Multi-tasking

When I first started coding I often struggled with remaining focused. While writing code, there is a lot of stuff going on in our heads that need to get put down into the computer. It turns out that code is best and the work is fastest if you can basically just type what you ‘see’ during prolonged stretches. Later you keep passing over it, again and again, editing it until it becomes clean and tidy.

As I grew in my abilities, I learned to focus for hours and hours at a time. In the heyday of programming, I had my own office, I could close the door and get emersed in the work. That was easily the best code I have ever written. It was very readable, the mechanics were discussed thoroughly and thought out clearly, the project had good strong conventions that we meticulously followed, and the code was consistent. Reuse was critical to the success of the project, otherwise, we would have ended up with way too much stuff. It was a very ‘technical’ development, a big distributed fault-tolerant cache that had high-performance specifications. Despite the complexity, we only ever had 1 known software bug in production. It was also my favorite job.

As Agile killed the profession, programming moved from being able being about deep focus to often just bouncing around throwing a lot of code fragments everywhere to see if we could get things working. Noisy open office spaces, lots of interruptions, stress, and drama, and maybe about 30% of the time necessary to get barely functional code.

It took some adjustments, but I gradually learned to work in a messy, loud, mutli-tasking environment. Part of the secret was learning to write a lot of stuff down. The core habit is to keep a ‘note’ of what you are going to do before you go and do it. If you are disrupted in the middle of working on something, you make a new note for the next task, jump to the new work and keep going. When you are finished, you go back over your notes and work your way back out of the pit. In rough environments, you might find yourself down 6 or 7 levels, really quickly.

Another great habit is time-boxing. That is, you try to allocate very specific times of the day to do specific work. Mornings are for big email replies, management issues, etc. After lunch, somedays are ‘coding days’, some are ‘meeting days’. Rather obviously, sticking all the meetings back to back helps keep you in ‘talking context’, rather than ‘coding mode’. It’s always been very different, at different companies, but trying to batch similar stuff together helps, thinking about it in terms of ‘focused work’ and ‘unfocused (chatty) work’ helps too.

Another thing to do is keep a separate list of boring tasks. When you are baked, work on stupid, boring, trivial stuff. When you have the energy, do the more cognitively demanding work. It does cause a bit of interference with scheduling though. When you are giving estimates, you have to give them in terms of the good days/hours. So, if most weeks consist of 2-3 days of insanity and drama, your capacity is really only 3-2 days of work, but it’s better to err on the lower side, so you are often down to 2. Thus 10 days of grueling, hard-ass coding are going to span 5 weeks unless you start clocking in overtime. If you keep clocking in overtime, eventually all of the days are ‘unfocused days’ and you’re down to 0, which gets messy with your employer (but at least the boring work is all done, isn’t it).

Sometimes I just cycle. Get four projects going at once, work on each until they hit a significant blocker, then move on to the next one. When doing this, the code itself can’t be rocket science, so it’s usually a little more brute forcey, domain-specific stuff. Then the blockers tend to be poor specifications, external decisions, ambiguities, etc. Most of them involve getting a conversation going with other people about some issue that needs to be addressed right away. So, the flow is to read back the items that need to get done (as per above), then work on them. Either close them off or jot down that you need to punt it into some discussion. When the little bits are all finished, flip over into ‘email mode’ and trigger all of the discussions. That project is now all jammed up, move on to the next project. Keep cycling between all of them, with the same flow.

Sometimes, shifting priorities will pop up that force a different order, but it’s best if you try to spend a bit of time first to close off the thing you are actively working on right now (by either finishing it or at least updating your notes). Working this way, keeps everything moving forward, and you keep a record of where you are and what didn’t get done yet. It doesn’t however produce great quality work.

My personal feeling is that spending the effort to get good or even great code is both faster and produces way better results for the users. We can rush through the work, at times they do only need crappy quality, but doing this all of the time gradually paints the entire effort into a corner. Technical debt is most often exponential, a little of it starts leaking in, and before you know it, getting anything done, at all, is a Herculean effort. Preventing that decline is equally, if not more important than getting out most features, but since that work is basically invisible to management and the users -- until it is too late -- people incorrectly ignore it. Still, the industry turned into this current direction, like a huge ship, so it’s not so easy to set it back on a good course. We have to deal with whatever madness that is paying the bills, so it is best to learn how to spew out lots of code, on many different tasks, all at once, even if you realize that people aren’t really getting what they need.

Theory and Practice

In many discussions, I’ve often referenced the need to know some ‘theory’ when writing specific types of code. What I’ve seen is that people who bypass acquiring this knowledge will end up writing code that just won’t ever work correctly. That is, there are some very deep, rather counter-intuitive problems buried at the heart of some discrete computational problems, and that approaching it with clever ideas is not enough.

If you are okay with your stuff falling back 30 to 50 years in behavior then you can attempt to reinvent these wheels, but if you need to get it out there, working as well as it can, then you should not ignore the underlying theories.

The first example is parsing. People have long noticed that string operations like ‘string split’ are pretty powerful, and they do some of what is needed to parse incoming text. So if you have a buffer of “x = y” statements all on their own line, then you could split and squeeze the whole thing into a dictionary. Easy, peasy.

It starts to fall apart when bits of the parsing becomes ‘modal’. The classic example is a quoted string in a .csv file that contains a comma, which is also the field separator for the file. The right way to handle this is to basically have 2 states. One that is normal and one that is ‘in quotes’. While ‘in-quotes’ you treat the comma as a normal character. Otherwise, you treat it as a separator.

And this is where things get messy. The comma character has 2 distinct meanings. It’s either just a normal character in a string, or it is a special character used to act as a separator between fields. If you don’t keep track of the state, the meaning is ‘ambiguous’. You just don’t know. Split might handle the work of breaking up the string into pieces, known as tokenizing, but it provides no means of state mechanics.

The syntax in a complex programming language has tonnes of these state-based issues. They are everywhere, from the operators we use to the way we handle blocks in the code. It’s a state-fest. This occurs because the bandwidth of our characters is somewhat smaller than the bandwidth of the things we want to do, so we pack our mechanics tightly into the symbols that we have available, thus causing lots of ambiguity. Language theory refers to this as an LR(1) grammar, but the underlying issue is that we often hit a token that is ambiguous, so we try to process it one way, realize that is wrong, then we have to back up and do it some other way. If the language is really gnarly, we might have to back up a few times before we can resolve the ambiguity.

Without the ability to back up, basically anywhere at any time, the code will continue to hit ambiguities and do the wrong thing. So the underlying problem is that if you try to hard code the mechanics, there will continue to be circumstances where that static coding won’t do what is expected. These special cases will continue to occur infinitely. In the beginning, you fix a lot of them, it will slow down as you get way more code, and the unobserved variance of the inputs reduces, but it will never, ever go away. The code is incomplete because the code can’t adapt dynamically to the input, so the code is weak by construction. The mechanics can’t be static they have to be dynamic enough to bend to any ambiguity that they find anywhere in the text.

Ambiguities are actually found all over the space of discrete systems. They are essentially just missing information, but they are missing in a way that sometimes the physical universe itself can’t resolve them. The most classic one is TGP, the two generals problem. It’s at the very heart of all distributed processing.

Basically, if you have more than one ‘serialized computation’, sometimes described as Turing Machines, the communication between them has an intrinsic difficulty. If one ‘machine’ sends a message to another ‘machine’ and after some time there is no response, what happened is ambiguous. The sending of the message may have failed, or the sending of the reply may have failed, or some code in between may have failed. You can’t ever be sure which side of the communication was dropped. You can’t be sure if the work was done or not.

If the message was just asking for some information, basically a ‘read’, then when the first machine gets impatient it can just redo the question.

If the message was asking the second machine to change a value, then it suddenly gets funny. If the message never made it, the second machine obviously didn’t make the change. If the message made it, but the reply didn’t come back from them the second machine may have actually made the change. The lack of reply doesn’t imply the state of the change.

If you don’t care whether the change has been made or not, it’s not much of a problem, but in a lot of cases we build up huge amounts of ‘related’ work that are distributed over a bunch of different computers. We wrap these in a ‘transaction’ so that we get the property that either all of the changes worked, or all of the changes failed. That’s critical in that we need to inform the user that their request, which was ‘singular’ to them, was actually fully and properly processed, or there was some problem so nothing was done. We refer to this as ‘transactional integrity’. It’s one transaction to the users, but it may span a lot of different machines, databases, etc. A trivial example is transferring money between two systems. The amount must be decremented from one database if and only if the amount is incremented in another. The transfer either works, or it does not. There can be no other intermediary states, or things will get ugly.

So, we have this problem of coordinating a bunch of different machines to either all do the work, or to all fail. But if one of them is late in replying, we don’t know whether that work was done or not. That ambiguity should halt the other machines from continuing until it’s been correctly resolved. Worse, there may be lots of other transactions piling up behind this one, and we don’t know whether they are independent of this change, or they rely on it for their changes. Say, transferring the new balance in one of those earlier accounts to a third system.

If we built this type of computation to be entirely reliable then every time one of the communications gets dropped, the entire system would grind to a halt, until that message was found or resent. That, of course, would perform incredibly badly, as there are lots of reasons in the physical world that communications can be interrupted.

In practice what we used to do was minimize the window of failure to be as small as possible. That is, a transaction is sent out to all participants to ‘stage’ the work. When they insist that their next step is tiny, the second round of messages is set out to turn it all live, or ‘commit it’. If it’s successfully staged, it is treated as if it will go through. If that first part of the conversation failed, it is all rolled back and an error is propagated upwards to let the initiator know that it didn’t work.

If you get this right, the behavior of the system overall is nicely deterministic. When you see a confirmation, it means everything really was updated. When you see an error, it means that it was not. When you get this wrong, neither a confirmation nor an error has an exact meaning. It could mean it mostly worked, or it could mean that it kinda failed. You really can’t trust the system at that point.

In modern practice, the whole notion of transactional integrity is often ignored. All of the subsystems just get the requests and process them or they error out. Everything is handled individually. The problem is amplified by the modern habits of adding more and more distributed systems. When most software was on the same hardware, the likelihood of problems was really tiny. When it is distributed across a lot of machines, the likelihood of problems is quite significant. It happens all of the time. If the data is spread all over the place, and there are a lot of mindless workers changing it, without any integrity, the quality of the data, as a whole, is going to be really, really bad. Constant problems will cause gaps, overlaps, and contradictions. Mostly it will go unnoticed, but every once in a while someone will do the work of reconciling their different data sources, and find out that collectively it is all a mess.

As if we needed more problems, we used to architect large scale systems with Transaction Management technology, but since they are slow and the programmers felt that they were difficult to use and somewhat pedantic, we switched over to using stateless API calls. No transactional integrity at all, on purpose. That might have been okay if we partitioned the systems based on data dependencies, but then along came ‘microservices’, where we break it all up even further into even more little, dependent, parts. Rather obviously that is a recipe for disaster. The finer grain we go, as well as making it more distributed across more hardware, while ignoring the necessity of transactions, the more we degrade the quality of the data we collect. This occurs as a natural result of tunnel visioning into small performance or organizational problems, while ignoring larger cross-cutting concerns like transactions. It’s not easy to cope with physical limitations, but it certainly isn’t advisable to just ignore them.

But it’s not just persistence that gets into trouble. Even though they are way faster, computers are still too slow. That oddly comes from their popularity. As we use them for more processing, we expect them to process way more stuff. The size of the stuff we are trying to handle these days has grown faster than the increases in hardware. That is, the workload for software outpaces Moore’s law in growth, right now.

To get around that, we don’t just want faster hardware, we want to involve more hardware as well. In the early days of computing, when we parallelized stuff, we were very careful to sandbox it as well. So, one computation as it runs could not be affected by any other running computation. We crafted a ‘process’ abstraction so that we could fit a bunch of these together on the same hardware, but they maintained their ‘safety’. Later, when programmers decided that safety was just too awkward, they came up with ‘threads’. These are independent computations that can cause all sorts of havoc with each other. Basically, they are fighting for dominance within the hardware, and if they bang into each other, the results of the collisions can be rather messy.

Realizing that multi-threaded code was a problem, we enhanced our knowledge of fine-grained synchronizations. That is, two threads could share a larger piece of data like a string if they both nicely agreed to use the same mechanics to synchronize their operations on it. The primitives to drive synchronization have been around since the early days, as they were needed for foundational technology like operating systems, networks, and databases. What changed was that we pushed their necessity upwards, higher and higher in the technical stacks, so that the programmers writing code on top now also needed to understand how they worked. It’s a form of unencapsulating stuff, where we solved a problem, made it nicely usable then went out and unsolved it again.

It’s not the only place where we provided mechanics to people that were highly likely to let them injure themselves. Colloquially we call it “giving them enough rope” and we’ve seen that over and over again in other areas like memory management, pointers, type management, and variable arguments.

It’s not enough to just avoid data collisions in the code, you also have to worry about starvation, race conditions, deadlocks, livelocks, etc. Synchronizing two dependent computations is non-trivial. But getting it wrong still lets the code mostly work. It works a lot, then strangely errors out, then magically works again. Mostly if the collisions are infrequent, the testing won’t notice them, and the users just put up with the occasional, unexplained problems.

There are many other known and unknown problems that quickly flow into theoretical issues, like caching or algorithmic growth, or even error handling to some degree. On the surface, they seem easy, but underneath it’s incredibly hard to get them done correctly, or at least close to the state-of-art knowledge.

Most code however is pretty trivial, and people often use brute force to make it even more trivial. Get some data from a database, throw it to the user, have them make some edits, and then save it again. Ask another system for an update, store it away forever. Spew out a few million slightly customized reports. Most of this type of programming needs a really good understanding of the way that the data should be modeled, but the technical side is quite routine. The trick isn’t coding it, it is keeping it all organized as it becomes large and hard to handle. Keeping millions of little pieces neatly arranged is a somewhat different problem from getting one little tricky piece to behave as expected. What we need to do better is to keep the tricky bits encapsulated, so that they aren’t wrong so often.

Plumbing

By now it should not be hard to synchronize data between different software systems. We have the storage capacity, the CPU power, and there are a plethora of ways to connect stuff. Still, the situation in the trenches isn’t much better than in the 90s, which is sad.

XML was in many ways an attempt to send around self-documenting data. The grand idea was that you could get stuff from anywhere, and there was enough meta-information included that it could be automatically, and seamlessly imported into your system. No need to write the same endless ETL code over and over again. Yeah.

But rather than mature into that niche, it got bogged down in complexity which made it hard to use. JSON took a few massive steps backward, but at least you could wire it up quickly.

One of the underlying problems is semantic meaning. If one system has a symbolic representation of part of a user’s identity in the real world you have to attach a crazy amount of stuff to it when it is exported in order for a second system to be able to interpret it correctly, in a completely different context. Just knowing the bits, or groking that they are byte encodings of particular glyphs, or even knowing that those glyphs collected together represent an identifier in a particular human language is not enough. The implicit ‘model’ of the information as it is mirrored from the real world in both systems have to hold enough common information that a stable mapping of them is possible. And it’s probably some offbeat artifact of reality that the stability itself probably needs to be bi-directional.

We could play with the idea that the two systems engage in a Q&A session. The second system keeps asking “what do you mean by xxxx?”, slowly descending the other system’s knowledge graph until it hits an ‘ah-ha’ moment. Oddly, if they were both event-based, then the conversation would only have to occur after one of them changed code or config, so not that frequently and probably not in haste.

That, of course, would be a crazy amount of code. As well as implementing the system you’d have to implement a malleable knowledge graph that spanned each and every line of code and a standard protocol to traverse it. It’s no surprise that we haven’t seen this type of code in the wild yet, and may never see it.

What if we went the other way? A system needing data contacted an intermediary and requested data. Another system contacts the same intermediary and offers to share data. That gives us two structural representations of a complex set of variables. Some map one to one, based on a mapping between the labels. Some have arbitrarily complex mappings, that in the worst case could include state. If the intermediary system had a way of throwing up its hands, admitting defeat, and bothering a few human beings for intervention, that version of the problem might be solvable. Certainly, the export and import from the two systems is well constrained. The interface is fairly simple. And, over time, as the intermediary built up a sophisticated database of the different sources and requestors, the effort to connect would converge on being fully automatable.

If the request for data had a property that any imports were idempotent with what amounts to a universal key for that specific entity, then the whole thing would also be fairly resilient to change. If the paradigm for the intermediary’s interface was a fancy to-do list based on incoming requests one could prioritize the connectivity based on organizational needs. 

So, one programmer’s assignment would be to send all of the public entities to X. X might choose to not save them, it isn’t a historic database, but it would accept any and everything.

Another programmer’s assignment would be to add in some background task to get an entity from X. It might poll if that works, or it might just register for an event, and get data as it is received.

Neither programmer cares about the other’s schema or field names or any other thing, They deal with what they need or already have. No fiddling.

In their calls, they include a contact email, each time. Then they wait.

A data administer sees that there is a publisher that kinda matches one or more requestors. Kinda. They can contact either side to get more info, or make suggestions. They have some ability to put small computations into the pipeline. They can contact others to sort through naming issues. At some point, hopefully, with minimal changes to either side, they just flip a switch, and each instance of the entities gets translated. The receiving system springs to life.

Obviously, they do this in a testing environment. Once it is agreed upon, the publishers go into production first, it is verified, then the receivers go live. The schedule for this work is fairly deterministic and estimable after the agreement from testing is reached. 

Ah, but what about ‘state’ you say? Yea, that is a problem. Some of it is mitigated by keys and idempotency. One can utilize that to support historic requests.

Other state variables more often tend to be embedded in the data itself. And with an event handling mechanism, you could have one system offer up a status to be consumed by another. Which segues nicely into error handling. The intermediary could fill in errors with a computation, say one system polls for the state of the other, since a specific time. The response could be computed to be ‘down’ if within that window there was no publishing of a heartbeat.

We can do similar things with abstract variables, like computing response time between heartbeats. It opens up a lot of great ways to monitor stuff.

A nice secondary attribute is that the intermediate system effectively documents the system interconnections. It can produce nice reports. 

But it won’t scale? Ya, sure it will. Not as one galactic intermediary system, but as many many smaller completely independent ones that could even be parallelized for big data flows. It won’t be super slow, but it's not the mechanism one needs for real-time data anyways, that is usually custom code, but very expensive code so you want to keep it to a minimum. It’s an 80/20 solution. If there are hundreds of these beasts running, you could still have one data management interface that binds them all together at an organizational level. Each instance has a synced persistent copy of only the data that it needs, not all of it.