Self-Inflicted Pain

The difference between regular programmers and 10x programmers is not typing speed. In some cases, it is not even knowledge.

It is that 10x programmers are aware of and strongly avoid self-inflicted injuries while coding.

That is, they tend to avoid shortcuts and work far smarter than harder. They don’t tolerate a mess, they don’t flail at their work.

They need some code, they think first before coding, they code what they need, and then they refine it rapidly until it works. Then they leverage that code, over and over again, to save crazy large amounts of time. This is why their output is so high.

If you watch other programmers, they jump in too fast. They don’t fully understand what they are doing. The code gets messier and messier. Debugging it sinks through massive effort. Then they abandon that work and do it all over again for the next part that is similar. They burn through time in all the wrong places.

All of these are self-inflicted injuries.

Writing code when you only half understand what it should do will go badly. It’s not that you should be able to predict the future, but rather that given your knowledge today it should span the code you write. If there is something you don’t understand, figure that out before starting to code. If you have to change the code later because things change, that is okay. But if you are coding beyond your current knowledge it will go badly and eat through time.

Trying to fix crappy code is a waste of time. Clean it up first, then fix it. If the code doesn’t clearly articulate what it was supposed to do, then any perceived bug may be predicated on top of a whole lot of other bugs. Foundations matter.

So, when debugging, unless it is some crazy emergency patch, you find the first bug you encounter and correct that first. Then the next one. Then the next one. You keep that up until you finally find and fix the bug you were looking for. Yes, it takes way longer to fix that bug, but not really, as you are saving yourself a lot of time down the road. Those other bugs were going to catch up with you eventually.

If you see bad names, you fix those. If you see disorganization, you fix it, or at a minimum write it down to be fixed later. If you see extra variables you get rid of them. If you see redundant functions, you switch to only using one instance. If you see poorly structured code or bad error handling, you fix that. If you see a schema or modeling problem, you either fix it now or write it down to fix it later. The things you wrote down to fix later, you actually fix them later.

The crap you ignore will always come back to haunt you. The time you saved by not dealing with it today is tiny compared to the time you will lose by allowing these problems to build up and get worse. You do not save time by wobbling through the code, fixing it at random. Those higher-level fixes get invalidated by lower-level changes, so they are a waste of time and energy.

And then, the biggest part. Once you have some good code that mostly does what you want it to do, you leverage that. That is, putting minimal effort into highly redundant code is as slow as molasses. Putting a lot of effort into a piece of code that you can use over and over again is exponentially faster. Why keep solving the same lower-level problems again and again, when instead you can lift yourself up and solve increasingly higher-level problems at faster speeds? That is the 10x secret.

If you have to solve the same basic problems again and again, it is self-inflicted. If you lose higher-level work because of lower-level fixes, it is self-inflicted. If you have to do spooky things on top of broken code, it is often self-inflicted. If you get lost or confused in your own code, it is self-inflicted. If you want to be better and faster at coding and to have less stress in your job, stop injuring yourself, it isn’t helping.

A Rose by Any Other Name

Naming is hard. Very hard. Possibly the hardest part about building software.

And it only gets harder as the size of the codebase grows, since there are far more naming collisions. Code scales very, very badly. Do not make it worse than it has to be.

This is why naming things correctly is such a fundamental skill for all programmers.

Coding itself is oddly the second most important skill. If you write good code but bury it under a misleading name, then it doesn’t exist. You haven’t done your job. Eventually, you’ll forget where you put it. Other people can’t even find it. Tools like fancy IDEs do not save you from that fate.

There are no one-size-fits-all naming conventions that always work correctly. More pointedly there can never be such a convention. Naming is not mindless, you have to think long and hard about it. You cannot avoid thinking about it.

The good news is that the more time you spend trying to find good names, the easier it gets. It’s a skill that takes forever to master, but at least you can learn to not do it badly.

There are some basic naming rules of thumb:

First is that a name should never, ever be misleading. If the name is wrong, it is as bad a name as possible. If someone reads it and comes to the wrong conclusion, then it is the author's fault. When you name something you have to understand what that thing is and give it the best possible name.

Second is that the name should be self-describing. That is, when someone reads the name, they should arrive at the right conclusion. The variable should hold the data they expect. The function should do what it says. The repo that holds a given codebase should be obvious.

“Most people never see the names I use in my code …”

No, they do see them. All of them.

And if they see them and they are poor or even bad, they will recommend that your code gets rewritten. They will throw away your work. Nothing else you did matters. If the code is unreadable, it will not survive. If it doesn’t survive, you aren't particularly good at your job. It’s pretty simple.

Occasionally, some really awful code does get frozen way deep in a ball of mud. But that unfortunate situation is not justification for you being bad at your job. Really, it isn’t.

Third, don’t put litter into your names. Made up acronyms, strange ‘pre’ or ‘post’ text. Long and stupid names are not helping. Stop typing in long crazy names, spend some time to thinking about it. Find short reasonable names that are both descriptive and correct.

Fourth, don’t put in irrelevant or temporary stuff in there either. If some unrelated thing in an organization changes and now the name is either wrong or needs to be changed, you did it wrong. Names should be nearly timeless. Only if the nature of the problem changes, should they need changing, and you should do that right away. Names that used to be correct suck.

Names are important. They form the basis of readability, and unreadable code is just an irritant. If you were asked to really write some code, you need to really write it properly. If it takes longer, too bad. Good naming only slows you down until you get better at it. You need to be better at it.

Natural Decompositions

Given a large problem, we start by breaking it down into smaller, more manageable pieces. We can then solve all of the smaller problems and combine them back together to solve the original problem.

The hiccup is that not all decompositions are created equal. If you break a big problem down into subparts, when they have any sort of cross dependencies with each other you can’t work on them independently. The dependencies invalidate the decomposition.

So we call any decomposition where all of the subparts are fully independent a ‘natural’ decomposition. It is a natural, complete, hard ‘line’ that completely separates the different parts.

Do natural decompositions actually exist?

Any subpart that has no dependencies on other outside parts is fully encapsulated. It is a black box.

A black box can have an interface. You can put things into the box. It’s just that whatever happens in the box stays in the box. You don’t need to know anything about how the box works inside, just on the outside.

A car engine is a good example. You put in fuel, and you play with the pedals, then the car moves. If you are just driving around, you don’t need to know much more than that. Maybe if you are pushing it on the highway or a racetrack, you’d need to understand gearing, acceleration, or torque better, but to go to the grocery store with an automatic transmission it isn’t necessary.

Cars have fairly good natural decompositions. They are complex machines, but most people don’t really need to understand how they work. Mechanics and race car drivers do.

Software though is much harder to decompose because it isn’t visible. The lines between things can be messed up and awful, but very few people would know this. A five wheeled car/truck/motorbike monstrosity would be quickly discounted in reality, but likely survive as a software component.

Although we don’t see it the same way, we can detect when a decomposition is bad. The most obvious test is that if you have to add a line of code, how many places are there that it would fit reasonably? The answer should be one. If that is not the answer then the lines are blurred somewhere.

And that is the crux. A good decomposition eliminates the degrees of freedom. There is just one place for everything. Then your code is organized if everything is in its one place. It’s simple, yet not simple at all.

For example, If you break off part of the system as a printing subsystem, then any and all code that is specifically tied to printing must be in that subsystem.

Now it’s not to say that there isn’t an interface to the printing subsystem. There is. Handling user context and the specific gui contexts is done elsewhere and must be passed in. But no heavy lifting is ever done outside. Only on the inside. You might have to pass in a print-it-this-way context that directs what is done, but it only directs it from the outside, the ‘doing it’ part is inside the box.

One of the hardest problems in software is getting a group of programmers to agree on defining one place for all of the different types of code and actually putting that code in the one place it belongs.

It fails for two reasons. The first is that it is a huge reduction in freedom. You aren’t free anymore to put the code anywhere. The culture of programming celebrates freedom, even when it makes our lives way harder or even tragic.

The other reason is in making it quick and easy for newer programmers to know where to put stuff. If we fully documented all of those places it would be far too much to read, and if we don’t most people won’t read the code to try to figure it out for themselves. Various standards and code reviews have tried to address it over the decades, but more often than not people just create a mess and pretend like they didn’t. Occasionally you see large projects with good discipline, it happens.

This shows up in other places too. Architecture is the drawing of lines between things. An enterprise architect should draw enough lines in a company to keep it organized; a system architect should draw enough lines in a system for the same effect. Again, these lines need to be natural to be useful. If they are arbitrary they make the problems worse not better.

Decomposition is the workhorse of software development, but it's far too easy to get it wrong. Fortunately it’s not hard to figure out if its wrong and fix it. Things go a lot smoother when the decompositions are natural and the work is organized. Programming is hard enough sometimes, we don’t need to find ways to make it worse.

Anti-patterns

“Calling something an anti-pattern is an anti-pattern.”

There are lots of ways to accomplish things with software. Some of them are better than others. But the connotation for the term ‘anti-pattern’ is that the thing you are doing is wrong, which is often not the case.

Realistically, the ‘pattern’ part of the phrase is abused. A design pattern is just a generalized abstraction of some work you are doing. It is less specific than an ‘idiom’. It is less specific than a ‘data structure’. It is just essentially a code structuring arrangement that is common. That is, it is effectively a micro-architecture, a way to structure some functionality so that it is easier to understand and will behave as expected.

So, mostly what people mean when they call something an anti-pattern is just that it is not the ‘best’ alternative. But even if it is not the best, that doesn’t make it a bad choice. Or basically, the set of alternatives for coding is not boolean. It’s not a case of right or wrong. It’s a large gradient, there are a huge number of ways to code stuff, some are better than others. And sometimes, for some contexts, a lesser approach is actually better.

We saw this in the 70s with sorting, but the understanding doesn’t seem to have crystalized.

There are lots of different ways to sort, with different performance. We can track ‘growth’ which is effectively how an algorithm performs relative to the size of the data. A cute algorithm like a pivot sort has nearly optimal growth, it is O(log N). Bubble sort however is considerably worse at O(N^2).

So, you should always implement a pivot sort if you have to implement your own sort?

No. If you have a large amount of data to sort, then you probably want to spend the time to implement a pivot sort. But… if you usually only have a few things to sort, then just putting in a bubble sort is fine.

Why?

Because the code for a bubble sort is way, way easier to write and visually validate. And performance is not even close to an issue, the set of data is always too small. With that tiny size, it wouldn’t matter if one algorithm was a few instructions different from the other, since it doesn't loop long enough for that to become a meaningful time.

So, in that reduced context, the shorter, easier, less likely to have a bug, code is the better alternative. More significantly, for front-end devs, whipping together a bubble sort is fine, for back-end ones, learning to implement pivot sorts is better.

But in modern programming, since most stacks implement pretty darn good sorting, the issue is moot. It is presented in data structure courses as a means of learning how to correctly think about implementation details, rather than an explicit skill.

In modern terms, I’m sure that a lot of people would incorrectly call a bubble sort an anti-pattern, which it is not. Most ‘lesser’ patterns are not anti-patterns. An actual anti-pattern would be to have multiple copies of the same globals, when what you really just needed was one. Another less common anti-pattern would be using string splits as the way to parse LR(1) grammars, as it would never, ever work properly but that is a longer and far more difficult discussion.

In general though, the software industry has a real problem with using hand waving to summarily dismiss significant technical issues. Programmers resort to claiming that something “right” or “wrong” far too quickly, when neither case applies. It is a form of boolean disease, caused by spending too much of the day crafting booleans, you start to see the rest of the world only in those terms.

Context

When I discuss software issues, I often use the term ‘context’. I’ll see if I can define my usage a little more precisely.

In software programs we talk about state. The setting of a boolean variable is its state. There are only two states.

For variables with larger ranges, i.e. possible settings, there can be a huge number of possible states, they are all discrete. An integer may be set to 42.

We usually use state to refer to a group of variables. E.g. the state of a UI is its settings, navigation, and all of the preferences.

Context is similar, but somewhat expanded. The context is all of the variables, whether explicit or implicit; formal or informal. It is really anything at all that can vary, digitally or even in reality.

Sometimes people just restrict context to purely digital usages, but it is far more useful if you open it up to include any informal variability in the world around us. That way we can talk about the context of a UI, but we can also talk about the context of the user using that UI. The first is a proper subset of the second.

The reason we want it to be wider than, say just a context in the backend code is because it affects our work. Software is a solution to one or more problems. Some of those problems are purely digital, such as computations, persistence, or communications, but most of our problems are actually anchored in reality.

For instance, consider a software system that inventories cogs created at a factory. The cogs themselves and the factory are physical. The software mirrors them in the computer in order to help keep track of them. So, some of the issues that affect the cogs, the factory, or the types of usage of the system, are really just ‘informal’ effects of reality. What people do with the software is heavily influenced by what happens in the real world. The point of an inventory system is to help make better real world decisions.

We may or may not map all of those physical influences onto digital proxies, but that does not mitigate their effect. They happen regardless. So if there are real events happening in the factory that affect the cogs but are not captured correctly, the digital proxies for those cogs can fall out of sync. We might have the wrong counts in the software for example because a bunch of cogs went missing.

As well, the mappings between reality and the software can be designed incorrectly. The factory might have twenty different types of cogs, but the software can only distinguish ten different types. The cogs themselves might relate to each other in some type of hierarchy, but the software only sees them as a flat inventory list.

In that sense the software developers are not free to model the factory and its cogs in any way they choose. The context in reality needs to properly bound the software context. So that whatever happens in the larger context can be correctly tracked in the software context.

The quality of the software is rooted in its ability to remain correct. Bad software will sometimes be wrong, so it is not trustworthy, thus not too useful.

Now if the factory was very complex, it might be a huge amount of work to write some software that precisely models everything down to each and every little detail. That would be a massive amount of work. So we frequently apply simplifications to the solution context. That works if and only if the solution context is still a proper generalized subset of the problem context.

From our earlier example if all twenty physical cogs map uniquely onto the ten software cogs, the context may be okay. But if some cogs can be mapped in different ways, or some cogs cannot be mapped at all, then the software solution will drift away from reality and people will see this as bugs. If there are manual procedures and conventions to occasionally fix the map, then at some point they'll degrade and it will still fail.

Which is one of the most common fundamental problems with software. There often isn’t time to do the context mappings properly, and the shortcuts applied were invalid. The software context is shifted out from under the problem context, so it will gradually break. More software or even manual procedures will only delay the inevitable. The data, e.g. proxies, in the computer will eventually drift away from reality.

So, if we see the context of the software as needing to be a proper subset of the context of the problem we intend to solve, it is easier to understand the consequences of simplifications.

This often plays out in interesting ways. If you build a system that keeps track of a large number of people you obviously want to be able to uniquely identify them. Some people might incorrectly assume that a full name, as first, middle, last, is enough, but most names are not particularly unique. Age doesn’t help and duplicate birthdays are far too common. You could use a home address as well, but even in some parts of the world that is not enough.

Correctly and uniquely identifying ‘all’ individuals is extraordinarily hard. Identifying a small subset for an organization is much easier. So we cheat. But any mapping only works correctly for the restricted domain context when you don’t have to fiddle with the data. If you have to have Bob and Bob1 for example, then the mapping is broken and should be fixed before it gets even worse.

So as a problem we want to track a tiny group of people and we don’t have to worry about the full context. Yet, if whatever we do forces fiddling with the data, that means our solution context is misfocused and should be shifted or expanded. Manual hacks are a bug. Seen this way, it ends any sort of subjective arguments about modeling or conventions. It’s a context misfit, it needs to be fixed. It’s not ‘speculative generation’ or over-engineering, it is just obviously wrong.

The same issues play out all over software development. We build solutions, but we build them to fit against one or more problem contexts, and those often get bounced around by larger organization or industry contexts.

That is, often people narrow down the context to make an argument about why something is right or wrong, better or worse, but the argument is invalid because the context is just too narrow. The most obvious example I know is the ancient arguments about why Betamax tapes would beat out VHS, when in reality it went the other way. I think the best reference to explain it all was Geoffrey Moore in “Crossing the Chasm” when he talks about the ‘whole product’ which is an expanded context.

All of that makes understanding the various contexts that bound the system very important. Ultimately we want to build the best fitting solutions given the problems we are trying to solve. Comparing the two contexts is how we figure out if we have done a good job or not.