Good vs. Bad

A good technology helps you solve your problems.

A bad technology is one that misleads you into thinking it has solved the problems, but instead, its side effects are actually creating other problems that now you also have to solve.

A good technology works as expected. No matter what the circumstances are, you should be able to predict what it is going to do, and it will do exactly that. You can rely on it to behave as expected.

A bad technology sometimes does what you think it will do, but sometimes it does very strange or unexpected things. You can’t predict it in advance, it is basically random from your perspective. You are caught by surprise often when using it, shock and disappointment follow.

A good technology is deterministic, at least in its ability to arrive at an expected outcome. You know what it is going to do, so you can figure out how it will accomplish those goals. If it is distributed or parallel, there may be a little bit of uncertainty with the finishing order of some intermediate operations, but it will correctly get to the desired outcomes.

A bad technology is non-deterministic. It may complete the work successfully or it may not. It may be restartable if there was a problem, or it may have left the system in a bad state that requires some manual interference. You pretty much don’t have any certainty of success when using it.

A good technology is obvious to use. You may need a bit of coaching about what it solves at a higher level, but after that, you can use it to do all of the things that you think it should be able to accomplish.

A bad technology needs lots of additional help. It may have strange abstractions, twisted behaviors, rude side-effects, etc. It needs really super-strong documentation, which is exceptionally hard to write, and quite rare to find. If you are confused while using it, that isn’t a particularly good sign. If it is really a brand new revolutionary technology that actually does present a new and innovative paradigm never seen before, that conceptual shift needs its own documentation, but once you’ve understood that, more documentation should not be necessary for any specific instance.

A good technology is effectively transparent. You see it, you understand what it does, you can figure out what is happening deep in its encapsulation since at some level it is pretty obvious. You can string together a bunch of good technologies to accomplish larger goals, it’s fairly clear how you would do this if you already know the individual components.

A bad technology is opaque. It’s a mystery. It does things behind the scenes that you won’t guess or be able to reason about. It hides stuff from you, that you need to know about it. You can’t string bad technologies together easily, they tend to fight with each other. Using the technology is just painful.

A good technology is malleable. Small changes have noticeable but contained effects. Everything is localized and the scope of changes is obvious. You can guess the overall impact.

A bad technology is brittle. A little change causes huge and unexpected differences in behavior. You have to be terrified with any type of change, you can’t use size as a measure of impact. One day it works, the next it is broken for unknown reasons.

A good technology is smooth. You decide you need it, it fits in easily and you are ready to start utilizing it for your goals. It’s isn’t a big deal to add it, and there isn’t a steep learning curve.

A bad technology has a lot of friction. It’s hard to get it set up, it’s hard to deal with it, and it’s even harder to accomplish any type of work reliably on a schedule. You might get some traction, but you pay for every inch that you get. It’s slow and plodding and acts as a time sinkhole.

Readability

In order to be useful, code needs to be readable.

If it’s not readable, then as things change, it is difficult to update. That has the unwanted side-effect of freezing the code in place with all of its defects. This leaves 2 bad choices: either spend a lot of effort to rewrite the code or just blindly build on top. Any code built on top or around defective code inherits its undesirable qualities.

So, unreadable code might suffice to meet a tight deadline, but its intrinsic tech debt will continue to grow.

You can reflector unreadable code into something better, but it is actually a lot less effort to just not make it unreadable in the first place. Some programmers wrongly assume that it is faster to create unreadable code, that any work to prevent that is extra and slows down the effort, but oddly unless the initial typing effort is perfect (which is so rarely the case) fixing the time spent the first round of syntactic and semantic flaws will quickly negate any perceived gains. So it is usually slower to write unreadable code and it only gets worse as it stays around in the codebase.

The most obvious factor in readability is naming. People generally believe this to be hard, but the names of most things in code come either from the problem domain or from the technical one. If naming is complicated it is usually a huge warning sign that either sufficient analysis and/or design has been skipped. That is if you don’t know what you are writing, then, of course, you don’t know what the pieces should be called, but you are also just taking a random shot at being able to craft the ‘correct’ piece of code, which usually ends up wasting a lot more time as you keep having to fiddle with it.

If you know the names of the data and functions involved in the code, the next big issue is structuring the code correctly. The easy problems in programming generally involve getting data from one location and sending it to another one, so the structure of the code is rather trivial. It gets a little more difficult if you also handle any errors properly, but there are plenty of straightforward paradigms for properly arranging this type of behavior. Some system-level code, like interpreters and operating systems, requires non-intuitive structures in order to keep the code organized, but they are well documented and there are plenty of resources available to learn the proper techniques. You just have to do the research first, before starting.

The biggest problem in structuring code is reuse. You might have dozens of screens that are each easy to write, but all pretty much share the same underlying code. It is an ‘epic’ waste of time to just write each one in its own silo. The code is hopelessly redundant, but the testing is also long and painful. So, the smart thing to do is write some underlying base code that contains the bulk of the work, then trivialize each different screen on top of it.

That reuse pattern also occurs when managing persistent data, doing imports and exports from other systems, and when keeping some other system synchronized. The super-difficult problem however is not just reusing the code, but rather doing it in a way that result is actually far more readable than just a large bunch of brute force silos. That can happen though if the levels between the parts of the code are not random, but rather they properly encapsulate different types of complexity for the different pieces.

When you encounter code that is well structured, you tend not to notice the care and effort that went into the workmanship. Instead, the code just seems rather obvious, it flows quite easily.

And that brings us to the most difficult aspect of readability. If you write some code, you will think that it is readable as you are working on it. But it still might be filled with a lot of weird, awkward, and eclectic stuff that other people would find difficult to understand quickly. That is, readability may be considered subjective when it comes to your own work, but as more people look at the code it becomes considerably more objective. So, the real measure of readability is in trying to make it quickly understandable by the largest possible audience. You really want to present the code in ways that make it inclusive to as many people as possible.

That means picking a few simple patterns that are applied consistently across all of the code. That means trying hard to not just throw in as many dependencies as possible. It means augmenting the code with any type of comments the reader needs in order to see the code the way you saw it when you wrote it. It means spending some time and effort trying to look at the code and see it the way other people will see it.

There are still people out there that believe that creating an obfuscated spaghetti codebase is job security, but that’s actually a very foolish belief. You don’t want to get stuck spending the rest of your life struggling with the mess you created, so it’s far better to create things that you can hand off to others. That helps with getting better work in the current company, but it also helps in creating lots of other opportunities in other organizations as well. Nobody wants to work with a difficult programmer again, which means that each bad experience is dwindling the possible market of new ones, rather than increasing it. In that sense, it’s not only better to write readable code now, it is also going to help you a lot in the future even if you switch jobs.

It’s also more fun to work on a well-written readable codebase. If there is a huge amount of redundant code, then it becomes increasingly likely that any given little code change will cause nasty unexpected side-effects somewhere else. So, it’s scary and super stressful to make changes. And since there is never enough time and effort put into testing, a lot of bugs seriously hurts the whole project. Thus making good changes becomes harder and harder, until the project is nearly frozen in place and everyone is angry.

Contrast that with a project where you can make a lot of good enhancements with very little stress. If the code has a lot of reuse, you can use the tools in the IDE to quickly visually check that the changes will not have unintentional side effects. A small amount of effort, but a huge gain.

As well, one little change may fix a bunch of related issues all over the system, so it looks like people have put in a lot of care and effort. So, it’s way less stressful, a little bit of extra work, but to the users, it looks like the project is moving forward really quickly now. And there are plenty of battle-tested components around that you can rely on to speed up the development, so you get much better quality at a faster pace. Instead of falling farther and farther behind in the development, you can start pursuing deeper and more meaningful changes, so people are excited and happy about the progress. It’s fun to work on, and you feel good about it.

Ultimately it’s not that hard to make code readable and it’s not that much “extra” effort. If the project starts off that way, the results will be far better. Skipping this work may get you to an early deadline slightly faster, but as the costs build up it will continually cripple the work. It’s harmful and crazy to intentionally write bad code, it’s insane that we keep encouraging it. Code isn’t disposable, it takes a lot of time and effort to get it working correctly, so it’s far better to write less code that is of good quality than it is to just spew out lots of junk code. We don’t need more code, we just need better code now.

Upside Down

Why is git so difficult to use? Way back, after I had already switched from CVS to Subversion, I finally started using Mercurial. I found the transition to be quick and easy; never had a problem getting it to do the things that I needed it to do. it was good, I really liked it. But later, I switched to git.

For whatever reason, I can’t seem to “get” git. It’s annoying and awkward and I pretty much have to keep looking up how to do stuff, even for a lot of simple, routine operations. Is it me?

I don’t think so. I think the problem is the way it was built. Someone once described Mercurial as James Bond and git as McGyver. Even though I’ve forgotten who said it, that analogy always stuck deeply in my head.

It seems that git is focused on doing the “right” things with the underlying data structures. It’s a programmer’s vision of the way that the data is stored. It doesn’t care about the users, it doesn’t want to be a nice and polite tool. It just exposes a rather huge, somewhat convoluted CLI, to get to the underlying data, and then you are left on your own to figure out how to map that back to the things you really need to do. Basically, it lacks any and all empathy for the people who end up having to use it on a daily basis.

And that would be very different from any tool that took a top-down approach to building up the functionality for people to accomplish their goals. That sort of high-level entry point into figuring out what code is needed to solve a problem starts right with the people doing the work and then maps it back down to the work that needs to be done. So, it’s cleaner and more organized around those higher perspectives.

Now it’s true that if you are coding from a bottom-up perspective it would be considerably faster and you’ll get a lot more reuse, so way less code and way less testing. As a way to build things, the bottom-up approach is clearly superior. But the big trick is that you don’t go all of the way back up to the top while doing that. Git is a good example of why this ends up just being awkward, and although it is usable, it wastes an awful lot of people’s time on what are essentially make-work issues.

This is why, for development, we always want to consider what we write from a top-down perspective first. We need to start there, we need to anchor the work in what the users need. What they really need. But we should switch and build it from a bottom-up approach afterward. The two sides really have to meet somewhere in the middle, where they are the least expensive. So it’s a bit of a flip-flop to get it right.

If you are curious, that is why so many development process tools and methodologies are also a mess. They usually pick the top-down side and force it on everyone. That makes sense from the design side, but it’s completely wrong from the construction side. It ends up promoting brute force as the construction style, which ends up crafting extreme amounts of redundant fragile code, which is inherently unstable and needs copious amounts of testing. So we end up doing way too much work that is forced into low quality because it takes way too long.

But if you go to the other extreme, it is wrong too as git so elegantly showed, and we keep ping-ponging between these two poles, without figuring out that the place we actually need to be is right there in between. I guess it’s a tendency to try to reduce everything to binary terms, just 2 colors, so we can overly crude logic on top. Multi-dimensional gradients are considered too complex, even if they do match our world properly. Probably also related to why most code doesn’t handle errors properly. It’s not just “works” or “error”, different types of errors often require very different types of processing. It’s the Achilles heel of the coding industry. We’re always in a crazy rush these days, so we cling to gross over-simplifications at our base, but those usually end up wasting even more time. just making it worse.

Non-destructive Refactoring

In order to evolve code, over increasingly sophisticated versions, you have to learn and apply a fairly simple skill for any language or tech stack.

It’s called non-destructive refactoring.

Although it has a fancy name, refactoring is really just moving around your lines of code and changing the way you partition them with abstract language concepts like functions, methods, and objects. There are lots of good books and references on refactoring, but few of them do a really good job at explaining why you should use the different techniques at different times.

The non-destructive adjective is a directive to constrain the wholistic effect of your changes. That is, you take an existing piece of working code, apply a bunch of refactoring to it, and when you are done the code will work 100% exactly as it did before. There will be no changes that users or even other programmers can or should notice.

All that has happened is that you’ve kept the same underlying instructions to the computer, but you have just moved them around to different places. The final effect of running those instructions is exactly the same as before.

A huge benefit of this is that you can minimize your testing. You don’t have to test new stuff, you just have to make sure what is there now is indistinguishable from what was there before.

If you are thinking that this is pretty useless, you are tragically mistaken.

The intent in writing code generally starts out really good. But the reality is that as the deadlines approach, programmers get forced to take more and more icky shortcuts. These shortcuts introduce all sorts of problems, that we like to refer to those as technical debt.

Maybe it’s massive over-the-top functions that combine too much work together. Maybe it’s the opposite, the logic is chopped up into tiny pieces and fragmented all across the source files. Maybe it is a complete lack of error handling. It doesn’t matter, and everyone was their own unique bad habits. But it does mean that as the code gets closer to the release, the quality of the code you are producing itself degenerates, often rapidly and to pretty low standards.

So, after a rather stressful release, possibly the worst thing you can do is just jump right back in again and keep going from where you left off. Because where you left off was hacking away badly at things in order to just toss the code out there. That is not a reasonable place to restart.

So, instead, you spend a little bit of time doing non-destructive refactoring.

You clean up the names of things, move code up and down in the function stack, add comments and decent error handling, etc. You basically fix all of the bad things you did right at the end and try to get yourself back to the code you wanted, not the code you ended up writing.

If you blurred the lines between layers, you correct that. If you skipped adding a layer, you put it in. If you added way too many layers, you flatten them down somewhat. You stop hardcoding things that would be nice to change and put them into cleaned-up configuration files. You document the things you missed.

If you have multiple copies of the same function all over the place, you pick one or merge the best parts, then you spend the time to point all of the other code back to that correct version. That’s one of the few places where this type of work isn’t truly non-destructive. One copy of the function maybe have been incorrect, you are rather accidentally fixing a bug or two by repointing it to something better. Because of that, you need to go up to everything that called that function and double-check that it was really calling it properly.

If you know that your next big work is building on top of your foundations, you rearrange things to make that work easier but restrict it to only changes that are non-destructive.

You take the time to do the right thing and to attend to all of the little details that you had to forget about earlier. You clean up your work first before you dive back in again.

If you keep doing this, even if you never seem to find enough time to do all of the work you know you should do, you will find that gradually, cleanup session after cleanup session, that you are moving in the right direction, and that the more often you do this, the less work you end up having to do for each release. That is, really messy code imposes a massive amount of friction which slows you down a lot. That friction is gone if the code is clean, so any work to get the code cleaner is also working to save you time and pain in the not too distant future.

Once you’ve mastered non-destructive refactoring, the other major skill is to extend what is already there (which is pretty clean cause you’ve made sure it is) instead of just bolting on more junk to the side. That is another super-strong habit that really helps to keep development projects ‘in control’ and thus make them a lot more fun to work on.

It’s Slow, We Should Rewrite it

Whenever a programmer says “it’s slow, we should rewrite it” there is probably like an 80% chance that that programmer doesn’t know what they are talking about. As a refrain, it’s pretty close to “works on my machine” for professional malfeasance.

The first issue is that X amount of work cannot be done in any less than X amount of time. In some super cool rare situations, one can achieve a logarithmic reduction of the work, but those types of optimizations do not come easily. It might be the case that the code is de-optimized, but it would be foolish to assume that without actually doing some performance testing.

The driving issue though is usually that the person making that suggestion does not want to actually go and spend time reading the older code. They like to write stuff, but they struggle to read it. So, as their offensive argument, they picked some attribute that is visible outside of the code itself, proclaim that that is the real problem, and use that as their justification to just throwing away someone else's work and starting over from scratch, without bothering to validate any of their assumptions.

Sometimes they have arrived at that argument based on a misunderstanding of the underlying technologies. They often assume that newer technologies are inherently better than older ones. Ironically, that is rarely the case in practice, the software crisis dictates that later programmers understand less of what they are actually doing, so it’s less likely that their repeated efforts will be better in general. It is true that there is some newer knowledge gained, which might feed into improved macro or micro optimizations, but those benefits can be lost to far more de-optimizations, so you can see why that is a really bad assumption. On top of all of that, the software industry has been rather stagnant for actual innovations for a long time now, most supposedly new technologies are just variations on older already existing themes. It just cycles around endlessly these days. Whatever is old is new again.

With all of that added up, you can’t say that an implementation in tech stack A would be faster or better than one in B. It’s not that simple. That’s been true for decades now. There are some pretty famous cases of people going right back to older technologies and using them to get far better results. The tech stack matters for other reasons, but usually not for performance or quality.

About the only thing you can say about one implementation is that it is a whole lot messier and disorganized than another. That the quality of work is poor. It’s just a pile of stuff hastily thrown together. But you cannot know that unless you actually dive in and read, then understand the code itself. You can’t look at 5% of it and draw that conclusion. And any outside behaviors are not enough to make those types of assertions.

Overall it is rarely ever a good idea to rewrite software anymore. There are times and situations when that changes, but it hasn’t been the default for a long, long time. The best alternative is to start refactoring the code so that you keep all of the knowledge that has already built up in it, and learn to leverage that into something that exceeds the scope and quality of the original code. You can’t do that by refusing to read it, or by ignoring the knowledge that went into it. If the code was in active development for a decade, then rewriting it would literally set you back 10 multiplied by the number of programmers involved over that period. Which is a huge backslide, and highly unlikely to be successful in any time frame. It takes an incredibly long time to slowly build up code, so even if it isn’t perfect, it represents a lot of time and knowledge. You need to mine and leverage that work, not just toss it blindly into a dustbin. It’s quite possible that the coders who wrote that original work were a lot better at their jobs than you are at yours.