Back in the eighties, when I was a student, there were only two choices: you used ‘vi’ or you used ‘emacs’.
I picked vi, my roommate picked emacs. The stuff he did with his editor was way cooler, but over the decades, vi has pretty much been everywhere I have stumbled.
I assumed it would fall out of fashion, but it persisted. Vim and all of those embeddings in the IDEs keep it alive.
I read something about how someone thought that using a complicated editor like vi was unnecessary. They implied that it was a waste of time to learn lots of its features. There were so many other things to learn, why not just use a lame mouse-based editor and move on?
For me, the answer is that it is all about craft.
When you build a large program, it’s a lot more than just throwing together the thousands of instructions necessary to tell the computer what to do. The code is just the output, but if you are hasty and careless on your way there, the whole thing ends up as a house of cards; any little breeze will knock it over. That is, it isn’t just coming up with some code; it is coming up with good, solid code that is readable and will survive all of the craziness that the world throws at it.
I’d rather have 150K lines of something that is super tight, abstract, solid, dependable than have 1M lines of stuff that could be served in an Italian restaurant. A whole lot of bad code is just a few steps closer to an apocalypse than a system.
So, it’s not about how much code you can create or how fast you can create it. Instead, it is about how carefully you embed precision into that code. Not for tomorrow, but for its entire lifespan. You want strong solutions to the real problems you need to solve, and you want them encoded in a way that you can keep them moving forward and leverage them for all sorts of related problems.
In the act of doing that sometimes tedious, thoughtful, and precise work, you want to develop habits that are more likely to produce high-quality output. Quality matters, not quantity.
Getting back to vi, if you are concerned about your code, you care very deeply about how it is ordered in any sort of file. It should not be random; you have come up with some consistent reason why one function is placed ahead of the others. In the daily grind, you have probably violated that order accidentally a few times. In most editors, you can use the mouse to highlight a chunk of code, hit ctrl-X to cut it, then ctrl-V to paste it back where it should go. It works, but it’s a touch haphazard. You could be interrupted, or get confused, or any other disruption that might interfere with the outcome you desire. In vi, you ‘mark’ the start, then ‘mark’ the end, then issue a few commands to move the code into place. It’s a small difference, but it’s transactional integrity. It either happens or it does not. If you get interrupted, it doesn’t matter; nothing is lost, nothing is messed up. If you're unsure about where to place the code, you can scroll around with the keyboard until you are sure. It isn’t any slower than using the mouse, but it is a whole lot more precise.
That’s a trivial example, for sure, but it’s just one of those things that you can do with a good tool when you’ve invested the time to understand it fully. Being a good programmer, a reliable one that is always there in a crisis, is the act of investing a whole lot of time in honing these acts of precision. It’s not that you can whack out a whole lot of code that you barely understand; it is that when you are given a piece of work, you can ensure that that work is done satisfactorily, even in environments that are difficult.
That is the essence of good programmers. They are the ones that everybody leans on, especially when it gets difficult, because they know that they’ll be there and make sure it is at least as good as it can get right now. They are reliable. Not always efficient, never fast, but always making sure that the work going out the door is solid.
We’ve always had the expression “garbage in, garbage out” in reference to the necessity to have good input data, but it also applies to code. If you get a group of people that churn millions of lines of questionable code, it’s just a field of landmines waiting to go off. If you get a group of people that solve complex problems with the least amount of code they can, carefully, then it doesn’t take that long before that quality of work starts to pay for itself. The craft of programming isn’t coding; the craft is in providing solutions, it just so happens that most of them involve code in some way.
Thursday, June 25, 2026
Thursday, June 18, 2026
Structureless
The most common mistake I have seen in big ugly balls of mud is to try to capture data without enough structure.
Chopping up some incoming data into a lot of little strongly typed fields is a pain. Sometimes it seems like an unwarranted pain. You may need to get a mailing address. Why not just give the user a big textbox to fill in?
The problem isn’t that the users can’t carefully type in the text with appropriate structure; it’s that sometimes they won’t.
And the code to parse unstructured text is stupidly complicated. They can type anything; you have to be able to apply some type of structure to each and every possible variation. Since there are an infinite number of those, you are going to lose.
You can add a ton of validation, but if it’s not rigorous within a fully interactive interface, then if there is any tiny way to bypass it, it was a total waste of effort.
You could scream at the users and make them format it correctly, but as time wears on, unless you keep screaming, eventually that practice will degrade. It will just delay the inevitable.
Which is to say that in a computer, a big box full of text is absolutely nothing more than a big box full of text. It has no other use or value. That makes it useful for someone putting a “personal note” somewhere in a report, or something like that, but there is nothing beyond that. It’s not really data; it’s just an extra external comment of some type.
If what you intend to do is collect data with a very specific structure, you should never subvert text boxes to do that for you. It’s not a shortcut; you haven’t “figured it out”, you just made a very bad mistake. And sadly, doing it right wasn’t that much more time.
Likewise, if you are using some questionable software, and it has a lot of text boxes, so you come up with a clever idea about how to put structured data into those, just because you can’t or don’t know how to change the program, then it isn’t brilliant. It’s just a sloppy, hacky way of trying to get around some other code. One step worse than duct tape.
Data is not data without structure. Untyped data is not data. A big string of characters is a mess. Mostly.
If it originated under the strict control of some code somewhere, and the pathway was closed and guarded, then sure, two programs can use strings and complicated parsers to pass data back and forth. It starts with structure, is transported in an unstructured container, and then it is restructured again. But if the pathway is open or one end of that game is a human, then a big string of stuff is just potential garbage. At some point, usually in the not-too-distant future, someone will fill that string with a problem, and it will end up wasting a lot of time.
Do note that there are subtle variations on this. A user might use one program to render XML that they upload manually into another. In a case like this, the human isn’t the endpoint; they are just the transfer medium. It's an open pathway, but still just between two programs. And you can close down that pathway by only strictly accepting XML with a very specific schema. Since you can verify that and reject garbage as necessary, it becomes closed.
The foundation of all software systems is to collect data. The full and complete structure of that data, as it relates to the real and digital worlds, is an essential part of that data.
There are extraordinary times when it makes sense to only sub-model some external data and live with the consequences of that choice. But the default should always be that if the program needs some data, it is key to its usage and computations, then the data needs to be fully and correctly modelled. That is, it needs the right structures, for example, not shoving a tree into a list, and it needs all of the individual fields in that data to be very strongly typed. It needs this in order to make the correct choices with its instructions based on what is actually there. It can’t be vague or ambiguous; computers are not smart enough to grok external context. They can only act on the very specific information that they have.
It’s also worth noting that while parsing may look easy, and in some tiny instances, it is not that complicated, it should always be considered a hard problem to properly solve. As such, if you can avoid parsing, or at least push it to a human somewhere, then you will get far fewer bugs and the code will be more likely to behave as expected. If you do have to venture into parsing, then it is one of the coding areas where reading a boatload of stuff in advance will pay off huge dividends. Trial and error with parsing is a massive bug generator.
Chopping up some incoming data into a lot of little strongly typed fields is a pain. Sometimes it seems like an unwarranted pain. You may need to get a mailing address. Why not just give the user a big textbox to fill in?
The problem isn’t that the users can’t carefully type in the text with appropriate structure; it’s that sometimes they won’t.
And the code to parse unstructured text is stupidly complicated. They can type anything; you have to be able to apply some type of structure to each and every possible variation. Since there are an infinite number of those, you are going to lose.
You can add a ton of validation, but if it’s not rigorous within a fully interactive interface, then if there is any tiny way to bypass it, it was a total waste of effort.
You could scream at the users and make them format it correctly, but as time wears on, unless you keep screaming, eventually that practice will degrade. It will just delay the inevitable.
Which is to say that in a computer, a big box full of text is absolutely nothing more than a big box full of text. It has no other use or value. That makes it useful for someone putting a “personal note” somewhere in a report, or something like that, but there is nothing beyond that. It’s not really data; it’s just an extra external comment of some type.
If what you intend to do is collect data with a very specific structure, you should never subvert text boxes to do that for you. It’s not a shortcut; you haven’t “figured it out”, you just made a very bad mistake. And sadly, doing it right wasn’t that much more time.
Likewise, if you are using some questionable software, and it has a lot of text boxes, so you come up with a clever idea about how to put structured data into those, just because you can’t or don’t know how to change the program, then it isn’t brilliant. It’s just a sloppy, hacky way of trying to get around some other code. One step worse than duct tape.
Data is not data without structure. Untyped data is not data. A big string of characters is a mess. Mostly.
If it originated under the strict control of some code somewhere, and the pathway was closed and guarded, then sure, two programs can use strings and complicated parsers to pass data back and forth. It starts with structure, is transported in an unstructured container, and then it is restructured again. But if the pathway is open or one end of that game is a human, then a big string of stuff is just potential garbage. At some point, usually in the not-too-distant future, someone will fill that string with a problem, and it will end up wasting a lot of time.
Do note that there are subtle variations on this. A user might use one program to render XML that they upload manually into another. In a case like this, the human isn’t the endpoint; they are just the transfer medium. It's an open pathway, but still just between two programs. And you can close down that pathway by only strictly accepting XML with a very specific schema. Since you can verify that and reject garbage as necessary, it becomes closed.
The foundation of all software systems is to collect data. The full and complete structure of that data, as it relates to the real and digital worlds, is an essential part of that data.
There are extraordinary times when it makes sense to only sub-model some external data and live with the consequences of that choice. But the default should always be that if the program needs some data, it is key to its usage and computations, then the data needs to be fully and correctly modelled. That is, it needs the right structures, for example, not shoving a tree into a list, and it needs all of the individual fields in that data to be very strongly typed. It needs this in order to make the correct choices with its instructions based on what is actually there. It can’t be vague or ambiguous; computers are not smart enough to grok external context. They can only act on the very specific information that they have.
It’s also worth noting that while parsing may look easy, and in some tiny instances, it is not that complicated, it should always be considered a hard problem to properly solve. As such, if you can avoid parsing, or at least push it to a human somewhere, then you will get far fewer bugs and the code will be more likely to behave as expected. If you do have to venture into parsing, then it is one of the coding areas where reading a boatload of stuff in advance will pay off huge dividends. Trial and error with parsing is a massive bug generator.
Thursday, June 11, 2026
Software Systems
I use the term ‘software system’ loosely. I usually intend it to mean: all of the boundaries for a set of related solutions that have been or will be implemented with software.
In that sense, it is less about the technical parts of the ‘system’ and more about how they all come together to help people.
I do this mostly because I tend to visualize a ‘problem space’ as a flat 2D terrain. It is a convenient oversimplification. It is a big, wide, open, empty field of grass which spans over related problems.
When I am doing greenfield work, I see the start as picking one spot in that field. You start there, building up enough structure to be useful. First, you lay down some common foundations, then you start adding in functionality that implements the features you know will help solve the problem.
As you do this, people will see the effort and start making suggestions. Some will want to go off in one direction, while others will prioritize the opposite way.
The trick to keeping it all as usable as possible is to slowly expand out your borders, but not in too many directions all at once.
Someone once told me that for software, you should never pick a path unless you are willing to walk it. From this perspective, it usually means that you won’t expand into another area in the field haphazardly. If you do choose to go there, it needs to be done correctly. That is, adding a few really good, solid features is way better than adding a million lame ones.
The same is true for the data. If you need new data, you add it carefully, properly structured, or not at all.
Overall, though, you start at one specific spot and keep growing. If it’s a good set of programs and people find it valuable, you’ll probably be at it for years, if not decades. So, it’s really crucial to its lifespan that the work you do in the very early days is as good as it can be. It needs to be neat, tidy, organized, and carefully thought out.
With that in mind, calling the current work and any of the rather obvious future work a ‘system’ works quite well. The system isn’t the code, but rather it is all of the territory that the code is trying to cover at some point. You might build a system for handling the account problems in a large corporation, for example. There might be lots of included pieces, and even some nearly stand-alone sub-systems, but they are all trying to fit together to deal with the same problems.
So, it’s similar to seeing the forest through the trees. The boundaries of the expected work are the system, but the system may not stretch right up to those boundaries yet.
From this view, it makes it easier to understand a bottom-up implementation. You might not know all of the features that people will ask for, but you should have a reasonable sense of the territory you are covering right now. Lots of that territory is similar, so building reusable components and engines will really help in getting more ground covered at a faster rate.
The classic example is reporting. You know people will need it at some point, so instead of just hardcoding a couple of static examples, it would be better to either offload it to somewhere else as exported data, or write some generic engine that is flexible enough to cover its rather massive width. The trick is not to write a lot of code, but rather to leverage any code you do write to cover the largest parts of that territory. In software, a little foresight goes a long way.
Thinking of a software system this way really helps in making a lot of the implementation decisions. If your field doesn’t cover having a million users, then designing an architecture to support that scale doesn’t make any sense.
More importantly, if you are located in one corner of the field, then trying to expand way over to the other side of a nearby hill doesn’t make a lot of sense either. That’s far enough away that it is clearly another system, another project and thus another codebase.
Building even medium-sized software is surprisingly complicated, so finding ways to frame it nicely really helps with making better decisions. Since time is a precious resource, getting to the right code as quickly as possible is important. Seeing it all as a system occupying some territory in an endless field is a good guide.
In that sense, it is less about the technical parts of the ‘system’ and more about how they all come together to help people.
I do this mostly because I tend to visualize a ‘problem space’ as a flat 2D terrain. It is a convenient oversimplification. It is a big, wide, open, empty field of grass which spans over related problems.
When I am doing greenfield work, I see the start as picking one spot in that field. You start there, building up enough structure to be useful. First, you lay down some common foundations, then you start adding in functionality that implements the features you know will help solve the problem.
As you do this, people will see the effort and start making suggestions. Some will want to go off in one direction, while others will prioritize the opposite way.
The trick to keeping it all as usable as possible is to slowly expand out your borders, but not in too many directions all at once.
Someone once told me that for software, you should never pick a path unless you are willing to walk it. From this perspective, it usually means that you won’t expand into another area in the field haphazardly. If you do choose to go there, it needs to be done correctly. That is, adding a few really good, solid features is way better than adding a million lame ones.
The same is true for the data. If you need new data, you add it carefully, properly structured, or not at all.
Overall, though, you start at one specific spot and keep growing. If it’s a good set of programs and people find it valuable, you’ll probably be at it for years, if not decades. So, it’s really crucial to its lifespan that the work you do in the very early days is as good as it can be. It needs to be neat, tidy, organized, and carefully thought out.
With that in mind, calling the current work and any of the rather obvious future work a ‘system’ works quite well. The system isn’t the code, but rather it is all of the territory that the code is trying to cover at some point. You might build a system for handling the account problems in a large corporation, for example. There might be lots of included pieces, and even some nearly stand-alone sub-systems, but they are all trying to fit together to deal with the same problems.
So, it’s similar to seeing the forest through the trees. The boundaries of the expected work are the system, but the system may not stretch right up to those boundaries yet.
From this view, it makes it easier to understand a bottom-up implementation. You might not know all of the features that people will ask for, but you should have a reasonable sense of the territory you are covering right now. Lots of that territory is similar, so building reusable components and engines will really help in getting more ground covered at a faster rate.
The classic example is reporting. You know people will need it at some point, so instead of just hardcoding a couple of static examples, it would be better to either offload it to somewhere else as exported data, or write some generic engine that is flexible enough to cover its rather massive width. The trick is not to write a lot of code, but rather to leverage any code you do write to cover the largest parts of that territory. In software, a little foresight goes a long way.
Thinking of a software system this way really helps in making a lot of the implementation decisions. If your field doesn’t cover having a million users, then designing an architecture to support that scale doesn’t make any sense.
More importantly, if you are located in one corner of the field, then trying to expand way over to the other side of a nearby hill doesn’t make a lot of sense either. That’s far enough away that it is clearly another system, another project and thus another codebase.
Building even medium-sized software is surprisingly complicated, so finding ways to frame it nicely really helps with making better decisions. Since time is a precious resource, getting to the right code as quickly as possible is important. Seeing it all as a system occupying some territory in an endless field is a good guide.
Subscribe to:
Posts (Atom)