Thursday, December 21, 2023

Software Knowledge

There are at least two different categories of knowledge in software.

One is the specifics in using a tech stack to make a computer jump through specific hoops to do some specific computations. It is very specific. For example, the ways to store certain arrangements of bits for persistence when using Microsoft stacks.

When you are building software, if you know how to do something, it will make the work go faster. But people are right when they say this type of knowledge has a half-life. It keeps changing all of the time, if you haven't done it for a while, you’ve forgotten it or it has moved out from under you.

This is the stuff you look up in StackOverflow.

The other category of knowledge is far more important.

There are all sorts of ways of working and building things with software that have not changed significantly for decades. They are as true now, as when I started forty years ago. They are the same no matter what tech stack you use, be it COBOL or JavaScript. Sometimes they are forgotten and then reinvented later under different names. A good example is that we used to edit our code, now we refactor it.

Fundamentally, building software is a construction effort. The medium appears as more malleable than most, but it is not immune from any other constructive issue. And because programmers intentionally freeze far too much, we change things too fast, and often need backward compatibility, it is rarely as malleable as it could be.

There are a couple of obvious anchors.

The first is that size matters. It is a whole lot easier to build tiny things than massive ones. Size is everything.

The second is that as the size of the effort grows, disorganization causes worse problems. If you write some tiny spaghetti, it is okay, you can still change it. But if you have a million lines of spaghetti you are screwed.

Organizing stuff isn’t fun, and oddly it isn’t a one-time task either. It is an ongoing effort. The data and code are only as organized as the explicit effort you put into them to keep them organized. If you aren’t doing anything, it is likely a mess.

But even more specifically, there is a lot of general knowledge about how to code things like data structures, algorithms, data normalization, or GUI conventions that hold true regardless of the stack. You may not need to create a hash table yourself anymore but you still need to understand how to leverage it and its limits. People will always need to ensure their data is at least 3NF or they will pay the price for storing it badly. A poorly wired GUI will diminish trust, it may be marginally workable but generating ill will.

The tools too. Learning to properly configure and use an editor or IDE tends to stay relevant for a very long time. There are all sorts of build tools and scripting, most of which haven't changed for decades, although sometimes they get obscured by trendy stuff that doesn’t last. But the need for the tools and usage of them never changes. If you spend time to figure one out, the others come easily. It also helps in understanding why some newer trends are poor and should be avoided.

Of course, all of the issues with people and politics never, ever change. We build software as a solution to some users' problems. If you don’t fully understand what you are trying to solve, then the things you’ve built are far less likely to work as needed. There is also a lot of gymnastics involved with funding software development, often resulting in too much stress and rushing through the work. Stress is bad for thinking; rushing is bad for quality.

Most of what you do specifically in software changes. The trends come and go in roughly five-year waves, developers need to keep up but not every wave. You can skip some waves, but if you skip too many your opportunities narrow. Once you are old and out, it is brutal getting back in.

Most of the general knowledge is far more important than the specifics. It is what keeps the projects from chaos, ensures that the work is at least good enough, and helps control the expectations of the people on the margins. If you know generally how to build things well, you can always look up the specifics. But if you don’t know how to properly persist the data, for instance, the work is doomed before it even starts. If you don’t understand fragmentation, you won’t understand why your work keeps failing when you bring it all together. If you don’t understand the components, you cannot craft a reasonable architecture.

You actually need more general knowledge to ensure that a large project is successful than specific knowledge. It is what keeps it all out of trouble while people are coding like mad. This is probably why the high failure rate of modern software is independent of the methodologies used. It's more likely experience-related. A bunch of kids who really know their specifics well will still usually fail in very predictable, general ways. That was true when I was a kid and it still holds true today.

No comments:

Post a Comment

Thanks for the Feedback!