Thursday, May 28, 2026

Versioning

If you start from the premise that a system is just a series of access points into a vast array of computations, then if you accept that there will always be a huge number of changes to this underlying code, you see why this is messy.

All the computer is really doing is taking a bunch of inputs, grinding through computation, then spitting it out. But we often end up changing these computations, sometimes because we had obvious or subtle bugs, sometimes because we’ve acquired new knowledge about how to do the work better, more accurately, or much faster.

At the high level, people interact with all of these access points, apply some variability to them, and then set the computations in motion. It might take a millisecond, an hour, or even a few days. The interaction might be rapid (real-time-ish), or it might just be infrequent. What is important to these people is that they can trust that the computer does the thing that they expect it to do. Trust is the bedrock.

We can skip over any sort of difference; the output is either a blob of text or a pretty little graphic of some type, it’s all just variations on presentation. The text could be typed and structured, which doesn’t matter either.

In order to trust the system (app, program, plugin, etc.), one key property is that the behaviour has to be ‘stable’. It should not change day to day, hour to hour. If you used it yesterday to do something, then you expect that with the same inputs, it will do pretty much the exact same thing (determinism). An added expectation is that if there were changes, then mostly those changes would be adding more features, not changing the old ones.

This is the core of what we call backward compatibility. Most programmers think of it in terms of APIs they are calling that are stable, but really, it is an overall property of the system itself. It is backward compatible if and only if any interactions, code, or humans are fully preserved after endless updates. If it worked ten years ago, it will work exactly the same today.

There is a loose exception for bug fixes, particularly bugs that have rendered the functionality to be useless. These are obviously not expected to be backward compatible, as the old behaviour does not correctly match expectations, so it needs to be changed to something else.

This relation is expressed nicely in using three-digit version numbers.

The last digit is bumped up for 1 or more bug fixes. Going from n.n.10 to n.n.11 means that at least one bug was fixed, maybe a dozen of them.

The second last digit is listed as a minor enhancement, but honestly, it is really just there for added functionality. Nothing else changed, nothing was deleted. So, if n.10.n is bumped up to n.11.n you expect backward compatibility for all of the existing functionality, and there is now some new functionality included.

That leaves the first digit to clearly state that you have broken backward compatibility. It is essentially a flag. There is some change that is major enough that the user is not necessarily going to be able to expect the code to be deterministic. Something big changed, and it will be noticed.

If people were strict in the usage of version numbers, and if they respected the notion that any 0.n.n version was just an initial demo or test instance, then even if the system was under active development for years, if it was backward compatible, then 1.9343523.14 would be a reasonable version number. 9M times new features were added, but the rest is still intact. Lots of stuff has been added, but all of it is backward compatible. The last round of added features needed 14 tries to get the bugs knocked out. All of these should have been in testing.

As it is with user interfaces, it is true for any pure computational dependencies below. Libraries, frameworks, languages, tools, etc. Strict usage of the version numbers is enough to get a very strong sense of both how the development is going and whether the authors even understand backward compatibility.

Probably the most embarrassing self-inflicted mistake a software developer can make is to push out a release that immediately crashes due to an underlying dependency change. If they were doing things reasonably, this would never occur. At minimum, an embarrassing non-backward-compatible library change would get picked up in testing. Untested code should never, ever get into a release. Subtle changes could slip through, but at the bare minimum, the work should have been smoke tested to catch exactly this sort of mistake. But the stronger habit is to only upgrade questionable libraries at the beginning of a long development cycle, while also doing lots of non-destructive refactoring. That is, before dumping in new stuff, you tidy up the junk from the last release and update some of the libraries. Run it a lot yourself until you are sure it is stable, then you go to town to add new stuff.

If the library is any good, and it has done an excellent job at being backward compatible, this is extremely low risk. You can kinda cheat the game sometimes. But if it is some dodgy little thing written by a couple of people as an advertising attempt, then you would have to wrap it in very expensive testing for each and every little thing you’ve used it for. It’s this that makes most libraries not worth integrating, either because the testing is too much work or the risk is just way too high. Reading the code and applying some of its better ideas is more suitable.

Often, you can get a sense of the quality of the library just by looking at the version numbers. For instance, 24.3.2 is a suspicious number if the work is only a couple of years old. They’re not taking backward compatibility very seriously; they are high risk.

It comes across with some of the larger tech stacks, too. If there is a major version bump that fundamentally breaks all backward compatibility, but someone has the newer and older versions haphazardly laid on top of each other, you pretty much know that the confusion caused by being too loose with the versioning is going to cause a lot of chaos that will either waste a lot of your time or result in embarrassing bugs. If the break was wide enough, the new work should really abandon the ‘brand’ of the old work. They are two different things, even if that means it is harder now for the new version to get a lot of traction. Just because you decided to change it doesn’t mean everyone else in the world should change too. Once you’ve committed to a particular set of computations, you have to stay committed and only grow from there. You can’t just pick up and move to some other spot farther away and claim it is the same work; it is not.

Backward compatibility is hard, really hard, which is why everyone loves to cheat the game so much. But it is an essential property of stability, which is necessary for trust. If you want to do a good job providing some complex computations to others, it is going to be hard. There is no way to avoid it. If you do the hard work, then you can communicate it quite clearly with the version numbers. That will let people know that your work is serious.

Thursday, May 21, 2026

Feedback

Recently, my blog has been getting a lot more views. Unfortunately, a lot of the incoming fields for these reads are just tagged with ‘Other’.

That tells me that the traffic is not coming from the older established sites I know, like HackerNews, but is either fake traffic or newer sites that I haven’t seen. It would be nice to know which is correct. Are people actually reading these posts?

So, if you are reading this, I’d really appreciate you taking a moment to comment. Anonymous is fine, and since my comments are screened before they are published, feel free to say ‘do not publish’ if you want. A ping is good; mentioning the source helps.

A long time ago, I briefly dreamed of monetizing my writing, but as I realized that the way to do that is to effectively change what I am saying, I decided not to do that.

I’ve always had to be careful not to upset any of my current employers, but beyond that, I write what I know, either from firsthand experience or from conversations with others.

Because of that, and my limited writing style, it’s never been a popular blog, but I still feel, after decades, that I want to get what I understand down somewhere. Maybe people read it, maybe not. It’s okay.

The software development industry varies hugely, so not surprisingly, plenty of other people have had very different experiences in their careers, but I also do suspect that there is way too much propaganda out there that is deliberately trying to mislead people. It’s an immature, messy and often ugly industry.

With all that in mind, if you could take a moment to say ‘Hi,’ at least I’ll know if you really exist or are just a figment of the web’s imagination.

UPDATE: Ok, I got a few responses, which is great. Thanks! Seems like at least some of the traffic is RSS and Atom, which doesn't show up in the stats. It might be those views where I do get a country and browser type, but that still leaves a great deal of traffic as Other. I guess I'll never know if those are real or not.

If anyone has suggestions about future topics, that would be great too. I feel like I am getting too repetitive in my old age :-)

Thursday, May 14, 2026

Security

Programmers hate adding security to their systems.

First, it is a huge amount of work, and second, since it is so often left to the end, it is very ugly and disruptive work. A patchwork of hacks.

But it’s misdirected. Without enough security, the system they built is useless, well, worse than useless. If people use it, it could severely screw them over. Nobody would intentionally use something that helps criminals more than it helps them. Even if it is in a walled garden, you can never really be sure that someone isn’t motivated to take a peek.

It’s worth noting that I am not a security expert, and although I’ve had to deal with it a lot in my career, my practice might not be as strong as the experts would like. That being said, I’ll continue.

There are only a few things you need to worry about in security. First is actually identifying any ‘users’. You always have to know who they are and have enough confidence in that decision that you don’t make a mistake.

Then the other part of it is that you want to protect both the data and the code from anyone who isn’t supposed to see or activate it. It’s not enough to protect just the data or just the code. You need both.

In that sense, security isn’t that hard if it is your concern from day one. There are a bunch of entry points that people will use to get to the features and functionality. First, you identify them, then you check to see if they can access the given functionality. If they can, then lower, you check to see if they can access all of the data input into that functionality. If they can, then they can see the output. Simple 🙂

Here’s where the trouble starts. First, there should be no anonymous endpoints. But people love adding them, but they open the door to leaks or denial of service attacks. If you have none, though, all of that goes away. If you can’t quickly identify someone right at the top, punt them immediately, send a log to some administrator. They might have to block the incoming address or put up some firewalls to stop botnets and other nasty things. You always need to flag a punted user as a serious problem.

Second is databases. For capitalist reasons, they charge by users, so the system users are not the same as the database users. That sucks, and it has always sucked. Life would be pretty easy if a person’s identity propagated all the way down to the metal. It should.

If there was a necessity for a group or functional account shared by a bunch of people, then the group is the identity, and that identity goes all the way down.

If your database or its license makes that impossible, then you need to wrap it. You need to wrap it thoroughly enough that pretty much nobody can get to it in any way without first passing an identity check. So, not just in the backend code, but also on the machine in the scripts, with the OS, etc. Everywhere.

Wrap the database. It won’t make it convenient, since that is the opposite of secure, but you need to do it.

Now, at the top, after you have checked identity, you take a quick look at whatever functionality is called. Are they allowed to use it? In some extreme cases, that is a messy lookup table, and it needs to be managed by data admins. It’s annoying, but really, in a large organization, that really should be a distinct piece that is shared by a whole bunch of systems. You just check with it, user X wants to call foo, is that okay?

If that’s good, then as the code executes and hits the wrapped database, the second check will trigger on the data. If it’s good, then it is all done. If you always reuse both the high and lower levels, then the security will be everywhere, and you don’t have to lie awake at night worrying about it failing.

The only other part is that if a user ever sends you ‘code’, you laugh and reject it. If you want some cool dynamic execution feature, great, but there have to be two paths, not one. The code comes in from somewhere else, having been fully and completely vetted, and then the user later asks for it to execute dynamically. That keeps it really simple, and sets you up with some external means for this uber dangerous code to be properly managed, vetted, and approved. That in itself is a huge task; you can’t just ignore it and hope for the best. Dynamic code can never be ‘open’ dynamic code; it has to be closed and come from a reliable source that actually has to be more reliable than just reliable.

So, in the end, if you wrap the database, always identify everyone, manage a lookup table or two, and punt anything that could ever be possibly executed by any downstream party or library, then you’re done. All of this code is reusable; you just need to do it once, at the beginning of the project, then leverage it for success and glory.

Thursday, May 7, 2026

Goodness

It really feels like the world has sunk to its lowest point in my lifetime. And it does not seem likely to improve anytime soon. We’re sliding downhill.

When I first started playing with computers, way back in the 80s, I felt like they had huge potential to help humanity. To lift us up, but it seems like they did the opposite. First, they trapped us; now they are forcing us to regress.

It’s not the computers themselves, but the types of monsters that latch onto them in order to make money, grab power, and manipulate people. Sadly, they find it too easy to use software to do this.

It seems like software developers made a rather tragic mistake in not preventing this earlier. We were just too eager to build stuff; we didn’t ask enough questions.

But the good news is that we can still do something about it now. We can build all new stuff that is empathy-driven and meant to really help people, not just pick their pockets or force their behaviour.

The trick is not to get hyper-focused on the technology itself; it doesn’t really matter. Instead, we put ourselves into the shoes of the users. Software without empathy is just a weapon waiting to be exploited.

The problem has always been that empathy-driven software is extraordinarily hard to write. It’s not just getting the technology to dance, or flooding it with domain data; you have to integrate all that very carefully into the full context of a user's life in order to shave off any of the hard spikes. It’s not just code and data; it is code and data that deliberately help people. It all emanates from their perspective, not from the builders or the operators.

To get us going, I’d suggest that everyone just start throwing any “non-monetizable” ideas they have out there. Pick a problem you know, write up a dream solution, and publish it. It doesn’t have to be practical; it doesn’t even have to be possible. It’s not about technology, but about seeing the world from the user’s perspective and making suggestions about how to really improve it. Too often, we first focus on technology, and then we try to shove it back into the solution space. That doesn’t work very well.

Once we have ideas, we can figure out how to implement these as solutions in ways that can’t be subverted by monsters. That, of course, is the difficult part. Serious software is still very expensive to build and run, and the cost of getting it funded has a lot of painful strings that we’ve seen over and over again are used to pull the efforts off in very bad directions.

If we can figure that out, then we just need to find a way to swap these technologies with the mess we’ve got right now.

I’ve occasionally dumped out some raw ideas:

https://theprogrammersparadox.blogspot.com/2024/05/identitynet.html

https://theprogrammersparadox.blogspot.com/2015/08/digital-discussions.html

https://theprogrammersparadox.blogspot.com/2009/04/end-of-coding-as-we-know-it.html

They were mostly unfundable, and since I had needed to pay the bills, they were beyond my ability to take further. But I’ve always thought it would be cool if someone was inspired to do something similar, so I wrote them up.

Other areas that desperately need our attention:

Source of Truth. I appreciate and admire Wikipedia, but I really want something more structured, like an ontology built on graphs or even hypergraphs, that contains all of human knowledge or at least as much as we can capture right now. It would assign a probability to any “knowledge”. For instance, a known mathematical proof would be 100% correct, but most other things we think are true are at best 99ish. And the myths and falsehoods are really low, maybe even 0. If there were multiple competing opinions, they would all exist in the data, but with some percentage of likely truth (as of today). It would be worldwide and not controllable by any country or dictator. Untouchable by monsters. A perfect use case for decentralization.

Privacy. We need to protect any facts about individuals, but still provide some (difficult) means of external verification. This would extend to group conversations as well. Some part of it would only allow retrospective external access if and only if the case made it to a territorial court accepted by all of the individuals. That is, they can’t spy on you, but if you did something bad in some jurisdiction that you have accepted, the information could be retrieved if there were actual court proceedings. It’s the notion that they have to do the policing legwork to catch you, but once you are in trouble, the whole truth will come out.

Time/Complexity Simulations. Being able to list out the consequences of a given decision over a complex circumstance. Lots of moving parts. You could throw together some approximate complexity for something, then play with any possible decisions to see how they fare in the long run. We need this, as too many people can’t see beyond extremely short horizons. Even if it is crude, it would help people think about more than just tomorrow, or next month, or next quarter. If you could come back with a chance that there is a 48.2% that “doing that” would turn the profits negative in the next seven years, it would be harder for someone to just forge ahead blindly. Or that there really are “century” events that we do need to protect against, like pandemics.

Consolidation. It sucks having to rely on dozens of different, widely inconsistent apps. Their collective value is eroded by the combined cognitive load. I’d want something simpler than a spreadsheet that brings together the common data and can trigger code in all sorts of remote places. A customized gateway that makes it easy to leverage the power of a computer, but just for you. The trick is to breach the complexity limits that so often hold us back. The abstraction that holds it together can’t be too abstract but still needs to be powerful enough that it is all-encompassing. If I could configure it for all of the repeatable parts of my life, like a crazy, distributed, super-integrated to-do list, with behaviours and data shareable for a wide range of scenarios, it would be my first point of contact on all my computers. It would have some deep way of reorganizing itself as I keep adding more to it. The key, though, is that it isn’t a remote service; you don’t rent it. You own it, it is yours, it can be seamlessly upgraded over the decades of your life, and it is fully private. The costs are trivial, but it will consume your time. There are parts you can share if you want, but there would never be a way to make money off your contributions; all you get for your efforts is a better life.

Guardrails. Lots of awful stuff happens on the web. Why? Why can’t we keep the good qualities of the Internet, without continuously opening doors for the bad ones? My guess is that capitalism drives an unquenchable thirst for monetization, so making that safe is just too costly. Eats into the profits. So we get half-baked stuff that eventually the monsters figure out how to leverage. From that perspective, it seems like we could put up some types of walls and fences that would protect this weak code from being exploited. Protect private data from going anywhere. You shouldn’t be subject to an attack unless you explicitly lowered your guard. It shouldn’t be possible to trick you into lowering your guard. All of the angst from this not being true today piles on the friction that devalues the capabilities of the computers. Finding a way to stop that is huge.

I’m sure there are a million more issues and ideas out there. Now is the time to flood the world with them, and then maybe we can figure out how to bring the best of them to life. If you do this, odds are you won’t get much credit, and it definitely won’t make you rich, but it is still a good thing to do, so it is worth the effort.

What we ultimately want is for computers to make our lives easier and more meaningful. To take away some of the drudgeries and difficulties of reality, but not numb us into a coma or stupor. Sure, we’ll still turn to the machines for assistance, but we won’t get caught in negative incentive loops like doom scrolling. We will live life in reality, not digitally.

To get this, we need to stop the people who are financially motivated from bending all of the technologies against us. They only see the bad potential, realize their use in carving off profits, and then find ways to slip these into our lives. They’re tearing us apart so they can own mansions, sports cars, jets, and yachts. We have to stop allowing this.

Thursday, April 30, 2026

Shortcuts and Makework

On the face of it, shortcuts and makework may seem like they are opposites.

A shortcut is a faster way to do something that effectively pushes out the consequences down the road. You take the quick and easy way now, only to pay for it later.

Makework, on the other hand, is anything that you are made to do that does not directly or indirectly contribute to the work at hand. For instance, you fill out a complicated form with copious details that is ultimately ignored forever.

Makework is usually some people trying to control or throttle others, often an abuse of power, or a justification of their value.

In bureaucratic organizations, the centralized control over poorly understood aspects of the company is usually thick with makework. There are plenty of administrators trying to control things that they do not understand. Thus, the rules and processes get weird and form the basis for lots of politics.

But the two are oddly related. Where you see one, you usually see the other.

The root cause is time. There is a project that has a tight timeline. But the people working keep losing huge blocks of time to makework. However, as makework is an integral part of the organization, blaming it for being late is not allowed. So, in order to try to catch up, they resort to a lot of shortcuts. The long-term consequences don’t matter if, in the short term, you will get in trouble for being late. The context of the project forces mistakes and panic.

It gets triggered the other way, too. Some people just take shortcuts out of habit; the project looks initially crazy successful. But as the long-term consequences come due, it collapses. In the downfall, lots of unrelated people jump in to “help”, like bureaucrats and generic management. Since they don’t understand and they don’t know why a once successful effort suddenly flipped, they propose a lot of work that they believe will fix the problem. More tracking, more documentation, more sign-offs, more meetings, etc. But all of this is effectively makework, and the real problem of replacing the shortcuts causing all of the grief gets ignored. The project ends up under the microscope, which amplifies its problems and does not correct them.

Mostly, though, the best approach is to be rigorously practical. Minimize both shortcuts and makework. Carefully assess any and all effort with respect to both of those categories. If it smells like one or the other, don’t do it. Get the core work done as best as possible.

The other part is to tackle the hardest parts first, don’t leave them for later, and don’t rush through them. While that gives the appearance of being late right from the get-go, it provides two valuable properties.

First, if you get stuck, you can raise a late flag early, rather than later, which tends to mitigate some of the bureaucrats coming out of the woodwork and drowning you with makework.

Second is that a shortcut on the hard stuff is way more destructive than a shortcut on the easy stuff. If time forces you to take shortcuts, then the ones with minimal consequences are far better. They are less costly to repair later. If the foundations are solid, you have a better shot of recovery when late. In many organizations, you are already late long before you even realize that you have work to do. It’s normal, so you need to adopt habits that mitigate it.

The biggest problem, though, is that coming up with shortcuts or makework is often a lot easier than doing things properly. It’s the easy path for both the workers and management. But it is an unsuccessful path too. It distracts from the things that really need to get done.

Ultimately, there is some work that needs to be finished with at least enough quality to keep the detractors at bay. Do that work, don’t get lost by trying to avoid it.