Analysis for software projects is the act of gathering, curating, and collating enough detailed information about a set of given problems that people would like to use software to solve. If it is complete enough, then it can be used as the sole reference for people creating designs for these solutions. It has all of the answers to all of the questions that a designer will have.
All software systems are anchored by the data that they collect.
The usage of these systems is confined within the boundaries of the data that they have access to. In that sense, this data sets the foundation on which any other features or functionality are built.
If there is a problem for which one wants to use software to solve it, then obviously the first and foremost thing that needs to be done is to figure out what data is necessary.
Data comes in two basic forms: a) you collect it directly from people, b) you cross-reference it from somewhere else.
So analysis goes through each and every datum and tries very hard to list it out with proper names, some understanding of expected frequency, location, properties, etc.
The more you know about the data, the smoother the development process will be.
For some data, it is not worth collecting literally; you can get it from relatively easy computations based on other raw data. So, the analysis needs to specify any ‘derived’ data necessary and how to compute it from the underlying raw data.
Since the software is a ‘solution’, it is important to understand how people will use it in order to solve their problems. Most of the time it is not ‘log in and click a button’. It is a considerably more complex ‘workflow’ where the users leverage specific functionality in a bunch of features, over a multi-step process in order to make decisions or create specific artifacts that they share.
The analysis needs to list out all of the known workflows, in as much detail as possible. When there are misfits with using the software, they begin and end in these workflows. That is, if it is only possible to do half the work in the system to accomplish something, then the system isn’t a particularly tight fit. If you can do it all and there is just one button to press, it is really tight, but if there is a lot of functionality in the system, the problem switches to being able to find that button, not pressing it.
Presenting data for decisions is straightforward, but then minimizing the navigation after that decision is critical too. If someone decides that the data indicates a specific action, it would be best if the button to trigger that action was presented at the same time, or at least very close by. So, presentations need to be linked back to their lists of expected actions. That is, you need to know why the data needed to be presented in order to wire up the navigation for the decisions.
Creating new artifacts is similar. They are essentially manually triggered derived data, but the tools to create and edit them can be very complex. The tools need to be arranged in a way that matches the expected usage for creating stuff. And more importantly, the tools should leverage as much of the existing data as they can; the user should not have to retype things, just rearrange them.
For cross-referenced data, there are usually performance vs. quality decisions that need to be made. If you grab the data early and keep it around, there is a chance it can get stale before you update it. That would be worse than if the process of using or presenting the data is really slow. So, there are accuracy expectations for all cross-reference data that need to be taken into account.
The analysis of a set of problems should give enough details about people dealing with those problems that a solid design can be produced. Analysis is nearly timeless. If the business does not change, the “business logic” within does not change either, even if the system gets reimplemented with many different technologies over the years. So, the descriptions, references, tables, etc. in the document should not reference, rely on, or be tied to any technology. In order to deal with a problem, you need to collect data and work with it in the following ways. It’s the ‘what’ portion of the issues, where the design document is the ‘how’.
For example, you might need to know a derived metric like the ‘yield’ of a given bond in the financial markets. There are many different ways to calculate that, all based on the bond’s prospectus. The yield is a real number, which for many technologies is represented using floating-point variables, although there are lots of other ways of representing it.
Analysis should really only be done once for a new line of business, then updated over the years and decades as the underlying domain changes. In this sense, it is independent of any sort of software implementation, and the work to keep it up to date should be disconnected from any IT initiatives. A business should be constantly working through its own analysis in order to keep its own line of business operating as well as possible. If that were done, then software projects would start at ‘design’ instead of ‘analysis’, which would likely take less time and improve their quality.
The Programmer's Paradox
Software is a static list of instructions, which we are constantly changing.
Thursday, November 19, 2026
Thursday, November 12, 2026
Unbounded Loops
[ Editor's note: I wrote this because I forgot, again :-( ]
Unbounded loops are bad. If there is a bounded loop that can be used instead, use that instead.
Code really has two major flow-of-control constructs. One is conditional execution. You execute the code “sometimes”. We see that as ‘if’ statements, but there are all sorts of variations on that theme.
The other is loops. Loops come in two flavours, bounded and unbounded. A bounded loop is one that has the number of iterations effectively set in advance. A good example is a ‘for’ statement, with a variable called ‘i’. It starts at 0 and finishes at some ‘size’. The 0..size is the bounds for that loop. It will “always” stay within them.
Sometimes we don’t have bounds. A REPL is a good example. As long as the user keeps typing in text, the loop will continue to run. At some point, the user types in something special that triggers the code to ‘break’ the loop. Could be ‘exit’ or ‘quit’ or ‘EOF’ or something. It could just be a dynamic terminal triggered by the end of work. There are plenty of other places, like communication protocols, where there are often unbounded loops. But all of them are special.
Sometimes, people encode their loops as unbounded, even though they are essentially bounded. If encoding it as a bounded loop is only slightly less convenient, then it should be encoded as a bounded loop. Being clever with loops is really bad.
The reason for this is that it is self-describing. If you hit an unbounded loop, you might have to search around through a whack load of spaghetti to find its bounds. That’s a huge waste of time.
Another good reason is computational complexity. If you are coding loops, you really, really, really need to know how long they are going to take. If it loops on 3 or 4 things, you can quickly ignore it, but if it might loop on a million things, you can’t just chuck code like that out into production and hope for the best. One day it will fail, and people will be angry about it. If it can loop ‘that hard,’ then at bare minimum you’ll need some sort of testing to be in place so that you have the certainty that it won’t be you causing a denial of service attack on the system.
So, if you quickly read the code, and you can instantaneously pick up all of the bounds, then often in your head you can have a reasonable degree of certainty that when this code gets out there in the world, its behaviour will not sufficiently degrade from the behaviour you’ve been seeing while you worked on it in development.
That certainty is everything.
You have become an expert in programming because the code you write does only and exactly what it should do relative to your expectations. Obviously, that doesn’t mean it is “right”; it just means that when it runs, you will not be surprised, and you really don’t want surprises, do you?
Thursday, November 5, 2026
Difficult Problems
In software, these days, if there is a fleet of existing interoperating systems, it is most likely that when they were developed, they focused on solving the majority of the easy problems.
Often, it is also the case that, in doing that specific work, they made any related difficult problems far more difficult.
What this usually means is that if things are not working smoothly, then there are probably no longer any easy solutions that would fix the mess. The development has painted itself into a corner.
Anything that would make significant progress in correcting some or all of these deficiencies is going to be extremely difficult now.
Why?
The obvious point is that the current solution is fragmented; e.g. there is more than one interconnected system. The reason they are fragmented is that people kept tearing off smaller parts and solving those. That behaviour is a side effect of people zeroing in on the easy stuff and then trying to solve only their bit in isolation. Thus you get a bunch of systems where interconnection is an afterthought.
If you have a space where there are a lot of these smaller independent fragments, not only do they, as separate pieces, cause artificial complexity, but also their focus, as tunnelled from the larger whole, forces them to skip over difficulties. That is, they are so focused on just getting their thing done that they do not care about the whole thing, so if they hit something difficult while doing their thing, they punt that out of their space. This leaves the space between all these systems filled with all of these punted difficulties. When this has been the approach for a while, most of the easy stuff is gone, and what remains is darn scary. The overall complexity of the Frankenstein mess is orders of magnitude higher than any sort of unified, difficult solution.
What to do?
Really, you have to go back over the mess and identify the difficulties. Pick a big one, start there, and grow it out to cover back over the easy stuff. That is, if hyper-focusing on solving simple things got you into this mess, then only hyper-focusing on difficult things will get you out of it.
Most difficult problems in software aren’t really that difficult; they are just difficult in a very restricted context. If you pull up and out a bit, there is probably a known way to get just that problem solved correctly. It definitely won’t be a cakewalk, but it is still likely solvable.
Given that, you take this difficult problem, a known thorn in everyone’s side, and you write up both a specification of the problem itself and a design for something that would fix it. With those, you see what other existing parts of other systems they now cover. Then, if you are sure that would really fix the difficult issues, you work out a slow, phased transition to get that difficult core into place, and gradually extend it out to everything else. If you keep at this, eventually everything else disappears, and what you have left is one single integrated, fully functional, “difficult” solution that works smoothly.
I’ll say it again. Simple solutions do not solve difficult problems; they only fragment them into bigger messes. If the problem is difficult, the solution needs to be at least as difficult. There is no getting around this. You cannot avoid difficult problems; you can only ignore them for a while. I’ll call this the first law of solutions.
Often, it is also the case that, in doing that specific work, they made any related difficult problems far more difficult.
What this usually means is that if things are not working smoothly, then there are probably no longer any easy solutions that would fix the mess. The development has painted itself into a corner.
Anything that would make significant progress in correcting some or all of these deficiencies is going to be extremely difficult now.
Why?
The obvious point is that the current solution is fragmented; e.g. there is more than one interconnected system. The reason they are fragmented is that people kept tearing off smaller parts and solving those. That behaviour is a side effect of people zeroing in on the easy stuff and then trying to solve only their bit in isolation. Thus you get a bunch of systems where interconnection is an afterthought.
If you have a space where there are a lot of these smaller independent fragments, not only do they, as separate pieces, cause artificial complexity, but also their focus, as tunnelled from the larger whole, forces them to skip over difficulties. That is, they are so focused on just getting their thing done that they do not care about the whole thing, so if they hit something difficult while doing their thing, they punt that out of their space. This leaves the space between all these systems filled with all of these punted difficulties. When this has been the approach for a while, most of the easy stuff is gone, and what remains is darn scary. The overall complexity of the Frankenstein mess is orders of magnitude higher than any sort of unified, difficult solution.
What to do?
Really, you have to go back over the mess and identify the difficulties. Pick a big one, start there, and grow it out to cover back over the easy stuff. That is, if hyper-focusing on solving simple things got you into this mess, then only hyper-focusing on difficult things will get you out of it.
Most difficult problems in software aren’t really that difficult; they are just difficult in a very restricted context. If you pull up and out a bit, there is probably a known way to get just that problem solved correctly. It definitely won’t be a cakewalk, but it is still likely solvable.
Given that, you take this difficult problem, a known thorn in everyone’s side, and you write up both a specification of the problem itself and a design for something that would fix it. With those, you see what other existing parts of other systems they now cover. Then, if you are sure that would really fix the difficult issues, you work out a slow, phased transition to get that difficult core into place, and gradually extend it out to everything else. If you keep at this, eventually everything else disappears, and what you have left is one single integrated, fully functional, “difficult” solution that works smoothly.
I’ll say it again. Simple solutions do not solve difficult problems; they only fragment them into bigger messes. If the problem is difficult, the solution needs to be at least as difficult. There is no getting around this. You cannot avoid difficult problems; you can only ignore them for a while. I’ll call this the first law of solutions.
Subscribe to:
Posts (Atom)