Quality

On the outside is the irrational world of the users, that has evolved chaotically over generations. It is murky and grey, with a multitude of intertwining special cases. It is ambiguous. It is frequently changing, sometimes growing, other times just cycling round and round again. The users have turned towards the computer to help them with this problem.

On the inside, deep down is the formal logic of a discrete system. It is nearly black and white, pure and it can be viewed with a rational, objective lense. There are plenty of theories, and although there are intrinsic trade-offs such as space vs. time, they can be quantified. If a system is built up well-enough from this base, it is a solution waiting to be applied.

Software is a mapping between these two worlds. It is composed of two distinct pieces: the computer code and the equally important underlying structure of the data that is being collected and manipulated. The quality of a mapping depends on the combination of both code and structure. A good piece of software has found a mapping that is effective; useful. As time progresses that mapping should continue to improve.

Both sides of the mapping are tricky, although the outside also has that extra level of irrationality. This leaves considerable leeway for there to be many different, but essentially equivalent maps that would act as reasonable solutions. There is no perfect mapping, but there are certainly lots of ill fitting ones.

To talk about quality in regards to software, we can assess this mapping. Mapping problems manifest as bugs and they diminish the usability of the software and its collected data. A good mapping needs to be complete, and it should not have gaps nor overlaps. It doesn’t have to solve all of the problems outside, but where it attempts to tackle one it should do it reliably. This means that all of the greyness on the outside needs to be identified, categorized and consistently mapped down to the underlying formal mechanics. Solving only a part of a problem is just creating a new one, which is essentially doubling the amount of work.

Because software development is so slow and time intensive, we also need to consider how the mapping is growing. Having a good initial mapping is nearly useless if the pace of development rapidly degrades it. Mappings are easy to break, in that small imperfections tend to percolate throughout. The longer the problem has been around, the more work it requires to fix it. For code, that is relative to adding more code, but for the structure it is relative to adding more data, so that imperfections are always growing worse even when no active development.

As a map, we can talk about the ‘fit’, and it is in this attribute that we can define quality. An ill fitting map is qualitatively poor, but as noted there are many different maps that would fit with higher quality. If we decomposed the map into submaps and assess their fit, we can get a linear metric for the overall system. The most obvious indicator of not fitting is a bug. That is, a system with 70% quality means that 30% of the code and structure has some form of noticeable deficiency in their usage. If you could decompose the code into 5 pieces and the structure into another 5 (ignoring weights between them), then if 6 of these pieces are complete, while 4 of them are really bad and need to be reworked the quality is 60%. If however, all ten pieces contained problems, then at least from the top-down perspective the overall quality of the system is 0%.

That may seem like an excessively brutal metric, but it is mitigated by the fact that we are only considering problems that directly bother the users, the future developers and the operations people. That is, theses are bugs, not trade-offs. Thus if the system is slow because the code was badly written, that is a bug, but if it is slow because it was weighted on a space trade-off made because of hardware limitations then unless the hardware situation changes it is not really a bug.

Obviously this metric is similar to measuring a coastline, in that getting down to finer details will change it. If you decompose part of the system into a single module of 100 lines with a bug, then the quality for the module is 0%, but if you isolate only the one line that is bad, it seems to jump back up to 99%. Quite the difference. But that points to a significant fundamental property of the mapping. Some parts of it are independent, while other parts are dependent. In the previous case of the 99 lines of code, if they are all dependent on the bad one then all 99 are tainted, thus the quality really is 0%. If only 49% are dependent, then the quality is 50%. A line of code is meaningless in isolation, it only has value when it is brought together with other lines within the system.

What we a have to assume with code though, is unless proven otherwise, there is a dependency. That again could be problematic. A programmer could see a huge code base with only 1 bug and assume that the quality is 0%. We do this quite naturally. But that illustrates the need for an architecture in a large code base that clearly delineates these dependencies. If the code can’t be easily decomposed, then the mapping is unlikely to be changeable, thus the quality really is lower, although probably not 0%.

Now that was for code, but the situation is even more dire for the underlying structure of the data. If you build a large system on a bad denormalized database, then those underlying bugs taint all of the code above it. Code written to collect bad data is inherently bad code. This is a reasonable proposition, since the code is 100% dependent on that broken structure. If the database is screwed, the value of the code is too.

Reuse and redundancy have an effect as well. If you have a tight engine for the main functionality of the system with 100K lines of code, and you have an administrative part, badly written, with 500K of spaghetti code, you might assess its quality incorrectly. Lets use the Pareto rules and say that 80% of the usage is that core engine. If there are lots of bugs in the administrative stuff, from an LOC perspective it might appear as if the overall quality was less 20%. However, if the engine was great then from a usage standpoint it is clearly at least 80%. This flip-flop shows that the value of any line of code is weighted by its functionality, and also its reuse or redundancy. A line of code in the engine is worth 20x more than one in the administration. That extremeness of the weighting is not surprising, and should be captured by the architecture at a higher level, and driven by usage or business needs. That difference shows why good architectures are necessary to assess priorities properly because they help pinpoint both dependencies and weights.

Another significant issue is trying to apply a static mapping to an inherently dynamic problem. If the essence of the problem keeps changing, then all of the collected static data is constantly out-of-date by definition. The mapping needs to keep pace with these changes, which is either a rapidly increasing amount of static coding work or the dynamic elements must be embedded in the map itself. The former approach might appear quicker initially, but will not scale appropriately. The latter is more difficult, but the scaling problems are mitigated. This comes across in the quality metric because the completeness attribute is always declining, making the structure out-of-date and then associated code. Note that this happens whether or not the code is in active development. Dynamic problems gradually degrade the quality in operations as do changes to any external dependency. They are both forms of rust.

Now given this metric, and its inherent weakness, it can be applied to any existing development project with the obvious caveat that an outsider is mostly going to undervalue quality. That is, where someone doesn’t know the weights and they don’t know the dependencies, the results will skew the effect of bugs, which every system has. That’s not as bad as it sounds, because what makes a project sustainable is the knowledge of this mapping. If that knowledge is lost, or blurry, or incorrect, then from a larger context the quality just isn’t there. If all of the original developers left and the new crew is just putting a detached bag on the side of the system instead of really enhancing it, although this new work is mostly independent, it is also redundant and full of gaps or overlaps with the existing pieces, which is reducing the quality. This gets caught quite nicely by this metric. If the knowledge is readily available, as it should be if the system is well-designed, then that feeds into both raising the quality but also making sure the existing deficiencies are less expensive to fix. In that sense a ball of mud really does have the quality of 0, even if it has a few well-written sections of code and a couple of decent structures. Good work on a small part of a bad system won’t change the fact that it is a bad system.

This metric is fairly easy to calculate in that to get a reasonable estimate all you need to know is the number of lines of code, the architecture, the denormalization of the database, the current bug list and the current usage. Often, you can even get close by just playing around with the main features of the application and guessing at the underlying code organization from the user interface. That is, if the screens are a mess and there are plenty of places where the functionality should have been reused but wasn’t, and the structure of the data being captured is erratic, then you know that the overall quality is fairly low even if you’ve never seen the code or the database. Bad mappings propagate all of the way to out to the interface. They can’t be hidden by a nice graphic design.

As Software Developers we want to build great systems; it’s why we were drawn to computers. We can’t know if the work we are contributing is any good unless we have some means of assessing the quality. We can’t know if some work is really an improvement if we have no way of understanding how it affects the overall system. Because we care about doing a good job and we want to know that our efforts have made things better, we need a means of quantifying quality that is oriented to what’s important for the usage of the software. Having a notion such as technical debt is fine, but because it is so vague it is easily misused or ignored. Knowing that there are rapidly growing problems in the mapping however allows us to visualize the consequences of our short-cuts, disorganization and design decisions. Realizing that haphazard work or muddled thinking really does cause a significant drop in the quality gives us a much better chance of raising the bar. It also gives us a means of determining if a project is improving over time, or if it is just getting worse. What the world needs now is not more code or more poorly structured data, but reliable mappings that really solve people’s problems. Once we have those, programming is just the act of making them accessible.

Factual

If I had a lot of money I’d create a new website. It would be similar to Wikipedia, but instead of descriptive text it would only contain cold, hard facts. Explicit relationships between specific things relative to time.

Structurally, the underlying encoding would be some formulaic knowledge based representation of the relationships, thus avoiding any vague or misrepresented information. Just facts, pure and simple.

It should be noted however that what we often think of facts are not necessarily ‘universally’ true. That is, even the most concrete facts are relative to a specific context. For example, we know that the sun always rises in the east and sets in the west. That will remain true for our lifetimes, but there will eventually come a time when the sun ages enough that engulfs the planet turning it into a molten cloud (or something equally as unappealing). Truth with respect to time is a tricky thing.

What is also true is that much of what we think are facts, are actually just opinions. Humans can’t grasp the full context of our existence, so we over-simplify relative to our own perspectives. In that way, informality creeps into the underlying representation. Any given fact is believed true by only a subset of our population, and there are many different subsets all with their own variety of beliefs. What is a fact to one person is guaranteed to be a falsehood to another. Only the purest of abstract mathematics is immune to this property.

To account for this, the underlying representation would contain everything that anyone would consider factual. It would, of course, also have some metric to show how popular one version is over the other, but it would allow for the encoding of all ‘alternate’ truths, theories or perspectives.

Each fact would also have an associated list of links back into the Internet, and an associated list of publications back into meatspace. These references would be there to backup the justification for preserving the relationship.

The point of collecting this huge, rather complex and highly interlinked data would be too allow us to get a sense of what we collectively know. It would also allow us to navigate through the relationships, as they are all deeply interconnected, and enhance or confirm our individual perspectives. We could use this knowledge to better understand reality, to make better decisions.

Of course this data would be massive. Eventually it would encompass everything we know, think we know or even guessed at. It would have every major and minor idea from our history. Somewhere it would list the world as being a sphere, but would also list out the flat earth concept (with the obvious metric that almost nobody believes it anymore). It would list out every political theory and every known historical interpretation. It would list everything, without bias and without fear of it being misinterpreted. Just the facts we think we know, drawn from billions of opinions, spanning the globe.

It would take a lot of money to create such a thing, and it would take considerable experimentation and research to find a suitably expressive knowledge representation, but if even a portion of it ever got created it would open up our ability to really see the world around us in all of its beautiful interconnectivity and complexity. It would enable us to become more realistic about what we collectively know, and how we often make decisions based on limited context. That, and it would be quite enjoyable to build...

Thinking Problems

What’s most interesting about reading lots of code is that you get a fairly deep insight into how people think. Building software solutions involves decomposing a bunch of related problems into discrete pieces and then recomposing them back into a solution that allows users to get their work completed. Examining these pieces shows both how the individuals involved break apart the world, and how much they understand of it to be able to put it all back together again.

Supplemented with an understanding of the history of the development and enough experience to be able to correctly visualize a better solution, this can allow us to analyze various types of common thinking problems.

The first and most obvious observation is whether or not the programmers are organized. That just flows right from how much structure and consistency is visible. Since most large bodies of code are written over years, there can sometimes be local organization, but that might change radically in different parts.

Organization is both external and internal. It applies both to the overall process of development and to how the analysts, designers, and programmers arrived at the code.

Internal organization, as it applies to thinking, is in a sense how fast one can head in the right direction, rather than just wander aimlessly in circles. Often times checking the repo sheds light, in that disorganization usually generates a large number of small fixes that come in sporadically. A good testing stage should hide this from the users, but in broken environments that doesn’t happen. So really messy code, and a boatload of little fixes, generally sets the base thinking quality. If the fixes are spread across fairly large time spans -- implying that they are driven by users, not testers -- then a significant amount of the thinking was rather disorganized.

Disorganized thinking is generally fairly shallow, mostly driven by the constant lack of progress. That is, people who make little progress in their thinking tend to want to avoid thinking about things because it doesn’t ultimately help them. That pushes them to the tendency to react to the world in the short-term. But long-term planning and goal achievement, almost by definition, require deep thinking about both the future and the means to get there. Thus disorganized thinkers are more often a-types that prefer the bull-in-the-china-shop approach to handling problems. For the most part there seems to be a real correlation between the brute force approach to coding and disorganized thinking. It’s the “work harder, not smarter” approach to getting things done.

Another really common thinking problem that is visible is with local simplification. It’s not uncommon to see programmers go from A to B via all sorts of other crazy paths. So instead of going straight there, they go A -> M -> Q -> B, or something equally convoluted. Generally, if you get a chance to inquire, many of them will rather confidently state that since A -> M is shorter than A -> B, that they have actually simplified the work and that they are correctly following an approach like KISS. What they seem to have difficulty understanding is that A -> M -> Q -> B is inordinately more complex. That the comparison between the two approaches is flawed because they aren’t comparing two equal things, rather just a part of the solution vs. the whole solution. You see this quite frequently in discussions about architecture, tools, design, etc. and of course these types of complexity choices get burned directly into the code and are quite visible.

In a programmatic sense, any code that isn’t directly helping to get to the destination is a possible suspect for being an M. More specifically, if you ignore the code, but follow the data instead, you can see its path through the runtime as being a long series of data copies and translations. If the requirement of the system is to get from the persistence storage to a bunch of widgets and then back again, then unless intermediate destinations are helping to increase performance, they are really just unnecessary waypoints. Quite often they come from programmers not understanding how to exploit the underlying primitive mechanics effectively, or from them blindly following poor conventions. Either way, the code and the practices combine to consume excess work that would have been better spent elsewhere. In that way, local simplifications are often related to shallow thinking. Either the programmer came to the wrong conclusion about what really is the simpler approach, or they just choose not to think about it at all and decided to blindly follow something they read on the web. Either way, the code points back to a thinking problem.

The third, and most complex problem is with the decomposition of things into primitive parts. It’s easy enough to break a large problem down into its parts, but what isn’t often understood well is that the size and relative positioning of the parts actually matters. There is nearly an infinite number of ways to decompose a problem, but only a select few really make it easy to solve it for a given context. Ideally, the primitives fit together nicely; that is there are no gaps in between them. Also, they do not overlap each other, so that there is a real canonical decomposition. They need to be balanced as well so that if the decomposition is multi-level, each level only contains the appropriate primitives. This whole approach is mathematical in nature since it adheres to a rigid set of relative rules, created specifically for the problem at hand.

A problem broken up in this manner is easily reassembled for a solution, but that’s generally not what we see in practice. Instead, people tear off little bits at a time, rather randomly, then they belt them out as special cases. The erraticness of the decomposition provides great obstacles towards cleanly building a solution and as it progresses over time, it introduces significant spaghetti back into the design. It’s the nature of decomposition that we have to start from the top, but to get it reasonable we also have to keep iterating up and down until the pieces fit properly; it's that second half that people have trouble with because that type of refinement process is often mistaken as not providing enough value. However, it really is one of those a-stitch-in-time-saves-nine problems, where initial sloppiness generates significantly more downstream work.

A more formal background makes it easier to decompose into solid primitives, but given the size and complexity of most problems we are tackling, knowledge or even intrinsic abilities does not alleviate the grunt work of iterating through and refactoring them to fit appropriately. This then shows up as two different thinking problems, the first being able to notice that the pieces don’t fit properly, while the second is being able to fix that specific mismatch. Both problems come from the way people think about the underlying system, and how they visualize what it should ultimately look like.

Now so far, I have been using the term ‘thinking’ rather loosely, to really refer to what is in between observations and conclusions. That is, people see enough of the problem, and after some thinking, they produce a solution. That skips over issues like cognitive bias, where they deliberately ignore what they see that does not already support their conclusions. Given that it is internal, I do consider that part of the thinking process; the necessity to filter out some observations that don’t fit the current context and focus. Rose-colored glasses, however, are a very common problem with software development. Sometimes it related to the time pressures, sometimes it's just unreasonable optimism, but sometimes it is an inability to process the relevant aspects of the world around us. It is a thinking failure that prevents us from being able to build up internal models that are sophisticated enough to match observed evidence. To some degree, everyone is guilty of wearing blinders, of filtering out things that we should not have, but in some cases, it is considerably more extreme than that. It’s really a deep-seated need to live within an overly simplified delusion, rather than reality. Strangely that can be as much of a plus as a minus in many cases, particularly if success is initially difficult. Ignoring the obvious roadblocks allows people to make a full-hearted attempt. But as much as it can launch unexpected successes (occasionally), it becomes increasingly negative as time progresses and the need for more refined thinking grows more significant.

Analysis, design, and programming are all heavily dependent on observation and thought. In the end, a software system is just an instantiation of various people’s understanding of a solution to a problem. In that, it is really only as strong as the sum of its commonly used weakest links (some parts of the system are just irrelevant). So, if the quantity of the observations is compromised by time, and this is combined with poor thinking, the resulting system most often falls far short of actually solving the problem. It is not uncommon to see systems that were meant to cut down on resources need significantly more resources than they save. That is, a grunt job for 4 people is so badly automated that 6 people have to be involved to keep it running. In that sense, the system is an outright failure and underlying that is not a technical problem, but rather one about how various people came to think about the world, and how they combined their efforts to solve things relative to their conclusions.

Thinking well is a prerequisite for handling complex problems, with or without technology. If people can’t clearly understand the issues, then any attempt to solve them is as equally likely to just make them worse. If they break down the problem into a convoluted mess of ill-fitting parts, then any solutions comprised of these are unlikely to work effectively and is more likely to make things worse. Software, as it is applied to sophisticated problems, is inordinately complex and as such cannot be constructed without first arriving at a strong understanding of the underlying context. This comes from observations, but it doesn’t come together until enough thinking has been applied to it to get it into a usable state. Software in this sense is just well-organized intelligence that gets processed by a computer. There is no easy way around this problem.