Quality and Scale

[LACK OF EDITOR'S NOTE: I wrote and posted this entirely on my iPad, so there are bound to be spelling and formatting problems. Please feel free to point them out, I'll fix them as I go.]

I use a couple of metrics to size development work. On one axes I always consider the underlying quality of the expected work. It's important because quality sits on at least in a logarithmic space. That is small improvements in quality get considerable more expensive as we try for better systems. On the other axes I use scale. Scale in also at least logrithmic in effort.
My four basic catigories for both are:

Software Quality
- Prototype
- Demo
- In-house
- Commercial

Project Scale
- Small
- Medium
- Large
- Huge

Software Quality

The first level of quality are prototypes. They are generally very specific and crude programs that show a proof of concept. There is almost no error handling, and no packaging. Sometimes these are built to test out new technologies or algorithms, sometimes they are just people playing with the coding environment. Proototypes are imortant for reducing risk, they allow experience to be gained before commiting to a serious development effort.

Demos usually bring a number of different things together into one package. They are not full solutions, rather they just focus on 'talking points'. Demos also lack reasonable error handlng and packaging, but they usually show the essence of how the software will be used. Occasionally you see someone release one as a commercial product, but this type of premature exposure comes with a strong likelihood of turning off potential users.

My suspicion is that in-house software accounts for most of modern software development. What really identifies a system as in-house is that it is quirky. The interface is a dumping ground for random disconnected functionality, the system looks ugly and it's hard to navigate around. Often, but not always, the internal code is as quirky as the external appearance. There is little architecture, plenty of duplication and usually some very strange solutions to very common problems. Often the systems have haphazaardly evolved into their present state. The systems get used but its more often because the audience is captive, given a choise they'd prefer a more usable product. Many enterprise commercial systems are really in-house quality. Sometime they started out higher, but have gradually degenerated to this level after years of people just dumping code into them. Pretty much an unfocused development project is constrained by this level. It takes considerable experience, talent, time and focus to lift the bar.

What really defines commercial quality is that the users couldn't imagine life without the software. Not only does it look good, it's stunningly reliable, simple and intuitive to use. Internally the code is also clean and well organized. It handles all errors correctly, is well packaged and requires minimal or no support. Graphic designs and UX experts have heavily contributed to give the solution both a clear narrative and a simple, but easily understood philosopy. A really great example really does solve all of the problems that it claims to. Even the smallest detail has been though-out with great care. The true mastery of programming comes from making a hard problem look simple; commercial systems require this both to maintain their quality and to support future extensions. Lots of programmers claim commercial quality programming abilities, but judging from our industry very few can actually operate at this level. As user's expectation for scale and shortened development times have skyrocketed, the overall quality of software has been declining. Bugs that in the past would have caused a revolt or lawsuit are now convientently overlooked. This may lead to more tools available, but it also means more time wasted and more confusion.

Project Scale

It is imossible to discuss project scale without resorting to construction analogies. I know programmers hate this, but without some tangilble basis, people easily focus on the wrong aspects or oversimplify their analysis. Grounding the discusion with links to physical reality really helps to visualise the work involved and the types of experience necessary to do it correctly.
A small project is akin to building a shed out back. It takes some skill, but it is easily learned and if there are the inevitable problems, they are relative well contained. A slightly wonky shack still works as a shack, it may not look pretty but hey, it's only a shack. Small projects generally come in around less than 20,000 lines of code. It's the type of work that can be completed in days, weeks or a few months by one or two programmers.

A medium project is essentially a house. There is a great deal of skill in building a house; it's not an easy job to complete. A well-built house is impressive, but somewhere in the middle of the scale it's hard for someone living in the house to really get a sense of the quality. If it works well enough, then it works. Medium projects vary somewhat, falling in around 50,000 lines of code and are generally less than 100,000. Medium projects usually require a team, or thet are spread across a large number of years.

You can't just pile a bunch of houses on top of each other to get an apartment building. Houses are made of smaller, lighter materials and are really scaled towards a family. Building an apartment building on the other hand, requires a whole new set of skills. Suddenly things like steel frames become necessary to get the size beyond a few floors. Plumbing and electricty are different. Elevators become important. The game changes, and the attention to detail changes as well. A small flaw in a house, might be a serious problem in an apartment building. As such more planning is required, there are fewer options and bigger groups need to be involved, often with specializations. For software, large generally starts somewhere after 100,000 lines but can also get triggered by difficult performance constraints or more than a trvial number of users. In a sense it's going from a single family dwelling into a system that can accomodate significatly larger groups. That leap upwards in complexity is dangerous. Crossing the line may not look like that much more work, but underneath the rules have changed. It's easy to miss, sending the whole project into what is basically a brick wall.

Skyscapers are marvels of modern engineering. They seem to go up quickly, so it's easy to miss their stunning degree of sophistication, but they are complex beasts. It's impressive how huge teams come together and managed to stay organized enough to achieve these monuments. These days, there are many similar examples within the software world. Sytems that run across tens of thousands of machines or cope with millions of users. There isn't that much in common between an apartment building and a skyscaper, although the lines may be somewhat blurred. It's another step in sophistication.

Besides visualization, I like these categories because it's easy to see that they are somewhat independent. That is, just because someone can build a house doesn't mean they can build a skyscaper. Each step upwards requires new sets of skills and more attention to the detail. Each step requires more organization and more manpower. It wouldn't make sense to hire an expert from one category and expect them to do a good job in another. An expert house-builder can't necessarily build a skyscraper and a skyscraper engineer may tragically over-engineer a simple shed. People can move of course, but only if they put aside their hubris and accept that they are entering a new area. This -- I've often seen -- is true as well for software. Each bump up in scale has its own challenges and its own skills.

Finally

A software project has an inherent scale and some type of quality goals. Combining these gives a very reliable way of sizing the work. Factors like the environment and experience of the developers also come to play, but scale and quality dominate. If you want a commercial grade skyscaper for instance, it is going to be hundreds of man-years of effort. It's too easy to dream big, but there are always physical constraints at work, and as Brookes pointed out so very long ago, there are no silver bullets.