Driving Design

It was late. I was sitting in front of the keyboard, excitement flowing through my veins like the swarm of electrons that was feeding my computer's circuit board. The statistics for my site were excellent, I think. Both the visitors and the subscriptions were at an all time high.

The fan on the computer was quietly humming in its case, drowning out the stillness of the night. The cursor silently blinking, begging me to get to work.

The problem with technology is that you can find out too much information too quickly. Without a proper context you may know something, but do you really understand it?

Inspired by the statistics from my site, I figured I needed to do it again. Another post, but this time I'll really push the envelope; perhaps I can lead with a scene that has lots and lots of color. Way back, I should have remembered being disgusted: it was while reading some technical book where the author went on and on about his dog. Apparently I learned little from that experience, as I quickly worked my pet into the subtext.

I set the scene -- walking the dog late at night -- but regrettably I think I failed in tying it back to the topic. I'm new at this, so you need to forgive my enthusiasm.

I got the usual sense of relief when I finished posting the entry, but that was soon replaced by a growing sense of dread as I kept checking my statistics each day. Down they went. Spiraling counterclockwise like water in a newly flushed toilet bowl. Even if I was in Australia, where rumor has it that the rotation is reversed, I wouldn't be saved from the momentum of my writing. The post was tanking, and it was tanking fast.

Ok, so maybe dogs are a really bad idea in a technological post? Sure, that's it. Must be. It is some type of doggie kiss of death. A big slobbery one. Mention your pooch and watch your stats plummet. Tossing in clever references like "toilet bowl" or "hairball" can't be great either, just more ways to drive people away. You'd think I'd learn this at some point.

So now that I've killed off my readership, I might as well get back on the methodology thread. I feel it is important, but it is one of those things that people really don't want to read about. Ironically, I think it would make their lives easier if only they worked through the issues. Most people however, can't see past the poodle, so to speak.

When last I stumbled over it, I was pushing the idea that the elements of the design are actually the responsibility of the methodology itself. As well, the flaws in the design flow directly into the implementation, making it paramount to not only get the design defined correctly, but also tightly. With all of that in mind, we need -- as part of the process -- to find the least destructive way of gathering together the details of the design and compressing them into a usable blueprint.

A design is composed of: constraints, the key functionality, and the bulk of the matter. These three elements form the essence, beyond that there remains the issue of how the design interacts with the world around it.

Constraints are those rules that must absolutely be in place or the project is considered a failure. Many projects truly have none that are absolute, but for some designs there are just things cannot afford to be wrong. These can be technical, such as meeting a bare minimum of performance, or behavioral, such as guaranteeing that the results meet a level of conformance. If there are constraints and they are not meet, then the project is a total failure. If that's not a true statement, then they're not really constraints and shouldn't be treated as such.

All systems are based around some key functionality; driven directly by the problem domain and addressed by the tool. That needs to be fleshed out, as the tool is generally useless without getting this right. This is the meat of the system. Mostly, this functionality is built on some set of complex algorithms, generally underlying the mechanics. Sometimes it a batch thing. Whatever it is, understanding these functions is understanding the system. You need to spend time designing here, because to not do so would result in the system not working.

The remainder of the system is everything that needs to exist to round out the product, but is not vital in terms of being specified. For instance, in any application with users, there must be administrative facilities to change the user data. De-constructing that example, the secondary functionality of most systems revolves around viewing and editing all of the "other" data in the system that is not necessarily core. Now, we could list this out in terms of its functionality -- most commonly done as requirements -- but truthfully that is a long redundant boring list of: view "this", and edit "that" descriptions. The "this" and "that" are data, and it is for handling this data that we require a design. This leads to our understanding that all we need do is list the data and a few rules about its visibility within the system. Structurally, if I said that the system contains users and groups and a bunch of screens to update and modify them -- accessible only by the administrator -- you know exactly what I mean. There are a few variations on implementing groups, but they are all similar. If it really mattered because it effects the core functionality, then it should be referenced in the core.

What exactly do the screens look like? If they are in a GUI-based interface, they must fit within the existing visual layout. This makes them well-defined if they follow the conventions of the rest of the screens.

The system lets you manipulate the user and group information; simple functionality that is bound by the rest of the system. There is very little flexibility in the design. It needs to match to be correct.

Not exactly rocket science, but still people feel the need to repeat all of the tiniest of details. Often in doing so, they create inconsistencies that then are transfered into the code.

Specifying it once is best, and least likely to be inconsistent.

If you can get the constraints, the main functionality and all of the data fleshed out, what remains is how the system interacts with the rest of the world. There are many contact points with people or systems, ranging from simple configuration files, application programming libraries, installation interfaces, up to command line interfaces and large graphical user interfaces. Wherever anyone -- other than the original programmers -- interacts with the code, data or configuration, that is an interface and that needs to be defined.

We need to know the interfaces to build them, but people also need to know the interfaces to use them. In that, what is really important is the consistency. Even in something simple like a configuration file, consistently makes it easy to use and leaves a good feeling about interacting with the system.

Inconsistencies piss people off, kinda like spurious dog references in technical writing. Not a good idea.

The specs should exist both for the developers and the users. If we can draft them only once we can leverage our effort. Not only does this save work, but it helps to ensure that the interfaces as documented are the interfaces as built. A labor saving way to get the interface right, implement them and make sure they are accessible.

The key elements and the interfaces provide the full specification for a system, but they are still subject to the basic problem of people over complicating the design. The great programmers simplify their designs naturally, everyone else must work hard at it manually. Applying brute force in the design stage leads to bloated code, which indirectly implies that the opposite should be true. If we work hard to compact the design, particularly when we have something that is small and abstract, that can be manipulated in our heads, we have a chance of finding those inspirations that would lead to a simpler, more elegant approach. And we find these improvements at a time when it is still possible to implement them into the code.

The standard approach to software design says to explicitly iterate each and every requirement, breaking down all compound ones. Recovering from that ill fated approach is what many programmers accomplish during their implementation if they are successful.

By compacting the pieces in the design there is a huge improvement in reducing both the size of the implementation and the risk of failure. Not only are we doing less work, but the work we are doing is more accurate. A small tight design leads to a better implementation. Practical experience has always shown me this, but for some reason it does not seem to be intuitive.

So much time has been wasted in needlessly expanding all of the minuscule details for designs. Effort that I know is misguided.

At the end of the day, a clear well-formed, reasonably detailed specification that is only five pages is far better than sixty pages of redundant, repetitive, over-stated requirements and screen shots. Even if the sixty pages is consistent and perfect from a design perspective, there is a still an increased chance that the programmers will not correctly get it into the implementation. It is more work to extract the data from a larger volume and more likely that is won't be done correctly.

For all of the extra work to be pedantic, all that is created is more work, more problems and more risk. Not something we really need on a big project. Personally I prefer to win.

So I figured out the statistics thing. I'm sure it is the dog's fault. It has to be. I can't think of any other reason, or at least one that I would really like to accept.

In thinking hard about this I've come to a simple conclusion. I need to buy a parakeet or possibly a ferret. See, if I color my posts with something other than a canine, that will attract readers, won't it? What is better than a good ferret story? People love that kind of thing. It can't fail. It has got to be an exotic pet thing, but if you don't think so you can always send me some feedback. It is actually subjective, so you can't be wrong. Fish, fowl or even something better...

Just don't try telling me to stick to dog stories, I've been there and I think I've actually managed to catch on that it doesn't work. It just takes me a while, that's all.