With Cat-like Dignity and Grace...

And I thought I was doing so well. I had found my voice, knew what I wanted to say and was starting to express it an enlightening and hopefully entertaining way. Then why, you ask would I deliberately go out and make my readers cough up a hairball?

"Why not", I say. "Sometimes, you just have to talk about the things that people really don't want to talk about". Sometimes they should listen.

Building software is challenging, but many of our problems spring from the way we are going about it, not from the technology itself. At the very heart of all development is the dreaded m word. Muggle, you say after having read too much Harry Potter? No, the other one: methodology. It is the 'thing' that sets the steps of the process; laying it out for us.

Perhaps it is an infancy thing, but for software the range of methodologies used is unusually vast, ranging from virtually nothing, all of the way up to inches and inches of documented process. The size of the range speaks volumes about where we are in our understandings.

Nothing, is really bad, of course. A complete lack of any recognizable process, when you see it, is a mess. Like a steel ball in a giant pinball machine, flipping from crisis to crisis, the organization never follows the same path twice. The inconsistencies fuel the game. There may be occasional wins but there will be lots of crashes too. Failure is an option. And a common one too, as the ball frequently is missed by the flippers and goes sinking back into the drain. Chaos quickly takes its toll on the organization: low morale, high turnover, and volatility.

Contrast that with one of those long, slow, overwhelming processes. You know the type that act like a giant poisonous gas cloud, sucking the life out of everything they touch. Its presence leaving the coders fearful and huddling in their endless maze of cubicles. A full three letter acronym of dread and despair that is sure to choke off even the most adventurous. The code gets done. Eventually. But it is so damn ugly by the end that most people turn a blind eye towards it, trying instead to develop hobbies outside of work. One last redeeming attempt to get some source of satisfaction in their lives.

Both ends of the spectrum are horrible places to be stuck, but they don't take away from the underlying need to have a good process. Even more important: the process you use absolutely shapes your output. Yep. You may throw all the cute technologies you want into the pile, you may even gather the best and the brightest, but the process always taints the results.

It is that important. It is that influential. It is that elementary.

I've used many processes over the years, none of which I would recommend. They fail more often because of their negative side-effects, not their central focus. It is sad because the process should be there to help me. I need something I can depend upon.

Over my next few blog entries I want to continue this discussion, focusing in on the areas where I think changing the process is crucial. Were it could work. We need new ideas for design, implementation and testing. Not complicated ones, but they need to fix the problems. Our current ones don't live up to our expectations. We either avoid the hairballs, or we rehash the same ideas, dressed differently, but still clinging to the same old flaws.

My ultimate goal is to write another book. A simple reference to a simple methodology. One that fixes the problems, not contributes to them. There is enough knowledge available, it just needs to be consolidated, sifted and repackaged in a way that it is usable. For this task, feedback is important to me.

We probably already know the next great methodology that will transform our industry for the 21st century, we just don't know that we know it, yet.