Altered States

I think I am judging our industry too harshly. My feelings towards software development have lately been ones of extreme disappointment. I keep thinking that we should have achieved some larger number of advances by now, particularly in the way we build systems. However, over the last couple of decades, while software has definitely not lived up to its potential, I do have to admit that there have been a few significant advances.

I've seen subareas of the software industry that are best described as being ten to twenty years behind the rest of us. The software looks and feels like it was written in the mid-nineties. It has that fugly aesthetic about it, the kind that makes it look like some old Soviet machine. All stiff and practical, without any consideration given to its actual appearance.

That I can judge a piece of software to be so outdated shows that we have made some type of progress. From a usage perspective, we have moved. Our gains, however, may have only been in the visual style of the interface, more interconnections between systems and little else.

We now have access to an unprecedented amount of data. But, we also have no real way of leveraging that. Early on we picked relational databases to be one of the cornerstones for implementations. That choice so often pushes us into architecting some strange and overly complicated solutions to get around their inherent limitations. There are mass amounts of data trapped in vertical silos with no easy way of liberating them. So, we can collect and store more data than ever before, but we cannot consolidate it, or easily mine it for higher knowledge.

We do have more functionality available to us. And with movements like OpenSource, we actually have a huge amount of available functionality, free and commercial. Unfortunately, large chunks of it are essentially unusable. There are so many badly designed software tools out there that are completely undependable. If we can't use the tool for practical use, then it's no good. Even many of our big famous tools that worked well in the past are rapidly degrading. The features that proved useful early on have bloated into untrustworthy, and unpredictable code. A clear sign of this instability problem is that we are doing less customization, automation, and scripting because the tools require more and more intelligence to work around their rapidly changing flaws. Our foundations -- which are getting bigger -- are so unstable that we are losing our ability to build on top of them.

We have more methodologies available to us, and we can choose between light-weight and heavyweight, but none of these appear to be an actual improvement in software development. They rehash old ideas or invest in unpractical new ones, but mostly they all choose to stay away from the really difficult issues. If we don't change the steps we use to develop software, then it seems a reasonable conclusion that the success of the projects and the quality of the systems will remain the same. That much has not changed in decades. If we play with fun ideas but don't deal with the real issues, then the problems will remain. There are more methodologies to choose from, but none of them seem designed to improve the results.

So, there does seem to be some progress, but it does not seem to have made significant changes in the software development industries. They remain, as always, producing things that are expensive and undependable. Software has a great deal of potential, but there is a long way to go before it will start to live up to it. Someday, it is easy to imagine that people will look back into these dark days and wonder how it was that we even managed to keep most of our systems running. A circumstance that becomes increasingly dangerous as we put more and more faith in our technologies.