I haven't posted anything for a while now. This has been the longest gap I've had since I started blogging five years ago. It's not that I don't have any anything to say -- my hard drive is littered with half-finished posts -- but rather that I just haven't had the time or energy to get my thoughts down nicely.
Lately I've been wrapped up in a large complicated system that has a wide array of problems, none of which are new to me but it's unusal to see them all together in the same project. I like this because it is extremely challenging, but I definitely prefer projects to be well-organized and focus on engineering problems, not organizational ones.
My sense over the last couple of decades is that software has shifted from being an intense search for well-refined answers to just a high-stress fight against allowing all of the earlier shortcuts to swamp the project. I find that unfortunate because the part of the job that I've always loved was that satisfaction from building something good. Just whipping together a barely functoning mess, I find depressing.
What I've noticed over the years is that there is a whole lot more progress made when the code is well-thought out and clean at all levels (from syntax to the interface). The increasingly rare elegant solution takes it one step higher. You get so much more out of the work, since there are so many more places to leverage it. But of course spending the time to get it right means coming out of the gate that much slower, and it seems that people are increasingly impaitent these days. They just want a quick bandaid whether or not that will make the probem worse.
Getting a mess back under control means having to say 'no' often. It's not a popular word and saying it comes with a lot of angst. It does not help that there are so many silver bullet approaches floating about out there. Anyone with little software knowledge is easily fooled by the over abundance of snake charmers in our industry. It's easy to promise that the work will be fast and simple, but experience teaches us that the only way to get it done is hard work and a lot of deep thinking. Writing code isn't that hard, you can teach it to high school students rapidly, but writing industrial strength code is a completely different problem.
I'd love to take off some more time and write another book that just lists out the non-controversial best practices that we've learned over the last few decades -- a software 101 primer -- but given that my last effort sold a massive 56 copies and I'm a wage slave it's not very likely that I'll get a chance to do this anytime soon. The trick I think is to shy away from the pop philosophies and stick to what we know actually works. Software development is easy to talk about, easy to thoerize about, but what often really works in practice is counter-intuitive for people with little experience. That's not unusal for complex systems and a large development projects contains millions of moving parts (people, code, technology, requirements, data, etc) with odd shifting dependencies. You can't understand how to organize such a volitile set of relationships without first devling deep into actual experience and even then it's hard to structure the understanding and communicate it.
What flows around the production of the code is always more complex than the code itself and highly influenced by its environment. A good process helps the work progress as rapidly as possible with high quality results, most modern methodologies don't do that. That's one of our industries most embarrassing secrets and it seems to be only getting worse.
Hopefully one of these days I'll catch my breath and get some of my half-finished posts completed. There are some good lessons learned buried in those posts, it just takes time and patience to convert them into some shareable.
