The last thing I ever wanted to be -- twenty-six years ago -- was an engineer. When I started university I had heard that 70% of all engineers hate their jobs and that they were often bored at work. Boredom sounded horrible.
I wanted, was to master the black art of coding. To follow in the foot
steps of those early magicians I had seen in the computer magazines. The
ones that crafted weird cryptic concoctions to do really neat things to
their machines. I wanted to be a hacker.
funny how time and experience temper one’s thoughts; how your actions
change once you really understand the consequences; how much you grow
once you see the larger picture.
shops span the range between the cowboys, who are just hacking at the
code furiously, and the bureaucrats, that are so ground down in
organizational paperwork that little actually gets accomplished.
my younger days the cowboys where fun. It’s a dynamic environment.
Things are happening. And they are happening fast. Well, OK, there are a
lot of needless problems cropping up, but hey, that just adds to the
excitement, doesn’t it? You could leave work feeling like you got
that lasts for a while. Until you start to get caught in the same
hollow accomplishments, over and over again. That is, you’ve fixed the
same bug before, wrote the same code before or stuck a band aid on the
same gaping wound before. Doing it once or twice is fun. Doing it
repetitively, while giving up your life to endless overtime, starts to
become depressing after a while. Then suddenly one day you find yourself
looking down at the growing pile of band-aids before you and you start
thinking “isn’t this all just a waste of my time and my abilities?”
on I moved away from the cowboys. They were driving me nuts. I landed
deep in the heart of a reasonable engineering culture. Some people on
the outside might have confused that development process with a classic
waterfall one, but it wasn’t really. Although there was a significant
design process up front, the last of the details were always worked out
in the various iterations. It was actually a healthy mix. Some thinking,
some paperwork, then a hard run at the code. Some testing, a release
and then back to thinking again.
when I set up my own shop, it naturally started as chaos. Startups are
sensitive to cash-flow problems so sometimes the only way to deal with
that is an ugly short-term hack. But as time progressed -- every time I
got the chance -- I applied long-term fixes. These gradually paid off
and when I finally moved on, the place was humming along smoothly
releasing a steady stream of mostly high quality code and the occasional
patch within a day or two of finding a bug. Each development cycle was a
bit easier than its predecessor. Everything was tracked. We had managed
to built a big application. One that was far larger, and far more
sophisticated than you would have expected from such a tiny shop. I
learned a lot in those years.
found that it isn’t all that complicated. If you boil it down again and
again, software development keeps coming back to its roots in
engineering. Not in the classic sense that I was afraid of when I was
younger -- the large, slow, boring, mindless process -- but rather in
the sense that as the work progresses, developers like myself needed to
be hacking less, guessing less, and understanding more. The project may
start with a lot of unknowns, but at some point it always gets down to
routine engineering. That is, we should be doing the ‘right’ things that
work because we know that they work, and fixing the little problems --
permanently -- before they become big serious ones.
earlier fears were misplaced. Engineering itself is not boring. I’m not
even sure where I heard that statistic about unhappiness. Engineering,
particularly when it comes to something like software, is simply knowing
what works and applying it correctly so that the project is successful.
It isn’t hack and hope, it isn’t wild guesses, and it isn’t a spin job
to cover up the fact that the system is horribly broken. No, quite
simply, it is the path to dreaming up cool solutions and then
implementing them as cool solutions.
there are boring times. There is a lot of digging to see what is
already out there. No sense reinventing the wheel especially if it’s
already decades old. Note that that doesn’t mean you can’t recode the
wheel -- making a better one -- but rather that you don’t try to rethink
the wheel, that work is already done and accessible.
is sometimes paperwork. But if one focuses on documentation that’s
useful, rather than on the documentation a management consultant would
want, then it’s not so bad either. It helps to refine ideas and
highlight missing corner-cases long before they become too serious to
fix. A little design goes a long way and any organizational chaos, at
any level, is always going to grow into a serious problem someday. What
you’re missing, or not thinking about, often becomes really obvious when
you try to write it down. It also helps you to understand the
vocabulary of the problem and to empathize with your users (if you can’t
easily explain it in a document then the users aren’t going to be able
to understand it either).
is a lot of thinking. And ironically, although programmers preach
heavily about their need for freedom, many choose to turn off their
minds when they code, resorting to as much brute force as possible. But
at the end of the day, code is just the by-product of thinking about the
solution, so it is only as good as the thinking that went into it. To
build sophisticated things you have to have a deep understanding of the
problem and you have to spend a lot of time working through it.
Thoughtless code is dangerous code because of it lacks intelligence. It
just causes stupid problems.
of course, if you’ve really set a firm direction and you know where you
are going with the development then the rest of it is just hard work.
Little issues have to be sorted out, details need gathering, but the
main flow of the development can be worked out in advance. The project
may drift but either the long-term perspective still holds, or it is
rethought again to insure that it is still valid.
remember reading decades ago in Scientific American about how the
pharmaceutical industry moved from art, to science and then on to
engineering. The pills we pop these days are highly likely to contain
exactly what they say they will. That wasn’t always the case. It took
them a long time to go from alchemy to engineering. The article said
that software also started as an art, and has some scientific basis, but
it will eventually follow the same path to engineering. I remember the
article because I figured I’d bail when that happened. Sounded boring.
But these days, as I am surrounded by more software with increasing
complexity and decreasing quality I realize that what I want most right
now is to get a whole lot more of the engineering mindset into the
software development industry. It is time, I think, that we started
taking our profession seriously.
is still room out there for the hackers and the cowboys. Sometimes a
band-aid is the best choice, or the solution is so small it doesn’t
matter how much of a mess it is. For quick one-offs I think hacking
will always be common, but when I go out and rely on someone else’s
code, on their thinking and on their intelligence, I would like to know
that they weren’t just mindlessly flailing at the keyboard or that they
skipped their homework and just tossed it together. It makes a
difference. I don’t want to build on a house of cards. I don’t want to
find out later that I’m dependent on somebody who didn’t really ‘get it’
at the end of the day. I don’t want that degree of instability in what I
am doing. I can’t build great software on top of a mess; dealing with
other people’s mistakes eats through my resources like a knife cuts
these days I am all in favor of us seriously moving into engineering.
Not doing so is a major obstacle that has prevented software from living
up to its true potential. I think we can do it, and do it in a way that
it is not boring and it isn’t just paperwork. We can still build
quickly, but we can do so from a place of understanding, not one of
panic. It’s about time that software development grew up and became a
real profession, one that is taken seriously. Even in our deplorable
state we are changing the world, so can you imagine what we could
achieve if our collective works didn’t suck?