Know your Limits

I’ve taken this week off from my day job to work around the house. It’s a yearly ritual -- my house is over a 100 years old and in constant need of repairs. This time my main focus is the front deck. It’s 16 feet in length and the top is about 8 feet deep. There are four deep steps running the whole length of the deck. It was probably built at least 20 years ago, but I suspect that it might be way older, perhaps as much as fifty.

Four years ago I rebuilt the inside of the deck because it was starting to rot in places. I put back the original covering boards since most of them were in reasonable shape. This year we decided that we wanted to get it painted, but many of the original boards we’re starting to go at their ends, so I figured it was time to recover much of the deck and the stairs with new boards.

Sometimes, when I am working on a project like this, people look at me strangely. I’m a bit geeky looking and their initial reaction is to get concerned when they see me playing with power tools. I guess they expect me to end up injuring myself, but I’ve actually been doing this sort of work since I was young and have plenty of experience with big tools of all shapes and sizes. So far I haven’t managed to nail my feet to floor. I guess people think that software guys are supposed to stick to virtual bits. But for me “building is building” it is only the medium that has changed; the work remains the same.

Of course natural materials like wood take a bit of getting used to. Wood warps, bends, and comes in weird sizes (a 2x4 is actually 1.5x3.5). Sometimes it does what you want, other times it takes a bit of ingenuity to get it to fit properly. In that way, it resembles software when you are building on top of quirky libraries. Sort of standardish, but not enough to make the work easy and plenty of unexpected surprises.

So far my project has gone well. As usual I made up a plan for what I was going to do and ordered a huge amount of wood to get shipped to the house. Once I opened up the stairs I found a bunch of problems. The bottom stair for instance, essentially had its cross-beams completely rotted out. A number of other steps needed new, but rather awkwardly sized little support beams.

Pretty typical for this type of project. An initial plan is great, but you have to keep re-adjusting it as you find more issues along the way. The issues are never quite predictable and you learn to not get easily frustrated. Sometimes it’s smoother then you expect, but rarely.

The stairs were a fairly simple job, but because of a few technical setbacks -- my power cord died, although at first I though it was the mitre saw -- it took longer than expected. When I finally got to do the main part of the deck -- it’s essentially a separate unit from the stairs -- I found that my earlier reconstruction efforts hadn’t worked as well as I had hoped.

The main deck was sagging horribly in the middle, which turned out to be caused by my fixing the cross-beams that had rotted. I cut off the bad parts, then extended the remaining pieces. Those extensions weren’t the same height as the original cross-beams (which are 8 inches) and I foolishly didn’t brace them on anything. With time they started to shift downwards.

Not a huge problem, but it meant taking them all off again (there are twelve), then cutting some new pieces to prop them up underneath, and then finally putting each one back in place. It took around three hours, but at least this time it was braced properly.

During my enthusiasm to fix these initial deficiencies, I knew that the new height would no longer be sagging, and that it was now somewhat higher than the original height. Way nicer and a lot more solid, but it left me with a new problem. The very last board on the top of the deck, which is also the first board as one steps up from the stairs, sits on a pair of 6x2s. The inside 6x2 helps to hold up the cross-beams, while the outside one is mostly decorative (and had to be replaced due to carpenter ants, another minor setback). With the cross-beams now raised and fixed, these two boards were sitting nearly an inch below the main deck height. A gap large enough that I couldn’t leave it that way, but awkward enough that I would never be able to find a pre-cut board that would fill it properly.

I’ve got lots of tools, but nothing that could cut such a small piece for that long distance. I basically needed a 1x4x16. Sixteen feet is a hard length to keep straight with a hand-held saw.

As I was sitting there pondering my problem, my friend Tom walked by. He’s constantly building or fixing things and he’s dropped in a few times to take a look at how the work is going. It’s a hugely valuable contribution, since I’ve found that although I have experience with this type of work, there are plenty of situations were I hit a novel issue that I’ve never encountered before. To be able to stop for a bit, chat about what I’ve done and what I’m about to do really helps in both clarifying the issues and getting someone else’s perspective. And given that Tom has way more experience than I do, he often has really helpful suggestions or points out things that I have missed.

In this instance I was really happy to see Tom. His timing was perfect. He took a quick look at the problem and told me that he could cut the boards that I needed. He disappeared for a bit, and came back with a specialized circular saw. The cutting was a bit tricky because of the length (and because he was holding the boards in the air with only one end touching the ground), but he cut six strips of nearly straight 1x1.5. Layered side-by-side, across the length, they fit in perfectly and my problem was resolved. He popped off for somewhere else -- it was nearly dinner time -- and I continued to finish up the top of the deck, thankful that I had been saved from a rather tricky and no doubt, time consuming problem.

I can relate this back to software development because it is often when we are building complex systems that we hit upon a thorny problem. Over the years, I’ve seen programmer after programmer hide away in their cubicles attempting to solves these issues in isolation. Programming is about solving problems, but there are always times when the problem is beyond any one person’s ability or experience to fix it properly. Sometimes it is best to go out and seek advice, help or at least a sounding board. Just trying to explain a solution to someone else often shows that it is not particularly viable or just too fragile. Sometimes there is an overlooked trivial solution. However, some programmers go out of their way to avoid seeking advice. I think it’s a culture problem with programmers at the deepest levels. Perhaps the fear of sounding stupid because of a lack of solution. Or possibly just a need to horde the more interesting bits. It’s different for different people, but all too frequent.

From the outside however it’s usually appears as a case of someone taking on work that is too big for them to chew. They go beyond their limits and as a result their solution causes serious side-effects. A badly designed or poorly architected piece of software won’t ever fix itself and will always get worse over time. Stopping the problem early saves a huge amount of wasted effort. Admitting that the problem is beyond their limits may be difficult for a lot of programmers, but it is far better than the alternatives. Seeking help is a good thing.

Had I stubbornly refused to get help with my deck project, I might have labored for hours to find a solution. My initial instinct was to cut a huge number of 2x4 shims with my mitre saw. But as Tom pointed out that would have looked incredibly ugly. Although I have the skills do to most of the work necessary, this one sticky problem was just beyond my abilities. I neither had the equipment nor the experience to be able to cut the required piece (at the time I was seriously wishing that I had bought a huge bandsaw for the basement, it would have worked fine but I’m sure that my wife wouldn’t be too impressed with such a large machine sitting around idle most of the time).

For me, bouncing ideas off other people or letting someone else do a piece that they are better at is normal at work. When I am doing complex modeling or schemas for relational databases I usually spend some time discussing the design with Gavin, he’s got a lot more experience in this area then I do. Igor is a wiz at algorithmic stuff and in fiddling with complex technical bits. It’s always worth checking in with him first (he also is great at finding excellent parts for custom PCs; both of my home machines came from his recommendations). I tend to specialize in architectural or product related issues, which seems to be my latest focus (I used to prefer performance optimizations, but I guess I am getting older now...).  I often consult with other programmers both internally and on the web whenever I need deeper advice. After twenty years, while I know lots of stuff, it’s just a drop in a rapidly increasing bucket. Getting things right the first time is far better than just guessing, and you’re just guessing if you don’t already have the knowledge.

A while ago I saw a blog post on how everyone should be doing code reviews. Formalizing that step is a good idea in places where the programmers are not communicating with each other, but certainly at my current work place it’s not necessary right now. As we are often bouncing ideas off each other and we do look at each other code (and point out problems), most of the code written by our rather small group was been reviewed at some point or another. That’s just a nice side-effect of our interactions. That, and we are able to gain indirect experience from each other’s efforts. If that culture doesn’t already exist naturally, then processes like code reviews are a great idea. Programming has gradually become a collaborative occupation where communication plays an important role in achieving success. The projects are too large and there are too many little details floating about for individuals to track properly.  

Building software is always a complex endeavor and the external pressures such as time or management don’t make it any easier. You can learn by experience, but that often means doing the same work a few times before getting it right. That’s not a particularly efficient process. Software is complex enough that there are always a huge amount of things people don’t understand. Knowing where your strengths and experience really lie makes it easier to seek outside advice when the problems get beyond your abilities. Hiding in a cubicle, unwilling to talk to anyone, and struggling alone generally doesn’t turn out well (and is never the best solution).

Styles, Standards and Conventions

Does it really matter how the code is formatted? Does it matter how you name the variables? Does it make a difference which variations of syntax you utilize for common tasks like loops?

For the most part any particular style, standard or convention is arbitrary. Sure, there are some that reduce or enhance the readability -- look at the obfuscated C contest for excellent reduction examples -- but overlooking the small trade-offs they all pretty much do the same thing. They set up some consistency in an otherwise unrestricted language environment. This doesn’t really matter to the computer, it has no sense of readability and it functions nearly identically no matter how the code is typed in.

So should programmers be free to create their code in any personalized or distinctive manner? Nope. We can not undervalue the utility in being able to easily read code, or in being able to spot obvious bugs because of how they violate consistency or naming conventions. A single file of code that is badly formatted, poorly named or bounces between different styles can easily hide bugs that would be obvious if the time and care were taken to add consistent formatting and naming. No one likes to waste time looking for problems that should be obvious, so that little bit of extra time and effort taken to make it clean easily pays off both in the short run, and over time.

Should a team of programmers all adopt the same style? Absolutely. Bugs are an indisputable fact of programing. Most systems are large enough, and complex enough, to require multiple people all working together. If one section of code is so eclectic that it prevents others from accessing it, it may work at the execution level, but it fails to be part of the system. And one is often blind to their own obvious coding flaws, so encouraging a second or third pair of eyes to help spot inconsistencies easily saves valuable time.

Job security is an often joked about excuse for excessive variations in styling, but it is far less stressful during vacations to know that any immediate problems can be solved by other people. It isn’t long before people notice that someone’s lack of presence means no progress, which generally builds up resentment. Pissing people off isn’t a very intelligent way of providing security (and never works in the long run).

An artistic desire for creative expression is another common motivation for group inconsistencies. That speaks to the idea that programming isn’t engineering and that the byproduct is a form of art. That may be true for some funky demo, but for serious projects one needs well thought-out industrial strength code. Failure to get this always means that the increasing acts of patching problems that were avoidable always grows large enough to consume any new efforts to extend the system. A mess begets a mess, and eventually eats up all of the time to do anything else. Thus ‘the art’ blocks out the ability to continue to build things. Creativity is nice, but getting something actually built is far better.

Styles, standards and conventions are a project/code-base specific issue. For each code-base (how ever large you want to define it) all of the programmers on the teams that are contributing to it, should follow the same rules as closely as possible. No excuses, no exceptions.

Prototype or System

It is more than just intent. It’s really the application of knowledge in order to achieve expected results. That is, it is one thing to just play around with building something in the hopes that it might work someday. It is another to have gained the knowledge and experience necessary to build it with the certainty that it will do exactly what it is supposed to do.

Wikipedia’s engineering page references the ECPD which states that engineering is:

[T]he creative application of scientific principles to design or develop structures, machines, apparatus, or manufacturing processes, or works utilizing them singly or in combination; or to construct or operate the same with full cognizance of their design; or to forecast their behavior under specific operating conditions; all as respects an intended function, economics of operation and safety to life and property.

Effectively, if you are just “guessing” when you build something, you’re not applying any engineering. If you can accurately predicate how it will operate because you understand it, then it is engineering. It’s also worth noting that in the Wikipedia page, if you are applying knowledge and science, you can be considered an ‘engineer’ even if that work isn’t certified (I always thought that the designation meant both).

Sometimes when developing software we read a bit of the documentation, then we just start hacking together a bunch of code. If the code is relatively simple and what we are doing is straightforward there is a very good chance the code will work as expected. We don’t need any depth -- understanding of what happens below each line of code or in other parts of the system -- we can just work at a higher level and hope that there are no unintentional side-effects, or that the lower parts won’t change significantly over time.

That works for smaller systems, but it tends to accumulate technical debt without us being aware. It’s not what we know; it’s what we don’t know that is slowly and dangerously building up. Thus the main behavior of the system works, but a great deal of what could happen remains unknown. Generally I like to call this type of code a prototype. It’s the basic functionality, but in quickly hacking it together we skipped over the depth required to insure that it will always function correctly.

When I work like this, I consider it to be hacking. It can be fast and fun, but there are too many uncertainties to feel comfortable having people dependent on this type of construction.

For larger systems that have complex architectures, relying on ‘guessing’ is usually a fatal error. A small problem in a little prototype can become a massive headache in a big system. It can require months or years of cleanup to rectify it. Thus, hacking is no longer a reasonable way to work. Instead we need to get more depth in our understanding. We don’t need to understand it all the way down to the chips, but we do need to understand it down to any levels that were not well encapsulated. That is, if someone else solved the problem correctly, we don’t need to rethink it, but as often is the case in software, if they only partially solved the problem we need to understand the consequences.

To me, when I sit down to build something with enough understanding of the technology to know precisely what it can and will do, that is ‘programming’. There is no guessing, and there is no ambiguity. A programmer knows the steps needed to be taken-- both for the results but also for handling any errors -- and applies that knowledge to produce something that they are certain will work.

The key difference here is the knowledge and certainty. If you know how to write the code, you are programming. If you are just taking wild guesses then you are hacking. You may get lucky hacking, and for prototypes you may not have an alternative, but you are essentially leaving the end result to chance.

In that sense, it is very clear that programming can and should become a form of engineering. There are many programmers out there that have a huge depth of knowledge that allows them to build exactly what they want, in a rather predictable time frame. It is also clear that someone that has only lightly-learned a programming language, and can do a few simple things with it, is mostly hacking. Hacking is no doubt a lot more fun, but it is akin to learning some magic tricks without any real understand of how they work. They may be successful most of the time, but Murphy’s Law insures that when you need them most, they will fail.

Generally we can see this differentiation in action. In the more modern languages that purport to take care of the memory management -- because people no longer understand what is happening underneath -- the run-times have become massively bloated. Moore’s law saves us from this being a disaster, but it goes to show how many people are really hacking at their work. Another good example is the proliferation of threading bugs. A language like Java makes it easy to create threads and protects against many types of crashes, but does nothing to stop spaghetti logic from disrupting the application’s behavior. Again people hack at threads without a deep enough understand of how they work. To work with them in predictable ways, you have to understand how they work and what you can do with them.

To sum it up, there are always times when we have to resort to hacking to figure out the underlying behavior or limits, but there is no substitute for knowing how it really works. Hacking together a prototype is often necessary, but when it comes to building a real system, a huge amount of knowledge is mandatory.