Micromanaging Software Development

Software development projects fail; often and badly. I’ve blogged a lot about the different causes so I won’t repeat most of that again, only to say if you include all of the various people involved in one way or another, at every level (including the end-users), the whole thing is known to be a very difficult exercise in ‘herding cats’. All of the problems stem from a lack of focus.

There are as many “theories” out there about how to prevent failure as there are ways to fail. They range all over the map, but a few of them rely on micromanagement at their core.

What is micromanagement? My first real introduction to it was from working in middle end restaurants as a kid. Most of them would be highly chaotic places but for their managers. The managers sit on the employees and make sure that they are all following the rules. They need to clock in when they start and clock out when they leave. They have to wash their hands, and the kitchen staff have to put on clean ‘whites’ and hairnets. Employees aren’t allowed to eat the food and they are always supposed to keep busy even if the restaurant is not. There are rules for preparing the food, rules for serving it and rules for handling the waste. As an employee your job is not to think, but to get your work done as quickly as possible, while obeying all of the rules.

When the manager is doing his or her job well, they’re running all over the place causing the restaurant to function like clockwork. A finely tuned machine for grinding out food and collecting revenues.

As a patron I’ve always appreciated a well-running restaurant with good food. As an employee I didn’t actually mind the discipline. At least you always knew where you stood and you didn’t have to think hard about stuff. After a while you just fell into the grove and did whatever the manager wanted you to do. Once work was over, it was over. You were done for the day.

What I  realized is that doing a job like ‘line cook’ in a restaurant is a semi-skilled labour position. It takes a bit to learn the ropes, but once you’ve acquired the skills they are pretty much constant. Even into my middle age, I can still grill a ‘mean’ steak (I’ve mastered medium rare and rare. mmmm). Micromanagement is a very good way to manage non-skilled and semi-skilled labour, since it really comes down to a manager just extending their capabilities by directing the physical work of others. And if the micromanager is found to be annoying (as sometimes they are), changes in moral really don’t significantly affect people’s ability to get the job done. It’s probably one of the best way of handling employees in this type of position.

For decades now I’ve seen many a person look enviously at how these types of organizations function and then try to apply this back to intellectual jobs like computer programming. As I’ve often said, by far the bulk of most programming is ‘jogging for the mind’ so it doesn’t always require the uber-deep levels of thought that something like research or mathematics needs. From the outside this seems to confuse people who mistake it for being closely aligned with semi-skill labour. Both require some thinking so bought ought to be the same they surmise. There is however a huge difference. When cooking for instance, my main output is the way I am handling the food as I prepare it. When programming, the byproduct of my work is typing on a keyboard or swinging a mouse around, but the main output of my work is actually in ‘understanding’ the solutions I am creating. An outsider can see that I am using the correct techniques to cook and if they have acquired the same skill set, they can help me correct flaws in how I am working. An outsider however cannot peer into my brain and see that I am correctly understanding the puzzles I am solving and worse, depending on what I am building, they may need decades to acquire the same skill set, just to be able to critique my working habits.

Even if programming is jogging, and it gets considerably easier the more you do it, there is still a huge amount of learning involved. We solve a vast array of problems, on a wide range of equipment, with a plethora of different technologies. Too many things for any single programmer to master them all, and they change fast and frequently. Stop coding for long enough and suddenly the landscape looks completely foreign. It isn’t really and a good knowledge of the many theories of computer science can really do help to keep abreast, but still the amount of knowledge required is staggering.

In principle this means that a micromanager in software is unlikely to have more expertise than his employees, and that he has no way of judging how well his employees are working through the problems (until it is too late).

I’ve seen the first aspect of this solved by creating half-managers/half-programmers. In some sense it has been very common, since well-run teams are usually lead by the seniors in the group. Some organizations have just formalized this relationship and built on it.

The second aspect of this however is more difficult to deal with. Nothing is worse than letting a bunch of programmers run wild for months or years, only to find out that they really didn’t understand what they were supposed to build or how they should build it. A great many projects have failed this way. The micromanagement solution is to reduce the work to very tiny time increment, say a few days. The manager issues the work, the programmer delivers it and then it becomes very obvious if the manager/programmer communication and the programmer’s skills are both at a functional level.

The downside to this is ownership. Once you take away a long-term commitment to building something from the coders, they’re just drones to belt out the pieces. A skilled micromanager is required to direct the process, but the inspiration and quality of work from the programmers is likely to be lackluster. The job has become akin to working in a restaurant and of course once the day is done, the programmers rush off to do better things with their time. This of course can be a functional way to develop software, but it is at the far end of the spectrum caused by trading off development risk for increased surveillance. Lacking satisfaction, the turnover for programmers is high and few of them will have the ability to become micromanagers effectively.  

There are no doubt circumstances where micromanagement works in software development, I’ve seen a couple of good examples. The work gets done, but from my personal experiences I’ve always seen that the overall quality suffers greatly for this. Attempts to formalize this relationship fail unless the significance of the manager’s role and exceptional level skill they require, is factored in correctly. Even then, with the manager basically over-extended and the staff turn-over often high, it’s no surprise that the organization is subject to significant volatility in both their output and quality. The risks have shifted from each individual programmer over to the relationship between the programmers and their handlers, but they are still there.

In general, although I find this type of organization structure interesting, I’d have to say that micromanagement is not a particularly effective way of getting over the herding cats problem. I’ve always felt that a well-educated and well-balanced team, with strong leadership is probably a more sustainable way to direct the work. A happy, engaged programmer is much more likely to catch significant issues before they’ve become too ingrained to fix. If they can transmit this upwards, while getting enough of a downwards context from management, they can be well positioned to deliver their individual work both quickly and correctly. Well that’s my “theory” anyways …