Developer Management

It’s very difficult to bring a group of software developers together and get a reasonably well-built software system out of the process.

The problems stem from the underlying programming culture. Coders tend to be introverted, their thinking is often extremely black-and-white and they go rogue quite frequently. 

This leads to a number of unfortunate outcomes. 

First, they are not really good at communicating the problems and issues that they are having getting the system built and running. They don’t like to admit problems. Coupled with the modern impatience to get things done too quickly, this often reduces the timeframes to considerably less than is what is necessary to do a good job. If the stakeholders aren’t aware of many of the ongoing problems, they aren’t going to have realistic expectations on the progress of the work.

Coding requires a rather rigorous logical mode of thinking. Stuff either works or it doesn’t. Things are either right or they are wrong. The computer does exactly what the programmer tells it to do, there is no room for ambiguities. If you spend your day writing code with these properties, it tends to leak out into the rest of your thinking, life, and interactions with people. The world, however, is grey and murky, and often requires a soft touch to work through the issues. A big team of people working together generates a lot of different agendas and politics. None of this is nicely black and white, so there is a lot of friction between how the programmers think the world ‘should’ operate and how it actually does operate. This leads to a lot of misunderstandings, anxiety, and confused goals. 

With a different perspective on the priorities, and a desire to not want to talk about it, programmers are infamous for just making a decision on their own and then going off with full confidence to get those things done. The problem is that those attempts are often not in sync with the rest of the project, so they basically go rogue and end up doing something that doesn’t provide value. A sub-group of coders incorrectly heading towards the wrong objectives will conflict with the important ones, so the result is poor or useless work.

It’s hard for management to distinguish between a rogue programmer and one not doing any work at all. In both cases, what gets accomplished is nothing. Usually given that their expectations are off too, this builds up a lot of tension between the developers and management. 

In the past, people like to blame the “waterfall methodology” for going off in the wrong direction and returning with stuff that was not useful. They insisted that it was the methodology that was at fault, that it was a ‘time’ problem, but there is a little more to it than that. 

If it's a big, well-defined project that takes 1.5 years to accomplish, doing it in one long continuous unit of work is a whole lot more efficient than breaking it up into little iterations and trying to stitch them together somehow. Mostly batching together similar work is more effective, is better for consistency, and for focus. 

The big failures before the turn of the century drove the stakeholders to seek out better ways of tightly controlling their software projects. The culture of programming itself helped. Both sides settled on an invasive form of micromanagement. The coders seeded control of their work. So, the prerequisites for deciding on the right work like analysis and design get ignored, while the whole effort is gamified with a rather childish bent on formality. You don’t have “excessive” daily status meetings, you have ‘standups’ instead. There isn’t a long list of pending work, it’s a ‘burndown’ chart. We don’t break down the work into tiny, little, verifiable chunks, it's called a ‘sprint’. Planning is a game, not a necessity, and the stick is called a ‘retro’ which is somehow supposed to good for you. 

Each time management was compelled to reach for the thumbscrews and lockdown inappropriate behavior, consultants came up with cutesy little names and games for implementing it, and pushing it as propaganda to the rest of the industry. It’s unfortunate. 

For me though, the fundamental problem is not upper management controlling how the system is constructed. Rather, it is the role of leading a group of developers that is confused. It’s not a technical role and it's not a business role. 

Ultimately, there are some high-level goals that need to be accomplished. The people setting those goals do not have the ability to break them down and express them as a very, very long list of complicated programming tasks. That’s the communications impedance mismatch. If you hire a bunch of programmers and can’t tell them what to do, and they won’t tell you what problems they are having, then it is pretty obvious that the project is not going to function.

So, you need an intermediary. Someone who has spent a lot of time programming, but also someone who has been around the higher-level objectives enough to understand them. They have to have a foot in both worlds because they have to accurately translate between them. 

They might not be the best programmer, or be able to solve silly little fake coding issues. They just need to have spent time in projects of similar scale. They need to get their priorities straight. They might not fully understand all of the business objectives, or be a domain expert, but they need to have empathy for the users and to have some depth in their specific domain problems. They sit in the middle, and they ensure that the upper goals are progressing, while the lower work isn’t going rogue. 

Over the decades, I’ve heard many an entrepreneur reach the conclusion that all they need is a group of students that can code a little bit in order to get their product to market. That’s kind of the classic delusion. It sees coding as a commodity that just requires enough ‘energy’ to drive it forward. Oddly, that can work for a demo or a proof-of-concept or some other introductory software that just needs to kinda work in order to grab more interest, but it fails miserably once it becomes real, mostly because the necessary skills to keep it all organized and keep it growing are missing. So, it’s a start that can be used to evaluate ideas, but not a product that will work when needed. 

Moving up to that next level means getting serious about keeping the work under control. It’s not a small gap, but actually a rather huge one. It’s not intuitive and the kids that threw together the prototype code won’t be able to magically pull it from the ethos. This is where coding switches from being a game to getting serious. It can’t afford to be ineffective anymore, it can’t afford to be disorganized anymore. Everything changes.

For medium, large and massive projects, even the smallest issues have huge, wide-ranging consequences. You learn to deal with them via experience, and it is these issues that are far more important at this point, than the actual coding itself. Fixing the code is cheap, unrolling a ball of mud is expensive. An intermediary who knows it is important to restrict ongoing dependencies, for example, is a much better asset than a coder who can craft unique algorithms. The wrong algorithm is useless, while the wrong dependencies are often fatal. 

In an industry known for its agism, and for still having a high rate of failure, you’d think it would be obvious by now that we’d know there is a missing critical component in the effort. But oddly, the stakeholders still think programmers are just cogs, and the coders still think that if they just had “more code” their problems would magically disappear. The technologies have changed, the methodologies have gotten crazier, but the underlying problems are still the same. Breaking up a months’ worth of work up into hundreds of artificial 2-week tasks doesn’t ensure that it will go any better or be more appropriate. Instead, it tends to build up a counter-culture of gaming that process. Since it’s all indecipherable from above, it does nothing to ensure that progress is actually moving as best as possible. It just provides a false sense of momentum. And the games that coders play may distract them for a while, but the necessary satisfaction from doing a good job is missing, so they aren’t getting what they want either. 

Part of programming is really boring, routine, software production. It’s just work that needs to be done. Some small parts of getting a big product out to market are creative, but more often than not the creative portions fall into the business, design, and architectural efforts. If the ideas and issues are worked through in advance, and any difficult technological issues are prototyped up front, then the rest of getting out a new release is just the careful assembling of all of the pieces in an organized manner. It’s not a game, it’s not a contest. Having someone who knows what is really important during this phase of the work is going to prevent a lot of predictable quality issues from materializing. Like any other profession, programming isn’t “fun”, but when it is done well it can be quite satisfying.