Micro vs. Macro

Economics is often cleverly broken up into at least two sub-disciplines: micro-economics vs. macro-economics. The first one is really the science of individual behavior, not to be confused (too much) with behavioral economics. It is what individuals are going to do relative to the economy.

The second sub-discipline is how the economy -- as a large entity -- acts, and moves. It is the more visible version of economics, but without its micro cousin, it would be far less robust.

Programming too can be broken up into a micro/macro decomposition. Micro-programming is algorithmic or connective. It focuses on getting some set of instructions to perform a specific set of operations on a computer.

Macro-programming is more architectural, and is primarily concerned with how the larger pieces interact, the technologies, the domain, the users, the interface, the “product”, etc. All of the larger issues circulating within a project.

Between the two, micro-programming is far easier. You simply focus your efforts on a finite set of instructions, and using writing and debugging skills get it to match your expectations. It is not uncommon for domain experts, with little computer science background, to be working on algorithmic sections of code. With a bit of training, most people can micro-program (although some people inherently make it more complex than necessary).

Macro-programming on the other hand is a world of conflicting trade-offs and easy mistakes. It involves understanding a lot more, particularly being able to correctly guess the overall direction of the project, the users and the market. It’s far harder, far more blurry and requires considerable experience to get it right.

A popular way to hire programmers right now is to give them ‘programming’ exams. Tricky questions that require prior micro-programming exposure, in order to answer them well. For junior or intermediate positions, where most of what they’ll do is micro-programming, this may be appropriate, but it is entirely the worst way to pick seniors.

What you don’t want at the top level of a software project is a group of people mostly focused on pounding out neat little algorithms. That’s what the kids are for. What you need is a group of people with real hardcore experience that can guide the entire project away from the many pitfalls that commonly sink coding projects. You get that if you look for attributes like experience, independence and leadership. Not coding ability. If your team is run by non-vocal people who think that every problem -- including the personal and organizational ones -- is solvable by just adding more code to the system, you can easily guess how this will quickly result in a mess.

Because there are so many seniors out there lacking the right kind of skills, you often see career managers, or domain experts stepping in to run the projects. But leaving a large engineering problem in the hands of someone without the prerequisite background is just courting disaster. If you don’t have enough micro-programming experience, and then even more macro-programming experience, how will you acquire the underlying knowledge to make good decisions? We’re completely unable, right now, to get any of this down into nice short textbooks, and it changes quickly depending on the project’s environment. Experience is the only source of understanding. Guessing at the standards or rigorously following “industry best practices” to the letter are both easy ways to find serious trouble.

Most people with little or no experience confuse micro-programming with macro-programming. They believe that the two are the same, and that experience with one is the same as the other. This colors their expectations and often their own self-confidence. It’s probably the main reason why so many think that code is malleable, and that years worth of work should be doable in a couple of months. It’s a nice delusion, but often a very costly one. We see it’s effect in our failure rate.

In an immature field, absolutely nothing beats experience. Without it, a project is dependent on sheer luck, and given the dismal state of our knowledge and technologies, it’s a massive amount of luck that is required. Software projects generally succeed when they are driven by experts who understand their volatile nature. They usually fail otherwise.