Some Lessons from Experience

I started by thinking about what a good strong definition for a software architect would be. That pushed me into reflecting about how I've changed over the years. The things I've learned, the differences in viewpoint, the way I approach problems.

Every so often, it's a great exercise to reflect on our progress, it helps in learning from our experiences. I've been at this software development thing for quite a while, in so many different contexts, that I ought to be able to distill some of that into a few discrete pieces of wisdom. Or at least show how I've changed.

My focus over the last few months has been on deconstructing the various roles that I've had while doing software development. I've been breaking it down into programming, analysis, architecture, vision and management. I do this in the hopes that if we better understand the parts, we'll draw stronger conclusions. When it is all a big giant blur, we tend to ascribe the wrong effects to the wrong causes.

Within the different roles, I want to find a simple sentence to identify my focus. Of course, since the basis for this post is how things have changed for me, my experiences as a junior, intermediate and senior are the central focus.

I thought about setting times on the entries as well, such as the number of years it took me to become an intermediate. But I would guess it's very different for different people, so I don't want anyone to draw the wrong conclusions. It takes as long as it takes to get to the next level, it's not about time, it's about knowledge and experience.

Currently, on a related track, a programmer in our industry will be promoted to a senior position within only five years. But that's somewhat misleading. Programming is just a small piece of software development. It's not being a senior programmer that is important, it is being a senior software developer, but that is a much harder challenge.

I really don't think that anybody, after only five years, really has a grasp on the whole picture. They may know how to code, but the real problems come from being able to see the problems and their solutions, all the way out to deploying them. Vision, analysis, design, deployment and support are all much broader than just programming.

It's a huge amount of knowledge and understanding, which includes groking people and politics. You're certainly not a senior software developer until you've been doing it for at least a decade, and even after putting in the time, many people will never grow up enough to even reach the senior designation. Immature people make for horrible software developers.


ROLES, RESPONSIBILITIES AND FOCUS

For this next section, I'll include a role, and my definition of its responsibilities. Then for each level of experience, a simple sentence that best describes my focus at the time. As a disclaimer, I reserve the right to change my senior viewpoint overtime, I don't want to ever be stuck into a static definition for any role.


PROGRAMMER

Programming is the act of taking a design, often in requirements, and turning it into code for a system. As programmers get more experience, the scope of the code gets larger.

Junior: I was struggling with just getting the code to work.

Intermediate: I developed an obsession with the 'right' way and performance.

Senior: I can build it, if I can see it.


ANALYST

All solutions must tie back to making piles of data solve real world issues. Mapping the messy real world onto an abstract mathematical one is the job of an analyst. They need to understand the size, frequency and structure of all of the data, as well as the relative merits of any of the algorithms underlying the user's functionality.

Junior: I still thought that terminology, formulas and definitions matter.

Intermediate: I saw vague patterns in the data and the functionality, but not quite getting it.

Senior: I see everything (including process) in terms of the underlying structure of information, its size, quality and timeliness. I accept that some things are just irrational.


ARCHITECT

An architect lays out the design of the system to meet the business, technological and environmental constraints of a project. This involves weaving the analysis into the standards and conventions, while overlaying a high-level structure onto the data, the code and the programming teams.

Junior: I was doing bottom-up design, hopefully getting enough pieces for it to work correctly.

Intermediate: I started top-down, breaking the system into major components, then building up enough pieces.

Senior: I have a more holistic approach, drawing the various lines in the project to accommodate the entire process including analysis, development, deployment and support.


VISIONARY

There are a billion problems out there that can be solved with a computer. The person who finds one and suggests a well-rounded solution to meet that need is a visionary. This is not a skill, it's something more, like being able to see the future. It is extremely rare, and is always underestimated in value. Everyone thinks they are visionary, but almost none really are.

Junior: I figured that if I solved a problem, the users will come.

Intermediate: I tried to find a niche, solve the problem then the users will come.

Senior: I finally understand why the users aren't coming, I've learned to go to them.


MANAGER

If you get enough people together, they get cranky so someone has to keep them working. Management is absolutely essential, even if it's subtle. Being a good manager of programmers is far more complicated then being a good manager of most other groups. Culture, intellect and a high degree of immaturity make it tough. Patience, understanding, some hand-holding and a firm but fair attitude are essential to keep things on track.

Junior: I figured that if I got a good-enough team together, they will build it.

Intermediate: I figured that if I treat the team well-enough, they will build it and it will be good.

Senior: I learned to remove all of the reasons for the team not to build it.


INDIVIDUAL VISION

Just a little side rant, that had no where else to go. It seems like a good fit right here:

In my above description of a visionary I left a huge gap for someone to creativity find the problem to be solved. In many ways, this is the very same creativity that so many programmers "want" to solve during their development phase. Often, at the coding level, programmers want unrestricted ability to put their own heavily creative touch into their works.

We see this again and again, where the programmers in a big company, for example, find a little place in the code and do something interesting and creative. That's great for the programmer, as they may get an enhanced sense of pride in having contributed to the over all project, but honestly it is the very same reason that most of the code coming out of these big companies is hideous, ugly and hard to use.

Dozens of young programmers deviating from the design to put in their own erratic little personal touches does not make for a good system. It makes an ugly inconsistent mess. It's ironic, when all of that extra effort, is in fact highly negative.

This phenomenon shows up in any large project, commercial or Open Source, when it is no longer being driven from one consistent perspective. For a tool to be good, it must hold together at a higher level to solve the user's problems. Little clever bits of functionality, badly placed, don't help the users. A nice feature, that's buried in some non-obvious place, is wasted effort. And a system full of too much wasted effort, is one that has taken a sharp negative turn. So many systems have turned to mush under the yoke of too much micro-creativity. It's an epidemic.


AND FINALLY

I put my experience in terms of myself, but I'm sure there will be a few people who disagree, as usual. Anytime we try to distill things down to their essence, we slip into subjective territory. Still, the more we decompose things, the better we get. Some day, in time, when all of the bias is removed, this will all be regulated to an introductory textbook somewhere. Until then, comments are expected. Please be nice :-)