We, the software development community, are often the source of our own really bad software. Sure, you can blame the managers or you can rail against the underlying technologies, but in the end the most frequent, most destructive problems come from the attitudes of the people involved in the development.
Software, as in any endeavor that involves constructing physical things, requires a great deal of discipline. A mess by definition is repetitively doing things differently. All of those inconsistencies add up, and unless it's some type of creative writing or a painting, the result is big and ugly. Picture a house built in ten different styles, or a car made up of several different models. The 'essence' of a mess is inconsistency.
Software development projects are always big. Big projects require big teams. And big teams need to work together. If they don't work together, the result is a big mess.
I have seen enough projects where the teams are maintaining hundreds of thousand lines of repetitive, redundant and inconsistent code when less than a quarter of that should have been enough. It is easy to see the effects of that type of problem, as the team wastes more and more effort on weak testing strategies and chasing inconsistencies throughout the code. It is a black hole for resources. You can't fix the problems if you are too busy slapping on band-aides.
By its very nature, building a software tool is difficult, but not, as many programmers want to believe, because of mapping functionality onto specific computer language code. That part of development is relatively straight-forward, the difficulties come from analyzing the user problems and choosing the appropriate functionality to implement. Well, they also come from dealing with a messy code-base, but that wound is entirely self-inflicted.
The analysis problems are unavoidable, but ultimately they are not related to programming. Once a team has decided on how to 'solve' the problem, the rest of the effort should just be work. Nothing but work. A group of professionals using their collective knowledge to come together and assemble the appropriate instructions to implement some functionality on well-known data.
In the midst of this, there are always changes, but those small unavoidable shifts in understanding can be easily managed across the lifetime of a sophisticated software product. If you were close to begin with, the changes are never really significant.
The really huge problem with our software projects is our own tragic culture of uniqueness.
It is not, in any way, shape or form, that I want to take away from the human individuality. We are all individuals, and we shouldn't be forced to band together as helpless sheep or powerless ants. But, and here is a huge point: it is that very sense of individualism that is destroying so many projects. The blame, often fails on the guy down the hall, or the manager in charge, or someone else, but the 'problem' at the end of the day is the inconsistent mess created by the programmers that is not working correctly.
It is inevitable that if a team of developers flails away at their keyboard, without working in some harmony, the code will become a mess. Developers will leave, managers will change, these things are normal. If every programmer is silo'ed into their own over-protective code-possessive region of the project, the inevitable changes and shifts are going to bring pain. Quite possibly disaster.
A team that is not working together, that does not share all of their code, that does not follow similar standards, etc. is a defective team. One that is just waiting for trouble. And the effort of a defective team, not unsurprisingly, is always a weak and fragile project.
Yet, even as we all know this intrinsically, and so many of us have had this as a direct experience in our professional lives, we still bend towards the dark side. We don't want to be seen as just 'resources', or to be 'interchangeable', or even 'replaceable', but by not being these things we are sabotaging our own projects.
Presumably, if you get any good professional who follows their industry's best practices properly, they should be able to perform the job for which they were commissioned. There is variation to some degree, but ultimately the very definition of 'professional' means that everybody lives up to some recognizable standard; by its nature that means they can be replaced.
Your life shouldn't rest on the skills of your lawyer, your doctor or your accountant. Scientists, mathematicians and engineers shouldn't be able to set their disciplines back hundreds of years because of incorrect theorems or approaches. You should be able to hire a competent professional to accomplish a specific job for which they have knowledge and experience. And yes, these people, are 'resources', that are 'replaceable'.
There is nothing short of insanity, that would lead one to believe that 'programming' as a practice was anything other than a professional activity. Don't confuse vision, analysis or leadership with programming, those are wishy-washy irrational things, not unlike business acumen. Programming, itself is just the act of taking the desired functionality and encoding it. It is a task that can be accomplished by any well-trained professional programmer.
There should only be one good standard practice of doing this. Some variation, of course, but a team of programmers should come together, pick their conventions, and then stick with them. Thus, beyond all of the work and the team itself -- if it is functioning correctly -- should be a team of 'resources' dedicated to getting the job done correctly. Losing one member, aside from the morale or leadership aspects should not change the outcome of the team. That would be bad. That would be unprofessional.
In truth, although the overall direction might change somewhat, a different analyst, product manager or even visionary leader in a well-healed team shouldn't jeopardize the overall efforts either. All professionals bring with them some minimal level of functioning. That should be enough to keep the project moving forward, even if it suffers some change in direction.
Catastrophic problems occur, but they are individual issues. A bad hire is a bad hire, not proof that a discipline is too chaotic to be standardized. Or too radical to be repeatable.
I do believe that a good product needs a strong unique vision, not a committee, to steer its direction. Consistency starts at the top, but even great direction and leadership at the higher levels fall short if there is chaos in the ranks. Once a development path is forged, the implementation team needs to follow it, not wander around aimlessly exploring options.
Filling in a blank is quite different from re-interpreting the design. A product is only unified if its developers are; inconsistent code sucks; inconsistent interfaces suck.
Honestly, I don't want to work with any more 'irreplaceable' programmers. I've been there, and it is bad. You get some programmer that is so far out on the edge, that virtually nobody else can understand or fix their code. That is, by any definition, crappy code.
If your team can't live without a specific person, then your team is dysfunctional. It is that simple.
This is true at most levels. The best managers know how to make themselves a dispensable part of the process, it is the poor ones that are so controlling that they hide the details from everyone else. It is absolutely true for technical people as well. Those people that horde their knowledge, hiding in their cubicles, smug in their own superiority, are not the ones you really want in a well-functioning team. I'd gladly trade the best of them for a good team player any day.
It is intrinsic to the very definition of a well-functioning team, that its members are 'resources'. It is intrinsic to the very definition of good code that it is readable and can be maintained by any competent programmer. It is intrinsic to a software development project that everybody is working towards the same goal. These are not good ideas, they are requirements; things that need to be urgently fixed if not true.
You can blame management all you want for various failures, but if you're not willing to play nicely with the other developers in one big sand box with all of the toys, then you're probably more at fault than you could ever realize. Programmers never work alone.
Quick question: How does this view that 'there are no indispensible' programmers square with the well known fact that programmers vary significantly in productivity (2-10X or 10-100X)?
ReplyDelete@js
ReplyDeleteThe fact (or requirement) that no programmer is indispensible does not mean that they are all the same. As I understand the 'there are not indispensible', they must all have at least some minimum of professionalism. Do not keep knowledge for yourself only, work with others when there is a problem, share common techniques and coding idioms... The kind of things that you expect from people working in a team toward a common goal.
Still, "being professional" is not the same as "being a field expert" or "being a [insert-your-language] guru". Those professional programmers always have different skills, be it technical or functional skills, different backgrounds, habits... Some might be more clever than others, understand things faster, have more experience with the language/tools/domain.
Being "productive" in IT demands a mix of skills that include intellectual skills, being able to work in teams, being able to organize one's work, knowledge of the tools and of the domain in which the software is used, etc. It should be no surprise that there can be a (sometimes huge) difference of productivity between programmers, even between profesional programmers. It is the same as in any other field, art, science, whatever. One can work to be better, one can work to help others be better (*) ; but, in the end, we will always be different.
(*) I strongly believe one should strive to do both. To me, improving one's skills *AND* sharing those knowledge and skills with one's co-workers are part of the requirements for a 'professional programmer'. That is not easy though, as time pressure and ego can distract one from helping others/accepting help. But it is something we should strive to do anytime we can.
Great question JS, and excellent answer Al.
ReplyDeleteI'd just like to add that sometimes I worry that the extreme differences in productivity come from measuring unharmonious teams.
Is my doing 10X the work of my other team members actually interfering with their ability to get their job done? If I've got my own quirky way of belting out code, that they have to rely on underneath, I can optimize it to allow me to write it faster at the expense of other people being able to use it. It's another lessor known trade-off: simplify the code or simplify the usability of the code. It's easier and faster to write overly-simple hard-to-use code.
People will all have different strengths and weaknesses. Good staffing balances their individual skills, with trying to give them a wider range of experiences. You might have a database expert on your team that you rely on for the critical design, but all of the members should get their chance to work on the database so that they are well-rounded (and your knowledge is not siloed). That is not the fastest way to get the work done in the short-term, but it is the most stable way to get it done in the long run.
Paul.
Hi Paul. I posted a very similar response to a comment of yours on Mark Turansky's blog, but I thought I'd leave one here as well.
ReplyDeleteAlthough I disagree with your fundamental premise, it doesn't mean that I think that software developers are irreplaceable or should act like prima donnas. There's a real cost to swapping people in and out of projects or positions, because no two people are exactly the same - in cognitive ability, experience, or background. When employers refer to employees (any type of employees!) as resources and assume that a task that should take Employee A 2 days will necessarily take the same time for Employee B, they're thinking of the employees as interchangeable. There's a big difference between being replaceable and being interchangeable. In contrast, one of the appeals of cloud computing (particularly the Amazon EC2 flavor of it) is the ability to allocate 10 identical virtual machine resources, have them work on a problem for you for 8 hours and then just give them back to the pool. Those virtual machines really are interchangeable. You're not suggesting that any type of employee is that interchangeable, are you?
Hi Matt,
ReplyDeleteThanks for your comments.
"You're not suggesting that any type of employee is that interchangeable, are you?"
Yes and no. I am suggesting that a professional, acting in their professional context is entirely interchangeable. If I hired you to be my database expert, I'd expect that you'd be able to produce a good to excellent schema. If I hired you to build me a standard GUI to handle tracking user documents, I also expect it would live up to my expectations.
There are plenty of examples of specific types of skills that vary hugely, even amongst skilled professionals. I wouldn't expect that any two artists would produce the same painting for example, or that two graphic designers would settle for the same color scheme. However, within a specific category, to whatever variability that is normally expected, if you've hired two good people for the same job, they both should be able to do it competently.
Two different business/domain analysts might see different resolutions to solve the user's problems, but two reasonable programmers should be pretty close in their implementations of a quick sort. Readability and elegance tend towards a minimally subjective base-line. We tend to confuse picking the solution, with implementing it, but the two are very different.
Paul.
Well its good that you are admitting your inefficiencies like this.You are very honest people.
ReplyDeleteCulturally, technical people often pride themselves on their honesty, but often, like most other human beings we can easily delude ourselves into believing what sounds better, or is more entertaining.
ReplyDeleteUnfortunately, we won't make any progress with improving our ability to build complex software if we can't first learn to see our efforts realistically.
Paul.