Why is it, that if you are building a large condo building you can unleash a team of electricians to do all of the wiring, with the expectation that in each an every unit, the wiring will be mostly identical? The plugs will be in the right places, or if not, at least reasonable places?
If you unleashed a team of programmers onto the building with similar directives, you'd come to find that the wiring in each and every unit is different. In some, the plugs would be all together in the same place, in others there wouldn't be any plugs. Probably in one unit there would be a giant plug directly in the center of the floor of the living room, while some other unit might have a few hundred plugs jammed together into the ceiling in some odd pattern.
The only thing you could be sure about is that very few, if any of the condos, would have the appropriate wiring, and if they did, it was more for accidental reasons, than not.
And when questioned, each and every programmer on the team would give you a good solid reason for their deviations. They would explain how they've optimized this problem, or dealt with a particular issue. They'd be proud of their work, while explaining how their other team members failed to recognize its particular brilliance. How, if only they had been allowed to wire all of the other units ...
In the same vein I read the following, from one of Filip van Laenen's entries in 97 Things Every Programmer Should Know, as he talks about standards:
"Once the project gets underway, though, these good intentions are abandoned, one at a time. When the project is finally delivered the code looks like a mess, and nobody seems to know how it came to be this way. When did things go wrong? Probably already at the kick-off meeting. Some of the project members didn't pay attention. Others didn't understand the point. Worse, some disagreed and were already planning their coding standard rebellion. Finally, some got the point and agreed but, when the pressure in the project got too high, they had to let something go."
The full text is here:
http://programmer.97things.oreilly.com/wiki/index.php/Automate_Your_Coding_Standard
I wouldn't necessarily believe what Filip says in this quote except that I've consistently seen this in practice with development teams. Particullary the part about the "code rebellion".
A group of programmers get together, decide on how they will proceed, and then they all go off and do their own thing (as if they had never met). In the most serious cases, the project becomes irredeemable, and even if there are a few OK initial releases, the long-term fate is doomed.
At any level, this is fairly self-destructive behavior. If the code is built on completely un-related ideas, standards, etc. then it is bound to show up badly. The interface is ugly, over-complicated and inconsistent; there are bugs caused by the different sections not inter-operating correctly; and it is nearly impossible to extend the system, so the long-term direction of the project is in jeopardy.
A single code-base divided into fiefdoms is at war with itself. An inconsistent mess, will always be nothing more than an inconsistent mess.
So why does this happen? I've been pondering this question since I first started seeing its results decades ago.
It just doesn't seem all that hard to assemble a team and get them all on the same page before you start building something. And it is necessary. Without this shared vision for how to proceed, it is only left to the slimiest of lucky chances that the final product will be coherent, and certainly as time passes and more extension are made, the chances of success fall even further. Without this shared vision, programmers can't work well in groups, and that restricts the type and size of the things that they can reliably build.
When I first started thinking about this problem, I assumed that it was related to background experience. That programmers choose particular styles, conventions, structures and even architectures based on where and when they learned to program.
As most programmers never got any formal training in how to write code, the large variety of approaches seemed to make sense. And overall, it meant that this was really a lack of training issue. It seemed that, if you spend the effort you could take any team of programmers and eventually train them to all work in a similar way. There'd be some distinction because of aptitude and skill, but mostly the output would be reasonably similar.
At least that's what I initially thought. But I guess that's one of the real benefits of time and experience; you can have these ideas, but reality has a way of wearing at them, gradually turning them into dust. Rudely showing you what you don't understand.
I was once again pondering this issue and I realized that there seemed to be a real correlation between the personalities of some of my fellow programmers, and their output. That is, over the years, I've worked with a large number of different programmers, all of who had their own style. Some programmers are pedantic, while others take a 10,000 feet view. Some prefer abstraction, while others just want to pound out the code, line by line. Some document well, while others are more concerned about being fast. Some are very ridged, while others are extremely sloppy. Some think that whatever works is best, while others have a more long-term approach to their efforts. There is a huge variety in their attitudes, and their outputs.
But as I sat there, and thought about my experience working with these different coders, I couldn't help but group them together by their style preferences. And I as grouped them, I started to realize that style wasn't the only common characteristic. Where a programmer was more detail-oriented for example, their overall personality was similar. Where they distrusted abstraction, they also tended to be very blunt in their approach to communication and to life. Where they obsessed about the smaller details in the code, that too carried over to other things in their environment, like their cloths or their environment.
And realistically, all of this makes perfect sense. All people have their own unique personality which naturally affects how they go through life, so why shouldn't this also affect their work, and its output? It all seems rather obvious.
Now, one would expect this if it came down to something like writing. We all have different wants, and styles, which come through as we write stuff.
But some forms of writing do require shedding our own voice. Work like technical manuals, research essays or marketing brochures, for instance. If I'm writing something personal, I leave lots of myself behind in the words, but I am still capable of writing something a little less egotistical, a little less about myself. I don't have to be there, if I don't want to be. I don't have to put my personal stamp on the work.
However, unlike most other professions, programmers don't seem to be able to operate at this larger more unified level. They seem incapable of 'egoless' work. Whether it is because we've just always allowed it to be this way, or there is something else magical buried at the depths is a good question to ask.
The magical answer is probably just wishful thinking. After all, underneath code is just a series of instructions assembled for a computer to execute. There is little art needed if the code is minimized to its extreme. There is little art needed if the code is re-used to its extreme. There is little art in organizing it, under some specific structure. There is little art necessary, if the coders are just implementing algorithms they found on the web, or in text books. There is art in the higher level choice of what to build, but at the programming level it is mostly just making the appropriate trade-offs. And these trade-offs are well-established (although not always well-known).
The only real art available in programming is by making it all more complex than necessary. By using one's imagination to add in things that just aren't required by the technical or the business domain. Imaginary problems can require great creativity to solve.
The more likely answer is that we just allow it. That it has grown up in the culture, and since we really don't teach programming, we don't teach any reasonable approaches towards getting the work done. Sometimes people learn the table scraps from Computer Science, but rarely, most just show up and start working one day. Working and learning from all of the attitudes around them; learning that being a prima-donna isn't just accepted, it is also the best way to get noticed. And learning that so long as the code appears to work, there are no consequences from doing a bad job. Getting it done fast is better than getting it right. We do a great job in teaching bad habits.
Still, for whatever reason -- necessary or not -- our standard programming practice is that most of the work is highly effected by a programmer's style, and thus by their personality.
The really disturbing part is how this affects our overall development strategies.
The deepest point is that one programmer's code is another programmer's disaster. That the personality variances are too great, and thus any code itself is entirely subjective. Well at least to people with varying tastes. Programmers that flock together, and are similar can agree on a definition of elegance, but for those on the opposite side of the personality spectrum, there can be no such agreement.
And so code will always be subjective, and there will always be at least one programmer out there that will want to re-write ANY work -- even if it was a work of pure genius -- because it wasn't done properly. In that way, nothing is ever finished, and nothing is ever elegant.
Obviously any of those ideas about programmers being interchangeable parts, that can be resources staffed from a 'matrix', also fail miserably. It's a rare occurrence that you can easily swap programmers, and it only works if they are similar in both an experience and personality level. Otherwise, the incoming programmer will not work effectively with the existing code. They'll have a pathological need to change it, to suit their own personality.
And while it is hard enough to assemble a small team of similar personalities, assembling a large one that isn't defective, may actually be impossible. It may be that there is no alternative but to watch as the different people all go off into their own directions. Maybe large scale software development is inherently bound to failure?
All in all, the idea that style is effectively bound to personality is highly problematic. It easily implies that organizing teams based on personality is the only hope for getting a big result. Mixing and matching personality types inevitably leads to failure. The final work won't be consistent. But it would be next to impossible to precisely only recruit only those with compatible personalities. That means that the teams themselves would have to be able to easily add and remove players, and be encouraged to do so. In turn that type of chaos would create new problems, such as scheduling, coup d'états or just making sure all of the different necessary technical positions are well-covered.
What we are really after are ways to make the process of software development significantly less risky. Once the design direction has been decided upon, the development of good, clean software should just be a matter of time. Right now, while time is a significant issue, it is often the least of the real problems. Personality conflicts are far more sever and they waste both time and morale. A team at war with itself is a dysfunctional team, so even if they somehow mange to deliver something that works, eventually they won't.
Sunday, May 16, 2010
Thursday, May 13, 2010
Why?
Because when I see it there, blinking away, patiently waiting for me to give it something to do, I can't help but realize how much power and potential it has. And how little of that is really being used right now.
Because even though it is crippled by a complete lack of understanding, there are still moments where it has actually made my life easier. Moments were I think "Without this, I would be suffering, yet now I am not."
Because I realize that it can remember things, better than I can. Calculate things faster than myself. Organize things better than an entire library, and lay over some incredible structure in an instant. It can perform feats of effort greater than whole institutions, and then make those results instantaneously available to the entire world.
Because I know that while it is intrinsically stupid, fragile and will always be that way, underneath it can dynamically manipulate a multitude of symbols in a near infinity of arrangements and present back to us new perspectives, new ways of looking at things. A higher view of what we put in. It can help us distinguish between what we know and what we really understand.
Because so many people proclaim the complexities to be art, and the art to be trivial, and the trivial to be complex. They spend all day buried in the minutia without ever understanding the context. They spend all of their effort hopelessly flailing hard when they needn't put in even a fraction of that effort to get it to work.
Because at our current rate of innovation, it is more likely that something really bad will happen to us, long before we've figured out how to avoid something really bad happening to us.
Because after twenty five years of exposure, and a lot of it agonizing about how to really use these things, I figured that my understanding can't be any worse than what is already out there, and probably because I cared and pondered and thought, that it was likely a whole lot better than most of it. Real practical experience trumps wishful thinking, every time.
Because ultimately, even though I am disappointed by a surrounding slimy, arrogant and foolish industry, I can't help but to realize that these things could do a whole lot more than just make our lives a tiny bit better. They have massive untapped potential, towards which we are doing absolutely nothing but belting out endlessly repeating lists of instructions, while believing that some how bad code and bad management will magically make our lists usable.
Because if it goes unsaid, it goes un-thought about. And if no one is thinking about it, then it won't ever change, and if it doesn't change we'll just keep on writing the same stuff over and over again, year after year. If we don't seek to change our direction, we'll just keep circling around the same tracks forever. And if you pop your head out of the coding mills long enough to see if what we are doing is actually working, you're only going to be disappointed. Really disappointed.
Because somebody has too, and more people should.
Because even though it is crippled by a complete lack of understanding, there are still moments where it has actually made my life easier. Moments were I think "Without this, I would be suffering, yet now I am not."
Because I realize that it can remember things, better than I can. Calculate things faster than myself. Organize things better than an entire library, and lay over some incredible structure in an instant. It can perform feats of effort greater than whole institutions, and then make those results instantaneously available to the entire world.
Because I know that while it is intrinsically stupid, fragile and will always be that way, underneath it can dynamically manipulate a multitude of symbols in a near infinity of arrangements and present back to us new perspectives, new ways of looking at things. A higher view of what we put in. It can help us distinguish between what we know and what we really understand.
Because so many people proclaim the complexities to be art, and the art to be trivial, and the trivial to be complex. They spend all day buried in the minutia without ever understanding the context. They spend all of their effort hopelessly flailing hard when they needn't put in even a fraction of that effort to get it to work.
Because at our current rate of innovation, it is more likely that something really bad will happen to us, long before we've figured out how to avoid something really bad happening to us.
Because after twenty five years of exposure, and a lot of it agonizing about how to really use these things, I figured that my understanding can't be any worse than what is already out there, and probably because I cared and pondered and thought, that it was likely a whole lot better than most of it. Real practical experience trumps wishful thinking, every time.
Because ultimately, even though I am disappointed by a surrounding slimy, arrogant and foolish industry, I can't help but to realize that these things could do a whole lot more than just make our lives a tiny bit better. They have massive untapped potential, towards which we are doing absolutely nothing but belting out endlessly repeating lists of instructions, while believing that some how bad code and bad management will magically make our lists usable.
Because if it goes unsaid, it goes un-thought about. And if no one is thinking about it, then it won't ever change, and if it doesn't change we'll just keep on writing the same stuff over and over again, year after year. If we don't seek to change our direction, we'll just keep circling around the same tracks forever. And if you pop your head out of the coding mills long enough to see if what we are doing is actually working, you're only going to be disappointed. Really disappointed.
Because somebody has too, and more people should.
Saturday, May 1, 2010
Insane Complexity
As a user, it is nice to have software features. Well, it is nice if they are actually accessible. Lately, I seem to be engaging with a lot more software that is feature-rich, but nearly impossible to use.
It feels like the programmers have become so excited about adding new things that they've forgotten to add them sensibly. A thousand features are no good to me, if I can't use them. They're no good to me, if I have to read pages and pages of badly written instructions, or pages and pages of web forums complaining about problems. They're no good to me, if I can't even guess at whether they exist or not.
Software plagued with too many features crosses over into "insane complexity". It is so horribly over-complicated that all of the extra programming effort to add this stuff is effectively wasted. It is unusable.
There are so many examples to choose from. I'll pick some easy ones. Most Microsoft products are feature-rich, but obtuse. I've seen several nasty products come from IBM as well. The old mainframe systems are a labyrinth. Even Unix, which started life as being consistent and philosophically driven, has really descended into chaos with spin-offs like Linux. The list goes on, and on.
Users don't want to be presented with large screens containing a million options. They don't want a million little tiny controls. Most of the time, they don't want to "tweek" their installs. They're not interested in the super cool ways of doing things, and most of them hate having to fiddle with any poorly named config parameter. As a user, all we want is something that is simple to setup, simple to use, and that works correctly. Great big drop-down menus of useless, obscure features are depressing. They're intimidating. They are not good design, and they aren't appreciated. Command line tools with a million arguments are hopeless too. There is just too much "stuff" to bother with.
It is easy to write a few lines of code that works. It is easy to add a menu item to an existing interface or add a parameter to the command line. It is easy. There are a huge number of people that can do this; that can brutally extend some existing piece or create something small that works. This, however is not good software development. It is equivalent to just gluing a garden shed onto the side of your house, knocking a hole in the wall and then calling it an "extension". It is hacking. It may seem constructive in the short run, but ultimately it is gradually taking away the existing value of the effort. It is a -1. Negative effort.
Programmers often get so excited about their "chance" to create new features that they forget about the the users. This lack of empathy, shows over and over again in our modern creations. Every so often, a few new technologies will come forth like the iPhone, with simple interfaces, clean lines and easy usability that remind us that 'simple is possible'. That our ugly massive constructs, needn't be so convoluted. It just ain't necessary.
But while simple user interfaces get high praise, we can't forget that they are the 10% of the iceberg that is most visible. What lies beneath, in terms of configuration or operation, also gets ugly. And what lies beneath that, the code itself, is ugly too. Generally, the sins of the coder propagate upwards, eventually polluting the user's experience. What starts as a bit of disorganization before a hasty release gradually spawns into a complexity explosion. Software arcs. That is, once it gets going, it gets better for a while until it gets to the hump. From that point onwards, each new version is worse than the previous one. It's the software equivalent of 'jumping the shark'. Generally for most products, once they've started this downward spiral, they stick to it. It is a nosedive.
As an industry, we are definitely in trouble. The 'software crisis' which began almost immediately, over fifty years ago when the industry started, gets worse on a daily basis. Access to free and open software simply accelerates the problem. Programmers can now toss in so much useless functionality that it is sometimes impossible to determine the purpose of the code. The "all singing, all dancing" software, that is also cryptic, is useless. Adding features doesn't make it better. Adding features doesn't make the core work. Adding features that are inaccessible doesn't actually solve the problem. It just wastes time, and makes it worse.
I don't want features any more. I don't want options. I don't want to have to read a dozen pages of text to "grok" it. I just want something simple, that works.
It feels like the programmers have become so excited about adding new things that they've forgotten to add them sensibly. A thousand features are no good to me, if I can't use them. They're no good to me, if I have to read pages and pages of badly written instructions, or pages and pages of web forums complaining about problems. They're no good to me, if I can't even guess at whether they exist or not.
Software plagued with too many features crosses over into "insane complexity". It is so horribly over-complicated that all of the extra programming effort to add this stuff is effectively wasted. It is unusable.
There are so many examples to choose from. I'll pick some easy ones. Most Microsoft products are feature-rich, but obtuse. I've seen several nasty products come from IBM as well. The old mainframe systems are a labyrinth. Even Unix, which started life as being consistent and philosophically driven, has really descended into chaos with spin-offs like Linux. The list goes on, and on.
Users don't want to be presented with large screens containing a million options. They don't want a million little tiny controls. Most of the time, they don't want to "tweek" their installs. They're not interested in the super cool ways of doing things, and most of them hate having to fiddle with any poorly named config parameter. As a user, all we want is something that is simple to setup, simple to use, and that works correctly. Great big drop-down menus of useless, obscure features are depressing. They're intimidating. They are not good design, and they aren't appreciated. Command line tools with a million arguments are hopeless too. There is just too much "stuff" to bother with.
It is easy to write a few lines of code that works. It is easy to add a menu item to an existing interface or add a parameter to the command line. It is easy. There are a huge number of people that can do this; that can brutally extend some existing piece or create something small that works. This, however is not good software development. It is equivalent to just gluing a garden shed onto the side of your house, knocking a hole in the wall and then calling it an "extension". It is hacking. It may seem constructive in the short run, but ultimately it is gradually taking away the existing value of the effort. It is a -1. Negative effort.
Programmers often get so excited about their "chance" to create new features that they forget about the the users. This lack of empathy, shows over and over again in our modern creations. Every so often, a few new technologies will come forth like the iPhone, with simple interfaces, clean lines and easy usability that remind us that 'simple is possible'. That our ugly massive constructs, needn't be so convoluted. It just ain't necessary.
But while simple user interfaces get high praise, we can't forget that they are the 10% of the iceberg that is most visible. What lies beneath, in terms of configuration or operation, also gets ugly. And what lies beneath that, the code itself, is ugly too. Generally, the sins of the coder propagate upwards, eventually polluting the user's experience. What starts as a bit of disorganization before a hasty release gradually spawns into a complexity explosion. Software arcs. That is, once it gets going, it gets better for a while until it gets to the hump. From that point onwards, each new version is worse than the previous one. It's the software equivalent of 'jumping the shark'. Generally for most products, once they've started this downward spiral, they stick to it. It is a nosedive.
As an industry, we are definitely in trouble. The 'software crisis' which began almost immediately, over fifty years ago when the industry started, gets worse on a daily basis. Access to free and open software simply accelerates the problem. Programmers can now toss in so much useless functionality that it is sometimes impossible to determine the purpose of the code. The "all singing, all dancing" software, that is also cryptic, is useless. Adding features doesn't make it better. Adding features doesn't make the core work. Adding features that are inaccessible doesn't actually solve the problem. It just wastes time, and makes it worse.
I don't want features any more. I don't want options. I don't want to have to read a dozen pages of text to "grok" it. I just want something simple, that works.
Subscribe to:
Posts (Atom)