This is a pure rant post, composed while I was wasting my life waiting for Eclipse to decide whether or not it can actually install some packages (it decided not to). Don’t read this if you don’t like ranting, or are sensitive about the nature or quality of modern software.
The end goal of all software is to utilize a computer to perform some tasks for the user. If the software makes it easy to perform these tasks, then it is a good. If not, then the software is likely the culprit, particularly if the tasks are simple.
What computers do well is remember things. Because of that, all I want to do is give the stupid machine a high level directive and let it fill in the blanks. I don’t mind having to tell it all of the details once, but when I have to do it over and over again, it quickly gets ridiculous. Some software does this well, but there a lots of examples of stuff that just takes the lazy approach, particularly when it comes to installations.
The other big annoyance with software is a boat load of functionality that is randomly scattered all over the place. Most problems come in groups of things you want to do and there is nothing worse than having to waste time flipping around endlessly inside of badly thought-out interfaces to find each of the little pieces necessary in order to accomplish a group of related tasks. Rarely do I do things that are exotic. Mostly I am using the software to perform its central task, the one that the programmers built it to do. So if I have to bounce around a lot, it shows how poor the collective thinking was on the part of the programmers.
Some software that bugs me:
Just about every product that has ever come out of Microsoft. Mostly because they maintain an army of young, smart, but poorly trained coders who work very hard on each piece in isolation. The finished products are always masterpieces of stupid complexity, disorganized interfaces and random functionality.
An example of this is that my Windows machine does weird things from time to time. I don’t know what cause these glitches, nor will I ever know. I’d list them out, but they change so frequently, and are always different with each descending new version of Windows. I’ve given up tracking them. I’d change some of my configuration parameters, but given that there are now an endless number of them distributed in overlapping sets across countless interfaces, I’m just as likely to make things worse as I am to make them better. It’s safer to just pretend they don’t happen.
Another example is “Security” in SQL Server (users, groups, roles, etc.) which is utterly demented. I’ve been in SQL Manager a billion times (OK, a few less than that), and by now I should at least be getting some idea of how the different pieces relate to each other. No doubt there is some relationship or model or some other twisted logic at play, but so far its hasn’t been obvious. The randomness in SQL Server reminds me of the insanity of the types and auto-formatting in Word. That used to work until they “improved” upon it so well that I’m always forced to always shut it off, or risk it destroying all of my work.
And thanks to a lack of overall design on their part, I’m running poorly written virus checkers, bogged down in policies and using other randomly dished out hacks, to mitigate the possibility of illegal access (even though my machine is physically secure). Between that mess and software bloat this causes my current box to run slower than DOS 6 on my IBM AT did twenty years ago (or is it thirty now?). If Moore’s law predicts the steady increase in hardware capabilities, an inverse that should be called Microsoft’s law, explains the corresponding decrease in software usability and performance: the more hardware performance you have, the more newbies will sloppily waste it.
But the very worst part of their works is the testing. I think they still use a 1-to-1 tester to programmer ratio, which means that while the applications are annoying, unpredictable and hard to use, they are patched with just enough duct tape that they “appear” workable.
Hibernate. This library is a classic example of awesome functionality and extreme usefulness buried beneath a layer of horrible APIs and bad documentation. The whole ORM concept is great and really useful, but I’ve always felt like the authors of Hibernate were so concentrated on their cool underlying algorithms that they couldn’t care less about the users. And in their case, the users are the other unfortunate programmers that want to utilize their technology. The libraries seem to go out of their way to make themselves obscure, weird, inconsistent and generally hard to grok. Once you get something figured out, you can use it to great success, but it’s always a beast to get there. Just because you know half of Hibernate, that knowledge sheds no light what so ever on the other half. Sophisticated underlying algorithms are great, but if the access to them sucks, then that severally diminishes the whole package.
Eclipse. Not sure, but I suspect that the Eclipse project is the single largest collection of programmers ever to come together to work on a single thing. A magnificent concept, but more for the fact that the by-product of this collaboration is possibly the most annoying, over-complicated, horrendously complex and silliest thing that humanity has ever produced. Recently I wanted to play around with Hibernate Tools, which exists as a plugin for Eclipse. Easy right, just install the basic Eclipse, point it to a repository with the Tools and then install them. Yep. Nope. Basically after wasting a day being bounced between useless web pages, and some really bizarre dependency handling, I’ve pretty much decided to give up. Whatever tools are out there are not worth the aggravation of having to navigate through a maze of randomly named gibberish, useless pages and other nonsense. Get something, install it, and then use it. How hard can that be?
All of the above -- and many more -- suffer from the same disease. The programmers have transferred their love of stupid complexity onto their users, as if they somehow expect the users to appreciate or thank them for it. In the Microsoft case I have no choice but to use their stuff, it is mandated from above. I am a captive audience. In the Hibernate case, the basic technology is too useful, so I have to be forgiving. But they should note that if something comes along that is even marginally better, I’d drop them in less than a second. In the Eclipse case, I used to use it, but then I switched to NetBeans and have never regretted it. NetBeans has its own set of annoying problems, but in comparison to Eclipse they are nearly trivial.
When programmers hyper-focus on the code they often forget about why they are building the software. Sophisticated code is nice, but not if it is wrapped in a shoddy package. It’s the whole thing that matters, not the bits.
Thursday, September 30, 2010
Thursday, September 9, 2010
Rock Hopping
The river is wide, its waters treacherous. On the bank stands a man -- a rock-hopper -- who wants to get over to the other side. In the river, scattered randomly, are a huge number of flat-topped boulders, spaced just close enough to each other that the rock-hopper knows he can leap from one to another.
He starts out, leaping from rock to rock, focusing his full might on the effort. Going left, and right, he manages to get from rock to rock without falling into the currents. In the end, he leaps off the last rock squarely onto the bank and proclaims victory; only to realize a moment later that he has landed right back onto the same bank from which he started.
Leaping from rock to rock is a tactical challenge. It is a short-term effort focused squarely on getting to the next rock. Getting to the other bank however, is a strategic challenge, it involves the longer term effort of picking the right path through the rocks. It often requires picking harder leaps, since the easier path may just lead back to the same bank.
What I often see with people is that they become so hyper-focused on the tactical challenges, that they completely ignore the strategic ones. It happens in every profession, we hear plenty of stories about how business people for example, walk away with huge performance bonuses only to have the company die from neglect shortly after. It is far easier to concentrate on the short term, and it is far easier to proclaim success.
But software development is not entirely about writing lots and lots of code. Sure that is the tactical side, but the strategic side is getting the right code to the right people, release after release. Its not just getting it started, but also keeping it going. All software projects, even the small ones, span years and they need continual effort to keep them working and viable. Pointed in the right direction. Winning a tactical battle doesn’t help if the overall strategy loses the war. Short term wins help, but real success comes from the long term.
In software development, we see this in several places. The first is in many people’s belief that the project will work, if and only if they can write enough code. Code is nice, but as it builds up, if it is sloppy and redundant it becomes harder and harder to manage. It’s not how much code we have, it’s how easily we can leverage it that really matters. Development projects should get easier as time rolls on. The momentum should speed up, not down. Rewriting “nearly” the same code again and again, a hundred times over, is wasted effort.
Often though, the reasons why we have so much redundancy in our code, is that people frequently make the tactical choice to not leverage what they have out of fear that it will slow them down. If the first version is somewhat independent of the second version, and you’ve written it once already, many choose to just re-write another version. They hop to the closest rock, without consideration for their final destination. Too many of these hops, and they wind up on the wrong bank.
Another place we see this is with functionality. A hundred quirky little cloned applications spreads more misery than assistance. A fractured, convoluted app that needs months of training is no better. At the user level, the things we build need to be integrated, and they need to have enough of a common philosophy that using them doesn’t require any more than a basic intuitive understanding. All the functionality in the world doesn’t matter if it’s hard to find, or not easy to use.
Still, we see programmer after programmer get the task of adding in some new feature, only to ignore what is already there in their rush to get it done quickly. It may initially look good that they only spent a few months on the effort, but when it hits the users and the complaints start rolling in, it gets harder and harder to justify the cut corners. What seems like a successful new feature, can often leave the developers back on the bank where they started.
Another instance of this comes directly from analysis. I’ve heard so many programmers over the years complain about how the requirements keep changing. A software tool automates something for the users, and often supplies data for management, but it generally does little else. The tasks being automated, in most domains have a long history. Normally there were users doing their work long before computers came onto the scene. Computer systems do cause change, but more often accidentally than by deliberate thought. So, most often, the changes in the requirements are not coming from changes in the domain; sure there are always some, but they do not account for the vast majority. The changes in the requirements more often come from rushing forward too quickly, before the full scope of the problem or technology is understood. They come from the lack of analysis into the real underlying problems that need to be solved. They come from short-cuts. They come from both technical debt, and from analysis debt.
Software developers, in our senior roles, like architects, designers and team leads are the experts in building software systems. We can’t expect the users to fully grasp the technical problems, or the complexities. We can’t leave it up to them to set the engineering schedules, or map out the most reasonable order to implement the code. Sure, we need to value what they say, but they hired us to be the experts, and it is our responsibility to insure that the system is up and functioning properly. That it is usable. Frequently changes in the requirements, particularly urgent ones that have not been noticed previously, are our fault, not theirs. It’s not their fault if they can’t express what they need in our lingo, using our methodologies. It is our fault that we didn’t ask the right questions, that instead we accepted the easy answers in a rush to move forward. Developers become so focused on hoping off the analysis rock that they often forget to head in a strategic direction. If the code is ready before the problem is understood, then it is unlikely to work properly.
This leads to the most significant problem of all. Software development isn’t about code or technical problems. It’s really all about people, and making their lives better in some way. Inexperienced programmers often want to hide out from their users, and hyper-focus only on the code. That is OK, but if they choose that option, then they need someone on the bank helping them with direction. And they need to listen. Someone has to look after the strategic direction, and learning to do that properly comes only from experience. Lots of experience. It doesn’t come from a couple of years of writing neat algorithms, nor does it come from a decade of constructing pretty screens. It only comes from getting down off the chair, and taking responsibility for the scope and direction of the project. From fighting it out, time and again. It’s a long and slow process. Computers are nice, polite, happy machines, people aren’t. But the real problems in software -- the ones that often mean the difference between success and failure -- are not technical. And managing those problems isn’t something that is learned overnight.
Our industry grossly over-values rock-hoppers, particularly the hyper-focused ones. There is too much emphasis on getting it done as soon as possible, and not enough on getting to the other bank. If we have an obscenely high failure rate, it is mostly because we still haven’t learned how to identify the necessary strategic skills needed for success. Anyone can build code. Literally anyone. It’s not a pile of code that matters, it’s what people can do with it that is the key. Returning to the same bank after an exhaustive effort is not only frustrating, but also a waste of time.
We’re faced with wide rivers everyday. It doesn’t matter how many rocks we hop, or how elegantly we do it, if we can’t get to the other bank. Real success comes from the long term.
He starts out, leaping from rock to rock, focusing his full might on the effort. Going left, and right, he manages to get from rock to rock without falling into the currents. In the end, he leaps off the last rock squarely onto the bank and proclaims victory; only to realize a moment later that he has landed right back onto the same bank from which he started.
Leaping from rock to rock is a tactical challenge. It is a short-term effort focused squarely on getting to the next rock. Getting to the other bank however, is a strategic challenge, it involves the longer term effort of picking the right path through the rocks. It often requires picking harder leaps, since the easier path may just lead back to the same bank.
What I often see with people is that they become so hyper-focused on the tactical challenges, that they completely ignore the strategic ones. It happens in every profession, we hear plenty of stories about how business people for example, walk away with huge performance bonuses only to have the company die from neglect shortly after. It is far easier to concentrate on the short term, and it is far easier to proclaim success.
But software development is not entirely about writing lots and lots of code. Sure that is the tactical side, but the strategic side is getting the right code to the right people, release after release. Its not just getting it started, but also keeping it going. All software projects, even the small ones, span years and they need continual effort to keep them working and viable. Pointed in the right direction. Winning a tactical battle doesn’t help if the overall strategy loses the war. Short term wins help, but real success comes from the long term.
In software development, we see this in several places. The first is in many people’s belief that the project will work, if and only if they can write enough code. Code is nice, but as it builds up, if it is sloppy and redundant it becomes harder and harder to manage. It’s not how much code we have, it’s how easily we can leverage it that really matters. Development projects should get easier as time rolls on. The momentum should speed up, not down. Rewriting “nearly” the same code again and again, a hundred times over, is wasted effort.
Often though, the reasons why we have so much redundancy in our code, is that people frequently make the tactical choice to not leverage what they have out of fear that it will slow them down. If the first version is somewhat independent of the second version, and you’ve written it once already, many choose to just re-write another version. They hop to the closest rock, without consideration for their final destination. Too many of these hops, and they wind up on the wrong bank.
Another place we see this is with functionality. A hundred quirky little cloned applications spreads more misery than assistance. A fractured, convoluted app that needs months of training is no better. At the user level, the things we build need to be integrated, and they need to have enough of a common philosophy that using them doesn’t require any more than a basic intuitive understanding. All the functionality in the world doesn’t matter if it’s hard to find, or not easy to use.
Still, we see programmer after programmer get the task of adding in some new feature, only to ignore what is already there in their rush to get it done quickly. It may initially look good that they only spent a few months on the effort, but when it hits the users and the complaints start rolling in, it gets harder and harder to justify the cut corners. What seems like a successful new feature, can often leave the developers back on the bank where they started.
Another instance of this comes directly from analysis. I’ve heard so many programmers over the years complain about how the requirements keep changing. A software tool automates something for the users, and often supplies data for management, but it generally does little else. The tasks being automated, in most domains have a long history. Normally there were users doing their work long before computers came onto the scene. Computer systems do cause change, but more often accidentally than by deliberate thought. So, most often, the changes in the requirements are not coming from changes in the domain; sure there are always some, but they do not account for the vast majority. The changes in the requirements more often come from rushing forward too quickly, before the full scope of the problem or technology is understood. They come from the lack of analysis into the real underlying problems that need to be solved. They come from short-cuts. They come from both technical debt, and from analysis debt.
Software developers, in our senior roles, like architects, designers and team leads are the experts in building software systems. We can’t expect the users to fully grasp the technical problems, or the complexities. We can’t leave it up to them to set the engineering schedules, or map out the most reasonable order to implement the code. Sure, we need to value what they say, but they hired us to be the experts, and it is our responsibility to insure that the system is up and functioning properly. That it is usable. Frequently changes in the requirements, particularly urgent ones that have not been noticed previously, are our fault, not theirs. It’s not their fault if they can’t express what they need in our lingo, using our methodologies. It is our fault that we didn’t ask the right questions, that instead we accepted the easy answers in a rush to move forward. Developers become so focused on hoping off the analysis rock that they often forget to head in a strategic direction. If the code is ready before the problem is understood, then it is unlikely to work properly.
This leads to the most significant problem of all. Software development isn’t about code or technical problems. It’s really all about people, and making their lives better in some way. Inexperienced programmers often want to hide out from their users, and hyper-focus only on the code. That is OK, but if they choose that option, then they need someone on the bank helping them with direction. And they need to listen. Someone has to look after the strategic direction, and learning to do that properly comes only from experience. Lots of experience. It doesn’t come from a couple of years of writing neat algorithms, nor does it come from a decade of constructing pretty screens. It only comes from getting down off the chair, and taking responsibility for the scope and direction of the project. From fighting it out, time and again. It’s a long and slow process. Computers are nice, polite, happy machines, people aren’t. But the real problems in software -- the ones that often mean the difference between success and failure -- are not technical. And managing those problems isn’t something that is learned overnight.
Our industry grossly over-values rock-hoppers, particularly the hyper-focused ones. There is too much emphasis on getting it done as soon as possible, and not enough on getting to the other bank. If we have an obscenely high failure rate, it is mostly because we still haven’t learned how to identify the necessary strategic skills needed for success. Anyone can build code. Literally anyone. It’s not a pile of code that matters, it’s what people can do with it that is the key. Returning to the same bank after an exhaustive effort is not only frustrating, but also a waste of time.
We’re faced with wide rivers everyday. It doesn’t matter how many rocks we hop, or how elegantly we do it, if we can’t get to the other bank. Real success comes from the long term.
Wednesday, September 1, 2010
Construction Metaphors
I know a lot of computer programmers hate construction metaphors and analogies. I think they fear that any comparison to physical labor demeans the intellectual requirements of building computer systems. It is too bad, because there is an awful lot to learn from watching other people toil in mammoth projects.
Construction sites aren’t just exclusively general laborers, there are usually a host of other complex jobs and professions. Skyscrapers, for instance, require architects, drafters, engineers, inspectors, project managers, electricians, plumbers and a host of specialty jobs like elevator installation or crane operation.
The same is true for medium or large software projects, you also get a large congregation of divergent professions and experience. Learning how all of these different perspectives and agendas can work together effectively is something urgently required by software projects. Too often software professionals get focused on the code -- the cement of the project -- only to miss the other necessary elements to achieve success. But a building isn’t functional until all of the other pieces are in place.
Even programmers tend to specialize in their work. It often takes GUI specialists, database admins, algorithmic gurus and system programmers just to get the core of the system going. Well-balanced system development also frequently includes domain experts, graphic designers, writers, editors and testers, not to mention the system admins and all of the support personnel (even occasionally lawyers and sales people). And one should never forget the necessity of having a good project manager around to coordinate and then pull it all together.
Technology and libraries have improved significantly since I started, but if anything, that has driven the expectation of the users towards bigger, more complex systems. Gone are the days of some guy hacking in a closet, then just installing it on a PC or two. If a computer system is really going to solve real problems (and not just add to them), it needs to integrate with the existing environment in the most seamless way possible. Dozens of little vertically-isolated systems are just adding to the problem.
This leads us to the most significant aspect of comparisons to construction: scale. Most people can build a toolbox out of wood, many can even build sheds for their backyard. These are great skills but they don’t easily transcend to larger projects such as building a house.
And being an expert in house building, doesn’t necessarily qualify someone to build an apartment building, or even to play in the big leagues with a skyscraper. They’re all construction, but each requires its own set of skills and experience. As the projects get larger, the underlying issues change, and the way people have to work together changes as well.
Failure to see this in programming has lead to a great number of colossal disasters. I’ve been on, or followed several great software projects lead by backyard enthusiasts or house-building experts who have leaped haphazardly over to trying to whip out skyscrapers with the same tools and style. Building a nice little app for a PC is an accomplishment, but because of scale, it is easier to get away with bad practices that would cripple the development of a major, integrated system. As the projects grow, the complexities grow exponentially and simple problems that would be barely noticed in a small project open up huge fatal cracks in a massive one. You might just wing it for a small shed -- fixing the problems as you go -- but that would be an insane approach to building a skyscraper.
Scale is one of those classic issues that many inexperienced programmers dismiss too quickly. I once saw a multi-million dollar system fail spectacularly, not because it wouldn’t run, but primarily because its performance was so poor that it could only handle a fraction of the number of users that would make it profitable. Just existing was not enough to claim success. That was a classic example of experienced house-builders mis-gauging the differences in size, technology and project organization. They built a pretty working system, but they missed the real criteria for success.
Construction isn’t just about digging ditches or spending all day hammering or even driving around big machines. It’s the act of manipulating materials in complex ways that breath life into something tangible. It isn’t that far removed from software. Despite all of the myths, we ultimately deliver something physical, even if we can’t see the bits on the disk, or in memory. Not unlike buildings, the users move into their software, and eventually help to change it, or morph it into an environment that is more suitable to their actions. And a poorly built software system is as unpleasant as a badly designed building. It may function, but nobody likes spending time there.
Building is building; it's the medium that changes. It doesn’t really matter if the work is more intellectually oriented or more physically oriented, the underlying issues are the same. Software may be seen as a more malleable resource, and it may seem like we have more options, but we’ve certainly learned that once a large code base comes into existence it is no easier to change than a tower of concrete and steel. I think its time for us to to look outside of our own myopic culture and start learning for the experiences of others. Construction has a long and mostly successful history, we do not. We can learn a lot from how they get their mightily works accomplished.
Construction sites aren’t just exclusively general laborers, there are usually a host of other complex jobs and professions. Skyscrapers, for instance, require architects, drafters, engineers, inspectors, project managers, electricians, plumbers and a host of specialty jobs like elevator installation or crane operation.
The same is true for medium or large software projects, you also get a large congregation of divergent professions and experience. Learning how all of these different perspectives and agendas can work together effectively is something urgently required by software projects. Too often software professionals get focused on the code -- the cement of the project -- only to miss the other necessary elements to achieve success. But a building isn’t functional until all of the other pieces are in place.
Even programmers tend to specialize in their work. It often takes GUI specialists, database admins, algorithmic gurus and system programmers just to get the core of the system going. Well-balanced system development also frequently includes domain experts, graphic designers, writers, editors and testers, not to mention the system admins and all of the support personnel (even occasionally lawyers and sales people). And one should never forget the necessity of having a good project manager around to coordinate and then pull it all together.
Technology and libraries have improved significantly since I started, but if anything, that has driven the expectation of the users towards bigger, more complex systems. Gone are the days of some guy hacking in a closet, then just installing it on a PC or two. If a computer system is really going to solve real problems (and not just add to them), it needs to integrate with the existing environment in the most seamless way possible. Dozens of little vertically-isolated systems are just adding to the problem.
This leads us to the most significant aspect of comparisons to construction: scale. Most people can build a toolbox out of wood, many can even build sheds for their backyard. These are great skills but they don’t easily transcend to larger projects such as building a house.
And being an expert in house building, doesn’t necessarily qualify someone to build an apartment building, or even to play in the big leagues with a skyscraper. They’re all construction, but each requires its own set of skills and experience. As the projects get larger, the underlying issues change, and the way people have to work together changes as well.
Failure to see this in programming has lead to a great number of colossal disasters. I’ve been on, or followed several great software projects lead by backyard enthusiasts or house-building experts who have leaped haphazardly over to trying to whip out skyscrapers with the same tools and style. Building a nice little app for a PC is an accomplishment, but because of scale, it is easier to get away with bad practices that would cripple the development of a major, integrated system. As the projects grow, the complexities grow exponentially and simple problems that would be barely noticed in a small project open up huge fatal cracks in a massive one. You might just wing it for a small shed -- fixing the problems as you go -- but that would be an insane approach to building a skyscraper.
Scale is one of those classic issues that many inexperienced programmers dismiss too quickly. I once saw a multi-million dollar system fail spectacularly, not because it wouldn’t run, but primarily because its performance was so poor that it could only handle a fraction of the number of users that would make it profitable. Just existing was not enough to claim success. That was a classic example of experienced house-builders mis-gauging the differences in size, technology and project organization. They built a pretty working system, but they missed the real criteria for success.
Construction isn’t just about digging ditches or spending all day hammering or even driving around big machines. It’s the act of manipulating materials in complex ways that breath life into something tangible. It isn’t that far removed from software. Despite all of the myths, we ultimately deliver something physical, even if we can’t see the bits on the disk, or in memory. Not unlike buildings, the users move into their software, and eventually help to change it, or morph it into an environment that is more suitable to their actions. And a poorly built software system is as unpleasant as a badly designed building. It may function, but nobody likes spending time there.
Building is building; it's the medium that changes. It doesn’t really matter if the work is more intellectually oriented or more physically oriented, the underlying issues are the same. Software may be seen as a more malleable resource, and it may seem like we have more options, but we’ve certainly learned that once a large code base comes into existence it is no easier to change than a tower of concrete and steel. I think its time for us to to look outside of our own myopic culture and start learning for the experiences of others. Construction has a long and mostly successful history, we do not. We can learn a lot from how they get their mightily works accomplished.
Subscribe to:
Posts (Atom)