I'm standing on the other side of what I think is some type of huge wall. A great wall that stretches for I don't know how far and is higher than I can see over or even climb. It is an obstacle through which I cannot pass.
We are drawn in by the simplest of things. You for instance, probably started reading this post because of its title "4 Easy Ways to Design Better Software". Your probably curious as to what simple and easy things you can do to design better systems, and this seems like a great place to find that information quickly. A quick simple solution is what you really want. Having to think about it, particularly thinking hard about it, will not do.
And so, software developers, programmers and other techies are drawn to the many fads that make up our daily fare of literary pulp for technical articles. 12 ways to do that; 5 ways for this; the best 20 of these. Most are seeking the easy answers, not understanding.
The things you didn't stop to read had big ugly titles, or maybe delved too deeply into the underlying makeup of the issues. Perhaps you bookmarked them into an optimistic 'readme' category or something similar, but still their ultimate fate, we can predict, is to languish in the dark corners of nowhere.
Ironic, because buried deep in all of that uninteresting stuff there lies at least four easy ways to design software, if not many more. If only you looked there, you might be able to find the real understanding that you are craving. Then again, there is always YouTube; have you seen the latest lolcats video?
I'm a horribly uncomfortable writer that feels compelled to continue churning out unreadable and uninteresting opinions because it turns out that after many years of searching, understanding and contemplating software, those answers that I was seeking were really close to home and I need to share. But, sadly what I am saying is bound to keep me on the other side of that looming wall of obscurity. The side that nobody really wants to visit because it is not easy or fun, and it requires too much thinking, which is clearly a drag.
I figure if I keep chucking things over the wall, someday, someone is going to come around and understand it. At some point, they may even forgive me for my pitiful attempts at self-promotion. It can be hard if you think there is value in what you are saying, to see it get ignored by the masses. Lost forever in a chorus of 95 million voices.
Ironically, in the end it is all really simple: the 4 easy ways to design better software are to stop looking for 4 easy ways to design better software. Well, almost. I still owe you three, but at least now -- maybe -- you'll be able to find them yourself if you want to.
Friday, August 31, 2007
Wednesday, August 29, 2007
Is Open Source hurting Computer Science?
Reality is bursting with counterintuitive examples. Thinking deeply on this statement shows that it makes perfect sense: we are still fairly limited intellectually, so we are drawn towards the conclusions we like, rather than the ones that are actually correct. Our intuition sometimes leads us astray.
You'd think that a whole lot of free stuff would be a good thing, wouldn't you? A movement, such as the Open Source one, of people willing to donate their time and effort should be a noble cause.
Not that the core of Open Source is really about free stuff. It is not. It is about having access to the underlying computer source code for all of your software dependencies. At least that was how it all started.
Poorly documented proprietary solutions that required Herculean effort to understand were driving people nuts. These early systems were often rushed to market, neglecting documentation as a consequence. There were few resources available for finding information, other than trial and error. Interfacing with another system was generally a slow and painful process.
Open Source came about as a reaction to this common and frustrating problem. But it was an unpopular reaction.
If you are building on someone else's underlying code, then having the source as a reference can make diagnosing problems easier. If of course, that source is readable, elegant and short, which is so rarely the case anymore. Still, just the ability to peek into the source provides a measure of comfort.
On the other side of the fence, if you make your source code visible to other programmers, it turns out that it makes it rather easy for them to create similar but legal copies. They don't need to fully understand the design to come up with a related one. If they get the general idea that is often enough. Not that they couldn't eventually figure it out, it is just that you might save them lots of time and millions of dollars in resources. Millions that you already had to pay.
The side effect of all this openness is to make it extremely risky to release an Open Source application. Once out there, your competitors have as much understanding about the system as you do. So risky, given the millions needed to write a commercial product, that the only people initially willing to write Open Source code were ones that could afford to give it away for free. The students and the academics. More poignantly they were going to give it away for free anyways, as they always had before the Open Source movement even started.
Along the way, this caused 'free' to became synonymous with Open Source, which started things rolling at an ever increasing rate.
Early on, most companies hated the movement. Open Source tools were commonly banned in IT departments for years. There is still some hesitation, based around support issues, but mostly that has changed.
Once it got larger, more and more people were willing to try to capitalize on Open Source.
Two things emerge from this type of momentum: the first is the tendency for Open Source to consume all of the low hanging fruit; the smaller coding challenges. The second, is for it to target popular commercial offerings, trying to replicate their success. Both are serious problems.
With so many programmers willing to spend their evenings and weekends writing small bits of code for free, there are less and less places for companies to get launched. The small libraries and applications are all taken. This means that there are less companies coming into the market, so there is less competition, and subsequently there is worse code. Quality becomes optional.
There is also less incentive for big companies to release new products as well. You can't come into the market if the smallest thing you can profitably write costs millions to build. It is difficult even if you are large and confident.
What about the existing Open Source code? Turning a free piece of code into a proprietary one creates enemies fast. You can't directly monetize the code once it has gone Open Source. A sudden conversion towards profit will quickly spawn rival projects that will copy the ideas and give them away for free. You'll lose, in the same way you'll lose if Microsoft decides to write a version of your application. In fact, that pressure from both sides keeps the market empty.
Software is always a huge risk with a lot of money. Eliminating the smaller projects just ups the ante.
Since Open Source often holds few corporate ties, it could conceivably be a means for great research. There is freedom to truly experiment, and to build things that are beyond the bleeding edge.
Oddly, the behavior is reversed, as most Open Source projects chase commercial ones in vein attempts to give away the commercial functionality for free. No doubt it is easier to redo something similar, then it is to create a brand new paradigm. People take the easy route. The less risky one. Even though they are not writing for commercial purposes, the programmers still want fame and ultimately, riches. The game may change, but the motivations stay the same.
There are a few innovative Open Source projects, but the big ones have all grown successful by displacing earlier commercial offerings. Some have lead the market, but many are just playing catchup. Far too often, the Open Source programmers try to do too much on their own, so for example the commercial products utilize experienced graphic designers and technical writers, while the Open Source versions just wing it. They may be close technically, but that is only part of the battle.
Getting back to the question in the title: do I think Open Source is hurting Computer Science? My answer would be 'yes'. Primarily because it is stifling innovation and keeping the smaller commercial players from entering into the market. Combined, these two issues have caused a huge glut of code, but mostly of the low quality knock-offs variety. Innovations have been rare. This has driven software to become a commodity, one that directly has little or no value. Companies can no longer rely on software revenues, instead they are driven by service contracts or advertising. Being almost worthless, means little incentive to explore, push the bounds or even fix the known bugs for existing products. Effort always follows the money.
While it is nice that I can download so much functionality onto my machine for free, so much of it is unstable that it diminishes its own utility. It becomes something to collect for the sake of collecting it, but not to depend on when you need to get something done. And it has become a sore point that I end up downloading so many bad offerings, in the hopes of finding some new usable tools. A lot of time is wasted on near-misses. It's almost worth paying someone to separate out the usable technologies.
We do seem to be getting it from both sides. The quality of commercial products have dropped because of a decrease in the value of software. The Open Source code is quickly assembled by part-timers seeking fame, not quality. We end up living in a world of low-quality software. Plentiful, but undependable. One probably caused by too much 'free' work.
If I could change anything, I would push for the Open Source programmers to stop trying to copy their commercial counterparts. They have an opportunity to leap frog the existing technologies, so to squander that on trying to produce replicas of already broken interfaces just seems cruel. Computer Science has stagnated. All we get are really old ideas, from decades ago, recycled in new shiny wrapping paper.
It has been quite a while since someone, anyone actually, has produced a really new innovative idea. Sure, there are a few cases now where they've managed better implementations, but the underlying ideas were there all along.
We've reached the saturation point with our current technological factoring. The same as in the past, the pieces that we have created will bear no further weight on them. The whole becomes too complex and too unstable to manage. As always, at this point, some brave souls need to head back to the basics and find yet another factoring, but one that supports a broader upper range. Instead of producing what are effectively brand-name knock-offs, the Open Source programmers have a chance to contribute to the industry and push it to the next level. It might help redeem some of the damage that they have caused.
You'd think that a whole lot of free stuff would be a good thing, wouldn't you? A movement, such as the Open Source one, of people willing to donate their time and effort should be a noble cause.
Not that the core of Open Source is really about free stuff. It is not. It is about having access to the underlying computer source code for all of your software dependencies. At least that was how it all started.
Poorly documented proprietary solutions that required Herculean effort to understand were driving people nuts. These early systems were often rushed to market, neglecting documentation as a consequence. There were few resources available for finding information, other than trial and error. Interfacing with another system was generally a slow and painful process.
Open Source came about as a reaction to this common and frustrating problem. But it was an unpopular reaction.
If you are building on someone else's underlying code, then having the source as a reference can make diagnosing problems easier. If of course, that source is readable, elegant and short, which is so rarely the case anymore. Still, just the ability to peek into the source provides a measure of comfort.
On the other side of the fence, if you make your source code visible to other programmers, it turns out that it makes it rather easy for them to create similar but legal copies. They don't need to fully understand the design to come up with a related one. If they get the general idea that is often enough. Not that they couldn't eventually figure it out, it is just that you might save them lots of time and millions of dollars in resources. Millions that you already had to pay.
The side effect of all this openness is to make it extremely risky to release an Open Source application. Once out there, your competitors have as much understanding about the system as you do. So risky, given the millions needed to write a commercial product, that the only people initially willing to write Open Source code were ones that could afford to give it away for free. The students and the academics. More poignantly they were going to give it away for free anyways, as they always had before the Open Source movement even started.
Along the way, this caused 'free' to became synonymous with Open Source, which started things rolling at an ever increasing rate.
Early on, most companies hated the movement. Open Source tools were commonly banned in IT departments for years. There is still some hesitation, based around support issues, but mostly that has changed.
Once it got larger, more and more people were willing to try to capitalize on Open Source.
Two things emerge from this type of momentum: the first is the tendency for Open Source to consume all of the low hanging fruit; the smaller coding challenges. The second, is for it to target popular commercial offerings, trying to replicate their success. Both are serious problems.
With so many programmers willing to spend their evenings and weekends writing small bits of code for free, there are less and less places for companies to get launched. The small libraries and applications are all taken. This means that there are less companies coming into the market, so there is less competition, and subsequently there is worse code. Quality becomes optional.
There is also less incentive for big companies to release new products as well. You can't come into the market if the smallest thing you can profitably write costs millions to build. It is difficult even if you are large and confident.
What about the existing Open Source code? Turning a free piece of code into a proprietary one creates enemies fast. You can't directly monetize the code once it has gone Open Source. A sudden conversion towards profit will quickly spawn rival projects that will copy the ideas and give them away for free. You'll lose, in the same way you'll lose if Microsoft decides to write a version of your application. In fact, that pressure from both sides keeps the market empty.
Software is always a huge risk with a lot of money. Eliminating the smaller projects just ups the ante.
Since Open Source often holds few corporate ties, it could conceivably be a means for great research. There is freedom to truly experiment, and to build things that are beyond the bleeding edge.
Oddly, the behavior is reversed, as most Open Source projects chase commercial ones in vein attempts to give away the commercial functionality for free. No doubt it is easier to redo something similar, then it is to create a brand new paradigm. People take the easy route. The less risky one. Even though they are not writing for commercial purposes, the programmers still want fame and ultimately, riches. The game may change, but the motivations stay the same.
There are a few innovative Open Source projects, but the big ones have all grown successful by displacing earlier commercial offerings. Some have lead the market, but many are just playing catchup. Far too often, the Open Source programmers try to do too much on their own, so for example the commercial products utilize experienced graphic designers and technical writers, while the Open Source versions just wing it. They may be close technically, but that is only part of the battle.
Getting back to the question in the title: do I think Open Source is hurting Computer Science? My answer would be 'yes'. Primarily because it is stifling innovation and keeping the smaller commercial players from entering into the market. Combined, these two issues have caused a huge glut of code, but mostly of the low quality knock-offs variety. Innovations have been rare. This has driven software to become a commodity, one that directly has little or no value. Companies can no longer rely on software revenues, instead they are driven by service contracts or advertising. Being almost worthless, means little incentive to explore, push the bounds or even fix the known bugs for existing products. Effort always follows the money.
While it is nice that I can download so much functionality onto my machine for free, so much of it is unstable that it diminishes its own utility. It becomes something to collect for the sake of collecting it, but not to depend on when you need to get something done. And it has become a sore point that I end up downloading so many bad offerings, in the hopes of finding some new usable tools. A lot of time is wasted on near-misses. It's almost worth paying someone to separate out the usable technologies.
We do seem to be getting it from both sides. The quality of commercial products have dropped because of a decrease in the value of software. The Open Source code is quickly assembled by part-timers seeking fame, not quality. We end up living in a world of low-quality software. Plentiful, but undependable. One probably caused by too much 'free' work.
If I could change anything, I would push for the Open Source programmers to stop trying to copy their commercial counterparts. They have an opportunity to leap frog the existing technologies, so to squander that on trying to produce replicas of already broken interfaces just seems cruel. Computer Science has stagnated. All we get are really old ideas, from decades ago, recycled in new shiny wrapping paper.
It has been quite a while since someone, anyone actually, has produced a really new innovative idea. Sure, there are a few cases now where they've managed better implementations, but the underlying ideas were there all along.
We've reached the saturation point with our current technological factoring. The same as in the past, the pieces that we have created will bear no further weight on them. The whole becomes too complex and too unstable to manage. As always, at this point, some brave souls need to head back to the basics and find yet another factoring, but one that supports a broader upper range. Instead of producing what are effectively brand-name knock-offs, the Open Source programmers have a chance to contribute to the industry and push it to the next level. It might help redeem some of the damage that they have caused.
Wednesday, August 22, 2007
Few Technological Conveniences
The photocopies are long overdue. There are several other more important things on my agenda for the day, but I've delayed doing them for so long I have no choice.
Wearily, I rush upstairs to the photocopier and insert my clump of papers. I enter a security code and a few other options causing the machine to spring to life, sucking paper in at one end and spiting it out in several places at the other.
But then -- as one always knows will eventually happen with photocopying -- the sounds of activity cease. It all comes clanking to a halt, my work being left undone.
The lights blink ... then there is a message on the little console screen: please change the paper in tray 2.
Ok. No problem, I think. I can handle this.
As I look down, to my horror I see that there is no tray 2. There never was. For some insane reason the machine has gone on a bender half way through my work and is now pining for something that never existed in the first place. Foog! Stupid technology. Fooled again.
But hey, I'm a software guy, this happens to me all the time. Maybe not with photocopies, but certain with at least 70% of the applications I use on a day to day basis. They suddenly choke up for no good reason.
Further investigation only shows that I should not have left my bed this morning. Some engineers, somewhere, decided for some reason that they don't have any empathy for their users, which I happen to be right now. They decided that they don't want to spend that extra effort to clean up the mess. They decided that it really ain't that bad. Or they even decided that they like the way it works, it speaks of their character.
The truth is, I don't much care. And if I could send them some bad karma, well, they should just be careful and not move around the house too much today.
So much crap, so little time to expose it.
At least with physical things like photocopiers, most people can immediately identify the crappiness of it, so it gradually gets fixed over time. Not so with software, where it may have been good once, but sooner or later some fun-starved geek is going to turn it into an uber-complexity nightmare. Even if it works today, it may be wrecked tomorrow.
Bad technology is so easy to build. Just sprinkle in a little bit of:
Nothing is more fun then changing some small seemingly insignificant parameter only to see that the whole application blows up spectacularly. Even better: have it blow up a few days later so it is even more difficult to trace back the crashing to the changing of the parameter. That kind of "well-designed" software is just such a joy to play with.
I could go on and on forever I think, as there are way too many good examples of bad technology out there. Worse though, is that people are becoming accustomed to the crappiness. They look at me like I am mad for just wanting something to work consistently or correctly. There was a time when much of what we use now would have been rated with an F-, but now it has become standard fare. The coders probably aren't even trying to fix it, are they?
I'm tired of complaining, but I keep getting sucked into this stuff. It is particularly awful with the stuff floating around the IDEs that people are using. "Use the new module X to solve all of your growing needs" they proudly exclaim while you watch another two to three of your hours get flushed away on another pitiful excuse for technology. I keep falling for this, I am such a sucker.
I get caught, I guess, because I am still looking for tools that work. And I am still being disappointed.
These days I can definitely say that the depth and frequency of my disappointment is increasing. If we have progressed, we have done so by learning how to churn out more crap at a faster rate and by convincing ourselves that it ain't so bad. Yes, thats it, progress from being able to tie the blindfold around our eyes that much faster. That's a form of accomplishment, isn't it?
So I kick the photocopier, then I open up all of the trays and close them. After that I turn it off, then on, then off again. I spin three times, kick the trays again (for good luck) hit the on switch and enter my security code. I hit go and it springs back into life, without any sense of why it had previously failed.
Whew, I exclaim, to nobody in particular. I dodged a big one. It could have been weeks before that machine worked again. If ever.
Wearily, I rush upstairs to the photocopier and insert my clump of papers. I enter a security code and a few other options causing the machine to spring to life, sucking paper in at one end and spiting it out in several places at the other.
But then -- as one always knows will eventually happen with photocopying -- the sounds of activity cease. It all comes clanking to a halt, my work being left undone.
The lights blink ... then there is a message on the little console screen: please change the paper in tray 2.
Ok. No problem, I think. I can handle this.
As I look down, to my horror I see that there is no tray 2. There never was. For some insane reason the machine has gone on a bender half way through my work and is now pining for something that never existed in the first place. Foog! Stupid technology. Fooled again.
But hey, I'm a software guy, this happens to me all the time. Maybe not with photocopies, but certain with at least 70% of the applications I use on a day to day basis. They suddenly choke up for no good reason.
Further investigation only shows that I should not have left my bed this morning. Some engineers, somewhere, decided for some reason that they don't have any empathy for their users, which I happen to be right now. They decided that they don't want to spend that extra effort to clean up the mess. They decided that it really ain't that bad. Or they even decided that they like the way it works, it speaks of their character.
The truth is, I don't much care. And if I could send them some bad karma, well, they should just be careful and not move around the house too much today.
So much crap, so little time to expose it.
At least with physical things like photocopiers, most people can immediately identify the crappiness of it, so it gradually gets fixed over time. Not so with software, where it may have been good once, but sooner or later some fun-starved geek is going to turn it into an uber-complexity nightmare. Even if it works today, it may be wrecked tomorrow.
Bad technology is so easy to build. Just sprinkle in a little bit of:
- Fiddly bits that you can play with, but shouldn't or things will break.
- Weird unintuitive abstractions that are unrelated to the problem.
- Inconsistency to show that the coder is a creative sort.
- Sensitivity to small changes, causing big random effects.
- Non-documentation and torture-torials, preferably with minimal editing.
- One-size-fits-all errors that are not traceable back to the original input.
Nothing is more fun then changing some small seemingly insignificant parameter only to see that the whole application blows up spectacularly. Even better: have it blow up a few days later so it is even more difficult to trace back the crashing to the changing of the parameter. That kind of "well-designed" software is just such a joy to play with.
I could go on and on forever I think, as there are way too many good examples of bad technology out there. Worse though, is that people are becoming accustomed to the crappiness. They look at me like I am mad for just wanting something to work consistently or correctly. There was a time when much of what we use now would have been rated with an F-, but now it has become standard fare. The coders probably aren't even trying to fix it, are they?
I'm tired of complaining, but I keep getting sucked into this stuff. It is particularly awful with the stuff floating around the IDEs that people are using. "Use the new module X to solve all of your growing needs" they proudly exclaim while you watch another two to three of your hours get flushed away on another pitiful excuse for technology. I keep falling for this, I am such a sucker.
I get caught, I guess, because I am still looking for tools that work. And I am still being disappointed.
These days I can definitely say that the depth and frequency of my disappointment is increasing. If we have progressed, we have done so by learning how to churn out more crap at a faster rate and by convincing ourselves that it ain't so bad. Yes, thats it, progress from being able to tie the blindfold around our eyes that much faster. That's a form of accomplishment, isn't it?
So I kick the photocopier, then I open up all of the trays and close them. After that I turn it off, then on, then off again. I spin three times, kick the trays again (for good luck) hit the on switch and enter my security code. I hit go and it springs back into life, without any sense of why it had previously failed.
Whew, I exclaim, to nobody in particular. I dodged a big one. It could have been weeks before that machine worked again. If ever.
Subscribe to:
Posts (Atom)