Problems and Solutions

The world is littered with a growing number of problems. Some of these problems are solvable by using a computer, some not. To solve a problem with a computer all it takes is to design and construct a software solution, then get people to use it. Easy?

Solving a problem with software is not nearly as simple as it looks. Many more solutions fail then succeed. The stats are ugly: 66% of all software projects fail, as do 90% of all startups (most of which are founded around software solutions). Why are these numbers so high?

I think the root of the problem is that a good software solution looks simple. It’s very easy to miss the depth of both the thinking and the work that have gone into making it possible. This leads to a lot of people casually tossing around ideas for solutions, with the expectation that they’ll actually solve a problem. But few of these ideas actually do, and in that very limited remaining subset, the execution and the details mean everything.

To actually solve somebody’s problem with software, the developers needs to look long and deeply at the underlying problem. They need to understand it, not only from the 10,000 foot view, but also right down to each of the tiny details. A shallow understanding of the problem is not nearly enough to solve it. The real issues are often counter-intuitive, hidden and not easily grasped.

Even more complex is understanding the nature of how people use technology. Without some type of incentive, many ‘features’ of a system are useless. Making software ‘sticky’ is a rather complex combination of various issues including stability, usability, psychology, feedback and empathy. Again a broad perspective is not enough to understand these depths.

As well, the developers need to understand the uses and limits of modern technology. Software is still fairly crude and marketing claims for it rarely match reality. Technological issues are often deeply rooted in their theory and history, so both of these need to be well understood before you can properly assess the limitations. You really need to understand these limits before you can apply the technology to solve a problem.

Running a system development project is no picnic either. Software development is like a large train moving slowly down the track, from station to station. You can’t just throw in a right turn here or there. You can’t really change its direction until you hit a station, and even then the number of other accessible stations is very limited. It is easy to miss these inherent and often dangerous properties.

Right down at the core, software is about programming. But just being able to program is not enough either. Some code is good, while some is just a disaster waiting to happen. Experience teaches one more about the code they shouldn’t write, then about what they should. Knowing what’s disorganized, or fragile, or outright dangerous isn’t well documented and sadly only comes with a tremendous amount of experience. Hacking one’s way through a problem space won’t work; won’t produce enough of a stable, coherent solution. Just a convoluted mess. Really understanding what that means in practice takes a lot of time and a lot of experience. It’s not something that people can just read about, or take a course in, or figure out on the fly.

So why is the rate of failure so high? Easy, most of the people out there proposing solutions don’t have a deep enough understanding of what works and what doesn’t, to be able to propose valid solutions. Most programmers are viewed as just ‘code monkeys’ whose task is to build someone else’s solution. So the success or failure of their labours depends heavily on the quality of the proposed solution. A crappy solution will either fail outright, or limp along for years. A good solution can be ruined by a lot of crappy sub-solutions, slowly mutating into something horrific.

It’s for this reason that I often distinguish between programmers and software developers. A programmer can code, but a software developer knows how to solve real problems correctly with software. There is a huge difference. It’s also for this reason that I’ve never liked the term ‘stakeholder’. It seems to imply that they have the right to choose the solution because they have a ‘stake’ in the outcome. But if they don’t have enough of a background to pick a valid solution, then the work will fail. I’ve also seen a great number of entrepreneurs out there with the belief that as business people they should be the ones to pick the solution. That somehow a business background is enough for them to fully grok the problem, the people and the technical issues. This is probably why most startups produce software that doesn’t actually solve any real problems. It just looks shiny and new. So they fail.

The takeaway from all of this is that if you want to create a software solution to solve people’s problems, you need to give the task to someone with experience in creating working solutions with software. An experienced software developer who understands the history, limits and possibilities of software. If you leave it to someone with no experience, the solution is unlikely to be valid.