Webapps

In the early days of the Web, people struggled to make websites more dynamic. The original web technologies were centred around static presentations; they were pretty good at this.

As the trends matured, more and more technologies became available to make the sites dynamic, but also to use them to essentially wrap other programs and systems. The web interface was born; people gave these the cool name of webapps.

In those days, most serious developers continued to produce native GUIs. The web technologies seemed hokey and crude. Many were just mindless fronts that called the real stuff in the back. Light clients calling APIs.

The problem with native apps was portability. There were more operating systems back then, and it was crazy expensive to rewrite the same program for each one. The technologies for writing stuff once and getting it to run everywhere were a great idea, but the implementations were generally too limited to be useful.

What the web promised, though, was a guaranteed way to avoid portability problems. These promises, however, were quickly disrupted by the browser wars and by the emergence of mobile devices.

That led to a huge wave of ‘frameworks’ that all promised portability across all of these different platforms and form factors. Initially, they helped; webapps grew a little more sophisticated. But somewhere along the way, they all got pretty convoluted. Suddenly crafting a reasonable webapp was a whole lot more effort than just crafting native apps. That bump in complexity was rewarded with a visible drop in quality. Webapps started to become extremely buggy.

The browsers themselves grew in sophistication, but their integration was compromised by huge security failures. The web became a free-for-all for scams, driving a lot of people into silos.

They are still just a limited window into some features, but most of the ways to extend them are too fiddly to be practical.

So we’re left with most interfaces starting on browsers, then getting mobile cousins, then maybe better native versions. Oddly ironic since, except for form factor, they're all a bunch of interactions on a whack load of widgets. The foundations for graphical user interfaces haven’t changed for decades, just after the client/server split. All of these modern technologies trace a close lineage to their earlier generations.

If we were going to rethink this, it would be to go way back to the portability days. We’d still like to write one set of code that covers all three locations, pushing each right to its limits. That is, you’d grab a webapp you like, run it natively, and it would save your files locally. If you run it on a phone, it would give you the option of local or hosted.

If you can wire in optional platform capabilities and some dynamic form factor support, then you really could return to a point where writing interfaces wasn’t the bulk of the development effort. If it was near trivial to dump out 80% of the boring screens in a few days, then you could spend more time deciding which widget arrangements were best suited for which tasks, rather than expensive widget/presentation wiring and refactoring.

Webapps suck. They almost didn’t, but then fate intervened. We don’t need more siloed clumps of monetizable half-baked features; we have enough already. We need better adaptive and integrated tools that allow us to spend less time on computers, not more. The answer to this is not probabilistic personalized interface generation; more opaque, crappy code will only make things worse. It is to rework our foundations and get back to some of the great ideas of the past that we skipped over too quickly.

Oversimplication

For a long time, the software industry turned against sophisticated power techniques like abstraction.

Partly because getting a little abstract is difficult for some people, but also because it slows down the number of lines they can grind. As an added bonus, it is a bit harder to debug.

People felt that rapidly churning out super simple, near-trivial, brute force code was the best approach.

Strangely, though, a great deal of the actual foundation of software rests on some pretty complex abstractions. Little of what is now possible with computers would be there or work properly if it didn’t. A lot of that code came from an age when people still cared about getting it correct. It’s not ‘legacy’, it’s ‘classic’.

As I watched these oversimplification trends take hold and gain strength, I was getting increasingly worried about the frailty of all of the software that we depend on. Everything is so buggy these days that it’s a wonder that anything works. A few decades back, people would dispute that claim and say it was crazy, but now it’s way worse and so obvious the tables have turned.

There are plenty of known problems out there where any and all attempts to brute-force some crude logic will fail. The scary part is that it may not ‘completely fail’. It may just look like it’s working, work some of the time, but then fail right in the middle of consequential moments.

Software has many essentially ‘physical’ boundaries. You can’t do anything to get past them; they are hard and fast limits. It’s also not particularly malleable, and certainly gets rapidly less malleable as the codebase grows. The crap at the bottom gets frozen there as more stuff gets piled on top. A digital form of gravity, I guess.

Any sort of ‘just jump in and pound it out’ philosophy is a recipe for failure. By the time you’ve clued into your most obvious mistakes, and there will be obvious mistakes, it is too late to go back and fix them. Then building on top of that shaky mess is doomed.

If the problem you are trying to solve is complicated, then any viable real solution to that problem is at least equally complicated. We’ll call this the equality rule.

Techniques like abstraction and generalization are the only means to take that unmanageable complexity and bring it down to a reasonable level where you can implement it correctly. All your other attempts to ‘hack through’ the issues are just a waste of time.

But some people are often so focused on trying to create the simplest solution possible that they end up ignoring the problem they are trying to solve. So, it’s disconnected. The solution is wonderful, but useless.

Instead, it’s way better to go grab a sheet of actual paper and start writing out any proposed solution first. Sketch some mechanics. Think of it as modelling clay, like what they use for car design. Then take that proposed solution and go back to properly explore the problem. If they don’t match up, go back to the design board. A sketch in time saves buckets of wasted coding. It’s orders of magnitude faster than coding up a bad solution, throwing it away, coding up another, throwing that away, ad nauseam.

Grungy coding habits like that are oddly okay for demos and quick prototypes, but it only works because the code is tiny. Scale screws with everything, and any code that is good enough will always grow. Maybe we should call that the destiny rule. If the code kinda works, then it is fated for continued expansion until it doesn’t work anymore.

The oddest part, though, is that the motivation to keep it simple is great. The thing should be as simple as possible; it’s just that any distance way over or way under that bar is exponentially worse. You need to hit the bar, and that bar is positioned by the equality rule.

If there is any sort of method to the madness, it is to start simple. But know immediately that it is too oversimplified. Bad design. Keep going back to the problem to figure out why it won’t work in specific situations. Put those complexities carefully back into the design. Iterate, over and over again.

If you think that might take too long, and you really don’t have decades to catch up with the state of the art, the shortcut is not to ignore the problem, but rather to jump out and do some reading on the state of the art. Leverage what other people have already figured out. Read a bit, learn a lot. Revise what you actually think simple really means. Know that if a kink in the problem space stumbles too close to a boundary, then find the best fit possible within your timeframe to actually address it, or at least carve out some space for it to be corrected later.

What you don’t want to do is hobble something together out of components you barely understand. While that keeps your code simple, what rests below explodes with overcomplexity.

It’s worth noting that the crazy balancing act necessary for modern software design is not something you can learn instantaneously. Because of this, the fastest way to gain knowledge is always mentoring. Get on a good project, work with good teams, and get someone who knows a bit to explain their rationale. Don’t be skeptical, at least not at first. Everyone is wrong, but some people are less wrong, thus closer to the knowledge you are seeking.

While it's tempting to believe that there is always a simple software solution for everything, it’s been proven incorrect so many times that it has become boring and forgotten, again and again. If you want to build software that really solves people’s problems for them, then that software has to really solve people’s problems for them. If their problems have become crazy complicated, then it’s obvious that anything simple will not even come close to fitting properly. I really should call ‘the equality rule’ a law or something...

Ordering

With complex work, the order in which you do it hugely affects the quality, speed, and accuracy.

If there are ten subtasks, you do not really have the option to do them in any order. You need to start with some tasks first, before you do the others. They are dependent.

That’s pretty obvious if you are building a house. You can’t just do the roof first if you don’t have the walls up. And you can’t get the walls up until you have completed the foundations.

The crazy part about software is that although it is exactly like that too, most people incorrectly believe that because it is “digital”, it is somehow different. Somehow it magically escapes any dependencies, so starting in any order is fine. It is not.

As a consequence, you get situations that are equivalent to the stakeholders getting obsessed over the paint colour for a penthouse, when the foundations for the apartment building haven’t even been finalized in design yet, totally out of order.

\While we would not expect non-technical people to understand or be concerned about ordering the work, it is critical for developers to do this correctly. Maybe the second most important thing after organization.

The stakeholders only see the outside of the top; the rest of the box is invisible to them. They always assume it is trivial.

But internally, software is closest to a house of cards. It is far from trivial, where everything rests on what is below it. That orientation sets its dependencies. An interface that calls a backend, for example, is way up there. The persistence that the backend uses is far below. With that in mind, you can assess the “height” of all components.

Some architectures may flip certain parts of the game, but there is still some sort of relative height there. Figure out those heights, and you get a quick rough sketch of the ordering.

The biggest notion is that everything is built on something else. Those times when a few independent components can be built in parallel, in isolation from each other, are rare anomalies. Take advantage when you find them, but don’t expect to find them and definitely don’t assume they exist. Go the other way: assume there are far more dependencies for anything than you are currently aware of, since that is most likely to be true.

Set a height, plan as best as you can, start at the bottom, then work upwards.

When you get it right, development will go smoothly along.

When you get it wrong, there will be lots of frustration and drama. Simple changes will become huge, oversized issues. The schedule will keep getting pushed back. Way too much stress, and the quality will bottom out, causing downstream operational problems which just keep it all boiling.

Not always, but frequently, if there is a lot of drama in a development project, a key ingredient is bad ordering. That is an issue that developers can and should control. It is one of the places where they are directly responsible for the outcomes.

It’s a grey zone, though, in that plenty of management tries to control it too. It is contested ground, mostly because of timelines.

What experience teaches is that the less you say about the tasks and their order, the more control you will have over them, which you need. It’s better to frame any project scheduling around outside events, not internal ones. That’s why along the way I learned to separate ‘features’ from ‘functionality’, where one is a set of things the user needs, while the other is the specific way each one is implemented. That disconnect is not there to over-complicate things, but instead to disconnect the requirements from the work in order to retain control over the ordering of the work.

When and where some non-technical management has stuck their fingers too deeply into the pie, because they don’t care about dependencies, their ordering choices have been disastrous. Erratic and prone to burning crazy amounts of time on wasted efforts that were started far too early. And although they are the problem, they keep complaining that it's developers that are too slow, which is often beyond frustrating.

Ordering matters. It matters a lot. Once you’ve worked on a well-ordered project, it's difficult to deal with a chaotic one again. The shortest path to getting the work done is oddly very strict. A few other paths are similar, but most others are far worse. Stay on a good path; keep people from distracting you.