Lessons Learned

You learn a lot during thirty years. I tried to write about most of it in this blog, at least from a higher level perspective bringing lots of different things that have happened together, but some things are smaller and just don’t fit. Each one of these is rooted in at least one epic failure.

• Doing the screens first and persistence last is a common top-down development approach, but it is a very bad mistake. The screens are driven by the irrationality of the users, they don’t map cleanly to the demands of persistence, and they never will. If they could, then lots of the very early application generators would have worked, but they didn’t. Persist first, then gradually move it up until it gets into the screens.
• If you have an RDBMS, use it to its nearly fullest ability to protect itself. You really don’t want to persist garbage data, that will cause all sorts of annoying bugs. You don’t want to double up stuff you are persisting too, it is wasting space and can cause stale or inconsistent data as well. People always try to cheat the database work, and they always pay a high price for it. It isn’t particularly fun work, but it anchors everything else.
• Don’t try to break dependent things into subparts, like say put the front and back ends for the same system into two different repos. People might decompose by language, for example, but really if there is dependency, like an API, that matters more. It’s hard to explain, but if things can’t stand on their own, then you shouldn’t try to force them to.
• Disorganization will always be the biggest problem. Organization is a place for everything, everything in its place, and not too many similar things in the same place. That is, if you have something new and you don’t know where to put it, then it is disorganized. It must go somewhere; if that is obvious, then you are okay.
• If the programmers don’t know what the system holds for data, and they don’t know why people are doing things with that data, then it will have a huge number of bugs. Programmers are the frontline for quality, if they can’t see problems as they work, there will be lots and lots more.
• Always only every move code in one direction. It goes from Dev to Release, with a few QA stops along the way. Never, never, break that chain. It will result in all sorts of problems including things getting accidentally rolled back, which is avoidable.
• Always clean up right after a release. Everyone is tired, and cleanup work is boring. If you do not clean up then, you will never clean up and the mess will get worse, far worse.
• Tackle the hard parts first, not the easy ones. The hard ones are unpredictable in time, if they don’t go well you can raise an early flag on the schedule. The other way around tends to mislead people into thinking that everything is going well when it isn’t.
• Do the right thing when you start. Only take more shortcuts the closer you are to the deadline. If you take a shortcut, note it, and clean it up right away after the release.
• Do not freeze code forever. If you freeze the code, you also free the bugs, which is then the foundation of everything else. Building on buggy foundations is problematic.
• Do not let people add in onion architectures. If they are trying to avoid the main code, and just do “their thing” around the outside, that work is usually very harmful. Push them to do the work properly.
• Don’t drink the Kool-Aid. There just isn’t an easy or right way to build stuff. The best you can do is make it readable and keep it organized. Most philosophies for coding are extreme and have worse side effects.
• If what happens underneath matters in the system, it is not fully encapsulated. In that case, you need to learn some stuff about what happens and why it happens. You can’t just ignore it. Some components will never be fully encapsulated.
• The ramp-up time for an experienced coder is proportional to the size of the codebase. The ramp-up time for an inexperienced coder is far longer.
• A weird and ugly interface is a strong disincentive against usage. Useful code has a long life. The point of writing professional code is to maximize its lifespan.
• Reduce friction, don’t tolerate it. Spending the time to mitigate or minimize it always pays off. Putting up with it always slides one downhill.

There is a lot more, but I’ll start with these. Writing code is fairly easy, but building huge reliable systems is exceptionally hard. The two are not the same.