Top-down and Bottom-up

When I was younger, there was a lot of debate about how to design and build software. The prevailing theories were that you either started at the top and worked your way into the details, or that you started at the bottom and built up the pieces.

Software systems are tools used by people, so it is important to understand what their problems are and what is necessary to solve them. The only way to do this, is to see it from their perspective. If you understand what the user is trying to accomplish and how they are doing that, you can find the best design that simplifies their lives.

Functionality should not be arbitrary buried in dis-organized menus, it needs to be at their finger-tips, right when they need it, and out of the way the rest of the time.

Design then, only comes together if you’re looking down at the problem. You have to start at the 10,000 foot view and then wind your way through all of the steps necessary for someone to complete their work. Empathy for the users, and a deep understanding of both their environment and their goals is key to creating the tools that actually make their lives better.

But, from the other perspective, software is extremely slow and expensive to write. As the user expectations have increased, even the small systems of today are significantly larger then just a few decades ago. And the more we depend on underlying libraries, the faster the complexity increases. Even if we don’t have to write all of the code, each underlying dependency brings with it a unique set of problems that requires time to understand it and time to manage it properly. Declining standards, poor design, forgotten knowledge and sloppy release procedures in these external pieces don’t help.

Our only defense against spiraling complexity is to try isolate the work as much as possible. That is, highly redundant code that haphazardly calls the underlying components, anywhere, is the type of spaghetti that quickly burns through all of the development resources by wasting time chasing sloppy mistakes. A well-architect-ed system that correctly encapsulates all of its underlying parts allows for the scope of change to be controlled. Once a problem has been settled, it should no longer crop back up in arbitrary places. Each piece must fully encapsulates a specific section of the system. This is the only way to provide a solid foundation on which bigger and better functionality can be added without fear of setting off a chain reaction of cascading bugs.

This type of design can only be achieved by building upwards. Starting with the depths of the system, not unlike a real building, each layer is carefully designed, built and stacked. The lines between the layers need to be well-understood and carefully mapped out. This type of bottom-up construction insures that the whole does not become unstable as it gets pushed and extended. It can also insures that the momentum of the project doesn’t grind to halt because of exponential explosions in complexity. It requires a bit more effort and foresight, but it is the only way to build large systems that are dependable and can continue to grow as the needs of the users increase.

So, is it top-down or bottom-up? Really it is both. The design and requirements need to be seen from the top, but the construction needs to focus on building up solid and reliable pieces from the bottom. Foresight and experience are necessary to understand the scale of the system, but projects still need to be adaptable to changes in the environment, technologies or the user’s priorities. A solid foundation is necessary to insure that the system will survive for its maximum lifespan while continuing to grow.

Although our industry pretends to turn over every five years, by now it is clear that the life expectancy for big systems is ten to twenty years. And for some systems, they may be around far longer than that. With that understanding of our history, it seems even more reckless to hack together something without a design, apply duct-tape excessively as leaks appear, or just hope to catch the flaws in testing. Good software development demands a longer-term focus as well as seeing the whole process from both the ground and the aerial view.