Right there, in the center of all software development projects, is a set of rather odd mappings from our irrational physical world onto a very strict formal computational one.
These difficult mappings form the core of any system.
Get them right, and it is easy to package up all of the other aspects of the system around them. Get them wrong and their faults permeate everything else, making it all rather hopeless.
The act of designing and building these mappings is not intrinsically difficult by itself, but there are enough places for variations and trade-offs that it is way too easy to get it completely wrong.
It's not something that you learn overnight, nor is it easily explainable to others. There are no blog posts to read, nor textbooks to lay it out in any detail.
The best way to learn it is to work with someone else who understands it, then gradually build up enough experience that you can eventually strike out on your own. Getting it wrong a dozen times doesn't help as there are millions of possible mistakes you can make. It’s only when you see it being done well, that you are able to get some type of sense of how it all comes together.
You can tell when development projects are led by someone who gets it. The big and small parts all fit together nicely. It’s also pretty obvious when the project is adrift without a strong lead.
Since this is the blurry intersection between technology and the problem domain, you really have to have spent time on both sides of the fence in order to see how everything lines up. A strong business background tells you about the problem. A strong technical background tells you about the possibilities of the solution. But neither side on their own helps with working through this mapping. You have to connect these two sides together, carefully, but since it's odd, crazy, and irrational, then throughout the process you have to try to balance out any hard edges in order to find some type of reasonable working compromise.
The strength of any system is the quality of these mappings, which comes directly from the skill level of the people putting them together. It might just be one person, which tends to be more coherent, but if the timescales are too short, a group of people may have to come together to sort it out.
We have no known defined way of ‘documenting’ this mapping. It underpins all of the other design work, peeking through in artifacts here and there. While you can’t describe it, it is clearly visible when it is missing.
Since it binds all of the other work together, it is always a huge disruption when the original author leaves and is replaced. The next person might know how to follow suit, maybe slowly changing it, but this only happens if they can ‘see’ the original effort.
Two or more competing mappings will always clash destructively. They will rub each other in the wrong direction, forming one larger broken mapping. Bad choices will gradually undo any good ones.
A development project is over when the mapping is complete for the whole problem domain or when the mapping is so hopelessly broken that any effort to fix it exceeds the effort to start over again. In the first case, the system just needs continual ongoing work to move the code forward with the constant changes in dependencies. In the second case, any work going into the system is wasted. The skill level necessary to correct a bad mapping is off the charts.