I am a software developer.
I will not lie to anyone on my team.
I will value engineering far above process.
I will value completing work far above discourse or politics.
I will spend time with the end users and empathize with their problems.
I will take the time to really understand what I am coding.
I will try to keep things organized even in the face of intense time pressure.
I care if my code does not meet the expectations of a) the end users b) operations c) testers and d) myself.
I will take the extra time and effort necessary to ensure that the code works.
I will spend the time to investigate anything I don’t understand.
I will follow suit with any already existing code in our codebase and always try to leverage it first, before blindly reinventing it.
I will use all existing dependencies to their fullest extent before dumping in new ones.
I will not blindly copy code from answer sites and assume it works. Instead, I will read it, figure out what it does, and then learn.
I will communicate with everyone and make sure they are all on the same page. I will not hide any relevant details. I will stick to the facts and just the facts. I will write down what I know, even if the formatting is simple.
I will always try to explain things as simply and clearly as I can, and tailor those conversations to the audiences I am addressing.
I will not waste time or effort on make-work. Everything I do will be necessary to keep the project moving forward and to get the software out to the users.
I will stay out of any politics. I will avoid drama and childish behaviour.
I will not continue to work on things that I know are causing harm to others. I may not be able to stop it, but I will not participate.
It’s not about money. It’s not about power. It’s not about ambition. It’s not about ego. It is only about building things that are good enough to actually help people.
I am a professional software developer. I will act professionally at all times.
Sunday, August 14, 2022
Thursday, August 4, 2022
Architecture
I was reading an online discussion about architecture. It’s always been a rather odd topic in that it has actually been extremely well codified, at least a few times, but that knowledge always seems to get lost. So more often, people just define it as anything they want it to be, as long as it benefits their personal agenda.
Pretty much all that is wrong, but rather than point to a bunch of better, but old references, I’ll continue a long and painful tradition of throwing around my own definition.
A software system has an architecture if and only if it is organized. That is, if it is just a mess of stuff thrown together, then it has no architecture. It may have had an architecture in its early years, but if subsequent work ignored that, then it is just a pile of stuff now.
The process of establishing an architecture is part structural and part political. That is, for any non-trivial system, it is both so slow and so expensive to build that the act of arranging for the significant costs of the work to be covered is pure politics. All sorts of people have their fingers in this pie, so the technologies and stacks that will be picked as the foundations are just artifacts of the act of raising money. It’s true for startups, but it is also true for any large organization.
Each foundational dependency has an obvious set of strengths and weaknesses. You could rationally choose between them based on those properties to arrive at the best fitting pieces for any given solution. That almost never happens, politics drives people to choose irrationally, often based on prejudice, personality or limited experiences. So, generally the technologies and tech stacks are picked way in advance, long before the real work begins. Often they are poorly matched to the requirements. A lot of work goes into sloppy patches to cover over these weaknesses.
Once you get past the effects of politics, the rest is structural. There are two primary areas.
The first is decomposing related pieces together. That is, all of the code related to a behaviour of the system like reporting should be placed together. But it’s not actually that simple, in that there are always at least two dominant dimensions at play. The problem domain, too often called the business, imposes vertical constraints on the system, while the technical domain, as a foundational layer, imposes horizontal constraints. These are at odds with each other, although in modern times it has become more popular to just pay lip service to any large scale technical issues and blindly follow the business.
Still, you can organize the codebase, even if it is huge, into layers of nicely fitting boxes that clearly delineate the subcomponents at both a high and medium level. The strength of doing this early is that if the pieces are all independent from each other, the work can effectively be scaled (parallized) and the overall quality will be far better.
You can go further though. Instead of breaking down the mechanics by clumping together similar bits of code, you can organize it by fundamental composite data types. When this is correctly applied, it tends to mitigate the vertical and horizontal mismatches, but also simplifies both the code and any visualizations of it. You do see this happen in practice, but because it is a very advanced technique, it is more often applied to high quality low-level commercial products, than to the massive amounts of domain specific application code that is far more frequently written these days.
The second primary area is performance related. All systems have to fit into a larger environment. They share a lot of data, they are constantly getting data feeds and sending out data to other systems. At this level, there are standardized enterprise ways of interconnecting the different systems.
The core problem is usually frequency. One system may be able to generate or collect data far faster than another can import it, so there are common solutions like queues that get placed in between to prevent synchronization problems.
As there are literally 50 years of different data formats, communications tools and protocols and a nearly unlimited number of different right and wrong ways to model the data itself, so a great deal of time is spent on these data transfers. In some cases there are well established communications patterns like having a common data bus for the whole organization, but oddly, impatience more often means that these are ignored and each and every feed is badly home rolled. Often even if the internals of the system is tightly organized, its imports and exports are not, usually because the different teams responsible don’t like to agree on any standardized ways of moving data around. They think it takes too long to implement (while ignoring the fact that they are just reinventing the same wheels, over and over again).
There are a few other key architectural issues, like centralization, avoiding massive duplication, stale data and choosing between interface types like cli, native, web, mobile, etc. All of these have been heavily explored and resolved in the past, but again much of that knowledge has been forgotten too.
That’s pretty much architecture in a nutshell.
At least across parts of a large company, we could really be a lot more organized and save ourselves huge amounts of redundant work, but strangely we are unable to do that. It’s possible that the initial political component ends up overshadowing the technical issues, so that the initial loss of time spent there forces all of the following work to get horribly rushed, and then done badly. Not sure, but over the decades we’ve learned more than enough to be able to avoid this fate, yet it still seems to befall most large internal projects.
Our attempts at, instead, just trying to be fully reactive and then letting the work magically evolve on its own tend to court pure disorganization as the “architecture”, so they are usually far, far worse.
Either way we keep building things badly, then realizing they are bad, then restarting from scratch again. By now there are crappy big systems out there that are in their 7th or 8th generation. Maybe we should just do a better job of getting things organized first before we rush off to repeat the same mistakes again. Not making a mess on purpose is really the aim of architecture. You can tell how good people are at it, by the results that they leave behind.
Pretty much all that is wrong, but rather than point to a bunch of better, but old references, I’ll continue a long and painful tradition of throwing around my own definition.
A software system has an architecture if and only if it is organized. That is, if it is just a mess of stuff thrown together, then it has no architecture. It may have had an architecture in its early years, but if subsequent work ignored that, then it is just a pile of stuff now.
The process of establishing an architecture is part structural and part political. That is, for any non-trivial system, it is both so slow and so expensive to build that the act of arranging for the significant costs of the work to be covered is pure politics. All sorts of people have their fingers in this pie, so the technologies and stacks that will be picked as the foundations are just artifacts of the act of raising money. It’s true for startups, but it is also true for any large organization.
Each foundational dependency has an obvious set of strengths and weaknesses. You could rationally choose between them based on those properties to arrive at the best fitting pieces for any given solution. That almost never happens, politics drives people to choose irrationally, often based on prejudice, personality or limited experiences. So, generally the technologies and tech stacks are picked way in advance, long before the real work begins. Often they are poorly matched to the requirements. A lot of work goes into sloppy patches to cover over these weaknesses.
Once you get past the effects of politics, the rest is structural. There are two primary areas.
The first is decomposing related pieces together. That is, all of the code related to a behaviour of the system like reporting should be placed together. But it’s not actually that simple, in that there are always at least two dominant dimensions at play. The problem domain, too often called the business, imposes vertical constraints on the system, while the technical domain, as a foundational layer, imposes horizontal constraints. These are at odds with each other, although in modern times it has become more popular to just pay lip service to any large scale technical issues and blindly follow the business.
Still, you can organize the codebase, even if it is huge, into layers of nicely fitting boxes that clearly delineate the subcomponents at both a high and medium level. The strength of doing this early is that if the pieces are all independent from each other, the work can effectively be scaled (parallized) and the overall quality will be far better.
You can go further though. Instead of breaking down the mechanics by clumping together similar bits of code, you can organize it by fundamental composite data types. When this is correctly applied, it tends to mitigate the vertical and horizontal mismatches, but also simplifies both the code and any visualizations of it. You do see this happen in practice, but because it is a very advanced technique, it is more often applied to high quality low-level commercial products, than to the massive amounts of domain specific application code that is far more frequently written these days.
The second primary area is performance related. All systems have to fit into a larger environment. They share a lot of data, they are constantly getting data feeds and sending out data to other systems. At this level, there are standardized enterprise ways of interconnecting the different systems.
The core problem is usually frequency. One system may be able to generate or collect data far faster than another can import it, so there are common solutions like queues that get placed in between to prevent synchronization problems.
As there are literally 50 years of different data formats, communications tools and protocols and a nearly unlimited number of different right and wrong ways to model the data itself, so a great deal of time is spent on these data transfers. In some cases there are well established communications patterns like having a common data bus for the whole organization, but oddly, impatience more often means that these are ignored and each and every feed is badly home rolled. Often even if the internals of the system is tightly organized, its imports and exports are not, usually because the different teams responsible don’t like to agree on any standardized ways of moving data around. They think it takes too long to implement (while ignoring the fact that they are just reinventing the same wheels, over and over again).
There are a few other key architectural issues, like centralization, avoiding massive duplication, stale data and choosing between interface types like cli, native, web, mobile, etc. All of these have been heavily explored and resolved in the past, but again much of that knowledge has been forgotten too.
That’s pretty much architecture in a nutshell.
At least across parts of a large company, we could really be a lot more organized and save ourselves huge amounts of redundant work, but strangely we are unable to do that. It’s possible that the initial political component ends up overshadowing the technical issues, so that the initial loss of time spent there forces all of the following work to get horribly rushed, and then done badly. Not sure, but over the decades we’ve learned more than enough to be able to avoid this fate, yet it still seems to befall most large internal projects.
Our attempts at, instead, just trying to be fully reactive and then letting the work magically evolve on its own tend to court pure disorganization as the “architecture”, so they are usually far, far worse.
Either way we keep building things badly, then realizing they are bad, then restarting from scratch again. By now there are crappy big systems out there that are in their 7th or 8th generation. Maybe we should just do a better job of getting things organized first before we rush off to repeat the same mistakes again. Not making a mess on purpose is really the aim of architecture. You can tell how good people are at it, by the results that they leave behind.
Sunday, June 19, 2022
Basic Rules of Coding
It’s fairly easy to write code for a computer to run. But it’s incredibly hard to write code that isn’t ugly, awkward, or broken.
Most code needs to work reliably. No matter what else is happening around it, such as faults, user errors, or upgrades, the code should always do what is expected. That is an extremely hard attribute to get correct.
It is always better to get the code right the first time because it will save a lot of time overall. Patching any mess in the wild is super expensive. Some bugs are just time or money sinks.
It is preferable to reuse battle-tested code than it is to have new stuff, even if it is well-tested (which is rare). Testing and bug fixing are way more expensive than writing code. If you spend more time writing better code, you’ll spend less time firefighting it, thus saving a lot of time overall.
So there are a few very simple, high-level, rules that should be followed while coding:
First Rule: Write the code to be readable by as many people as possible. Readability always outweighs any other property. Every time.
Second Rule: Name all things correctly. Default most things to being explicit, but shorter names are usually better. Acronyms are bad, far worse if they are completely made-up.
Third Rule: Keep it all super organized. Be explicit about the organizational lines. Encapsulate it into one place, don’t fragment it all over the codebase.
Fourth Rule: Use what you have first before adding new stuff. Understand what you use really well, don’t use any stuff if you don’t understand it. Stuff that you fully understand is always better than anything else.
Fifth Rule: Edit, edit, edit. It sounds sexier to call it refactoring, but really it is just pure editing. Raw code is raw until you’ve passed through editing enough times to make it refined. The code is not done until you have spent time cleaning it up and correctly documenting it. Anything less just is just ‘in-progress’, definitely not ‘finished’.
Most programming should be slow, it is far better for quality if you take your time. However these days it is usually horrifically rushed, so it is even more critical to not waste any precious time. You don’t have enough of it. Don’t waste it doing unnecessary things, don’t waste it doing sloppy things that will hurt you later. Work smarter, not harder.
Most code needs to work reliably. No matter what else is happening around it, such as faults, user errors, or upgrades, the code should always do what is expected. That is an extremely hard attribute to get correct.
It is always better to get the code right the first time because it will save a lot of time overall. Patching any mess in the wild is super expensive. Some bugs are just time or money sinks.
It is preferable to reuse battle-tested code than it is to have new stuff, even if it is well-tested (which is rare). Testing and bug fixing are way more expensive than writing code. If you spend more time writing better code, you’ll spend less time firefighting it, thus saving a lot of time overall.
So there are a few very simple, high-level, rules that should be followed while coding:
First Rule: Write the code to be readable by as many people as possible. Readability always outweighs any other property. Every time.
Second Rule: Name all things correctly. Default most things to being explicit, but shorter names are usually better. Acronyms are bad, far worse if they are completely made-up.
Third Rule: Keep it all super organized. Be explicit about the organizational lines. Encapsulate it into one place, don’t fragment it all over the codebase.
Fourth Rule: Use what you have first before adding new stuff. Understand what you use really well, don’t use any stuff if you don’t understand it. Stuff that you fully understand is always better than anything else.
Fifth Rule: Edit, edit, edit. It sounds sexier to call it refactoring, but really it is just pure editing. Raw code is raw until you’ve passed through editing enough times to make it refined. The code is not done until you have spent time cleaning it up and correctly documenting it. Anything less just is just ‘in-progress’, definitely not ‘finished’.
Most programming should be slow, it is far better for quality if you take your time. However these days it is usually horrifically rushed, so it is even more critical to not waste any precious time. You don’t have enough of it. Don’t waste it doing unnecessary things, don’t waste it doing sloppy things that will hurt you later. Work smarter, not harder.
Subscribe to:
Posts (Atom)