Code generally implements a series of steps for the computer to follow. I am using a slightly broader definition than just an ‘algorithm’ or ‘heuristic’, which are usually defined as mappings between input and output. It is widened to include any sort of code that interacts with one or more endpoints.
We’ll talk about three general possible versions of this code. The first does the steps in an obvious way. The second adds unnecessary extra steps as well. And the third does the steps in a non-intuitive way that is faster. We can call these normal, excessive, and optimized.
Most times when you see people “optimize code” they are actually just taking excessive code and replacing it with normal code. That is, they are not optimizing it, really they just aren’t doing the useless work anymore.
If you take excessive code and fix it, you are not doing premature optimization, you’re just coding it properly. The excessive version was a mistake. It was wasting resources, which is unnecessary. Not doing that anymore is not really optimizing stuff.
If you have good coding habits, for the most part, you will write normal code most of the time. But it takes a lot of practice to master. And it comes from changing how you see the code and how you construct it.
Sometimes normal code is not fast enough. You will need to optimize it. Most serious optimizations tend to be limited to logarithmic gains. That is, you start with O(n^2) and bring it down to O(n). Sorting, for example, starts with O(n!) and gets it to O(n log n). All of these types of optimizations involve visualizing the code from a very non-intuitive viewpoint and using that view to leverage some information that circumvents the normal, intuitive route. These are the hardcode optimizations. The ones that we are warned not to try right away.
It is easy while trying to optimize code to break it instead. It is also easy to make it a whole lot slower. Some optimizations like adding in caching seem to be deceptively easy, but doing it incorrectly causes all sorts of unwelcomed bugs.
Making tradeoffs like space-time are sort of optimizations. They may appear to alter the performance but it can be misleading. My favorite example is matching unique elements in sets. The obvious way to code it is with two for loops. You take each member of the first set and compare it to the second one. But you can swap time for space. In that case, you pass through the first set and hash it, then pass through the second set and see if it is in the hash. If the respective sizes are m and n, the obvious algorithm is O(n*m) where the hashed version is O(n+m). For small data, the extra hash table shifts the operation from being multiplicative to additive. But if you scaled that up to large enough data, the management of the hash table and memory could eliminate most of those gains. It’s also worth noting that it is bounded as logarithmic, you see that by setting m to be another n.
The real takeaway though is to learn to code just the instructions that are necessary to complete the work. You so often see code that is doing all sorts of unnecessary stuff, mostly because the author does not know how to structure it better or understand what happens underneath. You also see code that does and undoes various fiddling over and over again as the data moves through the system. Deciding on a canonical representation and diligently sticking to that can avoid a lot of that waste.
Debloating code is not optimizing it. Sure it makes it run faster and with fewer resources, but it is simply removing what should have not been there in the first place. We need to teach coding in a better way so that programmers learn how to write stuff correctly the first time. Premature optimizations, though, are still the root of all evil. You need to get your code working first before you start messing with logarithmic reductions. They can be a bit mind-bending at times.
We’ll talk about three general possible versions of this code. The first does the steps in an obvious way. The second adds unnecessary extra steps as well. And the third does the steps in a non-intuitive way that is faster. We can call these normal, excessive, and optimized.
Most times when you see people “optimize code” they are actually just taking excessive code and replacing it with normal code. That is, they are not optimizing it, really they just aren’t doing the useless work anymore.
If you take excessive code and fix it, you are not doing premature optimization, you’re just coding it properly. The excessive version was a mistake. It was wasting resources, which is unnecessary. Not doing that anymore is not really optimizing stuff.
If you have good coding habits, for the most part, you will write normal code most of the time. But it takes a lot of practice to master. And it comes from changing how you see the code and how you construct it.
Sometimes normal code is not fast enough. You will need to optimize it. Most serious optimizations tend to be limited to logarithmic gains. That is, you start with O(n^2) and bring it down to O(n). Sorting, for example, starts with O(n!) and gets it to O(n log n). All of these types of optimizations involve visualizing the code from a very non-intuitive viewpoint and using that view to leverage some information that circumvents the normal, intuitive route. These are the hardcode optimizations. The ones that we are warned not to try right away.
It is easy while trying to optimize code to break it instead. It is also easy to make it a whole lot slower. Some optimizations like adding in caching seem to be deceptively easy, but doing it incorrectly causes all sorts of unwelcomed bugs.
Making tradeoffs like space-time are sort of optimizations. They may appear to alter the performance but it can be misleading. My favorite example is matching unique elements in sets. The obvious way to code it is with two for loops. You take each member of the first set and compare it to the second one. But you can swap time for space. In that case, you pass through the first set and hash it, then pass through the second set and see if it is in the hash. If the respective sizes are m and n, the obvious algorithm is O(n*m) where the hashed version is O(n+m). For small data, the extra hash table shifts the operation from being multiplicative to additive. But if you scaled that up to large enough data, the management of the hash table and memory could eliminate most of those gains. It’s also worth noting that it is bounded as logarithmic, you see that by setting m to be another n.
The real takeaway though is to learn to code just the instructions that are necessary to complete the work. You so often see code that is doing all sorts of unnecessary stuff, mostly because the author does not know how to structure it better or understand what happens underneath. You also see code that does and undoes various fiddling over and over again as the data moves through the system. Deciding on a canonical representation and diligently sticking to that can avoid a lot of that waste.
Debloating code is not optimizing it. Sure it makes it run faster and with fewer resources, but it is simply removing what should have not been there in the first place. We need to teach coding in a better way so that programmers learn how to write stuff correctly the first time. Premature optimizations, though, are still the root of all evil. You need to get your code working first before you start messing with logarithmic reductions. They can be a bit mind-bending at times.
No comments:
Post a Comment
Thanks for the Feedback!