This brings up an important piece of wisdom I see frequently stated and frequently ignored: don't micro-optimize until you've measured that you need to.
There are obviously exceptions in specific domains, but even in high performance cases it is generally better to optimize once you know where the bottlenecks are in practice.
You can pretty much just drop that "micro-". With very few exceptions, you really shouldn't optimize until you have accurate measurements of what is going on.
Update: To me, a lot of responses here conflate "optimization" with "design". From where I'm sitting, "optimization" means making adjustments to and existing solution to make it faster. That has nothing to do with choosing an appropriate language or toolset, and limited amounts to do with overall design. Yes, sometimes you must project forward a bit, for example knowing you'll be able to optimize out exponential behavior with a better data structure, say.
But often people do too much, too early, with too little information.
The qualifier "micro-" is right. You have to pick the language, and design your data structures and algorithms for efficiency from the ground up. Doing that after the fact is a nightmare, and downright impossible if you made the wrong language choice.
I wanted to make a similar comment but I don't understand what you mean by language choice. In my experience broad design choices can make it different to optimize later on, if you mean that.
That isn't what "micro-" typically means in this context. You are talking about design.
real micro optimization examples: order of operation in an inner loop. Changing memory alignment to facilitate vectorization, figuring out your compiler is doing something dumb and tricking it into better code generation for that area,etc.
If you are looking at generated assembly, chances you are doing micro-optimization.
I may have expressed myself in a confusing way, but it was in fact my intention to express that while you don't need to worry about micro-optimization up front, you do need to worry about designing for efficiency up front.
While that's true, it is often confused with the (wrong) assumption that: "don't try to do things efficiently; just measure later and redo the slow parts".
More often than not, if you don't think about performance characteristics beforehand, then -- after taking care of the low hanging fruit -- you find out that time is wasted all over, and there's no specific culprit that's making things so slow; at which point, you need to rewrite the whole thing.
Efficiency should be considered from the beginning in most projects, or later optimizations will be on the scale of "massive undertaking" to "complete rewrite".
We can differ on what we mean by "micro," but I think our intent is more or less the same. There is such a thing as anticipating performance problems and designing for them, and there is such a thing as losing sight of the performance needs of the code you're writing.
At the end of the day, it's another expression of the 80/20 rule: do the 20% of the optimization work that affects 80% of the runtime. But outside of general knowledge of where the inner loops are, you won't know what that 20% of code that touches 80% of runtime is until you measure.
But in practice it's quite often the case that you know optimisations are worth it based on past experience.
For example, with floating point calculations, divides are slower (on x86 anyway) than multiplies, so quite a common one in graphics/physics is to calculate the reciprocal (1.0f / x) and multiply by that if you'd do more than one divide in the first place.
The compiler can do this in some cases, but generally only with fpmath=fast, and even then, not always.
Figuring out which compiler optimizations did fire, and which still need to be done by hand, is also extremely difficult. Compilers generally don't tell you.
So while in principle one should be able to expect a compiler to do the $1/x$ optimization all by itself, it might be important enough to just do it by hand and get on with things.
True, in practice there are some things that you can anticipate optimizing. However, there is such a thing as doing too much, before you know that there's a performance problem.
Another good graphics/physics example is that you don't have to take the square root in a distance computation if you only care about the monotonically increasing ordering and not the exact distance. If you know that you will be dividing by magnitude or distance frequently, another optimization is to store the inverse of the magnitude instead of the original magnitude. That issue sometimes shows up in inner loops.
But optimizations like that also don't have to be done up front because refactoring that kind of thing is relatively straightforward compared to switching out the underlying data structure or algorithm.
My rule of thumb is that if I know I'm writing an inner loop, then I will bake in whatever best practices I already know about. Otherwise I'm going to write the feature first with generally sane architectural choices, and profile it later.
Optimizing floating point calculations in this way will impact the precision of your results. Multiplying by the reciprocal will not produce the same results as a division, especially when the error is compounded over multiple operations. Using the fpmath=fast option will also let the compiler produce faster code at the cost of accuracy.
Will this reduction in accuracy affect your application? For graphics and physics, maybe not, or at least not until some compounded error in an edge case makes a collision detection fail, or causes an unsightly visual error.
Don't just sprinkle these optimizations throughout your code without understanding the accuracy requirements of your program.
> Don't just sprinkle these optimizations throughout your code without understanding the accuracy requirements of your program.
There is a relatively straightforward way to think about this. Either your program depends on fully accurate error analysis (e.g. for reproducibility), or not. If yes, you would not even think of enabling fast-math, because that's implementation-defined behaviour. If no, any mathematically valid substitution will do, unless it causes some kind of unlikely numerical instability.
In the case of evaluating a/x as t=1/x;a * t, it is possible to analyze the optimization exactly: without overflows, the relative error of a * (1/x) will be at most 2u/(1-u), while relative error of a/x will be at most u. Thus this substitution is absolutely fine, there's no need to be scared of it.
Mathematically sensible transformations generally produce sensible results. It is exceedingly uncommon for someone to need a perfectly rounded result.
I'd rather they worked on improving IDE support for Clang and LLVM when it comes to "unmanaged" languages, as well as moved some of those compiler and optimization engineers to improving LLVM. LLVM still lags a bit behind GCC when it comes to performance of the machine code it generates, so I'm sure improvements there would be quite welcome, and I'm also sure Microsoft has the expertise to make it happen. For extra brownie points, help the community implement a portable debugger that doesn't suck, because thus far all attempts at this have failed miserably.
Microsoft C++ compiler is a source of aggravation at best. Their unmanaged language tooling has been neglected for nearly a decade while FOSS community (led by none other than Apple) plowed ahead and delivered the work of art that is Clang/LLVM, and a profusion of languages like Rust and Swift that run on top of that infrastructure.
25 comments
[ 3.5 ms ] story [ 73.1 ms ] threadThere are obviously exceptions in specific domains, but even in high performance cases it is generally better to optimize once you know where the bottlenecks are in practice.
Update: To me, a lot of responses here conflate "optimization" with "design". From where I'm sitting, "optimization" means making adjustments to and existing solution to make it faster. That has nothing to do with choosing an appropriate language or toolset, and limited amounts to do with overall design. Yes, sometimes you must project forward a bit, for example knowing you'll be able to optimize out exponential behavior with a better data structure, say.
But often people do too much, too early, with too little information.
real micro optimization examples: order of operation in an inner loop. Changing memory alignment to facilitate vectorization, figuring out your compiler is doing something dumb and tricking it into better code generation for that area,etc.
If you are looking at generated assembly, chances you are doing micro-optimization.
How would one know if the optimization was successful if there is no measurements to prove it!
More often than not, if you don't think about performance characteristics beforehand, then -- after taking care of the low hanging fruit -- you find out that time is wasted all over, and there's no specific culprit that's making things so slow; at which point, you need to rewrite the whole thing.
Efficiency should be considered from the beginning in most projects, or later optimizations will be on the scale of "massive undertaking" to "complete rewrite".
At the end of the day, it's another expression of the 80/20 rule: do the 20% of the optimization work that affects 80% of the runtime. But outside of general knowledge of where the inner loops are, you won't know what that 20% of code that touches 80% of runtime is until you measure.
For example, with floating point calculations, divides are slower (on x86 anyway) than multiplies, so quite a common one in graphics/physics is to calculate the reciprocal (1.0f / x) and multiply by that if you'd do more than one divide in the first place.
The compiler can do this in some cases, but generally only with fpmath=fast, and even then, not always.
So while in principle one should be able to expect a compiler to do the $1/x$ optimization all by itself, it might be important enough to just do it by hand and get on with things.
Another good graphics/physics example is that you don't have to take the square root in a distance computation if you only care about the monotonically increasing ordering and not the exact distance. If you know that you will be dividing by magnitude or distance frequently, another optimization is to store the inverse of the magnitude instead of the original magnitude. That issue sometimes shows up in inner loops.
But optimizations like that also don't have to be done up front because refactoring that kind of thing is relatively straightforward compared to switching out the underlying data structure or algorithm.
My rule of thumb is that if I know I'm writing an inner loop, then I will bake in whatever best practices I already know about. Otherwise I'm going to write the feature first with generally sane architectural choices, and profile it later.
Will this reduction in accuracy affect your application? For graphics and physics, maybe not, or at least not until some compounded error in an edge case makes a collision detection fail, or causes an unsightly visual error.
Don't just sprinkle these optimizations throughout your code without understanding the accuracy requirements of your program.
There is a relatively straightforward way to think about this. Either your program depends on fully accurate error analysis (e.g. for reproducibility), or not. If yes, you would not even think of enabling fast-math, because that's implementation-defined behaviour. If no, any mathematically valid substitution will do, unless it causes some kind of unlikely numerical instability.
In the case of evaluating a/x as t=1/x;a * t, it is possible to analyze the optimization exactly: without overflows, the relative error of a * (1/x) will be at most 2u/(1-u), while relative error of a/x will be at most u. Thus this substitution is absolutely fine, there's no need to be scared of it.
Mathematically sensible transformations generally produce sensible results. It is exceedingly uncommon for someone to need a perfectly rounded result.
Umm.. it should be evaluated at the top of the loop each time.
Microsoft C++ compiler is a source of aggravation at best. Their unmanaged language tooling has been neglected for nearly a decade while FOSS community (led by none other than Apple) plowed ahead and delivered the work of art that is Clang/LLVM, and a profusion of languages like Rust and Swift that run on top of that infrastructure.