This is a good set of slides. Dan is a good guy. There are a few nits I would pick. Sqrt(N) convergence comes from independence not normality -- based on independence => linearity of variance. { So, N IID samples of any distribution have a sum with N times higher variance, but then dividing by N you et sqrt(N). } There is, of course, a higher order relationship between the variance / "scale^2" of the distro and its tails which statisticians refer to as "shape". He later goes on to mention the dependence problem, though, and the minimum dt solution that, relied upon by, e.g., https://github.com/c-blake/bu/blob/main/doc/tim.md. So, it's all good. He may have covered it in voice, even.
He also mentions the Sattolo used by https://github.com/c-blake/bu/blob/main/doc/memlat.md to do his memory latency measurements. One weird thing was how he said because of 1 byte/cycle is 4GB/s things are "easily CPU bound" while I feel like I've been "fighting The Memory Wall for at least 3 decades now..." even just from super-scalar CPUs, but he later does some vectorization stuff. That more relates to what calcs you are doing, of course, but high bandwidth memory is a big part of what nVidia is selling.
> Processors can 'learn' thousands of branches: benchmark over massive inputs.
> Pick a solution without branches when it provides the same performance.
Note that it is terrible idea to benchmark small parts of your software individually and compose together the parts that are individually the fastest. Branch history tables and branch target buffers are a shared resource, and performance can crater horribly if you routinely exceed what's provided. You should generally favor straight-line code even when it's slightly slower in microbenchmarks.
> Note that it is terrible idea to benchmark small parts of your software individually and compose together the parts that are individually the fastest
Certainly beats composing together the parts that are individually the slowest or not profiling at all. When you have to start worrying about shared CPU resources (main memory is a different matter), you are already doing much better than most software.
The most relevant topic in this space is: processors got wider, more speculative etc. But if you measure IPC across a wide range of workloads it way less than 1 instruction per cycle.
Talking about this to software engineers (this is pre vibe-code era), you get a response like "this is too low level and not worth it".
If you’re already competent with systems languages in keen to _go on a wild deep dive_ into this sort of perf engineering, where’s a good place or good resources to get started ?
12 comments
[ 2.8 ms ] story [ 33.3 ms ] threadhttps://x.com/lemire/status/1947615932702200138
edit - #64 E ! Also, i always say, human body is most error prone measuring device humans have in their disposal.
He also mentions the Sattolo used by https://github.com/c-blake/bu/blob/main/doc/memlat.md to do his memory latency measurements. One weird thing was how he said because of 1 byte/cycle is 4GB/s things are "easily CPU bound" while I feel like I've been "fighting The Memory Wall for at least 3 decades now..." even just from super-scalar CPUs, but he later does some vectorization stuff. That more relates to what calcs you are doing, of course, but high bandwidth memory is a big part of what nVidia is selling.
> Processors can 'learn' thousands of branches: benchmark over massive inputs.
> Pick a solution without branches when it provides the same performance.
Note that it is terrible idea to benchmark small parts of your software individually and compose together the parts that are individually the fastest. Branch history tables and branch target buffers are a shared resource, and performance can crater horribly if you routinely exceed what's provided. You should generally favor straight-line code even when it's slightly slower in microbenchmarks.
Certainly beats composing together the parts that are individually the slowest or not profiling at all. When you have to start worrying about shared CPU resources (main memory is a different matter), you are already doing much better than most software.
Talking about this to software engineers (this is pre vibe-code era), you get a response like "this is too low level and not worth it".
More such talks needed!