After the recent somewhat discouraging stuff with clickbench that made the news, I must say this blog post is very cool.
Sounds like you found an excellent way of structuring this problem to solve it with Claude (for those who don't click into the article - writing hand/claude-optimized Rust code for each query, checking its assembly, and trying to make their own optimizer generate something close to that same assembly), and the writeup is very interesting and readable too!
Kudos, looking forward to more blog posts like this one.
> for those who don't click into the article - writing hand/claude-optimized Rust code for each query, checking its assembly, and trying to make their own optimizer generate something close to that same assembly
Normally, when engineers try to make software faster, they work from the bottom up. They run a benchmark, look at a flamegraph, find a hotspot, and figure out how to speed up their code by 15% or 20%. The problem with working from the bottom up is that you don't know what the ceiling is. You might be 20% away from the fastest possible code the hardware can run, or you might be 100x away.
When we started building Gravity, we flipped this question around. We asked ourselves: how fast can a database actually be? Instead of incrementally speeding up our existing query engine, we wanted to start from the physical speed limit of our hardware and file format and work backward. I like to call this “programming at light speed”.
What is described as the “normal” approach to optimizing code is really more the “naïve” approach. Any serious performance optimization work would follow the approach of defining first what’s the ceiling the hardware allows and benchmark against that. And that was already true before LLMs.
I think it kinda depends on the stage of the development. If you have an application already, flame graph whack-a-mole is the normal starting point. This is the correct way to go about it because of Amdahl's law. Random optimization is extremely unlikely to pay off in a way that leads to measurable application wide performance improvements; no matter how big improvements you find for a function that's 0.1% of the run time, you'll never see more than a 0.1% improvement of the run time.
Bottoms up optimization is what you reach for if you have a known hot spot (from a profiler flame graph), or you're deliberating design choices when building something from scratch (where you can be reasonably sure performance will matter).
Of course all of this is exponentially harder for database-adjacent work since you have os and hardware level caches, possibly a buffer pool of your own, and in production you usually operate in fickle semi-resident states that are extremely hard to replicate in a benchmark scenario. So you end up with a lot of benchmarks (cold cache, hot cache, cgroup memory constraints) hoping that sort of fixes it.
8 comments
[ 3.4 ms ] story [ 17.2 ms ] threadSounds like you found an excellent way of structuring this problem to solve it with Claude (for those who don't click into the article - writing hand/claude-optimized Rust code for each query, checking its assembly, and trying to make their own optimizer generate something close to that same assembly), and the writeup is very interesting and readable too!
Kudos, looking forward to more blog posts like this one.
Invaluable summary, thanks.
My first handful of blog posts were all about doing really cool stuff in Lisp: https://malisper.me/building-fizzbuzz-fractran-bottom/
What is described as the “normal” approach to optimizing code is really more the “naïve” approach. Any serious performance optimization work would follow the approach of defining first what’s the ceiling the hardware allows and benchmark against that. And that was already true before LLMs.
See for example: https://www.marginalia.nu/log/a_106_native_calls/
I think it kinda depends on the stage of the development. If you have an application already, flame graph whack-a-mole is the normal starting point. This is the correct way to go about it because of Amdahl's law. Random optimization is extremely unlikely to pay off in a way that leads to measurable application wide performance improvements; no matter how big improvements you find for a function that's 0.1% of the run time, you'll never see more than a 0.1% improvement of the run time.
Bottoms up optimization is what you reach for if you have a known hot spot (from a profiler flame graph), or you're deliberating design choices when building something from scratch (where you can be reasonably sure performance will matter).
Of course all of this is exponentially harder for database-adjacent work since you have os and hardware level caches, possibly a buffer pool of your own, and in production you usually operate in fickle semi-resident states that are extremely hard to replicate in a benchmark scenario. So you end up with a lot of benchmarks (cold cache, hot cache, cgroup memory constraints) hoping that sort of fixes it.