I've seen a few of these in my career, if I understand the author correctly. You have a big ball of mud that can theoretically be 10x or 100x faster, but the costs are diffuse and can't be solved by just finding a hotspot and optimizing it.
It often happens for good reasons. Features get added over time, there are some scars from a mocking framework, simpler (faster) solutions don't quite work because they're supporting X which supports Y which supports Z (dead code, but nobody noticed), people use full datetime handling when they mean to access performance counters, the complexity of the thing means that you blow your branch prediction cache size budget, etc....
The solution is to deeply understand the problem (lots of techniques, but this comment isn't a blog post) and come up with a solution, like a ground-up rewrite of some or all of the offending section.
Great article. Can confirm, writing performance focused C# is fun. It's great having the convenience of async, LINQ, and GC for writing non-hot path "control plane" code, then pulling out Vector<T>, Span<T>, and so on for the hot path.
One question, how portable are performance benefits from tweaks to memory alignment? Is this something where going beyond rough heuristics (sequential access = good, order of magnitude cache sizes, etc) requires knowing exactly what platform you're targeting?
To the list of profiling tools I would like to add KDAB Hotspot and KDE Heaptrack.
The former, hotspot, is a visualiser for perf data, and it deals ok with truly massive files that made perfetto and similar just big down. It also supports visualing off-CPU profiles ("why is my program slow but not CPU bound?").
The latter, heaptrack, is a tool with very similar UI to hotspot (I think the two tools share some code even) to profile malloc/free (or new/delete). Sometimes the performance issue is as simple as not reusing a buffer but reallocating it over and over inside a loop. And sometimes you wonder where all the memory is going.
7 comments
[ 3.2 ms ] story [ 21.9 ms ] threadIt sounds like the “worst case“ here is that the program is already optimized.
It often happens for good reasons. Features get added over time, there are some scars from a mocking framework, simpler (faster) solutions don't quite work because they're supporting X which supports Y which supports Z (dead code, but nobody noticed), people use full datetime handling when they mean to access performance counters, the complexity of the thing means that you blow your branch prediction cache size budget, etc....
The solution is to deeply understand the problem (lots of techniques, but this comment isn't a blog post) and come up with a solution, like a ground-up rewrite of some or all of the offending section.
One question, how portable are performance benefits from tweaks to memory alignment? Is this something where going beyond rough heuristics (sequential access = good, order of magnitude cache sizes, etc) requires knowing exactly what platform you're targeting?
The former, hotspot, is a visualiser for perf data, and it deals ok with truly massive files that made perfetto and similar just big down. It also supports visualing off-CPU profiles ("why is my program slow but not CPU bound?").
The latter, heaptrack, is a tool with very similar UI to hotspot (I think the two tools share some code even) to profile malloc/free (or new/delete). Sometimes the performance issue is as simple as not reusing a buffer but reallocating it over and over inside a loop. And sometimes you wonder where all the memory is going.