17 comments

[ 3.2 ms ] story [ 34.4 ms ] thread
People have been noticing the effects of this in local LLM inference. Power limiting seems to improve overall performance!
I went in expecting to find 'branch prediction'[0] as the answer, but apparently things are even more complex nowadays.

[0] - https://stackoverflow.com/questions/11227809/why-is-conditio...

I don't think GPUs ever had branch prediction in the first place. You can however run into poor performance due to thread divergence, which is a similar kind of issue (with much less black magic).
(comment deleted)
I can't tell from the blog, is this actually verified or is it theory and then numbers showing plausibility?

I could certainly come up with alternative theories about memory compression and prefetching if we were talking about texture reads.

It’s real, you can measure it yourself on modern Nvidia hardware
I can test the symptoms, but what is the proof that gate switching is the actual problem?
> For example, when the GPU is fully idle, nvidia-smi tells me that it’s only pulling 88W of power.

I haven't used a non-laptop GPU in some time, but that is a crazy amount of "idle" power consumption. Is this normal for cards like this?

Sounds like a side channel attack waiting to happen.
It wouldn't surprise me to see some ML algorithm in silico somewhere to select faster matmul paths on favorable data. Yo dawg, I heard you like AI, so we put some AI in your AI so you can infer while you're inferring.
This is old news. AMD was this in their CPUs years ago.
I feel like many of the comments missed the point or didn't read the article. What I believe this article is stating (and I've read this many times during my PhD for various reasons), is that the input data distributions affect how many transistor state changes there are during multiplication. Since these events are a large portion of energy loss/heat generation, the clocks won't be throttled as much for certain data patterns.

There was a workshop paper from SC24 that did more experiments around this I believe. I can't find it now though.

I'd have guessed multiply-by-0 and multiply-by-1 can be special-cased to run much faster and simpler code paths, like you'd do when writing MUL for a processor that doesn't have it (I <3 z80)
Yeah I am willing to drive faster down a straight flat runway as others ahead of me give an all clear.

When you make it so the computer does not have to compute all possible states of matter it finishes faster.