I don't think GPUs ever had branch prediction in the first place. You can however run into poor performance due to thread divergence, which is a similar kind of issue (with much less black magic).
It wouldn't surprise me to see some ML algorithm in silico somewhere to select faster matmul paths on favorable data. Yo dawg, I heard you like AI, so we put some AI in your AI so you can infer while you're inferring.
I feel like many of the comments missed the point or didn't read the article. What I believe this article is stating (and I've read this many times during my PhD for various reasons), is that the input data distributions affect how many transistor state changes there are during multiplication. Since these events are a large portion of energy loss/heat generation, the clocks won't be throttled as much for certain data patterns.
There was a workshop paper from SC24 that did more experiments around this I believe. I can't find it now though.
I'd have guessed multiply-by-0 and multiply-by-1 can be special-cased to run much faster and simpler code paths, like you'd do when writing MUL for a processor that doesn't have it (I <3 z80)
17 comments
[ 3.2 ms ] story [ 34.4 ms ] thread[0] - https://stackoverflow.com/questions/11227809/why-is-conditio...
https://clehaxze.tw/gemlog/2025/04-21-programming-tensotrren...
https://clehaxze.tw/gemlog/2026/01-22-the-real-tenstorrent-t...
https://arxiv.org/html/2604.03279
I could certainly come up with alternative theories about memory compression and prefetching if we were talking about texture reads.
I haven't used a non-laptop GPU in some time, but that is a crazy amount of "idle" power consumption. Is this normal for cards like this?
There was a workshop paper from SC24 that did more experiments around this I believe. I can't find it now though.
When you make it so the computer does not have to compute all possible states of matter it finishes faster.