Tesla Patent Don't multiply, add. It saves time and energy
Tesla has filed a patent "High Precision Complex Number Based Rotary Positional Encoding Calculation On 8-Bit Compute Hardware"
(US20260017019)
There are several ideas in the patent but it appears that this work will be available in the latest AI5 chip. A good introduction is https://www.youtube.com/watch?v=GG9yOsPEGek
One of the key ideas is to use logarithms and addition rather than multiplication. Faster and less power needed.
NVIDIA is going to hurt from this.
3 comments
[ 2.7 ms ] story [ 13.2 ms ] threadSeems it could at least slightly open up the world for doing it on 9-bit Compute hardware or (partially) in software, for example.