Noise: Speedrun for Pretraining Without Backpropagation (qlabs.sh) 2 points by ZeroCool2u 6d ago ↗ HN
[–] pfdietz 5d ago ↗ https://x.com/industriaalist/status/2113365351455219760> In the 10 days since we released the Noise speedrun, the time to pretrain a transformer went from 1.76 hours to 1.6 seconds, a 4,000x (!!!) speedup. Dust itself was up to 10,000x more compute-efficient than the previous state-of-the-art.
1 comment
[ 0.25 ms ] story [ 16.8 ms ] thread> In the 10 days since we released the Noise speedrun, the time to pretrain a transformer went from 1.76 hours to 1.6 seconds, a 4,000x (!!!) speedup. Dust itself was up to 10,000x more compute-efficient than the previous state-of-the-art.