Training NanoGPT in 39.9 Seconds (hyperstition.cc)

Less mathy discussion on Twitter, that also covers some of the speedup that comes from outside the optimiser https://x.com/classiclarryd/status/2104688738354028962

Massive result