1 comment

[ 3.4 ms ] story [ 9.6 ms ] thread
Pretraining a frontier model is a one-time expense: 6,000 to 25,000 GPUs for one to four months, $5M to $150M depending on chip generation and luck.