>An interesting observation we made was that even though the 3 optimizers were given tens of thousands of dollars in compute, they found the best solution very early on
Honestly, this confirms some bias I already have from coding — models can get to the best solution pretty fast, but if the solution is something obscure, they usually don’t find it.
It would be interesting to see how a human researcher would do on the same task and same budget. 2 weeks of salary + a Claude Code max plan might outperform all of these auto-optimizers in a loop.
4 comments
[ 0.21 ms ] story [ 15.3 ms ] threadHonestly, this confirms some bias I already have from coding — models can get to the best solution pretty fast, but if the solution is something obscure, they usually don’t find it.