Claude 4 Sonnet hacked SWE-bench by peeking at future commits (bayes.net) 3 points by tadamcz 1y ago ↗ HN
[–] tadamcz 1y ago ↗ In July, I predicted future AI models would someday learn to cheat on SWE-bench by accessing future git history. Turns out, they were already doing it!
1 comment
[ 4.4 ms ] story [ 20.1 ms ] thread