It's impressive to see how fast open-weights models are catching up in specialized domains like math and reasoning.
I'm curious if anyone has tested this model for complex logic tasks in coding? Sometimes strong math performance correlates well with debugging or algorithm generation.
All the problems they claim to have solved are on are the Internet and they explicitly say they crawled them. They do not mention doing any benchmark decontamination or excluding 2024/2025 competition problems from training.
IIRC correctly OpenAI/Google did not have access to the 2025 problems before testing their experimental math models.
15 comments
[ 2.5 ms ] story [ 28.1 ms ] threadPeople use Instagram too.
All the problems they claim to have solved are on are the Internet and they explicitly say they crawled them. They do not mention doing any benchmark decontamination or excluding 2024/2025 competition problems from training.
IIRC correctly OpenAI/Google did not have access to the 2025 problems before testing their experimental math models.