* OpenAI researchers claimed or suggested that GPT-5 had solved unsolved math problems, but in reality, the model only found known results that were unfamiliar to the operator of erdosproblems.com.
* Mathematician Thomas Bloom and Deepmind CEO Demis Hassabis criticized the announcement as misleading, leading the researchers to retract or amend their original claims.
* According to mathematician Terence Tao, AI models like GPT-5 are currently most helpful for speeding up basic research tasks such as literature review, rather than independently solving complex mathematical problems.
After the circular financing schemes involving hundreds of billions of dollars were uncovered, nothing I read about the AI business and its artificial hype machine surprises me anymore.
Another instance of openAI manipulating results to prolong their unsustainable circular hype bubble.
The inevitable collapse could be even more devastating than the 2008 financial crisis.
All while so vast resources are being wasted on non-verifiable gen AI slob, while real approaches (neuro-symbolic like DeepMind's AlphaFold) are mostly ignored financially because they don't generate the quick stock market increases that hype does.
This honestly doesn’t surprise me. We have reached a point where it’s becoming clearer and clearer that AGI is nowhere to be seen, whereas advances in LLM ability to ‘reason’ have slowed down to (almost?) a halt.
This entire thing has been pretty disingenuous on both sides of the fence. All the anti-AI (or anti OpenAI) people are doing victory laps, but what GPT-5 Pro did is still very valuable.
1) What good is your open problem set if really its a trivial "google search" away from being solved. Why are they not catching any blame here?
2) These answers still weren't perfectly laid out for the most part. GPT-5 was still doing some cognitive lifting to piece it together.
If a human would have done this by hand it would have made news and instead the narrative would have been inverted to ask serious questions about the validity of some these style problem sets and/or ask the question how many other solutions are out there that just need pieced together from pre-existing research.
I try not to lose sight of the first time that I heard (some years back) that people were using this new LLM thing for DM'ing ("dungeon mastering", leading) a game of Dungeons and Dragons. I thought, this must be bullshit or some kind of witchcraft.
Definitely not anti-AI here. I think I have been disappointed though, since then, to slowly learn that they're (still) little beyond that.
Still amazing though. And better than a Google search (IMHO).
> it just solved Erdos Problem #339 (listed as open in the official database https://erdosproblems.com/forum/thread/339) by realizing that it had actually been solved 20 years ago
> Update: Mehtaab and I pushed further on this. Using thousands of GPT5 queries, we found solutions to 10 Erdős problems that were listed as open: 223, 339, 494, 515, 621, 822, 883 (part 2/2), 903, 1043, 1079.
It's clearly talking about finding existing solutions to "open" problems.
The main mistake is by Kevin Weil, OpenAI CTO, who misunderstood the tweet:
> you are totally right—I actually misunderstood
@MarkSellke's original post, embarrassingly enough. Still very cool, but not the right words. Will delete this since I can't edit it any longer I think.
Obviously embarassing, but completely overblown reaction. Just another way for people to dunk on OpenAI :)
This happening the same week as DeepMind’s seemingly legitimate AI-assisted cancer treatment breakthrough is a startlingly bad look for OpenAI.
My boss always used to say “our only policy is, don’t be the reason we need to create a new policy”. I suspect OpenAI is going to have some new public communication policies going forward.
To be fair to the OpenAI team, if read in context the situation is at worst ambiguous.
The deleted tweet that the article is about said "GPT-5 just found solutions to 10 (!) previously unsolved Erdös problems, and made progress on 11 others. These have all been open for decades." If it had been posted stand-alone then I would certainly agree that it was misleading, but it was not.
The "this" in question is what this second tweet is in turn quote-tweeting: https://x.com/SebastienBubeck/status/1977181716457701775?t=T... -- where the author says "gpt5-pro is superhuman at literature search: [...] it just solved Erdos Problem #339 (listed as open in the official database erdosproblems.com/forum/thread/3…) by realizing that it had actually been solved 20 years ago"
So, reading the thread in order, you get
* SebastienBubeck: "GPT-5 is really good at literature search, it 'solved' an apparently-open problem by finding an existing solution"
* MarkSellke: "Now it's done ten more"
* kevinweil: "Look at this cool stuff we've done!"
I think the problem here is the way quote-tweets work -- you only see the quoted post and not anything that it in turn is quoting. Kevin Weil had the two previous quotes in his context when he did his post and didn't consider the fact that readers would only see the first level, so wouldn't have Sebastien Bubek's post in mind when they read his.
That seems like an easy mistake to entirely honestly make, and I think the pile-on is a little unfair.
> GPT-5 is proving useful as a literature review assistant
No, it does not. It only produces a highly convincing counterfeit. I am honestly happy for people who are satisfied with its output: life is way easier for them than for me.
Obviously, the machine discriminates me personally. When I spend hours in the library looking for some engineering-related math made in the 70s-80s, as a last resort measure, I can try to play this gambling with chat, hoping for any tiny clue to answer my question. And then for the following hours, I am trying to understand what is wrong with the chat output. Most often, I experience the "it simply can't be" feeling, and I know I am not the only one having it.
Wouldn't be surprised if OpenAI employees are being asked to phrase (market) things this way. This is not the first time they claimed GPT-5 "solved" something [1]
It's becoming increasingly clear that gpt5 can solve MINOR open math problems, those that would require a day/few days of a good PhD student. Ofc it's not a 100% guarantee, eg below gpt5 solves 3/5 optimization conjectures. Imo full impact of this has yet to be internalized...
32 comments
[ 3.1 ms ] story [ 46.3 ms ] thread* OpenAI researchers claimed or suggested that GPT-5 had solved unsolved math problems, but in reality, the model only found known results that were unfamiliar to the operator of erdosproblems.com.
* Mathematician Thomas Bloom and Deepmind CEO Demis Hassabis criticized the announcement as misleading, leading the researchers to retract or amend their original claims.
* According to mathematician Terence Tao, AI models like GPT-5 are currently most helpful for speeding up basic research tasks such as literature review, rather than independently solving complex mathematical problems.
The inevitable collapse could be even more devastating than the 2008 financial crisis.
All while so vast resources are being wasted on non-verifiable gen AI slob, while real approaches (neuro-symbolic like DeepMind's AlphaFold) are mostly ignored financially because they don't generate the quick stock market increases that hype does.
Another case of culture flowing from the top I guess.
1) What good is your open problem set if really its a trivial "google search" away from being solved. Why are they not catching any blame here?
2) These answers still weren't perfectly laid out for the most part. GPT-5 was still doing some cognitive lifting to piece it together.
If a human would have done this by hand it would have made news and instead the narrative would have been inverted to ask serious questions about the validity of some these style problem sets and/or ask the question how many other solutions are out there that just need pieced together from pre-existing research.
But, you know, AI Bad.
Edit: we are in peak damage control phase of the hype cycle.
Definitely not anti-AI here. I think I have been disappointed though, since then, to slowly learn that they're (still) little beyond that.
Still amazing though. And better than a Google search (IMHO).
https://x.com/SebastienBubeck/status/1977181716457701775:
> gpt5-pro is superhuman at literature search:
> it just solved Erdos Problem #339 (listed as open in the official database https://erdosproblems.com/forum/thread/339) by realizing that it had actually been solved 20 years ago
https://x.com/MarkSellke/status/1979226538059931886:
> Update: Mehtaab and I pushed further on this. Using thousands of GPT5 queries, we found solutions to 10 Erdős problems that were listed as open: 223, 339, 494, 515, 621, 822, 883 (part 2/2), 903, 1043, 1079.
It's clearly talking about finding existing solutions to "open" problems.
The main mistake is by Kevin Weil, OpenAI CTO, who misunderstood the tweet:
https://x.com/kevinweil/status/1979270343941591525:
> you are totally right—I actually misunderstood @MarkSellke's original post, embarrassingly enough. Still very cool, but not the right words. Will delete this since I can't edit it any longer I think.
Obviously embarassing, but completely overblown reaction. Just another way for people to dunk on OpenAI :)
My boss always used to say “our only policy is, don’t be the reason we need to create a new policy”. I suspect OpenAI is going to have some new public communication policies going forward.
The deleted tweet that the article is about said "GPT-5 just found solutions to 10 (!) previously unsolved Erdös problems, and made progress on 11 others. These have all been open for decades." If it had been posted stand-alone then I would certainly agree that it was misleading, but it was not.
It was a quote-tweet of this: https://x.com/MarkSellke/status/1979226538059931886?t=OigN6t..., where the author is saying he's "pushing further on this".
The "this" in question is what this second tweet is in turn quote-tweeting: https://x.com/SebastienBubeck/status/1977181716457701775?t=T... -- where the author says "gpt5-pro is superhuman at literature search: [...] it just solved Erdos Problem #339 (listed as open in the official database erdosproblems.com/forum/thread/3…) by realizing that it had actually been solved 20 years ago"
So, reading the thread in order, you get
I think the problem here is the way quote-tweets work -- you only see the quoted post and not anything that it in turn is quoting. Kevin Weil had the two previous quotes in his context when he did his post and didn't consider the fact that readers would only see the first level, so wouldn't have Sebastien Bubek's post in mind when they read his.That seems like an easy mistake to entirely honestly make, and I think the pile-on is a little unfair.
Imagine if you were talking about your own work online, you make an honest mistake, then the whole industry roasts you for it.
I’m so tired of hearing everyone take stabs at people at OpenAI just because they don’t personally like sama or something.
No, it does not. It only produces a highly convincing counterfeit. I am honestly happy for people who are satisfied with its output: life is way easier for them than for me. Obviously, the machine discriminates me personally. When I spend hours in the library looking for some engineering-related math made in the 70s-80s, as a last resort measure, I can try to play this gambling with chat, hoping for any tiny clue to answer my question. And then for the following hours, I am trying to understand what is wrong with the chat output. Most often, I experience the "it simply can't be" feeling, and I know I am not the only one having it.
[1] https://x.com/SebastienBubeck/status/1970875019803910478
edit: full text
It's becoming increasingly clear that gpt5 can solve MINOR open math problems, those that would require a day/few days of a good PhD student. Ofc it's not a 100% guarantee, eg below gpt5 solves 3/5 optimization conjectures. Imo full impact of this has yet to be internalized...