Human mathematicians have been being out-counterexampled for at least two decades. The main difference, as I understand, is that (A) we now have a lot more compute to throw at such things, and (B) it is currently trendy to do so. But the sizes of counterexample we're seeing are around about what I'd expect pre-generative-AI counterexample search systems to be able to find.
It's not easy to find a counterexample to the Jacobian conjecture, by any means – by which I mean to say that naïve brute-force search will take too long – but the scope of existing searches listed on Wikipedia[0] suggest that many tricks are already known, and that people just hadn't looked, systematically, for a counterexample in three variables before. Wikipedia writes:
> Tzuong-Tsieng Moh checked the conjecture for polynomials of degree at most 100 in two variables.[17][18]
where reference 17 is from 1983, and reference 18 is a preprint with no given date. Knowing very little about this problem, my impulse is to side with the unnamed faculty member cited in the article:
> [who] said to me that the fact that the counterexample was so easy to find just indicated that humans had not spent enough time thinking about the problem,
For context, the auto-generated counterexample is in three variables, has degree 7, and was discovered in 2026.
> The conjecture was first stated for two variables by Ludwig Kraus in 1884 [...] an example of a difficult question in algebraic geometry that can be understood using little beyond a knowledge of calculus.
> The Jacobian conjecture is number 16 in Stephen Smale's 1998 list of Mathematical Problems for the Next Century. It was notorious for the large number of published and unpublished false proofs that turned out to contain subtle errors.
mathematicians have been using computers for well over half a century, but this was after "bounding" the problem first and then running through the cases with a computer. Now AI is doing the first part. However, mathematicians are still needed at crafting prompts, and knowing where to look, still. The prompt for the Jacobian conjecture was obviously not random. the search space is too big to just try all the combinations of 3 variable polynomials.
That's a good thing. It saves people wasting time trying to prove something they now know to be false, so that they can move on to other things to prove, it's a more fruitful use of humanity's time overall at least in the field of mathematics.
If the poster's (is it Kevin Buzzard?) suggestion works out and AI finds a counterexample to the Hodge conjecture, that would be a really big deal. It's one of the Millenium problems, for example.
One thing that he mentions that already quite surprising is that AI was able to autoformalize the Golod-Shaferevich theorem and proof.
Interestingly, Yitang Zhang of the twin-prime-conjecture fame spent 7 years working on the Jacobian conjecture under the advisor Tzuong-Tsieng Moh at Purdue. A key step in his thesis used a corollary of Moh's. It turned out that the corollary was incorrect. As a result, Moh refused to write any recommendation letter for Zhang, and Zhang couldn't find any teaching or research job and ended up spending years working at a Subway[1].
Imagine Zhag had ChatGPT in 1986 when he started working on the Jacobian Conjecture.
[1] Of course now this has become an inspiring story. That said, the story definitely invokes complex emotions. The best way to describe it is probably this Chinese poem, which I have no idea how to translate: 庾信平生最萧瑟，暮年诗赋动江关
When I was in grad school, I had the opportunity to take a course from my adviser in which he discussed his current research and some open questions. It was a relatively accessible subject area and the questions were sometimes easy enough that we could meaningfully contribute.
On one particular Friday afternoon, he stated a conjecture that he hoped was true, and invited us to try to help him prove or disprove it. It was the sort of thing that he really wanted to be true; he liked things smooth and beautiful. I, on the other hand, hoped it was false as I like the weird and exceptional in mathematics. It was also the case that I had absolutely no command of the sort of machinery that one would use to prove such a thing, but I could certainly look for a counterexample.
I learned on Monday that he had spent the entire weekend trying and failing to prove it. I, on the other hand, had put all my energy into finding a counterexample and had one within an hour.
My single (quite small) contribution to mathematical research was a counterexample because it was all I could do. The story does illustrate that it can be helpful to have people with different tools, hopes, and motivations working on a problem, though. I was not, and will never be, even a shadow of that great mathematiciam I studied under, but on that occasion, I had reason to look in a different direction than he did.
A lot of this math is beyond my comprehension, but it often seems to talk of proofs of theorems. What I want to know is if we continue on this accelerated AI mathematics trajectory, will we eventually be discovering new forms of math that will in turn have some applications down the line in engineering or biomedicine etc? I guess what I’m asking is are we on the cusp of a huge breakthrough for humanity, or largely just proving what was already known?
Most of maths is remarkably abstract and impractical, but sometimes real-world scenarios turn out to be related to some obscure branch of math. Internet security is based on elliptic curves in finite fields, and why would you ever study that if it wasn't powering internet security? Well some people did study it before, for no good reason, and that's how we knew about it.
You can personally work out how apply this one in engineering today!
If you have a spare hour or two, I'd encourage you to have a go at learning (however you learn best - I like just asking smarter people or robots stupid questions) what the maths means and why it's important.
And then once you feel like you have a vague grip on the principles, think about a problem in a domain you know a lot about. Try to see if the maths - and how it's changed our perception - could be used as a tool to solve that problem, or if the solution is analogous to a solution you could try in your own domain of expertise.
LLMs are good at speeding up, I think, the journey an idea has to take between "theoretical academic stuff for academics" and "a usable idea for regular people", because they increasingly allow you to ask an infinite number of stupid questions and give you (hopefully) reasonably good responses.
I've had loads of fun doing this today - specifically seeing if the idea this counter (from what I understand: a many-to-one conversion that kind of does and kind of does not preserve meaning) can tell me anything about the relationship between language and meaning.
I'm sure everything I've done today while mucking around has been the equivalent of a monkey with a typewriter (and Codex), but I think the huge breakthrough(s) you ask whether we're on the cusp of are relatively dependent on how many monkeys are throwing typewriters at problems they know a little bit about, after learning a bit about new ideas like this one. Historically, that's a really good way for broad cultural innovation to happen - distributed information applied across multiple domains by experts in them.
I must object. Because this is derivation, not composition. German is noteworthy for its long compositional word formations (though many other language are, too, e.g., Finnish). German can do derivation, too, but not significantly more than English (and the verb prefix 'out-' is hard to map into German in this case). For admiring derivation, you'd have to point to, e.g., Greenlandic instead (e.g. _iliorfigeqatigiissariaqaraluarput_ roughly 'they should work together' (from the Declaration of Human Rights)).
To be clear, I do like 'outcounterexampled' a lot!
I suppose it will fall to AI as well to compose the mathematical equivalent of The Ballad of John Henry. Who will be the human champion, the last great hero who can deliver proofs "from the book" that a machine cannot outperform?
This is an unhealthy view of mathematics. It's mathematics as envisioned by football fans.
The most valuable things in mathematics are not beautiful proofs. We need more useful definitions. Actually coming up with useful definitions (and building good conjectures out of them - not even theorems, conjectures) is something LLMs have not yet tried to conquer.
This morning I was reading a paper called "An enduring error" (2009) by Branko Grünbaum, regarding Archimedean polyhydra. From the introduction:
> ..Even more unexpected is the fact that many expositions of this topic commit serious mathematical and logical errors. Moreover, this happened not once or twice, but many times over the centuries, and continues to this day in many printed and electronic publications; the most recent case is in the second issue for 2008 of this journal. I will justify this harsh statement soon, after setting up the necessary background.
I wish I had LLM-built Lean formalisations in university, so much of the math in the slides had errors, and some professors are very bad and ungracious admitting it, while simultaneously rejecting requests for clarifications by saying "the proof is in the slides".
Of course Lean proofs are rarely a good way to understand proofs, but hopefully they can be used to generate more human understandable arguments.
> A few days earlier I had got an email from a professor in the maths department here at Imperial, expressing surprise that some of our graduate students were paying $200 per month to access models such as Sol and Fable. He said that he thought that these people were crazy. I did not immediately respond. But after meeting with Andrew I emailed the professor back and told him that in my opinion, any PhD student who was not paying $200 per month to access these tools was crazy. In fact during the workshop I learnt from Harvard PhD student Bryan Wang that Harvard were already giving free Fable access to all PhD students, post-docs and faculty at Harvard.
Yeah, given how much it accelerates grad students to produce meaningful output more quickly, why wouldn’t you make an investment of $2400/student/year. Seems like pennies overall.
Many graduate students view themselves as ethical beings, not machines that "produce meaningful output," and everybody here knows (useful) LLMs are indefensibly evil because of stolen training data and enormous environmental impact.
How do mathematicians view counter examples? Is it like an unexpected result in the physical sciences: annoying in the moment but potentially stupendously important as it reveals some inaccuracy in the current models? Or is it more like a bug report in coding… probably just, another little annoying detail?
I am sure finding holes in existing proofs is what counts as "another little annoying detail". Some people are probably relieved when they have failed to prove a hypothesis and someone finds a counterexample.
A large fraction of the problems assigned in the Ross Program were of the form "Prove or disprove, and salvage if possible." The rest were usually a calculation, meant to motivate a general proposition you would encounter soon after.
This is not new, most famous conjectures are believed to be true mainly because an already enormous brute force search was conducted finding no counter-example.
I wonder if at some point mathematicians will be over-flooded with proofs to check and eventually some over confident false claim will make it into math.
Maybe in the future the work of Mathematicians will be like the ones of SWEs with AI, check thousands of lines of AI generated proof and find the subtle errors
This will soon happen with theoretical physics, computer science, and everything which can be verified cheaply. Then, we will have long running projects augmented by agents for 2-4 years while AI companies are collecting data of human workflows. After that we will see AI being able to do those projects by themselves. This will lead to super fast human progress and cheap products. The price of things will be bound by energy and natural resources. Interesting times are ahead of us
46 comments
[ 0.19 ms ] story [ 58.9 ms ] threadIt's not easy to find a counterexample to the Jacobian conjecture, by any means – by which I mean to say that naïve brute-force search will take too long – but the scope of existing searches listed on Wikipedia[0] suggest that many tricks are already known, and that people just hadn't looked, systematically, for a counterexample in three variables before. Wikipedia writes:
> Tzuong-Tsieng Moh checked the conjecture for polynomials of degree at most 100 in two variables.[17][18]
where reference 17 is from 1983, and reference 18 is a preprint with no given date. Knowing very little about this problem, my impulse is to side with the unnamed faculty member cited in the article:
> [who] said to me that the fact that the counterexample was so easy to find just indicated that humans had not spent enough time thinking about the problem,
For context, the auto-generated counterexample is in three variables, has degree 7, and was discovered in 2026.
AI is only as good as a human mathemetician, which we don't have enough of? "just"?
https://en.wikipedia.org/wiki/Jacobian_conjecture
> The conjecture was first stated for two variables by Ludwig Kraus in 1884 [...] an example of a difficult question in algebraic geometry that can be understood using little beyond a knowledge of calculus.
> The Jacobian conjecture is number 16 in Stephen Smale's 1998 list of Mathematical Problems for the Next Century. It was notorious for the large number of published and unpublished false proofs that turned out to contain subtle errors.
One thing that he mentions that already quite surprising is that AI was able to autoformalize the Golod-Shaferevich theorem and proof.
Interestingly, Yitang Zhang of the twin-prime-conjecture fame spent 7 years working on the Jacobian conjecture under the advisor Tzuong-Tsieng Moh at Purdue. A key step in his thesis used a corollary of Moh's. It turned out that the corollary was incorrect. As a result, Moh refused to write any recommendation letter for Zhang, and Zhang couldn't find any teaching or research job and ended up spending years working at a Subway[1].
Imagine Zhag had ChatGPT in 1986 when he started working on the Jacobian Conjecture.
[1] Of course now this has become an inspiring story. That said, the story definitely invokes complex emotions. The best way to describe it is probably this Chinese poem, which I have no idea how to translate: 庾信平生最萧瑟，暮年诗赋动江关
On one particular Friday afternoon, he stated a conjecture that he hoped was true, and invited us to try to help him prove or disprove it. It was the sort of thing that he really wanted to be true; he liked things smooth and beautiful. I, on the other hand, hoped it was false as I like the weird and exceptional in mathematics. It was also the case that I had absolutely no command of the sort of machinery that one would use to prove such a thing, but I could certainly look for a counterexample.
I learned on Monday that he had spent the entire weekend trying and failing to prove it. I, on the other hand, had put all my energy into finding a counterexample and had one within an hour.
My single (quite small) contribution to mathematical research was a counterexample because it was all I could do. The story does illustrate that it can be helpful to have people with different tools, hopes, and motivations working on a problem, though. I was not, and will never be, even a shadow of that great mathematiciam I studied under, but on that occasion, I had reason to look in a different direction than he did.
If you have a spare hour or two, I'd encourage you to have a go at learning (however you learn best - I like just asking smarter people or robots stupid questions) what the maths means and why it's important.
And then once you feel like you have a vague grip on the principles, think about a problem in a domain you know a lot about. Try to see if the maths - and how it's changed our perception - could be used as a tool to solve that problem, or if the solution is analogous to a solution you could try in your own domain of expertise.
LLMs are good at speeding up, I think, the journey an idea has to take between "theoretical academic stuff for academics" and "a usable idea for regular people", because they increasingly allow you to ask an infinite number of stupid questions and give you (hopefully) reasonably good responses.
I've had loads of fun doing this today - specifically seeing if the idea this counter (from what I understand: a many-to-one conversion that kind of does and kind of does not preserve meaning) can tell me anything about the relationship between language and meaning.
I'm sure everything I've done today while mucking around has been the equivalent of a monkey with a typewriter (and Codex), but I think the huge breakthrough(s) you ask whether we're on the cusp of are relatively dependent on how many monkeys are throwing typewriters at problems they know a little bit about, after learning a bit about new ideas like this one. Historically, that's a really good way for broad cultural innovation to happen - distributed information applied across multiple domains by experts in them.
To be clear, I do like 'outcounterexampled' a lot!
[1] https://en.wikipedia.org/wiki/John_Henry_(folklore)
[2] https://en.wikipedia.org/wiki/Proofs_from_THE_BOOK
The most valuable things in mathematics are not beautiful proofs. We need more useful definitions. Actually coming up with useful definitions (and building good conjectures out of them - not even theorems, conjectures) is something LLMs have not yet tried to conquer.
> ..Even more unexpected is the fact that many expositions of this topic commit serious mathematical and logical errors. Moreover, this happened not once or twice, but many times over the centuries, and continues to this day in many printed and electronic publications; the most recent case is in the second issue for 2008 of this journal. I will justify this harsh statement soon, after setting up the necessary background.
Of course Lean proofs are rarely a good way to understand proofs, but hopefully they can be used to generate more human understandable arguments.
Yeah, given how much it accelerates grad students to produce meaningful output more quickly, why wouldn’t you make an investment of $2400/student/year. Seems like pennies overall.
1) I recommend the wonderful 1976 book Proofs and Refutations by Imre Lakatos.
2) There is a considerable list of books dedicated to counterexamples, e.g. in topology, probability, analysis, etc.
[1] https://en.wikipedia.org/wiki/Proofs_and_Refutations
[2] https://www.amazon.com/s?k=counterexamples
Maybe in the future the work of Mathematicians will be like the ones of SWEs with AI, check thousands of lines of AI generated proof and find the subtle errors