Agents basically did statistically guided brutforcing. There is no value in what they produced because it lead to no understanding of anything and most likely will hurt the field IMO
Corollary to the infinite monkey theorem: Any provable theorem can be found by a finite number of LLM agents with an infinite token budget in finite time.
As a side note, you can tell this wasn't written by an AI by the first sentence:
> mommy, I heard you got cooked! I heard that a robot solved the math problem you worked on for your whole career! OOF!
My 8yo talks exactly like that. I could totally imagine him saying this, the same way, at the dining room table.
I asked ChatGPT "pretend you're an 8/9 year old today. how would you insult your mom about having her job be replaced by an AI?", and the responses it offered were:
> “Mom, AI took your job because apparently even robots were like, ‘Yeah… we can do this better.’”
> “Mom, congratulations! You got replaced by a computer. Even Siri has a job now and you don’t!”
> “Mom, AI took your job? Dang. I guess even a robot looked at your work and said, ‘I got this.’”
> “Don’t worry, Mom. You can still be useful… like teaching the AI how to make my lunch.”
All of these seem to have a vaguely Millennial flavor, aside from being pretty awkward and mechanical roasts. Trust the children and linguistic drift to be the best AI detector.
I appreciate the counter argument, even as a chat bot skeptic.
However, there is the fact that you had to know how to poke the machine so it plucks the right vocabulary out of the training data.
There is very clearly no theory of mind: no inherent internalised modelling of how a human of a particular age thinks, speaks and what they do or do not know. These are the most obvious cracks in the “LLMs are (or will be) the superintelligence” narrative.
Mother dearest, it pains my effusive tendencies to inform you the untimely death of your life's work by the hands of a mere transistor-- moreover, my incontinence briefs have reached their capacity.
Eh I don't think the content is out of line with what an 8yo understands. Mine tells stories about bombs flying in the Russia/Ukraine war, which required a lesson about tact because some of his classmates are both Russian and Ukrainian refugees and this subject may be a little close to home (literally, his best friend's grandma lives in Ukraine right now).
You're probably still stuck with old beliefs from when all available AI checkers were snake oil. Pangram is, to my knowledge, the only AI checker today that actually passes accuracy assessments. See e.g. on their own site: https://www.pangram.com/blog/third-party-pangram-evals
(Incidentally, if you asked me a few years ago I'd have predicted that reliable AI detection was impossible. I still suspect that it's impossible in general and that Pangram would break as soon as companies figure out how to make LLMs stop all sounding the same - but until then, it'll work fine.)
I liked the metaphor of a climber teleported to the top of a fog shrouded mountain. And I agree that now that the teleporter exists, we need to use it to reach more peaks and explore. There's no going back to a world where AI doesn't exist.
Agreed! Specifically, compensation (monetary and reputational) for professional mathematicians was bundled into theorem proving--essentially, climbing the mountain. Now that a teleporter exists, we need to unbundle compensation.
I don't know what that means in practical terms, but I agree that's the issue.
That's the issue, but they can't come out and just say it, because that horse left the barn long ago for other professions. It's socially just fine to obsolete entire lines of work, and has been for centuries.
Maybe for a short amount of time until AI starts doing that too mathematicians could get monetary and reputational compensation for formulating the right questions.
>I have had this conversation with my PhD students yesterday. I am 100% sure that all of their problems can be solved by publicly-available models now (I solved a case of one myself as a test, it took 15 minutes). So the challenge for them is to see how much they can accomplish in their allotted period, and still pass a defence on at the end of it all. The PhD defence is going to become all about a test of understanding, not a test of quantity of publication.
Maybe they should target more ambitious results now that they can solve their other results in 15 minutes? This spirit of racing in academia is insane. Get in before the door closes.
While the developments are certainly difficult for new grad students, it should still be possible to do what many have done before: find an obscure enough problem that it’s extremely unlikely you get sniped. Worry about big problems after finishing the PhD.
Genuine question. How many big problems were solved by people who took the safe path for PhD? I would intuit there’s high correlation of unsafe PhD and big problem solvers.
for say FFT/integer multiplication or 3SUM, we have natural algorithms that have existed a long time with a given complexity (O(n \log n) and O(n^2), respectively). Given how long these natural algorithms have been the best algorithms we have, it is natural to conjecture they are optimal. Showing an O(n(\log n)^{.99999}) algorithm exists shows that these optimality conjectures are false.
Now, there are some critiques you can have of this. Namely, it is possible that these novel algorithms have significant trade-offs that make them almost never worthwhile in practice. "Fast" matrix multiplication algorithms are typically of this form. So perhaps this all points towards a deficiency in big O notation, which can be deceptive. But, for people who care about optimizing asymptotic complexity, it is still interesting.
to get some intuition about why this is such a big deal, look up the history of strassen's algorithm, which solved matrix multiplication in less than O(n^3). this was a truly stunning result because it seemed intuitively obvious that the output matrix had n^2 cells each of which was calculated via an independent O(n) loop over a row/column of the input matrices, so how could you do better than n^3. but once strassen proved that you could do some clever tricks and reduce the overall time to something less than O(n^3) it started an entire cottage industry of people getting better and better algorithmic bounds. the initial breakthrough was a qualitative one, independent of how much it improved things in numerical terms.
Tell that to the humans working on matrix multiplication who spent years of their lives getting it from n^2.3728596 to n^2.371866, only for openai to blow it away at n^2.25
This is a legitimate question, and it'll be interesting to see how these constants evolve in the future. In some cases, maybe it's true the AI did the minimum to break the barrier, and the best constants are much lower. It's still a notable result that the barrier was broken. But it's quite possible that there's not actually much more to improve.
Indeed, the bound of O(n log n) on integer multiplication was broken by 2^-182 in the results, and that improvement has already been improved, step by step, all the way to 2^-15 (unpublished and unverified, so far).
It's hard not to hold the original limit break causally responsible for the rest of it, though. If someone clocked a particle doing 299,792,500m/s, and it wasn't due to instrument error, that would justifiably be regarded as a giant leap in understanding.
> It feels like something written by someone who’s on psychedelics. So much unclear and doesn’t make sense. Lots of name dropping of previous work without discussing why it can be used despite impossibility results
> Basically the paper is so horribly written that it’s impossible to read it without AI help
That's interesting and haven't seen this in all the coverage of this event.
It sounds horrible to wade through - like trying to understand someone else's messy code that still produces the correct output.
It also aligns with the fear that these proofs present a risk to the ecosystem by out-competing attempts at more human-readable proofs. Perhaps though we end up with more math influencers who edit and annotate these proofs to bring them back to us.
I think that's ultimately a good thing. As proofs weren't supposed to be the point as stated by William Thurston long ago. Maybe now the focus can be more on better explanations and creating tools for growing understanding and intuition.
It's worthy of note that most humans, do not find most mathematicians understandable. As is frequently demonstrated in Calculus classes. Therefore it is arguable that even human produced results are not generally human understandable.
The ecosystem is (ahem) gated by hiring committees. There is no risk of AI replacement from the inside. "Replacement" is not even a possible movement. The funding for mathematics worldwide comes mostly from endowments, which are investment pools.
Why would anyone believe this (also) is not simply example N+1 of this is the worst it will ever be, as opposed to recognizing this as what will almost certainly prove to be an awkward moment, soon to be replaced by another order of magnitude of cleaner, clearer, more intelligible, etc.?
Ximm's Law: every critique of AI assumes to some degree that contemporary implementations will not, or cannot, be improved upon.
In that case, one would expect to see some progress in this direction, but AFAICT that hasn't shown up yet? If anything, it's getting worse, though that could just be the increasing scale and decreasing cleanup efforts.
Already the unit distance proof was substantially human-edited (per Thomas Bloom). Then with the ten problems from Astra you started getting the citation issues. Then Navier-Stokes was a rushed 160 pages with barely any citations, and some of the related papers were called (by their "authors") the ugliest mess they've ever seen.
And now here we are. At least it seems that mathematical ability and communication with a mathematical audience are independent skills, and progress in the first does not imply the second.
This doesn't surprise me much, given two analogies: 1) many smart people are nonetheless horrible lecturers. (You can't quite get the opposite extreme, since to explain math well you have to be able to do it.) 2) AI writing in general hasn't improved. The models have annoying verbal tics ("honestly") and have no sense of which part of what they say is obvious and which is relevant.
I'm thinking of somebody like Grant Sanderson (3blue1brown), doing pure exposition extremely well. For that you at least need to be able to work through examples, or to present why an intuitive approach might fail, and these things can be little theorems themselves. It doesn't have to be publishable in the current culture of novel results, but you do need a lot of competence with the tools.
Yes, Grant Sanderson might be a great example. I am not doubting the competence of the "good lecturers" I was referring to. But that is different from being able to introduce the big new concepts that give the big new results.
Math proofs need to produce the correct output correctly, which is not quite the same thing.
This looks like an AI IPO PR powerplay, because at this point the proofs haven't been checked and it may not be possible for a human to check them - because proofs should be clear, not horribly written and noisy.
The noise is suspicious because it's the difference between brute forcing and cognition. A human proof won't just be logically correct, it will be cognitively distilled and coherent. It may still take years to understand it, but the logical flow will be straightforward, not obfuscated.
You want the path through the maze to be as short as possible and the map to be as clear as possible.
This sounds like the opposite. There may be a genuine path through the maze, but if it's too convoluted and takes too long it will be impossible to confirm.
I think the next step is to demand that proofs either be human-scale or they prove that a human-scale proof is impossible and the machine proof is as good as it gets.
I suspect that's possible without tripping over the halting problem. (But I can't prove it.)
The post says there's a Lean certificate for this and other proofs ("some [...] not all of them").
> This looks like an AI IPO PR powerplay,
Interestingly, the post has actually also an argument for this:
> Experience has shown that, even now, there will still be people explaining in patronizing tones why none of this is real and none of it counts. If such people were capable of being impressed by anything that happens in the empirical world, of updating on anything, they would’ve already been impressed and already updated several years ago, long before things had reached the point of an actual Mathocalypse.
> So, they’ll say, maybe the alleged solutions are not solutions at all, but just “AI slop.”
Surely by the time of the IPO we will know whether the main results are correct, if only because a different AI will have produced a lean proof or found a logical flaw (the second case would be hard to verify but probably not impossible).
Also from what I can tell from the few fields I understand, the proofs aren't that long or complicated they are just terribly written.
Why should that make material difference to the IPO? What is the economic value of those results?
The entire US federal budget for math research is something like $100M annually. And mathematicians in other countries are hardly making bank either. How does one reconcile how the market has historically valued mathematics with the cash-strapped frontier labs ploughing so much money into that enterprise?
They're burying any doubt that the models are capable of superhuman performance on intellectual tasks. Neural nets aren't calculators, and were notably poor at mathematical reasoning tasks for a long time. Now they're not, and the labs are proving that by chewing through what would ordinarily be decades of progress in a month.
If the models can do this, they're almost certainly good at just about everything, because the reasoning and creativity required to solve these problems will translate. And even if they were only good at this stuff, that's still a tremendously valuable thing, because quantitative reasoning and analysis is the bedrock for many, many industries.
oAI is gunning for the largest IPO in history at this point, and they might actually get there.
> If the models can do this, they're almost certainly good at just about everything, because the reasoning and creativity required to solve these problems will translate.
The "then" in your "if-then" bears a heavy load. Why would society assign so little economic value to pure mathematics if the skills for proving math theorems translate to massive value in "just about everything"? Would you expect top mathematicians to cure cancer if you transplanted them from the math department to a medical research lab?
Do you think mathematicians are not already working on cancer research? There's quite a bit of heavy math in biostatistics, medical imaging, machine learning, etc.
I'd assume that the majority of people who study math take their skills and move onto some related STEM career that isn't pure math.
Because research mathematics is a subset of all quantitative work that gets done, but it's by far the most technically difficult subset. If the models can handle research grade math and produce ironclad proofs, they can probably handle the quantitative side of just about any discipline in a trustworthy fashion. Think about how many dinky spreadsheets have gone on to become critical tooling for large organizations. Even if you consider that many disciplines hide technically demanding work behind tooling (e.g. essentially no one is writing a stiffness matrix FE routine by hand), this model would be capable of writing a direct competitor from scratch to produce the same result.
All of STEM relies on mathematical analysis, and new models are now superhuman at that. And yeah, I'd go a step further and say that the reasoning and creativity required to solve cutting edge math problems probably does translate to other tasks like interpretation of the law, or medical diagnosis, or accounting, etc., for the same reasons that I think most top tier mathematicians would excel at those tasks were they so inclined.
That sort of worked for Eric Lander but as he moved higher up in the bureaucracy, he found himself suddenly having to manage people who weren't nerds like him. He was bad at that had to step down.
American mathematician Jim Simons was worth some $31 billion at the time of his death in 2024. The lack of economic value in pure math doesn't mean that applied math is of little value. Physicists and mathematicians with PhDs "sell out" to join Wall Street as quants and make a killing there, Jane Street is full of them.
> If the models can do this, they're almost certainly good
I would argue that "good at math" was a short hand for "good at X" because mathematicians were historically good at engaging with very complex ideas, distilling them and coming up with precise and concise answers they could validate by themselves.
Given that AI solutions are described as "psychedelical" and they rely on the outside source to validate the result, I don't think the same logic could apply to them.
I don't see anything stopping the models from getting to that point, though. So far we've gone from the models not being able to do math at all, to improving on progressively more difficult problems, to claims of "it's just a counterexample", to now producing constructive proofs and new mathematics on the hardest problems we currently have. All within less than two years.
Being able to do a task well and explain what you're doing in an organized way are fairly orthogonal, even in humans. I don't see why the skills the models currently have wouldn't transfer, even if the results look psychedelic. The next step would obviously be to improve trust by improving the way in which results are communicated.
> A human proof won't just be logically correct, it will be cognitively distilled and coherent. It may still take years to understand it, but the logical flow will be straightforward, not obfuscated.
Not a counterpoint, actually case in point, because:
>Mochizuki and a few other mathematicians claim that the theory indeed yields such a proof but this has so far not been accepted by the mathematical community.
We should consider the possibility that at some abstraction levels, we can safely stop chasing "clarity" or "coherence" which is circularly defined in such a way that it's capped by human processing power.
Developers and people in CS in general seem to have gotten used to the idea that most productive SWEs don't need to exactly know how to produce assembly or trace every branch prediction or even most of the optimization the CPU (or even their compiler) is running. Mathematicians will get there.
CS got that idea from mathematics. Theorems (with the definitions required to state them) are supposed to be self-contained units. Once the general consensus is that a theorem has been proven correct, people can use it without understanding the proof. Of course, people still want to understand how things work, and it often makes sense to understand them a couple of layers below the one you usually work at. But at some point, you should stop distracting yourself with irrelevant details and focus on your actual work.
Thing is that many of the theorems here are not useful work in and of themselves, but were posed as research problems because it wasn't clear how they could be resolved with current techniques, implying that the process of trying to find a proof might result in new techniques. It's those new techniques that are the actual goal, but if they can't be easily extracted because the proof isn't structured to enable this, that's a bit of a headache.
But it doesn't follow that these proofs - or any proofs - are automatically on the far side of that limit.
The human usefulness of a proof depends entirely on its human legibility. Much of the value of proofs is in inventing new techniques and concepts and having new insights into relationships. Occasionally you get some game changing insight into practical physics or engineering. But that's rare.
Without that, proving or disproving a conjecture is an excuse for new and original thinking.
Compilers are not the same problem. The point of code is to produce reliable-ish consequences from various possible inputs. It's not a creative exercise in logical consistency, which is what maths proofs are, ultimately.
> Developers and people in CS in general seem to have gotten used to the idea that most productive SWEs don't need to exactly know how to produce assembly or trace every branch prediction or even most of the optimization the CPU (or even their compiler) is running.
That's because there's a bunch of people who work on that stuff. Just because web developers don't care about it doesn't mean it doesn't exist. Utter magical thinking.
You seem to be confusing your quantifiers a little. (Not all SWEs must understand all CS) does not imply (there can exist parts of CS that no SWE understands).
> “All mathematicians are familiar with the concept of an open research problem. I propose the less familiar concept of an open exposition problem. Solving an open exposition problem means explaining a mathematical subject in a way that renders it totally perspicuous. Every step should be motivated and clear; ideally, students should feel that they could have arrived at the results themselves. The proofs should be “natural” in Donald Newman’s sense [13]:
> This term . . . is introduced to mean not having any ad hoc constructions or brilliancies. A “natural” proof, then, is one which proves itself, one available to the “common mathematician in the streets.””
If intelligence is compression, and these models are a different form of lesser intelligence than human, but being scaled up to brute force problems, then it makes sense the artifacts that produce (the proofs) would have worse compression than a human proof would.
In other domains I have seen first hand overwhelming evidence of how things that cause the AI to make mistakes also cause humans to make the same mistakes.
I wonder if the proofs being produced that are hard for humans to interpret are also hard for other LLMs to interpret.
In other words, I wonder if humans are still much better at compressing understanding into proofs than the best LLMs, and what it will take for LLMs to exceed them.
It kind of an explicit example of how the LLMs can be materially less intelligent than people, but still be more productive through scaling, and yet they also can't replace people because they are a categorically different kind of intelligence. It's like all the AI debates compressed into one example showing countwr-intuitive answers.
idk if i'd even say they're "lesser", just very different. so they look like gods/babies depending on what they're doing because we anthropomorphise them.
In terms of synapse density, neuron diversity, energy efficiency, memory access, etc. they are orders of magnitude lesser.
My mental model - for better or worse - is that intelligence has both shape and area, and LLMs are orders of magnitude smaller area but very different shape, and they have more intelligence area in the kind that humans have lesser of.
So yes very different, more in some material ways and lesser in others, but in total intelligence are still orders of magnitude lesser.
My gp comment was acknowledging that when you scale up many instances / brute force problems it confuses that "total area" claim a bit.
To follow the anthropomorphization... 1000 toddlers may have more total intelligence than a grown man, but does that matter?
The problem with these discussions probably/usually fold into differing/loose definitions of intelligence.
perhaps the chat-based ux has sort of fooled us into comparing these things to human intellegence. we don't really do this with chess, or other forms of ai, nor computers at large.
> I think the next step is to demand that proofs either be human-scale or they prove that a human-scale proof is impossible and the machine proof is as good as it gets.
Who do we demand this from? The AI companies? Or the mathematicians who are worried they will have nothing left to do?
> I think the next step is to demand that proofs either be human-scale or they prove that a human-scale proof is impossible and the machine proof is as good as it gets.
In 1976, the proof of the Four Color Theorem was controversial because it was done with a computer examining over 1000 cases by brute force and was essentially not comprehensible by humans. But mathematicians ended up accepting it. So mathematics has a 50-year precedent of not requiring human-scale proofs. How is the current situation different?
(Disclaimer: Apologies if this sounds dismissive or argumentative. I genuinely think that the Four Color Theorem should play a role in these discussions and suspect that many people are unaware of the controversy over it.)
I don't know much about this controversy, but looking at the Wikipedia page for the Four Color Theorem, it looks like since the initial Appel-Haken result, mathematicians have continued to work on the Four Color Theorem to try to find a simpler proof. So did they really accept it?
There are a lot of proofs for Pythagoras' Theorem and other theorems.
There are mappings between complex numbers and 2D matrices allowing problems to be solved in either domain.
There's research into The Langlands Program looking to connect number theory and harmonic analysis.
There's research into Category Theory looking to define core concepts and relate them do different fields so that results in one field can be applied to another due to equivalence.
> I think the next step is to demand that proofs either be human-scale or they prove that a human-scale proof is impossible and the machine proof is as good as it gets.
> I suspect that's possible without tripping over the halting problem. (But I can't prove it.)
I doubt that it is. For the language of proofs to be powerful enough to be able to
express an arbitrary proof it would have be Turing Complete. Proving that a proof
is the smallest proof of a given concept (i.e. there is no smaller human understandable proof)
would then be proving the minimality of a program in a Turing Complete language.
> The noise is suspicious because it's the difference between brute forcing and cognition. A human proof won't just be logically correct, it will be cognitively distilled and coherent. It may still take years to understand it, but the logical flow will be straightforward, not obfuscated.
This is basically how all AI approaches appear to work. They solve the problem you give them (in many cases) but in a very over-complicated way.
It's like they can't step back from the problem and realise that they need to simplify to make it work better. Nope, just keep hammering more code/proofs against the problem and eventually you'll hit the goal.
> It's like they can't step back from the problem and realise that they need to simplify to make it work better. Nope, just keep hammering more code/proofs against the problem and eventually you'll hit the goal.
There is nothing wrong with that approach if it accomplishes the goal faster, i.e. you can work at the speed of an AI.
> There is nothing wrong with that approach if it accomplishes the goal faster, i.e. you can work at the speed of an AI.
Sure, and god knows I've done the same myself when under time pressure (so similar to an AI in RL training, just get the goal).
The part that's lacking (that humans are much better at right now) is the meta-cognition around this works, but it's far more complicated than it needs to be.
Potentially because LLMs have larger "working memory" than humans, they can avoid this for a while, but once it blows up the context they fail in much worse ways than a human would.
Hmmm, this feels like a good metaphor, I need to write about this at some point.
> the difference between brute forcing and cognition
There’s not really a clear distinction between these things, in my opinion. Problem solving (and intelligence?) is a mix of search and compression. We like solutions that are elegant (high compression, simple search). But often what appears elegant to some is harder to appreciate for those without the same background knowledge or even the same amount of mental bandwidth (if you’ve ever worked with someone simply much, much smarter than you, you may intuit this!).
Isn't that just the default experience with AI these days? In small enough scale, AI models can express their ideas clearly. But the larger and more complex the ideas are, the less suitable the outputs are for human consumption. I guess AI models think too different from humans, and nobody has trained them to communicate complex ideas in the way human experts in that particular topic expect.
> Basically the paper is so horribly written that it’s impossible to read it without AI help
This basically describes every single PR at work for the past year. Diffs of 10k+ paragraphs of comments saying nothing. Just rubber stamp and move on, nothing else you can do.
Meanwhile you can use the model to help you out as Scott comments "Just now, however, Dana tells me that she’s been asking Astra all day to explain the new proof of the UGC to her and it’s been doing an amazing job and she’s starting to understand the construction."
This makes sense and looks exactly what something would look like that is smarter than us.
If it is correct it is making inferences that we can't see. At a stretch even working in more dimensions than our three dimension limited brains.
If it were submitted to a journal, it would be outright rejected. We should give the peer review process -however flawed it is- some credit here. Dumping hundreds of AI-generated preprint articles on the field should not be allowed.
No. Letting some tiny handful of individuals selected by a tiny set of review-delegators review this is immeasurably worse than just releasing it publicly and letting anyone with the sufficient expertise review it. You know nothing of how peer review actually works, or have not thought about why that process would be only harmful in this case.
Normal peer review works by submitting the article to the appropriate journal, suggesting some preferred reviewers that have expertise in the subject, after which the rest of the process ideally is then orchestrated by an experienced editor.
If a journal receives a paper that is unreadable, it is outright rejected with the comment that it should be made readable, regardless the results in that paper. What is the point of having results if you are unable to convince your audience of those results? What is harmful about this process?
Try thinking for just two seconds about what your process would result in, given the scale and amount of results here. You are basically asking for these results to become available to all experts only months / years from now, only after some select experts are chosen and first get to read these results. If there is any error or bias in that selection process, the whole thing is delayed even further or lost to the file drawer entirely. Or, since you say these are all unreadable, and should be rejected, you are in fact arguing these results should never be released at all.
Which is why it is obvious post-publication peer review is the only sane way to handle this. Formal peer review is not even remotely up to the task here.
For what it’s worth I’ve heard from several different ex mathematician colleagues that found papers in the stack that was adjacent to their work or solved a problem they were acquainted with in their career, and they all said the results are actually quite readable.
Maybe there’s some variance or it depends on the reader. That said, like with a lot of other complaints about AI: human papers can be poorly written and poorly explained too. That’s always been the case. And sometimes a proof is just complicated and hard to expose nicely.
> It sounds horrible to wade through - like trying to understand someone else's messy code that still produces the correct output.
That's what a lot of us do nowadays, but instead of maths, it's code that looks sensible on the surface, but when you try to understand it's some kind of "alien" logic , names don't really make sense, etc.
It’s been said every time about these LLM math results – but of course you don’t see it in press releases or articles written based on said press releases.
As mentioned in the first paragraph, the proofs come with Lean certificates. Lean is a theorem-proving language. If Lean accepts the proof, it’s legit.
There's a proposition written in Lean that we can read and understand, and a proof of that statement that Lean can verify, despite it being a huge unreadable mess. We only need to check the proof isn't using an obvious cheat ("sorry") or abusing some Lean bug.
They already had to withdraw three of the manuscripts and correct several others. Who knows how much broken stuff there's in there; unfortunately they were too lazy to check themselves.
What I don’t get: if someone that’s apparently not the average mathematician has so much trouble reading and understanding the paper, then why is everyone so sure that GPT really delivered a genuine proof? This is an honest question; I really don’t understand that.
they are Lean certified, so in theory they should work. It's automated programming language for proving math, but it doesn't mean that you can't make mistakes there, although way less likely
It's a well-established fact that Llama simply can't write comprehensible maths, and it's one of the major reasons that OpenAI are catching the flak they are. Dumping the manuscripts in their current form is a sign of laziness and incompetence.
And you're right, it's not really a theme around here, but there aren't that many mathematicians around HN. Check one of the maths forums and it'll be a common point of complaint.
That's what Buckmaster and Alpöge said about the Navier–Stokes equations paper: they had the meat of their own paper ready by the beginning of August thanka to LLM assistance, but it was so messy they spent the whole month turning that into a paper worth publishing. Then OpenAI heard about their work, spent a few millions of dollar of compute to beat them (likely exploiting their own work with ChatGPT in the process) and released something.
When all you value is being the first, you have not time producing useful papers.
The proofs could be optimized for human consumption.
OpenAI could optimize each proof for fewest theorems, low branching, or short dependency chains.
OpenAI could have an adversarial network enforce that proofs look human.
OpenAI could spend a couple more machine hours per paper with the prompt "edit for clarity".
I think OpenAI wants the outputs to be unreadable.
If outputs "too advanced for human comprehension" become the norm, then every interaction requires tokens.
If someone can buy a day pass, get what they need, and leave, then there's no recurring revenue stream.
I use/train models for financial markets.. equities analysis... some of the absolutely nonsensical sounding stuff that comes out of (even frontier) models on long running tasks will do your head in. The real skill in AI is wading through material that is akin to reading case law for a first year law student and trying to figure if it makes sense. Ie, evals don't actually work well for most long running task use cases.
Guy is misrepresenting AGMAI, talking about the Simons Institute (AI boosters), Quanta (AI boosting magazine from the Simons Foundation), Scoot Alexander (!) and Steven Pinker (!).
The he puts up preemptive straw man arguments against doomers. His blog has become a joke.
I’m still wondering why UT Austin is letting him teach a course (CS395T AI Alignment Theory) so completely outside his field of expertise (Quantum Computing).
I don't think Aaronson's post is swiping at AI doomers at all. The post's one use of "doom" is to call the position that math doesn't matter and that AI will never breach the realm of true human creativity as a "doomed worldview". The post later praises Scott Alexander's argument (for taking AI x-risk seriously, a position associated with "AI doomers") against Pinker.
Imagine if a team of mathematicians from OpenAI had gone on a university tour, gave demos of how powerful their models were for math research, and then gave mathematicians access to the model. Empower others rather than drop 700+ discoveries on GitHub that were made using a model only they have access to.
People might feel differently about AI if they were a part of the changes rather than being a helpless spectator.
The reaction/fallout would have been substantially improved even if OpenAI had just made a commitment to not scoop external researchers using an internal model until X months after the model had been made available to the public.
That would have given grad students who've been grinding towards a PhD for years a fighting chance to see if they could leverage the model to push their work forward, rather than watching years of work potentially turn to dust via a tool they don't even have access to.
It wouldn't delay the progress of mathematics by any meaningful amount in the long run for OpenAI to take this approach, and would help somewhat to preserve the health of mathematics as a field. Without it, the motivation for any young mathematician to devote years to a new problem must be sapped knowing there's an uneven playing field... an OpenAI team with access to colossal tools months before they'll ever be able to get access, willing to scoop them as soon as they can. (It's not the same as competing with other human mathematicians, where everyone has roughly the same arsenal of tools, constrained by human intelligence.)
> But it also appears that no human has understood just about any of these proofs yet
Has anyone verified any of the proofs produced by OpenAI or is everyone just assuming that it just be true because the Lean code checks out? Couldn’t the Lean code just be formulated incorrectly?
For example, the statement of e.g. Fermat's last theorem in Lean should be understandable to anyone who played The Natural Number Game [0] and knows a bit of mathematics and programming. For the proof, you trust the compiler.
The statement of other theorems can be much more delicate, and the Lean formalization may require an extensive introductory section which will need to be carefully checked.
Then there are the cases where no Lean formalization is currently available, and all we have right now is an often impenetrable pdf in the OpenAI repo. I would not at all be surprised if some of those claims contained logical gaps.
Time will surely tell, but there are certainly doubts and lots people are very busy checking these results.
> is everyone just assuming that it just be true because the Lean code checks out?
Kinda? It's only been 24 hours since they dumped 722 manuscripts on the world, most of which are apparently basically unreadable, and only some of which come with a Lean proof, which in itself is not a joy to read afaik.
Time will tell if OpenAi is doing what many people are right now: superficially checking the slop, and throwing it to other humans for deep analysis and understanding.
In other words, they save effort by wasting the effort of others.
The others are not in any way compelled to do that deep analysis and understanding. They're doing it because they find real value in the material produced.
If the problem statements in lean are correctly formalized, I think we can be pretty confident the proofs are correct, which does not mean their natural language counterparts are faithful representation of the proofs though.
I'm however puzzled by the number of proof claims without lean proofs. How does OpenAI have confidence in those, especially if, as noted in the blog posts, the papers are very hard to read?
> what problems do these people reorient towards after this?
The same problem almost every person on earth is going to have to reorient to in the next decade, which is: how do we eat and stay housed when we have no real economic value?
To some extent this was discussed in the article, and in a way I think their goal is actually the same as it was: become the first human to understand something.
It's just that we lost one of the important ways to demonstrate understanding.
I've asked my students whether they still want to learn maths even if there will be a machine that will answer any question instantly and they will be homeless. They said yes.
(To my credit, I have warned them since more than a year ago that we will reach this point.)
your students are crazy (neutral term); no one should learn maths if the condition is that they will be homeless and exposed to the elements. the will to subvert the hierarchy of needs is commendable
But are any of them going to be able to do any significant amount of studying maths without being homeless?
Why not? Maths studying correlates with high intellect, high intellect correlates with competitiveness in the job market. So their chances of finding a job that would allow to have much of free time (for doing math) without being homeless are pretty good.
They will continue to prove theorems and make discoveries, except now they will have AI to help them so hopefully progress will be faster. At the same time, new challenges will open up, for example how do you verify what the AI is doing and how do you explain it.
Math isn't about collecting random theorems, progress in math is about gaining understanding of new systems, and the theorems are guideposts to aid in that understanding.
You can prove 1000 theorems and not really increase any understanding about a subject, but gain knowledge of 1000 random facts. For example, I can write down some complicated equation and ask you "does this have a solution in the integers"? And if you do a maze of very complex and tedious algebra to show that there is a solution, you would have proved a theorem, but you would not have done much to move math forward at all.
On the other hand, if you introduce some completely new technique, say you take my equation and turn that into an algebraic surface, and then you count some special curves that live on this surface using geometric ideas, and then you show that if the number of such curves is odd, there must be a solution in the integers, and in this specific case, it is odd, so there is a solution -- well, then you have really pushed math forward and people will celebrate your proof, even though no one really cares if the equation I wrote down has a solution in the integers.
For example, there is a long history of failed attempts to prove Fermat's last theorem driving algebra and number theory forward by introducing the concept of ideals, for example, and this concept ended up much more important than whether Fermat's theorem is true or false, which is not too much more than a piece of trivia.
Or for example, the recent proof of the Poincare conjecture relies on the machinery of the Ricci flow introduced by Richard Hamilton, who then applied it to solve a number of open problems, but Perelman was able to take it even more forward to solve Poincare. So Ricci flow was massively important machinery.
For this reason, we celebrate people like Gromov, who didn't really prove that many theorems but introduced amazing machinery -- for example, the h-principle, or Gromov Compactness -- these were ideas and math is about the ideas. The ideas are then applied, using laws of logic, to form theorems.
So mathematicians will need to mine these proofs to see if there are any new techniques - new machinery - being introduced, or if the AI just used the existing machinery more efficiently. Here too, we are just looking at AI as a form of search, which it is really good at, since there are so many thousands of papers and so many ideas, that there might be a connection between two areas that lead to a solution and the human mathematician, not knowing all known results, can't make that connection. In the future, we may wonder how anyone did math without AI, much like we would wonder how anyone can be a writer without access to a dictionary or reference work. Is the AI just searching through a catalogue of known ideas and connecting them or is the AI coming up with genuinely new stuff like Ricci flow or the h-principle?
What is interesting is seeing whether we can get AI to actually discover new machinery for us. That would be huge.
And then we need to find efficient ways to detect these ideas and describe them.
Really this is very exciting and opens up whole new workstreams for mathematicians.
They may need to raise their game and reorientate if they haven’t already to a greater understanding of programming to augment their mathematical ability.
You can certainly do that - but it's quite the break from tradition to release a paper in the state described. Why they did is a really interesting question! It may be that AI math requires approaches that humans don't find intuitive and what you are describing is actually counter productive (because, in summarizing the work in a way humans understand, you're removing the context an AI would use to further the work an AI did). It also might be that OpenAI could have done that and chose not to - or maybe they tried and this was the best they could do. No matter what I don't think anything about how to react to a paper being released in this state is obvious.
> in summarizing the work in a way humans understand, you're removing the context an AI would use to further the work an AI did
The "summarization" process can be multi-step. Initial steps develop new frameworks and prerequisite concepts. Later steps build upon these frameworks and concepts. Some can be simplified too if this can be done without loss of fidelity. The summarization comes last, and is meant to preserve sufficient context.
> I don't think anything about how to react to a paper being released in this state is obvious.
If it helps, exaggerate the condition a bit by imagining coming across a repository of alien knowledge.
> Some can be simplified too if this can be done without loss of fidelity.
If you don't understand why the paper is in its current form, how do you know what changes would damage the fidelity for the AI? I think your example of alien knowledge is misleading because it suggests the aliens don't need to understand the info - but in this case we intend to keep using AIs to expand this work. So if you transform it into something that humans think makes more sense without understanding why it's in a form that we hate now - you risk losing something in the form that you don't understand.
AI is producing works that are so poorly written/explained that it requires AI to even parse the results, which in turn produces explanations that are still confusing. In fact, it seems the humans are required to understand and explain the results, and the fact that they need to use AI to do so is a shortcoming of AI.
And if humans decide to give up on mathematics because the process of using the machine is so tedious, then there will be no value in automated theorem proving.
I am sorry but your resistance will fail utterly as long as AI itself can understand prior AI works. This is the only requirement that need be met. If it helps, imagine that the work is by a culturally distinct alien race in an alien language, and if it isn't there now, it's might get there soon.
> resistance will fail utterly as long as AI itself can understand prior AI works
That's not the issue in the long term. Look further: where did these questions come from? They came from humans who had been exploring mathematics and noticed interesting patterns that cropped up again and again.
The people who posed these questions had a profound understanding of the background maths that led to them. Me - I don't. Take one of the results he quotes: "Positive solution to the Unitary Synthesis Problem". He may as well be one of your alien races yabbering to me in an alien language. The question is meaningless to me. The answer the AI came up with, correct though it may be, is equally meaningless. You may as well have told my dog how to find eigenvalues.
If we let AIs do all the understanding so we know nothing about math, there will be no human understanding that lets us find new questions, or understand the value of any questions AI might pose and find the answers to.
Progress in science assumes a certain level of signal to noise ratio in papers. Decreasing the signal to noise ratio in published results will eventually overwhelm the ability to successfully disseminate newly obtained knowledge. This is already happening in the open source community with maintainers being pushed to the point of burnout by AI slop. The same thing is now beginning to happen in other fields.
This is only going to get worse in the foreseeable future.
In certain fields if you are talented and just starting out, you have better do any work that’s important to you offline for privacy/pirate reasons I really can’t say that enough, those AI model companies are not even remotely your friend.
> Experience has shown that, even now, there will still be people explaining in patronizing tones why none of this is real and none of it counts. If such people were capable of being impressed by anything that happens in the empirical world, of updating on anything, they would’ve already been impressed and already updated several years ago, long before things had reached the point of an actual Mathocalypse.
Also a ton of comments in this thread: breathless frothing hype declaring mathematics is over and assuming these proofs are exactly what they claim they are at face value, giving the company with a vested interest in everyone unquestioningly believing this is all real every conceivable benefit of the doubt
Reading that section and the linked essay was kind of depressing. It reads like the most important part about the AI apocalypse is who called it first.
Heaven forbid, as a person who is apparently "[in]capable of being impressed by anything that happens in the empirical world", that I should patronise anyone!
My dislike for AI is entirely orthogonal to it's ability.
My dislike lies in how it devalues skill + knowledge acquisition, disrupts art/software communities, and that it's now bloody impossible to tell the difference between someone who actually knows something from someone who merely regurgitates shit from their chatbot.
> my 9-year-old son was taunting my wife… “mommy, I heard you got cooked! I heard that a robot solved the math problem you worked on for your whole career! OOF!”
Usually 9 year olds imitate adults when they regurgitate such words in these circumstances. What a sad state of affairs.
Aren’t there dozens of proofs of the Pythagorean theorem? The goal isn’t to just “prove” but create something well written and intuitive to the average practitioner. And by gaining a deeper understanding we can ask better questions.
But up until now, the mathematics community has valued the "prove" part much more highly than the "deliver an insight" part. Mostly because with a little work they went hand in hand.
And going from zero proofs to one proof (even a sloppy one) is a big deal regardless of whether it was written by AI or a human.
Getting recognition is a fundamental human motivation to work on hard problems. Being first is a really important part of getting recognition, measuring by the last few hundred years.
You might as well say "the obsession with sex has always held humanity back". Maybe, but it's complicated...
Something that often comes out is "it is about the journey, not the destination".
Many math problems are practically useless if you only care about the answer, the millennium prize about the Navier-Stokes equation is such a problem. The solution makes no physical sense, real life fluids don't follow the Navier-Stokes equations in such extreme conditions. But in the process of finding the solution, we may get insight into what will end up being really useful. The big mess that OpenAI produced is the solution no one really cared about, but it didn't deliver much of what people actually wanted.
One reason it is sometimes seen negatively despite being at least something is that it broke the incentive. Without the million dollar prize and with only the privilege of being second, people are much less likely to go for the insightful solution.
You are wrong if you consider academic incentives, funding, human nature(reproduction/survival) and capitalism.
It would be fantastic if university and science was like "here is 100M$, play around and develop some 'understanding'".
However the reality is that human societies are hierarchical and currently capitalistic which implies value creation and status building.
1) the funding bodies/agencies need proof of value that you're using the resources meaningfully to be able to assign resources
2) Humans are status seeking, power seeking, resource seeking and sexual reproduction seeking. If you hold a lot of power and make decisions, you have more of all of the above.
The short story about the digital animals ("The Lifecycle of Software Objects") is an interesting read in light of AI advancement. It is incredible to think that when this story was written, all concepts described were sci-fi, whereas current technology is more advanced than the AIs in the book.
Have you learned something from every Fields medalist's research? If so you are a member of the extreme mathematical elite and you should probably just dig into the results yourself.
Yes, there was controversy around the Navier-Stokes result. But here are >300 problems and no corresponding >300 complaints of theft. Things are indeed moving really fast, and it's hard to keep up even as someone folrowing this with more obsession than would be healthy. Maybe somebody should keep a running short summary of The Situation...
How do we know that these "internal models" are not just half computer and half a giant team of mathematicians?
How do we know that OpenAI actually came up with these solutions and didn't steal them from outside researchers?
(also answered similarly to another comment; this is a common question)
There are suddenly many new solutions to problems that have resisted sustained attacks (e.g. the Uniform Games Conjecture as detailed in TFA at some length). Where do you think they are coming from? Why is there suddenly a bunch of results to be stolen?
319 comments
[ 3.2 ms ] story [ 33.6 ms ] thread> mommy, I heard you got cooked! I heard that a robot solved the math problem you worked on for your whole career! OOF!
My 8yo talks exactly like that. I could totally imagine him saying this, the same way, at the dining room table.
I asked ChatGPT "pretend you're an 8/9 year old today. how would you insult your mom about having her job be replaced by an AI?", and the responses it offered were:
> “Mom, AI took your job because apparently even robots were like, ‘Yeah… we can do this better.’”
> “Mom, congratulations! You got replaced by a computer. Even Siri has a job now and you don’t!”
> “Mom, AI took your job? Dang. I guess even a robot looked at your work and said, ‘I got this.’”
> “Don’t worry, Mom. You can still be useful… like teaching the AI how to make my lunch.”
All of these seem to have a vaguely Millennial flavor, aside from being pretty awkward and mechanical roasts. Trust the children and linguistic drift to be the best AI detector.
However, there is the fact that you had to know how to poke the machine so it plucks the right vocabulary out of the training data.
There is very clearly no theory of mind: no inherent internalised modelling of how a human of a particular age thinks, speaks and what they do or do not know. These are the most obvious cracks in the “LLMs are (or will be) the superintelligence” narrative.
Reminds me of the memes with the little girl making astute comments about the patriarchy to her father.
(Incidentally, if you asked me a few years ago I'd have predicted that reliable AI detection was impossible. I still suspect that it's impossible in general and that Pangram would break as soon as companies figure out how to make LLMs stop all sounding the same - but until then, it'll work fine.)
We are quickly moving to a world where all symbolic and numeric reasoning for economic purposes is performed by AI.
I don't know what that means in practical terms, but I agree that's the issue.
>I have had this conversation with my PhD students yesterday. I am 100% sure that all of their problems can be solved by publicly-available models now (I solved a case of one myself as a test, it took 15 minutes). So the challenge for them is to see how much they can accomplish in their allotted period, and still pass a defence on at the end of it all. The PhD defence is going to become all about a test of understanding, not a test of quantity of publication.
Also, Ted Chiang's 2000 short story "Catching crumbs from the table" <https://www.reddit.com/r/singularity/comments/1wzu5gf/this_m...>.
I guess it deserves respect as progress, but it just rubs me the wrong way. Like the machine did the absolute minimum to beat the previous mark.
Now, there are some critiques you can have of this. Namely, it is possible that these novel algorithms have significant trade-offs that make them almost never worthwhile in practice. "Fast" matrix multiplication algorithms are typically of this form. So perhaps this all points towards a deficiency in big O notation, which can be deceptive. But, for people who care about optimizing asymptotic complexity, it is still interesting.
https://hideoushumpbackfreak.com/algorithms/algorithms-stras...
-Boaz Barak
― Alfred North Whitehead, "An Introduction to Mathematics" (1911)
> Basically the paper is so horribly written that it’s impossible to read it without AI help
That's interesting and haven't seen this in all the coverage of this event.
It sounds horrible to wade through - like trying to understand someone else's messy code that still produces the correct output.
It's worthy of note that most humans, do not find most mathematicians understandable. As is frequently demonstrated in Calculus classes. Therefore it is arguable that even human produced results are not generally human understandable.
- Feynman
Why would anyone believe this (also) is not simply example N+1 of this is the worst it will ever be, as opposed to recognizing this as what will almost certainly prove to be an awkward moment, soon to be replaced by another order of magnitude of cleaner, clearer, more intelligible, etc.?
Ximm's Law: every critique of AI assumes to some degree that contemporary implementations will not, or cannot, be improved upon.
Already the unit distance proof was substantially human-edited (per Thomas Bloom). Then with the ten problems from Astra you started getting the citation issues. Then Navier-Stokes was a rushed 160 pages with barely any citations, and some of the related papers were called (by their "authors") the ugliest mess they've ever seen.
And now here we are. At least it seems that mathematical ability and communication with a mathematical audience are independent skills, and progress in the first does not imply the second.
This doesn't surprise me much, given two analogies: 1) many smart people are nonetheless horrible lecturers. (You can't quite get the opposite extreme, since to explain math well you have to be able to do it.) 2) AI writing in general hasn't improved. The models have annoying verbal tics ("honestly") and have no sense of which part of what they say is obvious and which is relevant.
This looks like an AI IPO PR powerplay, because at this point the proofs haven't been checked and it may not be possible for a human to check them - because proofs should be clear, not horribly written and noisy.
The noise is suspicious because it's the difference between brute forcing and cognition. A human proof won't just be logically correct, it will be cognitively distilled and coherent. It may still take years to understand it, but the logical flow will be straightforward, not obfuscated.
You want the path through the maze to be as short as possible and the map to be as clear as possible.
This sounds like the opposite. There may be a genuine path through the maze, but if it's too convoluted and takes too long it will be impossible to confirm.
I think the next step is to demand that proofs either be human-scale or they prove that a human-scale proof is impossible and the machine proof is as good as it gets.
I suspect that's possible without tripping over the halting problem. (But I can't prove it.)
> This looks like an AI IPO PR powerplay,
Interestingly, the post has actually also an argument for this:
> Experience has shown that, even now, there will still be people explaining in patronizing tones why none of this is real and none of it counts. If such people were capable of being impressed by anything that happens in the empirical world, of updating on anything, they would’ve already been impressed and already updated several years ago, long before things had reached the point of an actual Mathocalypse.
> So, they’ll say, maybe the alleged solutions are not solutions at all, but just “AI slop.”
Also from what I can tell from the few fields I understand, the proofs aren't that long or complicated they are just terribly written.
The entire US federal budget for math research is something like $100M annually. And mathematicians in other countries are hardly making bank either. How does one reconcile how the market has historically valued mathematics with the cash-strapped frontier labs ploughing so much money into that enterprise?
If the models can do this, they're almost certainly good at just about everything, because the reasoning and creativity required to solve these problems will translate. And even if they were only good at this stuff, that's still a tremendously valuable thing, because quantitative reasoning and analysis is the bedrock for many, many industries.
oAI is gunning for the largest IPO in history at this point, and they might actually get there.
The "then" in your "if-then" bears a heavy load. Why would society assign so little economic value to pure mathematics if the skills for proving math theorems translate to massive value in "just about everything"? Would you expect top mathematicians to cure cancer if you transplanted them from the math department to a medical research lab?
I'd assume that the majority of people who study math take their skills and move onto some related STEM career that isn't pure math.
All of STEM relies on mathematical analysis, and new models are now superhuman at that. And yeah, I'd go a step further and say that the reasoning and creativity required to solve cutting edge math problems probably does translate to other tasks like interpretation of the law, or medical diagnosis, or accounting, etc., for the same reasons that I think most top tier mathematicians would excel at those tasks were they so inclined.
I would argue that "good at math" was a short hand for "good at X" because mathematicians were historically good at engaging with very complex ideas, distilling them and coming up with precise and concise answers they could validate by themselves.
Given that AI solutions are described as "psychedelical" and they rely on the outside source to validate the result, I don't think the same logic could apply to them.
Being able to do a task well and explain what you're doing in an organized way are fairly orthogonal, even in humans. I don't see why the skills the models currently have wouldn't transfer, even if the results look psychedelic. The next step would obviously be to improve trust by improving the way in which results are communicated.
https://en.wikipedia.org/wiki/Inter-universal_Teichmüller_th... seems like a counterpoint, but IANAM. (I am likely cherrypicking the far end of the bell curve re: straightforward here)
>Mochizuki and a few other mathematicians claim that the theory indeed yields such a proof but this has so far not been accepted by the mathematical community.
Proof can't be understood, proof doesn't matter.
Someone at OpenAI, please, work on this.
There is ongoing AI assisted work on this, and precisely the problematic gap in the theory (around Corollary 3.12).
As of now, a required step appears to definitely be missing, but it's not clear if this is a genuine / unrepairable defect.
https://zen.ac.jp/news/zmcpostevent0331e
https://github.com/lana-agents/iut
https://github.com/LANA-Project/genl
Developers and people in CS in general seem to have gotten used to the idea that most productive SWEs don't need to exactly know how to produce assembly or trace every branch prediction or even most of the optimization the CPU (or even their compiler) is running. Mathematicians will get there.
But it doesn't follow that these proofs - or any proofs - are automatically on the far side of that limit.
The human usefulness of a proof depends entirely on its human legibility. Much of the value of proofs is in inventing new techniques and concepts and having new insights into relationships. Occasionally you get some game changing insight into practical physics or engineering. But that's rare.
Without that, proving or disproving a conjecture is an excuse for new and original thinking.
Compilers are not the same problem. The point of code is to produce reliable-ish consequences from various possible inputs. It's not a creative exercise in logical consistency, which is what maths proofs are, ultimately.
That's because there's a bunch of people who work on that stuff. Just because web developers don't care about it doesn't mean it doesn't exist. Utter magical thinking.
Chow, T. Y. (2008). A beginner’s guide to forcing (arXiv:0712.1320). arXiv. https://doi.org/10.48550/arXiv.0712.1320
> “All mathematicians are familiar with the concept of an open research problem. I propose the less familiar concept of an open exposition problem. Solving an open exposition problem means explaining a mathematical subject in a way that renders it totally perspicuous. Every step should be motivated and clear; ideally, students should feel that they could have arrived at the results themselves. The proofs should be “natural” in Donald Newman’s sense [13]:
> This term . . . is introduced to mean not having any ad hoc constructions or brilliancies. A “natural” proof, then, is one which proves itself, one available to the “common mathematician in the streets.””
https://en.wikipedia.org/wiki/Nothing-up-my-sleeve_number
?
In other domains I have seen first hand overwhelming evidence of how things that cause the AI to make mistakes also cause humans to make the same mistakes.
I wonder if the proofs being produced that are hard for humans to interpret are also hard for other LLMs to interpret.
In other words, I wonder if humans are still much better at compressing understanding into proofs than the best LLMs, and what it will take for LLMs to exceed them.
It kind of an explicit example of how the LLMs can be materially less intelligent than people, but still be more productive through scaling, and yet they also can't replace people because they are a categorically different kind of intelligence. It's like all the AI debates compressed into one example showing countwr-intuitive answers.
My mental model - for better or worse - is that intelligence has both shape and area, and LLMs are orders of magnitude smaller area but very different shape, and they have more intelligence area in the kind that humans have lesser of.
So yes very different, more in some material ways and lesser in others, but in total intelligence are still orders of magnitude lesser.
My gp comment was acknowledging that when you scale up many instances / brute force problems it confuses that "total area" claim a bit.
To follow the anthropomorphization... 1000 toddlers may have more total intelligence than a grown man, but does that matter?
The problem with these discussions probably/usually fold into differing/loose definitions of intelligence.
perhaps the chat-based ux has sort of fooled us into comparing these things to human intellegence. we don't really do this with chess, or other forms of ai, nor computers at large.
Who do we demand this from? The AI companies? Or the mathematicians who are worried they will have nothing left to do?
In 1976, the proof of the Four Color Theorem was controversial because it was done with a computer examining over 1000 cases by brute force and was essentially not comprehensible by humans. But mathematicians ended up accepting it. So mathematics has a 50-year precedent of not requiring human-scale proofs. How is the current situation different?
(Disclaimer: Apologies if this sounds dismissive or argumentative. I genuinely think that the Four Color Theorem should play a role in these discussions and suspect that many people are unaware of the controversy over it.)
There are mappings between complex numbers and 2D matrices allowing problems to be solved in either domain.
There's research into The Langlands Program looking to connect number theory and harmonic analysis.
There's research into Category Theory looking to define core concepts and relate them do different fields so that results in one field can be applied to another due to equivalence.
> I suspect that's possible without tripping over the halting problem. (But I can't prove it.)
I doubt that it is. For the language of proofs to be powerful enough to be able to express an arbitrary proof it would have be Turing Complete. Proving that a proof is the smallest proof of a given concept (i.e. there is no smaller human understandable proof) would then be proving the minimality of a program in a Turing Complete language.
This is basically how all AI approaches appear to work. They solve the problem you give them (in many cases) but in a very over-complicated way.
It's like they can't step back from the problem and realise that they need to simplify to make it work better. Nope, just keep hammering more code/proofs against the problem and eventually you'll hit the goal.
RL has a lot to answer for, I guess.
There is nothing wrong with that approach if it accomplishes the goal faster, i.e. you can work at the speed of an AI.
Sure, and god knows I've done the same myself when under time pressure (so similar to an AI in RL training, just get the goal).
The part that's lacking (that humans are much better at right now) is the meta-cognition around this works, but it's far more complicated than it needs to be.
Potentially because LLMs have larger "working memory" than humans, they can avoid this for a while, but once it blows up the context they fail in much worse ways than a human would.
Hmmm, this feels like a good metaphor, I need to write about this at some point.
There’s not really a clear distinction between these things, in my opinion. Problem solving (and intelligence?) is a mix of search and compression. We like solutions that are elegant (high compression, simple search). But often what appears elegant to some is harder to appreciate for those without the same background knowledge or even the same amount of mental bandwidth (if you’ve ever worked with someone simply much, much smarter than you, you may intuit this!).
This basically describes every single PR at work for the past year. Diffs of 10k+ paragraphs of comments saying nothing. Just rubber stamp and move on, nothing else you can do.
Meanwhile you can use the model to help you out as Scott comments "Just now, however, Dana tells me that she’s been asking Astra all day to explain the new proof of the UGC to her and it’s been doing an amazing job and she’s starting to understand the construction."
If a journal receives a paper that is unreadable, it is outright rejected with the comment that it should be made readable, regardless the results in that paper. What is the point of having results if you are unable to convince your audience of those results? What is harmful about this process?
Which is why it is obvious post-publication peer review is the only sane way to handle this. Formal peer review is not even remotely up to the task here.
And yes, I am arguing that they should not be released if they cannot be verified within a reasonable amount of time.
Also, how will this impact PhD student who might need this research to get their career started. How will it affect the inflow of junior researchers?
By the way, your tone is very unpleasant. Go outside and touch grass, or have a wank for all I care. You are insufferable.
Maybe there’s some variance or it depends on the reader. That said, like with a lot of other complaints about AI: human papers can be poorly written and poorly explained too. That’s always been the case. And sometimes a proof is just complicated and hard to expose nicely.
That's what a lot of us do nowadays, but instead of maths, it's code that looks sensible on the surface, but when you try to understand it's some kind of "alien" logic , names don't really make sense, etc.
Makes you wonder if AI can build upon such proofs.
If the AI cannot create proper abstractions, then how can it build a tower of abstractions?
so then, if you don't understand the proof, how do you know it's a proof?
Welcome to lots of (most?) code PR's in the last year. Though for software at the PR level it has gotten better with the latest models.
And you're right, it's not really a theme around here, but there aren't that many mathematicians around HN. Check one of the maths forums and it'll be a common point of complaint.
When all you value is being the first, you have not time producing useful papers.
I think OpenAI wants the outputs to be unreadable. If outputs "too advanced for human comprehension" become the norm, then every interaction requires tokens. If someone can buy a day pass, get what they need, and leave, then there's no recurring revenue stream.
I have read many research papers in computer science, and the code there is written in math symbols... and my sentiment aligns exactly with yours
The he puts up preemptive straw man arguments against doomers. His blog has become a joke.
People might feel differently about AI if they were a part of the changes rather than being a helpless spectator.
That would have given grad students who've been grinding towards a PhD for years a fighting chance to see if they could leverage the model to push their work forward, rather than watching years of work potentially turn to dust via a tool they don't even have access to.
It wouldn't delay the progress of mathematics by any meaningful amount in the long run for OpenAI to take this approach, and would help somewhat to preserve the health of mathematics as a field. Without it, the motivation for any young mathematician to devote years to a new problem must be sapped knowing there's an uneven playing field... an OpenAI team with access to colossal tools months before they'll ever be able to get access, willing to scoop them as soon as they can. (It's not the same as competing with other human mathematicians, where everyone has roughly the same arsenal of tools, constrained by human intelligence.)
I bet sharing unreleased models with outsiders safely ain't that easy.
Has anyone verified any of the proofs produced by OpenAI or is everyone just assuming that it just be true because the Lean code checks out? Couldn’t the Lean code just be formulated incorrectly?
For example, the statement of e.g. Fermat's last theorem in Lean should be understandable to anyone who played The Natural Number Game [0] and knows a bit of mathematics and programming. For the proof, you trust the compiler.
The statement of other theorems can be much more delicate, and the Lean formalization may require an extensive introductory section which will need to be carefully checked.
Then there are the cases where no Lean formalization is currently available, and all we have right now is an often impenetrable pdf in the OpenAI repo. I would not at all be surprised if some of those claims contained logical gaps.
Time will surely tell, but there are certainly doubts and lots people are very busy checking these results.
[0] https://adam.math.hhu.de/#/g/leanprover-community/nng4
I believe so, even with all the usual safeguards properly in place: https://news.ycombinator.com/item?id=49672339
> is everyone just assuming that it just be true because the Lean code checks out?
Kinda? It's only been 24 hours since they dumped 722 manuscripts on the world, most of which are apparently basically unreadable, and only some of which come with a Lean proof, which in itself is not a joy to read afaik.
In other words, they save effort by wasting the effort of others.
I'm however puzzled by the number of proof claims without lean proofs. How does OpenAI have confidence in those, especially if, as noted in the blog posts, the papers are very hard to read?
The same problem almost every person on earth is going to have to reorient to in the next decade, which is: how do we eat and stay housed when we have no real economic value?
It's just that we lost one of the important ways to demonstrate understanding.
(To my credit, I have warned them since more than a year ago that we will reach this point.)
But are any of them going to be able to do any significant amount of studying maths without being homeless?
Why not? Maths studying correlates with high intellect, high intellect correlates with competitiveness in the job market. So their chances of finding a job that would allow to have much of free time (for doing math) without being homeless are pretty good.
Math isn't about collecting random theorems, progress in math is about gaining understanding of new systems, and the theorems are guideposts to aid in that understanding.
You can prove 1000 theorems and not really increase any understanding about a subject, but gain knowledge of 1000 random facts. For example, I can write down some complicated equation and ask you "does this have a solution in the integers"? And if you do a maze of very complex and tedious algebra to show that there is a solution, you would have proved a theorem, but you would not have done much to move math forward at all.
On the other hand, if you introduce some completely new technique, say you take my equation and turn that into an algebraic surface, and then you count some special curves that live on this surface using geometric ideas, and then you show that if the number of such curves is odd, there must be a solution in the integers, and in this specific case, it is odd, so there is a solution -- well, then you have really pushed math forward and people will celebrate your proof, even though no one really cares if the equation I wrote down has a solution in the integers.
For example, there is a long history of failed attempts to prove Fermat's last theorem driving algebra and number theory forward by introducing the concept of ideals, for example, and this concept ended up much more important than whether Fermat's theorem is true or false, which is not too much more than a piece of trivia.
Or for example, the recent proof of the Poincare conjecture relies on the machinery of the Ricci flow introduced by Richard Hamilton, who then applied it to solve a number of open problems, but Perelman was able to take it even more forward to solve Poincare. So Ricci flow was massively important machinery.
For this reason, we celebrate people like Gromov, who didn't really prove that many theorems but introduced amazing machinery -- for example, the h-principle, or Gromov Compactness -- these were ideas and math is about the ideas. The ideas are then applied, using laws of logic, to form theorems.
So mathematicians will need to mine these proofs to see if there are any new techniques - new machinery - being introduced, or if the AI just used the existing machinery more efficiently. Here too, we are just looking at AI as a form of search, which it is really good at, since there are so many thousands of papers and so many ideas, that there might be a connection between two areas that lead to a solution and the human mathematician, not knowing all known results, can't make that connection. In the future, we may wonder how anyone did math without AI, much like we would wonder how anyone can be a writer without access to a dictionary or reference work. Is the AI just searching through a catalogue of known ideas and connecting them or is the AI coming up with genuinely new stuff like Ricci flow or the h-principle?
What is interesting is seeing whether we can get AI to actually discover new machinery for us. That would be huge.
And then we need to find efficient ways to detect these ideas and describe them.
Really this is very exciting and opens up whole new workstreams for mathematicians.
Emotions can be funny.
1. Help understand, check, and explain the results.
2. Write new works explaining or refuting the results in more lucid language.
3. Advance the field further.
I don't know why this is not obvious.
#3 at this point might need more human intuition; but that might be a 2026 problem.
The "summarization" process can be multi-step. Initial steps develop new frameworks and prerequisite concepts. Later steps build upon these frameworks and concepts. Some can be simplified too if this can be done without loss of fidelity. The summarization comes last, and is meant to preserve sufficient context.
> I don't think anything about how to react to a paper being released in this state is obvious.
If it helps, exaggerate the condition a bit by imagining coming across a repository of alien knowledge.
If you don't understand why the paper is in its current form, how do you know what changes would damage the fidelity for the AI? I think your example of alien knowledge is misleading because it suggests the aliens don't need to understand the info - but in this case we intend to keep using AIs to expand this work. So if you transform it into something that humans think makes more sense without understanding why it's in a form that we hate now - you risk losing something in the form that you don't understand.
And if humans decide to give up on mathematics because the process of using the machine is so tedious, then there will be no value in automated theorem proving.
> resistance will fail utterly as long as AI itself can understand prior AI works
That's not the issue in the long term. Look further: where did these questions come from? They came from humans who had been exploring mathematics and noticed interesting patterns that cropped up again and again.
The people who posed these questions had a profound understanding of the background maths that led to them. Me - I don't. Take one of the results he quotes: "Positive solution to the Unitary Synthesis Problem". He may as well be one of your alien races yabbering to me in an alien language. The question is meaningless to me. The answer the AI came up with, correct though it may be, is equally meaningless. You may as well have told my dog how to find eigenvalues.
If we let AIs do all the understanding so we know nothing about math, there will be no human understanding that lets us find new questions, or understand the value of any questions AI might pose and find the answers to.
This is only going to get worse in the foreseeable future.
^^ half of the comments on this thread
My dislike lies in how it devalues skill + knowledge acquisition, disrupts art/software communities, and that it's now bloody impossible to tell the difference between someone who actually knows something from someone who merely regurgitates shit from their chatbot.
Usually 9 year olds imitate adults when they regurgitate such words in these circumstances. What a sad state of affairs.
And going from zero proofs to one proof (even a sloppy one) is a big deal regardless of whether it was written by AI or a human.
You might as well say "the obsession with sex has always held humanity back". Maybe, but it's complicated...
Many math problems are practically useless if you only care about the answer, the millennium prize about the Navier-Stokes equation is such a problem. The solution makes no physical sense, real life fluids don't follow the Navier-Stokes equations in such extreme conditions. But in the process of finding the solution, we may get insight into what will end up being really useful. The big mess that OpenAI produced is the solution no one really cared about, but it didn't deliver much of what people actually wanted.
One reason it is sometimes seen negatively despite being at least something is that it broke the incentive. Without the million dollar prize and with only the privilege of being second, people are much less likely to go for the insightful solution.
It would be fantastic if university and science was like "here is 100M$, play around and develop some 'understanding'". However the reality is that human societies are hierarchical and currently capitalistic which implies value creation and status building.
1) the funding bodies/agencies need proof of value that you're using the resources meaningfully to be able to assign resources
2) Humans are status seeking, power seeking, resource seeking and sexual reproduction seeking. If you hold a lot of power and make decisions, you have more of all of the above.
Highly recommend reading it. Very prescient for something written 26 years ago.
https://gwern.net/doc/fiction/science-fiction/2000-chiang.pd...
I also recommend "Exhalation", though that has nothing to do with AI.
https://web.archive.org/web/20111121100139/http://www.fantas...
https://en.wikipedia.org/wiki/Division_by_Zero_(short_story)
What was it about the other problems that made them unsolvable? Was it just a time constraint, or are they just harder problems?
In the past 2 years the AI's started solving math problems in roughly the order of "hardness" as ranked by humans.
https://cepr.net/publications/ai-didnt-steal-the-mathematici...
Frontier theft is just faster.
There are suddenly many new solutions to problems that have resisted sustained attacks (e.g. the Uniform Games Conjecture as detailed in TFA at some length). Where do you think they are coming from? Why is there suddenly a bunch of results to be stolen?