I sort of wonder if effectively using AI to solve math problems is a skill in its own right, distinct from traditional mathematical skills. I don’t just mean “prompt engineering” either. More so figuring out how to combine agents with other tools and approaches in an effective way.
I guess the better question is, can this skill be learned without being an extreme expert in mathematics? Can I be a sort of intelligence highschool student (not incredibly exceptional at all) and have the LLM teach me the required math for a specific problem and guide it towards a solution?
I think the answer to this question is becoming, in general, a big yes.
LLMs can arrive at a solution via two paths. The first one is via heavy guidance by an adept expert in the domain. The other path is via brute force... multiple agents (the more the faster it can arrive at a solution). The later path is what enables anybody to do this.
IMO, it _currently_ requires a lot of skill because it will frequently take wrong turns and dead ends and needs suggestions and steering to get there. You do need to understand what it is doing at least a little bit and to understand the general landscape of the problem, and to at least have a sense of whether and why the problem is tractable at all.
the fact that you have to encourage these models and tell them that they can solve these problems and warm up on easier problems seems to indicate that there's something to AI pairing above and beyond prompt
I would think this type of behavior by LLMs will always be desirable for ‘hard’ problems (things previously unsolved) because it’s a bit like shutting off all saftey mechanisms put in place to prevent hallucinations for people incapable of verifying correctness of the output. You’ve got to do a little work to prove you understand that it’s hard but it’s still something the LLM might be able to accomplish.
I think this is also a healthy mindset for a person to have in many cases. If my boss asks me "go solve P = NP", as extreme example, I would also give some pushback.
It will likely burn a lot of tokens, take a long time, and might not work, so hopefully they'll still ask if you really want to spend the money on the attempt.
I spent about a month walking through proving something with Claude a few months ago and it _constantly_ told me that it was impossible and I should stop working on it, right up until it proved it.
They are relying on training data of humans talking about how hard these problems are. Same way an un-reminded Claude gives estimates for work that are as if a human is doing it by hand, but then will drop them by 20x if you remind them it’s going to do the work.
One way they improved the hallucination problem was basically training the models to refuse to do or say something if they are not very sure they can do it. As a side effect, they refuse to work on problems they know are extremely hard.
Great short term achievement but humanity is better served by these kids doing it without AI. The ideas have to come from the next generation (eg would we be worried if a 5 year old wrote the great American novel with AI or not?)
I think it's more like how anyone learning to play chess is going to use chess engines, but you can't learn to play by always asking the chess engine for the answer.
I expect that all mathematicians are going to be working with power tools, so they might as well learn about that. They will still need to do math exercises by hand to learn the material.
> The students heavily utilized AI assistants, specifically Claude Opus 5 and GPT-5.6 Sol, for computational exploration, proof idea generation, and editing.
this is a bit like Enhanced Olympics(https://www.enhanced.com), except that you have someone else compete for you.
My issue is that it makes it hard to distinguish real insight/work etc. from effectively null one.
An old instance of the same issue was with what was called "script kiddie" back in the 90-00s
For short problems the LLM are just too good now, but for long problems they still get in trouble.
It's like bicycle or F1 race. It goes faster, but you still have to steer the boat to reach somewhere. (Or probably something in between, like a motorcycle race.) Also, the "kids" were guide by a postdoc, not completely on their own.
> Jarred Sumner, an Anthropic staff member (and non-mathematician), prompted Claude to "take a real stab" at the hypothesis itself, leaving the mathematical choices from there up to the model. ... Throughout this process, Jarred's input was mostly limited to sending Claude messages of encouragement (mostly variants of "keep going" or "believe in yourself"). This seems to have helped Claude overcome some initial skepticism that it could make meaningful progress.
It means that the solution is merely an interpolation of existing work and not fundamentally innovative as AI can only regurgitate, never creating something new.
Working backwards from the Navier-Stokes solution, I was able to walk Astra through the path used to solve it. It took some formulation, starting with the problem, then challenging it to look closer at the specific path, and iterating when it got stuck, it was able to reach the solution.
34 comments
[ 2.3 ms ] story [ 36.7 ms ] threadI think the answer to this question is becoming, in general, a big yes.
LLMs can arrive at a solution via two paths. The first one is via heavy guidance by an adept expert in the domain. The other path is via brute force... multiple agents (the more the faster it can arrive at a solution). The later path is what enables anybody to do this.
Once the idea of AI routinely solving conjectures enters the training data this encouragement will disappear like 2023-era prompt engineering did.
Title: BOUNDED RATIOS FOR LORENTZIAN POLYNOMIALS
https://arxiv.org/pdf/2609.05341
I expect that all mathematicians are going to be working with power tools, so they might as well learn about that. They will still need to do math exercises by hand to learn the material.
> Unsolved Problem by Fields Medalist Breached by Two High School Students with AI
My title recommendation: Fields Medalist Problem Solved With AI
Opus 5 and ChatGPT 5.6 Sol were used for “computation, proof idea generation, and editing assistance”.
(Although it's a tiny bit odd how they list the AIs used.)
> The students heavily utilized AI assistants, specifically Claude Opus 5 and GPT-5.6 Sol, for computational exploration, proof idea generation, and editing.
this is a bit like Enhanced Olympics(https://www.enhanced.com), except that you have someone else compete for you.
My issue is that it makes it hard to distinguish real insight/work etc. from effectively null one.
An old instance of the same issue was with what was called "script kiddie" back in the 90-00s
It's like bicycle or F1 race. It goes faster, but you still have to steer the boat to reach somewhere. (Or probably something in between, like a motorcycle race.) Also, the "kids" were guide by a postdoc, not completely on their own.
https://www.anthropic.com/research/riemann-zeta
https://arxiv.org/pdf/2609.05341