In late 2023, a Reddit user known as "evilscientist311" conducted a comprehensive analysis of Cleo's activity patterns and interactions with other Stack Exchange accounts.
Well, it does make the solutions less impressive, right? Because it's much easier to compute the inverse of an integral than an integral. What's impressive here is more in composing problems that have nice answers and can't be solved by computer algebra systems, then the sock puppet trick to conceal that you ha the answer all along.
Reshetnikov didn't confess to them being reverse engineered, though. He confessed to them being conjecture arrived at after working on the problem for a bit by tweaking similar integrals in symbolic math software / numerical approximations. Every post was a gamble because it could have been wrong.
Of course, that could be a lie, but he came clean about other things that make sense retrospectively.
> Every post was a gamble because it could have been wrong
Symbolic differentiation is a deterministic algorithm. There is no reason for his posts to be a gamble. I'm sure there was a lots of interesting trial and error in developing good question and answer pairs, but once he posted the question he knew what the answer was already.
As I understand, he was specifically targeting problems that symbolic math software had given up on. He was only using symbolic software to solve the similarly shaped problems first, which allowed him to be fairly confident in his conjecture, as he became quite keen on how to slip his way from the output of symbolic math software from one solvable formula to a closely related formula that broke the software.
That is frankly, obvious horseshit. It couldn’t possibly be a gamble because you can just trivially differentiate the answer and see if it came out to be the expression in the question.
This is a definite integral. The specific endpoints might allow substantial simplifications that obscure the anti-derivative in question.
This is similar to the way that $\int_{-\infty}^\infty exp(-x^2) dx = \sqrt \pi$. The answer is a constant that doesn't give you any information about the function erf(x).
Remember he was producing the anti derivative/integral and already had the original expression. So all he needed to do is show that when you differentiate the integral answer you get the original question. That’s a lot easier than you’re making out.
I think there are some interesting parallels between what the Cleo account did, and what the large AI companies are doing now with some of the proofs for previously unsolved theorems.
There may be some value in the answers, but for some the value is in the understanding of how to get from the problem to the solution.
I wouldn't call myself a mathematician, but I encountered a few problems in my CS classes where I could work through the proof by hand, see that an algorithm works, and even mathematically prove it, but yet still didn't feel like I understood why it worked, or how the algorithm was invented/derived. But for many, the fact that it did work was enough.
> for some the value is in the understanding of how to get from the problem to the solution
All of the value is in this understanding.
So many people are still in denial about it, but "AI" really is just a natural language search engine. AI helps organize what has already been written by humans and was technically self-evident from our work.
If we allow ourselves to continue down this anti-intellectual path and stop caring about why proofs work, we've basically entered another dark age.
I'm not saying AI can't possibly help at all with building intuition, but clearly there are a lot of idiots in the class who just gotta raise their hand first to blurt out whatever trash they come up with on impulse. Those people need to be told to sit the fuck down already. Their uncontrolled ADHD is ruining everything. There's nothing intelligent about mindlessly rushing to compute an answer. That is literally what the last few years have felt like for everyone else.
> AI helps organize what has already been written by humans and was technically self-evident from our work.
I am not 100% sure about the "self-evident" part here.
To your point: yes, AI doesn't come up with new axioms. But then we enter the topic of "math is invented or discovered?". Each of the discoveries are logical consequences of the axioms, but that doesn't mean they were "self-evident", especially for humans.
If everything was already self-evident, AI would just be a useless tautological machine. However, I think it is more than just that
I just gave AI the original Cleo integral, and it came with a solution dissimilar to any answer on Math.StackExchange page in question. The answer also differs in presentation from Cleo's answer, being "2π arccos(sqrt(5) − 2)" (the numerical values seem to be the same.)
Joe McCann and his audience discovered that Cleo was a ploy at ragebaiting the math forum in order to drive engagement for more useful proofs. A real-world Cunningham's Law social experiment.
So maybe OpenAI's ridiculous approach to mathematics has value in how it's ragebaiting us to scrutinize and/or come up with better proofs!
AI capabilities keep getting bumpier and bumpier. There's no reason to believe it won't stall out on a front that it is already quite bad at (correctly formalizing in human language)
I don't think that is the thing that makes AI better. In the metaphor the idea is that someone scored a goal and then people retroactively shift the goalpost to say it doesn't count as a goal.
Because making AI better is orthogonal to making human lives better. We are quite literally racing towards artificial superintelligence without any serious plan or preparation for the societal impacts that will have.
Did people prepare for or ever fully foresaw the downstream impact of any other major technological revolution, such as farming or the industrial revolutions or the Internet?
Has there been any other major technological revolution that underwent exponential recursive self improvement?
If AI development stopped right now, we would see a technological revolution at a scale similar to the Internet. AI development has not slowed down, let alone stopped. This technology demands far more planning than we are allocating.
AI inherently has misaligned incentives (as do the companies developing them). I don't trust them to outline the consequences of something like this and to develop an appropriate plan, and neither should you. I believe in humans, but we need time.
The point of understanding is to get better at proving new things. Now that humans are obsolete for proving things, human understanding is unnecessary.
The economy can show us whether the conclusion holds, but it does not tell us anything about the premise. Do you believe the point of humans is to produce things for the economy?
Treat it as an oracle problem. Posting exact closed forms without derivations forced the community to build the actual algorithmic bridges. High-effort trolling, higher-tier result.
The Wikipedia page says that in the early 2000s, he emigrated from his native Uzbekistan. Where too? Of course almost everyone would have the US as their first guess. And they'd be right. What a boon for the Americans to be the default destination for geniuses like this!
But now... What have you guys done? Would the guy even be allowed to come if he wanted to?
Nationalism/politics aside, it seems extremely unlikely he was actually a genius based on the fake account findings in the article.
More likely, he started with the answer he liked & did simpler operations to turn it into a much harder to undo integral problem until none of his symbolic math software could manage to solve it the right way around anymore. Then, he used his now known fake accounts to post the "question" he already had the "answer" to. His self-replies missing the one part (the steps to do it the hard way around by solving the integral instead of creating it) that would have proven he actually did figure out how to solve it 39 out of 39 times.
It'd be like if I generated a 2048 bit RSA key, posted under a fake account asking if anyone could crack it, and then posted under another fake account what the private key was.
> I was frustrated that when I posted questions about integrals on Math.SE, I often received comments like "Why is this interesting?" or "What makes you think that it may have a closed-form solution?"
Not surprised that the rotten culture of unhelpful gatekeeping at Stack Exchange extended beyond stack overflow.
>In January 2025, Cleo was revealed by internet sleuths
this type of sleuthing, when someone is not doing any harm, is typically quite off-putting. if someone is contributing good to the world but trying to maintain some anonymity, it's really rude to try and "out" them.
however, i guess reshetnikov didn't care too much about being exposed as the person behind the name, considering:
>"On 8 February 2025, Reshetnikov posted a Base64-encoded message on his Stack Exchange profile that, when decoded, read "Creator of Cleo"."
As a non-mathematician I'm confused by this Wikipedia article. Isn't it trivial to fabricate arbitrarily hard integrals by starting from a function with known values and differentiating it?
I understand that people enjoyed the prank, but I think the article doesn't make it clear enough that he was just trolling and wasn't actually solving hard mathematical problems
57 comments
[ 0.22 ms ] story [ 88.5 ms ] threadIn late 2023, a Reddit user known as "evilscientist311" conducted a comprehensive analysis of Cleo's activity patterns and interactions with other Stack Exchange accounts.
However, I and others were on the scent at least as far back as April, 2023. See for example: https://x.com/TheDavidSJ/status/1650957407902658571
> Wild to see a story I played a major part in on the front page… The first post outlining the theory and evidence was made by me a few years ago
https://www.reddit.com/r/todayilearned/comments/1wyzrji/comm...
Doesn’t necessarily make the solutions less impressive, but it’s kind of a relevant detail that is left out in the Wikipedia summary.
Of course, that could be a lie, but he came clean about other things that make sense retrospectively.
Symbolic differentiation is a deterministic algorithm. There is no reason for his posts to be a gamble. I'm sure there was a lots of interesting trial and error in developing good question and answer pairs, but once he posted the question he knew what the answer was already.
This is similar to the way that $\int_{-\infty}^\infty exp(-x^2) dx = \sqrt \pi$. The answer is a constant that doesn't give you any information about the function erf(x).
There may be some value in the answers, but for some the value is in the understanding of how to get from the problem to the solution.
I wouldn't call myself a mathematician, but I encountered a few problems in my CS classes where I could work through the proof by hand, see that an algorithm works, and even mathematically prove it, but yet still didn't feel like I understood why it worked, or how the algorithm was invented/derived. But for many, the fact that it did work was enough.
All of the value is in this understanding.
So many people are still in denial about it, but "AI" really is just a natural language search engine. AI helps organize what has already been written by humans and was technically self-evident from our work.
If we allow ourselves to continue down this anti-intellectual path and stop caring about why proofs work, we've basically entered another dark age.
I'm not saying AI can't possibly help at all with building intuition, but clearly there are a lot of idiots in the class who just gotta raise their hand first to blurt out whatever trash they come up with on impulse. Those people need to be told to sit the fuck down already. Their uncontrolled ADHD is ruining everything. There's nothing intelligent about mindlessly rushing to compute an answer. That is literally what the last few years have felt like for everyone else.
I am not 100% sure about the "self-evident" part here.
To your point: yes, AI doesn't come up with new axioms. But then we enter the topic of "math is invented or discovered?". Each of the discoveries are logical consequences of the axioms, but that doesn't mean they were "self-evident", especially for humans.
If everything was already self-evident, AI would just be a useless tautological machine. However, I think it is more than just that
So maybe OpenAI's ridiculous approach to mathematics has value in how it's ragebaiting us to scrutinize and/or come up with better proofs!
If AI development stopped right now, we would see a technological revolution at a scale similar to the Internet. AI development has not slowed down, let alone stopped. This technology demands far more planning than we are allocating.
One can also say: The point of humans is to produce things for the economy. Now that humans are obsolete for producing things, humans are unnecessary.
Would you agree?
But now... What have you guys done? Would the guy even be allowed to come if he wanted to?
More likely, he started with the answer he liked & did simpler operations to turn it into a much harder to undo integral problem until none of his symbolic math software could manage to solve it the right way around anymore. Then, he used his now known fake accounts to post the "question" he already had the "answer" to. His self-replies missing the one part (the steps to do it the hard way around by solving the integral instead of creating it) that would have proven he actually did figure out how to solve it 39 out of 39 times.
It'd be like if I generated a 2048 bit RSA key, posted under a fake account asking if anyone could crack it, and then posted under another fake account what the private key was.
Also, didn't Cleo post the problems themselves?
Not surprised that the rotten culture of unhelpful gatekeeping at Stack Exchange extended beyond stack overflow.
this type of sleuthing, when someone is not doing any harm, is typically quite off-putting. if someone is contributing good to the world but trying to maintain some anonymity, it's really rude to try and "out" them.
however, i guess reshetnikov didn't care too much about being exposed as the person behind the name, considering:
>"On 8 February 2025, Reshetnikov posted a Base64-encoded message on his Stack Exchange profile that, when decoded, read "Creator of Cleo"."
I understand that people enjoyed the prank, but I think the article doesn't make it clear enough that he was just trolling and wasn't actually solving hard mathematical problems