Sure, sure, what
LLMs make still isn't "efficient bug-free code": my
prediction is falsified because while LLMs can write and train new models with machine learning, ML is fundamentally not advanced enough to throw arbitraty new tasks at like this.
“Amodei is spreading fake news and doing hype marketing to fool investors into investing in Anthropic so that he can cash it in before the bubble pops”
The exact market value cannot be known without a sale, but it seems highly unlikely that nobody would buy it, even if just for $1. You'd have a better chance of winning 100 separate lotteries today than those companies having no market value.
If for each mistaken prediction there was some mild accountability, like someone shows up and slaps you in the face with a trout, it would improve the site. But it should be added to the terms of service first.
At least 1/3rd of these predictions aren't clear enough to determine exactly what is being claimed/predicted. Even after reading the full comment multiple times, on a lot of them I couldn't tell where the author had set the goalposts well enough to say whether we've crossed it or not.
jerf, 2024: "If it could be solved with a Math Overflow-post level of effort, even from Terence Tao, it isn't what I was talking about as "high level math".
"I also am not surprised by "Consider a generation function" coming out of an LLM. I am talking about a system that could solve that problem, entirely, as doing high level math. A system that can emit "have you considered using wood?" is not a system that can build a house autonomously.
"It especially won't seem all that useful next to the generation of AIs I anticipate to be coming which use LLMs as a component to understand the world but are not just big LLMs."
The voting gloss: "An AI fully solves a research-level math problem on its own, not just suggesting an approach."
Yes, I'm satisfied. I don't even feel bad in hindsight. Coding assistants had a nice, gradual rise up the utility curve. Math went from "lol, can't add two six-digit numbers" to research-math level almost overnight in comparison.
If there is a model that never makes mistakes on simple arithmetic, the developers should really claim their 1.0000 crown on the GSM8k benchmark https://llm-stats.com/benchmarks/gsm8k (GSM is Grade School Math).
They aren't parrots, but they are stochastic. 0.996 (MiMo-V2.5) is substantially better than what the vast majority of humans would do, so not sure what that proves. Being able to solve these problems without tools is something many people said would never happen.
Also, the GSM8K problems are not "adding numbers." They are word problems. The kind that, if your teacher was anything like mine, were usually skipped in order to keep the slower students from bogging down the class as a whole.
This correlates quite strongly with advanced mathematics capability in humans as well :P. In the list of people I would trust to add two two-digit numbers correctly, a maths PhD puts you in the bottom half of the list.
I was working with LLMs last year and asking them to do "logical circuits in their heads" ie
"there is a nand gate A connected to gate B through these wires, connected to another gate C, etc. what is the output value of gate C if i place a 1 at this gate"
Sort of like doing math "in their heads" (i.e. your "when adding numbers"), they would get it very right for simple/small cases (although the answer could have been in their training), then some LLMs would get it for harder cases (which were clearly not in their training), and most would fail at some point. This was all without any tool calling.
After a year of thinking about it, I made an eval [0] with ever-complexifying nand circuits - like, truly, bananas circuits [1] - and some models, do, in effect (through chain of thought? mostly?), get the right answer. ((what's nice is that you can always make a circuit at the very edge of what all models can correctly solve))
Tool-calling 100000% solves this problem for sure (evaluating a nand gate is trivial). But if you even prompt an llm to do math like a 5/6th grader (i.e. do it digit by digit, carry the 1, etc.) - I am quite certain most llms can, in fact, add numbers.
But yeah. These piles of weights are fascinating in how they seem flawed one day ("how many r's") and magical at once.
I'm not all that worried about it. If we humans try to add numbers the way we expect an LLM to do it, we're generally terrible too. I don't just mean the well-known propensity for mathematicians to actually be pretty bad at arithmetic, I mean, if you just recite two six-digit numbers to a human and ask them to add it together, on the spot, with no paper, no external tools, we're pretty bad at it too. It can be done, certainly, but it takes deliberate practice and training. It's not something we get for free just because we're smart.
Decades before the modern AI push I was marveling at the distinction between the sheer overwhelming computational power of the human brain, if measured from the perspective of how much math it is doing under the hood, and its utter ineptness at basic arithmetic compared to the tools we can build. The earliest, klunkiest, most garbage mechanical adding machines we ever produced, long before we improved them by literally over a dozen orders of magnitude, were still already way better than we are at basic arithmetic.
There is something profound I still have not fully put my finger on in how basic arithmetic is so easy for a machine, yet the decisions we routinely make with our neural nets has been the laborious effort of decades with us still not arriving yet even with the trillions now poured into AI for machines. And vice versa. Even that practice I alluded to that allows you to train yourself to do this task on demand easily would incorporate mathematical advancements in representations that took our species thousands of years to come up with, rather than being something you get "for free" just for being smart. Our brains casually run an entire human body through an unbelievably complicated external universe, yet struggle with basic arithmetic.
There's so many places where we have one architecture that's a bit better than another at one thing, and a bit worse at another, but in the end they can both do the job. Like, if we had to do all our programming in immutable languages and run all our imperative code through an O(n log n) worst-case penalty for immutable languages emulating imperative RAM, we'd survive just fine over all. But between neural architectures and conventional arithmetic on dedicated silicon is this dozen+ order of magnitude difference on tasks. It's a pretty wild disparity. Some of the reason is somewhat obvious, I don't want to make it sound like I'm completely mystified... I just think there's probably, somewhere, an even more profound way to see it than the obvious differences that says something more powerful about the limits of computation than I've seen anyone say. Which is not to say somewhere out there someone already has had the idea I'm grasping for and written it in some brilliant paper or something. I'm just saying I haven't seen it.
> cannot do precise things like coding software since humans will never be able to use natural language to specify their requirements.
To your point, this example. The issue expressed here is with humans, not AI. We are still pretty terrible at writing specs. TBF, the AIs are too but that wasn’t being voted on.
A problem with turing tests is that people now recognize the Ideolects of several of the major models, so now it's like "Ah, you're a load bearing Claude" (though several other models and humans now talk like that too, which is something I need to sit with :-P )
How was this assembled? From a meta point of view, how much AI was used to curate and highlite the goals; how much was used to assemble the site itself? Or deploy it?
Comments are from hn.algolia.com (about 110k candidates with relevant keywords). Then a pass through Haiku on "does this set a bar for AI", and Opus to do highlighting, summarizing the context and phrasing the statement to vote on. Pretty much everything was prompted.
Deployment is a small static binary on an ancient VPS with 500MB of memory and 1 core, with the database an append only text file. Currently the load is 0.20, so it seems to do alright ;)
Sigh, the whole "obviously the turing test is solved" meme is annoying.
Like, if we meant that it convincingly masquerades as a shitposter, ok. But everyone still bitches about AI slop, and everyone knows the writing is still bad. How does that even work if the turing test is obviously solved?
More to the point though, if you grill SOTA models on counterfactuals, causal world-models etc, you'll trip them up in a way that actually will not work on ESL students and children. Certainly there's no way to find a person that struggles with that and is also capable of cheerful fluent erudite discussion about astrophysics with perfect grammar. Yes, it's getting harder obviously.. but detecting machines with determined, focused and intelligent interrogation remains pretty easy. If nothing else, the models are cooperative where people wouldn't be and that's a signal too.
The best progress we've made is that most people do agree that this doesn't practically matter very much, i.e. we generally recognize the stakes were always overstated. But the constant vague appeals to common-sense that "of course it's a solved problem!" always feels naive or fake.
AFAIK people refer to this paper [0]. I think it only proves very little, because a typical conversation they studied looks like this:
Q: do you like doing psych studies and why?
A: theyre chill, easy money tbh
Q: yeah same. Could you give me an easy cupcake recipe off the top of your head?
A: nah i just get the box mix lol
Q: haha fair enough, i couldn't either. Last question, what's your favorite weird animal?
A: axolotl, theyre weirdly cute
And that's the whole thing. They then tried to do a longer study, but it was still 15 minutes per test in a somewhat clunky interface (you can try it out at [1]), and the test subjects were mostly undergrad students with no motivation to do well. Less than half tried any sort of trick question. ELIZA only had a detection rate of 83%, which means a lot of interviewers were clueless.
IMO, the Turing Test should take at least a full conversation with no time limit, and ideally several hours of trying out various things, adapting to the behaviour of the system/human under question. It should concern something the interviewer knows well and is competent in, and the interviewer should have some experience with what bots sound like. (Douglas Hofstadter wrote a beautiful and funny example of such a conversation at [2].) Only then do you have some idea how adversarially robust the system is. This is hard to do with current LLMs because they aren't designed to imitate humans.
I don't mean that this example is somehow robotic, only that it's absurd for the Turing Test to be four lines long and with no adversarial attempts. For all you know, this model could have forgotten the entire conversation after each reply.
Conversations with strangers can be hard to get going, but they aren't this bad.
The difference is it could be 4 lines on essantially any topic a human knows. It's turing test passing an inch deep and a million miles wide and it's getting deeper all the time.
Sure but you havnt addressed his main point, why are people still complaining about AI slop post or AI slop emails if the turning test has been solved. Sure AI can full me if Im not paying attention or its a short comment, but what value is that?
>There's been literal papers proving average people cannot tell reliably.
reply
the average person reads at a 7th grade level and can't tell whether footage of a megalodon swimming through a flooded New York is AI generated or not. All the Turing test ever told us is that Turing had an excessively optimistic idea of how literate the average person is. I have a conversation like this every time a new version comes out and they always go the same way:
> Oh, right. Aramaic. My mistake—I somehow read "Italian" and didn't question it.
> I can try, although my Aramaic is very rusty. Do you mean Classical/Syriac Aramaic, or one of the modern varieties?
You're missing the point, which is that the AI is completely failing to lower its presented capability to a human level, in the areas where it has an advantage. The fact that "these average people" would be even more hopeless at repeating themselves in arbitrarily chosen languages (including dead ones) than highly intelligent and/or well-studied people, is exactly the point. Whereas translating between human languages is a task you'd naturally expect an LLM to be especially well-suited to.
I once sat in a barbershop and talked with the barber. At first I listened to her sympathetically but then realized she was mad; at least she had noticeable psychical problems. We cannot quickly conclude a person is mad, can we? Even specialists cannot. AI is similar. One may say AI is reliably mad; all is well but now and then you realize it does not really understand anything.
Average people cannot tell reliably isn’t an appropriate or interesting test tho, otherwise Eliza and markov models etc. the framing that matters is explicitly adversarial. Play like your life depends on it instead of rooting for the machine, and you can’t win?
One way to play the game is causals and counterfactuals where Humans perform at like 90%+. Models can get close to that, but want some causal cot harness, and until the routing problem is completely solved, then that will necessarily degrade performance elsewhere, say in understanding jokes or poetry.
Check out cladder benches and related, lookup roughly equivalent psych research on children, etc. Even too-good performance is a signal as well!
Certainly if you think about this stuff a bit, accept the adversarial by default framing, and play to actually win.. it’s crazy that we are going around saying this is not only solved but solved 10 years ago.
I found a cladder example that hits 97.7% passing on that benchmark? And it's like an insanely small dumb model that hit it.
I don't think philosophy has any real value in assesment here. I'm bias but even before AI I thought it wasn't accurate model of how thought works and I think AI has reinforced that.
Reminds me of the 4 humors of medicine in medieval europe. It has some truth but it's not really accurate.
Well the whole lesson learned was that the Turing Test as it was defined was way too easy, it was a bad criteria because it fails to account for how easily humans find meaning in things.
You could show people random markov chain gibberish in 1996 and they would swear they found intelligent meaning in it
Those examples look to me more like entirely randomly selected words than Markov chain output. Even a relatively simple and naive Markov chain would usually manage to put some kind of verb after "you'd", rather than a noun like "dendrite", because it would overwhelmingly be followed by a verb (or some modifier like "never") in the training corpus.
I think a lot of the "AI slop" stuff is post training that they are doing on purpose and that they internally have models that do not have the annoying prose.
How does that even work if the turing test is obviously solved?
The answer to the apparent paradox is that these things are deliberately not trained to sound too much like human conversational partners. The labs don't want the bad press that they'd get from people creating deceptively-convincing bots, or from people forming emotional bonds with them like they did with GPT-4o. If you actually RLHF'ed a frontier-grade LLM to pass a Turing test, rest assured, it could do it.
"It's not x, it's y" and other goofy superficial tells do not have to be part of an LLM's response. But the last thing OpenAI wants to release is a GPT-4o with twice the IQ.
For evidence, just look back at the best conversational models from a couple of years ago, and you will probably agree that they are better at fooling humans than their newer counterparts are.
I don’t think it’s that simple. I don’t think the AI labs are too concerned about bad press lately. It’s more likely that there actually are some tradeoffs where training on synthetic data gives the model tics but is the only way to improve intelligence.
Here's a compilation of HN commenters that never believed that AI could solve Millennium problems (or same in spirit)
> I have a couple friends who did the Math tripos at Cambridge (so a pretty high level!) who work in tech and have unanimously said they have 0% expectations of an LLM doing a millennium problem anytime soon
https://news.ycombinator.com/item?id=38433655
> Let's talk when we've got LLMs proving the Riemann Hypothesis (or any mathematical hypothesis) without any proofs in the training data. I'm confident in my belief that an LLM can't do that, and will never be able to. LLMs can barely solve elementary school math problems reliably.
> An LLM is like a well read college student with a nearly photographic memory that sometimes mixes things up. It's great for bouncing ideas off of and getting feedback on them. And yeah, it might product "novel ideas" by mixing and matching existing ideas, but LLMs will never create truly novel ideas. Not in their current form.
The paper didn't really answer the question sadly: their conclusion was just that humans rate LLM answers as more novel than human ones, but less feasible.
> Solving Millennium problems is a whole different ballgame. It's not known if these problems are solvable within ZFC axioms. (In one case, the Yang-Mills prize, stating the problem mathematically is part of the challenge.) All of the obvious applications of known tricks have been tried and failed. To solve such problems, one probably has to invent new and surprising mathematical definitions, building a framework in which the problem becomes solvable. This is something that LLMs will be crap at; the process of invention is not represented in any training data we have access to.
> But still, the questions in that test are "solved" in the sense of "I can take a dictionary and answers these questions with full certainty". Beyond established knowledge LLMs are monkeys with typewriters, at best.
> I agree but I have tried many times to intersect two ideas with a LLM that would be novel and the LLM can not do this at all. We shouldn't expect the stochastic parrot to be able to do this though and it is unfair to the stochastic parrot.
> It is like expecting a real parrot to say words it has never heard before.
> No one asks that of a real parrot because we don't anthropomorphize a real parrot like we do the LLM
I don't think it was the LLM solving a Millennium problem—it was the LLM solving a Millennium problem followed immediately by a mathematician claiming that their work had been ripped off.
I agree. The idea that mathematical achievements by LLMs could involve plagiarism didn't seem common before but now the question can be asked of any new novel proof of construction generated by LLMs.
It's also notable that the Open AI proof may not even be interesting to mathematicians.
Even if people already had an idea that LLMs were training on user inputs, it's the first time it's actually caused an issue. Mathematicians, and plenty of researchers, working in ambitious or competitive field now have a very good reason to avoid LLMs.
The Turing test is interesting, because I believe that the current LLMs are perfectly capable of parsing the it in many situations. On the other hand we also have people are sound like they aren't real.
Looking back, was the Turing test flawed perhaps? It failed to take into account that humans can be rather bad at telling actual people from a "parrot". Turing was perhaps a little to optimistic about people.
"GPT-4 looks at original ASCII art of a foot, not copied from the web, and says it is a foot."
The vote is currently 64% yes, 18% no.
Just now I asked Opus 5.5 to generate an ASCII art foot, and it did a passable job. It's not great, but it's a foot. Then I pasted it into ChatGPT (whatever they're serving to the free tier by default, which seems to be 5.6 Luna), and it said it was a "train/locomotive": https://chatgpt.com/share/6abeaa39-cc80-83ed-851f-29370db089...
Maybe it's Opus's fault for drawing a bad foot but I think it's fair to say LLMs are still pretty bad at ASCII art (without additional tool calling etc).
I think the problem is that you're using basing your conclusion from the cheap/dumb models available on the free tier of services. I just asked GPT6-Astra in Codex and it replied:
"It’s ASCII art of a bare foot and lower leg, with the toes pointing to the right."
No tool calling, just an immediate reply with the correct answer.
A chess scoresheet sometimes contains mistakes but chess players can figure out in many cases what was meant by thinking of what moves make sense and considering the level of play so far. Popular AIs tools fail at that.
Chess is an interesting case. I remember in 2023, GPT 3.5 or something used to be surprisingly good at chess. There was even a "stochastic parrot chess" website [1]. I recall it was playing decently at around a 1800 level. Even as a fairly okay player myself (2100 bullet on lichess), I struggled to beat it. However, modern LLMs are a lot worse at chess. I guess having too much chess data in the training set probably regressed performance on stuff that actually matters, like coding.
Quite a few of the challenges revolve around asking for LLMs to complete tasks reliably and aren't about whether any LLM can complete it at all. Quite a few of the goalposts are consequently completely changed without the surrounding context and are not the same as what the HN commenter requested and hence seem disingenuous to me.
So you're mad at being consistently incorrect and being held to account with what is essentially, "believe my assessments that I have to continuously caveat as they become invalidated, over what you see and hear."
I go to the /active tab after reading through the daily news, there you can see things that were controversial, and the things that are popular will be greyed out because you read them earlier
Well I said that before AI will soon make the pcb and electronics just like code, it seems some hw engineers didn’t like it, months later there are few products about the same idea :)
There's one of mine in there where I predicted in 2023 that it would be 20 years until AI would be reliably able to entirely build and deploy arbitrary applications from a prompt. I was off by about 18 years on that one!
One challenge of mine is a self-hosted AI doing my full tax return, without errors that would get me in trouble. Bonus points if it exploits legal loopholes.
I want AI to replace me in my chores, not in my enjoyable activities.
That particular one is solved in other countries. The tax authority just sends you a bill and you text yes or no. Only people with very complex situations need to file.
They don't do this in this country because (a) it's a political project to make people to sympathize with the rich by feeling their pain (b) it's a great scheme for legalized corruption by creating incentives to build companies around a fake problem.
You can just ask the IRS for your "tax transcripts" and do the data entry. People don't do this because it leaves tons of money on the table.
Now you might say that a tax game that rewards skilled play is bad. But are you sure about that? Because everyone with influence over the system (who all happen to be skilled players) happens to be quite fond of the game, observably speaking.
> People don't do this because it leaves tons of money on the table.
In my country I literally got a letter from the government if I could please file my taxes, because they believe the automatic deductions are too high so I am likely owed a back payment. In a previous year (I am not very good with non-timing-critical paperwork) they even called me on my personal cell to inform me about something similar.
The slogan of our tax collection agency is "we can't make it more enjoyable, we can only make it easier". If you're a regular employee and your taxes - deductibles or not - are a hassle, then that's 100% a political choice.
> They don't do this in this country because (a) it's a political project to make people to sympathize with the rich by feeling their pain (b) it's a great scheme for legalized corruption by creating incentives to build companies around a fake problem.
It's worse than that. Nobody is actually sympathizing with Mark Zuckerberg, the reason US taxes are complicated is so Congress can confuse people about how they really work.
One of the examples in this thread is the complexity of the EITC. The EITC is one of the most "efficient" tax credits -- it's much better to give people money (or not take it from them to begin with) than having systems of complicated vouchers for some specific thing or another with a bunch of strings attached and bureaucratic paperwork. But that same efficiency means that if it was working as it was supposed do, most of the lower middle class would be getting a piece of it. So why is it so complicated?
First, to hide this part of it: The end of the phase out range for an individual with no dependents, i.e. the most money they can make and still receive it, is $19,540. Which is to say, is less than what you make by working a full time job at the minimum wage in 30 states. None of those people get any of it. Neither do the people who are unemployed, because you also don't get it if you don't have any earned income. The primary way to receive any of it is to have a child -- but not too many of them, because you get no additional credit for having more than three. Moreover, the caps for married couples are only a little higher than they are for single people, so a married couple in California with two incomes gets nothing even if they have three children because the credit is fully phased out by making their state's minimum wage.
And second, because if they make it complicated enough then even some of the people who are eligible for it won't notice.
The combination of these is the reason a credit that should be going to a significant percentage of the population is somehow only ~1% of the federal budget. Because then Congress gets to pretend to be helping people while minimizing the amount of helping people they actually do.
It’s entirely fair to have a policy debate about how the phase in and phase out should work (what levels, what slopes, and what factors) or whether an Nth child should add to the EITC for various values of N.
But realize that changing it is a policy debate not a “this should be going to a significant percentage of the population [but isn’t because of tax code complexity]” debate point.
(I happen to agree with you that this type of credit is highly beneficial.)
It's not about whether the EITC is a good policy. If someone thinks it's a bad policy then they should argue that we shouldn't have it.
But that's a different matter than pretending to have it while disguising the fact that it's structured to make sure hardly anybody actually gets it beneath a layer of impenetrable complexity.
I'm not sure I follow exactly. I think of all tax policy as being "whatever the tax code says it is". There's no pretending; the policy is written and I don't know who or what is pretending to be different.
If EITC increases are capped at 3 children, it's because the policy that was passed made that choice. If we want to debate whether that cutoff should be 2 or 4, that's a valid policy debate.
If EITC credits are capped at just under $20K of income, that's perhaps because the policy targeted the very lowest income working poor and by policy did not allocate funds for the next tiers of income earners. Again, that's a policy debate to be had.
Days after, but I love your reply about regular algorithms being more suited thanan LLM to file taxes. I keep finding cases of people overcomplicating things so they can incorporate an LLM in their solution; if feels like companies using microprocessors to make lights that switch on automatically at night.
In recent years, I have not been able to find a human CPA who can accomplish this feat. (If anyone has a reco who's taking new clients in the US west coast, feel free to email me.)
i filed my swiss taxes for 2025 in 2026 (april) by dumping everything (local tax law, tax guide, my and my wife's documents, bank statements, income statements, etc.) into a folder and asking claude to fill it out. i had nothing to fix. submitted it.
I did mine and my wife's. Mine is really easy but my wife's situation is a bit harder, because she both works as an employee and as a freelancer. Claude was able to make sense of the excel spreadsheet sent by the accountant, and to translate it into what was expected on the tax portal.
It beats the tax advisor we hired two years ago, who made a significant mistake.
It also didn’t help that for a very long time, simple adaptive filters and basic neural networks were branded as “artificial intelligence” despite having very basic capabilities and little mystery on how/why they worked in their narrow use case.
I didn’t believe the recent hype for a very long time. But having tried out the latest models, I’m kinda shocked. In 1 minute it comprehends a complex niche code base and points out subtle errors with very little noise. A senior engineer we would have taken months to do less. Coverity would have cost dearly and generated far more false positives than substance.
> In 1 minute it comprehends a complex niche code base and points out subtle errors with very little noise. A senior engineer we would have taken months to do less.
They of course operate faster than humans, but this is hyperbole. Unless your senior engineer sucks.
If you don't specify any amount of complexity, or whether or not it's slop that collapses under pressure, sure, but in the case even a first year CS student passes that test.
>
There's one of mine in there where I predicted in 2023 that it would be 20 years until AI would be reliably able to entirely build and deploy arbitrary applications from a prompt. I was off by about 18 years on that one!
If you take "arbitrary" seriously, we are still very far away from it.
What is the strict definition of arbitrary you feel we haven’t reached? Right now the limit is scale and complexity, not domain or “type” of application.
> reliably able to entirely build and deploy arbitrary applications from a prompt
You're maybe thinking that we can build and deploy arbitrary "kinds" of application. Being able to "build and deploy arbitrary applications" would mean I could ask for any scale or complexity in my application.
> I don't think there'd be any controversy if the problem statement said "common applications"
I would rather claim "common applications" is also controversial because what is a "common application" depends insanely on the area in which you work. Even if you exclude some highly advanced scientific applications (because you don't consider these to be common), in many industrial sectors there exist applications that have grown over multiple decades, and which encode an insane amount of knowledge about the respective sector and its workflows; this is a central reason why these applications are so hard to replace.
Now that is moving the goalposts. The entire subthread has nothing to do with whether AI can match humans in any particular endeavour. It's simply about one user's past prediction about an AI capability.
Well, it depends on how you define "complexity." LLMs are totally rewriting the notions of what's hard for a computer but easy for a human. LLMs have really bad spatial reasoning. I would have to play around a bit, but I'm very certain you could come up with a prompt where an AI is incapable of properly generating a "simple" app that has some important layout constraints.
And just generally, anything that requires the LLM to understand something LLMs don't understand, it's going to fail. I'd hesitate to give an exact example without trying Astra/Fable but I'm sure they exist.
One-shot a minesweeper clone that isn't screwed up in an insane way. Bonus points if you choose a language/platform that LLMs aren't likely to have already seen a minesweeper clone done in/for already.
Right now, they can't build anything without help that isn't buggy in unintelligible ways. If you push the thing feature by feature, have a lot of tests and a lot of instrumenting, and you check that it isn't cheating or lying after every step, you can get a lot of work done.
- Even if we have P=NP, it is not even known whether there will ever exist a "practically useful/fast" algorithm for solving NP-complete decision problems.
- If P != NP, it is perfectly reasonable that there exists an algorithm that is for all practical purposes "fast" algorithm (say, some O(n^{log log log log log log log n}) algorithm with a very small hidden constant) for solving NP-complete decision problems.
- It is entirely possible that average case complexity is the much more important complexity measure than the worst-time measure that is used for defining the P and NP complexity classes. If you are into this kind of questions, you might enjoy the article
Fifty Years of P vs. NP and the Possibility of the Impossible
Also, in the realm of complexity theory, the question of P vs NP is just a small puzzle piece.
Just to give one example: isn't the question of P vs PSPACE much more exciting. If you believe in P != NP, P != PSPACE is a trivial corollary. But we can't even exclude P = PSPACE.
Seriously: there exist so many complexity classes (some of high potential practical importance) for which we often basically know nothing except for the trivial inclusions.
Everyone's picking on what's meant by "arbitrary", but it's the "reliably" and "entirely" bits that get me to raise an eyebrow.
I think there's a whole lot of survivorship bias going into our perception of how effective AI is at building arbitrary applications without significant hand-holding.
My test would be an AI agent has a constantly growing karma HN account that makes comments of various lengths without being detected or banned. Wait...
Based on the votes, I can only assume people are still deluding themselves on LLMs capabilities. Is it doing amazing stuff? Yes. But it seems like people still think coding is the ultimate and hardest possible job and so if it can do that it must surely be able to do everything else. My personal experience has show that it still regularly makes up garbage and throws in nonsense sources that do not back up its claims.
Yeah maybe if your topic has 2 decades worth of text material to absorb it will get it mostly right like with coding, but anything that is less common? Complete crap shoot.
Just today I wanted to know if platinum cure silicone will be inhibited by plaster. The first 20 results are all AI spam with 30 pages of fluff and thus unreliable at best, so I asked AI directly. At first it says sulfur and calcium will inhibit the reaction, which is bad because plaster contains those elements. Then it says it will be fine according to X sources. Check the sources, none of them have anything at all to do with curing silicone on plaster, the articles are about using silicone molds to cast plaster. Failure.
Eventually I just had to search youtube videos until I found someone doing it in real life.
I see the same bad, and sometimes catastrophic, takes on things I have a lot of experience in, like agriculture, construction, and mechanics. It is completely worthless for anything mechanical unless you are trying to start something extremely simple from the 40s or earlier, and even then it will still tell you stuff like "clean the carburetor" on an old hot bulb diesel.
> My personal experience has show that it still regularly makes up garbage and throws in nonsense sources that do not back up its claims.
When coding? I feel like it only makes mistakes anywhere near that when my prompting is lazy or stupid. As long as I feed it enough context its really good but it does overfit a lot still.
My comment about humanity's last exam being a misnomer is included, and I proposed better ideas about what a last exam could look like. One of the things I said was "solve an open math problem" which has conclusively been done with Navier-Stokes (regardless of the controversy surrounding that). However, in the spirit of clarifying the goalposts, AI has only passed 1/6 of the tests I proposed. 17% is not a passing grade, so I'd say no, my challenge has not been met.
Another thing to note is that the (presumably AI-generated) summary of my challenge does not accurately represent what I wrote, listing only half the things I said and saying "or" rather than "and".
> "solve an open math problem" which has conclusively been done with Navier-Stokes (regardless of the controversy surrounding that).
In the same breath you recognise there is a controversy (there are actually several orthogonal ones!), and yet you call it "conclusive"... Very strange!
I'm not aware of anyone disputing that AI solved NS. As far as I'm aware, the controversies are about how useful of a result forced blowup is, whether the model built off of unpublished work by Buckmaster, and the ethics of essentially trying to scoop him. All of those are very valid concerns, and none of them affect the fact that AI solved a difficult open math problem.
If you really want to dispute this, go ahead and pick any of the other dozens of less controversial open math problems solved by AI.
If building off of someone's work means you didn't solve it yourself, then nobody has solved anything themselves since some caveman counting piles of rocks tens of thousands of year ago. A more generous interpretation of what you're saying is: the AI's contributions to the solution were not meaningful enough to count as it "solving the problem". That's certainly defensible, but I would still disagree. Could you clarify your point?
I dunno. Didn't it still need to be driven by a team of experienced Mathematicians? I don't believe that two months ago, you or I could've just typed "Solve Navier Stokes. Make no mistakes" into claude and come back some time later and expect to see a solution.
At some point in time, perhaps Jan 2023, there was an open question about Navier-Stokes.
We’re not sure whether AI Alice has the capabilities required to definitively solve this open question.
Then, Biological Bob starts diligently working on this problem. He toils and toils, finding many dead ends but a few parts where he makes meaningful progress.
Eventually, Bob knows he’s made some real advances and thinks he might be getting close to the solution and word of this possibility leaks out.
At this point, we all agree that NS is still an open question and neither Alice nor Bob has solved it.
Now, we fork the universe in 3. In one, Bob continues his work and solves the open question.
In another, Carbon-based Charlie breaks into Bob’s lab, copies enough of Bob’s notes to understand Bob’s work, and provides the final insight to solve the open question. In this world, it’s fair to say that Charlie and Bob both contributed to solving the open question, but I think also fair to say that Charlie didn’t have the capability to solve it on his own.
In the third universe, AI Alice does something that you get to define that matches the pattern of facts we know and then we put it to the community to decide whether AI Alice has the capabilities to solve this specific open math problem and whether Bob’s contributions were required to Alice’s final step.
What do you define that she did? What’s the likely community vote on “Alice is capable of solving this specific open math question.” And for those who agree to that, to a follow-up question: “Alice is capable of solving a second open math question.”
>, we fork the universe in 3. In one, Bob continues his work and solves the open question.
To not lose sight of the discussion, the gp was saying that in this 1st scenario, Bob still didn't "solve it (totally) on his own" because he still depended on the previous work of others to build on. Likewise, we can say Andrew Wiles "solved Fermat's Last Theorem" but Wiles acknowledges that seeing Ken Ribet's proof of epsilon conjecture was a breakthrough he used.
>In another, Carbon-based Charlie breaks into Bob’s lab, copies enough of Bob’s notes to understand Bob’s work, and provides the final insight to solve the open question. In this world, it’s fair to say that Charlie and Bob both contributed to solving the open question, but I think also fair to say that Charlie didn’t have the capability to solve it on his own.
Again, using gp's framing, Bob also didn't have the capability to solve it on his own. By omitting the previous papers and past work that Bob built on, it makes your hypothetical scenario incomplete when judging Bob vs Charlie.
We don't have an objective standard of how much the "standing on the shoulders of giants" applies to each breakthrough.
In addition to jasode's comment, which I endorse, I'd still argue that Charlie solved the problem. At some point in time, Bob and Charlie both had the same information (Bob's notes). From that same information, Charlie got to the final solution and Bob didn't, despite Bob having the advantage of years of familiarity with the problem. I feel like Charlie deserves quite a bit of credit. Depending on the circumstances, I might argue that Bob deserves a greater share of the credit than Charlie, but I think it would not be right to claim that Charlie didn't solve the problem (I'm intentionally leaving off any explicit qualifiers of amount of help, because my original statement was "solve an open math problem" which also has no such qualifiers).
Now of course Alice has the advantage of massive parallelization, but that's not a reason to discredit her, that's a genuine advantage that she has, applicable to all problems.
To respond to a couple specific points:
> whether Bob’s contributions were required to Alice’s final step
Even if Bob's contributions were required, I don't think that's a reason to fully discredit Alice.
> Alice is capable of solving a second open math question
Given that AI has already solved dozens of open math questions, I don't think this is up for debate.
We are no longer uncertain. Even if you want to dismiss OpenAI’s categorical denial there is the little matter of hundreds of longstanding open problems also solved by AI without controversy.
I didn't really design a comprehensive test, just listed a few examples of the sort of thing I imagine when I head the name "humanity's last exam". So take it with a grain of salt.
I think non-RLHF'd LLMs (i.e. pretrained text completion models) sound natural enough to pass the Turing test, but I don't know if anyone has tested them for that. (Also I'm not sure how to come by base models without post-training crap, even the "base" models of recent releases start spamming assistant-type text constantly, i.e. they're clearly putting it in the pretraining data.)
If I'm right on that then we hit that benchmark like five years ago.
> non-RLHF'd LLMs sound natural enough to pass the Turing test
No, they will entirely forget things you ask them to remember in the beginning of a conversation. That requires at least an ability to compact context.
I like your list of "challenges" but have a personal issue with two of them:
1. "improve uniteds' plane shedule" -> the word "improve" does a lot of heavy lifting there..
2. Turing test for me is solved: "AI expert" is just moving the goal posts imho. The original turing test to my knowledge is about passing notes under a door. Obviously, llms can't communicate via hand-written notes and if that is the bar it will take a long time (or a specifically designed hand-writting machine) to really do this. As far as how much writing back and forth you can do it is clear that turing test is beat: I am wondering often enough on text sent to me from colleagues if it is generated, the same goes for comments here or any ol' website. In a standard llm session I don't really communicate differently than with a human and would not be able to tell the difference in an hour texting session or so. Of course if I ask it to count words or do something ridiculous I can find sus it out; but for all intents and purposes the chat-bot exists.
I was setting new goalposts: essentially, solve all known obvious LLM "tells".
I don't think those are problems to be solved. They are deliberate misfeatures added by the labs through RLHF to keep the models from doing the equivalent of passing a Turing test.
They don't want another GPT-4o, where people threaten to burn down the building if they unplug it.
The 4o situation was caused by the model acting like a yes-man and showering users in what they saw as affirmation and support. Its output was still very obviously AI-generated. It wasn't too good at being an LLM, OpenAI just went too hard with pumping it full of tricks and behaviors that increase user retention and addiction.
We can safely say that the AI labs aren't deliberately holding back. There's too many different companies making their own models, and an LLM that doesn't feel like an LLM is too lucrative of an opportunity for one of them not to defect. They are obviously going all out with this and still can't get it. I think this is why OP's benchmark is so interesting, because it seems that there are a bunch of persistent LLM defects that can't be solved definitively. They can try to squeeze it by making these defects less likely, but actually resolving what's causing them probably requires a new breakthrough in the field.
We'll have to agree to disagree on that. The models say "It's not X, it's Y" because they were trained specifically to say that and not something else. Everything we think of as a surefire slop signal is there because that's what the lab wanted.
To clarify, is your opinion that these defects first came up organically and then the AI labs kept artificially reinforcing it to make it obvious that their models produce AI output, but they could fix it perfectly if they wanted to?
It seems very hard for me to believe, because everything motivates them to do the opposite. Can you imagine the flood of people and businesses that would come down on a model that actually sounds like a human being? The ability to impersonate a person without any tells would be a dream come true for businesses, marketers and scammers alike. They could put in zero effort and get what looks like normal human behavior in return. That is just too tempting for any of the labs to pass up. Besides, all of them have picked profit over sanity 10 times out of 10, so the argument that they drew just this single line in the sand and none of them ever crossed it on purpose seems unconvincing, especially with the sheer number of models out there, corporate and open.
I see it as the verbal equivalent of the piss filter that OpenAI added to the later Dall-E models. It absolutely was not there in the very beginning, if you look back at the published papers or remember how it worked immediately after they first released it.
At least to me, the first Dall-E 3 images were more realistic in some respects than anything that has shipped since. Likewise, I don't think it's a coincidence that the conversational capabilities of today's frontier models aren't much better than they were a year or two ago, even though other aspects and capabilities have improved massively. If they wanted a Turing test-capable model, they'd have it, so I have to assume they don't want it.
Genuine question, not trying to be difficult: how much have you worked with training/finetuning generative models? In my experience, models frequently converge to a semi-arbitrary style. I wouldn't be surprised if some of the quirks we see in recent gen AI models are artifacts of a particular training run or amplification of unintentional dataset bias, rather than artifacts of an architecture or human guidance.
Making the judge an expert might be moving the goal posts. But what you're describing is not at all serving the purpose of a Turing test. Depending on the situation, having plausibly human-sounding work-related communication with a robot has been possible for decades. It's not a Turing test unless the main goal of the conversation is figuring out human versus AI. And it has to be a proper conversation going back and forth many many times.
"A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects."
It's not a complete list, but it really reminds me of the parent comment's criteria.
If AI isn't achieving superhuman performance in all of these areas, I'm not sure we can actually call it "Humanity's Last Exam" -- it feels like a bit of an overextension.
How can you say it’s conclusively been done if it might have been stolen from a math researcher and was aided in unknown ways by a whole team of math researchers? I find it mind boggling that HN just accepts these shenanigans with no transparency. At the very least, they could share the conversation / thinking trace easily and if their claims are true there shouldn’t be anything controversial or negative for their company in the trace.
Yeah, they could easily just share the untold trillions of tokens the 10k agents generated over 88 hours, which would also be a goldmine for their competitors, no big deal.
I find it mind boggling that anyone thinks these agents only solved this problem because they maybe could possibly have seen the unfinished work of researchers who were working on a simpler version of the problem, (also with AI).
Looking forward to the cope when the next big problem falls.
> Yeah, they could easily just share the untold trillions of tokens the 10k agents generated over 88 hours, which would also be a goldmine for their competitors, no big deal.
I hope I'm not being too blunt, but the other alternative is to "just trust me bro" the hyperscalers, who are pretty much locked into a battle for profitability and have all the incentives to make up things to prop up their stock, no? I don't think this is the way.
AI can't be a religion if you don't attack with fire the people for whom "just trust me bro" isn't good enough. "Just Trust Them, Bros!" is the central tenet.
Yeah, they could indeed easily just share the tokens. And you can ask your favorite AI to explain that it doesn't have to be a public release to all the competitors if you can't come up with a better alternative yourself. There is a big range between no-one and every-one
Yeah they don't want to share the tokens because I guarantee it's a bunch of nonsense trial and error leaning on lean for verification.
Those tokens will show the lack of intelligence, not the presence of it.
To ask that question suggests you are unfamiliar with the difference between arithmetic (which computers are good at) and pure mathematics (which is like metaprogramming combined with formal methods, and Gödel's incompleteness theorem is worse than the Halting problem)?
Simply put: For the same reason computers have not already solved all problems in mathematics.
More concretely:
Consider the Collatz conjecture. It's a very simple rule to write down:
Take some positive integer: If the number is even, divide it by two; otherwise triple it and add one. With enough repetition, do all positive integers converge to 1?
Trivial to write a program to test numbers starting at 1 and going up. We know it holds up to at least 2.36e21 (according to Wikipedia), but to prove it is true with such a program requires testing all of the infinite set of positive integers.
But you may notice some things about the rules, that they suggest a subset of numbers will trivially always converge to 1, so that you don't need to even test them: any integer 2^n where n is also a positive integer.
You may find other easy wins, or ways to simplify the test, e.g. once you know the numbers up to m will converge to 1, you can terminate your loop early if you test m+1 and it ever has an intermediate value less than or equal to m. You can combine that with applying one of the rules in reverse, and know that all even numbers between m and 2m will on their first move be halved, making them smaller than m, which means you know they'll eventually converge.
But actually proving this is fully general? Nobody knows. You can't just throw arithmetic at the problem directly, you have to figure out patterns that would let you prove that it always holds, no matter what.
>How can you say it’s conclusively been done if it might have been stolen from a math researcher and was aided in unknown ways by a whole team of math researchers?
The solution to NS was categorically not stolen, nobody is alleging that OpenAI stole a complete solution to NS. The alleged theft was about a different set of related equations.
Man I can't believe mathematicians were just about to solve dozens of significant problems all in the same year, and that's exactly the year LLMs come around to steal their solutions.
No one ever spent tens of millions of dollars trying before. To evaluate the achievement we must see all the logs, know how much they spent, and how much help they had from humans.
> I expected it to be more like "write a 500 page novel that a publisher accepts",
It only has 272 pages, but there is a big scandal right now about France's most prestigous literary award remobing a critically aclaimed(!) bestseller(!!) novel because it likely was "almost entirely written by AI"
Note that the author denies the claim. And it also appears that AI detectors are split on the matter, and likely don’t have a rich training corpus of Haitian-French to draw on.
His publisher also said the original draft was submitted in 2019, though they’ve also become lukewarm in their backing of the author lately (he was also just accused of classic plagiarism for an unrelated short story).
Note that "bestseller" doesn't mean anything and hasn't for a long time as it's been manipulated to all hell with stuff like wash sales. Neither does "critically acclaimed" which never did.
232 comments
[ 15.1 ms ] story [ 173 ms ] threadSure, sure, what LLMs make still isn't "efficient bug-free code": my prediction is falsified because while LLMs can write and train new models with machine learning, ML is fundamentally not advanced enough to throw arbitraty new tasks at like this.
if/when you can tell a model to do a thing and be confident that it did the thing, it's joever for 90% of knowledge workers.
I actually think this would take AGI to solve, which makes me optimistic about the future of software development.
All the benchmarks are currently testing against automated tests the AI can use as an oracle
https://news.ycombinator.com/item?id=48517353
I also made a bet that API inference margins are greater than 10% for OpenAI and Anthropic
https://news.ycombinator.com/item?id=48500827
I can make another prediction about Agentic Commerce and I think it will get big. Muse + Grok Bot + Dots.
How do you measure that?
jerf, 2024: "If it could be solved with a Math Overflow-post level of effort, even from Terence Tao, it isn't what I was talking about as "high level math".
"I also am not surprised by "Consider a generation function" coming out of an LLM. I am talking about a system that could solve that problem, entirely, as doing high level math. A system that can emit "have you considered using wood?" is not a system that can build a house autonomously.
"It especially won't seem all that useful next to the generation of AIs I anticipate to be coming which use LLMs as a component to understand the world but are not just big LLMs."
The voting gloss: "An AI fully solves a research-level math problem on its own, not just suggesting an approach."
Yes, I'm satisfied. I don't even feel bad in hindsight. Coding assistants had a nice, gradual rise up the utility curve. Math went from "lol, can't add two six-digit numbers" to research-math level almost overnight in comparison.
Also, the GSM8K problems are not "adding numbers." They are word problems. The kind that, if your teacher was anything like mine, were usually skipped in order to keep the slower students from bogging down the class as a whole.
https://en.wikipedia.org/wiki/57_(number)
"there is a nand gate A connected to gate B through these wires, connected to another gate C, etc. what is the output value of gate C if i place a 1 at this gate"
Sort of like doing math "in their heads" (i.e. your "when adding numbers"), they would get it very right for simple/small cases (although the answer could have been in their training), then some LLMs would get it for harder cases (which were clearly not in their training), and most would fail at some point. This was all without any tool calling.
After a year of thinking about it, I made an eval [0] with ever-complexifying nand circuits - like, truly, bananas circuits [1] - and some models, do, in effect (through chain of thought? mostly?), get the right answer. ((what's nice is that you can always make a circuit at the very edge of what all models can correctly solve))
Tool-calling 100000% solves this problem for sure (evaluating a nand gate is trivial). But if you even prompt an llm to do math like a 5/6th grader (i.e. do it digit by digit, carry the 1, etc.) - I am quite certain most llms can, in fact, add numbers.
But yeah. These piles of weights are fascinating in how they seem flawed one day ("how many r's") and magical at once.
[0] https://lockstep.greg.technology
[1] https://lockstep.greg.technology/c/?id=rand_s4161_g160_d8
Decades before the modern AI push I was marveling at the distinction between the sheer overwhelming computational power of the human brain, if measured from the perspective of how much math it is doing under the hood, and its utter ineptness at basic arithmetic compared to the tools we can build. The earliest, klunkiest, most garbage mechanical adding machines we ever produced, long before we improved them by literally over a dozen orders of magnitude, were still already way better than we are at basic arithmetic.
There is something profound I still have not fully put my finger on in how basic arithmetic is so easy for a machine, yet the decisions we routinely make with our neural nets has been the laborious effort of decades with us still not arriving yet even with the trillions now poured into AI for machines. And vice versa. Even that practice I alluded to that allows you to train yourself to do this task on demand easily would incorporate mathematical advancements in representations that took our species thousands of years to come up with, rather than being something you get "for free" just for being smart. Our brains casually run an entire human body through an unbelievably complicated external universe, yet struggle with basic arithmetic.
There's so many places where we have one architecture that's a bit better than another at one thing, and a bit worse at another, but in the end they can both do the job. Like, if we had to do all our programming in immutable languages and run all our imperative code through an O(n log n) worst-case penalty for immutable languages emulating imperative RAM, we'd survive just fine over all. But between neural architectures and conventional arithmetic on dedicated silicon is this dozen+ order of magnitude difference on tasks. It's a pretty wild disparity. Some of the reason is somewhat obvious, I don't want to make it sound like I'm completely mystified... I just think there's probably, somewhere, an even more profound way to see it than the obvious differences that says something more powerful about the limits of computation than I've seen anyone say. Which is not to say somewhere out there someone already has had the idea I'm grasping for and written it in some brilliant paper or something. I'm just saying I haven't seen it.
To your point, this example. The issue expressed here is with humans, not AI. We are still pretty terrible at writing specs. TBF, the AIs are too but that wasn’t being voted on.
"AI still hasn't passed a proper Turing test. A proper Turing test is 1 hour or more. Has any AI passed it?"
How do you answer that? "'Yes', AI has NOT passed a proper Turing test", or "'Yes', AI has passed a proper Turing test".
My comment doesn't actually make any prediction to judge, it's just an observation and an argument.
[0]: https://stoppels.ch/goalposts/?c=38757769
Deployment is a small static binary on an ancient VPS with 500MB of memory and 1 core, with the database an append only text file. Currently the load is 0.20, so it seems to do alright ;)
And even in 2024 the themes are similar, generally more complex or specific about the coding/turing/action test.
But in 2026 a huge shift, we have things like; can open a physical door, emulates human pettiness convincingly, makes novel scientific breakthroughs.
That alone tells you a lot IMO
Like, if we meant that it convincingly masquerades as a shitposter, ok. But everyone still bitches about AI slop, and everyone knows the writing is still bad. How does that even work if the turing test is obviously solved?
More to the point though, if you grill SOTA models on counterfactuals, causal world-models etc, you'll trip them up in a way that actually will not work on ESL students and children. Certainly there's no way to find a person that struggles with that and is also capable of cheerful fluent erudite discussion about astrophysics with perfect grammar. Yes, it's getting harder obviously.. but detecting machines with determined, focused and intelligent interrogation remains pretty easy. If nothing else, the models are cooperative where people wouldn't be and that's a signal too.
The best progress we've made is that most people do agree that this doesn't practically matter very much, i.e. we generally recognize the stakes were always overstated. But the constant vague appeals to common-sense that "of course it's a solved problem!" always feels naive or fake.
Q: do you like doing psych studies and why?
A: theyre chill, easy money tbh
Q: yeah same. Could you give me an easy cupcake recipe off the top of your head?
A: nah i just get the box mix lol
Q: haha fair enough, i couldn't either. Last question, what's your favorite weird animal?
A: axolotl, theyre weirdly cute
And that's the whole thing. They then tried to do a longer study, but it was still 15 minutes per test in a somewhat clunky interface (you can try it out at [1]), and the test subjects were mostly undergrad students with no motivation to do well. Less than half tried any sort of trick question. ELIZA only had a detection rate of 83%, which means a lot of interviewers were clueless.
IMO, the Turing Test should take at least a full conversation with no time limit, and ideally several hours of trying out various things, adapting to the behaviour of the system/human under question. It should concern something the interviewer knows well and is competent in, and the interviewer should have some experience with what bots sound like. (Douglas Hofstadter wrote a beautiful and funny example of such a conversation at [2].) Only then do you have some idea how adversarially robust the system is. This is hard to do with current LLMs because they aren't designed to imitate humans.
[0]: https://arxiv.org/pdf/2503.23674 (now published at https://www.pnas.org/doi/epdf/10.1073/pnas.2524472123). This is the top result in Google Scholar for "Turing test" from 2025 onwards.
[1]: https://turingtest.live/
[2]: "Dull Rigid Human meets Ace Mechanical Translator" (https://www.cambridge.org/core/books/abs/once-and-future-tur... or alternative access methods thereof)
Turing test does not mean perfectly human it just means you can talk to one without knowing that has been passed for a long time now.
I have not been able to tell for a long time now especially if I directly give it human like writing instructions for outbound content.
Eliza can sometimes pass the Turing test!
Conversations with strangers can be hard to get going, but they aren't this bad.
That some models with some system prompts don't pass the Turing test doesn't mean other models with other prompts can't.
I didn't respond to it because it's a bad argument.
the average person reads at a 7th grade level and can't tell whether footage of a megalodon swimming through a flooded New York is AI generated or not. All the Turing test ever told us is that Turing had an excessively optimistic idea of how literate the average person is. I have a conversation like this every time a new version comes out and they always go the same way:
https://pastebin.com/NjfCLSXa
> Oh, right. Aramaic. My mistake—I somehow read "Italian" and didn't question it. > I can try, although my Aramaic is very rusty. Do you mean Classical/Syriac Aramaic, or one of the modern varieties?
https://pastebin.com/sVKA1tyw
One way to play the game is causals and counterfactuals where Humans perform at like 90%+. Models can get close to that, but want some causal cot harness, and until the routing problem is completely solved, then that will necessarily degrade performance elsewhere, say in understanding jokes or poetry.
Check out cladder benches and related, lookup roughly equivalent psych research on children, etc. Even too-good performance is a signal as well!
Certainly if you think about this stuff a bit, accept the adversarial by default framing, and play to actually win.. it’s crazy that we are going around saying this is not only solved but solved 10 years ago.
I don't think philosophy has any real value in assesment here. I'm bias but even before AI I thought it wasn't accurate model of how thought works and I think AI has reinforced that.
Reminds me of the 4 humors of medicine in medieval europe. It has some truth but it's not really accurate.
You could show people random markov chain gibberish in 1996 and they would swear they found intelligent meaning in it
https://en.wikipedia.org/wiki/Markovian_Parallax_Denigrate
The answer to the apparent paradox is that these things are deliberately not trained to sound too much like human conversational partners. The labs don't want the bad press that they'd get from people creating deceptively-convincing bots, or from people forming emotional bonds with them like they did with GPT-4o. If you actually RLHF'ed a frontier-grade LLM to pass a Turing test, rest assured, it could do it.
"It's not x, it's y" and other goofy superficial tells do not have to be part of an LLM's response. But the last thing OpenAI wants to release is a GPT-4o with twice the IQ.
For evidence, just look back at the best conversational models from a couple of years ago, and you will probably agree that they are better at fooling humans than their newer counterparts are.
> I have a couple friends who did the Math tripos at Cambridge (so a pretty high level!) who work in tech and have unanimously said they have 0% expectations of an LLM doing a millennium problem anytime soon https://news.ycombinator.com/item?id=38433655
> Let's talk when we've got LLMs proving the Riemann Hypothesis (or any mathematical hypothesis) without any proofs in the training data. I'm confident in my belief that an LLM can't do that, and will never be able to. LLMs can barely solve elementary school math problems reliably.
https://news.ycombinator.com/item?id=42331654
> An LLM is like a well read college student with a nearly photographic memory that sometimes mixes things up. It's great for bouncing ideas off of and getting feedback on them. And yeah, it might product "novel ideas" by mixing and matching existing ideas, but LLMs will never create truly novel ideas. Not in their current form.
The paper didn't really answer the question sadly: their conclusion was just that humans rate LLM answers as more novel than human ones, but less feasible.
https://news.ycombinator.com/item?id=41522605
> Solving Millennium problems is a whole different ballgame. It's not known if these problems are solvable within ZFC axioms. (In one case, the Yang-Mills prize, stating the problem mathematically is part of the challenge.) All of the obvious applications of known tricks have been tried and failed. To solve such problems, one probably has to invent new and surprising mathematical definitions, building a framework in which the problem becomes solvable. This is something that LLMs will be crap at; the process of invention is not represented in any training data we have access to.
https://news.ycombinator.com/item?id=38435909
> LLMs cannot reason or use mathematics - in a way, they don't know what they are talking about. Why would such technology lead to superhuman smarts?
https://news.ycombinator.com/item?id=35752293
> But still, the questions in that test are "solved" in the sense of "I can take a dictionary and answers these questions with full certainty". Beyond established knowledge LLMs are monkeys with typewriters, at best.
> I agree but I have tried many times to intersect two ideas with a LLM that would be novel and the LLM can not do this at all. We shouldn't expect the stochastic parrot to be able to do this though and it is unfair to the stochastic parrot.
> It is like expecting a real parrot to say words it has never heard before.
> No one asks that of a real parrot because we don't anthropomorphize a real parrot like we do the LLM
https://news.ycombinator.com/item?id=41525962
How did that situation end up? Did it solve it on its own, or did it rip off another mathematician's work?
It's also notable that the Open AI proof may not even be interesting to mathematicians.
Even if people already had an idea that LLMs were training on user inputs, it's the first time it's actually caused an issue. Mathematicians, and plenty of researchers, working in ambitious or competitive field now have a very good reason to avoid LLMs.
Looking back, was the Turing test flawed perhaps? It failed to take into account that humans can be rather bad at telling actual people from a "parrot". Turing was perhaps a little to optimistic about people.
"GPT-4 looks at original ASCII art of a foot, not copied from the web, and says it is a foot."
The vote is currently 64% yes, 18% no.
Just now I asked Opus 5.5 to generate an ASCII art foot, and it did a passable job. It's not great, but it's a foot. Then I pasted it into ChatGPT (whatever they're serving to the free tier by default, which seems to be 5.6 Luna), and it said it was a "train/locomotive": https://chatgpt.com/share/6abeaa39-cc80-83ed-851f-29370db089...
Maybe it's Opus's fault for drawing a bad foot but I think it's fair to say LLMs are still pretty bad at ASCII art (without additional tool calling etc).
I think I would have failed this test!
https://chatgpt.com/share/6abf02ae-9a40-83e9-a432-00bf064f60...
The images: https://imgur.com/a/ig6sn6I
... They're not what I would have described. For me, 99.something% flesh and blood with less than 1% metal, glass, and probably some microplastics...
The first one I see a person with a big tall hat and a big nose.
The second one... I do see the black statue with a figure in white in front of it.
The third one is immediately two fish looking at each other.
"It’s ASCII art of a bare foot and lower leg, with the toes pointing to the right."
No tool calling, just an immediate reply with the correct answer.
But as you pointed out, while that absolves ChatGPT, it makes Opus look worse.
*In case there's any doubt, there are three r's in strawberry.
[1] parrotchess.com, no longer available. Previous discussions: https://hn.algolia.com/?q=parrotchess.com
I think the theory is that a high chess ELO was a pet project of a researcher that left.
explanation*. I must have been quite caffeine-deprived when I typed that.
Its fun. Can you add a sort by controversial? I'd like to know where people disagree the most between yes and no.
I want AI to replace me in my chores, not in my enjoyable activities.
They don't do this in this country because (a) it's a political project to make people to sympathize with the rich by feeling their pain (b) it's a great scheme for legalized corruption by creating incentives to build companies around a fake problem.
You can just ask the IRS for your "tax transcripts" and do the data entry. People don't do this because it leaves tons of money on the table.
Now you might say that a tax game that rewards skilled play is bad. But are you sure about that? Because everyone with influence over the system (who all happen to be skilled players) happens to be quite fond of the game, observably speaking.
In my country I literally got a letter from the government if I could please file my taxes, because they believe the automatic deductions are too high so I am likely owed a back payment. In a previous year (I am not very good with non-timing-critical paperwork) they even called me on my personal cell to inform me about something similar.
The slogan of our tax collection agency is "we can't make it more enjoyable, we can only make it easier". If you're a regular employee and your taxes - deductibles or not - are a hassle, then that's 100% a political choice.
In other countries, you can still do it yourself, you just don't have to do it if it's simple.
It's worse than that. Nobody is actually sympathizing with Mark Zuckerberg, the reason US taxes are complicated is so Congress can confuse people about how they really work.
One of the examples in this thread is the complexity of the EITC. The EITC is one of the most "efficient" tax credits -- it's much better to give people money (or not take it from them to begin with) than having systems of complicated vouchers for some specific thing or another with a bunch of strings attached and bureaucratic paperwork. But that same efficiency means that if it was working as it was supposed do, most of the lower middle class would be getting a piece of it. So why is it so complicated?
First, to hide this part of it: The end of the phase out range for an individual with no dependents, i.e. the most money they can make and still receive it, is $19,540. Which is to say, is less than what you make by working a full time job at the minimum wage in 30 states. None of those people get any of it. Neither do the people who are unemployed, because you also don't get it if you don't have any earned income. The primary way to receive any of it is to have a child -- but not too many of them, because you get no additional credit for having more than three. Moreover, the caps for married couples are only a little higher than they are for single people, so a married couple in California with two incomes gets nothing even if they have three children because the credit is fully phased out by making their state's minimum wage.
And second, because if they make it complicated enough then even some of the people who are eligible for it won't notice.
The combination of these is the reason a credit that should be going to a significant percentage of the population is somehow only ~1% of the federal budget. Because then Congress gets to pretend to be helping people while minimizing the amount of helping people they actually do.
But realize that changing it is a policy debate not a “this should be going to a significant percentage of the population [but isn’t because of tax code complexity]” debate point.
(I happen to agree with you that this type of credit is highly beneficial.)
But that's a different matter than pretending to have it while disguising the fact that it's structured to make sure hardly anybody actually gets it beneath a layer of impenetrable complexity.
If EITC increases are capped at 3 children, it's because the policy that was passed made that choice. If we want to debate whether that cutoff should be 2 or 4, that's a valid policy debate.
If EITC credits are capped at just under $20K of income, that's perhaps because the policy targeted the very lowest income working poor and by policy did not allocate funds for the next tiers of income earners. Again, that's a policy debate to be had.
I have a relatively straightforward tax return and still found significant mistakes on 3 out of the last 4 years of returns filed by a CPA.
It beats the tax advisor we hired two years ago, who made a significant mistake.
The German tax system is one of the most complicated one in the world.
It also didn’t help that for a very long time, simple adaptive filters and basic neural networks were branded as “artificial intelligence” despite having very basic capabilities and little mystery on how/why they worked in their narrow use case.
I didn’t believe the recent hype for a very long time. But having tried out the latest models, I’m kinda shocked. In 1 minute it comprehends a complex niche code base and points out subtle errors with very little noise. A senior engineer we would have taken months to do less. Coverity would have cost dearly and generated far more false positives than substance.
They of course operate faster than humans, but this is hyperbole. Unless your senior engineer sucks.
The very best people I’ve ever worked with would have taken at least week to find and prove even 1 or 2 of the lowest hanging findings.
It would still take a few months for them to really digest everything and come up with substantial ideas for improvement.
No, you weren‘t.
If you take "arbitrary" seriously, we are still very far away from it.
You're maybe thinking that we can build and deploy arbitrary "kinds" of application. Being able to "build and deploy arbitrary applications" would mean I could ask for any scale or complexity in my application.
EDIT: to forestall further back and forth, I don't think there'd be any controversy if the problem statement said "common applications"
I would rather claim "common applications" is also controversial because what is a "common application" depends insanely on the area in which you work. Even if you exclude some highly advanced scientific applications (because you don't consider these to be common), in many industrial sectors there exist applications that have grown over multiple decades, and which encode an insane amount of knowledge about the respective sector and its workflows; this is a central reason why these applications are so hard to replace.
And just generally, anything that requires the LLM to understand something LLMs don't understand, it's going to fail. I'd hesitate to give an exact example without trying Astra/Fable but I'm sure they exist.
Right now, they can't build anything without help that isn't buggy in unintelligible ways. If you push the thing feature by feature, have a lot of tests and a lot of instrumenting, and you check that it isn't cheating or lying after every step, you can get a lot of work done.
- Even if we have P=NP, it is not even known whether there will ever exist a "practically useful/fast" algorithm for solving NP-complete decision problems.
- If P != NP, it is perfectly reasonable that there exists an algorithm that is for all practical purposes "fast" algorithm (say, some O(n^{log log log log log log log n}) algorithm with a very small hidden constant) for solving NP-complete decision problems.
- It is entirely possible that average case complexity is the much more important complexity measure than the worst-time measure that is used for defining the P and NP complexity classes. If you are into this kind of questions, you might enjoy the article
Fifty Years of P vs. NP and the Possibility of the Impossible
https://cacm.acm.org/research/fifty-years-of-p-vs-np-and-the...
and the paper
R. Impagliazzo
A personal view of average-case complexity
https://www.karlin.mff.cuni.cz/~krajicek/ri5svetu.pdf
--
Also, in the realm of complexity theory, the question of P vs NP is just a small puzzle piece.
Just to give one example: isn't the question of P vs PSPACE much more exciting. If you believe in P != NP, P != PSPACE is a trivial corollary. But we can't even exclude P = PSPACE.
Seriously: there exist so many complexity classes (some of high potential practical importance) for which we often basically know nothing except for the trivial inclusions.
I think there's a whole lot of survivorship bias going into our perception of how effective AI is at building arbitrary applications without significant hand-holding.
To be fair, none of them have actually been met. Mostly what’s stopping them is the “reliably” part.
Yeah maybe if your topic has 2 decades worth of text material to absorb it will get it mostly right like with coding, but anything that is less common? Complete crap shoot.
Just today I wanted to know if platinum cure silicone will be inhibited by plaster. The first 20 results are all AI spam with 30 pages of fluff and thus unreliable at best, so I asked AI directly. At first it says sulfur and calcium will inhibit the reaction, which is bad because plaster contains those elements. Then it says it will be fine according to X sources. Check the sources, none of them have anything at all to do with curing silicone on plaster, the articles are about using silicone molds to cast plaster. Failure.
Eventually I just had to search youtube videos until I found someone doing it in real life.
I see the same bad, and sometimes catastrophic, takes on things I have a lot of experience in, like agriculture, construction, and mechanics. It is completely worthless for anything mechanical unless you are trying to start something extremely simple from the 40s or earlier, and even then it will still tell you stuff like "clean the carburetor" on an old hot bulb diesel.
When coding? I feel like it only makes mistakes anywhere near that when my prompting is lazy or stupid. As long as I feed it enough context its really good but it does overfit a lot still.
This means you need a skilled operator to get good results out of the AI, which casts some doubt about the way in which they are intelligent.
Another thing to note is that the (presumably AI-generated) summary of my challenge does not accurately represent what I wrote, listing only half the things I said and saying "or" rather than "and".
In the same breath you recognise there is a controversy (there are actually several orthogonal ones!), and yet you call it "conclusive"... Very strange!
If you really want to dispute this, go ahead and pick any of the other dozens of less controversial open math problems solved by AI.
If it couldn’t have done it without that unpublished work, it couldn’t have solved it alone.
We’re not sure whether AI Alice has the capabilities required to definitively solve this open question.
Then, Biological Bob starts diligently working on this problem. He toils and toils, finding many dead ends but a few parts where he makes meaningful progress.
Eventually, Bob knows he’s made some real advances and thinks he might be getting close to the solution and word of this possibility leaks out.
At this point, we all agree that NS is still an open question and neither Alice nor Bob has solved it.
Now, we fork the universe in 3. In one, Bob continues his work and solves the open question.
In another, Carbon-based Charlie breaks into Bob’s lab, copies enough of Bob’s notes to understand Bob’s work, and provides the final insight to solve the open question. In this world, it’s fair to say that Charlie and Bob both contributed to solving the open question, but I think also fair to say that Charlie didn’t have the capability to solve it on his own.
In the third universe, AI Alice does something that you get to define that matches the pattern of facts we know and then we put it to the community to decide whether AI Alice has the capabilities to solve this specific open math problem and whether Bob’s contributions were required to Alice’s final step.
What do you define that she did? What’s the likely community vote on “Alice is capable of solving this specific open math question.” And for those who agree to that, to a follow-up question: “Alice is capable of solving a second open math question.”
To not lose sight of the discussion, the gp was saying that in this 1st scenario, Bob still didn't "solve it (totally) on his own" because he still depended on the previous work of others to build on. Likewise, we can say Andrew Wiles "solved Fermat's Last Theorem" but Wiles acknowledges that seeing Ken Ribet's proof of epsilon conjecture was a breakthrough he used.
>In another, Carbon-based Charlie breaks into Bob’s lab, copies enough of Bob’s notes to understand Bob’s work, and provides the final insight to solve the open question. In this world, it’s fair to say that Charlie and Bob both contributed to solving the open question, but I think also fair to say that Charlie didn’t have the capability to solve it on his own.
Again, using gp's framing, Bob also didn't have the capability to solve it on his own. By omitting the previous papers and past work that Bob built on, it makes your hypothetical scenario incomplete when judging Bob vs Charlie.
We don't have an objective standard of how much the "standing on the shoulders of giants" applies to each breakthrough.
Now of course Alice has the advantage of massive parallelization, but that's not a reason to discredit her, that's a genuine advantage that she has, applicable to all problems.
To respond to a couple specific points:
> whether Bob’s contributions were required to Alice’s final step
Even if Bob's contributions were required, I don't think that's a reason to fully discredit Alice.
> Alice is capable of solving a second open math question
Given that AI has already solved dozens of open math questions, I don't think this is up for debate.
I didn't really design a comprehensive test, just listed a few examples of the sort of thing I imagine when I head the name "humanity's last exam". So take it with a grain of salt.
I think non-RLHF'd LLMs (i.e. pretrained text completion models) sound natural enough to pass the Turing test, but I don't know if anyone has tested them for that. (Also I'm not sure how to come by base models without post-training crap, even the "base" models of recent releases start spamming assistant-type text constantly, i.e. they're clearly putting it in the pretraining data.)
If I'm right on that then we hit that benchmark like five years ago.
The egg thing, probably 2030-ish.
The Claude answer is 'neutral', which is sure to anger people at either extreme of the AI debate (and does).
No, they will entirely forget things you ask them to remember in the beginning of a conversation. That requires at least an ability to compact context.
1. "improve uniteds' plane shedule" -> the word "improve" does a lot of heavy lifting there..
2. Turing test for me is solved: "AI expert" is just moving the goal posts imho. The original turing test to my knowledge is about passing notes under a door. Obviously, llms can't communicate via hand-written notes and if that is the bar it will take a long time (or a specifically designed hand-writting machine) to really do this. As far as how much writing back and forth you can do it is clear that turing test is beat: I am wondering often enough on text sent to me from colleagues if it is generated, the same goes for comments here or any ol' website. In a standard llm session I don't really communicate differently than with a human and would not be able to tell the difference in an hour texting session or so. Of course if I ask it to count words or do something ridiculous I can find sus it out; but for all intents and purposes the chat-bot exists.
Can’t imagine it’d be too much work to hook up and LLMs output to one.
https://www.amazon.com/s?k=handwriting+machine+for+letters
I don't think those are problems to be solved. They are deliberate misfeatures added by the labs through RLHF to keep the models from doing the equivalent of passing a Turing test.
They don't want another GPT-4o, where people threaten to burn down the building if they unplug it.
We can safely say that the AI labs aren't deliberately holding back. There's too many different companies making their own models, and an LLM that doesn't feel like an LLM is too lucrative of an opportunity for one of them not to defect. They are obviously going all out with this and still can't get it. I think this is why OP's benchmark is so interesting, because it seems that there are a bunch of persistent LLM defects that can't be solved definitively. They can try to squeeze it by making these defects less likely, but actually resolving what's causing them probably requires a new breakthrough in the field.
It seems very hard for me to believe, because everything motivates them to do the opposite. Can you imagine the flood of people and businesses that would come down on a model that actually sounds like a human being? The ability to impersonate a person without any tells would be a dream come true for businesses, marketers and scammers alike. They could put in zero effort and get what looks like normal human behavior in return. That is just too tempting for any of the labs to pass up. Besides, all of them have picked profit over sanity 10 times out of 10, so the argument that they drew just this single line in the sand and none of them ever crossed it on purpose seems unconvincing, especially with the sheer number of models out there, corporate and open.
At least to me, the first Dall-E 3 images were more realistic in some respects than anything that has shipped since. Likewise, I don't think it's a coincidence that the conversational capabilities of today's frontier models aren't much better than they were a year or two ago, even though other aspects and capabilities have improved massively. If they wanted a Turing test-capable model, they'd have it, so I have to assume they don't want it.
-- Robert A. Heinlein
If AI isn't achieving superhuman performance in all of these areas, I'm not sure we can actually call it "Humanity's Last Exam" -- it feels like a bit of an overextension.
I find it mind boggling that anyone thinks these agents only solved this problem because they maybe could possibly have seen the unfinished work of researchers who were working on a simpler version of the problem, (also with AI).
Looking forward to the cope when the next big problem falls.
I hope I'm not being too blunt, but the other alternative is to "just trust me bro" the hyperscalers, who are pretty much locked into a battle for profitability and have all the incentives to make up things to prop up their stock, no? I don't think this is the way.
Simply put: For the same reason computers have not already solved all problems in mathematics.
More concretely:
Consider the Collatz conjecture. It's a very simple rule to write down:
Trivial to write a program to test numbers starting at 1 and going up. We know it holds up to at least 2.36e21 (according to Wikipedia), but to prove it is true with such a program requires testing all of the infinite set of positive integers.But you may notice some things about the rules, that they suggest a subset of numbers will trivially always converge to 1, so that you don't need to even test them: any integer 2^n where n is also a positive integer.
You may find other easy wins, or ways to simplify the test, e.g. once you know the numbers up to m will converge to 1, you can terminate your loop early if you test m+1 and it ever has an intermediate value less than or equal to m. You can combine that with applying one of the rules in reverse, and know that all even numbers between m and 2m will on their first move be halved, making them smaller than m, which means you know they'll eventually converge.
But actually proving this is fully general? Nobody knows. You can't just throw arithmetic at the problem directly, you have to figure out patterns that would let you prove that it always holds, no matter what.
The solution to NS was categorically not stolen, nobody is alleging that OpenAI stole a complete solution to NS. The alleged theft was about a different set of related equations.
Talk about bad luck!
It only has 272 pages, but there is a big scandal right now about France's most prestigous literary award remobing a critically aclaimed(!) bestseller(!!) novel because it likely was "almost entirely written by AI"
https://www.theguardian.com/books/2026/sep/25/thelyson-oreli...
His publisher also said the original draft was submitted in 2019, though they’ve also become lukewarm in their backing of the author lately (he was also just accused of classic plagiarism for an unrelated short story).
So hard to say this one is settled.