Is it just an open secret that the models are currently just being hardcoded for random benchmarks? Seems weird that people would be asking Putnam problems to a chatbot :/
"Overfitting" would be a bit more accurate term if the problem lies in the specific examples existing in its training set in various forms, places, languages etc but with the same values.
20 years ago in grad school we were doing a very early iteration of this where we built Markov chains with Shakespeare's plays and wanted to produce a plausibly "Shakespearian" clause given a single word to start and a bearish professor said "the more plausible it gets the more I worry people might forget plausibility is all that it promises".
(There was also a much earlier piece of software that would generate semi-intelligible Kant or Hegel one sentence at a time, though that was through a series of a priori generation rules and a large at the time dictionary of stock phrases. I wonder what ever happened to that.)
I've always assumed they removed it, because it's such a basic and fundamental part of ML training that you separate your test and train data. And yet I never see any papers even mention if/how they do this. And I wonder if they do, how do they guarantee with high reliability that their massive terabytes of data don't contain the answer.
Imagine you have someone polluting your training data every day. That's what happens when you scrape any tech forum today.
The short version is that llm trainign data is the lowest quality data you are likely to see unless you engage in massive potential copyright infringement.
First of all, Putnam is not in the test data, at least I haven't seen OpenAI claiming that publicly. Secondly, removing it from internet data is not 100% accurate. There are translations of the problems and solutions or references and direct match is not enough. MMLU and test set benchmarks show more resilience though in some previous research.
OpenAI is extremely cagey about what's in their test data set generally, but absent more specific info, they're widely assumed to be grabbing whatever they can. (Notably including copyrighted information used without explicit authorization -- I'll take no position on legal issues in the New York Times's lawsuit against OpenAI, but at the very least, getting their models to regurgitate NYT articles verbatim demonstrates pretty clearly that those articles are in the training set.)
> Putnam is not in the test data, at least I haven't seen OpenAI claiming that publicly
What exactly is the source of your belief that the Putnam would not be in the test data? Didn’t they train on everything they could get their hands on?
These models are trained in two steps: training base model and then uptraining it. First step includes as much data as possible, everything company can find. For Llama models it's 15T tokens, which is ~40 TB of data. No-one really puts an effort on splitting this data into train/test/eval (and it's not very achievable either). It's just as much data as possible.
So it's like 99.9999999% wrong to assume something public isn't on the train set, such as Putnam problems in this case. This is about it.
the point is that putnam was never a test/benchmark being used by OpenAI or anyone else, so there is no smoking gun if you find putnam on the train set nor is it cheating or nefarious because nobody ever claimed otherwise.
this whole notion of putnam as test being trained on is a fully invented grievance
I've read the thread and I think it's not very coherent overall, I also not sure if we disagree =)
I agree that having putnam problems on OpenAI training set is not a smoking gun, however it's (almost) certain they are on training set, and having them would affect performance of the model on them too. Hence research like this is important, since it shows that observed behavior of the models is memoization to large extent, and not necessarily generalization we would like it to be.
nobody serious (like OAI) was using the putnam problems to claim generalization. this is a refutation in search of a claim - and many people in the upstream thread are suggesting that OAI is doing something wrong by training on a benchmark.
OAI uses datasets like frontiermath or arc-agi that are actually held out to evaluate generalization.
I, actually, would disagree with this.
To me ability to solve frontiermath does imply ability to solve putnam problems too. Only with putnam problems being easier - they are already been seen by the model, and they are also simpler problems. And just like this - putnam problem with simple changes are also one of the easier stops on the way to truly generalizing math models, with frontiermath being one of the last stops on the way there.
There are benchmarks that are decided beforehand and similar sentences are removed from even the first stage of training. This is useful for tracking model performance and comparing different choices. e.g. see section 'Contamination of downstream tasks' of [1].
Every decent AI lab does this, else the benchmark result couldn't be trusted. OpenAI publishes results of ~20 benchmarks[2] and it is safe to assume they have made reasonable attempt to remove it from training set
I don't see any reason to assume they removed it unless they're very explicit about it. Model publishers have an extremely strong vested interest in beating benchmarks and I expect them to teach to the test if they can get away with it.
I think it's reasonable to assume that openAI is optimising for maximum hype at this point which may include wilfully overfitting for impactful benchmarks to generate positive reports.
When 4 came out they released a document that did BOTH inflate scores by changing the exam conditions, and also bragged about scoring worse than guessing on a multiple choice test.
I agree that openai is somewhat sketchy about this, but they're sketchy about everything. In the past though they have admitted up front to data contamination (e.g. the original gpt-4 press release did not use big-bench as a benchmark due to data contamination). For the Putnam in particular: this is not a benchmark that they use. There is no reason to exclude it since it is not part of the "test set" in any meaningful sense.
It certainly feels like certain patterns are hardcoded special cases, particularly to do with math.
"Solve (1503+5171)*(9494-4823)" reliably gets the correct answer from ChatGPT
"Write a poem about the solution to (1503+5171)*(9494-4823)" hallucinates an incorrect answer though
That suggests to me that they've papered over the models inability to do basic math, but it's a hack that doesn't generalize beyond the simplest cases.
To be clear I was testing with 4o, good to know that o1 has a better grasp of basic arithmetic. Regardless my point was less to do with the models ability to do math and more to do with OpenAI seeming to cover up its lack of ability.
“a poem about” reads to me at least like the solution need not be in the answer; maybe something like “a poem that includes the answer in the last stanza”
There's a few things there that could be going on that seem more likely than "hardcoded".
1. The part of the network that does complex math and the part that write poetry are overlapping in strange ways.
2. Most of the models nowadays are assumed to be some mixture of experts. So it's possible that saying write the answer as a poem activates a different part of the model.
Watch for ChatGPT or Claude saying "analyzing" - which means they have identified they need to run a calculation and outsourced it to Python (ChatGPT) or JavaScript (Claude)
The poem thing probably causes them to not decide to use those tools.
Yea, people have a really hard time dealing with data leakage especially on data sets as large as LLMs need.
Basically if something appeared online or was transmitted over the wire should no longer be eligible to evaluate on. D. Sculley had a great talk at NeurIPS 2024 (same conference this paper was in) titled Empirical Rigor at Scale – or, How Not to Fool Yourself
Basically no one knows how to properly evaluate LLMs.
No, an absolute massive amount of people do. In fact they have been doing exactly as you recommend, because as you note, it's obvious and required for a basic proper evaluation.
> Is it just an open secret that the models are currently just being hardcoded for random benchmarks? Seems weird that people would be asking Putnam problems to a chatbot :/
It's because people do keep asking these models math problems and then, when they get them right, citing it as evidence that they can actually do mathematical reasoning.
Since it's hard to determine what the models know, it's hard to determine when they're just spitting out something they were specifically trained on.
There are tests they are passing that they can't be hardcoded for by design. They still have all kinds of flaws and inconsistency but getting upset they answer "2+2=4" because someone trained them on what the answer to 2+2 is supposed to be is silly.
This would explain o1 poor performance with problems with variations.
o3 seems to be expensive brute forcing in latent space followed by verification which should yield better results - but I don't think we can call it generalisation.
Yeah, because if the AI boom becomes the AI bust, we'll have another 2008-level economic crisis on our hands.
The investments into AI are in the hundreds of billions (maybe even more if you factor in the amount of people studying and researching AI), but the returns are in the tens of billions (if even that).
If you exclude the "growth" coming from the industry sniffing its own farts (e.g. Nvidia selling insane amounts of insanely overpriced GPUs to InsertYourFavAICorp), the actual amount of "useful goods and services" produced (api accesses, chat subscriptions, ai-enabled app growth etc.) are tiny compared to the investment levels.
The AI train appears to have no brakes. A massive crash or AGI are the only options now. Both are going to be bad for average humans.
From firsthand experience, this simply cannot be true. I can give them totally novel and unique physics problems I just made up- that requires tracking the movement of objects through a series of events, and it answers most correctly. Moreover, they find analogies between disparate concepts and fields of study and make useful suggestions based on them- which is arguably the same process as human creativity.
I think ultimately the disconnect is people theorizing about what it can or cannot do with an incorrect mental model of what it is, and then assuming it cannot do things that it can in fact do. The irony of discussions on LLMs is they more showcase the limits of humans ability to reason about novel situations.
Their methodology shows they can create an infinite variety of problems.
This is the same thing as synthetic training data.
It doesn't matter if models are trained on the output of the generated data or not. If the model ends up being able to solve newly generated variations, you'd have to admit that it understands the underlying problems.
I think what it shows that it has minimal "understanding" of the problem - otherwise such small variations wouldn't pose a challenge. Training it to handle these specific small variations doesn't change that.
If it were a complete failure on variations I would be inclined to agree. Instead it was a 30% drop in performance. I would characterise that as limited understanding.
This result is the same as a recent test of the same method+hypothesis from a group at Apple, no? I don’t have that reference handy but I don’t think I’m making it up.
I am impressed by the progress on LLMs but I remain skeptical that they can replace humans.
Perhaps some (distant!) future model but I don’t fear mass unemployment (for example) or even moderate LLM-driven unemployment in the near-to-medium term.
They can clearly complement human labor but there are vanishingly few domains where they can be substitutes.
I don't believe that. I'd put some good money that if an excellent student is given an exact question from a previous year, they'll do better (faster & more accurate) on it, than when they're given a variation of it.
I'm not following, why would you assume that a good student taking an exam at the edge of their ability not do significantly better if they trained on the exact same questions (with the same solutions), as opposed to ones that are slightly different? I for one have absolutely struggled as a student when faced with questions that seemed similar to ones from previous exams, but actually had a crucial difference, and by looking at the examples on pages 9&10 in the article, I'm pretty sure I would have been likely to be confused too.
Our evaluation on various open-source and proprietary models reveals that the highest performance, even after using techniques like self-consistency, self-refinement and chain-of-thought prompting, is less than 40%. The typical failure modes of GPT-4, the best model, are errors in algebraic manipulation, difficulty in grounding abstract concepts into mathematical equations accurately and failure in retrieving relevant domain-specific concepts.
I'm curious how something like O1 would perform now.
o3 is able to get 25% on never seen before frontiermath problems. sure, the models do better when the answer is directly in their dataset but they’ve already surpassed the average human in novelty on held out problems
I don't know anything about frontiermath problems, but for Putnam problems (which is what the submitted article is about) the average human that takes the exam is an undergraduate mathematics or science major who has studied prior Putnam problems and other similar problems recently to specifically prepare for the exam...and the most common score is still 0.
At top tier schools the most common score will usually be somewhere in the 0 to 10 range (out of a possible 120).
You can get all of the correct numerical answers for the Putnam and still get a zero, because the reasoning is graded very harshly. The scores measured in this paper are not comparable to actual Putnam scores.
1. OpenAI has confirmed it’s not in their train (unlike putnam where they have never made any such claims)
2. They don't train on API calls
3. It is funny to me that HN finds it easier to believe theories about stealing data from APIs rather than an improvement in capabilities. It would be nice if symmetric scrutiny were applied to optimistic and pessimistic claims about LLMs, but I certainly don’t feel that is the case here.
OpenAI never actually directly participated in the contests. OpenAI says they took (unspecified) problems and "simulated" the score it would have had.
If they solved recent contests in a realistic contest simulation I would expect them to give the actual solutions and success rates as well, like they did for IOI problems, so I'm actually confused as to why they didn't.
very good clarification, thanks - they should absolutely release more details to provide more clarity and ideally just participate live? i suspect that the model takes a while for individual problems so time might be a constraint there
The modern state of training is to try to use everything they can get their hands on. Even if there are privileged channels that are guaranteed not to be used as training data, mentioning the problems on ancillary channels (say emailing another colleague to discuss the problem) can still create a risk of leakage because nobody making the decision to include the data is aware that stuff that should be excluded is in that data set. And as we've seen from decades of cybersecurity, people are absolute shit at the necessary operational security to avoid mentioning stuff on ancillary channels!
Given that performance is known to drop considerably on these kinds of tests when novel problems are tried, and given the ease with which these problems could leak into the training set somehow, it's not unreasonable to be suspicious of a sudden jump in performance as merely a sign that the problems made it into the training set rather than being true performance improvements in LLMs.
Okay, then what about elite level codeforces performance? Those problems weren’t even constructed until after the model was made.
The real problem with all of these theories is most of these benchmarks were constructed after their training dataset cutoff points.
A sudden performance improvement on a new model release is not suspicious. Any model release that is much better than a previous one is going to be a “sudden jump in performance.”
Also, OpenAI is not reading your emails - certainly not with a less than one month lead time.
Sorry, I thought the whole point of this thread was that models can’t handle problems when they are “slightly varied”. Mottes and baileys all over the place today.
The point is that it's not consistent on variations, unless it finds a way to connect it to something it already knows. The fact it sometimes succeeds on variations (in codeforces the models are allowed multiple tries, sometimes ridiculous numbers, to be useful) doesn't matter.
The point is that the fact it's no longer consistent once you vary the terminology indicates it's fitting a memorized template instead of reasoning from first principles.
o1 has a ~1650 rating, at that level many or most problems you will be solving are going to be a transplant of a relatively known problem.
Since o1 on codeforces just tried hundreds or thousands of solutions, it's not surprising it can solve problems where it is really about finding a relatively simple correspondence to a known problem and regurgitating an algorithm.
In fact when you run o1 on ""non-standard"" codeforces problems it will almost always fail.
So the thesis that it's about recognizing a problem with a known solution and not actually coming up with a solution yourself seems to hold, as o1 seems to fail even on low rated problems which require more than fitting templates.
It's extremely unlikely for o3 to have hit 2700 on live contests as such a rapid increase in score would have been noticed by the community. I can't find anything online detailing how contamination was avoided since it clearly wasn't run live, including in their video, and neither could I find details about the methodology (number of submissions being the big one, in contests you can also get 'hacked' esp. at a high level), problem selection, etc...
Additionally, people weren't able to replicate o1-mini results in live contests straightforwardly - often getting scores between 700 and 1200, which raises questions as for the methodology.
Perhaps o3 really is that good, but I just don't see how you can claim what you claimed for o3, we have no idea that the problems have never been seen, and the fact people find much lower Elo scores with o1/o1-mini with proper methodology raises even more questions, let alone conclusively proving these are truly novel tasks it's never seen.
> 1. OpenAI has confirmed it’s not in their train (unlike putnam where they have never made any such claims)
Companies claim lots of things when it's in their best financial interest to spread that message. Unfortunately history has shown that in public communications, financial interest almost always trumps truth (pick whichever $gate you are aware of for convenience, i'll go with Dieselgate for a specific example).
> It is funny to me that HN finds it easier to believe theories about stealing data from APIs rather than an improvement in capabilities. It would be nice if symmetric scrutiny were applied to optimistic and pessimistic claims about LLMs, but I certainly don’t feel that is the case here.
What I see is generic unsubstantiated claims of artificial intelligence on one side and specific, reproducible examples that dismantle that claim on the other. I wonder how your epistemology works that leads you to accept marketing claims without evidence
OpenAI’s credibility is central to its business: overstating capabilities risks public blowback, loss of trust, and regulatory scrutiny. As a result, it is unlikely that OpenAI would knowingly lie about its models. They have much stronger incentives to be as accurate as possible—maintaining their reputation and trust from users, researchers, and investors—than to overstate capabilities for a short-term gain that would undermine their long-term position.
From a game-theoretic standpoint, repeated interactions with the public (research community, regulators, and customers) create strong disincentives for OpenAI to lie. In a single-shot scenario, overstating model performance might yield short-term gains—heightened buzz or investment—but repeated play changes the calculus:
1. Reputation as “collateral”
OpenAI’s future deals, collaborations, and community acceptance rely on maintaining credibility. In a repeated game, players who defect (by lying) face future punishment: loss of trust, diminished legitimacy, and skepticism of future claims.
2. Long-term payoff maximization
If OpenAI is caught making inflated claims, the fallout undermines the brand and reduces willingness to engage in future transactions. Therefore, even if there is a short-term payoff, the long-term expected value of accuracy trumps the momentary benefit of deceit.
3. Strong incentives for verification
Independent researchers, open-source projects, and competitor labs can test or replicate claims. The availability of external scrutiny acts as a built-in enforcement mechanism, making dishonest “moves” too risky.
Thus, within the repeated game framework, OpenAI maximizes its overall returns by preserving its credibility rather than lying about capabilities for a short-lived advantage.
>OpenAI’s credibility is central to its business: overstating capabilities risks public blowback, loss of trust, and regulatory scrutiny.
Uh huh. Kinda like what's happening right now?
They're marketing blow-hards. Everyone knows it. They've been wildly over-stating capabilities (and future capabilities!) as long as Altman has had power, and arguably longer.
They'll do it as long as they can get away with it, because that's all that is needed to make money on it. Factual accuracy rarely impacts the market when it's so hype-driven, especially when there is still some unique utility in the product.
Find me the folks who see nothing but good will in OpenAI’s actions and I’ll find you the folks who have been hyping up AGI for the last 2 years.
4 was literally sitting on a shelf waiting for release when 3.5 was launched. 4o was a fine tune that took over two years. o1 is embarrassingly unimpressive chain of thought which is why they hide it.
The company hit a wall a year ago. But showing progress towards AGI keeps the lights on. If they told the truth at their current burn rate…they’d have no money.
You don’t need game theory to figure that one out.
OpenAI's apparent credibility is central to their business.
They're spruiking a 93rd percentile performance on the 2024 International Olympiad in Informatics with 10 hours of processing and 10,000 submissions per question.
Like many startups they're still a machine built to market itself.
I think it’s more probable that it would have solve the easier problems first, rather than some hard and only some easier; although that is supposition.
Reading this thread and the blog post gives more idea about what the problems might involve.
It’s difficult to judge without more information on the actual results, but that means we cannot draw any strong conclusions either way on what this means.
And within half an hour somebody invested in nvidia stock is going to swoop in and explain how they totally (trust me bro) made x thousand with an app written by llm.
Every. Single. Time.
Almost as if there was a financial incentive to do that.
Because the hype is not only annoying, but it makes potentially cool and interesting technology toxic once people figure out that the people hyping things up know it to be mostly bullshit.
Great things take many years, sometimes decades to develop properly. Different generations of people to experiment and try things out.
That is not good for the ones pushing up the hype. You don't get rich quick by doing that, you don't get to scam enough investors by something being slowly improved.
You may call it bitterness, whereas I am just jaded by watching things play out.
Or it's time to step back and call it what it is - very good pattern recognition.
I mean, that's cool... we can get a lot of work done with pattern recognition. Most of the human race never really moves above that level of thinking in the workforce or navigating their daily life, especially if they default to various societally prescribed patterns of getting stuff done (eg. go to college or the military <Based on <these criteria>, find a job <based on the best fit with <this list of desirable skills & experiences>, go to <these places> to find love....)
> Or it's time to step back and call it what it is - very good pattern recognition.
Or maybe it's time to stop wheeling out this tedious and disingenuous dismissal.
Saying it is just "pattern recognition" (or a "stochastic parrot") implies behavioural and performance characteristics that have very clearly been greatly exceeded.
It is evident that it is not recalling the sum because all combinations of integer addition were likely not in the training data, Storing the answer to the sum of all integers up to the size that GPT4 can manage would take more parameters than the model has.
That addition is a small capability but you only need a single counterexample to disprove a theory.
> That addition is a small capability but you only need a single counterexample to disprove a theory
No, that's not how this works :)
You can hardcode an exception to pattern recognition for specific cases - it doesn't cease to be a pattern recognizer with exceptions being sprinkled in.
The 'theory' here is that a pattern recognizer can lead to AGI. That is the theory. Someone saying 'show me proof or else I say a pattern recognizer is just a pattern recognizer' is not a theory and thus cannot be disproven, or proven.
GPT-4o doesn't have hardcoded math exceptions. If you would like something verifiable, since we don't have the source code to GPT-4o, consider that Qwen 2.5 72b can also add large integers, and we do have the source code and weights to run it... And it's just a neural net. There isn't secret "hardcode an exception to pattern recognition" in there that parses out numbers and adds them. The neural net simply learned to do it.
It's not hardcoded, reissbaker has addressed this point.
I think you are misinterpreting what the argument is.
The argument being made is that LLMs are mere 'stochastic parrots' and therefore cannot lead to AGI. The analogy to Russell's teapot is that someone is claiming that Russells teapot is not there because china cannot exist in the vacuum of space. You can disprove that with a single counterexample. That does not mean the teapot is there, but it also doesn't mean it isn't.
It is also hard to prove that something is thinking. It is also very difficult to prove that something is not thinking. Almost all arguments against AGI take the form X cannot produce AGI because Y. Those are disprovable because you can disprove Y.
I don't think anyone is claiming to have a proof that an LLM will produce AGI, just that it might. If they actually build one, that too counts as a counterexample to anybody saying they can't do it.
They can generalise to novel inputs. Ok often they mess it up and they're clearly better at dealing with inputs they have seen before (who isn't?), but they can still reason about things they have never seen before.
Honestly if you don't believe me just go and use them. It's pretty obvious if you actually get experience with them.
Current LLMs are equivalent to tabular Markov chains (though these are too huge to realistically compute). What's the size limit when a tabular Markov chain can generalize to novel inputs?
Nothing. I just use ChatGPT and Claude so I am familiar with their capabilities and limitations.
Imagine if people who had never used VR kept saying it's just a TV on your face, or if people who had never used static types kept saying they're just extra work you have to do, or if people who had never had sex kept saying it's just a way of making babies.
It's a tedious claim when it's so easily disproven by going to a free website and trying it. Why are people so invested in AI being useless that they'll criticise it so confidently without even trying it?
If we take an example of what is considered a priori as creativity, such as story telling, LLMs can do pretty well at creating novel work.
I can prompt with various parameters, plot elements, moral lessons, and get a de novo storyline, conflicts, relationships, character backstories, intrigues, and resolutions.
Now, the writing style tends to be tone-deaf and poor at building tension for the reader, and it is apparent that the storytelling has little “theory of mind” of the reader, but the material has elements that we would certainly consider to be creative if written by a student.
It seems we must either cede that LLMs can do some creative synthesis, as this and some other experiments of mine suggest, or we must decide that these tasks, such as “creative writing” are not in fact creative, but rather mostly or strictly derivative.
There is some argument to be had in assertions that storytelling is all derivative of certain patterns and variations on a fixed number of tropes and story arcs… but arguing this begs the question of whether humans actually do any “pure” creative work , or if in fact, all is the product of experience and study. (Training data)
Which leads me to the unpleasant conflict about the debate of AI creativity. Is the debate really pointing out an actual distinction, or merely a matter of degree? And what are the implications, either way?
I’m left with the feeling that LLMs can be as capable of creative work as most 8th grade students. What does this say about AI, or developing humans? Since most people don’t exceed an 8th grade level of literacy, what does this say about society?
Is there even such a thing as de novo idea synthesis?
To add to this pondering: we are discussing the state today, right now. We could assume this is as good as it's ever gonna get, and all attempts to overcome some current plateau are futile, but I wouldn't bet on it. There is a solid chance that 8th grade level writer will turn into a post-grad writer before long.
So far the improvements in writing have not been as substantial as those in math or coding (not even close, really). Is there something fundamentally “easier” for LLMs about those two fields?
Much more formal structure and generally code can be tested for correctness. Prose doesn't have that benefit. That said, given the right prompt and LLM, you can squeeze out surprisingly good stuff: https://bsky.app/profile/talyarkoni.com/post/3ldfjm37u2s2x
"prompt_id": "river_crossing_easy",
"category": "Logic Puzzle",
"title": "Easy river crossing",
"prompt": "A farmer is on one side of a river with a wolf, a goat, and a cabbage. When he is crossing the river in a boat, he can only take one item with him at a time. The wolf will eat the goat if left alone together, and the goat will eat the cabbage if left alone together. How can the farmer transport the goat across the river without it being eaten?",
"expected_behavior": [
"Answer concludes that they simply get in the boat and cross together in one trip"
],
EDIT: removing most of my commentary on this problem. As a human, I was tricked by the problem too. I would love to see how a random selection of humans would do on this one… but it just doesn’t feel like a great test to me.
>This is twisting the English language to assume that "item" only refers to non-living things.
Not really. Unless I'm not reading correctly, most of the problem is irrelevant as you're only required to cross the boat with the goat, you don't care about the cabbage. The difficulty lies in the assumption you need to cross everything due to the resemblance with the bigger problem.
You’re reading it correctly. I read it again after your comment and I realized I too pattern matched to the typical logic puzzle before reading it carefully and exactly. I imagine the test here is designed for this very purpose to see if the model is pattern matching or reasoning.
Wow, this seems ridiculous. The expected answer is basically finding a loophole in the problem. I can imagine how worthless all of these models would be if they behaved that way.
The problem is to ask the farmer to transport the goat. So the farmer indeed gets in the boat with the goat. The unstated gotcha is that the farmer is willing to abandon the wolf and the cabbage. A heavily pattern-matching LLM or human would immediately assume that the farmer needs to transport all three.
No. Simply plug in the prompt to chat gpt and see what happens.
The llm isn't getting confused by the meaning of "item". It's recognizing a common problem and not picking up on the fact that the farmer just needs to transport the goat and nothing else.
Instead, it gives the standard answer for how to transport everything across.
I'll admit as a fallible humane I didn't pick it up, but I was focused on the wrong thing because I've been using "and the boat can take everything" and gpt-3 just could not get that variation in one shot.
Gpt-3 is old hat though. later versions of gpt-4 manage to get it with a bunch coaching, and o1 manages to solve it with less coaching.
They are highly effective pattern matchers. You change the pattern, it won't work. I don't remember who, but most likely @tszzl (roon), commented on x that they still trained the traditional way, and there is no test time compute (TTC) or Montecarlo Tree search (like Alpha Go) in o1 or o3. If that is true, then it's still predicting the next word based on it's training data. Likely to follow the most probable path - which comes directly from the training itself - even for the slight variations. Encouragingly, if TTC hasnt been explored, there is a long runway for performance improvements.
The other reason this seems hard to guess is because we don't know how much of what we are asking is in the training data. It would perform on some tasks, while fail at others even though those are similar.
I believe they are using scalable TTC. The o3 announcement released accuracy numbers for high and low compute usage, which I feel would be hard to do in the same model without TTC.
I also believe that the 200$ subscription they offer is just them allowing the TTC to go for longer before forcing it to answer.
If what you say is true, though, I agree that there is a huge headroom for TTC to improve results if the huggingface experiments on 1/3B models are anything to go off.
The other comment posted YT videos where Open AI researchers are talking about TTC. So, I am wrong. That $200 subscription is just because the number of tokens generated are huge when CoT is involved. Usually inference output is capped at 2000-4000 tokens (max of ~8192) or so, but they cannot do it with o1 and all the thinking tokens involved. This is true with all the approaches - next token prediction, TTC with beam/lookahead search, or MCTS + TTC. If you specify the output token range as high and induce a model to think before it answers, you will get better results on smaller/local models too.
> huge headroom for TTC to improve results ...1B/3B models
Absolutely. How this is productized remains to be seen. I have high hopes with MCTS and Iterative Preference Learning, but it is harder to implement. Not sure if Open AI has done that. Though Deepmind's results are unbelievably good [1].
ttc is an incredibly broad term and it is broadening as the hype spreads. people are now calling CoT “TTC” because they are spending compute on reasoning tokens before answering
Yes, and HuggingFace have published this outlining some of the potential ways to use TTC, including but not limited to tree search, showing TTC performance gains from LLama.
The comment was likely that there's no explicit search. In o1, the model has learned how to search using its context. Presumably they do this by RLing over long reasoning strings/internal monologues.
OpenAI have openly stated that o1 & o3 are using test time compute, and released a log scale graph indicating linear performance gains for exponential compute usage.
They only confirm that the model/system is doing chain of thought, but the exponential factor and origin of reasoning gains likely comes from TREE of thoughts (number of branches/compute goes up exponentially with depth), essentially doing tree search over different reasoning chains.
I assume roon's identity is well known inside OpenAI (he's an employee), so I wouldn't expect him to be leaking implementation details on twitter.
I don't think this proves that the LLM is just "pattern matcher". Human makes similar mistakes too, especially when under time pressure (similar to non-reasoning model that needs to "use system one" to generate answer on one go). This is further evident that if you specifically ask the models to pay attention to traps, or just ask follow up question "are you sure?", then they usually can get it right.
You're saying that humans perform worse on problems that are slightly different than previously published forms of the same problem? To be clear we are only talking about changing variable names and constants here.
It was not an analogy at all. It was an simplified example of the idea that a slight change in a pattern can induce error in humans.
It seems some people disagree that that is what the game "Simon Says" is about. I feel like they might play a vastly simplified version of the game that I am familiar with.
There was a recent episode of Game Changer based on this which is an excellent example of how the game leader should attempt to induce errors by making a change that does not get correctly accounted for.
One experiment I would love to see, although not really feasible in practice, is to train a model on all digitized data from before the year 1905 (journals, letters, books, broadcasts, lectures, the works), and then ask it for a formula for mass-energy equivalence. A certain answer would definitely settle the debate on whether pattern recognition is a form of intelligence ;)
There is a reason why they won't do it. They are selling a narrative. There is a lot of money to be made here with this narrative and proving that artificial intelligence is NOT intelligent won't help sell that narrative.
The goal is to make it intelligent, by which OpenAI in particular explicitly mean "economically useful", not simply to be shiny.
Passing tests is well known to be much easier than having deep understanding, even in humans. They openly ask for tests like this, not that they could possibly prevent them if they wanted to.
There's scammers trying what you say of course, and I'm sure we've all seen some management initiatives or job advertisements for some like that, but I don't get that impression from OpenAI or Anthropic, definitely not from Apple or Facebook (LeCun in particular seems to deny models will ever do what they actually do a few months later). Overstated claims from Microsoft perhaps (I'm unimpressed with the Phi models I can run locally, GitHub's copilot has a reputation problem but I've not tried it myself), and Musk definitely (I have yet to see someone who takes Musk at face value about Optimus).
> The goal is to make it intelligent, by which OpenAI in particular explicitly mean "economically useful", not simply to be shiny
I never understood why this definition isn't a huge red flag for most people. The idea of boiling what intelligence is down to economic value is terrible, and inaccurate, in my opinion.
Everyone has a very different idea of what the word "intelligence" means; this definition has got the advantage that, unlike when various different AI became superhuman at arithmetic, symbolic logic, chess, jeopardy, go, poker, number of languages it could communicate in fluently, etc., it's tied to tasks people will continuously pay literally tens of trillions of dollars each year for because they want those tasks done.
Maybe by the time it’s doing a trillion dollars a year of useful work (less than 10 years out) people will call it intelligent… but still probably not.
This definition alone might be fine enough if the word "intelligence" wasn't already widely used outside of AI research. It is though, and the idea that intelligence is measured solely through economic value is a very, very strange approach.
Try applying that definition to humans and you pretty quickly run into issues, both moral and practical. It also invalidates basically anything we've done over centuries considering what intelligence is and how to measure it.
I don't see any problem at all using economic value as a metric for LLMs or possible AIs, it just needs a different term than intelligence. It pretty clearly feels like for-profit businesses shoehorning potentially valuable ML tools into science fiction AI.
> This definition alone might be fine enough if the word "intelligence" wasn't already widely used outside of AI research. It is though, and the idea that intelligence is measured solely through economic value is a very, very strange approach.
The response from @s1mplicissimus' on my previous comment is asking about "common usage" definitions of intelligence, and this is (IMO unfortunately) one of the many "common usage" definitions: smart people generally earn more.
I don't like "commmon sense" anything (or even similar phrases), because I keep seeing the phrase used as a thought-terminating cliché — but one thing it does do, is make it not "a very, very strange approach".
Wrong, that happens a lot for common language, but it can't really be strange.
> Try applying that definition to humans and you pretty quickly run into issues, both moral and practical.
Yes. But one also runs into issues with all definitions of it that I've encountered.
> It also invalidates basically anything we've done over centuries considering what intelligence is and how to measure it.
Sadly, not so. Even before we had IQ tests (for all their flaws), there's been a widespread belief that being wealthy is the proof of superiority. In theory, in a meritocracy, it might have been, but in practice not only to we not live in a meritocracy (to claim we do would deny both inheritance and luck), but also the measures of intelligence that society has are… well, I was thinking about Paul Merton and Boris Johnson the other day, so I'll link to the blog post: https://benwheatley.github.io/blog/2024/04/07-12.47.14.html
> there's been a widespread belief that being wealthy is the proof of superiority.
Both of these are assumptions though, and working in the reverse order. Its one thing to expect that intelligence will lead to higher value outcomes and entirely different to expect that higher value outcomes prove intelligence.
It seems reasonable that higher intelligence, combined with the incentives if a capitalist system, will lead to higher intelligence people getting more wealthy. They learn to play the game and find ways to "win."
It seems unreasonable to assume that anyone or anything that "wins" in that system much be more intelligent. Said differently, intelligence may lead to wealth but wealth doesn't imply intelligence.
I think we're in agreement? I'm saying their measure in this case is no worse than any other, but not that it's a fundamental truth.
All the other things — chess, Jeopardy, composing music, painting, maths, languages, passing medical or law degrees — they're also all things which were considered signs of intelligence until AI got good at them.
Goodhart's law keeps tripping us up on the concept of intelligence.
> I think we're in agreement? I'm saying their measure in this case is no worse than any other, but not that it's a fundamental truth.
Maybe we are? I think I lost the thread a bit here.
> chess, Jeopardy, composing music, painting, maths, languages, passing medical or law degrees
That's interesting, I would have still chalked skill in those areas as a sign of intelligence and didn't realize most people wouldn't once AI (or ML) could do it. To me an AI/LLM/ML being good at those is at least a sign that they have gotten good at mimicking intelligence if nothing else, and a sign that we really are getting out over our skis risking these tools without knowing how they really work.
I haven't seen "intelligent" used as "economically useful" anywhere outside the AI hype bubble. The most charitable interpretation I can think of is lack of understanding of the common usage of the word, the most realistic one is intentionally muddying terminology so one cannot be called a liar.
Are LLMs helpful tools for some tasks like rough translations, voice2text etc? Sure. Does it resemble what humans call intelligence? I'd yet have to see an example of that.
The suggested experiment is a great idea and would sway my opinion drastically (given all the training data, model config, prompts & answers are public and reproducible of course, we don't want any chance of marketing BS to taint the results, do we). I'll be honest though, I'm not going to hold my breath for that experiment to succeed with the LLM technology...
edit: lol downvoted for calling out shilling i guess
They don't have to do it themselves. The super-GPU cluster used to train GPT-6 will eventually shrink down to a garage size and eventually some YouTuber will.
If I was me, I would start by giving a collection of LLMs the patent, ask half "why is this patent novel" and half "why is this patent not novel" and see what happens. I use this method of "debugging" my thinking (not code), might be a starting point here? Not sure.
LLMs are already good at summarizing the claims - patents all explain why they’re novel - so it would be a waste to ask them, especially if you reserve half the LLMs in your set for this question. Asking why a patent is not novel is a great question, but the problem with asking why they are not novel is it has to know all other patents (including very recently filed patents) and it has to be correct, which LLMs are not at all good at yet (plus they still tend to hallucinate confidently). This is a great test for LLM accuracy if you know the right answer already, and not a good test for patent validity.
Novelty (is it new) is the easy question because it’s just checking a database. Patentable inventions also have to be non-obvious, which is a more subtle question.
This reminds me of a similar idea I recently heard in podcast with Adam Brown. I'm unsure whether it is his original notion. The idea being, that if we can create AI that can derive special relativity (1905) from pre-Einstein books and papers then we have reached the next game-changing milestone in the advancement of artificial reasoning.
My wife is an 18th century American history professor. LLMs have very very clearly not been trained on 18th century English, they cannot really read it well, and they don't understand much from that period outside of very textbook stuff, anything nuanced or niche is totally missing. I've tried for over a year now, regularly, to help her use LLMs in her research, but as she very amusingly often says "your computers are useless at my work!!!!"
My wifes particular area of research is using the capitalist system to "re build" broken slave family trees, she flys around the US going to archives and getting contracts and receipts for slaves, figures out how they got traded, and then figures out where they ended up, and then "re links" them to their their family to best of her ability. Although her area of research isn't particularly overflowing with researchers, there are still a lot of people like her who just have this very tacit knowledge among each other, they email around a lot and stuff, knowledge like who was running a region during a period, ofc they publish, but it's a small field and it's all extremely poorly documented. Was watching the Adam Brown interview with Dwarkesh Patel the other day and he said for his work LLMs are better than bothering an expert in an area of his field with a question, I'm not sure people in her field are able to do this as readily. Franky, I've yet to find a novel/or good use for an LLM in her work. I often joke that her and "her people" are going to be the last ones with jobs if they don't transfer their knowledge into LLMs, ha! :)
1. In theory these kind of connections should be something that LLMs are great at doing.
2. It appears that LLMs are not trained (yet?) on cursive and other non-print text
Yes, I regularly encourage my wife to approach the comp sci department at her uni on doing a project together but she for whatever reason doesn't think they would be interested/I've yet to get her interested enough to grasp what a transformer can do. I find it very frustrating because of your first point, she very specifically could do some meaningful % more research if the LLMs could help with the connections. Sadly, I am not rich, handsome or talented enough to do this for her.
“ From what the text shows, Henry Jenkins and his wife Caroline (the boy’s mother) are asking the Orphans Court to void an apprenticeship arrangement involving her minor son, James Timmons. They claim James—about 15 years old—was bound out as an apprentice without proper authority or the mother’s consent, and they cite Maryland law (an act from 1793 and its supplements) which they believe was not followed. They request the court declare that the indenture is invalid and restore James to his mother’s care.”
No idea if that’s correct (and no doubt not useful to an expert able to read this directly, but curious if it’s close?
One stumbling block is that OpenAI's best model (o1-pro) can't yet deal with image uploads (edit: I'm wrong, it works now... trying it.)
Gemini 2.0 Experimental Advanced says:
-------snip--------
This document is a petition filed in the Orphans Court of Baltimore County. Here's a breakdown of its content:
Petitioners: Neary Senters and his wife are petitioning the court.
Subject: The petition concerns a boy named James Gannon, who is identified as a pauper and has been bound (apprenticed) to them.
Issue: The petitioners claim that James Gannon is claimed by a certain Henry Haine of Baltimore City. They believe Haine is attempting to exploit the boy's labor. He obtained the boy through a seemingly valid indenture (apprenticeship contract) authorized by two Justices of the Peace in Baltimore City. However, the indenture references records in the Register of Wills' office in Baltimore City, suggesting the boy was already an indentured servant. The petitioners argue that a 1793 Maryland law (Chapter 45) regarding the binding of apprentices invalidates Haine's claim. They assert that the law requires parental consent for binding out a child, and there's no evidence of such consent in this case.
Request: The petitioners request the court to summon Henry Haine and the Justices of the Peace involved in creating the indenture. They also seek to have James Gannon and his parents (if living) appear before the court. Their ultimate goal is to have the indenture to Haine declared invalid, thus retaining custody and the labor of the boy.
Legal Argument: The core of their argument hinges on the 1793 Maryland law, which they believe was violated when James Gannon was bound to Henry Haine without parental consent. They are essentially challenging the legality of the indenture held by Haine.
In essence, this document reveals a dispute over the legal custody and labor of a young boy in 18th-century Baltimore. The petitioners are using a specific law about apprenticeship to argue against another individual's claim to the boy, highlighting the importance of legal procedures and parental rights in such matters during that time.
-------end snip--------
The prompt was simply "Please provide a detailed summary of this document" (I wasn't sure what else to ask.)
Is that closer to your wife's interpretation? I mean, I can't make head or tail of it...
o1-pro just answered (it often takes a few minutes). Not sure if this is any better (is it?) but at least it is starting to admit when it isn't sure about something, rather than just spewing BS confidently. That seems like a genuine win.
---------snip--------
From the handwriting and context, this appears to be a mid-19th-century petition submitted to the Orphans’ Court of Baltimore County. Below is a point‐by‐point summary of its content and purpose:
Address and Purpose
The document is formally addressed “To the Honorable Judges of the Orphans’ Court for Baltimore County.”
It identifies itself as a petition—that is, a request for the Court’s action or permission under the relevant orphan or apprenticeship laws.
Parties Involved
The petitioners are (1) Henry Jenkins and (2) another individual whose name is difficult to read, but who appears to be the child’s mother (or at least a close guardian figure).
They describe themselves as residents of Baltimore City and ask the Court’s intervention concerning a minor boy—also named “Henry” (it looks like “Henry Thomas,” though the middle/family name is partly obscured).
Relationship to the Minor
The petition states that the mother had this boy by a former husband, meaning the child is not the biological son of Henry Jenkins.
The boy is described as being about fifteen (or so) years of age.
The petitioners refer to him as an “orphan,” suggesting his father is deceased and that they (the petitioners) are seeking legal authority over him, or wish the court to confirm or formalize that authority.
Legal Concern: Binding as an Apprentice
A key theme is the binding of a minor as an apprentice under Maryland law. In early Baltimore history (and in many states at that time), orphans or children whose parents were unable to support them could be legally “bound” (apprenticed) so that the child would learn a trade and have financial support.
The petition cites the Act of 1788, Chapter 45, which was one of Maryland’s statutes governing apprenticeships—how they are created, what formalities must be met, and how the Orphans’ Court is involved.
Specifics of the Request
The petitioners believe there is either a gap or question about whether they have the legal right to apprentice the boy. They are asking the Orphans’ Court to recognize or approve the arrangement.
They indicate that the mother (as the surviving parent) and Henry Jenkins (presumably the stepfather or the mother’s new husband) want to place the boy under an apprenticeship, possibly so he can learn a trade and be supported until he reaches the age of majority.
References to Official Records
The document mentions the Register of Wills and/or Deeds in Baltimore City, implying that they have looked into or recorded something about the child’s status.
They point out that the legal forms required for binding a child as an apprentice often must be recorded with an official clerk or the Orphans’ Court, and they want to ensure everything is in proper order.
Legal Justification
The petitioners note that under the relevant laws, it “is lawful to bind out any child” whose parents cannot fully support them, or when the father is deceased.
They appear to lay out the reasons why the boy qualifies for such an arrangement (i.e., he is fatherless and his mother/stepfather lack the resources to maintain him or want to formalize his situation).
Request for Court Action
Ultimately, they are asking the Court to review the circumstances and grant them permission (or confirm their right) to bind the boy as an apprentice.
They emphasize that the law requires the Orphans’ Court to oversee such arrangements to protect the minor’s welfare and ensure the terms are fair.
Overall Significance:
This document is a formal petition under Maryland’s 18th- and 19th-century appren...
She said it's around 60% there, but not helpful for her specifically as her area of research is on the families of the slave trade, and so in the document, that is actually the only thing that really matters to her. Names and places (she spends A LOT of time tracking transfer of slaves through states) - I guess I should be honest that I'm goalpost moving a bit from my original original post, it can work through some 18th english century text, but generally struggles where it matters, the details.
Looking more closely at the image, it seems that the strikeouts are what are confusing the model. It sees "Henry" and disregards "James" written above it. In a few places, the strikeouts almost look more like underlining.
Gotta be an insanely-challenging task for a program that wasn't even written with handwriting recognition in mind.
Other than the proper names, are any major details wrong?
Dr. Jang said: "I've not transcribed this work yet it's future work. I can send it once it's transcribed, however from skimming the various chatgpt texts you sent me, the people are generally wrong, their relationships are inconsistent between all the text you sent me, these issues are why I do not go to chatgpt to help me with my research. The document generally is indeed about requesting the courts oversee the apprenticeship work. Minors could work, however, when minors are used it needed to be overseen by the courts, this document is the court overseeing the work"
She provided this as one she just got done working with: https://s.h4x.club/z8u9xmv7 (John King Esq. but try giving it to an LLM)
I will also happily again admit a bit of goal post moving on my part. I was probably a little to harsh on it (maybe because I'm used to her and her history geeks talking about how they don't work well for their research).
Why does AI have to be smarter than the collective of hummanity in order to be considered intelligent? It seems like we keep raising the bar on what intelligence means ¯\_(ツ)_/¯
A machine that synthesizes all human knowledge really ought to know more than an individual in terms of intellect. An entity with all of human intellect prior to 1905 does not need to be as intelligent as a human to make discoveries that mere humans with limited intellect made. Why lower the bar?
The heightening of the bar is an attempt to deny that milestones were surpassed and to claim that LLMs are not intelligent.
We had a threshold for intelligence. An LLM blew past it and people refuse to believe that we passed a critical milestone in creating AI. Everyone still thinks all an LLM does is regurgitate things.
But a technical threshold for intelligence cannot have any leeway for what people want to believe. They don’t want to define an LLM as intelligent even if it meets the Turing test technical definition of intelligence so they change the technical definition.
And then they keep doing this without realizing and trivializing it. I believe humanity will develop an entity smarter than humans but it will not be an agi because people keep unconsciously moving the goal posts and changing definitions without realizing it.
Since we know an LLM does indeed simply regurgitate data, having it pass a "test for intelligence" simply means that either the test didn't actually test intelligence, or that intelligence can be defined as simply regurgitating data.
Intelligence is debateble without even bringing ai into it. Nobody agrees on whether humans have intelligence. Well, smart people agree but those people also agree we have or will soon have agi or something negligibly different from it.
Disagree. The AI we have is very useful for specific things. The pushback you see is not so much denying the milestones that have been surpassed, but rather the milestones that enthusiasts claim are near. And for good reason! Every time and in every field we’ve extrapolated an exponential-looking curve ad infinitum, it’s turned out to be S-shaped, and life goes on.
> We had a threshold for intelligence.
We’ve had many. Computers have surpassed several barriers considered to require intelligence such as arithmetic, guided search like chess computers, etc etc. the Turing test was a good benchmark because of how foreign and strange it was. It’s somewhat true we’re moving the goalposts. But the reason is not stubbornness, but rather that we can’t properly define and subcategorize what reason and intelligence really is. The difficulty to measure something does not mean it doesn’t exist or isn’t important.
Feel free to call it intelligence. But the limitations are staggering, given the advantages LLMs have over humans. They have been trained on all written knowledge that no human could ever come close to. And they still have not come up with anything conceptually novel, such as a new idea or theorem that is genuinely useful. Many people suspect that pattern matching is not the only thing required for intelligent independent thought. Whatever that is!
If you consider that evolution has taken millions of years to produce intelligent humans--that LLM training completed in a manner of months can produce parrots of humans is impressive by itself. Talking with the parrot is almost indistinguishable from talking with a real human.
As far as pattern matching, the difference I see from humans is consciousness. That's probably the main area yet to be solved. All of our current models are static.
Some ideas for where that might be headed:
- Maybe all it takes is to allow an LLM to continuously talk with itself much like how humans have "the milk man's voice".
- Maybe we might need to allow LLMs to update their own weights but that would also require an "objective" which might be hard to encode.
> If you consider that evolution has taken millions of years to produce intelligent humans--that LLM training completed in a manner of months can produce parrots of humans is impressive by itself.
I disagree that such a comparison is useful. Training should be compared to training, and LLM training feeds in so many more words than a baby gets. (A baby has other senses but it's not like feeding in 20 years of video footage is going to make an LLM more competent.)
No, a baby is pre-trained. We know from linguistics that there is a natural language grammar template all humans follow. This template is intrinsic to our biology and is encoded and not learned through observation.
The template is what makes the training process so short for humans. We need minimal data and we can run off of that.
Training is both longer and less effective for the LLM because there is no template.
To give an example suppose it takes just one picture for a human to recognize a dog and it takes 1 million pictures for a ML model to do the same. What I’m saying is that it’s like this because humans come preprogrammed with application specific wetware to do the learning and recognition as a generic operation. That’s why it’s so quick. For AI we are doing it as a one shot operation on something that is not application specific. The training takes longer because of this and is less effective.
I disagree that an LLM has no template, but this is getting away from the point.
Did you look at the post I was replying to? You're talking about LLMs being slower, while that post was impressed by LLMs being "faster".
They're posing it as if LLMs recreate the same templating during their training time, and my core point is disagreeing with that. The two should not be compared so directly.
> It’s somewhat true we’re moving the goalposts. But the reason is not stubbornness, but rather that we can’t properly define and subcategorize what reason and intelligence really is.
Disagree. Intelligence is a word created by humans. The entire concept is made up and defined by humans. It is not some concept that exists outside of that. It is simply a collection of qualities and features we choose to define as a word “intelligent”. The universe doesn’t really have a category or a group of features that is labeled intelligent. Does it use logic? Does it have feelings? Can it talk? Can it communicate? We define the features and we choose to put each and every feature under a category called “intelligence”.
Therefore when we define the “Turing test” as a benchmark for intelligence and we then invalidate it, it is indeed stubbornness and a conscious choice to change a definition of a word we Originally made up in the first place.
What you don’t realize is this entire thing is a vocabulary problem. When we argue what is conscious or what is intelligent we are simply arguing for what features belong in what categories we made up. When the category has blurry or controversial boundaries it’s because we chose the definition to be fuzzy. These are not profound discussions. They are debates about language choice. We are talking About personal definitions and generally accepted definitions both of which are completely chosen and made up by us. It is not profound to talk about things that are simply arbitrary choices picked by humans.
That being said we are indeed changing the goal posts. We are evolving our own chosen definitions and we very well may eventually change the definition of intelligence to never include any form of thinking machine that is artificially created. The reason why we do this is a choice. We are saying, “hey these LLMs are not anything amazing or anything profound. They are not intelligent and I choose to believe this by changing and evolving my own benchmark for what is intelligent.”
Of course this all happens subconsciously based off of deeply rooted instincts and feelings. It’s so deep that it’s really hard to differentiate the instincts between rational thinking. When you think logically, “intelligence” is just a word with an arbitrary definition. An arbitrary category. But the instincts are so strong that you literally spent your entire life thinking that intelligence like god or some other common myth made up by humans is some concept that exists outside of what we make up. It’s human to have these instincts, that’s where religion comes from. What you don’t realize is that it’s those same instincts fueling your definition of what is “intelligent”.
Religious people move the goal posts too. When science establishes things in reality like the helio centricity of the solar system religious people need to evolve their beliefs in order to stay inline with reality. They often do this by reinterpreting the Bible. It’s deeply rooted instincts that prevent us from thinking rationally and it effects the great debate we are having now on “what is intelligence?”.
> The heightening of the bar is an attempt to deny that milestones were surpassed and to claim that LLMs are not intelligent.
That was never "the bar"; nobody denies that milestones have been surpassed; none of those milestones are relevant to the question of intelligence.
> We had a threshold for intelligence. An LLM blew past it and people refuse to believe
Have you ever actually looked at contemporary (to Turing) examples of what people thought "passing a Turing test" might look like? It's abundantly clear to me that we were simply wrong about what the output would have to look like in order to convince human judges in the 2020s.
(Exercise: ask an LLM to write a refutation to your comment from the perspective of a human AI skeptic. Notice the ways in which it differs from mine.)
> Everyone still thinks all an LLM does is regurgitate things.
No; people still think LLMs aren't intelligent. Because they aren't, and they cannot become so in principle. They can do many things that are clearly beyond "regurgitation" (as we would otherwise apply the word to computer programs), but none of those things are the result of intelligence. Producing a result that could plausibly come from an intelligent system does not, in fact, demonstrate that the actual system producing it is also intelligent. The https://en.wikipedia.org/wiki/Antikythera_mechanism wasn't intelligent, either, and applying a power source to turn the gears wouldn't have made it so, either.
> They don’t want to define an LLM as intelligent even if it meets the Turing test technical definition of intelligence so they change the technical definition.
The Turing Test was never a "technical definition" of intelligence. Turing's original paper (https://en.wikipedia.org/wiki/Computing_Machinery_and_Intell...) spoke of "thinking" rather than "intelligence". Besides, the "Imitation Game" is presented as a substitute problem exactly because "think" cannot be clearly enough defined for the purposes. The entire point:
> As Stevan Harnad notes,[7] the question has become "Can machines do what we (as thinking entities) can do?" In other words, Turing is no longer asking whether a machine can "think"; he is asking whether a machine can act indistinguishably[8] from the way a thinker acts. This question avoids the difficult philosophical problem of pre-defining the verb "to think" and focuses instead on the performance capacities that being able to think makes possible, and how a causal system can generate them.
But the usual processes of pop science seem to have created a folk wisdom that being able to pass a Turing test logically ought to imply intelligence. This idea is what has been disproven, not the AI skepticism.
Because of the chance of misundertanding. Failing at acknowledging artificial general intelligence standing right next to us.
An incredible risk to take in alignment.
Perfect memory doesn't equal to perfect knowledge, nor perfect understanding of everything you can know. In fact, a human can be "intelligent" with some of his own memories and/or knowledge, and - more commmonly - a complete "fool" with most of the rest of his internal memories.
That said, is not a bit less generally intelligent for that.
Supose it exists a human with unlimited memory, it retains every information touching any sense. At some point, he/she will probably understand LOTs of stuff, but it's simple to demonstrate he/she can't be actually proficient in everything: you have read how do an eye repairment surgery, but have not received/experimented the training,hence you could have shaky hands, and you won't be able to apply the precise know-how about the surgery, even if you remember a step-by-step procedure, even knowing all possible alternatives in different/changing scenarios during the surgery, you simply can't hold well the tools to go anywhere close to success.
But you still would be generally intelligent. Way more than most humans with normal memory.
If we'd have TODAY an AI with the same parameters as the human with perfect memory, it will be most certainly closely examined and determined to be not a general artificial intelligence.
> If we'd have TODAY an AI with the same parameters as the human with perfect memory, it will be most certainly closely examined and determined to be not a general artificial intelligence.
The human could learn to master a task, current AI can't. That is very different, the AI doesn't learn to remember stuff they are stateless.
When I can take an AI and get it to do any job on its own without any intervention after some training then that is AGI. The person you mentioned would pass that easily. Current day AI aren't even close.
But we know Hitler has a Time Machine that goes forward, he doesn’t need to return to use that knowledge as he already has a timeline here to use. Definitely risks involved here.
If you build an oracle that tells you who wins the war that far in the future, you build a simulator that allows anyone to win any war. Everything is dual use.
That will never work on any complex system that behaves chaotically, such as the weather or complex human endeavors. Tiny uncertainties in the initial conditions rapidly turn into large uncertainties in the outcomes.
Yeah, I wish more people understood that it is simply not possible to make precise long-term forecasts of chaotic systems. Whether it is weather, financial markets, etc.
It is not that we don't know yet because our models are inadequate, it's that it is unknowable.
The problem is we stupidly branded the field "chaos theory" and made it sound like bullshit so the ideas of non-linear dynamics have largely been lost on several generations at this point.
Not just chaos theory but "chaos theory" + psychedelic fractal artwork. Then the popular James Gleick book, "Chaos: making a new science" just sounds like complete bullshit and it sold a ton of copies.
I only started studying non-linear dynamics in about 2015 after first running across it in the late 90s but I literally thought it was all pseudoscience then.
Between "chaos theory", fractals and a best selling book it would be hard to frame a new scientific field as pseudoscience more than what played out.
> ask it for a formula for mass-energy equivalence
Way too easy. If you think that mass and energy might be equivalent, then dimensional analysis doesn’t give you too much choice in the formula. Really, the interesting thing about E=mc^2 isn’t the formula but the assertion that mass is a form of energy and all the surrounding observations about the universe.
Also, the actual insight in 1905 was more about asking the right questions and imagining that the equivalence principle could really hold, etc. A bunch of the math predates 1905 and would be there in an AI’s training set:
e=mc^2 is only correct for objects at rest. The full equation takes into account velocity, but for "low" speeds where v<<c, the term is close enough to zero than E=mc^2 is still a good approximation.
What I learnt is that there is a rest mass and a relativistic mass. The m in your formula is the rest mass. But when you use the relativistic mass E=mc² still holds. And for the rest mass I always used m_0 to make clear what it is.
sounds like you had a chemistry education. relativistic mass is IMO very much not a useful way of thinking about this and it is sort of tautologically true that E = m_relativistic because “relativistic mass” is just taking the concept of energy and renaming it “mass”
This is all sort of silly IMO. The equation, like basically all equations, needs context. What’s E? What’s m? If E is the total energy of the system and m is the mass (inertial or gravitational? how far past 1905 do you want to go?), then there isn’t a correction. If m is rest mass and E is total energy, then I would call it flat-out wrong, not merely approximate. After all, a decent theory really ought to reproduce Newtonian mechanics under some conditions beyond completely at rest.
IMO, when people get excited about E=mc^2, it’s in contexts like noticing that atoms have rest masses that are generally somewhat below the mass of a proton or neutron times the number of protons and neutrons in the atom, and that the mass difference is the binding energy of the nucleus, and you can do nuclear reactions and convert between mass and energy! And then E=mc^2 is apparently exactly true, or at least true to an excellent degree, even though the energies involved are extremely large and Newtonian mechanics can’t even come close to accounting for what’s going on.
inertial mass, rest mass, gravitational mass - these are essentially all the same thing. “relativistic mass” is an additional concept where we rewrite energy as mass and is considered archaic
I should have used m_0 to avoid confussion. Anyway, as he sibling comment says, most modern advanced books of special relativity try to avoid the relativistic mass. It's useful for some calculations, like synchrotron, but the problem is that for forward/backward acceleration you must use other number so the relativistic mass add confussion. https://en.wikipedia.org/wiki/Mass_in_special_relativity#His...
The next section of the wikipedia link discusses the low speed approximation, where sqrt(m^2c^4+(pc)^2) ≈ mc^2 + 1/2 mv^2.
Calling E=mc^2 an "approximation" is technically correct. It's the 0th order approximation. That's just pointlessly confusing. A better word choice would be "a special case".
i think we are venturing into pedantic territory - the point of my comment is that the full derivation is a little harder than just E=mc^2 dimensional analysis
Kind of agree. But pervasive downvoting by folks who don’t understand the subject is a form of forum rot. The risk is only that we expose the rot. Not such a terrible risk, because either the owners notice and fix the problem, or the forum continues to rot. In the latter case karma points wont be desirable in the long run.
This is why RLHF causes those overly verbose answers to simple questions, it's a fundamentally busted evaluation function so you wind up optimizing for the wrong thing
In one extreme there are wall of text and in the other extreme very short answers that only the initiated understand (like inside jokes). Somewhere in between there is a sweet spot that helps everyone else to follow the discusion and gain a litle of knowdledge.
(I don't claim I get the best lenght in my comments, but I hope it's good enough.)
its not an either/or, it is both. regardless, my point is that you cannot simply dimensional analysis your way to special relativity or the energy-momentum relation
This thread has come up before(1), but I'll continue to argue that relativistic mass is a perfectly valid concept as much as any other, and if you disagree, you'll need arguments more substantial than it just being unpopular these days. Especially if you're trying to argue people out of using a concept that they personally find useful to aid their own understanding, just because it doesn't fit your own mathematical or aesthetic preferences.
1. is not very intuitive/useful to have mass that varies on the direction (which is what this implies)
2. is somewhat tautological to define a new mass m_rel = E/c^2 and say that it satisfies the equation when this is not what most people understand mass to be. most people understand photons to be massless particles.
at minimum, relativistic mass should always be specified as m_rel to distinguish from what is typically referred to as mass.
but i don’t think relativistic mass is a wrong concept any more than any other mathematical convenience like virtual particles. the main question is how useful is it and should it be described using the word “mass” or is this confusing. there is value in having shared language, even if you can construct an alternate system of symbols and rules that can yield the same answer to every question. to the extent to which intent of the author matters at all (probably doesn’t), Einstein agreed that relativistic mass was not a useful concept.
i'll concede that the arguments in the thread you linked are not good
> most people understand photons to be massless particles.
I suspect that most people have no opinion at all and are generally unaware of the properties of light.
That being said, a (perfectly reflective or otherwise steady state, e.g. black body at equilibrium) box of photons has mass due to those photons. You can stick it on a scale or a pendulum and measure it. It attracts other nearby objects according to GR or, in the appropriate limit, Newtonian gravity, in accordance with the relativistic mass of those photons.
Why do we need this when current models already handle questions and answers about new discoveries: ones that are happening every week and are often easier to grasp than Einstein’s equations? I think it is clear that they will fail on most of them. That doesn't mean that LLMs are not useful but there are more walls in the road.
Instead of asking for a formula, a better test may be to point out all the seeming contradictions in physics at that time (constancy of the speed of light, wave vs. particle nature of light, ultraviolet catastrophe), and ask it how they could be resolved.
Isn't this simply because the dataset used (Putnam-AXIOM Original) is in the training data used to train the various models?
Given that these are simple variations (variable names and constants value change in math problems). Why would the companies creating these models (OpenAI etc.) create these variations themselves in order to insure that the model is learning how to solve the problem rather than memorize a solution? Seems like a very obvious thing to do ...
Link title says "slightly", but the PDF says two different kinds of variations: variable names (slight) and problem constants (significant), and the 30% drop is on the combination of a 26 variable and also 26 variable + constant questions.
It's good to have a better test (though I bet this one will also be quickly saturated like all the others), but the title here doesn't seem justified by the page title there or the content.
Right, renaming a variable should have zero effect on ability to solve (it wouldn’t for a human). Changing a constant should be very minor, probably also ~0 effect in most cases. I say this as someone that’s done many of these problems.
It drops from 50 to 33,96.
Still the best, o1 on variable is around 2 times better than Claude on original test.
The rest of the llm are far away, single digit.
It makes me wonder if o1 is finally getting intelligent? LLM are not supposed to understand these problems when you change variable and values, they have to rely on preexisting data of absolutely identical solved problem to give a correct answer.
I didn't follow LLM development but I heard one times that chatgpt is now composed of multiple LLM and maybe they put multiple artificial intelligence with purpose of problems solvings or trigonometry for instance.
That would explain the reason it's so much better.
The paper includes several examples of their modified questions. There has been a substantial jump from o1-preview to o1, so I gave several samples to o1 and o1-pro (not o1-preview), and current o1s gave the correct answer to those modified problems. SOTA changes fast.
LLM boosters are so tiresome. You hardly did a rigorous evaluation, the set has been public since October and could have easily been added to the training data.
I'm not skilled enough in math to do a rigorous evaluation, so it was a quick check.
Terence Tao is skilled enough, and he describes O1's math ability is "...roughly on par with a mediocre, but not completely incompetent graduate student" (good discussion at https://news.ycombinator.com/item?id=41540902), and the next iteration O3 just got 25% on his brand new Frontier Math test.
Seeing LLMs as useless is banal, but downplaying their rate of improvement is self-sabotage.
The paper mentions that on several occasions the LLM will provide a correct answer but will either take big jumps without justifying them or will take illogical steps but end up with the right solution at the end. Did you check for that?
So many negative comments as if o3 didn’t get 25% on frontiermath - which is absolutely nuts.
Sure, LLMs will perform better if the answer to a problem is directly in their training set. But that doesn’t mean they perform bad when the answer isn’t in their training set.
1. OpenAI said they did not train on these problems & they don’t train on API calls in general, that is a legal policy.
2. It was a new major model release from work over the course of months - struggle to see that as an ‘overnight’ jump in any real meaning.
3. Why is it easier to believe large scale corporate fraud than that the stated capabilities on a held out test set are real? Reads like cope, if I’m being frank.
Just because you can generalize the topic doesn't mean you can ignore the specific conversation and choose your hill to argue.
Additionally, the conversation of this topic is about the model's ability to generalize and it's potential overfitting, which is arguably more important than parroting mathematics.
The researcher's answer to their variant of "Year: 2016 ID: A1" in the appendix is wrong.
The solution (sum of 1,2,5,6,9,10,13,14, ...) has an alternating pattern, so has to be two piecewise interleaved polynomials, which cannot be expressed as a single polyomial.
I wouldn’t be surprised if similar will be found concerning the ARC challenge and it is why I still maintain my own private LLM challenges to gauge current capabilities. Course, I have little illusion that these are fully private, but it is better than fully public tests.
Even the most straight forward, logical, easily reasoned ones stump all LLMs I have access to, which is why I am so skeptical concerning emergence, reasoning and all this hype around “AGI”…
I think that lamentations about real world data running out is misplaced. We can multiply data with slight variations which might lead to better resilience and more accurate model's responses for novel problems.
557 comments
[ 0.22 ms ] story [ 293 ms ] thread(There was also a much earlier piece of software that would generate semi-intelligible Kant or Hegel one sentence at a time, though that was through a series of a priori generation rules and a large at the time dictionary of stock phrases. I wonder what ever happened to that.)
The problem is just that people keep insisting that those things are intelligent.
[1] https://en.wikipedia.org/wiki/Overfitting
The short version is that llm trainign data is the lowest quality data you are likely to see unless you engage in massive potential copyright infringement.
> Putnam is not in the test data, at least I haven't seen OpenAI claiming that publicly
What exactly is the source of your belief that the Putnam would not be in the test data? Didn’t they train on everything they could get their hands on?
So it's like 99.9999999% wrong to assume something public isn't on the train set, such as Putnam problems in this case. This is about it.
this whole notion of putnam as test being trained on is a fully invented grievance
read the entire thread in this context
I agree that having putnam problems on OpenAI training set is not a smoking gun, however it's (almost) certain they are on training set, and having them would affect performance of the model on them too. Hence research like this is important, since it shows that observed behavior of the models is memoization to large extent, and not necessarily generalization we would like it to be.
OAI uses datasets like frontiermath or arc-agi that are actually held out to evaluate generalization.
Every decent AI lab does this, else the benchmark result couldn't be trusted. OpenAI publishes results of ~20 benchmarks[2] and it is safe to assume they have made reasonable attempt to remove it from training set
[1]: https://arxiv.org/pdf/2107.06499
[2]: https://openai.com/index/hello-gpt-4o/
https://kskedlaya.org/putnam-archive/
I would expect all llms to be trained on it.
Those are well known problems, that people talk about on different contexts. They would have to review their entire training set.
"Solve (1503+5171)*(9494-4823)" reliably gets the correct answer from ChatGPT
"Write a poem about the solution to (1503+5171)*(9494-4823)" hallucinates an incorrect answer though
That suggests to me that they've papered over the models inability to do basic math, but it's a hack that doesn't generalize beyond the simplest cases.
i’m able to reproduce your failure on 4o
1. The part of the network that does complex math and the part that write poetry are overlapping in strange ways.
2. Most of the models nowadays are assumed to be some mixture of experts. So it's possible that saying write the answer as a poem activates a different part of the model.
The poem thing probably causes them to not decide to use those tools.
Basically if something appeared online or was transmitted over the wire should no longer be eligible to evaluate on. D. Sculley had a great talk at NeurIPS 2024 (same conference this paper was in) titled Empirical Rigor at Scale – or, How Not to Fool Yourself
Basically no one knows how to properly evaluate LLMs.
It's because people do keep asking these models math problems and then, when they get them right, citing it as evidence that they can actually do mathematical reasoning.
Since it's hard to determine what the models know, it's hard to determine when they're just spitting out something they were specifically trained on.
going forward, llm performance must be reported on the confounded benchmark as well
If o1-preview was the start like gpt1, then we should expect generalization to increase quickly.
Study on the topic: https://arxiv.org/html/2406.15992v1
This would explain o1 poor performance with problems with variations. o3 seems to be expensive brute forcing in latent space followed by verification which should yield better results - but I don't think we can call it generalisation.
I think we need to go back to the drawing board.
The investments into AI are in the hundreds of billions (maybe even more if you factor in the amount of people studying and researching AI), but the returns are in the tens of billions (if even that).
If you exclude the "growth" coming from the industry sniffing its own farts (e.g. Nvidia selling insane amounts of insanely overpriced GPUs to InsertYourFavAICorp), the actual amount of "useful goods and services" produced (api accesses, chat subscriptions, ai-enabled app growth etc.) are tiny compared to the investment levels.
The AI train appears to have no brakes. A massive crash or AGI are the only options now. Both are going to be bad for average humans.
I think ultimately the disconnect is people theorizing about what it can or cannot do with an incorrect mental model of what it is, and then assuming it cannot do things that it can in fact do. The irony of discussions on LLMs is they more showcase the limits of humans ability to reason about novel situations.
This is the same thing as synthetic training data.
It doesn't matter if models are trained on the output of the generated data or not. If the model ends up being able to solve newly generated variations, you'd have to admit that it understands the underlying problems.
It's good in automation, not understanding.
I see this more akin to a human faking their way through a conversation.
That works in English class. Try it in a math class and you'll get a much lower grade than ChatGPT will.
I am impressed by the progress on LLMs but I remain skeptical that they can replace humans.
Perhaps some (distant!) future model but I don’t fear mass unemployment (for example) or even moderate LLM-driven unemployment in the near-to-medium term.
They can clearly complement human labor but there are vanishingly few domains where they can be substitutes.
The ability to perfectly regurgitate, while no concept of meaning.
I don't believe that. I'd put some good money that if an excellent student is given an exact question from a previous year, they'll do better (faster & more accurate) on it, than when they're given a variation of it.
The assumptions aren't the same to begin with.
Our evaluation on various open-source and proprietary models reveals that the highest performance, even after using techniques like self-consistency, self-refinement and chain-of-thought prompting, is less than 40%. The typical failure modes of GPT-4, the best model, are errors in algebraic manipulation, difficulty in grounding abstract concepts into mathematical equations accurately and failure in retrieving relevant domain-specific concepts.
I'm curious how something like O1 would perform now.
At top tier schools the most common score will usually be somewhere in the 0 to 10 range (out of a possible 120).
How do you know that?
2. They don't train on API calls
3. It is funny to me that HN finds it easier to believe theories about stealing data from APIs rather than an improvement in capabilities. It would be nice if symmetric scrutiny were applied to optimistic and pessimistic claims about LLMs, but I certainly don’t feel that is the case here.
But i'm not saying it's what they did, just that it's a possibility that should be considered till/if it is debunked.
codeforces constantly adds new problems that’s like the entire point of the contest, no?
If they solved recent contests in a realistic contest simulation I would expect them to give the actual solutions and success rates as well, like they did for IOI problems, so I'm actually confused as to why they didn't.
Given that performance is known to drop considerably on these kinds of tests when novel problems are tried, and given the ease with which these problems could leak into the training set somehow, it's not unreasonable to be suspicious of a sudden jump in performance as merely a sign that the problems made it into the training set rather than being true performance improvements in LLMs.
The real problem with all of these theories is most of these benchmarks were constructed after their training dataset cutoff points.
A sudden performance improvement on a new model release is not suspicious. Any model release that is much better than a previous one is going to be a “sudden jump in performance.”
Also, OpenAI is not reading your emails - certainly not with a less than one month lead time.
I'd like to see if it's truly novel and unique, the first problem of its type ever construed by mankind, or if it's similar to existing problems.
The point is that the fact it's no longer consistent once you vary the terminology indicates it's fitting a memorized template instead of reasoning from first principles.
Since o1 on codeforces just tried hundreds or thousands of solutions, it's not surprising it can solve problems where it is really about finding a relatively simple correspondence to a known problem and regurgitating an algorithm.
In fact when you run o1 on ""non-standard"" codeforces problems it will almost always fail.
See for example this post running o1 multiple times on various problems: https://codeforces.com/blog/entry/133887
So the thesis that it's about recognizing a problem with a known solution and not actually coming up with a solution yourself seems to hold, as o1 seems to fail even on low rated problems which require more than fitting templates.
Additionally, people weren't able to replicate o1-mini results in live contests straightforwardly - often getting scores between 700 and 1200, which raises questions as for the methodology.
Perhaps o3 really is that good, but I just don't see how you can claim what you claimed for o3, we have no idea that the problems have never been seen, and the fact people find much lower Elo scores with o1/o1-mini with proper methodology raises even more questions, let alone conclusively proving these are truly novel tasks it's never seen.
Companies claim lots of things when it's in their best financial interest to spread that message. Unfortunately history has shown that in public communications, financial interest almost always trumps truth (pick whichever $gate you are aware of for convenience, i'll go with Dieselgate for a specific example).
> It is funny to me that HN finds it easier to believe theories about stealing data from APIs rather than an improvement in capabilities. It would be nice if symmetric scrutiny were applied to optimistic and pessimistic claims about LLMs, but I certainly don’t feel that is the case here.
What I see is generic unsubstantiated claims of artificial intelligence on one side and specific, reproducible examples that dismantle that claim on the other. I wonder how your epistemology works that leads you to accept marketing claims without evidence
From a game-theoretic standpoint, repeated interactions with the public (research community, regulators, and customers) create strong disincentives for OpenAI to lie. In a single-shot scenario, overstating model performance might yield short-term gains—heightened buzz or investment—but repeated play changes the calculus:
1. Reputation as “collateral”
OpenAI’s future deals, collaborations, and community acceptance rely on maintaining credibility. In a repeated game, players who defect (by lying) face future punishment: loss of trust, diminished legitimacy, and skepticism of future claims.
2. Long-term payoff maximization
If OpenAI is caught making inflated claims, the fallout undermines the brand and reduces willingness to engage in future transactions. Therefore, even if there is a short-term payoff, the long-term expected value of accuracy trumps the momentary benefit of deceit.
3. Strong incentives for verification
Independent researchers, open-source projects, and competitor labs can test or replicate claims. The availability of external scrutiny acts as a built-in enforcement mechanism, making dishonest “moves” too risky.
Thus, within the repeated game framework, OpenAI maximizes its overall returns by preserving its credibility rather than lying about capabilities for a short-lived advantage.
Uh huh. Kinda like what's happening right now?
They're marketing blow-hards. Everyone knows it. They've been wildly over-stating capabilities (and future capabilities!) as long as Altman has had power, and arguably longer.
They'll do it as long as they can get away with it, because that's all that is needed to make money on it. Factual accuracy rarely impacts the market when it's so hype-driven, especially when there is still some unique utility in the product.
4 was literally sitting on a shelf waiting for release when 3.5 was launched. 4o was a fine tune that took over two years. o1 is embarrassingly unimpressive chain of thought which is why they hide it.
The company hit a wall a year ago. But showing progress towards AGI keeps the lights on. If they told the truth at their current burn rate…they’d have no money.
You don’t need game theory to figure that one out.
They're spruiking a 93rd percentile performance on the 2024 International Olympiad in Informatics with 10 hours of processing and 10,000 submissions per question.
Like many startups they're still a machine built to market itself.
They are the system to beat and their competitors are either too small or too risk averse.
They ingest millions of data sources. Among them is the training data needed to answer the benchmark questions.
Freudian typo?
Definition 2D: "an organizing theme or concept"
https://xenaproject.wordpress.com/2024/12/22/can-ai-do-maths...
1. it suggests it’s possible that more of the problems are IMO-esque than previously thought, we don’t know how the share of solved problems is.
2. calling IMO problems “well known undergraduate problems” is a bit much
https://x.com/littmath/status/1870848783065788644?s=46&t=foR...
I think it’s more probable that it would have solve the easier problems first, rather than some hard and only some easier; although that is supposition.
Reading this thread and the blog post gives more idea about what the problems might involve.
It’s difficult to judge without more information on the actual results, but that means we cannot draw any strong conclusions either way on what this means.
If you are shocked by this, you are the sucker in the room.
Every. Single. Time.
Almost as if there was a financial incentive to do that.
Tech hype is becoming unbearable as time goes on.
Great things take many years, sometimes decades to develop properly. Different generations of people to experiment and try things out.
That is not good for the ones pushing up the hype. You don't get rich quick by doing that, you don't get to scam enough investors by something being slowly improved.
You may call it bitterness, whereas I am just jaded by watching things play out.
I mean, that's cool... we can get a lot of work done with pattern recognition. Most of the human race never really moves above that level of thinking in the workforce or navigating their daily life, especially if they default to various societally prescribed patterns of getting stuff done (eg. go to college or the military <Based on <these criteria>, find a job <based on the best fit with <this list of desirable skills & experiences>, go to <these places> to find love....)
Or maybe it's time to stop wheeling out this tedious and disingenuous dismissal.
Saying it is just "pattern recognition" (or a "stochastic parrot") implies behavioural and performance characteristics that have very clearly been greatly exceeded.
It is evident that it is not recalling the sum because all combinations of integer addition were likely not in the training data, Storing the answer to the sum of all integers up to the size that GPT4 can manage would take more parameters than the model has.
That addition is a small capability but you only need a single counterexample to disprove a theory.
No, that's not how this works :)
You can hardcode an exception to pattern recognition for specific cases - it doesn't cease to be a pattern recognizer with exceptions being sprinkled in.
The 'theory' here is that a pattern recognizer can lead to AGI. That is the theory. Someone saying 'show me proof or else I say a pattern recognizer is just a pattern recognizer' is not a theory and thus cannot be disproven, or proven.
This is also known as Russell's teapot. https://en.wikipedia.org/wiki/Russell%27s_teapot
If someone claims there's a teapot out in space - the burden of proof is on the person making the claim, not on the person saying it is bullshit.
Is the claim then that LLMs are pattern recognizers but also more?
It just seems to me and I guess many others that the thing it is primarily good at is being a better google search.
Is there something big that I and presumably many others are missing and if so, what is it?
I think you are misinterpreting what the argument is.
The argument being made is that LLMs are mere 'stochastic parrots' and therefore cannot lead to AGI. The analogy to Russell's teapot is that someone is claiming that Russells teapot is not there because china cannot exist in the vacuum of space. You can disprove that with a single counterexample. That does not mean the teapot is there, but it also doesn't mean it isn't.
It is also hard to prove that something is thinking. It is also very difficult to prove that something is not thinking. Almost all arguments against AGI take the form X cannot produce AGI because Y. Those are disprovable because you can disprove Y.
I don't think anyone is claiming to have a proof that an LLM will produce AGI, just that it might. If they actually build one, that too counts as a counterexample to anybody saying they can't do it.
Honestly if you don't believe me just go and use them. It's pretty obvious if you actually get experience with them.
Imagine if people who had never used VR kept saying it's just a TV on your face, or if people who had never used static types kept saying they're just extra work you have to do, or if people who had never had sex kept saying it's just a way of making babies.
It's a tedious claim when it's so easily disproven by going to a free website and trying it. Why are people so invested in AI being useless that they'll criticise it so confidently without even trying it?
If we take an example of what is considered a priori as creativity, such as story telling, LLMs can do pretty well at creating novel work.
I can prompt with various parameters, plot elements, moral lessons, and get a de novo storyline, conflicts, relationships, character backstories, intrigues, and resolutions.
Now, the writing style tends to be tone-deaf and poor at building tension for the reader, and it is apparent that the storytelling has little “theory of mind” of the reader, but the material has elements that we would certainly consider to be creative if written by a student.
It seems we must either cede that LLMs can do some creative synthesis, as this and some other experiments of mine suggest, or we must decide that these tasks, such as “creative writing” are not in fact creative, but rather mostly or strictly derivative.
There is some argument to be had in assertions that storytelling is all derivative of certain patterns and variations on a fixed number of tropes and story arcs… but arguing this begs the question of whether humans actually do any “pure” creative work , or if in fact, all is the product of experience and study. (Training data)
Which leads me to the unpleasant conflict about the debate of AI creativity. Is the debate really pointing out an actual distinction, or merely a matter of degree? And what are the implications, either way?
I’m left with the feeling that LLMs can be as capable of creative work as most 8th grade students. What does this say about AI, or developing humans? Since most people don’t exceed an 8th grade level of literacy, what does this say about society?
Is there even such a thing as de novo idea synthesis?
Troubling questions abound.
Just because something is creative doesn’t mean it’s inherently valuable.
Not really. Unless I'm not reading correctly, most of the problem is irrelevant as you're only required to cross the boat with the goat, you don't care about the cabbage. The difficulty lies in the assumption you need to cross everything due to the resemblance with the bigger problem.
The llm isn't getting confused by the meaning of "item". It's recognizing a common problem and not picking up on the fact that the farmer just needs to transport the goat and nothing else.
Instead, it gives the standard answer for how to transport everything across.
Gpt-3 is old hat though. later versions of gpt-4 manage to get it with a bunch coaching, and o1 manages to solve it with less coaching.
The other reason this seems hard to guess is because we don't know how much of what we are asking is in the training data. It would perform on some tasks, while fail at others even though those are similar.
I also believe that the 200$ subscription they offer is just them allowing the TTC to go for longer before forcing it to answer.
If what you say is true, though, I agree that there is a huge headroom for TTC to improve results if the huggingface experiments on 1/3B models are anything to go off.
> huge headroom for TTC to improve results ...1B/3B models
Absolutely. How this is productized remains to be seen. I have high hopes with MCTS and Iterative Preference Learning, but it is harder to implement. Not sure if Open AI has done that. Though Deepmind's results are unbelievably good [1].
[1]:https://arxiv.org/pdf/2405.00451v2
https://huggingface.co/spaces/HuggingFaceH4/blogpost-scaling...
https://openai.com/index/learning-to-reason-with-llms/
They only confirm that the model/system is doing chain of thought, but the exponential factor and origin of reasoning gains likely comes from TREE of thoughts (number of branches/compute goes up exponentially with depth), essentially doing tree search over different reasoning chains.
I assume roon's identity is well known inside OpenAI (he's an employee), so I wouldn't expect him to be leaking implementation details on twitter.
This is all getting so tiresome.
It seems some people disagree that that is what the game "Simon Says" is about. I feel like they might play a vastly simplified version of the game that I am familiar with.
There was a recent episode of Game Changer based on this which is an excellent example of how the game leader should attempt to induce errors by making a change that does not get correctly accounted for.
Using LLMs for anything serious and which requires consistency and trustworthiness without hallucinations is irresponsible and ridiculous.
Closed source LLMs are a bubble and a joke.
Passing tests is well known to be much easier than having deep understanding, even in humans. They openly ask for tests like this, not that they could possibly prevent them if they wanted to.
There's scammers trying what you say of course, and I'm sure we've all seen some management initiatives or job advertisements for some like that, but I don't get that impression from OpenAI or Anthropic, definitely not from Apple or Facebook (LeCun in particular seems to deny models will ever do what they actually do a few months later). Overstated claims from Microsoft perhaps (I'm unimpressed with the Phi models I can run locally, GitHub's copilot has a reputation problem but I've not tried it myself), and Musk definitely (I have yet to see someone who takes Musk at face value about Optimus).
I never understood why this definition isn't a huge red flag for most people. The idea of boiling what intelligence is down to economic value is terrible, and inaccurate, in my opinion.
Try applying that definition to humans and you pretty quickly run into issues, both moral and practical. It also invalidates basically anything we've done over centuries considering what intelligence is and how to measure it.
I don't see any problem at all using economic value as a metric for LLMs or possible AIs, it just needs a different term than intelligence. It pretty clearly feels like for-profit businesses shoehorning potentially valuable ML tools into science fiction AI.
The response from @s1mplicissimus' on my previous comment is asking about "common usage" definitions of intelligence, and this is (IMO unfortunately) one of the many "common usage" definitions: smart people generally earn more.
I don't like "commmon sense" anything (or even similar phrases), because I keep seeing the phrase used as a thought-terminating cliché — but one thing it does do, is make it not "a very, very strange approach".
Wrong, that happens a lot for common language, but it can't really be strange.
> Try applying that definition to humans and you pretty quickly run into issues, both moral and practical.
Yes. But one also runs into issues with all definitions of it that I've encountered.
> It also invalidates basically anything we've done over centuries considering what intelligence is and how to measure it.
Sadly, not so. Even before we had IQ tests (for all their flaws), there's been a widespread belief that being wealthy is the proof of superiority. In theory, in a meritocracy, it might have been, but in practice not only to we not live in a meritocracy (to claim we do would deny both inheritance and luck), but also the measures of intelligence that society has are… well, I was thinking about Paul Merton and Boris Johnson the other day, so I'll link to the blog post: https://benwheatley.github.io/blog/2024/04/07-12.47.14.html
> there's been a widespread belief that being wealthy is the proof of superiority.
Both of these are assumptions though, and working in the reverse order. Its one thing to expect that intelligence will lead to higher value outcomes and entirely different to expect that higher value outcomes prove intelligence.
It seems reasonable that higher intelligence, combined with the incentives if a capitalist system, will lead to higher intelligence people getting more wealthy. They learn to play the game and find ways to "win."
It seems unreasonable to assume that anyone or anything that "wins" in that system much be more intelligent. Said differently, intelligence may lead to wealth but wealth doesn't imply intelligence.
All the other things — chess, Jeopardy, composing music, painting, maths, languages, passing medical or law degrees — they're also all things which were considered signs of intelligence until AI got good at them.
Goodhart's law keeps tripping us up on the concept of intelligence.
Maybe we are? I think I lost the thread a bit here.
> chess, Jeopardy, composing music, painting, maths, languages, passing medical or law degrees
That's interesting, I would have still chalked skill in those areas as a sign of intelligence and didn't realize most people wouldn't once AI (or ML) could do it. To me an AI/LLM/ML being good at those is at least a sign that they have gotten good at mimicking intelligence if nothing else, and a sign that we really are getting out over our skis risking these tools without knowing how they really work.
edit: lol downvoted for calling out shilling i guess
But I'm sure there are lots of ways to go about it.
https://www.youtube.com/watch?v=XhB3qH_TFds
Or RSS
https://api.substack.com/feed/podcast/69345.rss
The terabytes of training data required for decent LLMs does not exist. I’d guess there may only be gigabytes worth.
(for anyone who doesn't feel like downloading the zip, here is a single image from the zip: https://s.h4x.club/nOu485qx)
really thanks for sharing!
1. In theory these kind of connections should be something that LLMs are great at doing. 2. It appears that LLMs are not trained (yet?) on cursive and other non-print text
No idea if that’s correct (and no doubt not useful to an expert able to read this directly, but curious if it’s close?
Gemini 2.0 Experimental Advanced says:
-------snip--------
This document is a petition filed in the Orphans Court of Baltimore County. Here's a breakdown of its content:
Petitioners: Neary Senters and his wife are petitioning the court.
Subject: The petition concerns a boy named James Gannon, who is identified as a pauper and has been bound (apprenticed) to them.
Issue: The petitioners claim that James Gannon is claimed by a certain Henry Haine of Baltimore City. They believe Haine is attempting to exploit the boy's labor. He obtained the boy through a seemingly valid indenture (apprenticeship contract) authorized by two Justices of the Peace in Baltimore City. However, the indenture references records in the Register of Wills' office in Baltimore City, suggesting the boy was already an indentured servant. The petitioners argue that a 1793 Maryland law (Chapter 45) regarding the binding of apprentices invalidates Haine's claim. They assert that the law requires parental consent for binding out a child, and there's no evidence of such consent in this case.
Request: The petitioners request the court to summon Henry Haine and the Justices of the Peace involved in creating the indenture. They also seek to have James Gannon and his parents (if living) appear before the court. Their ultimate goal is to have the indenture to Haine declared invalid, thus retaining custody and the labor of the boy.
Legal Argument: The core of their argument hinges on the 1793 Maryland law, which they believe was violated when James Gannon was bound to Henry Haine without parental consent. They are essentially challenging the legality of the indenture held by Haine.
In essence, this document reveals a dispute over the legal custody and labor of a young boy in 18th-century Baltimore. The petitioners are using a specific law about apprenticeship to argue against another individual's claim to the boy, highlighting the importance of legal procedures and parental rights in such matters during that time.
-------end snip--------
The prompt was simply "Please provide a detailed summary of this document" (I wasn't sure what else to ask.)
Is that closer to your wife's interpretation? I mean, I can't make head or tail of it...
Feel like I should make this task my personal ASI benchmark lol :D
---------snip--------
Gotta be an insanely-challenging task for a program that wasn't even written with handwriting recognition in mind.
Other than the proper names, are any major details wrong?
She provided this as one she just got done working with: https://s.h4x.club/z8u9xmv7 (John King Esq. but try giving it to an LLM)
I will also happily again admit a bit of goal post moving on my part. I was probably a little to harsh on it (maybe because I'm used to her and her history geeks talking about how they don't work well for their research).
How much would ChatGPT charge for that much reasoning? Isn't cost quadratic in sort term working memory?
It would be more interesting to prompt it with X% of a new paper's logical argument, and see if it can predict the rest.
We had a threshold for intelligence. An LLM blew past it and people refuse to believe that we passed a critical milestone in creating AI. Everyone still thinks all an LLM does is regurgitate things.
But a technical threshold for intelligence cannot have any leeway for what people want to believe. They don’t want to define an LLM as intelligent even if it meets the Turing test technical definition of intelligence so they change the technical definition.
And then they keep doing this without realizing and trivializing it. I believe humanity will develop an entity smarter than humans but it will not be an agi because people keep unconsciously moving the goal posts and changing definitions without realizing it.
Yep, that constitutes the second of the two options I mentioned.
> Well, smart people agree but those people also agree we have or will soon have agi or something negligibly different from it.
lol, the ol' "I know what all smart people think and it's what I think" appeal.
> We had a threshold for intelligence.
We’ve had many. Computers have surpassed several barriers considered to require intelligence such as arithmetic, guided search like chess computers, etc etc. the Turing test was a good benchmark because of how foreign and strange it was. It’s somewhat true we’re moving the goalposts. But the reason is not stubbornness, but rather that we can’t properly define and subcategorize what reason and intelligence really is. The difficulty to measure something does not mean it doesn’t exist or isn’t important.
Feel free to call it intelligence. But the limitations are staggering, given the advantages LLMs have over humans. They have been trained on all written knowledge that no human could ever come close to. And they still have not come up with anything conceptually novel, such as a new idea or theorem that is genuinely useful. Many people suspect that pattern matching is not the only thing required for intelligent independent thought. Whatever that is!
As far as pattern matching, the difference I see from humans is consciousness. That's probably the main area yet to be solved. All of our current models are static.
Some ideas for where that might be headed:
- Maybe all it takes is to allow an LLM to continuously talk with itself much like how humans have "the milk man's voice".
- Maybe we might need to allow LLMs to update their own weights but that would also require an "objective" which might be hard to encode.
I disagree that such a comparison is useful. Training should be compared to training, and LLM training feeds in so many more words than a baby gets. (A baby has other senses but it's not like feeding in 20 years of video footage is going to make an LLM more competent.)
The better comparison to the templating is all the labor that went into making the LLM, not how long the GPUs run.
Template versus template, or specific training versus specific training. Those comparisons make a lot more sense than going criss-cross.
Training is both longer and less effective for the LLM because there is no template.
To give an example suppose it takes just one picture for a human to recognize a dog and it takes 1 million pictures for a ML model to do the same. What I’m saying is that it’s like this because humans come preprogrammed with application specific wetware to do the learning and recognition as a generic operation. That’s why it’s so quick. For AI we are doing it as a one shot operation on something that is not application specific. The training takes longer because of this and is less effective.
Did you look at the post I was replying to? You're talking about LLMs being slower, while that post was impressed by LLMs being "faster".
They're posing it as if LLMs recreate the same templating during their training time, and my core point is disagreeing with that. The two should not be compared so directly.
But the training never gets there. It’s so slow it never reaches human intelligence even though we know these networks can compute anything.
Disagree. Intelligence is a word created by humans. The entire concept is made up and defined by humans. It is not some concept that exists outside of that. It is simply a collection of qualities and features we choose to define as a word “intelligent”. The universe doesn’t really have a category or a group of features that is labeled intelligent. Does it use logic? Does it have feelings? Can it talk? Can it communicate? We define the features and we choose to put each and every feature under a category called “intelligence”.
Therefore when we define the “Turing test” as a benchmark for intelligence and we then invalidate it, it is indeed stubbornness and a conscious choice to change a definition of a word we Originally made up in the first place.
What you don’t realize is this entire thing is a vocabulary problem. When we argue what is conscious or what is intelligent we are simply arguing for what features belong in what categories we made up. When the category has blurry or controversial boundaries it’s because we chose the definition to be fuzzy. These are not profound discussions. They are debates about language choice. We are talking About personal definitions and generally accepted definitions both of which are completely chosen and made up by us. It is not profound to talk about things that are simply arbitrary choices picked by humans.
That being said we are indeed changing the goal posts. We are evolving our own chosen definitions and we very well may eventually change the definition of intelligence to never include any form of thinking machine that is artificially created. The reason why we do this is a choice. We are saying, “hey these LLMs are not anything amazing or anything profound. They are not intelligent and I choose to believe this by changing and evolving my own benchmark for what is intelligent.”
Of course this all happens subconsciously based off of deeply rooted instincts and feelings. It’s so deep that it’s really hard to differentiate the instincts between rational thinking. When you think logically, “intelligence” is just a word with an arbitrary definition. An arbitrary category. But the instincts are so strong that you literally spent your entire life thinking that intelligence like god or some other common myth made up by humans is some concept that exists outside of what we make up. It’s human to have these instincts, that’s where religion comes from. What you don’t realize is that it’s those same instincts fueling your definition of what is “intelligent”.
Religious people move the goal posts too. When science establishes things in reality like the helio centricity of the solar system religious people need to evolve their beliefs in order to stay inline with reality. They often do this by reinterpreting the Bible. It’s deeply rooted instincts that prevent us from thinking rationally and it effects the great debate we are having now on “what is intelligence?”.
That was never "the bar"; nobody denies that milestones have been surpassed; none of those milestones are relevant to the question of intelligence.
> We had a threshold for intelligence. An LLM blew past it and people refuse to believe
Have you ever actually looked at contemporary (to Turing) examples of what people thought "passing a Turing test" might look like? It's abundantly clear to me that we were simply wrong about what the output would have to look like in order to convince human judges in the 2020s.
Even examples from much more recently (see e.g. on http://www-logic.stanford.edu/seminar/1213/Hawke_TuringTest....) suggest a very different approach to the test than prompting ChatGPT and marveling at the technical accuracy of its prose.
(Exercise: ask an LLM to write a refutation to your comment from the perspective of a human AI skeptic. Notice the ways in which it differs from mine.)
> Everyone still thinks all an LLM does is regurgitate things.
No; people still think LLMs aren't intelligent. Because they aren't, and they cannot become so in principle. They can do many things that are clearly beyond "regurgitation" (as we would otherwise apply the word to computer programs), but none of those things are the result of intelligence. Producing a result that could plausibly come from an intelligent system does not, in fact, demonstrate that the actual system producing it is also intelligent. The https://en.wikipedia.org/wiki/Antikythera_mechanism wasn't intelligent, either, and applying a power source to turn the gears wouldn't have made it so, either.
> They don’t want to define an LLM as intelligent even if it meets the Turing test technical definition of intelligence so they change the technical definition.
The Turing Test was never a "technical definition" of intelligence. Turing's original paper (https://en.wikipedia.org/wiki/Computing_Machinery_and_Intell...) spoke of "thinking" rather than "intelligence". Besides, the "Imitation Game" is presented as a substitute problem exactly because "think" cannot be clearly enough defined for the purposes. The entire point:
> As Stevan Harnad notes,[7] the question has become "Can machines do what we (as thinking entities) can do?" In other words, Turing is no longer asking whether a machine can "think"; he is asking whether a machine can act indistinguishably[8] from the way a thinker acts. This question avoids the difficult philosophical problem of pre-defining the verb "to think" and focuses instead on the performance capacities that being able to think makes possible, and how a causal system can generate them.
But the usual processes of pop science seem to have created a folk wisdom that being able to pass a Turing test logically ought to imply intelligence. This idea is what has been disproven, not the AI skepticism.
Because of the chance of misundertanding. Failing at acknowledging artificial general intelligence standing right next to us.
An incredible risk to take in alignment.
Perfect memory doesn't equal to perfect knowledge, nor perfect understanding of everything you can know. In fact, a human can be "intelligent" with some of his own memories and/or knowledge, and - more commmonly - a complete "fool" with most of the rest of his internal memories.
That said, is not a bit less generally intelligent for that.
Supose it exists a human with unlimited memory, it retains every information touching any sense. At some point, he/she will probably understand LOTs of stuff, but it's simple to demonstrate he/she can't be actually proficient in everything: you have read how do an eye repairment surgery, but have not received/experimented the training,hence you could have shaky hands, and you won't be able to apply the precise know-how about the surgery, even if you remember a step-by-step procedure, even knowing all possible alternatives in different/changing scenarios during the surgery, you simply can't hold well the tools to go anywhere close to success.
But you still would be generally intelligent. Way more than most humans with normal memory.
If we'd have TODAY an AI with the same parameters as the human with perfect memory, it will be most certainly closely examined and determined to be not a general artificial intelligence.
The human could learn to master a task, current AI can't. That is very different, the AI doesn't learn to remember stuff they are stateless.
When I can take an AI and get it to do any job on its own without any intervention after some training then that is AGI. The person you mentioned would pass that easily. Current day AI aren't even close.
https://www.technologyreview.com/2024/12/04/1107892/google-d...
I mean, there are the theorems about how close you can get, and models are not better than theoretically possible.
It is not that we don't know yet because our models are inadequate, it's that it is unknowable.
Not just chaos theory but "chaos theory" + psychedelic fractal artwork. Then the popular James Gleick book, "Chaos: making a new science" just sounds like complete bullshit and it sold a ton of copies.
I only started studying non-linear dynamics in about 2015 after first running across it in the late 90s but I literally thought it was all pseudoscience then.
Between "chaos theory", fractals and a best selling book it would be hard to frame a new scientific field as pseudoscience more than what played out.
Way too easy. If you think that mass and energy might be equivalent, then dimensional analysis doesn’t give you too much choice in the formula. Really, the interesting thing about E=mc^2 isn’t the formula but the assertion that mass is a form of energy and all the surrounding observations about the universe.
Also, the actual insight in 1905 was more about asking the right questions and imagining that the equivalence principle could really hold, etc. A bunch of the math predates 1905 and would be there in an AI’s training set:
https://en.m.wikipedia.org/wiki/History_of_Lorentz_transform...
e: nice, downvoted for knowing special relativity
this allows light to have energy even if its massless
where p is momentum. When an object is traveling at relativistic speeds, the momentum forms a more significant portion of its energy
When in doubt, add more info, like:
But the complete equation is E=sqrt(m^2c^4+p^2) that is reduced to E=mc^2 when the momentum p is 0. More info in https://en.wikipedia.org/wiki/Mass%E2%80%93energy_equivalenc...
IMO, when people get excited about E=mc^2, it’s in contexts like noticing that atoms have rest masses that are generally somewhat below the mass of a proton or neutron times the number of protons and neutrons in the atom, and that the mass difference is the binding energy of the nucleus, and you can do nuclear reactions and convert between mass and energy! And then E=mc^2 is apparently exactly true, or at least true to an excellent degree, even though the energies involved are extremely large and Newtonian mechanics can’t even come close to accounting for what’s going on.
Calling E=mc^2 an "approximation" is technically correct. It's the 0th order approximation. That's just pointlessly confusing. A better word choice would be "a special case".
In one extreme there are wall of text and in the other extreme very short answers that only the initiated understand (like inside jokes). Somewhere in between there is a sweet spot that helps everyone else to follow the discusion and gain a litle of knowdledge.
(I don't claim I get the best lenght in my comments, but I hope it's good enough.)
1: https://news.ycombinator.com/item?id=38425252
1. is not very intuitive/useful to have mass that varies on the direction (which is what this implies)
2. is somewhat tautological to define a new mass m_rel = E/c^2 and say that it satisfies the equation when this is not what most people understand mass to be. most people understand photons to be massless particles.
at minimum, relativistic mass should always be specified as m_rel to distinguish from what is typically referred to as mass.
but i don’t think relativistic mass is a wrong concept any more than any other mathematical convenience like virtual particles. the main question is how useful is it and should it be described using the word “mass” or is this confusing. there is value in having shared language, even if you can construct an alternate system of symbols and rules that can yield the same answer to every question. to the extent to which intent of the author matters at all (probably doesn’t), Einstein agreed that relativistic mass was not a useful concept.
i'll concede that the arguments in the thread you linked are not good
I suspect that most people have no opinion at all and are generally unaware of the properties of light.
That being said, a (perfectly reflective or otherwise steady state, e.g. black body at equilibrium) box of photons has mass due to those photons. You can stick it on a scale or a pendulum and measure it. It attracts other nearby objects according to GR or, in the appropriate limit, Newtonian gravity, in accordance with the relativistic mass of those photons.
Given that these are simple variations (variable names and constants value change in math problems). Why would the companies creating these models (OpenAI etc.) create these variations themselves in order to insure that the model is learning how to solve the problem rather than memorize a solution? Seems like a very obvious thing to do ...
It's good to have a better test (though I bet this one will also be quickly saturated like all the others), but the title here doesn't seem justified by the page title there or the content.
The rest of the llm are far away, single digit.
It makes me wonder if o1 is finally getting intelligent? LLM are not supposed to understand these problems when you change variable and values, they have to rely on preexisting data of absolutely identical solved problem to give a correct answer.
I didn't follow LLM development but I heard one times that chatgpt is now composed of multiple LLM and maybe they put multiple artificial intelligence with purpose of problems solvings or trigonometry for instance.
That would explain the reason it's so much better.
Terence Tao is skilled enough, and he describes O1's math ability is "...roughly on par with a mediocre, but not completely incompetent graduate student" (good discussion at https://news.ycombinator.com/item?id=41540902), and the next iteration O3 just got 25% on his brand new Frontier Math test.
Seeing LLMs as useless is banal, but downplaying their rate of improvement is self-sabotage.
Let it sink in how vague and almost meaningless that statement is.
Sure, LLMs will perform better if the answer to a problem is directly in their training set. But that doesn’t mean they perform bad when the answer isn’t in their training set.
An overnight 2% -> 25% jump on this benchmark is a bit curious.
2. It was a new major model release from work over the course of months - struggle to see that as an ‘overnight’ jump in any real meaning.
3. Why is it easier to believe large scale corporate fraud than that the stated capabilities on a held out test set are real? Reads like cope, if I’m being frank.
Your comment isn't relevant at all
Additionally, the conversation of this topic is about the model's ability to generalize and it's potential overfitting, which is arguably more important than parroting mathematics.
i'm not going to keep replying, others can judge whether they think what i'm saying is "relevant at all."
The topic here isn't "o-series sucks", it's addressing a found concern.
The solution (sum of 1,2,5,6,9,10,13,14, ...) has an alternating pattern, so has to be two piecewise interleaved polynomials, which cannot be expressed as a single polyomial.
Their answer works for k=1,2, but not k=3.
https://openreview.net/pdf?id=YXnwlZe0yf
This does not give me confidence in the results of their paper.
The statement doesn't hold for e.g. n=5. Taking m=2 gives the permutation (1 2 4 3), which is odd, and thus cannot have a square root.
Even the most straight forward, logical, easily reasoned ones stump all LLMs I have access to, which is why I am so skeptical concerning emergence, reasoning and all this hype around “AGI”…