so o1 seems like it has real measurable edge, crushing it in every single metric, i mean 1673 elo is insane, and 89th percentile is like a whole different league, and it looks like it's not just a one off either, it's consistently performing way better than gpt-4o across all the datasets, even in the ones where gpt-4o was already doing pretty well, like math and mmlu, o1 is just taking it to the next level, and the fact that it's not even showing up in some of the metrics, like mmmu and mathvista, just makes it look even more impressive, i mean what's going on with gpt-4o, is it just a total dud or what, and btw what's the deal with the preview model, is that like a beta version or something, and how does it compare to o1, is it like a stepping stone to o1 or something, and btw has anyone tried to dig into the actual performance of o1, like what's it doing differently, is it just a matter of more training data or is there something more going on, and btw what's the plan for o1, is it going to be released to the public or is it just going to be some internal tool or something
If its actually true in practice, I sincerely cannot imagine a scenario where it would be cheaper to hire actual junior or mid-tier developers (keyword: "developers", not architects or engineers).
1,673 ELO should be able to build very complex, scalable apps with some guidance
currently my workflow is generate some code, run it, if it doesn't run i tell LLM what I expected, it will then produce code and I frequently tell it how to reason about the problem.
with O1 being in the 89th percentile would mean it should be able to think at junior to intermediate level with very strong consistency.
i dont think people in the comments realize the implication of this. previously LLMs were able to only "pattern match" but now its able to evaluate itself (with some guidance ofc) essentially, steering the software into depth of edge cases and reason about it in a way that feels natural to us.
currently I'm copying and pasting stuff and notifying LLM the results but once O1 is available its going to significantly lower that frequency.
For example, I expect it to self evaluate the code its generate and think at higher levels.
ex) oooh looks like this user shouldn't be able to escalate privileges in this case because it would lead to security issues or it could conflict with the code i generated 3 steps ago, i'll fix it myself.
I'm not sure how well codeforces percentiles correlate to software engineering ability. Looking at all the data, it still isn't. Key notes:
1. AlphaCode 2 was already at 1650 last year.
2. SWE-bench verified under an agent has jumped from 33.2% to 35.8% under this model (which doesn't really matter). The full model is at 41.4% which still isn't a game changer either.
3. It's not handling open ended questions much better than gpt-4o.
i think you are right now actually initially i got excited but now i think OpenAI pulled the hype card again to seem relevant as they struggle to be profitable
Claude on the other hand has been fantastic and seems to do similar reasoning behind the scenes with RL
1) The "bitter lesson" may not be true, and there is a fundamental limit to transformer intelligence.
2) The "bitter lesson" is true, and there just isn't enough data/compute/energy to train AGI.
All the cognition should be happening inside the transformer. Attention is all you need. The possible cognition and reasoning occurring "inside" in high dimensions is much more advanced than any possible cognition that you output into text tokens.
This feels like a sidequest/hack on what was otherwise a promising path to AGI.
lol come on it’s not the exact same thing. At best this is like gagging yourself while you talk about it then engaging yourself when you say the answer. And that presupposing LLMs are thinking in, your words, exactly the same way as humans.
Admittedly not my most articulate, my exasperation showed through. To some extent it seems warranted as it tends to be the most effective tactic against hyperbole. Still trying to find a better solution.
> Does that mean human intelligence is cheapened when you talk out a problem to yourself?
In a sense, maybe yeah. Of course if one were to really be absolute about that statement it would be absurd, it would greatly overfit the reality.
But it is interesting to assume this statement as true. Oftentimes when we think of ideas "off the top of our heads" they are not as profound as ideas that "come to us" in the shower. The subconscious may be doing 'more' 'computation' in a sense. Lakoff said the subconscious was 98% of the brain, and that the conscious mind is the tip of the iceberg of thought.
The similarity is cosmetic only. The reason it is used is because it's easy to leverage existing work in LLMs, and scaling (although not cheap) is an obvious approach.
On the contrary, this suggests that the bitter lesson is alive and kicking. The bitter lesson doesn't say "compute is all you need", it says "only those methods which allow you to make better use of hardware as hardware itself scales are relevant".
This chain of thought / reflection method allows you to make better use of the hardware as the hardware itself scales. If a given transformer is N billion parameters, and to solve a harder problem we estimate we need 10N billion parameters, one way to do it is to build a GPU cluster 10x larger.
This method shows that there might be another way: instead train the N billion model differently so that we can use 10x of it at inference time. Say hardware gets 2x better in 2 years -- then this method will be 20x better than now!
I'd be shocked if we don't see diminishing returns in the inference compute scaling laws. We already didn't deserve how clean and predictive the pre-training scaling laws were, no way the universe grants us another boon of that magnitude
Attention is about similarity/statistical correlation which is fundamentally stochastic , while reasoning needs to be truthful and exact to be successful.
I think it's too soon to tell. Training the next generation of models means building out entire datacenters. So while they wait they have engineers build these sidequests/hacks.
Karpathy himself believes that neural networks are perfectly plausible as a key component to AGI. He has said that it doesn't need to be superseded by something better, it's just that everything else around it (especially infrastructure) needs to improve. As one of the most valuable opinions in the entire world on the subject, I tend to trust what he said.
Imagine instead the bitter lesson says: we can expand an outwards circle in many dimensions of ways to continuously mathematically manipulate data to adjust outputs.
Even the attention-token approach is on the grand scale of things a simple line outwards from the centre; we have not even explored around the centre (with the same compute spend) for things like non-token generation, different layers/different activation functions and norming / query/key/value set up (why do we only use the 3 inherent to contextualising tokens, why not add a 4th matrix for something else?), character, sentence, whole thought, paragraph one-shot generation, positional embeddings which could work differently.
The bitter lesson says there is almost a work completely untouched by our findings for us to explore. The temporary work of non-data approaches can piggy back off a point on the line; it cannot expand it like we can as we exude out from the circle..
This kind of short-sighted, simplistic reasoning / behaviour is what I worry about the most in terms of where our society is going. I always wonder - who will be the people buying or using your software (build very cheaply and efficiently with AI) once they can do the same, or get replaced by AI, or bankrupt themselves?
Everybody seems to be so focused on how to get ahead in race to profitability, that they don't consider the shortcut they are taking might be leading to a cliff.
Except that these aren't thoughts. These techniques are improvements to how the model breaks down input data, and how it evaluates its responses to arrive at a result that most closely approximates patterns it was previously rewarded for. Calling this "thinking" is anthropomorphizing what's really happening. "AI" companies love to throw these phrases around, since it obviously creates hype and pumps up their valuation.
Human thinking is much more nuanced than this mechanical process. We rely on actually understanding the meaning of what the text represents. We use deduction, intuition and reasoning that involves semantic relationships between ideas. Our understanding of the world doesn't require "reinforcement learning" and being trained on all the text that's ever been written.
Of course, this isn't to say that machine learning methods can't be useful, or that we can't keep improving them to yield better results. But these are still methods that mimic human intelligence, and I think it's disingenuous to label them as such.
Models don't really predict the next word, they predict the next token. Strawberry is made up of multiple tokens, and the model doesn't truely understand the characters in it... so it tends to struggle.
What's amazing is that given how LLMs receive input data (as tokenized streams, as other commenters have pointed out) it's remarkable that it can ever answer this question correctly.
The model performance is driven by chain of thought, but they will not be providing chain of thought responses to the user for various reasons including competitive advantage.
After the release of GPT4 it became very common to fine-tune non-OpenAI models on GPT4 output. I’d say OpenAI is rightly concerned that fine-tuning on chain of thought responses from this model would allow for quicker reproduction of their results. This forces everyone else to reproduce it the hard way. It’s sad news for open weight models but an understandable decision.
I think they mean that you won’t be able to see the “thinking”/“reasoning” part of the model’s output, even though you pay for it. If you could see that, you might be able to infer better how these models reason and replicate it as a competitor
You can see an example of the Chain of Thought in the post, it's quite extensive. Presumably they don't want to release this so that it is raw and unfiltered and can better monitor for cases of manipulation or deviation from training. What GP is also referring to is explicitly stated in the post: they also aren't release the CoT for competitive reasons, so that presumably competitors like Anthropic are unable to use the CoT to train their own frontier models.
> Presumably they don't want to release this so that it is raw and unfiltered and can better monitor for cases of manipulation or deviation from training.
My take was:
1. A genuine, un-RLHF'd "chain of thought" might contain things that shouldn't be told to the user. E.g., it might at some point think to itself, "One way to make an explosive would be to mix $X and $Y" or "It seems like they might be able to poison the person".
2. They want the "Chain of Thought" as much as possible to reflect the actual reasoning that the model is using; in part so that they can understand what the model is actually thinking. They fear that if they RLHF the chain of thought, the model will self-censor in a way which undermines their ability to see what it's really thinking
3. So, they RLHF only the final output, not the CoT, letting the CoT be as frank within itself as any human; and post-filter the CoT for the user.
This is a transcription of a literal quote from the article:
> Therefore, after weighing multiple factors including user experience, competitive advantage, and the option to pursue the chain of thought monitoring, we have decided not to show the raw chains of thought to users
The open source/weights models so far have proved that openAI doesn't have some special magic sauce. I m confident we ll soon have a model from Meta or others that s close to this level of reasoning. [Also consider that some of their top researchers have departed]
On a cursory look, it looks like the chain of thought is a long series of chains of thought balanced on each step, with a small backtracking added whenever a negative result occurs, sort of like solving a maze.
I suspect that the largest limiting factor for a competing model will be the dataset. Unless they somehow used GPT4 to generate the dataset somehow, this is an extremely novel dataset to have to build.
I’d say depends. If the model iterates 100x I’d just say give me the output.
Same with problem solving in my brain: Sure, sometimes it helps to think out loud. But taking a break and let my unconcious do the work is helpful as well. For complex problems that’s actually nice.
I think eventually we don’t care as long as it works or we can easily debug it.
It'd be helpful if they exposed a summary of the chain-of-thought response instead. That way they'd not be leaking the actual tokens, but you'd still be able to understand the outline of the process. And, hopefully, understand where it went wrong.
Given the significant chain of thought tokens being generated, it also feels a bit odd to hide it from a cost fairness perspective. How do we believe they aren't inflating it for profit?
No, its the fraud theory of charging for usage that is unaccountable that has been repeatedly proven true when unaccountable bases for charges have been deployed.
Yeah, if they are charging for some specific resource like tokens then it better be accurate. But ultimately utility-like pricing is a mistake IMO. I think they should try to align their pricing with the customer value they're creating.
AFAIK, they are the least open of the major AI labs. Meta is open-weights and partly open-source. Google DeepMind is mostly closed-weights, but has released a few open models like Gemma. Anthropic's models are fully closed, but they've released their system prompts, safety evals, and have published a fair bit of research (https://www.anthropic.com/research). Anthropic also haven't "released" anything (Sora, GPT-4o realtime) without making it available to customers. All of these groups also have free-usage tiers.
> I'd say OpenAI is rightly concerned that fine-tuning on chain of thought responses from this model would allow for quicker reproduction of their results.
Tested cipher example, and it got it right. But "thinking logs" I see in the app looks like a summary of actual chain of thought messages that are not visible.
o1 models might use multiple methods to come up with an idea, only one of them might be correct, that's what they show in ChatGPT. So it just summarises the CoT, does not include the whole reasoning behind it.
I don't understand how they square that with their pretense of being a non-profit that wants to benefit all of humanity. Do they not believe that competition is good for humanity?
Am I right that this CoT is not actual reasoning in the same way that a human would reason, but rather just a series of queries to the model that still return results based on probabilities of tokens?
It could just be programmed to follow up by querying itself with a prompt like "Come up with arguments that refute what you just wrote; if they seem compelling, try a different line of reasoning, otherwise continue with what you were doing." Different such self-administered prompts along the way could guide it through what seems like reasoning, but would really be just a facsimile thereof.
after weighing multiple factors including user experience, competitive advantage, and the option to pursue the chain of thought monitoring, we have decided not to show the raw chains of thought to users.
Saying "competitive advantage" so directly is surprising.
There must be some magic sauce here for guiding LLMs which boosts performance. They must think inspecting a reasonable number of chains would allow others to replicate it.
They call GPT 4 a model. But we don't know if it's really a system that builds in a ton of best practices and secret tactics: prompt expansion, guided CoT, etc. Dalle was transparent that it automated re-generating the prompts, adding missing details prior to generation. This and a lot more could all be running under the hood here.
Lame but not atypical of OpenAI. Too bad, but I'm expecting competitors to follow with this sort of implementation and better. Being able to view the "reasoning" process and especially being able to modify it and re-render the answer may be faster than editing your prompt a few times until you get the desired response, if you even manage to do that.
No direct indication of what “maximum test time” means, but if I’m reading the obscured language properly, the best scores on standardized tests were generated across a thousand samples with supplemental help provided.
Obviously, I hope everyone takes what any company says about the capabilities of its own software with a huge grain of salt. But it seems particularly called for here.
Honestly, it doesn't matter for the end user if there are more tokens generated between the AI reply and human message. This is like getting rid of AI wrappers for specific tasks. If the jump in accuracy is actual, then for all practical purposes, we have a sufficiently capable AI which has the potential to boost productivity at the largest scale in human history.
It starts to matter if the compute time is 10-100 fold, as the provider needs to bill for it.
Of course, that's assuming it's not priced for market acquisition funded by a huge operational deficit, which is a rarely safe to conclude with AI right now.
yeah this is kinda cool i guess but 808 elo is still pretty bad for a model that can supposedly code like a human, i mean 11th percentile is like barely scraping by, and what even is the point of simulating codeforces if youre just gonna make a model that can barely compete with a decent amateur, and btw what kind of contest allows 10 submissions, thats not how codeforces works, and what about the time limits and memory limits and all that jazz, did they even simulate those, and btw how did they even get the elo ratings, is it just some arbitrary number they pulled out of their butt, and what about the model that got 1807 elo, is that even a real model or just some cherry picked result, and btw what does it even mean to "perform better than 93% of competitors" when the competition is a bunch of humans who are all over the place in terms of skill, like what even is the baseline for comparison
edit: i got confused with the Codeforce. it is indeed zero shot and O1 is potentially something very new I hope Anthropic and others will follow suit
But what does that mean? I can't do "pip install nurture" or "pip install patience". I can generate a bunch of answers and take the consensus, but we've been able to do that for years. I can do fine-tuning or DPO, but on what?
What's the statement that OpenAI are making today which you think they're violating? There very well could be one and if there is, it would make sense to talk about it.
But arguments like "you wrote $x in a blog post when you founded your company" or "this is what the word in your name means" are infantile.
> “Therefore, after weighing multiple factors including user experience, competitive advantage, and the option to pursue the chain of thought monitoring, we have decided not to show the raw chains of thought to users.”
Trust us, we have your best intention in mind. I’m still impressed by how astonishingly impossible to like and root for OpenAI is for a company with such an innovative product.
Kinda disappointed that they're hiding the thought process. Hopefully the open source community will figure out how to effectively match and replicate what OpenAI is doing.
I wonder how far we are from having a model that can correctly solve a word soup search problem directly from just a prompt and input image. It seems like the crossword example is close. For a word search it would require turning the image into an internal grid representation, prepare the list of words, and do a search. I'd be interested in seeing if this model can already solve the word grid search problem if you give it the correct representation as an input.
> Hopefully the open source community will figure out how to effectively match and replicate what OpenAI is doing.
No need for that, there is a Reflection 70B model that does the exact same thing - with chains of thought being separated from the "final answer" via custom 'tag' tokens.
Some reported benchmarks do seem to be rather dubious, but the 70B model itself is quite real. Sample output:
$ ollama run reflection:70b-q4_0
>>> hello
<thinking>
To respond to "hello", I'll need to consider several factors:
1. The user's intent: They're likely saying hello as a greeting.
2. Appropriate responses: Common ways to respond to "hello" are:
- Hello back
- Hi
- Hey
3. Formality level: Since this is an AI response, I'll aim for a friendly but professional tone.
<reflection>
The approach of responding with a standard greeting seems appropriate in this context. It acknowledges the user's hello and provides a polite response.
</reflection>
Given these considerations, the most suitable response would be to echo "hello" back to the user.
</thinking>
<output>
Hello!
</output>
There was a hackernews post a few days ago, pointing to a reddit thread where some guys proved that the founder/s of relection AI were faking their model by just passing the input to Claude (Sonnet 3.5) and stripping the word "Claude" from the output, amongst other things. Then when they got caught they switched it to GPT 4-o.
After this, I will be very skeptical to anything they claim to achieve.
I have access to the model via the web client and it does show the thought process along the way. It shows a little icon that says things like "Examining parser logic", "Understanding data structures"...
However, once the answer is complete the chain of thought is lost
It can't do large portions of the parts of an IQ test (not multi-modal). Otherwise I think it's essentially superhuman, modulo tokenization issues (please start running byte-by-byte or at least come up with a better tokenizer).
> We have found that the performance of o1 consistently improves with more reinforcement learning (train-time compute) and with more time spent thinking (test-time compute).
Wow. So we can expect scaling to continue after all. Hyperscalers feeling pretty good about their big bets right now. Jensen is smiling.
This is the most important thing. Performance today matters less than the scaling laws. I think everyone has been waiting for the next release just trying to figure out what the future will look like. This is good evidence that we are on the path to AGI.
It'd be interesting for sure if true. Gotta remember that this is a marketing post though, let's wait a few months and see if its actually true. Things are definitely interesting, wherever these techniques will get us AGI or not
Even when we start to plateau on direct LLM performance, we can still get significant jumps by stacking LLMs together or putting a cluster of them together.
Microsoft, Google, Facebook have all said in recent weeks that they fully expect their AI datacenter spend to accelerate. They are effectively all-in on AI. Demand for nvidia chips is effectively infinite.
> I really hope people understand that this is a new paradigm: don't expect the same pace, schedule, or dynamics of pre-training era.
I believe the rate of improvement on evals with our reasoning models has been the fastest in OpenAI history.
I LOVE the long list of contributions. It looks like the credits from a Christoper Nolan film. So many people involved. Nice care to create a nice looking credits page. A practice worth copying.
A lot of skepticism here, but these are astonishing results! People should realize we’re reaching the point where LLMs are surpassing humans in any task limited in scope enough to be a “benchmark”. And as anyone who’s spent time using Claude 3.5 Sonnet / GPT-4o can attest, these things really are useful and smart! (And, if these results hold up, O1 is much, much smarter.) This is a nerve-wracking time to be a knowledge worker for sure.
I have been deploying useful code from LLMs right and left over the last several months. They are a significant force accelerator for programmers if you know how to prompt them well.
We’ll see if this is a good idea when we start having millions of lines of LLM-written legacy code. My experience maintaining such code so far has been very bad: accidentally quadratic algorithms; subtly wrong code that looks right; and un-idiomatic use of programming language features.
ah i see so you're saying that LLM-written code is already showing signs of being a maintenance nightmare, and that's a reason to be skeptical about its adoption. But isn't that just a classic case of 'we've always done it this way' thinking?
legacy code is a problem regardless of who wrote it. Humans have been writing suboptimal, hard-to-maintain code for decades. At least with LLMs, we have the opportunity to design and implement better coding standards and review processes from the start.
let's be real, most of the code written by humans is not exactly a paragon of elegance and maintainability either. I've seen my fair share of 'accidentally quadratic algorithms' and 'subtly wrong code that looks right' written by humans. At least with LLMs, we can identify and address these issues more systematically.
As for 'un-idiomatic use of programming language features', isn't that just a matter of training the LLM on a more diverse set of coding styles and idioms? It's not like humans have a monopoly on good coding practices.
So, instead of throwing up our hands, why not try to address these issues head-on and see if we can create a better future for software development?
Maybe it will work out, but I think we’ll regret this experiment because it’s the wrong sort of “force accelerator”: writing tons of code that should be abstracted rather than just dumped out literally has always caused the worst messes I’ve seen.
Yes, same way that the image model outputs have already permeated the blogosphere and pushed out some artists, the other models will all bury us under a pile of auto-generated code.
We will yearn for the pre-GPT years at some point, like we yearn for the internet of the late 90s/early 2000s. Not for a while though. We're going through the early phases of GPT today, so it hasn't been taken over by the traditional power players yet.
Honestly the code it's been giving me has been fairly cromulent. I don't believe in premature optimization and it is perfect for getting features out quick and then I mold it to what it needs to be.
same...but have you considered the broader implications of relying on LLMs to generate code? It's not just about being a 'force accelerator' for individual programmers, but also about the potential impact on the industry as a whole.
If LLMs can generate high-quality code with minimal human input, what does that mean for the wages and job security of programmers? Will companies start to rely more heavily on AI-generated code, and less on human developers? It's not hard to imagine a future where LLMs are used to drive down programming costs, and human developers are relegated to maintenance and debugging work.
I'm not saying that's necessarily a bad thing, but it's definitely something that needs to be considered. As someone who's enthusiastic about the potential of code gen this O1 reasoning capability is going to make big changes.
do you think you'll be willing to take a pay cut when your employer realizes they can get similar results from a machine in a few seconds?
In a way it's not surprising that people are getting vastly different results out of LLMs. People have different skill levels when it comes to using even Google. An LLM has a vastly bigger input space.
What's a sample prompt that you've used? Every time I've tried to use one for programming, they invent APIs that don't exist (but sound like they might) or fail to produce something that does what it says it does.
Even then it is hit and miss. If you are doing something that is also copy/paste-able out of a StackOverflow comment, you're apt to be fine, but as soon as you are doing anything slightly less common... Good luck.
True. It can be good at giving you pointers towards approaching the problem, even if the result is flawed, for slightly less common problems. But as you slide even father towards esotericism, there is no hope. It won't even get you in the right direction. Unfortunately – as that is where it would be most useful.
No matter the prompt, there's a significant difference between how it handles common problems in popular languages (python, JS) versus esoteric algorithms in niche languages or tools.
I had a funny one a while back (granted this was probably ChatGPT 3.5) where I was trying to figure out what payload would get AWS CloudFormation to fix an authentication problem between 2 services and ChatGPT confidently proposed adding some OAuth querystring parameters to the AWS API endpoint.
I just ask it for what I want in very specific detail, stating the language and frameworks in use. I keep the ideas self-contained -- for example if I need something for the frontend I will ask it to make me a webcomponent. Asking it to not make assumptions and ask questions on ambiguities is also very helpful.
It tends to fall apart on bigger asks with larger context. Breaking your task into discrete subtasks works well.
I think that's just the same as using an autocomplete efficiently, though. I tend to like them for Search, but not for anything i have to "prompt correctly" because i feel like i can type fast enough that i'm not too worried about auto-completing.
With that said i'm not one of those "It's just a parrot!" people. It is, definitely just a parrot atm.. however i'm not convinced we're not parrots as well. Notably i'm not convinced that that complexity won't be sufficient to walk talk and act like intelligence. I'm not convinced that intelligence is different than complexity. I'm not an expert though, so this is just some dudes stupid opinion.
I suspect if LLMs can prove to have duck-intelligence (ie duck typing but for intelligence) then it'll only be achieved in volumes much larger than we imagine. We'll continue to refine and reduce how much volume is necessary, but nevertheless i expect complexity to be the real barrier.
'Not useful' is a pretty low bar to clear, especially when you consider the state of the art just 5 years ago. LLMs may not be solving world hunger, but they're already being used in production for coding
If you're not seeing value in them, maybe it's because you're not looking at the right problems. Or maybe you're just not using them correctly. Either way, dismissing an entire field of research because it doesn't fit your narrow use case is pretty short-sighted.
FWIW, I've been using LLMs to generate production code and it's saved me weeks if not months. YMMV, I guess
It’s definitely the case that there are some programming workflows where LLMs aren’t useful. But I can say with certainty that there are many where they have become incredibly useful recently. The difference between even GPT-4 last year and C3.5/GPT-4o this year is profound.
I recently wrote a complex web frontend for a tool I’ve been building with Cursor/Claude and I wrote maybe 10% of the code; the rest with broad instructions. Had I done it all myself (or even with GitHub Copilot only) it would have taken 5 times longer. You can say this isn’t the most complex task on the planet, but it’s real work, and it matters a lot! So for increasingly many, regardless of your personal experience, these things have gone far beyond “useful toy”.
I don’t want to be in the business of LLM defender, but it’s just hard to imagine this aging well when you step back and look at the pace of advancement here. In the realm of “real math and science”, O1 has improved from 0% to 50% on AIME today. A year ago, LLMs could only write little functions, not much better than searching StackOverflow. Today, they can write thousands of lines of code that work together with minimal supervision.
I’m sure this tech continues to have many limitations, but every piece of trajectory evidence we have points in the same direction. I just think you should be prepared for the ratio of “real” work vs. LLM-capable work to become increasingly small.
I can probably climb a tree faster than I can build a rocket. But only one will get me all the way to the moon. Don't confuse local optima for global ones.
> The sooner those paths are closed for low-effort high-pay jobs, the better, IMO. All this money for no work is going to our heads.
> It's time to learn some real math and science, the era of regurgitating UI templates is over.
You do realize that software development was one of the last social elevators, right?
What you're asking for won't happen, let alone the fact that "real math and science" pay a pittance, there's a reason the pauper mathematician was a common meme.
At this point, you're either saying "I don't understand how to prompt them" or "I'm a Luddite". They are useful, here to stay, and only getting better.
Familiarize yourself with a tool which does half the prompting for you, e.g. cursor is pretty good at prompting claude 3.5 and it really does make code edits 10x faster (I'm not even talking about the fancy stuff about generating apps in 5 mins - just plain old edits.)
Is it? They talk about 10k attempts to reach gold medal status in the mathematics olympiad, but zero shot performance doesn't even place it in the upper 50th percentile.
Maybe I'm confused but 10k attempts on the same problem set would make anyone an expert in that topic? It's also weird that zero shot performance is so bad, but over a lot of attempts it seems to get correct answers? Or is it learning from previous attempts? No info given.
The correct metaphor is that 10,000 attempts would allow anyone to cherry pick a successful attempt. You’re conflating cherry picking with online learning. This is like if an entire school of students randomized their answers on a multiple choice test, and then you point to someone who scored 100% and claim it is proof of the school’s expertise.
Yeah but how is it possible that it has such a high margin of error? 10k attempts is insane! Were talking about an error margin of 50%! How can you deliver "expert reasoning" with such an error margin?
It’s undeniably less impressive than a human on the same task, but who cares at the end of the day? It can do 10,000 attempts in the time a person can do 1. Obviously improving that ratio will help for any number of reasons, but if you have a computer that can do a task in 5 minutes that will take a human 3 hours, it doesn’t necessarily matter very much how you got there.
How long does it take the operator to sift through those 10,000 attempts to find the successful one, when it's not a contrived benchmark where the desired answer is already known ahead of time? LLMs generally don't know when they've failed, they just barrel forwards and leave the user to filter out the junk responses.
Even if it's the other way around, if the computer takes 3 hours on a task that a human can do in 5 minutes, using the computer might still be a good idea.
A computer will never go on strike, demand better working conditions, unionize, secretly be in cahoots with your competitor or foreign adversary, play office politics, scroll through Tiktok instead of doing its job, or cause an embarrassment to your company by posting a politically incorrect meme on its personal social media account.
The blog says "With a relaxed submission constraint, we found that model performance improved significantly. When allowed 10,000 submissions per problem, the model achieved a score of 362.14 – above the gold medal threshold – even without any test-time selection strategy."
I am interpreting this to mean that the model tried 10K approaches to solve the problem, and finally selected the one that did the trick. Am I wrong?
Even if you disregard the Olympiad performance OpenAI-O1 is, if the charts are to be believed, a leap forward in intelligence. Also bear in mind that AI researchers are not out of ideas on how to make models better and improvements in AI chips are the metaphorical tide that lifts all boats. The trend is the biggest story here.
I get the AI skepticism because so much tech hype of recent years turned out to be hot air (if you're generous, obvious fraud if you're not). But AI tools available toady, once you get the hang of using them, are pretty damn amazing already. Many jobs can be fully automated with AI tools that exist today. No further breakthroughs required. And although I still don't believe software engineers will find themselves out of work anytime soon, I can no longer completely rule it out either.
Even without AI, it's gotten ~10,000 times easier to write software than in the 1950s (eg. imagine trying to write PyTorch code by hand in IBM 650 assembly), but the demand for software engineering has only increased, because demand increases even faster than supply does. Jevons paradox:
I'm skeptical because "we fired half our programmers and our new AI does their jobs as well as they did" is a story that would tear through the Silicon Valley rumor mill. To my knowledge, this has not happened (yet).
The local decline in open software engineering positions has _nothing_ to do with AI. The best orgs are using AI to assist developers in building out new systems and write tests. Show me someone who is doing anything bigger than that, please I'd love to be proven wrong.
The big decline is driven by a few big factors. Two of which are 1- the overhiring that happened in 2021. This was followed by the increase of interest rates which dramatically constrained the money supply. Investors stopped preferring growth over profits. This shift in investor preferences is reflected in engineering orgs tightening their budgets as they are no longer rewarded for unbridled growth.
> it's gotten ~10,000 times easier to write software than in the 1950s
It seems many of the popular tools want to make writing software harder than in the 2010s, though. Perhaps their stewards believe that if they keep making things more and more unnecessarily complicated, LLMs won't be able to keep up?
> People should realize we’re reaching the point where LLMs are surpassing humans in any task limited in scope enough to be a “benchmark”.
Can you explain what this statement means? It sounds like you're saying LLMs are now smart enough to be able to jump through arbitrary hoops but are not able to do so when taken outside of that comfort zone. If my reading is correct then it sounds like skepticism is still warranted? I'm not trying to be an asshole here, it's just that my #1 problem with anything AI is being able to separate fact from hype.
I think what I’m saying is a bit more nuanced than that. LLMs currently struggle with very “wide”, long-run reasoning tasks (e.g., the evolution over time of a million-line codebase). That isn’t because they are secretly stupid and their capabilities are all hype, it’s just that this technology currently has a different balance of strengths and weaknesses than human intelligence, which tends to more smoothly extrapolate to longer-horizon tasks.
We are seeing steady improvement on long-run tasks (SWE-Bench being one example) and much more improvement on shorter, more well-defined tasks. The latter capabilities aren’t “hype” or just for show, there really is productive work like that to be done in the world! It’s just not everything, yet.
I have written a ton of evaluations and run countless benchmarks and I'm not even close to convinced that we're at
> the point where LLMs are surpassing humans in any task limited in scope enough to be a “benchmark”
so much as we're over-fitting these bench marks (and in many cases fishing for a particular way of measuring the results that looks more impressive).
While it's great that the LLM community has so many benchmarks and cares about attempting to measure performance, these benchmarks are becoming an increasingly poor signal.
> This is a nerve-wracking time to be a knowledge worker for sure.
It might because I'm in this space, but I personally feel like this is the best time to working in tech. LLMs still are awful at things requiring true expertise while increasingly replacing the need for mediocre programmers and dilettantes. I'm increasingly seeing the quality of the technical people I'm working with going up. After years of being stuck in rooms with leetcode grinding TC chasers, it's very refreshing.
I like your phrasing - "any task limited in scope enough to be a 'benchmark'". Exactly! This is the real gap with LLMs, and will continue to be an issue with o1 -- sure, if you can write down all of the relevant context information you need to perform some computation, LLMs should be able to do it. In other words, LLMs are calculators!
I'm not especially nerve-wracked about being a knowledge worker, because my day-to-day doesn't consist of being handed a detailed specification of exactly what is required, and then me 'computing' it. Although this does sound a lot like what a product manager does!
> And as anyone who’s spent time using Claude 3.5 Sonnet / GPT-4o can attest, these things really are useful and smart! (And, if these results hold up, O1 is much, much smarter.) This is a nerve-wracking time to be a knowledge worker for sure.
If you have to keep checking the result of an LLM, you do not trust it enough to give you the correct answer.
Thus, having to 'prompt' hundreds of times for the answer you believe is correct over something that claims to be smart - which is why it can confidently convince others that its answer is correct (even when it can be totally erroneous).
I bet if Google DeepMind announced the exact same product, you would equally be as skeptical with its cherry-picked results.
> And as anyone who’s spent time using Claude 3.5 Sonnet / GPT-4o can attest, these things really are useful and smart!
I have spent significant time with GPT-4o, and I disagree. LLMs are as useful as a random forum dweller who recognises your question as something they read somewhere at some point but are too lazy to check so they just say the first thing which comes to mind.
Here’s a recent example I shared before: I asked GPT-4o which Monty Python members have been knighted (not a trick question, I wanted to know). It answered Michael Palin and Terry Gilliam, and that they had been knighted for X, Y, and Z (I don’t recall the exact reasons). Then I verified the answer on the BBC, Wikipedia, and a few others, and determined only Michael Palin has been knighted, and those weren’t even the reasons.
Just for kicks, I then said I didn’t think Michael Palin had been knighted. It promptly apologised, told me I was right, and that only Terry Gilliam had been knighted. Worse than useless.
Coding-wise, it’s been hit or miss with way more misses. It can be half-right if you ask it uninteresting boilerplate crap everyone has done hundreds of times, but for anything even remotely interesting it falls flatter than a pancake under a steam roller.
I asked GPT-4o and I got the correct answer in one shot:
> Only one Monty Python member, Michael Palin, has been knighted. He was honored in 2019 for his contributions to travel, culture, and geography. His extensive work as a travel documentarian, including notable series on the BBC, earned him recognition beyond his comedic career with Monty Python (NERDBOT) (Wikipedia).
> Other members, such as John Cleese, declined honors, including a CBE (Commander of the British Empire) in 1996 and a peerage later on (8days).
Maybe you just asked the question wrong. My prompt was "which monty python actors have been knighted. look it up and give the reasons why. be brief".
Yes yes, there’s always some “you're holding it wrong” apologist.¹ Look, it’s not a complicated question to ask unambiguously. If you understand even a tiny bit of how these models work, you know you can make the exact same question twice in a row and get wildly different answers.
The point is that you never know what you can trust or not. Unless you’re intimately familiar with Monty Python history, you only know you got the correct answer in one shot because I already told you what the right answer is.
Oh, and by the way, I just asked GPT-4o the same question, with your phrasing, copied verbatim and it said two Pythons were knighted: Michael Palin (with the correct reasons this time) and John Cleese.
¹ And I’ve had enough discussions on HN where someone insists on the correct way to prompt, then they do it and get wrong answers. Which they don’t realise until they shared it and disproven their own argument.
Unless I'm mistaken, isn't all the math behind them... ultimately probabilistic? Even theoretically they can't guarantee the same answer. I'm agreeing with you, by the way, just curious if I'm missing something.
If you take a photo the photons hitting the camera sensor do so in a probabilistic fashion. Still, in sufficient light you'll get the same picture every time you press the shutter button. In near darkness you'll get a random noise picture every time.
Similarly language models are probabilistic and yet they get the easiest questions right 100% of the time with little variability and the hardest prompts will return gibberish. The point of good prompting is to get useful responses to questions at the boundary of what the language model is capable of.
(You can also configure a language model to generate the same output for every prompt without any random noise. Image models for instance generate exactly the same image pixel for pixel when given the same seed.)
The photo comparison is disingenuous. Light and colour information can be disorganised to a large extent and yet you still perceive the same from an image. You can grab a photo and apply to it a red filter or make it black and white and still understand what’s in there, what it means, and how it compares to reality.
In comparison, with text a single word can change the entire meaning of a sentence, paragraph, or idea. The same word in different parts of a text can make all the difference between clarity and ambiguity.
It makes no difference how good your prompting is, some things are simply unknowable by an LLM. I repeatedly asked GPT-4o how many Magic: The Gathering cards based on Monty Python exist. It said there are none (wrong) because they didn’t exist yet at the cut off date of its training. No amount of prompting changes that, unless you steer it by giving it the answer (at which point there would have been no point in asking).
Furthermore, there’s no seed that guarantees truth in all answers or the best images in all cases. Seeds matter for reproducibility, they are unrelated to accuracy.
Language is fuzzy in exactly the same way. LLMs can create factually correct responses in dozens of languages using endless variations in phrasing. You fixate on the kind of questions that current language models struggle with but you forget that for millions of easier questions modern language models already respond with a perfect answer every time.
You think the probabilistic nature of language models is a fundamental problem that puts a ceiling on how smart they can become, but you're wrong.
No. Language can be fuzzy, yes, but not at all in the same way. I have just explained that.
> LLMs can create factually correct responses in dozens of languages using endless variations in phrasing.
So which is it? Is it about good prompting, or can you have endless variations? You can’t have of both ways.
> You fixate on the kind of questions that current language models struggle with
So you’re saying LLMs struggle with simple factual and verifiable questions? Because that’s all the example questions were. If they can’t handle that (and they do it poorly, I agree), what’s the point?
By the way, that’s a single example. I have many more and you can find plenty of others online. Do you also think the Gemini ridiculous answers like putting glue on pizza are about bad promoting?
> You think the probabilistic nature of language models is a fundamental problem that puts a ceiling on how smart they can become, but you're wrong.
One of your mistakes is thinking you know what I think. You’re engaging with a preconceived notion you formed in your head instead of the argument.
And LLMs aren’t smart, because they don’t think. They are an impressive trick for sure, but that does not imply cleverness on their part.
I think your iPhone analogy is apt. Do you want to be the person complaining that the phone drops calls or do you want to hold it slightly differently and get a lot of use out of it?
If you pay careful attention to prompt phrasing you will get a lot more mileage out of these models. That's the bottom line. If you believe that you shouldn't have to learn how to use a tool well then you can be satisfied with your righteous attitude but you won't get anywhere.
No one’s arguing that correct use of a tool isn’t beneficial. The point is that insisting LLMs just need good prompting is delusional and a denial of reality. I have just demonstrated how your own prompt is still capable of producing the wrong result. So either you don’t know how to prompt correctly (because if you did, by your own logic it would have produced the right response every time, which it didn’t) or the notion that all you need is good prompting is wrong. Which anyone who understands the first thing about these systems knows to be the case.
> ChatGPT Plus and Team users will be able to access o1 models in ChatGPT starting today. Both o1-preview and o1-mini can be selected manually in the model picker, and at launch, weekly rate limits will be 30 messages for o1-preview and 50 for o1-mini. We are working to increase those rates and enable ChatGPT to automatically choose the right model for a given prompt.
Weekly? Holy crap, how expensive is it to run is this model?
It's probably running several lines of COT. I imagine, each single message you send is probably at __least__ 10x to the actual model. So in reality it's like 300 messages, and honestly it's probably 100x, given how constrained they're being with usage.
The human brain uses 20 watts, so yeah we figured out a way to run better than human brain computation by using many orders of magnitude more power. At some point we'll need to reject exponential power usage for more computation. This is one of those interesting civilizational level problems. There's still a lack of recognition that we aren't going to be able to compute all we want to, like we did in the pre-LLM days.
For 20 watts of work on stuff like this for about 4 hours a day counting vacations and weekends and attention span. So 20 hours of rest, relaxation, distraction, household errands and stuff, so that maybe bumps it up to 120 watts per work hour. Then 22.5 years of training or so per worker, 45 year work period, 22.5 year retirement. So double it there to 240 watts. We can't run brains without bodies, so multiply that by 6 giving 1440 watts + the air conditioning, commuting to school and work, etc., maybe 2000 watts?
We're getting close to parity if things keep getting more efficient as fast as they have been. But that's without accounting for the AI training, which can on the plus side be shared among multiple agents, but on the down side can't really do continuous learning very well without catastrophic forgetting.
> Therefore, after weighing multiple factors including user experience, competitive advantage, and the option to pursue the chain of thought monitoring, we have decided not to show the raw chains of thought to users.
What? I agree people who typically use the free ChatGPT webapp won't care about raw chain-of-thoughts, but OpenAI is opening an API endpoint for the O1 model and downstream developers very very much care about chain-of-thoughts/the entire pipeline for debugging and refinement.
I suspect "competitive advantage" is the primary driver here, but that just gives competitors like Anthropic an oppertunity.
They they've taken at least some of the hobbles off for the chain of thought, so the chain of thought will also include stuff like "I shouldn't say <forbidden thing they don't want it to say>".
1,249 comments
[ 3.0 ms ] story [ 559 ms ] threadWell, the model doesn't start with "GPT", so maybe they have come up with something better.
If its actually true in practice, I sincerely cannot imagine a scenario where it would be cheaper to hire actual junior or mid-tier developers (keyword: "developers", not architects or engineers).
1,673 ELO should be able to build very complex, scalable apps with some guidance
with O1 being in the 89th percentile would mean it should be able to think at junior to intermediate level with very strong consistency.
i dont think people in the comments realize the implication of this. previously LLMs were able to only "pattern match" but now its able to evaluate itself (with some guidance ofc) essentially, steering the software into depth of edge cases and reason about it in a way that feels natural to us.
currently I'm copying and pasting stuff and notifying LLM the results but once O1 is available its going to significantly lower that frequency.
For example, I expect it to self evaluate the code its generate and think at higher levels.
ex) oooh looks like this user shouldn't be able to escalate privileges in this case because it would lead to security issues or it could conflict with the code i generated 3 steps ago, i'll fix it myself.
1. AlphaCode 2 was already at 1650 last year.
2. SWE-bench verified under an agent has jumped from 33.2% to 35.8% under this model (which doesn't really matter). The full model is at 41.4% which still isn't a game changer either.
3. It's not handling open ended questions much better than gpt-4o.
Claude on the other hand has been fantastic and seems to do similar reasoning behind the scenes with RL
Idk if I'm "feeling the AGI" if I'm being honest.
Also... telling that they choose to benchmark against CodeForces rather than SWE-bench.
1) The "bitter lesson" may not be true, and there is a fundamental limit to transformer intelligence.
2) The "bitter lesson" is true, and there just isn't enough data/compute/energy to train AGI.
All the cognition should be happening inside the transformer. Attention is all you need. The possible cognition and reasoning occurring "inside" in high dimensions is much more advanced than any possible cognition that you output into text tokens.
This feels like a sidequest/hack on what was otherwise a promising path to AGI.
It's the exact same thing here.
At best it maybe vaguely resembles thinking
I've never found this sort of argument convincing. it's very Chalmers.
In a sense, maybe yeah. Of course if one were to really be absolute about that statement it would be absurd, it would greatly overfit the reality.
But it is interesting to assume this statement as true. Oftentimes when we think of ideas "off the top of our heads" they are not as profound as ideas that "come to us" in the shower. The subconscious may be doing 'more' 'computation' in a sense. Lakoff said the subconscious was 98% of the brain, and that the conscious mind is the tip of the iceberg of thought.
This chain of thought / reflection method allows you to make better use of the hardware as the hardware itself scales. If a given transformer is N billion parameters, and to solve a harder problem we estimate we need 10N billion parameters, one way to do it is to build a GPU cluster 10x larger.
This method shows that there might be another way: instead train the N billion model differently so that we can use 10x of it at inference time. Say hardware gets 2x better in 2 years -- then this method will be 20x better than now!
source: https://youtu.be/hM_h0UA7upI?t=973
Even the attention-token approach is on the grand scale of things a simple line outwards from the centre; we have not even explored around the centre (with the same compute spend) for things like non-token generation, different layers/different activation functions and norming / query/key/value set up (why do we only use the 3 inherent to contextualising tokens, why not add a 4th matrix for something else?), character, sentence, whole thought, paragraph one-shot generation, positional embeddings which could work differently.
The bitter lesson says there is almost a work completely untouched by our findings for us to explore. The temporary work of non-data approaches can piggy back off a point on the line; it cannot expand it like we can as we exude out from the circle..
Everybody seems to be so focused on how to get ahead in race to profitability, that they don't consider the shortcut they are taking might be leading to a cliff.
Human thinking is much more nuanced than this mechanical process. We rely on actually understanding the meaning of what the text represents. We use deduction, intuition and reasoning that involves semantic relationships between ideas. Our understanding of the world doesn't require "reinforcement learning" and being trained on all the text that's ever been written.
Of course, this isn't to say that machine learning methods can't be useful, or that we can't keep improving them to yield better results. But these are still methods that mimic human intelligence, and I think it's disingenuous to label them as such.
Same as Bitcoin being the right combination of things that already existed.
They also worked with Devin to benchmark it on Devin's internal benchmarks, where it's twice as good as GPT-4o: https://x.com/cognition_labs/status/1834292718174077014 https://www.cognition.ai/blog/evaluating-coding-agents
https://community.openai.com/t/incorrect-count-of-r-characte...
After the release of GPT4 it became very common to fine-tune non-OpenAI models on GPT4 output. I’d say OpenAI is rightly concerned that fine-tuning on chain of thought responses from this model would allow for quicker reproduction of their results. This forces everyone else to reproduce it the hard way. It’s sad news for open weight models but an understandable decision.
My take was:
1. A genuine, un-RLHF'd "chain of thought" might contain things that shouldn't be told to the user. E.g., it might at some point think to itself, "One way to make an explosive would be to mix $X and $Y" or "It seems like they might be able to poison the person".
2. They want the "Chain of Thought" as much as possible to reflect the actual reasoning that the model is using; in part so that they can understand what the model is actually thinking. They fear that if they RLHF the chain of thought, the model will self-censor in a way which undermines their ability to see what it's really thinking
3. So, they RLHF only the final output, not the CoT, letting the CoT be as frank within itself as any human; and post-filter the CoT for the user.
> Therefore, after weighing multiple factors including user experience, competitive advantage, and the option to pursue the chain of thought monitoring, we have decided not to show the raw chains of thought to users
On a cursory look, it looks like the chain of thought is a long series of chains of thought balanced on each step, with a small backtracking added whenever a negative result occurs, sort of like solving a maze.
https://old.reddit.com/r/LocalLLaMA/comments/1fc98fu/confirm...
Same with problem solving in my brain: Sure, sometimes it helps to think out loud. But taking a break and let my unconcious do the work is helpful as well. For complex problems that’s actually nice.
I think eventually we don’t care as long as it works or we can easily debug it.
Also, what is going to be their excuse to defend themselves against copyright lawsuits if they are going to "understandably" keep their models closed?
how can you access it without an account?
I don't agree with this, but it definitely carries higher weight in their decision making than leaking relevant training info to other models.
Why? They're called "Open" AI after all ...
There must be some magic sauce here for guiding LLMs which boosts performance. They must think inspecting a reasonable number of chains would allow others to replicate it.
They call GPT 4 a model. But we don't know if it's really a system that builds in a ton of best practices and secret tactics: prompt expansion, guided CoT, etc. Dalle was transparent that it automated re-generating the prompts, adding missing details prior to generation. This and a lot more could all be running under the hood here.
Obviously, I hope everyone takes what any company says about the capabilities of its own software with a huge grain of salt. But it seems particularly called for here.
Of course, that's assuming it's not priced for market acquisition funded by a huge operational deficit, which is a rarely safe to conclude with AI right now.
edit: i got confused with the Codeforce. it is indeed zero shot and O1 is potentially something very new I hope Anthropic and others will follow suit
any type of reasoning capability i'll take it !
I would suggest re-reading more carefully
honestly im spooked
at least NVDA should benefit. i guess.
go play more, your priorities and focus on it being work are making you think this to be harder than it is, and the models can even tell you this.
you don’t have to like the answer, but take it seriously, and you might come back and like it quite a bit.
you have to have patience because you likely wont have scale - but it is not just patience with the response time.
But arguments like "you wrote $x in a blog post when you founded your company" or "this is what the word in your name means" are infantile.
Trust us, we have your best intention in mind. I’m still impressed by how astonishingly impossible to like and root for OpenAI is for a company with such an innovative product.
I wonder how far we are from having a model that can correctly solve a word soup search problem directly from just a prompt and input image. It seems like the crossword example is close. For a word search it would require turning the image into an internal grid representation, prepare the list of words, and do a search. I'd be interested in seeing if this model can already solve the word grid search problem if you give it the correct representation as an input.
No need for that, there is a Reflection 70B model that does the exact same thing - with chains of thought being separated from the "final answer" via custom 'tag' tokens.
After this, I will be very skeptical to anything they claim to achieve.
https://news.ycombinator.com/item?id=41484981
However, once the answer is complete the chain of thought is lost
Where it says "Thought for 20 seconds" - you can click the Chevron to expand it and see what I guess is the entire chain of thought.
Awesome!
Probably 1% of users will get access to it, with a 20/message a day rate limit. Until early next year.
Wow. So we can expect scaling to continue after all. Hyperscalers feeling pretty good about their big bets right now. Jensen is smiling.
This is the most important thing. Performance today matters less than the scaling laws. I think everyone has been waiting for the next release just trying to figure out what the future will look like. This is good evidence that we are on the path to AGI.
> I really hope people understand that this is a new paradigm: don't expect the same pace, schedule, or dynamics of pre-training era. I believe the rate of improvement on evals with our reasoning models has been the fastest in OpenAI history.
> It's going to be a wild year.
https://x.com/willdepue/status/1834294935497179633
https://openai.com/openai-o1-contributions/
legacy code is a problem regardless of who wrote it. Humans have been writing suboptimal, hard-to-maintain code for decades. At least with LLMs, we have the opportunity to design and implement better coding standards and review processes from the start.
let's be real, most of the code written by humans is not exactly a paragon of elegance and maintainability either. I've seen my fair share of 'accidentally quadratic algorithms' and 'subtly wrong code that looks right' written by humans. At least with LLMs, we can identify and address these issues more systematically.
As for 'un-idiomatic use of programming language features', isn't that just a matter of training the LLM on a more diverse set of coding styles and idioms? It's not like humans have a monopoly on good coding practices.
So, instead of throwing up our hands, why not try to address these issues head-on and see if we can create a better future for software development?
We will yearn for the pre-GPT years at some point, like we yearn for the internet of the late 90s/early 2000s. Not for a while though. We're going through the early phases of GPT today, so it hasn't been taken over by the traditional power players yet.
Which is how we ended up here, which I guess is tolerable, where a webpage with a bit of styling and a table uses up 200MB of RAM.
If LLMs can generate high-quality code with minimal human input, what does that mean for the wages and job security of programmers? Will companies start to rely more heavily on AI-generated code, and less on human developers? It's not hard to imagine a future where LLMs are used to drive down programming costs, and human developers are relegated to maintenance and debugging work.
I'm not saying that's necessarily a bad thing, but it's definitely something that needs to be considered. As someone who's enthusiastic about the potential of code gen this O1 reasoning capability is going to make big changes.
do you think you'll be willing to take a pay cut when your employer realizes they can get similar results from a machine in a few seconds?
It's like infinitely tailored blog posts, for me at least.
I had a funny one a while back (granted this was probably ChatGPT 3.5) where I was trying to figure out what payload would get AWS CloudFormation to fix an authentication problem between 2 services and ChatGPT confidently proposed adding some OAuth querystring parameters to the AWS API endpoint.
It tends to fall apart on bigger asks with larger context. Breaking your task into discrete subtasks works well.
With that said i'm not one of those "It's just a parrot!" people. It is, definitely just a parrot atm.. however i'm not convinced we're not parrots as well. Notably i'm not convinced that that complexity won't be sufficient to walk talk and act like intelligence. I'm not convinced that intelligence is different than complexity. I'm not an expert though, so this is just some dudes stupid opinion.
I suspect if LLMs can prove to have duck-intelligence (ie duck typing but for intelligence) then it'll only be achieved in volumes much larger than we imagine. We'll continue to refine and reduce how much volume is necessary, but nevertheless i expect complexity to be the real barrier.
If you're not seeing value in them, maybe it's because you're not looking at the right problems. Or maybe you're just not using them correctly. Either way, dismissing an entire field of research because it doesn't fit your narrow use case is pretty short-sighted.
FWIW, I've been using LLMs to generate production code and it's saved me weeks if not months. YMMV, I guess
I recently wrote a complex web frontend for a tool I’ve been building with Cursor/Claude and I wrote maybe 10% of the code; the rest with broad instructions. Had I done it all myself (or even with GitHub Copilot only) it would have taken 5 times longer. You can say this isn’t the most complex task on the planet, but it’s real work, and it matters a lot! So for increasingly many, regardless of your personal experience, these things have gone far beyond “useful toy”.
It's time to learn some real math and science, the era of regurgitating UI templates is over.
I’m sure this tech continues to have many limitations, but every piece of trajectory evidence we have points in the same direction. I just think you should be prepared for the ratio of “real” work vs. LLM-capable work to become increasingly small.
> It's time to learn some real math and science, the era of regurgitating UI templates is over.
You do realize that software development was one of the last social elevators, right?
What you're asking for won't happen, let alone the fact that "real math and science" pay a pittance, there's a reason the pauper mathematician was a common meme.
Maybe I'm confused but 10k attempts on the same problem set would make anyone an expert in that topic? It's also weird that zero shot performance is so bad, but over a lot of attempts it seems to get correct answers? Or is it learning from previous attempts? No info given.
A computer will never go on strike, demand better working conditions, unionize, secretly be in cahoots with your competitor or foreign adversary, play office politics, scroll through Tiktok instead of doing its job, or cause an embarrassment to your company by posting a politically incorrect meme on its personal social media account.
I am interpreting this to mean that the model tried 10K approaches to solve the problem, and finally selected the one that did the trick. Am I wrong?
That's the thing, did the operator select the correct result or did the model check it's own attempts? No info given whatsoever in the article.
I get the AI skepticism because so much tech hype of recent years turned out to be hot air (if you're generous, obvious fraud if you're not). But AI tools available toady, once you get the hang of using them, are pretty damn amazing already. Many jobs can be fully automated with AI tools that exist today. No further breakthroughs required. And although I still don't believe software engineers will find themselves out of work anytime soon, I can no longer completely rule it out either.
https://en.wikipedia.org/wiki/Jevons_paradox
https://x.com/catalinmpit/status/1831768926746734984
The big decline is driven by a few big factors. Two of which are 1- the overhiring that happened in 2021. This was followed by the increase of interest rates which dramatically constrained the money supply. Investors stopped preferring growth over profits. This shift in investor preferences is reflected in engineering orgs tightening their budgets as they are no longer rewarded for unbridled growth.
It seems many of the popular tools want to make writing software harder than in the 2010s, though. Perhaps their stewards believe that if they keep making things more and more unnecessarily complicated, LLMs won't be able to keep up?
Can you explain what this statement means? It sounds like you're saying LLMs are now smart enough to be able to jump through arbitrary hoops but are not able to do so when taken outside of that comfort zone. If my reading is correct then it sounds like skepticism is still warranted? I'm not trying to be an asshole here, it's just that my #1 problem with anything AI is being able to separate fact from hype.
We are seeing steady improvement on long-run tasks (SWE-Bench being one example) and much more improvement on shorter, more well-defined tasks. The latter capabilities aren’t “hype” or just for show, there really is productive work like that to be done in the world! It’s just not everything, yet.
> the point where LLMs are surpassing humans in any task limited in scope enough to be a “benchmark”
so much as we're over-fitting these bench marks (and in many cases fishing for a particular way of measuring the results that looks more impressive).
While it's great that the LLM community has so many benchmarks and cares about attempting to measure performance, these benchmarks are becoming an increasingly poor signal.
> This is a nerve-wracking time to be a knowledge worker for sure.
It might because I'm in this space, but I personally feel like this is the best time to working in tech. LLMs still are awful at things requiring true expertise while increasingly replacing the need for mediocre programmers and dilettantes. I'm increasingly seeing the quality of the technical people I'm working with going up. After years of being stuck in rooms with leetcode grinding TC chasers, it's very refreshing.
This seems like a bold statement considering we have so few benchmarks, and so many of them are poorly put together.
I'm not especially nerve-wracked about being a knowledge worker, because my day-to-day doesn't consist of being handed a detailed specification of exactly what is required, and then me 'computing' it. Although this does sound a lot like what a product manager does!
If you have to keep checking the result of an LLM, you do not trust it enough to give you the correct answer.
Thus, having to 'prompt' hundreds of times for the answer you believe is correct over something that claims to be smart - which is why it can confidently convince others that its answer is correct (even when it can be totally erroneous).
I bet if Google DeepMind announced the exact same product, you would equally be as skeptical with its cherry-picked results.
I have spent significant time with GPT-4o, and I disagree. LLMs are as useful as a random forum dweller who recognises your question as something they read somewhere at some point but are too lazy to check so they just say the first thing which comes to mind.
Here’s a recent example I shared before: I asked GPT-4o which Monty Python members have been knighted (not a trick question, I wanted to know). It answered Michael Palin and Terry Gilliam, and that they had been knighted for X, Y, and Z (I don’t recall the exact reasons). Then I verified the answer on the BBC, Wikipedia, and a few others, and determined only Michael Palin has been knighted, and those weren’t even the reasons.
Just for kicks, I then said I didn’t think Michael Palin had been knighted. It promptly apologised, told me I was right, and that only Terry Gilliam had been knighted. Worse than useless.
Coding-wise, it’s been hit or miss with way more misses. It can be half-right if you ask it uninteresting boilerplate crap everyone has done hundreds of times, but for anything even remotely interesting it falls flatter than a pancake under a steam roller.
> Only one Monty Python member, Michael Palin, has been knighted. He was honored in 2019 for his contributions to travel, culture, and geography. His extensive work as a travel documentarian, including notable series on the BBC, earned him recognition beyond his comedic career with Monty Python (NERDBOT) (Wikipedia).
> Other members, such as John Cleese, declined honors, including a CBE (Commander of the British Empire) in 1996 and a peerage later on (8days).
Maybe you just asked the question wrong. My prompt was "which monty python actors have been knighted. look it up and give the reasons why. be brief".
The point is that you never know what you can trust or not. Unless you’re intimately familiar with Monty Python history, you only know you got the correct answer in one shot because I already told you what the right answer is.
Oh, and by the way, I just asked GPT-4o the same question, with your phrasing, copied verbatim and it said two Pythons were knighted: Michael Palin (with the correct reasons this time) and John Cleese.
¹ And I’ve had enough discussions on HN where someone insists on the correct way to prompt, then they do it and get wrong answers. Which they don’t realise until they shared it and disproven their own argument.
Similarly language models are probabilistic and yet they get the easiest questions right 100% of the time with little variability and the hardest prompts will return gibberish. The point of good prompting is to get useful responses to questions at the boundary of what the language model is capable of.
(You can also configure a language model to generate the same output for every prompt without any random noise. Image models for instance generate exactly the same image pixel for pixel when given the same seed.)
In comparison, with text a single word can change the entire meaning of a sentence, paragraph, or idea. The same word in different parts of a text can make all the difference between clarity and ambiguity.
It makes no difference how good your prompting is, some things are simply unknowable by an LLM. I repeatedly asked GPT-4o how many Magic: The Gathering cards based on Monty Python exist. It said there are none (wrong) because they didn’t exist yet at the cut off date of its training. No amount of prompting changes that, unless you steer it by giving it the answer (at which point there would have been no point in asking).
Furthermore, there’s no seed that guarantees truth in all answers or the best images in all cases. Seeds matter for reproducibility, they are unrelated to accuracy.
You think the probabilistic nature of language models is a fundamental problem that puts a ceiling on how smart they can become, but you're wrong.
No. Language can be fuzzy, yes, but not at all in the same way. I have just explained that.
> LLMs can create factually correct responses in dozens of languages using endless variations in phrasing.
So which is it? Is it about good prompting, or can you have endless variations? You can’t have of both ways.
> You fixate on the kind of questions that current language models struggle with
So you’re saying LLMs struggle with simple factual and verifiable questions? Because that’s all the example questions were. If they can’t handle that (and they do it poorly, I agree), what’s the point?
By the way, that’s a single example. I have many more and you can find plenty of others online. Do you also think the Gemini ridiculous answers like putting glue on pizza are about bad promoting?
> You think the probabilistic nature of language models is a fundamental problem that puts a ceiling on how smart they can become, but you're wrong.
One of your mistakes is thinking you know what I think. You’re engaging with a preconceived notion you formed in your head instead of the argument.
And LLMs aren’t smart, because they don’t think. They are an impressive trick for sure, but that does not imply cleverness on their part.
If you pay careful attention to prompt phrasing you will get a lot more mileage out of these models. That's the bottom line. If you believe that you shouldn't have to learn how to use a tool well then you can be satisfied with your righteous attitude but you won't get anywhere.
Will the next model be named "1k", so that the subsequent models will be named "4o1k", and we can all go into retirement?
> ChatGPT Plus and Team users will be able to access o1 models in ChatGPT starting today. Both o1-preview and o1-mini can be selected manually in the model picker, and at launch, weekly rate limits will be 30 messages for o1-preview and 50 for o1-mini. We are working to increase those rates and enable ChatGPT to automatically choose the right model for a given prompt.
Weekly? Holy crap, how expensive is it to run is this model?
We're getting close to parity if things keep getting more efficient as fast as they have been. But that's without accounting for the AI training, which can on the plus side be shared among multiple agents, but on the down side can't really do continuous learning very well without catastrophic forgetting.
What? I agree people who typically use the free ChatGPT webapp won't care about raw chain-of-thoughts, but OpenAI is opening an API endpoint for the O1 model and downstream developers very very much care about chain-of-thoughts/the entire pipeline for debugging and refinement.
I suspect "competitive advantage" is the primary driver here, but that just gives competitors like Anthropic an oppertunity.