Yes, two things: 1. GPT-5.1 Codex is a fine tune, not the "vanilla" 5.1 2. More importantly, GPT 5.1 Codex achieves its performance when used with a specific tool (Codex CLI) that is optimized for GPT 5.1 Codex. But…
[comment removed]
They explicitly address this in page 11 of the report. Basically perfect recall for up to 1M tokens; way better than GPT-4.
Yes
Weird - just tried this and it worked for me.
Tech report seems to hint at the fact that GPT-4 may have had some training/testing data contamination and so GPT-4 performance may be overstated.
I'd be shocked - given the incentives - if it hasn't already happened to a great extent. Many of the types of people Google DeepMind hires are also the types of people hedge funds hire.
Sure, but this time in 2022, you probably wouldn't have let it do that much (or even had this thought). Progress!
? BERT was 5 years ago. Of course it's worse than anything introduced more recently (both inside and outside Google). https://en.m.wikipedia.org/wiki/BERT_(language_model)
The US has had the Bill of Rights and its constitutional system / property protection for a long time but only recently has had the degree of NIMBYism that it's had. It's easy to pretend the bug is actually an important…
> If only we would fight for the real issues like... I've heard these arguments many times and they never make sense to me. Most of the people I know working on AI do so precisely because they want to solve the "real…
Two Sigma and Renaissance (albeit not Medallion) primarily invest outside capital.
Doesn’t work on mobile (iOS) for me.
Yes, two things: 1. GPT-5.1 Codex is a fine tune, not the "vanilla" 5.1 2. More importantly, GPT 5.1 Codex achieves its performance when used with a specific tool (Codex CLI) that is optimized for GPT 5.1 Codex. But…
[comment removed]
They explicitly address this in page 11 of the report. Basically perfect recall for up to 1M tokens; way better than GPT-4.
Yes
Weird - just tried this and it worked for me.
Tech report seems to hint at the fact that GPT-4 may have had some training/testing data contamination and so GPT-4 performance may be overstated.
I'd be shocked - given the incentives - if it hasn't already happened to a great extent. Many of the types of people Google DeepMind hires are also the types of people hedge funds hire.
Sure, but this time in 2022, you probably wouldn't have let it do that much (or even had this thought). Progress!
? BERT was 5 years ago. Of course it's worse than anything introduced more recently (both inside and outside Google). https://en.m.wikipedia.org/wiki/BERT_(language_model)
The US has had the Bill of Rights and its constitutional system / property protection for a long time but only recently has had the degree of NIMBYism that it's had. It's easy to pretend the bug is actually an important…
> If only we would fight for the real issues like... I've heard these arguments many times and they never make sense to me. Most of the people I know working on AI do so precisely because they want to solve the "real…
Two Sigma and Renaissance (albeit not Medallion) primarily invest outside capital.
Doesn’t work on mobile (iOS) for me.