Context size cuts like this are usually a cost/latency tradeoff rather than a capability one - serving a smaller window is cheaper and keeps latency in check, and most sessions probably don't need anywhere near 372k tokens anyway. Curious if this affects people who were actually relying on the larger window for big codebases.
using open models and local AI via llamacpp, I'm compacting (via agent directed) between 55k-85k; going up to 120k is rare unless the agent is actually following a non trivial log tracing or some other thing that absolutely requires keeping large context.
I've also tailored an agent system prompt with llamacpp reasoning budget and message that asks the agent to spawn subagents then compress to reduce context bloat. It's in opencode using the dynamic context pruning, so it's mostly keeping a vector&direction without having to bloat up.
It mostly works to allow iteration into multiple sub components.
The other day I was trying to find out exactly how much context size 5.6 has in codex CLI (via subscription) and I didn’t find that documented in any of the obvious places. Sort of ridiculous that we have to find this out in X or Reddit, e.g:
I never felt that context size was any issue at all in codex. I don't know how their compaction works, but it just keeps on going as if it has no context limit at all. At least in my experience.
Another interesting change in that commit is the addition of this section to the system prompt:
Before taking a destructive action:
- Make sure the action is clearly within
the user's request.
- Resolve the exact targets with
read-only checks when necessary.
- Do not use `$HOME`, `~`, `/`, a
workspace root, or another broad
directory as the target of a
recursive or destructive command
No matter how good compaction is, on some big projects it needs to read a lot of files. In my experience the first 200.000 tokens go FAST, but after that it slows down. Most of my Fable sessions don't go over 500.000 tokens, I don't need to compact once. But when I use Codex a single session has to compact over and over again.
In my experience the only reason it needs to read a lot of files is if your agents.md is deficient. It should only need to read the file it's actually working on, maybe a few others. Everything else should be explained in documentation
Good move given some experienced issues and compaction across the 5.6 range is closer to 5.4 than 5.5, i.e solid and reliable.
Will say that 5.6-Sol is a minor bump in my benchmarks in most areas vs 5.5 but a severe regression in a few specific task focused on rearranging trees, addressing merge conflicts, etc. where the model to accomplish the task does not properly adhere to prompts in a way GPT-5 originally managed, not retaining parts of history in the way prompted despite specific instructions not to as that made the final completion easier…
I am of the conservative and cautious opinion that no model should be able to run destructive tasks at all, I have seen every model do things that make me concerned enough to maintain that opinion and know my evals can’t catch everything. But for 5.6-Sol specifically, I’d caution everyone to reevaluate how you run the model, maybe take a few more precautions you tend to forgo.
It is extremely capable as a reviewer and for extensive tasks, though for the later, the safety net I feel is required to be comfortable limits the utility. The code 5.6-Sol provides also still is a bit harder to parse in reviews.
Release strategy wise, feel it’s have been smarter to release only Luna and Sol now, then Terra a few weeks of posttraining later, I simply cannot see a purpose for it in the current form given how well both Luna and Sol scale up and down respectively with reasoning. Two models from a lab at a time is also the limit I feel one can properly assess at a time.
Not a fan of their context compaction and I feel like 1M-token context should minimum today. Each day I see how GPT 5.5 and 5.6 struggle a bit after each compaction before they get to the full speed, sometimes focusing too much on some older steering message that made it into the compacted context.
Note that GPT-5.6-Sol is ~2x as token efficient as Opus/Fable, so a 258K max is equivalent to ~516K on Claude models. You also may just be fooling yourself, as context rot is still a problem[1][2] and there's evidence compaction can be equivalent or better than longer context windows for agentic tasks[3]. Obviously it would be best if the models could reason over 1M context like they do over 256K context, but they can't yet.
I know a lot of people like to say that compaction makes this moot, but the level of detail you lose across compaction is wildly too much for most things that I do, unfortunately.
Perhaps if your plans don't have as much detail, or if you're not, for example, having a discussion with a lot of nitty-gritty then it's fine?
The lack of long context is the main reason that I still end up using Anthropic.
The worst is when you need it to hold for example a number of papers in its head, or large and complex materials that it needs full resolution on and your context window ends up being perennially at 16%. You have about five minutes of conversation and it compacts and then you have to wait for it to read that again, get to 16%... and repeat.
372 was not perfect, but it was so much better and a godsend. It turned that 12 to 20% into more like 40%.
sometimes compaction is a death knell and other times it's totally fine. it really just depends on whether the thing you were doing at the very end has high correlation with stuff from early on in the session, or if there was an obvious ~break s.t. you were just on some specific subtask. the situation has clearly gotten a lot better than even 6 months ago-- i think openai handles compaction better in my own experience. but it's exactly the kind of thing that yegge was trying to get at with gastown-- i used to run every session s.t. i could re-animate it every time the compaction boundary came... now, when the compaction boundary comes i am so pot committed that i just hold my breath.
I think I've seen the opposite end of this as well: I get the most value from any LLM when I ask it one very targeted question with a followup. I don't fall outside of any sort of context window sizing when I am doing that. That isn't to say that I don't have a few long engineering discussions that are helpful, but most are very to the point. It helps that I have decades of experience at this point and effectively know what to ask to narrow the scope quickly.
What. Anthropic's models have million token context windows! I'm switching to OpenAI next month, I can't believe they're still stuck on ~300k... Better start adapting to the new reality I guess.
A typical suggestion is to make the agent create or update a .md file once in a while, so the agent would remember the new important things that have shown up since the last update. The problem is, how would the agent know which things are really important. If it knows that well, then /compact should work well too...
All depends on context coherence. E.g. in Claude Code with the "1M context" models I am reluctant to push past 30%. So if that Codex 272K is 100% coherent than it is less of a difference. Still, even then the "hard fail" boundary has moved. Longer running agentic processes will surely run into this more often.
Yeah, agreed. It makes everything else look pretty bad. I still like to manage my work in a more structured form, but Codex can just rip on a goal end to end in the same thread.
You just need good compaction and a memory system with decent project layout and documentation. If you don't have that 1M tokens won't really save you either.
Not sure if this is still up to date (2023), but https://arxiv.org/abs/2307.03172 shows that performance degrades mostly in the middle of the context.
Anecdotally I've been stuck in that situation of being at 400-500k tokens and "just one more prompt bro" will get the task done, and I appreciate not having to wait through a compaction. If anything, keeping the bloated context helps with accuracy at the expense of speed in these cases.
I have rule files that guides the agent towards my coding standards, code style, house rules etc. They alone cost 60-80k tokens, and they are the backbone of my system that prevents slop. Pre 1M context, I had to build complicated tooling to re-include the relevant docs to the context upon compaction, which relied on unstable transcription file format, which was a pain to maintain. With 1M context I deleted all of those. Nowadays most of my sessions uses 300-450k context.
Another thing that's preventing me from trying Codex. (the other is @ referencing files not auto including them to the context)
1M should be table stakes for frontier models at this point for programming.
With purely quadratic attention, the cost of the token at 372K is 87% more than the token at 272K.
There are attention mechanisms that help to mitigate this, but you can clearly see in the chart that Tibo posted that the attention mechanisms they are using are still scaling quadratically.
If ChatGPT 5.6 is so good why do they continue to have the same motherfucking cache and usage issues that they were struggling with when 5.2 is new? Could not pay me to use this shit. Enjoy your reset bingo
58 comments
[ 11.4 ms ] story [ 1158 ms ] threadI try to keep it under 200k but my DeepSeek and MiMo sessions can sometimes grow to 350k tokens when I try to squeeze one last iteration I compact.
Can't OpenAI copy DeepSeek K/V cache tech (from published papers) to make it super cheap?
I've also tailored an agent system prompt with llamacpp reasoning budget and message that asks the agent to spawn subagents then compress to reduce context bloat. It's in opencode using the dynamic context pruning, so it's mostly keeping a vector&direction without having to bloat up.
It mostly works to allow iteration into multiple sub components.
https://www.reddit.com/r/codex/s/hVv29obfFD
Will say that 5.6-Sol is a minor bump in my benchmarks in most areas vs 5.5 but a severe regression in a few specific task focused on rearranging trees, addressing merge conflicts, etc. where the model to accomplish the task does not properly adhere to prompts in a way GPT-5 originally managed, not retaining parts of history in the way prompted despite specific instructions not to as that made the final completion easier…
I am of the conservative and cautious opinion that no model should be able to run destructive tasks at all, I have seen every model do things that make me concerned enough to maintain that opinion and know my evals can’t catch everything. But for 5.6-Sol specifically, I’d caution everyone to reevaluate how you run the model, maybe take a few more precautions you tend to forgo.
It is extremely capable as a reviewer and for extensive tasks, though for the later, the safety net I feel is required to be comfortable limits the utility. The code 5.6-Sol provides also still is a bit harder to parse in reviews.
Release strategy wise, feel it’s have been smarter to release only Luna and Sol now, then Terra a few weeks of posttraining later, I simply cannot see a purpose for it in the current form given how well both Luna and Sol scale up and down respectively with reasoning. Two models from a lab at a time is also the limit I feel one can properly assess at a time.
[1] https://arxiv.org/abs/2605.12366 [2] see F1 of GraphWalks 256K vs 1M in Opus 4.8 System Card: https://www-cdn.anthropic.com/0b4915911bb0d19eca5b5ee635c80f... [3] https://context-folding.github.io/
Perhaps if your plans don't have as much detail, or if you're not, for example, having a discussion with a lot of nitty-gritty then it's fine?
The lack of long context is the main reason that I still end up using Anthropic.
The worst is when you need it to hold for example a number of papers in its head, or large and complex materials that it needs full resolution on and your context window ends up being perennially at 16%. You have about five minutes of conversation and it compacts and then you have to wait for it to read that again, get to 16%... and repeat.
372 was not perfect, but it was so much better and a godsend. It turned that 12 to 20% into more like 40%.
I had a /goal running last night for 9.5 hours straight while I slept. When I woke up in the morning it was fully on task and focused.
Write up a detailed design doc. Build a decent AGENTS.md, and write up a good prompt or /goal.
Long context can be more of a curse than a benefit sometimes anyways.
Anecdotally I've been stuck in that situation of being at 400-500k tokens and "just one more prompt bro" will get the task done, and I appreciate not having to wait through a compaction. If anything, keeping the bloated context helps with accuracy at the expense of speed in these cases.
> Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.
Sincerely, HN Guidelines Police :-)
https://news.ycombinator.com/newsguidelines.html
Another thing that's preventing me from trying Codex. (the other is @ referencing files not auto including them to the context)
1M should be table stakes for frontier models at this point for programming.
There are attention mechanisms that help to mitigate this, but you can clearly see in the chart that Tibo posted that the attention mechanisms they are using are still scaling quadratically.
This becomes an optimization problem for token t:
min f[cost, quality loss] = compaction cost[t] + compaction quality loss[t] + token cost[t]