Claude Code sends 33k tokens before reading the prompt; OpenCode sends 7k (systima.ai)
This started based off of a hunch. We usually use OpenCode, but were 'forced' to use Claude Code for a while due to issues with Meridian. In that time, we saw the usage meter rise much, much more quickly than when using OpenCode.
This was the initial anecdotal evidence, but we undertook this small study to collect empirical data:
We added logging between the agentic coding tool (Claude Code and OpenCode) and Anthropic's endpoint, and captured all requests (and the returned usage blocks).
With one caveat (toward the end of the post) we found unambiguously that Claude Code was far more inefficient in terms of its cache strategy and its harness token usage than OpenCode.
276 comments
[ 0.69 ms ] story [ 58.0 ms ] threadSo not only is this article AI-written, but the testing was entirely done by AI, too? I can't see any other reason to use such an old model.
> Our traffic passes through a local LLM gateway that wraps requests in its own envelope, a constant we measured at roughly 6,200 tokens with bare calibration requests
Why do you need to do calibration requests to figure out how your own gateway is affecting requests?
> Its subagent lane did not complete cleanly through our gateway
> We attempted to toggle extended thinking in both harnesses and are declining to publish numbers. Our gateway applies its own thinking policy, neither harness's toggle demonstrably survived the path, and anything we quoted would be noise.
Why is your own gateway screwing with your testing?
https://quesma.com/blog/the-true-cost-of-saying-hi-to-an-ai-...
Tokenflation seems very real. The number of tokens needed for simple things is increasing.
I read that this is because it wastes time looking through past conversations and other context to figure one what you might want it to do - a less ambiguous prompt would be better.
The entire agent system prompt can be seen here:
https://github.com/earendil-works/pi/blob/main/packages%2Fco...
But I'll investigate how that works in a session. You got me curious.
It works great for long-horizon tasks, and feels like it saves a boatload of tokens.
I used mitmproxy (setup assisted by Claude, natch) to capture Claude Code's entire initial system prompt and the whole thing was (I just double-checked) 162k of JSON.
This led me to start experimenting with Pi, OpenCode, and Hermes...
I was simply supporting the article's data - their reported 33k tokens is probably roughly 150-165k.
Current /context on a fresh session (compare to that above) is:
My $20 sub using gpt 5.6 sol thinking-off lasts for hours using pi.
Doesn't it need at least a basic system prompt to understand how to call tools?
Qwen 3.6 35B A3B and Qwen 3.6 27B can both do reliable tool calls on Pi at Q4_K_M using llama.cpp
That would depend entirely on what your device is. This sounds likely not to be an issue with the harness, but the capabilities of the models you've tried.
I experience almost no tool call failure using my nothing-special harness and DSv4 Flash.
Ultimately this combo worked:
1. https://pi.dev/packages/pi-tool-guard —- corrects key name synonyms and common structure errors, so tool calls succeed automatically (e.g if the model hallucinates old_str instead of oldText). It also wraps top level oldText/newText in an edits array if the tool didn’t do it.
2. https://pi.dev/packages/@aboutlo/pi-smart-edit - white-space-tolerant edits, as Qwen would sometimes add a fifth space to a four space indent
Hashline edit tools didn’t work well for me at all, they confused the model and it still failed to edit correctly. Also line removals would invalidate the rest of the file requiring re-reads. I tried pi-hashline-edit-pro, though I see it now keeps a database of hashes to help keep them stable across edits. Regardless Qwen kept thinking that the hashline prefixes were part of the source.
and the interesting thing about system prompt wastage is its a cost that scales non linearly with subagent use.
Not sure if intentionally meant as a reference, but it gives "I use Arch btw" vibes.
At this point I think Dario is just in his wellness retreat adjusting a revenue/profit dial.
Increase the price by 70% and then cut it by 50%, resulting in a 15% cut that sounds like a major deal.
Its not 'nefarious' in that its in their best business interests.
But it'd be difficult to take anyone serious who thinks Anthropic's motivation was to improve the UX, and the other effect were by accident. At the time they specifically started blocking based on openclaw prompt text. Its a walled-garden tactic.
A walled garden is nefarious to people who do not want to be inside one.
So what? When you care about optimising the entire experience, you offer sane defaults.
When you prevent people from changing the defaults, it's about control, not experience.
The nefarious part is because it's non optional. They could give you an option and compete by being better, instead you're given the finger as the option is taken from you. Competition is hard and banning people to create more FUD serves business need better.
You've obviously been gaslit so badly you're desperate to find a way to defend a shitty move and pretend it's the only way to increase usability. But you don't have to deny really! You're allowed to admit control is easier for a company than competition, and that they didn't have to, but did because it increases their control of the ecosystem.
If you want to defend someone, good? But at least save it for someone who actually deserves it. They don't; and you insult you and your readers intelligence by trying.
The only issue is that Anthropic optimizes the entire experience for their bottom line. User experience and price only suffer becaue of that.
no amount of alignment will stop aomeone drom just shutting up.
On the other hand, the newer variants also tend to benchmark higher so it's not quite a clean argument of "hey the new version eats more tokens"
Its rather frustrating, slower tokens and more tokens.
OTOH, this makes typical subscriptions usages consume more tokens, which are included in their flat fee.
This sounds more like incompetence than malice.
To remind Goodhart's law: "When a measure becomes a target, it ceases to be a good measure".
..also to parent's point, yes the upsell is only appealing once user run's out of tokens.
When using Pi, one way to significantly reduce input tokens it yields is to ignore common bookkeeping "dot directories", such as `.git`. How to do so can be found with the following interactive Pi prompt:
Other local assets to consider ignoring are `.pi`, `.agents`, `*.md`, and language specific output directories such as `__pycache__`, `bin`, `obj`, `target`, etc.I asked both a trivial question (summarize last commit). Opus cost 50 cents, Fable about $1.
That checks out because Fable's twice as much in the API (though I think its emphasis on correctness makes the difference larger for bigger tasks).
But, at $1 per question, I think I will stick to the subscription for now! I was certainly glad GPT-5.6-Sol is included in OpenAI's subscription, and I'm curious if they'll be able to do the same for GPT-6.
All the VC money appears to have run out a few weeks ago.
Are we measuring and caring about the right thing?
> When context gets too long, maki compacts history automatically: strips images, thinking blocks, and summarizes older turns.
Don’t the summaries of older turns effectively invalidate the context cache, such that you consume less tokens but more expensive tokens?
I don’t like that Claude became more opaque around February, including the system prompts. 33k feels way too much.
I appreciate that Codex is open source and OpenAI has explicitly said using the subscription with other agents is ok. OpenAI has been much more consumer-friendly recently.
If I let the main agent do the same task sequentially, it was no problem at all. I don't know if it's really just communication and orchestration that makes sub agents so inefficient, or if Anthropic figured that most people using sub agents pay per token on a big corporate account, so this is an easy way to make more money from tokenmaxxers.
That said, if your project is "do this well-planned thing on a bunch of things in parallel" then you should absolutely be instructing to have subagents "step down" to less curious models. Their output may well be more cohesive as a result!
A major complaint with AI code was that AIs struggle with complex codebases, don't respect existing conventions, reinvent functionality multiple times over, etc. So, newer high end AIs are tuned with the "explore/exploit" dial turned towards "explore".
You could probably get it to do things "quick and dirty" with prompting, but that, of course, requires prompting for it.
Give it only what it needs and do things usually 1 file at a time.
Feels like I'm a sort of manual tape editor, if the context was a tape fed into machine, I assemble that and then watch the machine output the results I need.
I feel like most mainstream programming languages do this sort of work for their standard libraries and their official docs. Go and Python come to mind, but plenty others do this reasonably well to the point where one mostly doesn’t need to read the implementation code to effectively use the standard library itself.
All the while, it could have just done a two-line probe test and see what happens when it calls the API with "None" for that parameter. Or just assum it would act as expected and wire in debug logs in case it doesn't.
The downside is the code isn’t as good but it is produced a lot faster and more cheaply and often it’s actually fine.
CoT has made LLMs better (say 50% improvement or something) but increases cost by an order of magnitude. That graph is going in the wrong direction and has been for a while now
I write a good prompt, paste the code then copy the output code and place it into my project.
So in the end I hand assemble and I only give it what it needs to know so no extra context wasted.
The human in the loop is of course the secret sauce but this way I am highly efficient, no vibecode and I work really fast too. Everything is audited.
Depends what I want but I can give a completely new context for every generation.
I try to make everything as simple and human readable as possible because I want the audit to go fast.
I think for me I lean towards an audit optimized approach. Everything is still generated but revolves around the human-in-the-loop for review.
Unless they are orthogonal they most likely require similar context anyway so multiple sub agent is just wasteful.
I vaguely understand you argument with the context, however is that not solved by sum agents handing their results in to the planner (or a third agent) to run on them again? I'd assume that's what is happening anyway. Let me know if that's wrong
Or are you saying my sub agents burned so many tokens because they were all using Fable, whereas my main agent could do the same job with a lesser model?
Using VS code if it matters.
If you do it sequentially you only read those files approximately once, and everything hits the same prefix cache
If you plan on continuing on in the parent, and aren't going to necessarily be touching the systems the other agents are exploring, it can be worth it.
It's useful in certain situations where the parent context may need the "10,000 foot" view of something without going back in there. But subsystem-specific AGENTS.md/CLAUDE.md files are still superior and accomplish the same thing. The problem with those is they can become stale.
Probably because the general purpose subagents inherit the parent model.
I tell Claude explicitly to use Explore subagents, which use Haiku only, now.
https://code.claude.com/docs/en/changelog#2-1-198
> The built-in Explore agent now inherits the main session’s model (capped at opus) instead of running on haiku
https://gist.github.com/joshcartme/dd71df7b4c51c356760b28d7f...
I run it basically 24/7 on a ~500k line repo, and only rarely run out of quota before the end of the week.
My experience with Claude Code was very good until about 2.5 months ago, and then it suddenly turned unbelievably terrible for me.
I have not and will hopefully never look back.
I still have PTSD from how ungodly terrible it was that last week of using it.
Please, for the sake of everyone suffering from actual PTSD: Don't. It's hard enough already for victims to communicate what difficulties they are facing without people watering down terminology like that.
Sorry just teasing.
How else are we supposed to learn from each other, voice our opinions, point out our mistakes to each other? For me, this is communication. And currently 8 upvotes seem to agree with me and my request. Feel free to ignore it, or consider it, for your own use of language. But, sorry, to me, you're the one acting like a jerk and trying to "police", not me.
If anything, it's a net positive people are talking about mental health and recognize different ailments such as OCD, PTSD, and others.
I disagree that it's insensitive to those or have the illness. If it hurts you when people use PTSD as a literary special effect, I invite you to articulate why exactly it makes you feel (however it makes you feel - I don't want to put words in your mouth). I get the sense that some people take offense the same way a religious fundamentalist doesn't want to hear their Prophet or Messiah used disrespectfully. The source of the anger is built on faith and dogma.
Just tried Claude Code yesterday, and nope, it's the same old bad.
> CRITICAL: Do NOT spawn sub-agents for any reason. Perform all work in the main session. If a task is too large, ask me to break it down manually.
> This is a big task, and can easily get too large. However, sub-agents make the situation worse, and eat through our token budget way too fast. Do not use them.
> Take on manageable tasks. Don't try to do everything at once. When you start on a big task, break it down into smaller tasks, and make sure you finish each task before starting on the next one.
Or actually Claude put it there for me. Maybe it's a bit much, but it seems to work.
This is why the subscription plans are forced through the harness (the "OpenClaw Wars"): it creates a false equivalence in the minds of many customers between API tokens (latency sensitive, easy to measure) and Claude Code tokens (remnant backfill to stay to the right of the roofline, marginal cost often zero).
Selling sausage as sirloin is a great business if people go for it. And there's nothing inherently wrong with spot pricing, as long as you're honest about it...
What's happening this year, with secrecy and all, is saddening, but expected.
I feel like maybe it could have asked for clarification or something rather than go and try to calculate all the digits of pi all of a sudden.
Yesterday, I wanted to review a complex piece after a large refactoring, and requested a review plan beforehand. The first step was 8 agents + one more to verify the findings (all Fable). Looks good, approved.
The verification step turned into an attempt to throw a party with 41 Fable verifiers.
It will find a way.
That'll be $50 — please.
Part of my prompt:
---
Start of your prompts...
If a dynamic workflow is used, set its subagents to Opus 4.8 — do not inherit Fable 5. This applies only to dynamic-workflow subagents; leave ordinary subagents (Plan, Explore, etc.) on their defaults or current configuration.
... rest of your prompts.
Maybe when they realize there is need to change this they come up with a more configurable interface for us mere mortals who can't afford to gamble their house on a pay as you go subscription.
The best you can do in such an environment is seek to introduce new features at the top tier, and then pull old features down the stack as the cost of those features has been amortized out, or to hurt your competitors by raising the ladder.
But somehow the cost has doubled in the last few months.
Supposed to be hacker news and half the posts are like "this harness steals this" like it cant be avoided.
These API costs are mad.
It pays to be marginally ahead of people stuck on open models.
After reading PUSH_AX's valid comment: ``` This is like saying contractor (A) asked for $33,000 to undertake the work and contractor (B) asked for $7,000 Are we measuring and caring about the right thing? ``` We will update the post to include:
1) A more in-depth task. 2) Qualitative results comparison. 3) As soon as possible, a reproduction of the inputs and outputs.
I wonder if a lot of the 33k is context, like from recent conversations.
Therefore, you should include the actual costs associated with the task in API token usage or subscription level. Is there a reasonable way to do apples to apples cost comparison?
If you don't use a subscription, and pay per token instead, you can easily move to another harness.