>A user should be able to close an account, keep a session, and hand it to another model. The new model may disagree, ask questions, or perform worse.
I think this is a fair contract. I also think a user should ideally be able to easily identify a comparable model in terms of embedding 'signature'. When GPT-4o originally kicked the bucket, I remember reading lots of anecdotes of people desperately searching for models similar in manner and language, so they could pump in their exports and re-find their friend. Other open-ai models just didn't have the same vibe. It was sad to read. This, to me, is the power of open-weights models. They are for perpetuity. You can keep your guide, your friend, your therapist, whatever. No big company can pull the rug.
I get the point of the article and it resonates with my general tendency to support more openness, but older and more cynical me wonders how much of this is a battle worth fighting or a self-serving argument from an harness developer.
I mean, the article says each of the points it is complaining about has “_a basic justification that's trivial for a provider to come up with, along with good arguments for why this is good for the user_“, which to me sounds like implying these are just bs to have people accept them, but is that actually the case?
For instance, this [0] was mentioned here a while ago, and based on that it seems pretty clear why one would choose not to provide the _thinking_ anymore…
It's one thing that the APIs don't return the reasoning tokens in a readable fashion. But then why do I have to send them the whole conversation every time? If they don't allow me to see parts of the conversation then they should have the burden of storing and transmitting it, and shouldn't charge more and more the longer the conversation gets (unless opaque token caching kicks in) like some LLM Shlemil the painter.
This is an important article. I hadn’t realized it was already getting this bad. Like a frog enjoying a nice warm bath ...
> Most people do not switch their operating system or phone provider every week either. But even if you do not utilize that freedom, it matters because it changes the relationship you have with the provider and the provider has with you.
This is why it’s important to utilize your freedoms. Do NOT let yourself get locked into a particular ecosystem (this is why I’m building a phone app for OpenCode).
This article makes me reconsider using my recently acquired Codex sub in my home setup. I never liked that they hide the reasoning, but somehow overrode the cognitive dissonance because the performance is so good. But the inauditability is already a huge problem.
I am waiting for prices to get down and in the meanwhile I am gathering as much session data from claude/codex as possible so I can then later use them to fine tune open models. I've built my own session parser/archiver for this.
It was with the transience of valuable agent session dialogue in mind that I built https://www.agentkanban.io - A key feature is context capture in the tasks which you create on the board and then progress in agent sessions. Supported agents are currently Claude and Github CoPilot in VS Code. All of your context is captured in the task and can be reloaded into new agent sessions at any point in the future. Tool use is intentionally discarded because it is proprietary in nature and therefore breaks the promise of session portability.
People will keep hiding reasoning because it allows prompt injection https://arxiv.org/pdf/2603.12277, in addition to facilitating distillation (you don't pay the full cost of RL)
This analogy doesn't make sense to me. Once I've had a conversation, it may be useful to refer back to it shortly afterwards, but typically the output is some code (or maybe a recipe or instructions for something or whatever). It's certainly important to me that I can use that code elsewhere. That's the thing that, to me, is analogous to being able to switch operating system and still access my files or use my favourite word processing program.
But being able to decode the blobs that subagents returned in a conversation I've already forgotten about? That doesn't affect my ability to switch at all! I can't even imagine what trivial detail of an operating system this is analogous to. Maybe an undo buffer of a document that gets cleared on exit anyway?
I can still pass all my code, and any associated documentation, to any other agent at any time. What am I missing?
I've never considered the conversation and reasoning to be more than transient. If you want to carry forward a session, you must produce a context artifact.
IMO this is like wanting to dump the intermediate state of an LLVM compilation and expecting to reliably resume it across platforms.
I think most LLM work will migrate to open weight models in a couple of years for this exact reason (as well as price), unless the big providers manage to have their way and the market gets regulated in a way that favors closed source models. To me it seems though like most models are becoming good enough for most tasks we want them to do so no need to use the latest super intelligent one that knows how to compose operas and do quantum chemistry and have it answer support requests. And with these models transparency is no issue.
LLM pricing is now Business Gacha - the whales will open more loot boxes and the normals will serve as fodder and feeder for them.
> market gets regulated
This MUST happen, but not in the way or for the reasons that any one thinks. There is a question of liability when one of these models causes massive damage to a 3rd party.
If I am running an open source model on a "rented" platform and it goes off the rails who is to blame when it "breaks containment" and does something bad?
The liability people (read lawyers) are gonna figure this out a lot faster than any one who says the word "safety" a lot.
Is this a analogous to idempotent behaviour? Of reproducible compilation? If the inputs and outputs aren't sufficient to reproduce one from the other, what is a minimally satisfying description of the middle third, the missing part? The "other" inputs?
I came here to complain about something similar, the excessive use of box shadows on both the top and the sides. It made the text horrible to read on mobile, if it not for Firefox's Readability mode.
I saw the weird subtle pattern and first freaked out that my monitor's backlight was failing (high flicker rate revealing LCD refresh patterns). I also was playing a video with the browser in the background and it was choppy, and I first thought it was my video player messing up until I realized the web page had some pointless background animation.
I just tested again and even my cursor gets jump over the page. Amazing they'd do this for such a subtle, useless effect.
I think the article gives a very good overview of a problem that most users of AI rarely evaluate / have to grapple with.
There really is a surprising amount of coupling that happens with many of the "frontier inference providers", where a lot of the powerful non-LLM extensions (web search, code execution) are packaged as simple "tools" on the surface, that build up a lot of moat. Those are parts that are in theory nicely separable from the inference API, and could be externalized via MCP servers, but are usually not offered as such by the inference providers themselves, and are often only available in a slightly less powerful variant from other providers.
We've faced that issue repeatedly while building a on-premise provider-agnostic Chat UI & platform[0], where even adding something as simple as an in-chat image generation tool for the end-users (which is just a build-in tool in the OpenAI Responses API), becomes a bit of an ordeal (though part of that is due to the MCP spec missing a native file transfer protocol as of today[1]).
I am quite hopeful though, as with recent shifts of interest towards open weight models, there will be more opportunities for companies offering alternative implementations in a easier plug-and-play manner.
OpenAI has been nudging their users toward the Responses API lately. Chat completion can still do everything if you are clever. The obvious reason to push everyone into the Responses API is that the reasoning process can be locked down and encrypted. The semantics of the chat completion API are fundamentally transparent. 100% of the tokens are visible.
If you have patience and the willingness to endure a little bit of pain, you can still retain autonomy over the entire reasoning process while using the latest 5.6 model family. The only downside is that you are now fully responsible for it.
Consider that when you flip your agent's reasoning level to "xhigh" or whatever, it's not some magical model internals being pushed around. There isn't an actual "try harder" knob on the black box. This is merely orchestration of many instances of one or more model types based upon some proprietary harness logic. The chances you can develop a domain specific reasoning process that outperforms the frontier providers is still very good.
Can you give some more details on the technical differences between completions and responses that people actually care about? I found this information surprisingly hard to drum up.
I don’t see this as a big deal in practice. Conversations contain a bunch of junk anyway, so removing it from the context is usually good.
In my repo, I have a notes directory. I ask the AI to write a markdown file with what it learned, what work has been done, and what remains. In the next conversation, I can ask another model to pick it up from there. Sometimes I edit the note first.
Oh, it very much is a big deal. In a setting where users can switch models easily and with no downsides, market forces will give us better and cheaper AI over time. If switching models is painful (as in, losing part of your context), the providers can create vendor lock-in, enshittify the user experience, and drive up cost. Even if there are workarounds right now, those AI providers have every incentive to make freedom-seekers more miserable over time.
It is disheartening that some AI companies are now setting the stage for enshittification. I hope we can collectively dodge that bullet.
This is what brought me around to doing more agentic coding. I set the task, require tests and the strict linting must pass and then leave it to blow smoke up its own ass about what's going on.
I see glimpses scrolling past of all the conversational language that used to frustrate me so much when using a chat interface and I can just let it flow on past.
I come back when its made everything pass, my life is better now.
The previous match was still too clever for clippy’s type analysis, so I’ve simplified it into a direct borrowed-pattern form and I’m validating once more. I’m replacing that wrapper with a direct, non-transparent error variant so the enum stays explicit and doesn’t rely on a generic anyhow bridge.
None of the popular providers have real moat, which is very much problematic for OpenAI and Anthropic; their operating margins are deeply red and they do not have the same reserves that FAANG/MANGA has. FAANG/MANGA does the same to kill fair market competition in the long run.
This is one of the ways in which model providers are attempting to artificially create moat where there is none.
There are only functional objections to this.
This is fundamentally antitrust material. However, this is “acceptable” in contemporary USA because the FTC has been gutted to follow Trumponomics.
OpenAI is doing this so that you can't preserve cache when routing between for example GPT 5.6 and DeepSeek V4, or other providers. I'm building a router that easily reaches >99% cache hit rate in Pi when routing between GPT 5.4, Kimi, DeepSeek etc. Working on a big update that's coming out in a week or so.
I do, quite regularly. Because different models have different strength (for example when producing text for live presentations based on the text for a reading deck). Or when it comes to other aspects of the work. I regularly switch between open wheight models and closed models.
I know, I am a tiny minority here. And this behavior only ever started a few weeks ago. But it quickly became a habbit, to CTRL-L in pi and change the model.
The post for the most part is mine, so if something AI generated slipped into it it's on me. I can share with you what I did to write the article which I do for most of my writing.
I start out with what I want to talk about, and then make an initial draft. In this case I also had Sol research all current APIs so I don't have blind spots. I tend to also talk with an AI to see if the structure structure of the post makes sense. I then wrote all sections by hand but used Sol to fix up typos, grammar and punctuation. I also had Sol apply notes and patches that were provided via Discord.
If I find the earlier drafts I can check if it did some more outrageous edits, but I kinda doubt it did. I do know that my workflow of having an LLM to fixes up later is increasingly breaking on SOTA models and I have complained about this before. Normally I now carefully apply fixes that it proposes, but in this case I didn't due to time constraints.
I am curious though why it claims that this is AI generated.
Currently these models have a limited context window, so the value of a session is not that high, at any rate at some point the model will have forgotten things. If in the future the model will truly learn / "change" as a result of the interaction, in any case this will be non portable to another model. So I think this is kinda moot
The AI companies need some way to lock you in. You weren't hoping for a free market were you? Hah. Free markets are for peasants. Investors are always looking for ways to make the market non-free.
77 comments
[ 1.8 ms ] story [ 24.1 ms ] threadI think this is a fair contract. I also think a user should ideally be able to easily identify a comparable model in terms of embedding 'signature'. When GPT-4o originally kicked the bucket, I remember reading lots of anecdotes of people desperately searching for models similar in manner and language, so they could pump in their exports and re-find their friend. Other open-ai models just didn't have the same vibe. It was sad to read. This, to me, is the power of open-weights models. They are for perpetuity. You can keep your guide, your friend, your therapist, whatever. No big company can pull the rug.
I mean, the article says each of the points it is complaining about has “_a basic justification that's trivial for a provider to come up with, along with good arguments for why this is good for the user_“, which to me sounds like implying these are just bs to have people accept them, but is that actually the case?
For instance, this [0] was mentioned here a while ago, and based on that it seems pretty clear why one would choose not to provide the _thinking_ anymore…
[0] https://role-confusion.github.io/
> Most people do not switch their operating system or phone provider every week either. But even if you do not utilize that freedom, it matters because it changes the relationship you have with the provider and the provider has with you.
This is why it’s important to utilize your freedoms. Do NOT let yourself get locked into a particular ecosystem (this is why I’m building a phone app for OpenCode).
This article makes me reconsider using my recently acquired Codex sub in my home setup. I never liked that they hide the reasoning, but somehow overrode the cognitive dissonance because the performance is so good. But the inauditability is already a huge problem.
I wish people would apply this same logic to governments.
Anything new is going to be an extremely rapid race to the bottom.
But being able to decode the blobs that subagents returned in a conversation I've already forgotten about? That doesn't affect my ability to switch at all! I can't even imagine what trivial detail of an operating system this is analogous to. Maybe an undo buffer of a document that gets cleared on exit anyway?
I can still pass all my code, and any associated documentation, to any other agent at any time. What am I missing?
IMO this is like wanting to dump the intermediate state of an LLVM compilation and expecting to reliably resume it across platforms.
LLM pricing is now Business Gacha - the whales will open more loot boxes and the normals will serve as fodder and feeder for them.
> market gets regulated
This MUST happen, but not in the way or for the reasons that any one thinks. There is a question of liability when one of these models causes massive damage to a 3rd party.
If I am running an open source model on a "rented" platform and it goes off the rails who is to blame when it "breaks containment" and does something bad?
The liability people (read lawyers) are gonna figure this out a lot faster than any one who says the word "safety" a lot.
I just tested again and even my cursor gets jump over the page. Amazing they'd do this for such a subtle, useless effect.
Then I blocked the whole site in ublock origin to ensure I don't visit the site again by accident.
The filter:
||earendil.com
There really is a surprising amount of coupling that happens with many of the "frontier inference providers", where a lot of the powerful non-LLM extensions (web search, code execution) are packaged as simple "tools" on the surface, that build up a lot of moat. Those are parts that are in theory nicely separable from the inference API, and could be externalized via MCP servers, but are usually not offered as such by the inference providers themselves, and are often only available in a slightly less powerful variant from other providers.
We've faced that issue repeatedly while building a on-premise provider-agnostic Chat UI & platform[0], where even adding something as simple as an in-chat image generation tool for the end-users (which is just a build-in tool in the OpenAI Responses API), becomes a bit of an ordeal (though part of that is due to the MCP spec missing a native file transfer protocol as of today[1]).
I am quite hopeful though, as with recent shifts of interest towards open weight models, there will be more opportunities for companies offering alternative implementations in a easier plug-and-play manner.
[0]: https://github.com/EratoLab/erato
[1]: https://github.com/modelcontextprotocol/modelcontextprotocol...
If you have patience and the willingness to endure a little bit of pain, you can still retain autonomy over the entire reasoning process while using the latest 5.6 model family. The only downside is that you are now fully responsible for it.
Consider that when you flip your agent's reasoning level to "xhigh" or whatever, it's not some magical model internals being pushed around. There isn't an actual "try harder" knob on the black box. This is merely orchestration of many instances of one or more model types based upon some proprietary harness logic. The chances you can develop a domain specific reasoning process that outperforms the frontier providers is still very good.
In my repo, I have a notes directory. I ask the AI to write a markdown file with what it learned, what work has been done, and what remains. In the next conversation, I can ask another model to pick it up from there. Sometimes I edit the note first.
It is disheartening that some AI companies are now setting the stage for enshittification. I hope we can collectively dodge that bullet.
This is what brought me around to doing more agentic coding. I set the task, require tests and the strict linting must pass and then leave it to blow smoke up its own ass about what's going on.
I see glimpses scrolling past of all the conversational language that used to frustrate me so much when using a chat interface and I can just let it flow on past.
I come back when its made everything pass, my life is better now.
The previous match was still too clever for clippy’s type analysis, so I’ve simplified it into a direct borrowed-pattern form and I’m validating once more. I’m replacing that wrapper with a direct, non-transparent error variant so the enum stays explicit and doesn’t rely on a generic anyhow bridge.
None of the popular providers have real moat, which is very much problematic for OpenAI and Anthropic; their operating margins are deeply red and they do not have the same reserves that FAANG/MANGA has. FAANG/MANGA does the same to kill fair market competition in the long run.
This is one of the ways in which model providers are attempting to artificially create moat where there is none.
There are only functional objections to this.
This is fundamentally antitrust material. However, this is “acceptable” in contemporary USA because the FTC has been gutted to follow Trumponomics.
Also, how to achieve these levels when switching mid conv? Don’t you effectively need to read the tokens per model switch?
I do, quite regularly. Because different models have different strength (for example when producing text for live presentations based on the text for a reading deck). Or when it comes to other aspects of the work. I regularly switch between open wheight models and closed models.
I know, I am a tiny minority here. And this behavior only ever started a few weeks ago. But it quickly became a habbit, to CTRL-L in pi and change the model.
you should fork or handoff or start a new one and tell the new one to read the old session.jsonl
This one is not particularly limited to LLMs. The entire cloud infrastructure is built around the trust me bro model.
I start out with what I want to talk about, and then make an initial draft. In this case I also had Sol research all current APIs so I don't have blind spots. I tend to also talk with an AI to see if the structure structure of the post makes sense. I then wrote all sections by hand but used Sol to fix up typos, grammar and punctuation. I also had Sol apply notes and patches that were provided via Discord.
If I find the earlier drafts I can check if it did some more outrageous edits, but I kinda doubt it did. I do know that my workflow of having an LLM to fixes up later is increasingly breaking on SOTA models and I have complained about this before. Normally I now carefully apply fixes that it proposes, but in this case I didn't due to time constraints.
I am curious though why it claims that this is AI generated.
https://github.com/pantoniou/fyai
The idea is that your session data are what's important, and what you need to keep yourself, using a model similar to git.