35 comments

[ 0.23 ms ] story [ 59.2 ms ] thread
[flagged]
> a unique message ID in the system prompt broke the entire KV cache. Since the cache requires byte-exact matches, that changing ID forced a full re-compute on every turn, turning warm contexts into cold fills.

This is (part of) the same problem that initially lead Anthropic to ban non-Claude Code clients from using the subsidized subscription: A full to-the-second datetime stamp in the system prompts of OpenCode, and I believe Pi as well, invalidated the caches, making this a very expensive use of their compute very quickly.

They even had Anthropic employees submit PRs (or maybe just open issues, I’d have to check) to these other clients/harnesses because the cache misses were hitting them so hard.

fascinating! That's a useful thing to know.
> The most counter-intuitive bug was that a unique message ID in the system prompt broke the entire KV cache.

How is that counter-intuitive? You changed the key, so it didn't return a value. Thats a KV cache working as intended.

Impressive debugging skills, and thank you for the benchmarks. Now I'm wondering if mlx-engine / mlx-lm have these bugs too.
I appreciate the amount of detail in the post, I think it's a useful addition to the space.

That said, I have to read LLM output all day all the time, and I would implore you to take the time to explore your own voice a bit more.

> Two separate things then happened, and it is worth keeping them apart.

Is one of those phrases claude spits out nonstop.

I second this, I read too much AI slop already so when something triggers that part of my brain, at this stage I immediatley lose the capacity to engage outside of work, largly because it feels like work. Scrolling through, this article looks like it holds useful info. Info i'd likely love to engage with, but realistically I cannot force myself to spend my weekend reading more ai outout, even if human seeded.
Fair enough, nothing more infuriating than the sycophantic way LLMs write.

But I’d definitely be dishonest if I said I’d stop using LLMs to tidy my writing (out of principle or otherwise), my hold on the English language has seriously degenerated over the last eight years since I pivoted into IT from customer facing roles.

It will continue to degenerate if you use LLMs to do the cleanup work.
(comment deleted)
Nonstop. There is too much wholesale reliance on LLMs to generate content. When I use LLMs for scientific writing, I approach it differently: I write the paragraph dirty, then ask an LLM to perform a minor rewrite for "clarity", using Claude's now retired Concise Mode. This has been a great approach for scientific writing. It tends to prevent these overly used turns of phrase, it makes sure that the writing is making the points I want to make, and shortens writing time by cleaning up the dirty edges of my grammar (especially since my writing can tend towards convoluted constructions). More artistic/creative writing, I'd probably not use it at all, because then, it's usually (for me) about rthym and emotional flow.

BTW It IS an effective rhetorical phrase, but given it's ubiquity in Claude's output, I have to avoid it.

Ahhh yeah, I’m not going to sit here and say I don’t use Claude to clean up my writing (ie make it actually coherently laid out). In all honestly I tend to write in a rambling stream of consciousness style across random scrap markdown files (ha), but point is taken.
> In all honestly I tend to write in a rambling stream of consciousness style

At this point that'd be a welcome respite from every single blog post being written in the same exact AI tone.

I was thinking to myself as I was reading the post that here we go again - it's like how much can a man take. It's not that it's unclear... because it is easy to read. I also long for regular old human writing, warts and all.
I don't mind reading stream of consciousness, thats how I think anyway. Not everything you write has to be tight and polished, it just has to have value to the reader one way or the other.

Try using claude to suggest structural improvements, but take the time to implement those yourself. Don't use claude to make the changes directly. You'll find that what you want to say changes when you have to write it out.

I caught the title "The real work..." labelizing things in a sort of weird phrasing like this is the one I've seen a lot.
Thats honestly a load bearing phrase, it truely wires clean sentences together, would be a footgun with an unknown blast radius if you didn't use it.
I wonder how many readers turned to claude to summarize TFA.
I was triggered by the "Honest Numbers" section
Sorry but I'm not reading an AI slop article no matter how pertinent or interesting the subject is. You want my attention? Earn it. Write it yourself.
It's fine, if you don't want to read it you don't have to read it, nobody's forcing you.
I'd rather have a ai-written article then no article at all. I think this is a real struggle. Writing an article takes time that could be spent on improving the product. Would I prefer a human article, yes. But if the alternative is "no article", then that's how it is.
Do you have any rough stats on how many total tokens per day you end up using with this setup?
(comment deleted)
I'm not sure I'd call 1.5min processing compared to 3-5 min processing as "interactive".
I pointed my Claude/Retort evaluator at this blog post and it ran an experiment to test what it found. We got better cache hit refill but it didn’t improve the overall coding results on my M5Pro/64GB system. Results in experiment-24 at GitHub.com/adrianco/retort
People are rough here.

Thanks for your work, thanks for sharing, thanks for the fork.

Is this issue specific to 122B or will it lurk in other 3.5 r 3.6 models? I run a 9B model locally so while this fix doesn't help me I'm wondering if it's general?