It seems like this could be easily solved in models that support tool calling by providing them with a tool that takes a token and returns the individual graphemes.
It doesn't seem valuable for the model to memorize the graphemes in each of its tokens.
OpenAI definitely tarnished the name of GPT-5 by allowing these issues to occur. It's clearly a smaller model optimized for cost and speed. Compare it to GPT-4.5 which didn't have these errors but was "too expensive for them".
This is why Anthropic naming system of haiku sonnet and opus to represent size is really nice. It prevents this confusion.
Oh wow, I stare at those model names every day, and I only just now after reading your comment realized what “haiku”, “sonnet”, and “opus” imply about the models! Seems super obvious in retrospect but never thought about it!
The Internet has been poking fun at LLMs failing to count letters for many months. Are the AI companies really living in an echo chamber? They should have implemented a thing to count letters (and to search for palindromes, and…), and just have the LLM pass the request to the count_letters function, as they do for many other tasks already…
> Are the AI companies really living in an echo chamber?
The author tested 12 models, and only one was consistently wrong. More than half were correct 100% of the time.
A better conclusion would be that there’s something in particular wrong with GPT-5 Chat, all the other GPT 5 variants are OK. I wonder what’s different?
I gave it a puzzle recently to test it out "An Ex-cop lost has home, his car and his girlfriend. What did he loose first?"
The AI thought and concluded that he had lost his job first, until I pointed out that it was not the first thing he had lost - which was his umbilical cord, a far better answer, in the AI's opinion.
Which raises many aspects - Can an AI disagree with you? Will AI develop solid out-of-the-box thinking as well as in-the-box thinking, will it grasp applying both for a thru the box thinking and solutions...
After all, we have yet to perfect the teaching of children, so the training of AI, has a long way to go and will get down to quality over quantity, just deciding what is quality and what is not. After all - Garbage in, Garbage out, is probably more important today than it ever was in the history of technology.
The hilarious thing is that LLMs will happily explain the token problem to you, and will even list out exactly which words where will have these problems. Then in the same breath when you ask it about letter counts, it will screw up.
>This is because the transformers are not able to take in or output actual text efficiently. Instead, the text is converted into numerical representations of itself, which is then contextualized to help the AI come up with a logical response. In other words, the AI might know that the tokens “straw” and “berry” make up “strawberry,” but it may not understand that “strawberry” is composed of the letters “s,” “t,” “r,” “a,” “w,” “b,” “e,” “r,” “r,” and “y,” in that specific order. Thus, it cannot tell you how many letters — let alone how many “r”s — appear in the word “strawberry.”
This is a great example. The LLM doesn't know something but it makes up something in it's place. Just because it made up something doesn't mean it's incapable of reasoning.
The thing with LLMs is that they can reason. There's evidence for that. But they can also be creative. And the line between reasoning and creativity at a low level is a bit of a blur as reasoning is a form of inference but so is creativity. So when an LLM reasons or gets creative or hallucinates it's ultimately doing the same type of thing: inference.
For us, we have mechanisms in our brain that allow us to tell the difference most of the time. The LLM does not. That's the fundamental line. And I feel because of this we are literally really close to AGI. A lot of people argue the opposite. They think reasoning and is core to intelligence and a separate concept from creativity and that all LLMs lack reasoning. I disagree.
In fact humans ourselves have trouble separating hallucination from reasoning. Look at religion. Religion permeates our culture but it's basically all hallucinations that we ultimately mistake for reasoning. Right? Ask any christian or muslim, the religions make rational sense to them! They can't tell the difference.
So the key is to give the LLM the ability to know the difference.
Is there some way to build into the transformer, some way to quantify whether something is fact or fiction? Like let's say the answer to a prompt created an inferenced datapoint that's very far far away from a cluster of data. From that we can derive some metric that quantifies how likely the response is based on evidence?
Right? The whole thing is on a big mathematical multidimensional durve. If the inferenced point on the curve is right next to existing data then it must be more likely to be true. If it's far away in some nether region of the curve then it's more likely to be false.
If the LLM can be more self aware and we can build this quantitative metric into the network then use reinforcement learning to kind of have the network be less sure about an answer if it's far away from a cluster of training data points we can likely very much improve the hallucination problem.
Of course I'm sure this is a blunt instrument as even false inferences data can be very close to existing training data. But at least this gives the LLM some level of self awareness of how reliable it's own answer was.
It might help a bit to expand this test to a short phrase. With such a small test the model can be right for the wrong reasons; opening up a bit of space to be wrong in might sharpen the differences.
(My one-off test of the default ChatGPT model, whatever that is, got 'How many b's are there in "Billy Bob beat the record for bounding?"' correct first try, with correct reasoning given.)
perhaps apocryphal but “A single lie discovered is enough to create doubt in every truth expressed.”. Why trust these tools for the hard things when we don't for the 'easy' ones.
i tried with three different models :
- hf.co/lmstudio-community/DeepSeek-R1-Distill-Llama-8B-GGUF:Q8_0
- qwen3:4b-instruct-2507-fp16
- hf.co/ibm-granite/granite-3.3-2b-instruct-GGUF:F16
ollama run qwen3:4b-instruct-2507-fp16
>>> how many b’s are there in blueberry?
Let's break down the word *"blueberry"* step by step to count how many *b's* are in it.
Step 1: Write out the word clearly:
*b l u e b e r r y*
Step 2: Go through each letter one by one:
- b → yes, a *b*
- l → no
- u → no
- e → no
- b → yes, another *b*
- e → no
- r → no
- r → no
- y → no
Step 3: Count the *b's*:
- First *b* at the beginning
- Second *b* after "blue"
So, there are *two b's*.
Final answer: *2* b's in "blueberry".
>>>
ollama run hf.co/ibm-granite/granite-3.3-2b-instruct-GGUF:F16
>>> how many b’s are there in blueberry?
The word "blueberry" contains two 'b's.
(fastest lol, granite models are pretty underated)
r1-distill output was similar to qwen instruct one but it double checked it's thinking part
A while back I encoded the message "Can you read this?" in rot13 (or something very similar) and sent it to chatGPT[. It answered back (yes) in with a sentence in rot13. Why could it do that but not count letters?
21 comments
[ 0.18 ms ] story [ 50.0 ms ] threadIt doesn't seem valuable for the model to memorize the graphemes in each of its tokens.
This is why Anthropic naming system of haiku sonnet and opus to represent size is really nice. It prevents this confusion.
Dev: “What about Bs in blueberry?”
PM: “you’ll need to open a new jira ticket”
The author tested 12 models, and only one was consistently wrong. More than half were correct 100% of the time.
A better conclusion would be that there’s something in particular wrong with GPT-5 Chat, all the other GPT 5 variants are OK. I wonder what’s different?
The AI thought and concluded that he had lost his job first, until I pointed out that it was not the first thing he had lost - which was his umbilical cord, a far better answer, in the AI's opinion.
Which raises many aspects - Can an AI disagree with you? Will AI develop solid out-of-the-box thinking as well as in-the-box thinking, will it grasp applying both for a thru the box thinking and solutions...
After all, we have yet to perfect the teaching of children, so the training of AI, has a long way to go and will get down to quality over quantity, just deciding what is quality and what is not. After all - Garbage in, Garbage out, is probably more important today than it ever was in the history of technology.
This is a great example. The LLM doesn't know something but it makes up something in it's place. Just because it made up something doesn't mean it's incapable of reasoning.
The thing with LLMs is that they can reason. There's evidence for that. But they can also be creative. And the line between reasoning and creativity at a low level is a bit of a blur as reasoning is a form of inference but so is creativity. So when an LLM reasons or gets creative or hallucinates it's ultimately doing the same type of thing: inference.
For us, we have mechanisms in our brain that allow us to tell the difference most of the time. The LLM does not. That's the fundamental line. And I feel because of this we are literally really close to AGI. A lot of people argue the opposite. They think reasoning and is core to intelligence and a separate concept from creativity and that all LLMs lack reasoning. I disagree.
In fact humans ourselves have trouble separating hallucination from reasoning. Look at religion. Religion permeates our culture but it's basically all hallucinations that we ultimately mistake for reasoning. Right? Ask any christian or muslim, the religions make rational sense to them! They can't tell the difference.
So the key is to give the LLM the ability to know the difference.
Is there some way to build into the transformer, some way to quantify whether something is fact or fiction? Like let's say the answer to a prompt created an inferenced datapoint that's very far far away from a cluster of data. From that we can derive some metric that quantifies how likely the response is based on evidence?
Right? The whole thing is on a big mathematical multidimensional durve. If the inferenced point on the curve is right next to existing data then it must be more likely to be true. If it's far away in some nether region of the curve then it's more likely to be false.
If the LLM can be more self aware and we can build this quantitative metric into the network then use reinforcement learning to kind of have the network be less sure about an answer if it's far away from a cluster of training data points we can likely very much improve the hallucination problem.
Of course I'm sure this is a blunt instrument as even false inferences data can be very close to existing training data. But at least this gives the LLM some level of self awareness of how reliable it's own answer was.
GPT-5: "How many times does the letter b appear in blueberry?" - https://news.ycombinator.com/item?id=44832908 - Aug 2025 (321 comments)
(My one-off test of the default ChatGPT model, whatever that is, got 'How many b's are there in "Billy Bob beat the record for bounding?"' correct first try, with correct reasoning given.)
ollama run qwen3:4b-instruct-2507-fp16 >>> how many b’s are there in blueberry? Let's break down the word *"blueberry"* step by step to count how many *b's* are in it.
Step 1: Write out the word clearly: *b l u e b e r r y*
Step 2: Go through each letter one by one:
- b → yes, a *b* - l → no - u → no - e → no - b → yes, another *b* - e → no - r → no - r → no - y → no
Step 3: Count the *b's*:
- First *b* at the beginning - Second *b* after "blue"
So, there are *two b's*.
Final answer: *2* b's in "blueberry".
>>> ollama run hf.co/ibm-granite/granite-3.3-2b-instruct-GGUF:F16 >>> how many b’s are there in blueberry? The word "blueberry" contains two 'b's. (fastest lol, granite models are pretty underated)
r1-distill output was similar to qwen instruct one but it double checked it's thinking part
GPT-5: "How many times does the letter b appear in blueberry?"
https://news.ycombinator.com/item?id=44832908