If you asked me last year(2025) I would have still said LLMs are a silly toy.
As of Jan 2026 I have come to accept that LLMs are at least part of the puzzle of how intelligence works. They are at this point better than the majority of humans at various intellectual tasks. It may not be or ever be a 1:1 but good enough ran the world already before llms.
There is not even a formal definition of what intelligence is so saying LLM's are intelligent can't even be "right/wrong". Its just arguing semantics and definitions.
> Ask it "what is the capital of the state containing Dallas" and you can observe, in order:
> the Dallas feature goes active,
> which causes the Texas feature to light up,
> which then causes Austin to light up.
> It seems fairly clear that this is tracing semantic relationships between high-level concepts — and in doing so, performing a kind of pseudo-symbolic inference, similar to what some philosophers would describe as "higher reasoning."
Uhhh no reasoning is required for Austin to follow Texas after Dallas, let alone "higher reasoning".
I don't recall being promised a black box. Are we certain llms didn't write this article and just came up with one of those It's Not Whatever-random-thing pithy zinger kind of things that they're prone to?
It's nice to see sparse interpretable LLMs being made.
This is similar to factor rotation in factor analysis (or PCA). A varimax rotation, for example, can produce an equivalent factor analysis with sparse loadings, and which is generally more interpretable. Fortunately for us the world is not just a complete mess, and sparse loadings can often be found. There seem to be "natural" concepts that we have observed rather than invented.
(Many examples in other simple machine learning methods too, I am sure.)
LLM written article. It's also not accurate; the fact that language models have human-interpretable representations and neurons has been known since BERT.
Circuits research also does not come from Anthropic. Mech interp is a huge field in academia and most of the core circuit analysis papers were from OpenAI/GDM/academia. However, Anthropic tends to produce a lot of blog posts where they draw poorly supported but hype-able analogies between LLMs and biological intelligence. It's wild.
For a better understanding of mech interp and circuits, including what we actually do know about LLM internals, I would recommend reading this paper: https://arxiv.org/pdf/2501.16496
18 comments
[ 2.9 ms ] story [ 47.5 ms ] threadAs of Jan 2026 I have come to accept that LLMs are at least part of the puzzle of how intelligence works. They are at this point better than the majority of humans at various intellectual tasks. It may not be or ever be a 1:1 but good enough ran the world already before llms.
There is not even a formal definition of what intelligence is so saying LLM's are intelligent can't even be "right/wrong". Its just arguing semantics and definitions.
> the Dallas feature goes active,
> which causes the Texas feature to light up,
> which then causes Austin to light up.
> It seems fairly clear that this is tracing semantic relationships between high-level concepts — and in doing so, performing a kind of pseudo-symbolic inference, similar to what some philosophers would describe as "higher reasoning."
Uhhh no reasoning is required for Austin to follow Texas after Dallas, let alone "higher reasoning".
This is really grasping as straws
This was the mantra of applied machine learning c. 2010 - 2024 for anyone paying attention. No longer the case.
This is similar to factor rotation in factor analysis (or PCA). A varimax rotation, for example, can produce an equivalent factor analysis with sparse loadings, and which is generally more interpretable. Fortunately for us the world is not just a complete mess, and sparse loadings can often be found. There seem to be "natural" concepts that we have observed rather than invented.
(Many examples in other simple machine learning methods too, I am sure.)
Biology? Anthropic really needs to stop anthropomorphizing these things so much. I'm with Dijkstra on this one.
I know they do it as a sort of marketing but still...
I didn't see him mentioned in the article and I can't recall what he ever said about biology before..
Circuits research also does not come from Anthropic. Mech interp is a huge field in academia and most of the core circuit analysis papers were from OpenAI/GDM/academia. However, Anthropic tends to produce a lot of blog posts where they draw poorly supported but hype-able analogies between LLMs and biological intelligence. It's wild.
For a better understanding of mech interp and circuits, including what we actually do know about LLM internals, I would recommend reading this paper: https://arxiv.org/pdf/2501.16496