It's sad how many people are falling for the narrative that there's more at play here than predict-next-token and some kind of emergent intelligence is happening.
No, that is just your interpretation of what you see as something that can't possibly be just token prediction.
And yet it is. It's the same algorithm noodling over incredible amounts of tokens.
And that's exactly the explanation: People regularly underestimate how much training data is being used for LLMs. They contain everything about writing a compiler, toy examples, full examples, recommended structure yadda yadda yadda.
I love working with Claude and it regularly surprises me but that doesn't mean I think it is intelligent.
I'm concerned too, and awareness of this externality is what stays my hand, even if I set aside other concerns with LLMs. Mere efficiency, efficiency that we were fine without, and one we haven't built out infrastructure around it yet, doesn't seem worth the negative impact.
Sounds like most of that blog was written by AI. I'm getting tired of reading it. I had patience for reading the peculiarities of human expression. But that just went on and on and was quite redundant and I'm left feeling my time and attention were wasted by someone who couldn't even be bothered to write all the words I took time to read.
6 comments
[ 3.2 ms ] story [ 30.7 ms ] threadNo, that is just your interpretation of what you see as something that can't possibly be just token prediction.
And yet it is. It's the same algorithm noodling over incredible amounts of tokens.
And that's exactly the explanation: People regularly underestimate how much training data is being used for LLMs. They contain everything about writing a compiler, toy examples, full examples, recommended structure yadda yadda yadda.
I love working with Claude and it regularly surprises me but that doesn't mean I think it is intelligent.
> And yet it is. It's the same algorithm noodling over incredible amounts of tokens.
That's all fine and dandy, until your token prediction algorithm tries to blackmail you[1] or harass you publicly[2]
[1] https://www.bbc.com/news/articles/cpqeng9d20go
[2] https://www.pcgamer.com/software/ai/a-human-software-enginee...
I honestly think we’re gonna ignore it like we do with plastic. Another technology too ubiquitous, cheap and convenient to trade off.