Let's quote all the CEO's benefiting from bubble spending, is their fake "AI" llm going to blow up the world or take all our jobs!? Find out in this weeks episode!
Well, I think because we know how the code is written, in the sense that humans quite literally wrote the code for it - it's definitely not thinking, and it is literally doing what we asked, based on the data we gave it. It is specifically executing code we thought of. The output of course, we had no flying idea it would work this well.
But it is not sentient. It has no idea of a self or anything like that. If it makes people believe that it does, it is because we have written so much lore about it in the training data.
The author searches for a midpoint between "AIs are useless and do not actually think" and "AIs think like humans," but to me it seems almost trivially true that both are possible.
What I mean by that is that I think there is a good chance that LLMs are similar to a subsystem of human thinking. They are great at pattern recognition and prediction, which is a huge part of cognition. What they are not is conscious, or possessed of subjective experience in any measurable way.
LLMs are like the part of your brain that sees something and maps it into a concept for you. I recently watched a video on the creation of AlexNet [0], one of the first wildly successful image-processing models. One of the impressive things about it is how it moves up the hierarchy from very basic patterns in images to more abstract ones (e. g. these two images' pixels might not be at all the same, but they both eventually map to a pattern for 'elephant').
It's perfectly reasonable to imagine that our brains do something similar. You see a cat, in some context, and your brain maps it to the concept of 'cat', so you know, 'that's a cat'. What's missing is a) self-motivated, goal-directed action based on that knowledge, and b) a broader context for the world where these concepts not only map to each other, but feed into a sense of self and world and its distinctions whereby one can say: "I am here, and looking at a cat."
It's possible those latter two parts can be solved, or approximated, by an LLM, but I am skeptical. I think LLMs represent a huge leap in technology which is simultaneously cooler than anyone would have imagined a decade ago, and less impressive than pretty much everyone wants you to believe when it comes to how much money we should pour into the companies that make them.
This reads like 2022 hype. It's like people stil do not understand that there's a correlation between exaggerating AI's alleged world-threatening capabilities and AI companies' market share value – and guess who's doing the hyping.
He's the GOAT in my opinion for "thinking about thinking".
My own thinking on this is that AI actually IS thinking - but its like the MVB of thinking (minimum viable brain)
I find thought experiments the best for this sort of thing:
- Imagine you had long term memory loss so couldn't remember back very long
You'd still be thinking right?
- Next, imagine you go to sleep and lose consciousness for long periods
You'd still be thinking right?
- Next, imagine that when you're awake, you're in a coma and can't move, but we can measure your brain waves still.
You'd still be thinking right?
- Next, imagine you can't hear or feel either.
You'd still be thinking right?
- Next, imagine you were a sociopath who had no emotion.
You'd still be thinking right?
We're just not used to consciousness without any of the other "baggage" involved.
There are many separate aspects of life and shades of grey when it comes to awareness and thinking, but when you take it down to its core, it becomes very hard to differentiate between what an LLM does and what we call "thinking". You need to do it by recognizing the depths and kinds of thoughts that occur. Is the thinking "rote", or is something "special" going on. This is the stuff that Hofstadter gets into(he makes a case for recursion and capability being the "secret" piece - something that LLMs certainly have plumbing in place for!)
BTW, I recommend "Surfaces and Essences" and "I am a strange loop" also by Hofstadter. Good reads!
Many people who object to the idea that current-generation AI is thinking do so only because they believe AI is not "conscious"... but there is no known law in the universe requiring that intelligence and consciousness must always go together. With apologies to René Descartes[a], intelligence and consciousness are different.
Intelligence can be verified and quantified, for example, with tests of common sense and other knowledge.[b] Consciousness, on the other hand, is notoriously difficult if not impossible to verify, let alone quantify. I'd say AI is getting more intelligent, and more reliable, in fits and starts, but it's not necessarily becoming conscious.
The definitions of all these words have been going back and forward and never reached any 100% consensus anyways, so what one person understands of "thinking", "conscious", "intelligence" and so on seems to be vastly different from another person.
I guess this is why any discussion around this ends up with huge conversations, everyone is talking from their own perspective and understanding, while others have different ones, and we're all talking past each other.
There is a whole field trying to just nail down what "knowledge" actually is/isn't, and those people haven't agreed with each other for the duration of hundreds of years, I'm not confident we'll suddenly get a lot better at this.
I guess ultimately, regardless of what the LLMs do, does it matter? Would we understand them better/worse depending on what the answer would be?
"However exactly you divide up the AI landscape, it is important to distinguish what I call AI-as-engineering from what I call AI-as-cognitive-science. AI-as-engineering isn’t particularly concerned with mimicking the precise way in which the human mind-brain does distinctively human things. The strategy of engineering machines that do things that are in some sense intelligent, even if they do what they do in their own way, is a perfectly fine way to pursue artificial intelligence. AI-as-cognitive science, on the other hand, takes as its primary goal that of understanding and perhaps reverse engineering the human mind.
[...]
One reason for my own skepticism is the fact that in recent years the AI landscape has come to be progressively more dominated by AI of the newfangled 'deep learning' variety [...] But if it’s really AI-as-cognitive science that you are interested in, it’s important not to lose sight of the fact that it may take a bit more than our cool new deep learning hammer to build a humanlike mind.
[...]
If I am right that there are many mysteries about the human mind that currently dominant approaches to AI are ill-equipped to help us solve, then to the extent that such approaches continue to dominate AI into the future, we are very unlikely to be inundated anytime soon with a race of thinking robots—at least not if we mean by “thinking” that peculiar thing that we humans do, done in precisely the way that we humans do it."
I think we are getting to point where we are trying to figure how important is human experience to intelligence.
Things we do like sleep, meditate, have fun, listen to music etc. do they add to our intelligence? Do they help us have a consistent world model that we build on everyday?
Will we be able to replicate this is in a artificial neural net which is extremely smart in spurts but does not "enjoy" the world it operates in?
If we assembled the sum total of all published human knowledge on a storage medium and gave a computer the ability to search it extremely well in order to answer any question falling within its domain, there, you would have a Nobel Prize beating "A.I".
But this is as "earth-shattering" (/s) as the idea that human knowledge can be stored outside the brain (on paper, flash drives, etc), or that the answer to complex questions can be deterministic.
And then there is the fact that this Noble winner beating "A.I" is highly unlikely to propound any ground-breaking novel ways of thinking and promote and explain it to general acceptance.
TFA is a part of what seems like a never-ending series about concepts that lack a useful definition.
"Thinking" and "intelligence" have no testable definition or specification, therefore it's a complete waste of time to suppose that AI is thinking or intelligent.
The debate around whether or not transformer-architecture-based AIs can "think" or not is so exhausting and I'm over it.
What's much more interesting is the question of "If what LLMs do today isn't actual thinking, what is something that only an actually thinking entity can do that LLMs can't?". Otherwise we go in endless circles about language and meaning of words instead of discussing practical, demonstrable capabilities.
Helpful to remember that we humans often say "I think" to mean "I am fairly confident based on my hunch", and in that sense AI is very good at hunching.
This is merely a debate about what it means to "think." We didn't really previously need to disambiguate thinking / intelligence / consciousness / sentience / ego / identity / etc.
Now, we do. Partly because of this we don't have really well defined ways to define these terms and think about. Can a handheld calculator think? Certainly, depending on how we define "think."
So much of the debate of whether AI can think or not reminds me of this scene from The Next Generation: https://youtu.be/ol2WP0hc0NY
LLMs hit two out of the three criteria already - self awareness and intelligence, but we're in a similar state where defining consciousness is such a blurry metric. I feel like it wont be a binary thing, it'll be a group decision by humanity. I think it will happen in the next decade or two, and regardless of the outcome I'm excited I'll be alive to see it. It'll be such a monumentous achievement by humanity. It will drastically change our perspective on who we are and what our role is in the universe, especially if this new life form surpasses us.
I think the challenge with many of these conversations is that they assume consciousness emerges through purely mechanical means.
The “brain as a computer” metaphor has been useful in limited contexts—especially for modeling memory or signal processing; but, I don’t think it helps us move forward when talking about consciousness itself.
Penrose and Hameroff’s quantum consciousness hypothesis, while still very speculative, is interesting precisely because it suggests that consciousness may arise from phenomena beyond classical computation. If that turns out to be true, it would also mean today’s machines—no matter how advanced—aren’t on a path to genuine consciousness.
That said, AI doesn’t need to think to be transformative.
Steam engines weren’t conscious either, yet they reshaped civilization.
Likewise, AI and robotics can bring enormous value without ever approaching human-level awareness.
We can hold both ideas at once: that machines may never be conscious, and still profoundly useful.
I think something that's missing from AI is the ability humans have to combine and think about ANY sequence of patterns as much as we want.
A simple example is say I think about a sequence of "banana - car - dog - house". I can if I want to in my mind, replace car with tree, then replace tree with rainbow, then replace rainbow with something else, etc... I can sit and think about random nonsense for as long as I want and create these endless sequences of thoughts.
Now I think when we're trying to reason about a practical problem or whatever, maybe we are doing pattern recognition via probability and so on, and for a lot of things it works OK to just do pattern recognition, for AI as well.
But I'm not sure that pattern recognition and probability works for creating novel interesting ideas all of the time, and I think that humans can create these endless sequences, we stumble upon ideas that are good, whereas an AI can only see the patterns that are in its data. If it can create a pattern that is not in the data and then recognize that pattern as novel or interesting in some way, it would still lack the flexibility of humans I think, but it would be interesting nevertheless.
The New Yorker is owned by Advance Publications, which also owns Conde Nast. "Open" "AI" has struck a deal with Conde Nast to feed SearchGPT and ChatGPT.
This piece is cleverly written and might convince laypeople that "AI" may think in the future. I hope the author is being paid handsomely, directly or indirectly.
113 comments
[ 4.7 ms ] story [ 92.3 ms ] threadhttps://news.ycombinator.com/user?id=jsomers
But it is not sentient. It has no idea of a self or anything like that. If it makes people believe that it does, it is because we have written so much lore about it in the training data.
What I mean by that is that I think there is a good chance that LLMs are similar to a subsystem of human thinking. They are great at pattern recognition and prediction, which is a huge part of cognition. What they are not is conscious, or possessed of subjective experience in any measurable way.
LLMs are like the part of your brain that sees something and maps it into a concept for you. I recently watched a video on the creation of AlexNet [0], one of the first wildly successful image-processing models. One of the impressive things about it is how it moves up the hierarchy from very basic patterns in images to more abstract ones (e. g. these two images' pixels might not be at all the same, but they both eventually map to a pattern for 'elephant').
It's perfectly reasonable to imagine that our brains do something similar. You see a cat, in some context, and your brain maps it to the concept of 'cat', so you know, 'that's a cat'. What's missing is a) self-motivated, goal-directed action based on that knowledge, and b) a broader context for the world where these concepts not only map to each other, but feed into a sense of self and world and its distinctions whereby one can say: "I am here, and looking at a cat."
It's possible those latter two parts can be solved, or approximated, by an LLM, but I am skeptical. I think LLMs represent a huge leap in technology which is simultaneously cooler than anyone would have imagined a decade ago, and less impressive than pretty much everyone wants you to believe when it comes to how much money we should pour into the companies that make them.
[0] https://www.youtube.com/watch?v=UZDiGooFs54
He's the GOAT in my opinion for "thinking about thinking".
My own thinking on this is that AI actually IS thinking - but its like the MVB of thinking (minimum viable brain)
I find thought experiments the best for this sort of thing:
- Imagine you had long term memory loss so couldn't remember back very long
You'd still be thinking right?
- Next, imagine you go to sleep and lose consciousness for long periods
You'd still be thinking right?
- Next, imagine that when you're awake, you're in a coma and can't move, but we can measure your brain waves still.
You'd still be thinking right?
- Next, imagine you can't hear or feel either.
You'd still be thinking right?
- Next, imagine you were a sociopath who had no emotion.
You'd still be thinking right?
We're just not used to consciousness without any of the other "baggage" involved.
There are many separate aspects of life and shades of grey when it comes to awareness and thinking, but when you take it down to its core, it becomes very hard to differentiate between what an LLM does and what we call "thinking". You need to do it by recognizing the depths and kinds of thoughts that occur. Is the thinking "rote", or is something "special" going on. This is the stuff that Hofstadter gets into(he makes a case for recursion and capability being the "secret" piece - something that LLMs certainly have plumbing in place for!)
BTW, I recommend "Surfaces and Essences" and "I am a strange loop" also by Hofstadter. Good reads!
Intelligence can be verified and quantified, for example, with tests of common sense and other knowledge.[b] Consciousness, on the other hand, is notoriously difficult if not impossible to verify, let alone quantify. I'd say AI is getting more intelligent, and more reliable, in fits and starts, but it's not necessarily becoming conscious.
---
[a] https://en.wikipedia.org/wiki/Cogito%2C_ergo_sum
[b] For example, see https://arxiv.org/abs/2510.18212
I guess this is why any discussion around this ends up with huge conversations, everyone is talking from their own perspective and understanding, while others have different ones, and we're all talking past each other.
There is a whole field trying to just nail down what "knowledge" actually is/isn't, and those people haven't agreed with each other for the duration of hundreds of years, I'm not confident we'll suddenly get a lot better at this.
I guess ultimately, regardless of what the LLMs do, does it matter? Would we understand them better/worse depending on what the answer would be?
The Robots Are Coming
https://www.bostonreview.net/articles/kenneth-taylor-robots-...
"However exactly you divide up the AI landscape, it is important to distinguish what I call AI-as-engineering from what I call AI-as-cognitive-science. AI-as-engineering isn’t particularly concerned with mimicking the precise way in which the human mind-brain does distinctively human things. The strategy of engineering machines that do things that are in some sense intelligent, even if they do what they do in their own way, is a perfectly fine way to pursue artificial intelligence. AI-as-cognitive science, on the other hand, takes as its primary goal that of understanding and perhaps reverse engineering the human mind.
[...]
One reason for my own skepticism is the fact that in recent years the AI landscape has come to be progressively more dominated by AI of the newfangled 'deep learning' variety [...] But if it’s really AI-as-cognitive science that you are interested in, it’s important not to lose sight of the fact that it may take a bit more than our cool new deep learning hammer to build a humanlike mind.
[...]
If I am right that there are many mysteries about the human mind that currently dominant approaches to AI are ill-equipped to help us solve, then to the extent that such approaches continue to dominate AI into the future, we are very unlikely to be inundated anytime soon with a race of thinking robots—at least not if we mean by “thinking” that peculiar thing that we humans do, done in precisely the way that we humans do it."
Things we do like sleep, meditate, have fun, listen to music etc. do they add to our intelligence? Do they help us have a consistent world model that we build on everyday?
Will we be able to replicate this is in a artificial neural net which is extremely smart in spurts but does not "enjoy" the world it operates in?
I don't even know what this means.
If we assembled the sum total of all published human knowledge on a storage medium and gave a computer the ability to search it extremely well in order to answer any question falling within its domain, there, you would have a Nobel Prize beating "A.I".
But this is as "earth-shattering" (/s) as the idea that human knowledge can be stored outside the brain (on paper, flash drives, etc), or that the answer to complex questions can be deterministic.
And then there is the fact that this Noble winner beating "A.I" is highly unlikely to propound any ground-breaking novel ways of thinking and promote and explain it to general acceptance.
"Thinking" and "intelligence" have no testable definition or specification, therefore it's a complete waste of time to suppose that AI is thinking or intelligent.
What's much more interesting is the question of "If what LLMs do today isn't actual thinking, what is something that only an actually thinking entity can do that LLMs can't?". Otherwise we go in endless circles about language and meaning of words instead of discussing practical, demonstrable capabilities.
Now, we do. Partly because of this we don't have really well defined ways to define these terms and think about. Can a handheld calculator think? Certainly, depending on how we define "think."
Eh... Plato would like a word with you. Philosophy has been specifically trying to disentangle all that for millennia. Is this a joke?
And the LLM part of our intelligence isn't really thinking.
And some people out there have a very, very small "unknown black box".
LLMs hit two out of the three criteria already - self awareness and intelligence, but we're in a similar state where defining consciousness is such a blurry metric. I feel like it wont be a binary thing, it'll be a group decision by humanity. I think it will happen in the next decade or two, and regardless of the outcome I'm excited I'll be alive to see it. It'll be such a monumentous achievement by humanity. It will drastically change our perspective on who we are and what our role is in the universe, especially if this new life form surpasses us.
The “brain as a computer” metaphor has been useful in limited contexts—especially for modeling memory or signal processing; but, I don’t think it helps us move forward when talking about consciousness itself.
Penrose and Hameroff’s quantum consciousness hypothesis, while still very speculative, is interesting precisely because it suggests that consciousness may arise from phenomena beyond classical computation. If that turns out to be true, it would also mean today’s machines—no matter how advanced—aren’t on a path to genuine consciousness.
That said, AI doesn’t need to think to be transformative.
Steam engines weren’t conscious either, yet they reshaped civilization.
Likewise, AI and robotics can bring enormous value without ever approaching human-level awareness.
We can hold both ideas at once: that machines may never be conscious, and still profoundly useful.
It's an illusion that's good enough that our brains accept it and it's a useful tool.
Now I think when we're trying to reason about a practical problem or whatever, maybe we are doing pattern recognition via probability and so on, and for a lot of things it works OK to just do pattern recognition, for AI as well.
But I'm not sure that pattern recognition and probability works for creating novel interesting ideas all of the time, and I think that humans can create these endless sequences, we stumble upon ideas that are good, whereas an AI can only see the patterns that are in its data. If it can create a pattern that is not in the data and then recognize that pattern as novel or interesting in some way, it would still lack the flexibility of humans I think, but it would be interesting nevertheless.
This piece is cleverly written and might convince laypeople that "AI" may think in the future. I hope the author is being paid handsomely, directly or indirectly.