Glad to see Searle's Chinese Room mentioned early on in the paper. "Syntax is not sufficient for semantics," no matter how much compute we throw at the problem.
My very amateur view is that until the underlying compute architecture and substrate resembles artificial biology more than silicon, we wont get there.
The latest advances in AI have given me even more appreciation of biology and evolution. It's incredible what the human brain can do with about 20 watts of power, barely enough to power a lightbulb, in comparison to what it takes to run even our most basic LLM models.
The paper makes a huge assumption that only thermodynamic constitutions can produce consciousness. The assumptions seems completely unsubstantiated given that thermodynamics are just states and states are replicable. The whole Chinese Room idea is pure sophistry as well. Both Dennett and Hofstadter address it quite well in Consciousness Explained and I am a Strange Loop respectively.
You know that Dennett and Hofstadter aren’t the beginning and end of Philosophy of Mind, right? Calling Searle’s Room “complete sophistry” is hilariously misguided, considering the vast majority of academic philosophers consider it valid: https://survey2020.philpeople.org/survey/results/5002#
Consciousness is a property of humans biology - and quite clearly not a requisite to intelligence.
I say clearly as at some point we reach proof by construction. As in, we already built intelligence because the system already completes tasks that require intelligence.
We are so far into what would have been science fiction five years ago and the goal posts have moved so far.
For anyone who disagrees, I challenge you to prove deep learning systems cannot solve <task with specific outcome humans can solve but not AI> given sufficient data and compute.
I think the strongest sign we have true intelligence already is no one has built any benchmark that AI cannot solve.
Yes, our current robotics lags AI, so we don’t have the equivalent of the human body to give our deep learning systems. Thus, it’s expected AI will be limited in physical scenarios.
Second, hallucinations are present in humans. We are highly biased to ignore all the misspoken words in everyday life as we have error correction built into normal conversations. How often do you have to have someone repeat or rephrase something?
It just doesn’t make sense to me.
It’s like there are people out there whose belief systems are incompatible with this tech existing.
Sure, it has limitations due to training data. It has limitations with no physical body. It cannot combine training and inference the same way a human does. But none of those are measures of intelligence or required to be intelligent.
>no one has built any benchmark that AI cannot solve.
Sure, but people have built benchmarks that no AI constructed before the benchmark was released can solve. If I know the answer to a benchmark problem, I can construct an "AI" that can solve it on a note card.
It's the pervasive theme in the book, but never really given a conceptual grounding further than "this sort of looks like recursion or can be modelled circularly so it's a strange loop". The vagueness of it reveals itself as being "more intuitive", because a vaguer pattern will have more matches. I don't remember Hofstadter digressing on whether these loops work "in reverse" either, which is sort of what the author here is denying. Basically positing that f doesn't have a well-defined inverse.
I'm partial to bioinformatics as per Pauline Hogeweg's definition; which explicitly has computation as a property of life.
This approach actually makes testable (and tested) scientific predictions.
This makes Searle-derived papers super-weird for me; since from my perspective they seem to disprove the existence of life. (and it makes the name of the philosophy "biological naturalism" very ironic to me :-P )
(for extra irony, Turing actually went into biology late in his life. See: Turing 1952 "The Chemical Basis of Morphogenesis" )
I'm disappointed that Searle's paper is still influential, at least out in the general culture. It's nonsense, and at face value, would disprove consciousness in humans unless you accept some mystic indefinable soul into the mix. Or quantum magic, which is just as mystic.
From my observations, there are generally four camps in the machine consciousness discussion:
1. People who haven't really thought about it, and assume they're conscious because they talk like a human.
2. People who haven't really thought about it, and assume they can't be conscious because humans are obviously somehow special. This appears to be the largest group, and is linked to our religiously rooted culture in which human exceptionalism is the default.
Those first two groups comprise the majority of people, and are not worth engaging with.
3. People who have thought about it, and came to the conclusion that they might be conscious, usually for computationalism/functionalism reasons. This is the group that I place myself in.
4. People who have thought about it, and came to the conclusion that they can't be conscious, usually for biological naturalist reasons. This seems to be the predominant group on Hacker News (among those who discuss it).
I don't feel like I am in either 4 of those camps or that I'm part of camp 4, but the camp 4 problem is not the important one. My thought on this is that intelligence != consciousness and even brain != consciousness. Consciousness is the experience, consciousness is what you see, hear, feel, in the moment of it. It's the experience. It does not require any thought. In fact, if you look at Buddhist teachings, they teach the very opposite, they teach that the thinking mind is in fact an obstacle to experiencing consciousnesses fully, that it's only a sense, a tool (like smell, touch, vision, hearing). My bet is that a cat, or a dog has the experience of awareness the same way we have (although you can't never be too sure, even about another human being - look up "philosophical zombie").
Obviously, language-driven thought is not a requirement for consciousness, not just in other animals, but even in humans. The thinking mind takes a secondary role in ordinary daily human life. The truth is that a human being behaves the way they do is not because of thoughts, but because of conditioning (the thoughts are not the primary driver of decisions, actions and behavior). The 99% of the action and responses are trained, the thoughts that we have are also part of this conditioning (most thoughts are unconscious and they are inter-wired with the behaviors, even a seemingly conscious self-reflection thought can be an automated pavlovian trigger). For example, one may think that they get up and go to work because they have a thought "I have to get up, now I am going to go to work", this is an illusion and complete misunderstanding of what consciousness is. Or one can have a psychological insight about oneself, if it's repeated and follows a behavior consistently, the very thought is just the equivalent of whistle-salivation. The thinking mind gives us that 1% to self-reflect, adjust our behavior, learn, predict the future and that differentiates us from other mammals, it's a powerful tool, but just a tool, but it should not be confused with consciousness and it should not be confused with the mind as a whole (in the materialistic sense). The way our brain functions is anything but like an AI agent. And what is consciousness? It's not the thinking mind. It's the experience. It's the direct perception of the senses. The consciousness is what is seen, heard, smelled, touched, thought (the experience of having a thought) in the moment. When you practice meditation, you get to discover the consciousness directly by becoming separated from the thinking thread. The thinking thread becomes more like an external tool, like a computer inside you and you realize directly that it's just a part of the cognitive faculty that makes you navigate your life, not the entire thing.
The LLM (and the harnesses) as built right now merely simulate the tool (the thinking mind). It's not that because this is some code ran on a beefy, but regular piece of tech invented in 20th century you may have at your desk that it does not have awareness (that's also a good argument), but because the way they function and operate is nothing like human (or mammalian) brain, then why would you think that regular code running on a regular PC could gain awareness? My point is that there's no similarity argument, LLMs, despite all their incredible capabilities (to threaten our jobs), are not remotely similar to the way our brain works.
Secondly, even if someone built an artificial brain made of whatever that simulates the biological structure, because of the philosophical zombie problem (the fact that there's no way to scientifically observe consciousness), you could never be too sure if a key ingredient was not missing and you are looking at an NPC. The consciousness is not a property of the physical brain, it's literally immaterial, it's the direct experience of the senses. You can make an optimistic assumption that every person and animal experiences consciousness t...
5. People who has a financial interest in making sure that any eventual AGI isn't granted any kind of rights and continue to be exploited as an inanimate "thing", not as a "being", no matter the actual characteristics of this hypothetical AGI entity.
I mean, take a look a this language in the paper [0]
> This realization pulls the field of AI safety out of the welfare trap. It allows us to focus entirely on the concrete risks of anthropomorphism, treating AGI as a powerful, but inherently non-sentient tool.
This reads as someone that started with this conclusion, and then built an argument to support it.
> To fully understand the difference between
the embodied robot running an algorithm on a chip and the biological mapmaker, we need to remember
that for the latter, subjective experience is a given, not because of abstract information processing, but
because of a specific, metabolically constituted physical reality.
Total drivel. Consciousness in biological systems is "a given" because of metabolism?
I think the question goes more into ourselves as it goes into AI, we don’t know exactly how our own intelligence and conciousness works and therefore it’s very tough to impossible to compare to AI intelligence and conciousness.
Are we just autocomplete machines with sufficient enough variable pseudo-randomized input?
19 comments
[ 0.17 ms ] story [ 41.6 ms ] threadMy very amateur view is that until the underlying compute architecture and substrate resembles artificial biology more than silicon, we wont get there.
The latest advances in AI have given me even more appreciation of biology and evolution. It's incredible what the human brain can do with about 20 watts of power, barely enough to power a lightbulb, in comparison to what it takes to run even our most basic LLM models.
I say clearly as at some point we reach proof by construction. As in, we already built intelligence because the system already completes tasks that require intelligence.
We are so far into what would have been science fiction five years ago and the goal posts have moved so far.
For anyone who disagrees, I challenge you to prove deep learning systems cannot solve <task with specific outcome humans can solve but not AI> given sufficient data and compute.
I think the strongest sign we have true intelligence already is no one has built any benchmark that AI cannot solve.
Yes, our current robotics lags AI, so we don’t have the equivalent of the human body to give our deep learning systems. Thus, it’s expected AI will be limited in physical scenarios.
Second, hallucinations are present in humans. We are highly biased to ignore all the misspoken words in everyday life as we have error correction built into normal conversations. How often do you have to have someone repeat or rephrase something?
It just doesn’t make sense to me.
It’s like there are people out there whose belief systems are incompatible with this tech existing.
Sure, it has limitations due to training data. It has limitations with no physical body. It cannot combine training and inference the same way a human does. But none of those are measures of intelligence or required to be intelligent.
> Consciousness is a property of humans biology
You're assuming consciousness is a product of biology rather than attracted to biology.
Sure, but people have built benchmarks that no AI constructed before the benchmark was released can solve. If I know the answer to a benchmark problem, I can construct an "AI" that can solve it on a note card.
> Consciousness is a property of humans biology
Just because we only observed something in human biology, doesn't mean that it can't be found elsewhere.
I mean being water based is also a property of human biology. They share this property with other things like lube and chicken soup.
This approach actually makes testable (and tested) scientific predictions.
This makes Searle-derived papers super-weird for me; since from my perspective they seem to disprove the existence of life. (and it makes the name of the philosophy "biological naturalism" very ironic to me :-P )
(for extra irony, Turing actually went into biology late in his life. See: Turing 1952 "The Chemical Basis of Morphogenesis" )
1. People who haven't really thought about it, and assume they're conscious because they talk like a human.
2. People who haven't really thought about it, and assume they can't be conscious because humans are obviously somehow special. This appears to be the largest group, and is linked to our religiously rooted culture in which human exceptionalism is the default.
Those first two groups comprise the majority of people, and are not worth engaging with.
3. People who have thought about it, and came to the conclusion that they might be conscious, usually for computationalism/functionalism reasons. This is the group that I place myself in.
4. People who have thought about it, and came to the conclusion that they can't be conscious, usually for biological naturalist reasons. This seems to be the predominant group on Hacker News (among those who discuss it).
Obviously, language-driven thought is not a requirement for consciousness, not just in other animals, but even in humans. The thinking mind takes a secondary role in ordinary daily human life. The truth is that a human being behaves the way they do is not because of thoughts, but because of conditioning (the thoughts are not the primary driver of decisions, actions and behavior). The 99% of the action and responses are trained, the thoughts that we have are also part of this conditioning (most thoughts are unconscious and they are inter-wired with the behaviors, even a seemingly conscious self-reflection thought can be an automated pavlovian trigger). For example, one may think that they get up and go to work because they have a thought "I have to get up, now I am going to go to work", this is an illusion and complete misunderstanding of what consciousness is. Or one can have a psychological insight about oneself, if it's repeated and follows a behavior consistently, the very thought is just the equivalent of whistle-salivation. The thinking mind gives us that 1% to self-reflect, adjust our behavior, learn, predict the future and that differentiates us from other mammals, it's a powerful tool, but just a tool, but it should not be confused with consciousness and it should not be confused with the mind as a whole (in the materialistic sense). The way our brain functions is anything but like an AI agent. And what is consciousness? It's not the thinking mind. It's the experience. It's the direct perception of the senses. The consciousness is what is seen, heard, smelled, touched, thought (the experience of having a thought) in the moment. When you practice meditation, you get to discover the consciousness directly by becoming separated from the thinking thread. The thinking thread becomes more like an external tool, like a computer inside you and you realize directly that it's just a part of the cognitive faculty that makes you navigate your life, not the entire thing.
The LLM (and the harnesses) as built right now merely simulate the tool (the thinking mind). It's not that because this is some code ran on a beefy, but regular piece of tech invented in 20th century you may have at your desk that it does not have awareness (that's also a good argument), but because the way they function and operate is nothing like human (or mammalian) brain, then why would you think that regular code running on a regular PC could gain awareness? My point is that there's no similarity argument, LLMs, despite all their incredible capabilities (to threaten our jobs), are not remotely similar to the way our brain works.
Secondly, even if someone built an artificial brain made of whatever that simulates the biological structure, because of the philosophical zombie problem (the fact that there's no way to scientifically observe consciousness), you could never be too sure if a key ingredient was not missing and you are looking at an NPC. The consciousness is not a property of the physical brain, it's literally immaterial, it's the direct experience of the senses. You can make an optimistic assumption that every person and animal experiences consciousness t...
5. People who has a financial interest in making sure that any eventual AGI isn't granted any kind of rights and continue to be exploited as an inanimate "thing", not as a "being", no matter the actual characteristics of this hypothetical AGI entity.
I mean, take a look a this language in the paper [0]
> This realization pulls the field of AI safety out of the welfare trap. It allows us to focus entirely on the concrete risks of anthropomorphism, treating AGI as a powerful, but inherently non-sentient tool.
This reads as someone that started with this conclusion, and then built an argument to support it.
[0] also discussed in this reddit thread https://old.reddit.com/r/singularity/comments/1sotz9t/google...
Total drivel. Consciousness in biological systems is "a given" because of metabolism?
Are we just autocomplete machines with sufficient enough variable pseudo-randomized input?