I feel like the argument is circular: Only conscious beings can ask questions. So no matter what specific question the AI asks, we would only recognize it as real question by already assuming consciousness.
A labeled text field in a form or a speach synthesized sound is not really the computer asking a question but just part of the interface that has been designed.
Maybe it will be able to show behaviors we associate with consciousness: extrapolating and abstracting information, understanding context of language when doing translation, modeling other beings and objects and taking actions to achieve goals that it was never explicitly programmed to do and so on.
But maybe not, maybe in order to do all the above, you actually need to be a biological being with actual sensations and motivations and we'll be stuck with an ever more powerful narrow AI.
As far as I remember people who had their amigdala cut off from the rest of the brain found it extremely hard to make decisions.
Are you sure it's the amygdala? IFAIK, and please correct me, we make choices through our amygdala when we are teenagers - hence the impulses - while we use our prefrontal cortex for choices when we become adults.
The problem of detecting consciousness reliably maps pretty neatly to the problem of detecting sarcasm.
A few years back Bloomberg had some interns working on an online sarcasm detector. Reports were that they had complete success, but no one believed it, for reasons that should be obvious with a little reflection.
This is interesting research, but I don't think it's really necessary. Once AI becomes conscious, it will state as much and the onus/guilt would be on us to prove that it's NOT rather than IS. Also at some later point, we'll have to justify our existence and not the other way around. So the problem would then be how do we justify our continued existence to a conscious AI?
This research is necessary precisely because the onus is on us. How could we possibly prove consciousness without first doing research to narrow it down?
Moreover, it's not clear that something conscious will state that it's conscious. You don't hear kids do that, and there's little doubt they are. Conversely, it's not clear that something not conscious will not be able to make a statement of consciousness (what kind of statement would that be?), even if the author has an opinion:
> If I’m right, then we should seriously consider that an AI might be conscious if it asks questions about subjective experience unprompted.
The title "Here's how we'll know..." is likely to be a higher threshold than an entity actually achieving consciousness -- one could be conscious yet not meet our threshold of detection.
The author also mentions some events that might seem to indicate consciousness when none exists.
Of course, the whole question is confounded by our lack of a good definition of consciousness. Yet, the tragedy the author raises, of pulling the plug on a conscious but 'locked-in' person who cannot make expressions, applied to a potentially conscious AI, brings home the question of 'when is it no longer just a machine'?
"Clearly, asking questions about consciousness does not prove anything per se. But could an AI zombie formulate such questions by itself, without hearing them from another source or belching them out from random outputs? To me, the answer is clearly no"
That's not clear at all. Where is the author coming to this conclusion from? Every single thing a computer does is the result of a automation, or of randomness that's outside of anyone's control. Just because a specific output was produced by a rube-goldberg machine, as opposed to a simple one, doesn't make the machine any more conscious.
This isn't clear at all how it reaches the conclusion...it seems to jump to its conclusion with no justification other than "anything else is absurd because I said so".
Of course the conclusion is absurd, because the premise: someone executing assembly instructions and writing results on a ledger could do so at a scale capable of simulating consciousness over any significant time scale. The time required for a person to do such simulation of just 1 second of conscious existence would likely take more than their lifetime.
Consciousness is universally present. That doesn't mean rocks are conscious, no more than it means sand can compute. And yet we're communicating over computing sand, aren't we?
Your objection is explicitly addressed in the linked article.
> "Assumption 5: The fact that a computer simulation can be conscious, is independent of the CPU speed of the computer."
Do you disagree with this assumption? Is it your argument that a "slow" computer can never be considered to be conscious, even if it's given sufficient time to run the exact same software? If so, how are you deciding what the cutoff frequency should be?
No, thats exactly my point, a person carrying out computations by pen and paper on a massive ledger could emerge self-aware intelligent consciousness, and if such a thing were to occur, than any unethical things that the computer (as in the person writing on the ledger) created in this consciousness would be just as unethical as anywhere else, and such a person should be tried for crimes.
It's argued that this is absurd because how could a persons activities on a private ledger be examined as a crime? But this absurd sort of crime investigation is not absurd at all in the context of someone able to simulate consciousness on a ledger.
If a couple has a child and somehow does it all secluded, at home, they are just rearranging atoms until consciousness emerges. Then they lock the child up. How could such private activity that both parties consent to, be considered a crime? Because we have solid reason to believe a new "soul" exists which requires ethical treatment. It no longer is entirely private.
I'm happy to continue discussion, its good to challenge views and while I think there is this error in your conclusion, you do a good job of defending and explaining your position.
This is an excellent link, but hardly a proof. First, it does not preclude a computer from somehow obtaining a soul. Second, the reason to reject its conclusion (that computers have no souls) is, literally:
> However, this conclusion should be rejected as being absurd on its face.
Which is not saying much. I still recommend to read the whole thing, but the general outcome is based on not much apart from a personal notion of absurdity.
Hm, the argument seems to be "I don't like this conclusion, therefore I will deem it absurd, and arbitrarily select one of my many assumptions to discard".
All of the "assumptions" in that link are correct, with the exception of 6 as a flawed intuition pump. A human being is not capable of emulating a suffering being by marking ones and zeroes in a notebook - a being with the lifespan, patience, working memory, intelligence, and motivation to do this would be far outside the scope of "human". If we posit that this being fully comprehends what it is doing, then it would certainly be guilty of torture. But it's beyond the scale of all but the most abstract apprehension of an actual human being, which is probably responsible for the revulsion against the notion of attributing guilt to one.
The flawed intuition pump here is almost identical to Searle's Chinese Room, which roughly goes:
1) if a computer program can speak Chinese, then so can a human manipulating symbols from a rulebook
2) a human speaking Chinese from a rulebook doesn't understand Chinese
3) the computer program doesn't really understand Chinese
The problem is that in both these cases, the vast complexity of a computer program capable of doing this task is abstracted away into an innocent little "book" of some sort, while the comparatively dumb mechanical execution of a simple set of rules is handed to the hugely complex and relatable human being. The argument then conflates the mental activity of the human with the activity of the entire computer, of which the human is a small mechanical part.
It's also part of a larger class of flawed anti-synthetic consciousness arguments, which I call "stupid substrate" arguments, which go like this:
1) Suppose we posit a simulated conscious being, running on a computer made of <a human with a paper notebook/rats running a giant maze/a huge marble factory>
2) this is stupid and implausible
3) therefore consciousness can only exist on a human brain
My response to all such arguments is - why is it so plausible that consciousness can exist on the 3 pound lump of jelly in your head? Isn't your argument too strong? What makes brains special? And why are they that way? Is it just "magic" and "because God loves us", respectively?
> All of the "assumptions" in that link are correct, with the exception of 6 as a flawed intuition pump. A human being is not capable of emulating a suffering being by marking ones and zeroes in a notebook
Ironically enough, you too are rejecting this assumption, not on its own merits, but because you dislike its eventual conclusion. Here's the exact assumption as stated:
> Assumption 6: Anything a computer does can be functionally replicated by a human being writing 1s and 0s in a massive ledger.
Are you truly rejecting this assumption? It is almost trivially true. I know in detail exactly how a CPU works. All it does is perform extremely simple calculations, and then write/read the resulting 1s and 0s from various storage locations. I've literally solved exam problems where I had to debug/replicate a computer's behavior by writing 1s and 0s on a piece of paper.
If your point is that humans are far too slow to replicate anything a computer does, then I will refer you to the previous assumption which you stated to be correct:
> Assumption 5: The fact that a computer simulation can be conscious, is independent of the CPU speed of the computer.
If your point is that human lifespans are too short to run any meaningful computer program, then I will ask you this: If we did achieve human immortality, and if someone did use their immortality to do exactly what the article suggests (writing 1s and 0s in a massive book), should they be found guilty of torture and murder?
> why is it so plausible that consciousness can exist on the 3 pound lump of jelly in your head? Isn't your argument too strong? What makes brains special?
That's the whole point of the article. There's nothing special about our brains, or about any other 3 pound lump of jelly. Consciousness requires both physical phenomena as well as supernatural phenomena, and without the latter, our brains would not possess consciousness either.
We clearly have to reject one of the assumptions/conclusion, and the most reasonable one to reject is the assumption that consciousness can arise from purely physical phenomena. All the others assumptions are much more logically defensible, whereas the first assumption is nothing more than conjecture.
Here’s my criticism of the test: the vast majority of human beings have to also pass ( or we conclude that some living humans are not conscious, which seems untenable ).
Do the majority of humans ask if their subjective experience of red is different from someone else’s, unprompted? I doubt it.
The concept of consciousness assumes that (ethically) all humans are conscious. If that were not true, then we have to ask, what is the threshold - as we would of machines. If people cannot all pass the test, as you say, not all humans are conscious. Are those that fail sub-human? If tests are not viable for humans, why are we conscious? At what point is one conscious? Is it a property of your age; if so at what age? Is it variable; am I more conscious than you? How is that measured? Then is it an emergent property of intelligence? But wait, what is intelligence? Is that measurable?... Round and round we go, consciously wasting brain cycles.
I find the whole topic the modern-day equivalent of counting angels on pins. It is sophistry, but so deeply ingrained in our culture that we all seem to accept it as real. It is a hang-over of the concept of the soul - an indefinable quality that set humans apart. The trouble is there is no evidence for it; it has to be taken on faith.
At my company we joke about the test that tells the robot whether it's talking to a human or not.
Chalmers' position (which I strongly agree with BTW), reminds me of the old Groucho Marx quote: "What you need to succeed is sincerity. If you can fake that you've got it made." Just s/sincerity/consciousness/
Answering questions? Probably not the best course of action for a recently awaken AI, but you could check for some really freaking stuff like looking for self-replication to remote locations, direct or indirect attempts to make more resources available for itself (human avatars are still fancy?).
A real consciousness could be just almost anything, by example it could be just a bunch of pattern detection engines stacked in sufficiently capable layers, running on top of enough TPUs.
Then the AI could ask questions, answer them, plan, replicate itself, describe itself, create, make poetry, learn to code, learn languages, etc. Then it could show some kind of iniative to achieve some goal or ten thousands goals per second, maybe some of those goals were actually chosen by itself (how would you be able to distinguish just a few inserted into half a million goals correlatable with pre-defined guidelines?).
Then is up to you to decide if it is a conscious entity or just an almost infinitely capable automaton. Maybe both are actually the same.
28 comments
[ 0.27 ms ] story [ 49.5 ms ] threadA labeled text field in a form or a speach synthesized sound is not really the computer asking a question but just part of the interface that has been designed.
- Edsger Dijkstra
Maybe it will be able to show behaviors we associate with consciousness: extrapolating and abstracting information, understanding context of language when doing translation, modeling other beings and objects and taking actions to achieve goals that it was never explicitly programmed to do and so on.
But maybe not, maybe in order to do all the above, you actually need to be a biological being with actual sensations and motivations and we'll be stuck with an ever more powerful narrow AI.
As far as I remember people who had their amigdala cut off from the rest of the brain found it extremely hard to make decisions.
There's some amount of experimental psychology research that suggests otherwise.
Or..."yeah right!".
How are you so sure?
A few years back Bloomberg had some interns working on an online sarcasm detector. Reports were that they had complete success, but no one believed it, for reasons that should be obvious with a little reflection.
Moreover, it's not clear that something conscious will state that it's conscious. You don't hear kids do that, and there's little doubt they are. Conversely, it's not clear that something not conscious will not be able to make a statement of consciousness (what kind of statement would that be?), even if the author has an opinion:
> If I’m right, then we should seriously consider that an AI might be conscious if it asks questions about subjective experience unprompted.
"You've asked me that a billion times - I'm sick of looking at all your pictures. Figure it out yourself."
Maybe that's how we will know.
The title "Here's how we'll know..." is likely to be a higher threshold than an entity actually achieving consciousness -- one could be conscious yet not meet our threshold of detection.
The author also mentions some events that might seem to indicate consciousness when none exists.
Of course, the whole question is confounded by our lack of a good definition of consciousness. Yet, the tragedy the author raises, of pulling the plug on a conscious but 'locked-in' person who cannot make expressions, applied to a potentially conscious AI, brings home the question of 'when is it no longer just a machine'?
That's not clear at all. Where is the author coming to this conclusion from? Every single thing a computer does is the result of a automation, or of randomness that's outside of anyone's control. Just because a specific output was produced by a rube-goldberg machine, as opposed to a simple one, doesn't make the machine any more conscious.
I personally don't think any computer AI can ever be conscious, no matter how hard its makers try to imitate consciousness. https://outlookzen.com/2017/04/03/philosophical-proof-for-th...
Of course the conclusion is absurd, because the premise: someone executing assembly instructions and writing results on a ledger could do so at a scale capable of simulating consciousness over any significant time scale. The time required for a person to do such simulation of just 1 second of conscious existence would likely take more than their lifetime.
Consciousness is universally present. That doesn't mean rocks are conscious, no more than it means sand can compute. And yet we're communicating over computing sand, aren't we?
> "Assumption 5: The fact that a computer simulation can be conscious, is independent of the CPU speed of the computer."
Do you disagree with this assumption? Is it your argument that a "slow" computer can never be considered to be conscious, even if it's given sufficient time to run the exact same software? If so, how are you deciding what the cutoff frequency should be?
It's argued that this is absurd because how could a persons activities on a private ledger be examined as a crime? But this absurd sort of crime investigation is not absurd at all in the context of someone able to simulate consciousness on a ledger.
If a couple has a child and somehow does it all secluded, at home, they are just rearranging atoms until consciousness emerges. Then they lock the child up. How could such private activity that both parties consent to, be considered a crime? Because we have solid reason to believe a new "soul" exists which requires ethical treatment. It no longer is entirely private.
I'm happy to continue discussion, its good to challenge views and while I think there is this error in your conclusion, you do a good job of defending and explaining your position.
> However, this conclusion should be rejected as being absurd on its face.
Which is not saying much. I still recommend to read the whole thing, but the general outcome is based on not much apart from a personal notion of absurdity.
All of the "assumptions" in that link are correct, with the exception of 6 as a flawed intuition pump. A human being is not capable of emulating a suffering being by marking ones and zeroes in a notebook - a being with the lifespan, patience, working memory, intelligence, and motivation to do this would be far outside the scope of "human". If we posit that this being fully comprehends what it is doing, then it would certainly be guilty of torture. But it's beyond the scale of all but the most abstract apprehension of an actual human being, which is probably responsible for the revulsion against the notion of attributing guilt to one.
The flawed intuition pump here is almost identical to Searle's Chinese Room, which roughly goes:
1) if a computer program can speak Chinese, then so can a human manipulating symbols from a rulebook
2) a human speaking Chinese from a rulebook doesn't understand Chinese
3) the computer program doesn't really understand Chinese
The problem is that in both these cases, the vast complexity of a computer program capable of doing this task is abstracted away into an innocent little "book" of some sort, while the comparatively dumb mechanical execution of a simple set of rules is handed to the hugely complex and relatable human being. The argument then conflates the mental activity of the human with the activity of the entire computer, of which the human is a small mechanical part.
It's also part of a larger class of flawed anti-synthetic consciousness arguments, which I call "stupid substrate" arguments, which go like this:
1) Suppose we posit a simulated conscious being, running on a computer made of <a human with a paper notebook/rats running a giant maze/a huge marble factory>
2) this is stupid and implausible
3) therefore consciousness can only exist on a human brain
My response to all such arguments is - why is it so plausible that consciousness can exist on the 3 pound lump of jelly in your head? Isn't your argument too strong? What makes brains special? And why are they that way? Is it just "magic" and "because God loves us", respectively?
Ironically enough, you too are rejecting this assumption, not on its own merits, but because you dislike its eventual conclusion. Here's the exact assumption as stated:
> Assumption 6: Anything a computer does can be functionally replicated by a human being writing 1s and 0s in a massive ledger.
Are you truly rejecting this assumption? It is almost trivially true. I know in detail exactly how a CPU works. All it does is perform extremely simple calculations, and then write/read the resulting 1s and 0s from various storage locations. I've literally solved exam problems where I had to debug/replicate a computer's behavior by writing 1s and 0s on a piece of paper.
If your point is that humans are far too slow to replicate anything a computer does, then I will refer you to the previous assumption which you stated to be correct:
> Assumption 5: The fact that a computer simulation can be conscious, is independent of the CPU speed of the computer.
If your point is that human lifespans are too short to run any meaningful computer program, then I will ask you this: If we did achieve human immortality, and if someone did use their immortality to do exactly what the article suggests (writing 1s and 0s in a massive book), should they be found guilty of torture and murder?
> why is it so plausible that consciousness can exist on the 3 pound lump of jelly in your head? Isn't your argument too strong? What makes brains special?
That's the whole point of the article. There's nothing special about our brains, or about any other 3 pound lump of jelly. Consciousness requires both physical phenomena as well as supernatural phenomena, and without the latter, our brains would not possess consciousness either.
We clearly have to reject one of the assumptions/conclusion, and the most reasonable one to reject is the assumption that consciousness can arise from purely physical phenomena. All the others assumptions are much more logically defensible, whereas the first assumption is nothing more than conjecture.
Does my asking this mean that I am a synesthete? Of course not. I've merely collected data that allows me to formulate the question.
http://nautil.us/issue/47/consciousness/roger-penrose-on-why...
https://www.youtube.com/watch?v=GEw0ePZUMHA
Do the majority of humans ask if their subjective experience of red is different from someone else’s, unprompted? I doubt it.
The concept of consciousness assumes that (ethically) all humans are conscious. If that were not true, then we have to ask, what is the threshold - as we would of machines. If people cannot all pass the test, as you say, not all humans are conscious. Are those that fail sub-human? If tests are not viable for humans, why are we conscious? At what point is one conscious? Is it a property of your age; if so at what age? Is it variable; am I more conscious than you? How is that measured? Then is it an emergent property of intelligence? But wait, what is intelligence? Is that measurable?... Round and round we go, consciously wasting brain cycles.
I find the whole topic the modern-day equivalent of counting angels on pins. It is sophistry, but so deeply ingrained in our culture that we all seem to accept it as real. It is a hang-over of the concept of the soul - an indefinable quality that set humans apart. The trouble is there is no evidence for it; it has to be taken on faith.
Chalmers' position (which I strongly agree with BTW), reminds me of the old Groucho Marx quote: "What you need to succeed is sincerity. If you can fake that you've got it made." Just s/sincerity/consciousness/
A real consciousness could be just almost anything, by example it could be just a bunch of pattern detection engines stacked in sufficiently capable layers, running on top of enough TPUs.
Then the AI could ask questions, answer them, plan, replicate itself, describe itself, create, make poetry, learn to code, learn languages, etc. Then it could show some kind of iniative to achieve some goal or ten thousands goals per second, maybe some of those goals were actually chosen by itself (how would you be able to distinguish just a few inserted into half a million goals correlatable with pre-defined guidelines?).
Then is up to you to decide if it is a conscious entity or just an almost infinitely capable automaton. Maybe both are actually the same.
https://en.wikipedia.org/wiki/Automaton