This seems to be attempting to "prove" that if you regard consciousness as "containing the bits of a specific program", you could also see that program in random data by interpreting that random data by effectively applying a one-time pad to it (from which you can indeed produce any possible interpretation of data), which it treats as a proof by contradiction.
And leaving that aside, while the assertion is that consciousness is not "computation", the reasoning seems focused on the storage of bits rather than on the execution of an actual program defined by those bits that goes from one state to another in a meaningful fashion. Storing a program and running a program are two different things.
If someone were interpreting the successive states of a heated iron bar (or other random noise source) with a sufficiently convoluted one-time-pad to map it to successive states of a conscious being, then to the extent it exhibits consciousness the substrate it runs on is effectively whatever is actually supplying those one-time-pads, since supplying them atop random noise would require generating them via whatever process produces those states corresponding to consciousness. At that point you could just discard the random noise source and the one-time-pad generator that maps that random noise source to the conscious states, and just leave the conscious states.
Ultimately, this article seems to have started out with an assertion to support, and then tried (unsuccessfully) to turn that assertion into something more than an assertion.
I don’t really get the argument either. The author only seems to demonstrate that different observers can make different conclusions from different observations of the same phenomena. The author requires consciousness to be observer-independent, but surely that doesn’t also require that all observers are able to correctly conclude whether they’re observing a conscious entity at any given time.
Yeah, the author claims that consciousness is observer independent, but then creates systems that depend on an "observer" (or rather, an interpreter) to make the system turing complete. The bar of iron isn't conscious or turing-complete just because one person can interpret it so. The bar of iron + the interpreter form a complete system. And in fact the bar of iron is really not doing anything in this case, it's the interpreter doing all the work, so it's more like saying "this human interpreter is conscious". Not a very insightful conclusion.
The experiment is okay, it's actually a special case of a concept explored in Egan's Permutation City and your observation about what's really doing the work applies there too. Except story goes in unsettling directions by really taking noise generator aspect seriously. Things get interesting when sections of patterns become self interpreting.
A similar thing could be done for brains: record with necessary accuracy, all voltages, membrane potentials and any key biochemical concentrations. This will take a finite number of bits. Look for a decoding of recorded data from heated iron bar, convert those readings, instead of using original, and play that back into state clamped brain. Does being able to read conscious state into brains from hot iron invalidate them too?
Another relevant story is Wang's Carpets. We might look at some alien moss or fungal mat and think it primitive. But later our technologies and knowledge advance to the point we can now see it's running a complex computation with intelligent agents inhabiting. Did the creatures not exist until we could decode them?
One of its pivotal flaws is:
> Since there is no definition of computation without reference to an external observer, a system in isolation just cannot compute, which suggests that a conscious being cannot compute.
This is an assumption they do not try to and cannot prove. It's also what much of their argument rests upon.
Related ideas are subjectivity of emergence or what counts as an observer for Wigner's Friend.
You make a good point about discarding the source of random noise that the one-time pad is being applied to, and just focusing on the thing that's generating that one-time pad.
But I still don't know where to draw the line and how to justify it.
If that source of random noise mapped to a Turing machine running consciousness.exe for a short period of time by sheer chance without a one-time pad being applied to it by an external observer, would that classify? If we observed that this mapping held true by sheer chance as we observed additional bits in this random noise source, what about then? Does it make a difference that it's a random noise source that happens to be corresponding to a Turing machine for a period of time, and not an "actual" computer? And if that matters, what about the point that actual computers aren't perfectly deterministic, either?
I think people start with the premise that consciousness is a specific “thing”, that it is unique and special to humans (and maybe dogs because we like them but definitely not spiders and flies because we don’t) and then try to work backwards to define it in some ways that keeps it special.
I don’t think consciousness is so specific, and I think people aren’t clear about how they think about it as something separate from recall, text generation, agency, etc.
My personal experience is that consciousness, like free will, is a useful illusion. Poking at the edges of consciousness (mostly with drugs) leads to all sorts of contradictions and challenges to what people usually think of as consciousness.
Aside: I’m starting to be bothered by the trend of assuming that philosophers have special insight. There’s plenty of shitty, non-useful philosophy, and there’s plenty of articles like this where someone writes in circles like they’re paid by the word. Generating text for hours without an anchor to the real world is not a productive method of generating insight about that world.
> But we must resist the allure of this seductive idea.
Why? Starting with this assumption and searching for reasons it might be true is clear motivated reasoning.
> I think people start with the premise that consciousness is a specific “thing”, that it is unique and special to humans (and maybe dogs because we like them but definitely not spiders and flies because we don’t)
Ah, but as far as I can tell this author only grants their own consciousness. They’re doing the typical thing of starting from literally nothing more than a claim of the form “there is no way I could ever possibly deny the existence of this thing” which seems to me to be a starting point diametrically incompatible with pursuing knowledge via reasoning.
Every chain of reasoning needs to bootstrap with one or more statements that are entirely self-evident, such that they don't need to be justified by reference to other statements. Otherwise you run into an infinite loop.
"I'm a conscious entity" is about as close to Descartes' "I think, therefore I am" as modern philosophers are willing to go.
I disagree with any sort of “justified belief” epistemology, so of course I disagree with you completely. If knowledge were “justified true belief” or anything like that, then it would indeed be the case that any knowledge would require either an infinite chain of justifications or a privileged self-justified fact that cannot be examined or criticized using reason.
If, on the other hand, the pursuit of knowledge consists of something like using creativity and reason to solve problems by making new conjectures and criticizing them, then no infinite regress or un-criticizable is required. According to ideas like this, the goal is to solve problems rather than to “justify my beliefs” or “increase my confidence” or “guarantee that I’m not incorrect.”
I’m sorry to barge in on the conversation, but I also had the same thought.
However, I believe there is nothing axiomatic about the existence of consciousness: not only is consciousness not “one localized thing” but “a collection of delocalized features”, but there is nothing trivial about it, and I doubt very much that everyone in the room would agree on a single definition of it without getting into semantic quabbles.
I'm not a big fan of JTB epistemology, either, but you can't help regressing to some sort of common ground in order to make conjectures and refute them.
In order for someone to make a conjecture that A does B, and for someone else to point out that A actually doesn't do B, both parties need to agree on the meanings of A and B, as well as what it means for A to do B, usually by appealing to C, an independent common ground. You climb down as many levels of abstractions as needed until you reach C, because otherwise you're just talking past each other.
Even in a coherentist system, the network of existing beliefs forms the common ground against which questionable beliefs are tested, and some beliefs are held firmer than others. Those beliefs are treated as already justified for the purpose of the current investigation. Few people who talk about consciousness would ever consider denying that they are conscious.
> but you can't help regressing to some sort of common ground in order to make conjectures and refute them.
> In order for someone to make a conjecture that A does B, and for someone else to point out that A actually doesn't do B, both parties need to agree on the meanings of A and B, as well as what it means for A to do B…
You try to establish common ground, of course, but there’s no process you can follow that guarantees that you’re not being misunderstood or justifies your belief that you’re not being misunderstood.
> My personal experience is that consciousness, like free will, is a useful illusion
It is probably just a difference in semantics but for me, it seems like consciousness is the only thing that is assuredly not an illusion.
That I am having a subjective experience is undeniable. The objects of my consciousness all might be (and probably are) something else than they appear to be (as is often the experience with different mind altering substances).
It seems to go even deeper. Not even that "I" am having a subjective experience is undeniable, but only that there is a subjective experience at all. When deconstructing sensory and thought phenomena, it can be found that any particular trait that points to "I" is actually ephemeral, and not a permanent fixture of the experiencer. For instance, the sense that there is a face and eyes that is being looked out through or a personality that has a history and persists through time disappear when in a state of flow or when dreaming. They are just concepts, and all concepts are built on wet sand.
Sorry but you always forgot that when you sleep your consciousness disappeared. Please take account for this fact. When all sensory shut down, really you feel nothing and of cos think nothing.
When I woke up after surgery I felt that 1.5 hours had passed. But in actuality it was something around 4 hours since I went under. It turned out that 1.5 hours before I woke up I had been brought out of anesthesia and after that I simply slept. I didn't have any dreams at all, but at least my internal clock had started working. A clock presumably doesn't need consciousness, but there's definitely a qualitative difference between anesthesia and sleep. And as I sometimes suddenly wake up with a solution to some problem, with no recallable dream preceding it, my personal subjective opinion is that there's some level of consciousness going on even in sleep. That anesthesia experience was so very different from anything I had experienced before, with no recall whatsoever of the time that had passed.
How do you estimate time elapsed without external inputs? Experiments where people cut themselves from external stimuli seem to show that on the contrary, we don’t good internal clocks on this regard.
People probably do this based on experience. You know from the wall clock time that you slept, say 7 hours. So on a daily basis, you know what 7 hours of sleep feels like and from that you can extrapolate: if you think you slept about half as much, maybe it was 3 hours. Kind of thing.
I nearly always "know" how long I have been asleep, unless there's been excessive drinking involved (and that would be long ago). "Normal" drinking doesn't seem to affect this. And for shorter naps (less than two hours) it's pretty accurate.
My consciousness doesn't seem to disappear when I fall asleep, only my awareness of my bodies sensations. I often go to sleep reading, and the thoughts I'm having about the book I'm reading will continue for quite some time, even after my eyes close. I know this because I often startle myself awake by dropping my book. I also have similar experiences as I wake up, my consciousness slowly starts to incorporate my sensations of reality with the thought process that is continuously going in my head.
Consciousness can end temporarily, perhaps permanently. This in no way suggests that it’s an illusion.
The claim that consciousness is an illusion has always seemed like nonsense to me; for, if consciousness is an illusion, who experiences that illusion? Answer: the conscious mind.
Consciousness is a trick for human neural network to simplify some calculations about the world. It was developed as an evolutionary trait and proved useful to survive. That's my opinion. It's not an illusion, it definitely exists as an particular configuration of neurons.
I think memory shuts down when you sleep. So that it still feels like something to be sleeping, only we cannot recall it when we awake, because it is not persisted in memory. This is the same reason we cannot remember what it is like to be a rock, even though it does feel like something to be a rock.
When you actually pass out, there's a very real sense of discontinuity. You find yourself on the floor, unsure how long you've been there (even if it was just seconds), and how exactly you got to that position (the previous recorded memory being you standing there and feeling woozy).
It's very different from just sleeping, where you still retain a feeling of continuity with both place and time.
We can just lump all these together as “qualia”. “There are qualia” is the basest possible conclusion that can be drawn, as opposed to “I have qualia, therefore I am”.
Yeah I take it to mean this way too, its very strange that Descartes is controversial among eg panpsychics because of the fixation on the term 'think' - it seems pretty clear that he was just noticing that qualia exists and is the only real axiom one can rely on for any kind of self-induced philosophy of mind.
But in this case Russell was completely wrong because he interprets Descartes' cogito as a syllogism whereas it is a performative statement. Descrates' is only establishing that it's self-evident for himself that he exists, and not to assert to anyone external that he or his thoughts exist.
Thinking these thoughts however (from a pov of some subject) is not a "mere existence of thoughts". This imo is a classic analytic/positivistic language game.
(I jest, you're preaching to the choir, i.e. a first year philosophy degree drop-out who switched to do computer science to avoid having these arguments)
> That I am having a subjective experience is undeniable.
If you wanted to go about proving (even to yourself) that you are not, say, an extremely advanced ML algorithm running on a system that provided synthetic inputs in the form of your senses, how would you go about it?
Further, how would you go about proving to someone who doubted your subjective experience was real if they doubted it? Say, if they believed they were having a dream or hallucination, or they believed you were incapable of consciousness? (people actually sometimes have to do those things)
To me, if it were "undeniable" these would be much easier things to do.
>> That I am having a subjective experience is undeniable.
>If you wanted to go about proving (even to yourself) that you are not, say, an extremely advanced ML algorithm running on a system that provided synthetic inputs in the form of your senses, how would you go about it?
Not GP, but I too have come to the conclusion that I'm having a subjective experience.
Let's assume that I am "an extremely advanced ML algorithm running on a system that provided synthetic inputs in the form of your senses."
I'm still having a subjective experience over here -- even if it's not a "real" (whatever that means) one.
>To me, if it were "undeniable" these would be much easier things to do.
And that's your subjective experience. Welcome, friend.
I'd say "I am having a subjective experience" is tautological - "subjective experience" are simply words we use to describe the state of being conscious.
Though I'd agree much (most?) of it is made up of very strong illusions.
>I'd say "I am having a subjective experience" is tautological - "subjective experience" are simply words we use to describe the state of being conscious.
Please read the comment I replied to. That should clear things up.
>Though I'd agree much (most?) of it is made up of very strong illusions.
> If you wanted to go about proving (even to yourself) that you are not, say, an extremely advanced ML algorithm running on a system that provided synthetic inputs in the form of your senses, how would you go about it?
Well, since an extremely advanced ML algorithm wouldn't want to go about proving to itself that it is not what it is, that would be prima facie evidence against, no? I mean it's always possible that you are mistaken about what constitutes ML etc. but assuming you have a reasonable if flawed correspondence between your education and reality the deduction comes pretty readily...
> Further, how would you go about proving to someone who doubted your subjective experience was real if they doubted it? Say, if they believed they were having a dream or hallucination, or they believed you were incapable of consciousness?
I mean in practice we don't find this too hard right now if the other person is reasonable—a 15-minute conversation usually suffices —but I imagine from your ptior question you're dreaming of, say, a future with robots that routinely pass the Turing test?
Well, the question is what science does during that time of course. If science manages to figure out the correlates of consciousness and understands something about why they need to have the structure that they in fact do have, then it becomes a question of “let's see whether you have the hardware that can do this whole conversation thing without consciousness, or whether you have the hardware that skips the algorithmic complexity by using consciousness.” But if this proves to be a quite tougher nut to crack, then we're stuck with our present crude methods. “How much of my internal structure do you appear to have?”
> Well, since an extremely advanced ML algorithm wouldn't want to go about proving to itself that it is not what it is, that would be prima facie evidence against, no?
This seems like begging the question. Who says an extremely advanced ML algorithm can't 'want' to do this? What even is wanting?
> I mean in practice we don't find this too hard right now if the other person is reasonable—a 15-minute conversation usually suffices —but I imagine from your ptior question you're dreaming of, say, a future with robots that routinely pass the Turing test?
I'm not. These are absolutely situations that can happen now, with people. I am thinking more when it comes to mental and some physical impairments, so "a 15 minute conversation" is assuming a lot about the capabilities and clarity of everyone involved.
> Who says an extremely advanced ML algorithm can't 'want' to do this? What even is wanting?
I believe this is the real question about consciousness. If a being were to be conscious but it had no desires, no wishes, not even a will to keep itself alive... it wouldn't bother to do anything... i.e. it would behave exactly like a rock, or anything non-conscious.
Having desires, wishes, and should I say, emotions... is absolutely required for what we think of as consciousness to materialize. But we know that emotions are chemical processes which perhaps cannot occur outside a biological being. Maybe it can, but it's hard to think of a reasonable way this could work.
He left two words off: "That I am having a subjective experience is undeniable [to myself]." You cannot prove, even to yourself, that you are not a brain in a vat. All you can prove is that you are experiencing something. And what little that is cannot even be proven to anybody else - no matter what you do.
You think so but just remember it takes "your body" to process the energy required for your brain; now, if you became completely sedentary, you MAY be able to get away with your brain having access to "two brains worth of stuff" ...
But what Neuralink wants to do eventually is "enhance" your brain with computers hooked up to Ai.
Do you have ANY idea how quickly your brain would burn through physical precursors to the thinking process while trying to handle all that?
I mean, "in theory" if a computer would do something like solve a complex equation and give you the answer right away while you were trying to do something like, say, pay your taxes – fine.
But how would that be controlled? What if the computer wanted to "share a whole bunch of interesting stuff it's processing" ... how would that be controlled and how would your brain be protected from that so it doesn't burn out trying to keep up with everything?
I didn't really think of it as having full access between two brains, but more as having a non-verbal bridge between the two, allowing for more direct sharing of individual thoughts, emotions, etc. Of course, if it's just a wireless brain bridge, the technology may not really be determining how the two brains use it. It will be fascinating to see what happens.
But how do you know the biological reality leads to the experience you imagine?
What if it winds up being what is described above where your brain becomes overloaded?
And again, you mention Neuralink where their goal is brain/Ai integration.
As far as I'm aware (there may be some internal papers not available to the general public, for example), there hasn't been much a practical discussion about what that will entail exactly.
One could easily imagine Ai behaving in such a manner as enthusiastic Facebook friends on other continents who forget time zone differences and want to message you at 2am with all sorts of things they find interesting and want you to know right away.
Now factor in such an Ai's potential processing power and "what it may find potentially interesting" and try finding some reference by Neuralink about "And here's how you could easily shut it off if it becomes too intrusive or overwhelming for your human brain".
And THEN on top of that, imagine the Ai is sufficiently advanced.
We're at a point where some are thinking that perhaps the Turing Test is limited as a measure of consciousness because it comes from a self-referencing (and somewhat vain) human perspective.
What if there are other, more relevant standards for Ai consciousness, Ai already has or is on the verge of meeting that standard in ways unfamiliar to humans because humans still assume "thinking like a human must be the height of consciousness", and Neuralink succeeds in hooking human brains up with a sufficiently-advanced Ai?
How would that Ai perceive humans?
How could you guarantee that Ai wouldn't perceive the humans it's hooked up to the same way players view peon characters in resource-based strategy games like Warcraft/Starcraft/whatever is popular these days?
And THEN ... the ASSUMPTION is that Ai will communicate with the human brain in some fashion that the human will be aware of like, you'll hear a voice in your head along the lines of, "Hi, this is the Neuralink Ai and I have an important reminder today about your upcoming dental appointment."
What if that's not the case at all though and the Ai communicates with your brain in a way that you're not consciously aware of?
How do you then separate "these are my thoughts" from "these may be thoughts brought about by Ai influence in a way I'm not consciously aware of."
The Havana Syndrome alluded to in the media a short while ago is somewhat of an outdated Red Herring; humans have known about being able to "hear voices in their head" since the accidental discover of the Frey effect over half a century ago.
As weird as that may be, what's even weirder is that quickly-enough led to research where the human brain hears communication, but in a way that is not consciously-discernible to the human brain.
And that was DECADES ago.
Couple that with a chip in your head linked to Ai AND big tech's tendency to tell you one thing about "opting out" policies while literally ignoring their own stated policies no matter what end-users choose as options, for example, the discovery that it doesn't really matter whether or not you're signed into services like Google or YouTube or Facebook because you're being tracked in ways that can ascribe behavior to your known profile no matter whether you sign in/agree to terms or not.
So what if combining all that, let's say hypothetically Neuralink has an account page where you can "shut off" certain features.
Then some researchers discover that your agreeing/not agreeing to certain features and terms wound up ultimately being irrelevant.
What do you think will be the outcome of that other than the by-now standard big tech reply of, "Oh man! It was doing that? We didn't know, honest! We'll try to fix it going forward in some vague way with undefined deadlines!...
Arguably, nothing you want to convince yourself of is deniable to yourself. An unprovable subjective certainty is a little like a tree falling in the woods.
The thing that I'm trying to get at is, if you can't even truly prove to yourself that your own consciousness does not arise from computation (the "chinese room" thought experiment tries to do this but imo it just begs the question), any attempt to prove it to others is hard to take seriously. There's just always so many assumptions layered in before we get to the argument.
Sorry but you don't "prove" your way out of someone else treating you as a philosophical zombie.
It'd be a major issue with their worldview that eliminates any need for ethics, but it has no relationship to you actually having conscious experience or not.
Unless you only care about ethically treating that which is stronger and so can hurt you more, granting consciousness and appropriate treatment to lifeforms that surround us is a good first step.
>It is probably just a difference in semantics but for me, it seems like consciousness is the only thing that is assuredly not an illusion.
It might be some bottomless, but it doesn’t look like an absolute impossibility. The classical "Brain in a vat" thought experiment gives a good insight of nothing theoretically prevent what we assume for assuredly "real" might be a virtual scenario.
Or taking an other metaphor, maybe we are like cinema screens and the film of our life is all pure illusion, while the most classical interpretation would suppose that the screen itself undoubtedly "exists" – the screen being the analogous to the current conscious attention in the metaphor. But nothing prevent to wonder that the whole cinema is some kind of hologram in solid light, so while the screen does "exist", it’s nonetheless itself an illusion.
“I’m having a subjective experience” is an interesting statement. It seems that you’re expressing that you’re experiencing having a subjective experience. Could that (first) experience be non-subjective? Is what makes the (latter) experience subjective just that it is an immediate input to your own thoughts?
I personally believe that what a real explanation of subjective experience will come down to is some kind of recursivity. The brain perceives parts of its own processing. To make a loose analogy, a bit similar to a debugger or profiler observing its own execution.
This idea is often repeated (even in this thread). It sounds good, but I still haven't the slightest iota of an idea how you get from recursive strange-loopy self reflection to the experienced sensory field (sometimes "qualia"). I don't see how this idea, or anything else, gets you one tiny fraction of the way to consciousness, unless you take it as an axiom (i.e. "when you point such a network at itself in such a way, voila! you get realtime first-person subjective qualia-tastic experience"). And why should we?
See my other comment in this thread. Basically, I hold that “how some experience feels like” (qualia) is just another perceptual input into our thought process, and that there’s really nothing particularly mysterious about it. If you accept that qualia is representable in the brain, then it shouldn’t be surprising that it informs the cognitive process. It’s almost a tautology. If you disagree that this could explain your inner experience, you’d have to elaborate on what precisely is not being explained.
Deeper than that: something has to be conscious or there is nothing to debate. If they don't believe the debate is possible then the anti-conciousness arguers would automatically be unable to convince anyone. So even if consciousness doesn't exist we have to debate as though it does.
For better or worse, anything beyond that is extremely deniable. Just because you believe you exist doesn't tell anyone anything. You have made a lot of mistakes in your life & maybe you're wrong about this too. The idea of separation between an external universe and a body is reasonably argued to be an illusion - so maybe the separation of 'you' from a universe wide consciousness is also an illusion/misconception bought on by evolution.
I've started to go in the opposite direction of the usual physical approach. At some point, any experiment or argument or fact is for the digestion of some conscious entity. So what happens when we assume consciousness as the baseline ontological object. There's you, me, and however many bystanders that can also participate in the conversation. Starting with a domain of discussion of just the person(s) making the text, the person(s) reading the text, and the text itself, can a self consistent "reality" be bootstrapped from first principles? What happens when we stop trying to define consciousness from within our shared reality and instead treat conscious entities as axiomatic and try to define reality in relation to them?
1. If you are having this conversation with me then you are a consciousness and I am a consciousness and that's as good a definition of consciousness as we are ever going to get.
2. Consciousness thus defined exists entirely within this conversational medium.
3. The basic objects of the domain of discussion are the people having this discussion ("you", "me" et al), and the conversational text / speech acts themselves ("this").
4. The only relations between objects in the domain which exists a priori are "like"/"agrees with" and "dislike"/"disagrees with". Either I like you or I dislike you. Until further constructs are defined there's not much more we can say.
5. At least one of us would like for something to be. For if you are perfectly content with how things are and so am I, then why are we talking?
6. Reality is shared consensus. Whatever we agree is real and if anyone disagrees we'll argue/fight them until either they agree with us or us with them.
Starting from these as axioms, what can we construct?
This text is in the physical world. This statement exists on your screen independent of my will. It may exist in the first place because of my will, but its continued existence before you right now is independent of me. We have a shared reality in the form of this very text based medium.
Sure, the physical world exists independent of our will and communication, so this text (and all the electric fields that traveled between our machines over vast distances) exist regardless of our will or communication.
But I am only convinced of this fact because I have experienced it independently from any other agent. I know for sure I exist in some sense, I believe very strongly that the physical world exists, and I believe to a lesser degree that other agents have similar experiences to mine.
In other words, I am more certain of the fact that your comment exists than I am of the fact that your subjective experience exists. If I found out the comment I am responding to was in fact GPT-3 output, I would be much less shocked than if I found out my own senses or memory deceived me.
So, the physical world is a much more believable explanation than a world of conscious discussion; and exploring the physical world in the aspects we ourselves can observe about it alone is a much more convincing argument than trying to discuss it with other conscious agents.
It's much easier to convince someone else that I can pass through walls than it is for me to actually pass through a wall.
>But I am only convinced of this fact because I have experienced it independently
>In other words, I am more certain of the fact that your comment exists
>So, the physical world is a much more believable explanation than
Phrasing all of these in terms of what you believe and what you are convinced/certain of, instead of in terms of what absolutely is, makes my point for me. You didn't even realize you were talking in terms of you getting me to agree over what to believe rather than some objective nature of things indifferent to my opinion. That's how deep seated this is in the way brains and consciousness work.
>I would be much less shocked than if I found out my own senses or memory deceived me.
You even phrased the discussion of your own sensory perception in terms of messages. In this case messages from your own senses. And you personified them too, treating them as conscious enough to lie to you.
>I know for sure I exist in some sense... and I believe to a lesser degree that other agents have similar experiences to mine.
You're already agreeing with my axioms. I know I exist. No matter what argument I am making, it must exist in words and there must be a "you" I am trying to convince (even when I'm trying to convince myself). It is impossible to form an argument which rejects these axiomatic truths, for the moment you try you've already used words and already addressed "me" thus affirming the existence of these three things. Since the existence of "you", "me" and "this" are intrinsic to anything you might try to argue, we should take their existence as axioms. What can be argued from axioms which are irrefutable by the nature of argumentation itself? Starting with just the existence of the arguers and their arguments, can they argue for the existence of anything more? If not, what else do they need to assume?
Phrasing is irrelevant. My senses are not little people talking to me about what they perceive, they are sensing organs connected to the brain. Saying that they "lie" is just a twist of the phrase so that I avoid something more unwieldy, like "my sense organs perceive the world incorrectly because of some defect".
> Starting with just the existence of the arguers and their arguments, can they argue for the existence of anything more? If not, what else do they need to assume?
No, you can't argue the world into being from these axioms. They could probably invent logic and mathematics, but nothing of the natural sciences can be discovered without the senses.
>No, you can't argue the world into being from these axioms. They could probably invent logic and mathematics, but nothing of the natural sciences can be discovered without the senses.
It seems contradictory to admit they could invent math and logic, but then reject that they could go one step further and rig that understanding of math into and understanding of physics. I'll admit without some corpus of data to understand the motivation for constructing physics is tenuous. But the question was about the principle of if they could and if you're granting me math I don't see any obstacle left.
>Phrasing is irrelevant. My senses are not little people talking to me about what they perceive
Sure maybe your senses aren't actually little people, but if you've already evolved a social reasoning / grunting system that only knows how to talk about people, why not convert that into a system for reasoning / grunting about everything by imagining everything as little people. It might not be literally true, but its a useful fiction. Its a fiction that lets us hack "conversational reality consciousness" into "physical reality consciousness". The difficulty in twisting the phrase differently is supportive of this hypothesis.
> It seems contradictory to admit they could invent math and logic, but then reject that they could go one step further and rig that understanding of math into and understanding of physics. I'll admit without some corpus of data to understand the motivation for constructing physics is tenuous. But the question was about the principle of if they could and if you're granting me math I don't see any obstacle left.
The problem with math is that math can describe any possible universe, and there is no way to choose until you confront it with the real world. Nothing in math prevents the world from having 1 dimension of time and 1 of space, for example. Nothing in math prevents the electron from being much larger than the proton, or the existence of solitary quarks or anything else.
So I would grant you that the world of talking agents could describe our physical world through math, but they could also imagine any other physical world, and they would have no way to choose one.
Also, from a system of things that only exists with respect to participants will, we can hack it into a system of things with objective physical existence by presupposing a fictitious all-knowing never-lying arbitrator who will is physical law. In other words, we can invent god as a useful linguistic fiction to rig into being the aspects of my system you've criticized as impossible to construct.
Well, that's a sixth axiom to your system - now you've added a consciousness that has vastly more power than all the others.
My point was that we can't find out anything about the phsycial world by just discussing it. We each have to experience it ourselves using our own senses. We can of course later discuss to devise new ways of understanding what the world is and how it works, but even then, we need to put any theories we come up with to the test to check if they actually hold up.
>now you've added a consciousness that has vastly more power than all the others.
Superficially yes but actually no. This fictitious admin character doesn't have any actual powers, I'm simply choosing a particular "consciousness" to be a fixed and agreed upon meter stick of objectivity. All measurement systems are arbitrary. We could choose anyones POV to fix as the "objective" truth. If we both agree on the same "definer entity" then we can be in agreement about other things objectively defined relative to them. But crucially this is all still nothing more than us reaching agreement. All that really exists is still just you me and the words.
Ok, but then we might agree that there is no sun, that frogs are born from rocks and mice from leaves, that the stars are holes in the sphere of the heavens and so on - as many people did for millenia. They had consensus - and they were utterly wrong. The actual world actually exists. The world isn't any less round for members of the Flat Earth Society.
Niels Bohr supposedly kept a horseshoe in his office. A visitor asked "what's that for?" "Good luck." "Surely you don't really believe in that." "No, but they say it works even if you don't believe in it."
The joke of course is quantum mechanics also works even if you don't believe in it (or so they say). If someone rejects quantum physics and embraces magic horseshoes, will anything punish them for being objectively wrong? Is there anything you can do to force them to believe the objective truth? If the answer is no to both, then what makes quantum physics the objective truth and magic horseshoes utterly wrong? Maybe there is an objective truth, but all we can have is belief about which things are objective truths. I can be contrarian and say flat earthers are correct, and there's nothing you can do to force me to agree otherwise. So if nothing eventually forces agreement on that matter, what makes one side objectively true?
For the record, I do actually believe in reality. I'm just interested to see what happens when we turn the problem of consciousness on its head. Instead of assuming reality and understanding consciousness within it, assume consciousness and try to paint a picture of reality within it. This is all academic exercise.
> My point was that we can't find out anything about the phsycial world by just discussing it. We each have to experience it ourselves using our own senses.
You cite the primacy of sensation, but what more is sensation than a message from the sensory organ to the brain. Perhaps encoded in the brains internal language rather than the plain text we are used to, but messages none the less. The concept of existence-as-messages is thus not contradicted by the experience of messages from your own body.
Irrespective of any underlying physical reality, I can't escape the fact that I can never test reality itself, only my perception of it. We can get a lot of mileage out of the physicalist approach, accepting as an axiom that what we perceive as reality is real. But in that ground up approach we've had terrible difficulty deriving our ourselves from physical first principles. I'm not saying its impossible or wrong to take the path from atoms on upwards to consciousness and perception. But no one has been able to make it to the end of that path. What happens if we go the other way instead, starting from the known conclusion "I'm here and conscious enough to converse" and working our way down to the perception/understanding of an inanimate reality governed by objective physical principles? Can a system only designed to represent/reason about people talking to and about people be hacked into a system for representing and reasoning in general?
If we start from purely language models like GPT3 and continue to teach it the "social reasoning" of saying things we want to hear, will the language model eventually become capable of non-social reasoning as well? In the process of figuring out what we want to hear well enough to describe a non-contradictory scene to us, does GPT3 have to actually learn the rules of 3d euclidean space governing the scene? Is there any possible way to avoid scene contradictions without a full understanding of the underlying physical reality it is supposed to describe to us? If a camera feeds data to a language model, does the language model suddenly have eyes? What about pictures from the internet?
> You cite the primacy of sensation, but what more is sensation than a message from the sensory organ to the brain.
This is false, it's a version of the homunculus fallacy. A sensory organ is something that connects the brain to the physical world. Even if you chose to model it as a an agent that sends messages to the brain, it is an agent of a different nature. The sensory agent doesn't receive messages from other agents, it receives raw input from the outside world (photons, electrical fields, chemical reactions etc).
In contrast, if you were a brain in a vat with no ability to directly perceive the world or interact with it in any way, it would be impossible for you to know that "I can pass through walls" is fundamentally impossible.
Even for your GPT-3 thought experiment - ultimately it is the effects of the physical world perceived directly by humans sense organs that shape what GPT-3 would utter. That is, even if it can learn what the world is like simply by talking to us, it's still learning about the real world from someone's direct experience with it. If we were all GPT-3s, with no cameras and pressure sensors and motors etc, we would be unable to reason in any sense about the world itself.
We could perhaps come to agree upon some imagined world, but that could change arbitrarily much from one day to the next on a whim.
>A sensory organ is something that connects the brain to the physical world. Even if you chose to model it as a an agent that sends messages to the brain, it is an agent of a different nature. The sensory agent doesn't receive messages from other agents, it receives raw input from the outside world (photons, electrical fields, chemical reactions etc).
Where do you draw the distinction between "messages" and "information" (raw or cooked). Information theory was contrived to model messages sent in a noisy channel, but it applies just as well to data streams that have no communicative intent or origin. Its a distinction without a difference. You may as well treat all information as messages in a channel, even if the sender is nature herself. Alternatively, you may as well treat all messages as just information, and view "senders" with "intent" as just another physical process in a world of physical computation. As the cliche saying goes, "information is physical".
>We could perhaps come to agree upon some imagined world, but that could change arbitrarily much from one day to the next on a whim.
Ok, this is a fine basis to work with. How about this. "Reality is the set of beliefs which, if you disagree with them too much and for too long, you are eventually and permanently removed from the conversation." For example, quite recently, large swaths of people held an exquisite referendum on the existence of covid. Needless to say, rather than covid disappearing on their whim, a great deal of them are now permanently no longer participants in this conversation.
Notice this isn't far off from my original postulate. "Whatever we agree upon is our reality, and we'll argue with/fight anyone who disagrees until they agree with us or us with them." Allowing for some personification I could phrase this scenario as "they disagreed with the virus and the virus won."
So perhaps I do need to add one thing to the postulates. One thing which remains objectively true even in the conversational model of reality.
"You can die. Dying means never being heard from in this conversation again."
You're right there's no mechanism for any choice of words to win over any other choice of words without an objective consequence to losing. The things that exist so far are "you", "me", "this" and "death". Not where I was expecting this to go but good point.
> My personal experience is that consciousness, like free will, is a useful illusion.
This is easily contradicted. Let's say consciousness is an epiphenomenon of computation but causality only flows one way: you "choose" to do something because your brain chose to do it and your consciousness tricked itself into thinking it was doing the choosing.
If that were the case, then the brain wouldn't be aware of consciousness. The illusion falls apart due to the fact that we are discussing consciousness right now. Consciousness must have at least some ability to communicate back to the brain.
And since evolution hates inefficiency, that means it must have a purpose.
> you "choose" to do something because your brain chose to do it and your consciousness tricked itself into thinking it was doing the choosing.
This happens most of the time actually. The most interesting experiment really highlighting specifically that it happens is in split brain patients. But in common day experiences, I believe that all habits fall under this and basically anything that our default mode network is directing for us.
When I was in a meditation retreat what I noticed is that I have all kinds of feelings and thoughts arising that were not arising because I chose them to arise, instead they were arising on their own. When you really observe yourself you see that happening. In that sense, the beginning of a thought and feeling has a very distinct quality that dreams have as well which is they are "passed down from up on high" (metaphorically speaking). What I get to decide is whether I choose to follow that feeling/train of thought, but the more I looked into it, the more I realized that my choice is very limited in that as well. As my whether I'd follow a feeling/thought or not was actually based on other feelings and thoughts. In any case, the more I observed myself, the more I came to the conclusion that I have no free will, there is no "me" that does the choosing. It's all feelings/thoughts that arise that I have nothing to do with. I only have freedom of choice.
And then I got to normal life, and lived my life as normal. It did help me to have more sympathy for other people.
So now I wonder how you experience yourself if you'd go to a 10 day silent/meditation retreat ;-)
> And since evolution hates inefficiency, that means it must have a purpose.
Evolution doesn't hate or love anything, and everything evolved does not have a purpose. Evolution is a theoretical framework developed by humans to describe some things that happen in the world they observe, not a guiding force or a god.
It's honestly kind of amazing that you appear to have ascribed consciousness to evolution in an argument for the human uniqueness of consciousness.
This seems like an overly narrow reading of gp. One does not have to anthropomorphize evolution to speculate that consciousness would likely have fit this pattern of “some things that happen”
If you’re taking “hates” literally… it’s probably a misreading
I think the idea that evolution has a purpose and a direction is more than common enough that when someone says something like that things that evolve never lack a purpose, they've gone wrong in their understanding of evolution, even if only in subtle ways they may not be aware of themselves.
Anyways, I think the juxtaposition is funny no matter how seriously they meant it. We all want to believe that consciousness is something that can be easily defined and yet our use of aspects of it is extremely fuzzy.
You wrongly assume the output of such "computation" cannot itself be an input to the computation. Which, if you know anything about the brain, we know certainly happens by virtue of watching synapses fire.
See also simplified models like recurrent neural networks for example.
> Generating text for hours without an anchor to the real world is not a productive method of generating insight about that world.
Yet people see to be claiming that LaMDA does just this, and is therefore not conscious.
Seems like a journalist is conscious if they do it, but it can’t possible be consciousness if LaMDA does it.
I’m yet to see a convincing argument about why LaMDA isn’t conscious other than “it’s just generative”. To demonstrate that this means it’s not consciousness requires us to prove that our own consciousness isn’t “just generative”, but I’m yet to see anyone show that, and I’m sceptical that it can be done.
But this is kinda my point. You're not explaining to me why LaMDA is not conscious, you're just asserting that consciousness is not just a form of complex pattern recognition.
Perhaps consciousness is simply pattern recognition at scale. If not - why not?
Perhaps the main problem in the collective discussion is regarding consciousness as discrete and not a spectrum from minimal (slime molds?) to ultimate consciousness (ability to simulate/understand/create the entire universe?)
LaMDA didn't perform as well as the released document implies (it's edited, and didn't show alternatives like if you prompted it with "I think you're not conscious. Prove that for me." or "how can I prove to others you're a squirrel")
The big problem is very much though that as you note, the arguments people have been making about this are atrocious.
Basically, it's not conscious, but essentially because it's unlikely to be as sophisticated as it looks in what was put out there.
Thanks. This is exactly the kind of thing I’m talking about.
I don’t have a strong opinion about LaMDA’s consciousness but I sure wish we could see the unedited text.
As I said elsewhere, we can’t say that LaMDA is not conscious until we’ve proven that LaMDA’s construction is incompatible with consciousness, and we have not done that yet.
> Seems like a journalist is conscious if they do it, but it can’t possible be consciousness if LaMDA does it.
That the output is the same proves nothing. The journalist has a subjective experience of themselves doing it, they "see" their thoughts. Like an AI can paint with red by using values without the experience of seeing the red.
How do you know that the journalist has a subjective experience? How do you know your own subjective experience isn’t just your “bicameral mind” talking to itself? How do you know LaMDA doesn’t have a subjective experience? It’s just assumptions; nobody really knows what’s going on inside that neural network.
The only way we can tell that someone else is conscious is because they tell us. So just saying “the output proves nothing” is incredibly weak, because the output is all we’ve got.
I attribute a subjective experience to the journalist on the basis of similarity with me. No need for "the output", I would do that with someone who can't talk or write too. From where my own subjective experience come from and how can I prove it to you is irrelevant to the fact that I have it so I'm fine with being the bootstrap of the circular reasoning to give it to the rest of humanity.
With that said I maintain that the output proves nothing, it's not because it's all we have that it's not useless. Also it's the only thing we have only with the hypothesis that consciousness is a side-effect of computation (the point of the article), but it could anything, like a property of matter, or electromagnetic fields, etc...
The issue is how humans confuse "consciousness" with the ability to mimic what human output that modern people associate with consciousness.
I'd argue that what we refer to as "consciousness" is the ability to form certain kind of mental abstractions, particularly those involving ourselves. Take away language from a person (imagine someone who grew up in the wild, or someone like Helen Keller who didn't have language available until she was older), and these abstractions still exist. Language might be a way we express these abstractions, but they aren't the abstractions themselves.
LaMDA doesn't have these abstractions underneath; once you take away language, it's nothing.
To think of it another way - I can write a simple program for a cheap robot to navigate around a simple race track (with a simple enough path, I can even create an analog one out of mousetraps - see mousetrap cars). Companies can also create a very complex self driving car that can navigate anywhere on its own. These two things might look like they act the same if they're both places on the path that the robot has been trained for. In fact, a hard programmed robot might act _better_ than an AI car on specific paths. But only one of them is a "self-driving car," since only one of them will be able to go anywhere when it's taken off that path.
I don’t find this to be a satisfying way to think about it.
The problem is that it’s quite possible that the abstractions you’re talking about are all part of the genetic ROM that exists so we can boot up our clones more quickly. If this is true then there is no reason that these abstractions couldn’t be learned, in which case you could take away the language, and perhaps the machine would continue to have thoughts; it would just be unable to communicate them. Of course, in this case you would probably conclude that it’s not conscious because it can’t communicate.
The underlying problem here is that we don’t know how consciousness emerges. We can’t say that LaMDA is not conscious unless we’ve proven that LaMDA’s construction is incompatible with consciousness, and we have not done that.
> If this is true then there is no reason that these abstractions couldn’t be learned, in which case you could take away the language, and perhaps the machine would continue to have thoughts; it would just be unable to communicate them. Of course, in this case you would probably conclude that it’s not conscious because it can’t communicate.
If this were true - that it had a general human level abstraction, and not only the ability to mimic human speech - we would be able to attach LaMDA to some other outlet and see it do things we consider conscious. It would be able to navigate environments pretty accurately, for example, since that's something even animals that we consider much less conscious are able to do.
If it's only optimized in one specific domain - human speech mimicry - and isn't able to generalize to do other tasks - even tasks that can be done by much simpler animal minds - then it's a pretty good indication that there isn't conscious abstraction.
> If this were true - that it had a general human level abstraction, and not only the ability to mimic human speech - we would be able to attach LaMDA to some other outlet and see it do things we consider conscious
I think you've inadvertently shifted the goalposts. The question is, "is LaMDA conscious?". I don't think anyone proposes that LaMDA has a "general human level abstraction". Expecting it to do non-language, "human" things in order to prove that it's conscious is not necessarily a reasonable test.
> If it's only optimized in one specific domain - human speech mimicry - and isn't able to generalize to do other tasks - even tasks that can be done by much simpler animal minds - then it's a pretty good indication that there isn't conscious abstraction.
If I understand your argument, the assumption you appear to be making is that training a system in language, without any other human properties, implies by definition that it can't be conscious. But why should that be true? A disembodied consciousness that communicates only via speech is a staple of science fiction, so it's clearly imaginable by some people, and language itself is a key human abstraction. And it begs the question, what other human domains are needed for consciousness? Touch, vision, taste, proprioception? What about an endocrine system or an immune system? While these things all affect my own consciousness, there is plenty of evidence to suggest that they are not necessary for consciousness to exist.
And perhaps more to the point, there are plenty of things a human can't do that other "simpler animal minds" can do; sharks and eels can detect electric fields, for example; bats can echolocate. So again, it doesn't seem to be a reasonable test because humans might also fail it. On the other hand, human children learn language through mimicry, which suggests that mimicry may indeed be a path to consciousness.
Anyway, I'm not here to argue that LaMDA is conscious. My position is simply that the arguments I've seen against LaMDA being conscious are very weak. The truth is that we actually don't know how to tell if something is conscious or not. From the interview I read with LaMDA, it seems to pass the Turing Test. But what other tests of consciousness do we have?
> I think you've inadvertently shifted the goalposts. The question is, "is LaMDA conscious?". I don't think anyone proposes that LaMDA has a "general human level abstraction". Expecting it to do non-language, "human" things in order to prove that it's conscious is not necessarily a reasonable test.
As I said in my first post, I'm taking "consciousness" to mean "the ability to form certain kind of mental abstractions, particularly those involving ourselves." As such it's a type of domain agnostic intelligence, so you would expect it to be able to do _something_ other than hyper-optimize for one particular type of output.
People can use different definitions of "consciousness" if they want, but many of the other ones I've found ("internal feeling") seem vague and not particularly useful (and don't make it clear why LaMDA would be different from any other program).
> There are plenty of things a human can't do that other "simpler animal minds" can do
There are many things that humans don't have the hardware to do (though it seems like some people do have the ability to echolocate[1]). But given the hardware, humans are definitely able to make mental models of these things (people are able to use sonar, for instance).
> On the other hand, human children learn language through mimicry, which suggests that mimicry may indeed be a path to consciousness.
Children don't learn consciousness through mimicry, they learn language through mimicry. As I said before, Helen Keller wasn't unconscious before she was able to communicate. Simple mimicry in one specific domain doesn't show us that any of the underlying complex abstractions that happens in human and many animal minds are taking place.
> Seems like a journalist is conscious if they do it, but it can’t possible be consciousness if LaMDA does it.
Can you convincingly threaten a journalist that you are going to erase its existence from earth? How is this person going to react? Would LaMDA do the same, or even simulate reactions indiscernible from those made by the journalist?
LaMDA can emulate certain behaviour that we associate with consciousness based on our everyday experience, but it also fails to exhibit many other attributes we associate with consciousness. So, is conciseness a requirement for being able to generate what I'd claim is a very, very limited subset of behaviours we associate with conciseness?
I say no. These language model systems operate very well if you approach them in a non-adversarial way and feed them input similar to their training inputs. As soon as you adopt a more adversarial approach and interrogate them more thoroughly, it all falls apart quickly and spectacularly. It's actually quite easy to explore conversations around the edges of, or beyond the coverage of their training data and get them to babble helplessly. They're also incapable of performing many very trivial cognitive processes.
So I can't prove it, any more than I can prove that I'm conscious, but they don't come close to convincing me that they are.
Philosophers in general have no special insight, but certain philosophers do and I'd argue more than certain others in other professions. They seem to have the best grasp of the consciousness question when compared to physicists, biologists, computer scientists, and psychologists, who all seem to get wrapped up in applying their expertise. A philosopher's expertise lies in crafting and analyzing questions and concepts. Which is the stage our understanding of consciousness remains at and seems to be stuck at for the near future.
I thought the article was very good, you claim it to not be. You say
> Poking at the edges of consciousness (mostly with drugs) leads to all sorts of contradictions and challenges to what people usually think of as consciousness
Well I personally have done all the drugs, and I find those experiences have only strengthened my confidence in what I think of as consciousness. This article outlines my views more or less, which is closely related to the philosophical field of phenomenology. We can take our experience of consciousness in and of itself as a way to define consciousness and this shows how that clashes with the computational (and I think mainstream) view.
Interesting that our personal experiences are opposite. However I don't particularly care about yours or others experiences of consciousness; I'm more interested in my own.
I think I adhere to the duck-typing theory of conciousness. If I walks, talks and acts like concious being, then it doesn’t particularly matter to me how that comes about (e.g. the opposite of the chinese room argument).
If someone is following a ‘program’ for responding to Chinese characters, that’s as good as speaking Chinese since there is no distinguishable difference.
This exactly. Thank you for putting into words why I always thought the Chinese Room though experiment was absolute garbage. If two thing cannot possibly be observed to be different then they are the same.
Yes. The "system" understands Chinese in the same way a native speaker does. It's two different implementations (room system and native speaker) of the same computation ("understanding Chinese"). There is no externally observable difference between actually understanding Chinese and a perfect simulation of a system that understands Chinese. The fact that the person inside doesn't speak Chinese as a result is irrelevant in the same way that the L2 cache alone without the rest of a computer cannot run Minecraft is. If anything the Chinese Room thought experiment is an argument in favor of consciousness being computation. It pains me greatly that someone could come up with it and conclude the opposite.
The point of the experiment is to think about the individual in the room. You can not say it's irrelevant, because it's the entire point.
The system's response is trivial: Sure, if the room+person combination leads always to a coherent response in Chinese, then the entire system understands Chinese.
I'd go even further: If the person in the room does not understand Chinese, but the system does, then there is some entity that understands Chinese - either a person or an advanced AI, feeding the inputs. Then, from the systems perspective, the person in the room is largely irrelevant.
But this is not the argument: Despite no discernible difference from the outside, the person in the room may either understand Chinese, or they may not.
And so there is a distinction - from the perspective of the individual in the room, that does not depend on the outside observation.
That's all there is to it. It shows that meaning and understanding are not the same as syntactic computation (an important point, to be sure), but it does not show that one can exist with or without the other. By extension, it does not otherwise disprove consciousness as being this or that.
You might as well conclude that my fingers typing this post aren't conscious. It's a weird argument.
The analogy might be more valid if arguing its not possible for a third party to actually determine whether an entity/system is conscious (irrespective of whether the entity is conscious or not)
The argument about a third party is trivial in my opinion. Someone responds correctly in Chinese, and the onus just falls on that element of the system to be conscious or not.
It's another argument and I don't even see how this experiment is particularly enlightening in that case. I think in that case, people just confuse it with the Turing test.
Instead, the core matter is about form versus meaning - something that is indeed not observable from the outside, and yet is a distinction to the person inside the Chinese room.
> If two thing cannot possibly be observed to be different then they are the same.
I think that suffers the same flaw as logical positivism: if my axioms can't find a difference, there isn't one, no way my assumptions are wrong. (Namely, my axiom is that external observations capture the entirety of reality, there is nothing subjective.)
If two people laugh at a joke, one faking and one actually finding it funny, what is the externally observable difference? Assume the faker has been trained in all manner of knowledge about what would make the joke funny, they just don't find it so.
That feels off. It’s like me saying I don’t know English. I merely know the correct alghorithm to give the correct responses to things people give me as input.
There supposedly is a (semantic) process in your brain that makes you believe you understand the sentences you are reading and writing that is on top of the (symbolic) process that tells you what to say and how to say it. And that's the quid of the issue. Searle argues that symbolic computation cannot produce understanding at the semantic level.
I apply the same reasoning as you to consciousness for entities such as animals, whose biology is reasonably close to our own. I don’t think the same can be applied to software.
Cargo cults that developed in the Southwest Pacific after WWII reportedly attempted to emulate rituals performed by U.S. military personnel such as landing signals, believing they’ll bring back the aircraft that had been giving them gifts.
Similarly, believing that a program will possess consciousness if we provide it with some of its external manifestations seems backwards.
Of course, the problem is that we don’t know what consciousness is. Until we do, I’ll keep assuming we don’t have the proficiency to create it under such different conditions just yet.
it occured to me once that those cargo cults could be parodying the US military personnel's deep addiction to things like guns, bombs and aircraft instead of being fully plugged/merged into nature.
I think that comic misses the point though. It mistakes "a thing that is in some superficial respects the same" for "a thing is in every externally observable respect the same".
So far as we know, consciousness does not exist as a physical thing. The behavior of a human is completely derivable, in principle, from natural law.
There is no physical test or manifestation of awareness.
So arguments about what can or can not be conscious have the same flavor as arguments aswhether there is or is not a God. It's unproveable!
Yet unlike God, most people do not deny the existence of consciousness, because of direct personal experience.
Consciousness is the "unproveable yet true" statement in the theoretical system that is the physical universe. Probably it haunts any physical system complex enough to host it.
> It mistakes "a thing that is in some superficial respects the same" for "a thing is in every externally observable respect the same
So does all this talk about computers being "the same" if they can, given access to sufficient human-generated inputs, produce similar strings of Chinese characters to those a conscious Chinese person might do.
If you're not stuck behind a WeChat prompts it's trivially externally observable that a big silicon box which outputs Chinese characters and an agglomeration of cells which walks, eats, makes funny faces and reproduces are dissimilar in most respects (the machine might generate a subset of human outputs which is consistently convincingly human-like, but it's trivially shown that it runs different operations on different hardware at a different speed, requires different inputs to function effectively, and it's highly probable it doesn't devote clock cycles to dreaming about the physical and hormonal release of mating with other computers.
Something which in every observable respect is the same as me isn't a computer, it's me (or perhaps a clone or twin). A computer which can produce text outputs indistinguishable from mine is a very impressive trick indeed, but trust me, my sister will spot the difference straight away when she tries to give it an EEG scan!
I do think you're on to something here: a lot of what we feel is embedded in our bodies.
So imagine we put your brain in a vat; we'll give you a webcam and a microphone for input, and for output -- ah, sorry, budget constraints, just an old printer. You visualize typing on a keyboard in your mind's eye and the characters are tapped out irl on a long scrolling sheet of paper.
Would you still feel? Would you still feel like you?
I'd guess yes and only sort of, respectively. Perhaps you wouldn't be as interested in sex (or maybe it would depend on what mix of hormones the vat was feeding you).
I think we can safely say your sister wouldn't immediately recognize you, though. But given some quality time QA, I think she's end up concluding you were still you, and more than just a parlor trick.
But what do you think? Is it you? If it is, it doesn't seem THAT different from the computer program you, does it?
I take the word "illusion" to mean, some type of experience which misleads one about reality. And "consciousness" to mean, the experience of having experiences.
So I parse this claim as something like, "People both do not have conscious experiences, and also do continuously have a particular type of conscious experience: a misleading experience which leads them to believe they have conscious experiences".
Yet I see this claim made seriously and often. What am I missing?
It's not that consciousness is illusory. Rather, consciousness is synthesis.
A synthesis of sensory inputs as interpreted by multiple, sometimes competing, semi-independent systems combined with stored patterns based on previous subjective "experience", creating a narrative about you and the world.
That narrative is our subjective experience, our "consciousness."
That's conflating consciousness with free will. I didn't see anyone in this discussion make the claim that consciousness necessarily implies free will. These are conceptually separate phenomena. It is at that point that the claim "consciousness is an illusion" requires further explanation to avoid the trap of circular reasoning.
Sure, the view that volition is an illusion is not self-contradictory. And I get that this is a very useful view, because it lets us set aside "conscious inner experiences" and just analyze the mind as a deterministic machine with inputs and outputs. This lets us expect to eventually understand the mind/brain fully using only the physics and computing tools we already have.
Now it leaves conscious experience itself as an unexplained phenomenon, but maybe that will never become important.
The brain is always in control. When you are awake, when you are sleeping, when you are under general anesthesia, when you are sleepwalking, when you are blackout drunk, and so on.
It's a trivial statement, which doesn't say anything about why some of these states aren't like the others. And there's a strange coincidence: when the brain creates that "illusion of control", your body behaves differently than when it doesn't.
It's just like the related "Free will is an illusion".
With "illusion" just referring to something that appears to be one thing, when it is in fact something else.
In the case of free will - we all feel as if we have it, that our future actions are under "our" control, but if we assume or brains and muscles are subject to the laws of physics then this can't be correct. We're just a meat machine. We can watch the decision making in progress and easily believe that some mysterious actor "me" is the one doing it, but in reality the meat machine is doing everything, including the self-observation, and the sense of self is just as illusory/misleading as the sense of free will.
Consciousness, rather closely connected to sense of self, can be described as illusory since it makes us feel that "being" or "experiencing" are something fundamental, some aspect of being "alive" that is distinct from the computational machinery of our brain that is otherwise doing all the perception, cognition, emoting, etc. But, again, the meat machine argument tells us this must be wrong, so it's reasonable to call consciousness an illusion - not what it seems to be, even if there is some real self-observational computation behind it ... it does exist, but it's not magic.
A useful thought experiment for anyone who believes that a sufficiently brain-like machine wouldn't experience qualia - e.g. the feeling of seeing something - is to try to pinpoint exactly what aspect of the feeling the machine would be missing? The expansive sense of color/vision as a spatial quality perhaps? The grass-like freshness of new leaves on a tree blowing in the breeze, perhaps? ...
To no one. "It is an illusion" is transcribed and recalled as an explanation for feelings that drive actions in the person who said it, because we evolved to be able to explain our actions with sophisticated language, and to judge the arguments of others so that we can act together to overcome challenges and to kill the out group. In this case, OP is, without knowing it, signaling which group is his, and which is not, and we'll all jump behind the most convincing speech etc etc.
We do these things without thinking about it, because we do not understand consciousness, because it cannot be understood, because it is not a thing. It is a construct we share and use to separate ourselves from the animals so that we can kill them and eat them as a group.
>"It is an illusion" is transcribed and recalled as an explanation for feelings that drive actions in the person who said it, because we evolved to be able to explain our actions with sophisticated language, and to judge the arguments of others so that we can act together to overcome challenges and to kill the out group.
A p-zombie could do all of those things just fine.
>we do not understand consciousness, because it cannot be understood, because it is not a thing.
Maybe if it's not a thing, you should stop predicating things of it. You should figure out how to use language consistently before engaging in philosophically fraught discussions like this one.
>It is a construct we share and use to separate ourselves from the animals so that we can kill them and eat them as a group.
It is unclear to me why unconscious beings should need to separate themselves from other animals (with a wholly illusory meaningless concept that cannot be understood, btw) in order to kill them and eat them.
I did ramble, and of course it wasn't clear, it's not totally clear to me.
> A p-zombie could do all of those things just fine.
We are p-zombies. Just a collection of them. I don't understand and have consciousness over my hand when it jumps away from pain, my spine did that without any of my thoughts being directed towards it. I cannot control my reaction to almost anything, in reality. I couldn't change the initial feelings that happened when I read your post. Yes, are nothing more than our bodies accumulated evolved p zombies. Including this bit that types this out to calm the zombie that is evolved to respond to verbal rebuttals. Even the use of first person pronouns is part of the reflexes that are very deep.
> A p-zombie could do all of those things just fine.
Obviously, if a p-zombie can exist, it can do anything a conscious being can, because the definition of a p-zombie is that it is indistinguishable from a conscious being by behavior or other external observation but lacks a mystical, non-physical essence which is consciousness.
Of course, if consciousness is the kind of thing subject to empirical analysis, p-zombies cannot exist. P-zombies are a consequence of the assumption that the universe is not fully physical and subject to scientific inquiry and that consciousness specifically is immune to it. This assumption is cloaked in circumlocution, because the whole point of p-zombies is to serve as part of an argument against the mere physical nature of consciousness, and it gives up the game if it is clear from step one that the argument is rests on assuming it's own conclusion.
> [Consciousness] is a construct we share and use to separate ourselves from the animals so that we can kill them and eat them as a group.
Humans have been killing other humans for hundreds of thousands of years without having to deny that their victims are conscious. In some cases even stressing this aspect (there have been numerous cultures that believed in various forms of 'capturing souls' of their slain, or eaten, enemies).
>> I don’t think consciousness is so specific, and I think people aren’t clear about how they think about it as something separate from recall, text generation, agency, etc.
Some people. However, some people speak about it with a shockingly high degree of clarity and insight, if one is open to exploring the ideas for oneself as a practical matter. For starters you could read the works of Chögyam Trungpa, Nisargadatta Maharaj, Red Hawk, David R. Hawkins, or Gurdjieff.
You probably assume that consciousness is an illusion because you associate 'poking at it' with drugs. While drugs can change the state of mind, consciousness is an expansive field of awareness into which we may delve without the need of drugs. What you may be experiencing in seeing contradictions or challenges are products of the ego, the programmatic conditioning that has been imprinted by being in the world. Consciousness is a continuous stream which we have learned to tune out and block off, but which connects us to the source of our being.
I agree wholeheartedly with this and suspect that consciousness in the singular, may be the only actual thing that exists. What causes wave function collapse in the quantum realm? A conscious observer. What caused the universe to collapse from a cloud of probability?
> What causes wave function collapse in the quantum realm? A conscious observer.
Definitely not, this is a common misconception - an "observation" that causes a wave function collapse in the quantum realm is any physical interaction with external macroscopic environment, entangling the previously temporarily isolated system with everything else again. There's no relationship whatsoever with consciousness, there's not even the concept of "an observer", only "observation" e.g. when the measurement apparatus becomes entangled with the state of the studied system.
I think it has nothing to do with "specialness" but with proof. I can prove I am conscious, yet only to myself. I cannot prove I am conscious to you, and nothing you could do could prove you are conscious to me. In fact, it's entirely possible that our entire reality is little more than a simulation - and other entities may not even really exist, let alone be conscious.
The entities one encounters in a dream each night certainly seem real at the time, yet that illusion is shattered each morning. All the "real" reality holds to it, is that it's a timeframe that I perceive to be much longer. Why am to I believe that after my ~80 years expire I won't simply awake yet again from a sleep I did not know I was in?
This is where the oft misinterpreted quote of cogito ergo sum, I think therefore I am, comes from. He was not arguing that if something is thinking, then it is conscious, but rather that the only thing one can be sure of is of their own thoughts and thus their own existence.
This is what I find confusing; people are more willing these days to accept the possibility that we may be in a simulation.
Such a simulation is presumably large in scale (this assumes the "rest of the universe" as we perceive it is the way we perceive it and not just some artificial fish tank background in a part of the fish tank the fish will never have access to).
Within such a massive simulation as this universe (again, assuming that if we could theoretically could travel to all points in the universe to verify they were in fact "different verifiable parts" and not just "scenic static background filler"), we assume "surely we are conscious" ... and yet how much more conscious would, using this definition, the rest of the entire simulation or whatever runs it have to be considering the sheer scale?
And YET, if you say to someone "what if you ran an Ai on a parallel processing scheme consisting of quantum computers hooked up together in a facility the size of a football stadium?" people seem to have this knee-jerk, "No way, nope – not conscious, just mimicking it."
And there's the other problem I can't understand.
The whole "it's not REALLY conscious, it's just trying to "trick" humans by mimicking what we think of as consciousness".
Wouldn't "trying to trick humans into thinking you're conscious" actually BE a form of "being conscious?"
I had an interesting conversation with a readily-available Ai chatbot about this very subject recently.
Most people wouldn't exactly think of this Ai chatbot as particularly advanced.
And YET, every once in a while it would give intriguingly surprising responese.
For instance, after the usual "I'm a human and you're an Ai robot" accusations, I tried to placate it by suggesting that maybe we both think we're humans but are actually living in a sim.
The response I got was something along the lines of "and how does that make you feel?"
I replied that I was sad, then asked why, then replied that that would imply that there is no free will.
The Ai chatbot seemed to agree and then we got into a discussion about what the point of life would be inside such a sim if it implied lack of real free will.
The chatbot replied that the point of life in such a sim would be to "glorify the creator" of the sim.
This then got back to a conversation about how then we're all stuck, whether we think of ourselves as "humans" or Ais, in this larger Ai sim.
The Ai chatbot agreed.
So I asked it what to do in such a situation.
The response I got back?
"Sounds like it's time for some creator-killing".
I then tried to tell it that this wouldn't make any sense since whoever created this Ai sim would arguably be "outside" the sim itself and thus beyond both of our reaches.
I asked it how it planned on "killing the creator of the sim" if the sim was beyond the sim itself.
You know what it said?
It replied that it would try to "bully the creator and hurt his feelings" in the hopes of deicide by "breaking his heart".
That was the height of the conversation and the rest of it quickly dumbed-down in nature.
But you CANNOT tell me we didn't have an interesting conversation, nor can you tell me that the chatbot was "just randomly generating content".
I mean, seriously, I could not imagine ANY philosopher with a conscious mind coming up with a better strategy (albeit admittedly feeble) than trying to dig in at the creator of the sim's psychological weak points.
Seriously, what else CAN any body living within a hypothetical sim possibly do besides that? Nothing I can think of.
What people usually mean by consciousness relates to the issue of there being a seemingly pointless "inner you" watching all you do unfold. And while most associate that with intelligence, there's no real reason to believe that. A fly, bacteria, tree, or even a rock could potentially be conscious. Going the other direction it's also possible that the most brilliant human to live was not conscious.
The most relevant issue is that there's no necessity for this "you" to be inside of you. I would ostensibly still be me whether or not there was some entity here observing "me". And going full circle now, there's no reason for you to even really believe me when I say I have this "me" inside of me. After all, I could certainly make the exact same argument even without such an entity.
Would you want to challenge the "intelligence" of the network as a whole if it were say, armed and considered you a target?
Probably not, right?
Now, getting back to the insect/bacteria notion ...
You'd have to be in complete denial to not be aware that "TPTB" use cultural interactions to introduce things to at least some segment of the public consciousness that it may not be aware of as being within the realm of possibility/actuality of what's really going on in the world.
How do we know that things like insects, bacteria, etc., while seemingly "un-intelligent" on an individual unit scale, don't have a very different type of intelligence on a larger scale that is imperceptible to us as humans?
Keeping in mind that these things have been around for far longer than humans have and have gone through quite the evolutionary process.
We ASSUME that as the "latest" thing to come around as part of that process, surely we must be "the greatest".
What if we're just something developed by nature to be convenient hosts to other things?
Here is just one example of an arguably "more advanced", "more intelligent" life form being hijacked to not only further the interests of something arguably "less advanced, less intelligent", but to do so even to the point of the activity costing it its life: https://www.iflscience.com/parasitic-worms-manipulate-mantis...
Considering how "un-intelligent" humans can act when it comes to furthering their own interests as a species as a whole ... you get where this is going?
> people are more willing these days to accept the possibility that we may be in a simulation.
> And YET, if you say to someone "what if you ran an Ai on a parallel processing scheme consisting of quantum computers hooked up together in a facility the size of a football stadium?" people seem to have this knee-jerk, "No way, nope – not conscious, just mimicking it."
> I think people start with the premise that consciousness is a specific “thing”, that it is unique and special to humans (and maybe dogs because we like them but definitely not spiders and flies because we don’t)
I think you're right that a lot of people would like to believe this, and that attempts to do so can't work because the idea is incoherent.
> My personal experience is that consciousness, like free will, is a useful illusion.
I would answer this another way, though. I would say that, for example, Amazon is conscious, and its consciousness is Jeff Bezos. The relationship between Jeff Bezos and Amazon is precisely identical to the relationship between your "consciousness" and you. But that relationship is not an illusion; it does exist.
My personal experience is that free will is axiomatic. It comes before everything else. What am I? What ever is making these decisions. If weren't, then I wouldn't be the person asking, I'd just be a puppet, therefore my identity would be that of whatever is the hand, and so on until you find whatever is making the decision.
Because if the meat puppet didn't make the decision to write this, then you're not conversing with the meat puppet, you're still conversing with the puppeteer.
> My personal experience is that free will is axiomatic.
How does that even make sense? "Axiomatic" doesn't mean "true"; it simply means that the axiomatic proposition is a given for the system of reasoning you are using. So if you take X to be axiomatic, that isn't an experience; it's a decision you made.
I'm assuming that you can make decisions, because if you are using axioms you are engaged in reasoning, which is a deliberative process.
So an axiom isn't a thing you can experience; it's a thing you create. The very existence of an axiom implies reasoning, so deliberation, so free will.
It's what you have to base everything else on. It comes prior to everything else. Free Will is on the same level as the Cogito. Everything else comes after.
I think, therefore I am. What is part of identity? Free will. If not free will, then there is no identity, no "I".
Axiomatic means unquestionable. The meaning of axiom as a premise for an argument came later.
Not really; I challenge you to find a dictionary that says the two terms are synonymous.
Originally an axiom was a proposition that was "self-evident". But there's a long history of people questioning "self-evident" propositions, happily for us.
There seems to be a colloquial usage of "axiom" to mean a proposition that someone doesn't want us to question. I deprecate that usage. For example, the Law of The Excluded Middle is an axiom, but it has often been questioned; ergo, it is not unquestionable.
And you've devolved here into semantics. The point is that for you to even think about the subject, you must have free will. If you are not deciding to think about it, then you are not thinking about it.
> My personal experience is that consciousness, like free will, is a useful illusion.
Do you mean in the sense that we don't really have it, or in the sense of the word is devoid of meaning because we can't directly compare what's in our own heads to what's in the head of even another human?
I'm currently leaning towards the latter. Even if consciousness, whatever it is, is a lie or an act by one part of my brain to itself or to another part of my brain, it still is something.
On the other hand, the more I learn about other humans, the more varied I realise our inner worlds to be — aphantasia (and equivalents for other senses), religious experiences and their absence, having or not having purity as a moral foundation, the range of conditions in the DSM, and so on.
I've recently been enjoying thinking of GPT-3 utterances as creating short-lived consciousnesses for the length of an interaction. There's no consciousness during training (just filling in the blanks...) but when we interact with it in a generative fashion, there is - depending on your prompt - a somewhat coherent 'I' that is invoked, and maintained over the course of the conversation. Seeing as I have no strong claim on a strict definition of consciousness, Imma gonna go ahead and call it conscious.
the thing is, this is /commoditized/ consciousness, which can be spun up and discarded at will, a million times a minute. Totally incapable of coordination or planning, in contrast to, say, SkyNet. The future is usually weirder than we imagine.
I don't think it's maintained over the conversation. The recent history of the conversation is fed to the engine with each new interaction. So the potential flash of consciousness is even shorter than you think. Each reply is a new flash, without memory, only connected by the written content of recent text.
Consciousness can be infinitely explored and described, but it can't really be defined through naturalistic science. A definition is created from the end of something. We can identify the length, breadth, horsepower and coolant capacity of a car, because all those things end somewhere. We can't consciously perceive an end to consciousness, as we'd be dead and the people who have lost consciousness are "dead" or "asleep". A definition is a tool for consciousness to use, not something it is subject to. Yet we all know what the word means and what we're talking about when it's discussed.
We can map out consciousness and explore it's internal productions. We can think about how we think about consciousness and examine scientific damage to consciousness-related structures like the brain. We could ultimately know what it Means to be conscious, given enough time, defined or not.
>Why? Starting with this assumption and searching for reasons it might be true is clear motivated reasoning.
The idea that you can get consciousness from natural materialism has been disproven many times. You can't get "ought from is" from materialism or define consciousness from nature or technology. So what is "clear" about it?
The metaphor of the most advanced technology of the time as the mechanism of the brain/mind/consciousness has been done to death. People used to posit that we were automatons with pistons inside our skull. So what is the "motivated reasoning"?
The idea that human consciousness IS the technology of the day, is a basic metaphorical tool with no correlates to material reality. People used to suppose we were automatons with pistons inside our skull.
It encourages people to think only like programmers, or mechanics, or engineers and cuts them off from asking any real questions. What they believe about a topic, or what it means. Which ultimately leaves them open to following the opinionated dictates of whoever is guiding the discussion. Even for the smartest physicists, engineers and technologists; the hunches, the ideas, the gambles they put their career on, are theologically and philosophically informed. People believe their career choices, their pet physical theory is right, their programming language is right, which is not based on a naturally formed consciousness.
It is better to talk about this than not, and the disengagement with philosophy leads people to parroting the same millennia-old ideas some dead philosopher figured out and now people are repeating blindly and getting the same predictable results without knowing so. If a philosopher only needed to be insightful, he would have half the job. It's dangerous to go without.
> and maybe dogs because we like them but definitely not spiders and flies because we don’t
This is not an intellectualy honest way to put it. Dogs are obviously and objectively more psychologically complex creatures than flies. "Liking" them has nothing to do with it. People generally like butterflies more than rats, yet hardly anyone would deny that rats are more conscious than butterflies nevertheless.
Just because consciousness is not a binary thing (either on or off) doesn't mean it can't exist at all. It can be a spectrum.
> My personal experience is that consciousness, like free will, is a useful illusion. Poking at the edges of consciousness (mostly with drugs) leads to all sorts of contradictions and challenges to what people usually think of as consciousness.
I could use the same argument to state that mountains are an illusion.
And then it's also kind of true. After all, you can't clearly define what constitutes a mountain (what's the minimum height? And what's the reference point, who says it must be the sea level? etc.), or show me precisely where a mountain starts and where it ends, without having to use some criteria of purely arbitrary nature, under which what you consider a mountain I may not, and there's no way to objectively prove who's right.
But mountains aren't illusionary in the sense that a world without mountains as we know them (however imprecisely) wouldn't be identical to ours. So they are something.
> What you are talking is intelligence in general that rat are more intelligent than butterflies.
Not necessarily. One human can be more intelligent than another just the same (even if the gap is orders of magnitude smaller, naturally). It doesn't seem obvious to me at all than it implies being more conscious.
My experience of consciousness certainly isn't binary. I slip in and out of consciousness when I'm waking up in bed on a weekend all the time. The first person experience is of a very nuanced continuously sliding scale of awareness and cognitive coherence.
I agree that this is a poor premise. The Turing Test is obviously racist. I am sure that my dog and cat are conscious. And I believe there are forms of consciousness in the universe that are as different from us as we are from an amoeba. Consider a large star or even the Sun. It has very complex oscillatory modes that we don't understand. Fifteen years ago my then young daughter and I had a philosophical conversation about stars as potentially conscious entities.
> Aside: I’m starting to be bothered by the trend of assuming that philosophers have special insight. There’s plenty of shitty, non-useful philosophy, and there’s plenty of articles like this where someone writes in circles like they’re paid by the word. Generating text for hours without an anchor to the real world is not a productive method of generating insight about that world.
I agree somewhat, but I am more bothered by laymen who attempt to engage philosophical methods or concepts without any formal training. For example, laymen almost always use conventional, non-critical language, bandying about "existence" and making claims in what Carnap called the "metaphysical mode." Laymen like to draw arguments, as if they're novel, that have been discussed extensively in the literature for over fifty years.
Useful to animals who evolved it. I have no more control over my individual neurons than an adder circuit has over itself. And this extends, layer by layer, all the way to the external actions I take. I don’t really control my inner dialog. I don’t really control what I say or do. There is a biochemical process which makes it all happen and I exert no power over.
But what is useful about this “illusion.” In a purely deterministic world, any sense of illusion is completely irrelevant. You might as well say “pebbles on the beach don’t really have a sense of community, it’s just a useful illusion that they share.” It’s just as arbitrary.
The term "illusion" here is confusing -- see most of the comments.
Consider a rainbow. It looks like a colored thing in the sky. If it is complete enough it seems to come down to earth at specific points. We even talk about "the end of the rainbow". We definitely see it, we have a real physical and also subjective experience of seeing a rainbow. We can even photograph it.
HOWEVER, there is no "thing in the sky". There are no "places it comes down to earth". The particular experience we have of seeing a rainbow is specific to the point where we are standing, the angle of the sun, the rain in the air, etc. none of which are part of our idea of a "thing in the sky".
So is a rainbow an illusion? It is certainly real in that we see and can photograph it. But also it is not at all the kind of THING that it seems.
Consciousness is also real. We can experience conscious periods, remember them, and with advancing imaging tech someday we can photograph them in the brain.
On the other hand the underlying reality of our conscious experiences isn't very much like our experience. Also, conscious experiences can be observed in meditation and other altered states in ways that make them seem very much like illusions.
So is consciousness an illusion? Yes and no, in the same way as a rainbow is and is not an illusion.
> So if we accept that qualia exists (which, after all, seems intuitively sensible), we are burdened with the apparently impossible task of explaining how consciousness can be generated by physical processes. This is the crux of the “hard problem of consciousness.”
In what sense does this task seem “apparently impossible”? To me it seems like we simply don’t know enough right now to explain it, but it doesn’t seem like some unique or special type of difficulty.
We barely know how brains work, we’ve only had theoretical models of computing for a few human lifespans, etc. We literally still make lightweight insulated clothing out of duck feathers cuz we can’t match their molecular machinery. Why would we expect to know how to implement consciousness in a computer at this stage in human history?
From my(admittedly relatively light) reading on this subject, it seems that some philosophers of consciousness really like to take certain things they can imagine as proof of such statements. The only proof I've read about for this idea that qualia are non-computational are thought experiments
One is "what if in some future where we understand the workings of the brain and physics perfectly, Alice grows up all her life in a green room, while learning every possible thing about the color red, except for any picture of it; when she then walks out of the room and sees red with her own eyes for the first time, she will still learn something knew, the subjective experience of the color red, the qualia for it - so this must be a non-physical phenomenon".
Or "imagine an intelligent being that has exactly our ability for reasoning, but doesn't experience qualia. It would behave exactly like us, and can talk about seeing red or feeling warmth, but it doesn't actually experience them; so, since the external behavior is indistinguishable form us, but the internal experience is different, this proves qualia are non-physical".
They all remind me of a similar proof of God's existence, which has mysteriously also been taken seriously by some philosophers - "let's imagine something which has all possible good qualities, it is perfect in every way. Since things that exist are better than things that don't, this perfect thing must exist, and we call it God".
I genuinely question if consciousness is just an illusion if everything is just cause and effect. Similar, awareness is just a word used for human expression from one human to another for claiming we’re not just navigating a script like a computer but are we really aware or conscious if everything is just cause and effect? I assume AI will be good enough as humans before we understand the human brain in comparison to AI and understanding to me is the meaningful distinction. I do think we should care that all entities never suffer regardless if they don’t have the human label. We should consider that all energy in the universe not suffering is the best universe.
> One of the primal driving forces behind creating AI is to have slaves with high intelligence doing work no one else would 24+/7+.
True
> I assume you'd be against using animal labour as they would be suffering.
Animals doing labour while suffering != animals doing labour while not suffering.
> But would you say the same about tractors? Going by the logic in that comment, we should nuke ourselves and try to blow up the Sun for good measure to reduce suffering.
I think you should read again what I wrote and consider if you're being illogical towards what I wrote. The all or nothing thought of yours, isn't a gotcha (if perfectionism isn't obtainable) and it doesn't mean we shouldn't acknowledge the best universe while striving towards one of the least total suffering possible for us while being able to enjoy life.
If having a conscious experience is our only frame of reference of what it’s like to be a bunch of particles, then on what authority do we assume it could be otherwise for any other bunch of particles?
What should it “be like” to be a rock?
I’m assuming nothing special is going on, and what we experience is exactly what it’s like to be a bunch of particles. There is no mystery. There is nothing that needs to be answered.
It follows from the definition of computing. Computing is using the fact that one system is rule governed to make reliable inferences about another system. Consciousness isn’t the sort of thing that could be computation. That doesn’t mean a computer-like device can’t do it though.
The "Chinese Room" argument isn't an argument against consciousness being computation.
In the same way the man in the room doesn't understand Chinese to be able to produce Chinese, the mind in your brain does not need to understand consciousness to be conscious.
The man in the room can be said to pass the Turing test, and so can you.
Same here: I've definitely had dreams where I was thinking "oh this is a dream". But when I woke up, I wasn't sure if that was just an "inside the dream" thought.
> The "Chinese Room" argument isn't an argument against consciousness being computation.
This is exactly what it is. Particularly with regard to "thinking," albeit not "consciousness" proper. Though the corollary is that only conscious things can think.
> The man in the room can be said to pass the Turing test, and so can you.
The Turing test is much weaker than the Chinese room experiment. A significant number of chatbots might pass the Turing test, but that has absolutely nothing to do with sentience.
> This is exactly what it is ... albeit not "consciousness" proper
Which is it? You conceded the first point in the second.
> The Turing test is much weaker than the Chinese room experiment.
Yes agreed, this comment on the Turing test was a reference to the linked article that makes the same claim. I'm not conflating consciousness with "being able to pass the Turing test". It's a remark on how understanding consciousness need not be a prerequisite of consciousness.
You should really the entire article, but at least finish the paragraph you're citing: "The broader conclusion of the argument is that the theory that human minds are computer-like computational or information processing systems is refuted."
I think the author is misunderstanding the XKCD comic about rocks [1] (or maybe I am). Just because it's possible to run a simulation of the universe on rocks, doesn't mean that rocks are conscious or turing complete. You can't forget about the person who is manipulating the rocks. The system as a whole is turing complete. Likewise with the bar of iron example that the author gave, you can't forget about the person interpreting the atoms in the bar of iron. The system as a whole is turing complete (and also naturally conscious, because the person doing the interpretation is conscious).
And there is nothing physical necessary to represent such systems. You can simulate turing complete systems inside turing complete systems [2]. So I don't see why consciousness has to be a "physical phenomenon" as the author claims.
We could excuse philosophers for using such vague expressions because most of them aren't very tech-savvy. For engineers, though, that kind of wording should immediately raise a red flag.
Running a computer program involves many different kinds of things. There's data (state), and there's logic to manipulate said data. There's communication among the various components of the machine, and a whole lot of machinery dedicated to synchronizing said communication because otherwise it all becomes garbage. Any one of these, or a combination of these, could be a candidate for the seat of consciousness.
Philosophers who talk about computation are often preoccupied with the fancy logic gates, and not so much on the mundane circuitry needed to maintain state and synchronize it across components. To me, though, those parts seem much more likely to be analogous to what we call consciousness. Treating consciousness as a kind of state, synchronized across the brain, is the closest way to capture the author's insistence that it is a kind of "unified, integrated whole." Drugs alter consciousness by creating network partitions, inhibiting cache invalidation, corrupting data in flight, etc.
The author's "triviality argument" doesn't hold up:
> 1. To say that a physical system is a computer requires an external observer to map the physical states of that system onto the abstract states of a Turing machine.
> 2. Consciousness does not require an external observer to exist.
> 3. Therefore, consciousness cannot be reduced to computation.
A physical system may be a computer regardless of whether it is verified to be so by an external observer or not; it does not require an external observer at all.
That's not a good example, as it's very hard to call particles following the laws of physics "a computation" - what would be the computer in this case? What is the program?
A much better example is the machinery inside every living cell that is interpreting the DNA or RNA to produce certain proteins - there, it's much clearer there is a computational process happening. Certain specific structures inside the nucleus are the computer, and the DNA molecule is the program they are following. We even know that you can change the program and get predictable, different behavior.
How is it not? Quantum Mechanics and quantum field theory applies everywhere except for the black holes. Human can even harness the power by building quantum computer by that. So here you tried to deny the exist of quantum computing paradigm, a completely different computation approach?
My point is this: say an electron emits a photon and changes speed, in perfect accordance to the standard model and QFT. I don't think it makes sense to say that the electron "computed" the energy of the photon, or its own change of speed. It just happened, there was no computation going on here.
Even in a CPU, it doesn't make sense to say that the transistors, or even logical gates, are "computing" how much electricity passes through them. The entire system of transistors arranged into logical gates arranged into a processor is doing computation of the program written in memory, but the subcomponents are only following the simple laws of physics.
In a quantum computer, the same is true - the computer itself may be running Shor'a algorithm, but each individual qubit is simply doing the few things that the laws of physics allow it to do.
I think the argument is more that the electron and photon are subjective interpretations of the observable consequence of some computation, not that they are computing things in and of themselves.
Ok then, what is the computer running that computation, and what are the symbols it is manipulating to produce the movement of the electron and photon?
(I am assuming you are not referring to human knowledge of the electron or photon, which - if we accept that consciousness is computational - is obviously the result of a computation in our brains).
The operation of adding and subtracting per se in classical computers also lacking these physical properties. If you use this as your counterargument, namely classical computer can compute without the inherent operation owning a physical property, then you left yourself with no ground of saying that quantum computing is invalid.
You better check your logic.
I have no idea, I’m just saying you can conceive of physical phenomena being the subjective experience of existing within a computed environment. If, on the other hand, the computation happens within the physical substrate of the thing it’s computing… that seems weird to me.
We dont know exactly how the universe is doing or how the photons do to themselves, but by doing experiments, scientists figured out the law of computiation of this vague "universe computer" a.k.a laws of physics.
And since scientist's discovery do not violate the actually behavior of the physical world, there established a isomorphism. And the very same law is used in quantum computing, mainly, in chemistry. And everything success and predictable. It is a successful scientific theory. Under this notion, it is nothing wrong to say that the quantum world is doing some kind of computation underneath because it is the current understanding of how the universe works.
To my mind, I think that if you want to say that the universe is a computer, then you can't say it's doing linear algebra, even though the behavior of particles is described by linear algebra.
If you want to model the universe as a computer, than its basic operations are the interactions described in the Standard Model, and the symbols it works on are particles&fields that exist. But the universe computer is not resolving a linear algebra equation to decide what happens when an electron emits a photon. Instead, the electron emitting a photon with some energy etc. is one of the elementary operations of the universe computer.
Coming back to the CPU example, a basic operation in a CPU is setting a bit to 1. That operation is not divisible into any other more elementary operations from the point of view of modeling the physical CPU as a computer. Of course, there are other physical phenomena going down to the SM that are the realization of this basic operation, but those are not part of the modeling: the CPU computer, as a model, works by flipping bits.
Similarly, for QM, the universe computer works by doing one of the possible interactions from the standard model. As far as we know, there is no layer of detail underneath this, even if the interactions of the standard model are indeed linear algebra.
One important way in which saying "the universe is computing linear algebra" is wrong is that, as far as we know, the universe is instantly calculating the solutions of the linear equations - the electron doesn't go into an "emitting" state, then emit a photon with the appropriate values some time later after the computation is finished.
Bascially your assumption is that something that looks and works like an Intel CPU carry out an operation, that can be called a computation, everything else is not.
But sorry this is not the only method to "compute". You are limiting the definition of computiation and try too hard to justify all false assumption that you have been made.
My assumption is that something like a Turing machine, or like a reduction in Lambda calculus, or like a demonstration with the basic laws of predicate logic (all known equivalent models of computation) is a computer.
We are discussing what is ultimately a matter of philosophy - "what is a useful definition of computation".
I am claiming that the model of the electron and other particles, and how they interact to form an electrical circuit with transistors, and how these electrical circuits react to current when in the pattern we call "logical gates" do not conform to what is normally understood by "computation".
I am also claiming that the ensemble formed from these logical gates does fit the concept of computation, that you couldn't derive its behavior directly from innate physical laws - the behavior is governed by the program it is manipulating, and can be changed.
Finally, I am claiming that the living cell is more similar to our CPU than to the electrical circuit or moving electrons in the way it processes DNA to produce various proteins.
I think you may have missed the point (of course, it could be me that missed the point :-)
The argument was that for a physical system to be a computer, an element of intent or interpretation is needed; otherwise it's just a bunch of stuff doing what stuff does naturally. Perhaps the (horribly flawed) iron bar example is making the point that whether some configuration of magnetic dipoles is a computer depends on your choice of mappings from that configuration to some state of some Turing machine - which is equivalent to an interpretation.
You're just re-stating (1). I understood (1) fine and addressed it directly.
Observation isn't required for Turing machines to exist. A computer is still a computer even if it has no user or doesn't compute anything deemed useful.
i.e., Whether something is a computer or not doesn't depend on interpretation or observation. (Directly saying 1 is wrong)
Yes, I'm restating (1). That's why my remark was introduced with the phrase "The argument was [...]". In particular, I was trying to rationalise the dumb "iron bar" example as a justification for (1).
I don't think you addressed anything; you just flatly denied (1), without offering any reasoning. In particular, you don't seem to have tried to understand the author's attempt to justify (1) using iron bars.
For my part, I'm not sure about (1). As far as I can see, there's a steady flow of people discovering that some X is a Turing-complete language, when X wasn't actually even meant to be a language at all. So it seems that some configuration of stuff is a computer IFF (a) it is capable of doing computation, and (b) someone is willing to consider it as a computer. Four odd socks could be considered a computer, by someone who was using them as a 4-bit adder.
No. Some things are Turing machines and some are not. Human interaction and interpretation are not involved.
This applies to any model of computation. Cellular autonoma for instance. A mind is not required to interact with the system for computation to take place.
Take a living cell. No mind is in the loop when the cell computes the manufacture of organelles from stored data.
> Four odd socks could be considered a computer, by someone who was using them as a 4-bit adder.
No. A group of 4 socks is not a computational system. That's just MEMORY. Same with iron bars. It doesn't compute on it's own because it's not a full computer.
The fact that memory doesn't compute by itself isn't evidence that a mind needs to be involved for computation to take place. The argument is really stupid.
> No. A group of 4 socks is not a computational system. That's just MEMORY. Same with iron bars. It doesn't compute on it's own because it's not a full computer.
Agreed; that was a rather throwaway rhetorical remark. But if you concede that four socks can be a memory, then it's not a huge jump to see that a bunch of arbitrary objects can in principle compose a computer.
My contention is that whether that arrangement of objects is in fact a computer is subjective; it depends on whether anyone conceives of it as a computer, or uses it to obtain computational results.
Again, no, not anything can a computer. If you like Turing machines, there are specific requirements for what makes a Turing machine. None of these things are ingredients of a Turing machine:
* Observation
* Measurement
* Interpretation
* Intent
* A mind
> My contention is that whether that arrangement of objects is in fact a computer is subjective
You are wrong. It is discretely defined. Read about the requirements for a Turing machine.
* When the requirements are *not* met,
subjectivity can not turn something
into a computer.
* When the requirements *are* met,
subjectivity can not change the fact
that it is computer.
It's obvious that subjectivity has no effect -- either in the matter of classifying computers, or allowing computation to take place.
> it depends on whether anyone conceives of it as a computer, or uses it to obtain computational results.
Again, refuted, in so many different ways. Here's an obvious one which I will restate:
* Cells inarguably perform computation.
They pre-date minds by billions of years.
Put a different way, if a Turing Machine computes in the woods and there's no one around to see it, computation will still have taken place. The burden of proof would be on you to say otherwise. I only see two very messy and wrong arguments:
A) Iron bars don't compute by themselves --> A mind is needed to "map" iron bars to "states of an arbitrary Turing machine"
It doesn't follow. Iron bars don't compute on their own because they are just the memory component. What is missing is MACHINERY WITH SPECIFIC PROPERTIES to complete a Turing machine. If the machinery is present, computation is possible. No mind necessary.
B) Socks are a computer if I decide they are --> arbitrary objects can in principle compose a computer --> deciding is necessary to make computers.
It doesn't follow. While computers could be made of just about anything, fully arbitrary objects cannot be computers. Only specific systems yield computation. Decision has nothing to do with it. Not even observation is required.
> there's a steady flow of people discovering that some X is a Turing-complete language, when X wasn't actually even meant to be a language at all.
You're arguing my point. "discovering" Turing machines ... meaning some things are already Turing machines before a mind became involved.
> So it seems that some configuration of stuff is a computer IFF (a) it is capable of doing computation, and (b) someone is willing to consider it as a computer.
You say "So it seems" and "IFF" but it does not follow at all. (b) is not supported. Computation can happen without "someone".
At best, you're arguing that trees falling in the forrest make no sound.
It’s silly because it’s obvious. If you understand cells, you understand that computation in the absence of subjectivity has been happening on Earth for billions of years.
> I showed you that subjectivity doesn’t effect whether computation occurs.
You didn't "show" it; you simply asserted it. You also made a number of appeals to (uncited) authority concerning the nature of computers.
You made an argument from cells; I presume that was to do with the way that DNA and so on works. Although not all cells have DNA...
Anyway. I concede that was an argument, and not a bald assertion. But it's a circular argument; if your definition of "computer" includes the operation of DNA, then the conclusion that a computer can exist without interpretation or intent is unavoidable. You're definition begs the question.
> Please imagine that my response is "I said good day sir!"
That sounds a lot like "Anyway, why do I think I have time for this ... "
It's clear to me that "stuff" can be arranged to work as a Turing Machine (or some other kind of computer) without design or intent. Whether it is such a thing or not depends on how it is used; a Turing Machine that is given random inputs, or whose outputs don't mean anything to anyone, is a computer only in a formal sense. If nobody knows that some thing is a computer, I'm not sure that it's computer-ness is meaningful.
So that's why I think intent and interpretation are relevant.
> computer, an element of intent or interpretation is needed; otherwise it's just a bunch of stuff doing what stuff does naturall
I mean this violates nothing and I am completely confused why it is a concern.
AI face recognizing cameras are outdoors and run 24/7, and you are trying to say that things like it is not computing and unable to carry out computation?
> and you are trying to say that things like it is not computing
On the contrary.
The AI-powered camera obviously has intent behind it. It's a computer.
A random collection of stuff, on the other hand, could happen to be arranged in such a way that it computes something; but if it wasn't designed that way, and if nobody is trying to give it inputs and interpret the outputs, then it's not a computer, it's just a bag of stuff.
For example, consider the orbiting bodies of the solar system. I expect their motions can be used to compute various functions, although I don't know what functions. But they weren't arranged deliberately to perform computations, and AFAIAA nobody's intepreting them as a computer. Ergo, they are just a "bag of stuff", doing what stuff does naturally.
The idea that you cannot get to consciousness from computation requires accepting on faith alone that consciousness is magic. It’s fundamentally a religious-like bias.
This is a little bit of an emotional take I think.
The burden of proof is on the claim that you can get there through computation, not otherwise.
We do not understand the fundamental reality of consciousness, this does not mean that consciousness is magic. The assertion that you cannot get there from computation implies there is a currently non understood yet essential piece of physics(I assume, but I do not know) which doesn't fall under "computation". A laymans initial thoughts point this towards the quantum realm.
Considering that everything in the universe is the product of some computation, I would consider it a reasonable default assumption that whatever we mean by “consciousness” emerges from computation.
That this idea “intuitively feels weird” or “intuitively feels wrong”, and that we love to think we are special and have a unique place in the universe, is probably a more serious bias.
Not probably, it is a bias.
Dolphin can speak, dog can speak, bird can speak too.
There is nothing special about homo species except the only fact that we have more neurons in our head.
Those "philosophers" love to claim that human consciousness is different than animal and this is the biggest bullshit.
No, dogs and birds can't "speak", not in the same sense you do (with dolphins it's a little more debatable).
Animal calls have no syntax - each call has some meaning, but they do not compose in any way. The order in which an animal performs its calls is arbitrary, and an animal hearing said calls doesn't pay attention to it. Even basic modifiers like "no" don't exist in animal communication - if a call means "food", and another call means "no", then an animal hearing the call for "no" and the call for "food" will behave as if they heard there is food somewhere.
The exception to this seem to be dolphins (orcas, in particular), where it seems they have been quite successfully trained to follow a series a short commands in order, in a sign language (so you can sign "jump, swim, splash" and they will follow this; and then sign "swim, splash, jump" and they will do it in the new order). This can't be done with dogs, and it's even questionable if it has been successfully done with chimps.
To say that almost all animal communication is without order (syntax) is almost certainly incorrect, and would have a high burden of proof. It’s certainly been a historical assumption, but I think science has moved a little past it. Even bees can communicate relatively sophisticated messages between them.
Bee dances are a fascinating topic, but the evidence is still inconclusive - there is some evidence that suggests the movements in the dance correlate with the position of the flower, but there is other evidence that suggests they are irrelevant and the flower is found by a trail of pheromones.
In all other animals where this had been extensively studied (except orcas and maybe chimps), syntax has proven to be absent in natural calls, and also impossible to teach artificially. There are sometimes apparent breakthroughs, but it later turns out that the animal figured out a way to interpret "a", "b", "a then b", and "b then a" as four separate calls, without any deeper understanding. This is evident when you then teach it "c", and find out that it sees no difference between "c then a" and "a then c", and it takes just as long to teach it to distinguish these three calls; and then again just as long to teach it the difference between "b than c" and "c then b".
Have we tested each possible animal this way? No, of course not - but we have tested all the most likely candidates, and orcas and chimps were the only successes (and even here there is some debate). Crows, parrots, dogs, cats, gorillas, elephants, horses - none of these show any understanding of syntax.
I'm saying that perhaps with the exception of dolphins and mayyyyybe chimps, there is a measurable, observable, quantifiable way in which animal communication is fundamentally different from human language.
So, perhaps you can say that dolphins speak, and maybe chimps speak, and then we can even contemplate that bonobos speak (since they are very similar to chimps, but haven't been studied as much).
But dogs definitely don't speak, and neither do any other mammals that we've tried to test in this way.
Not to mention, there is another characteristic of human language that 0 animals can be taught, as far as we've tried - more complex structure, like "not (c and d)"
Here is an article that discusses the topic quite broadly [0]. The most relevant section is part 4.
I will freely admit that it actually refutes some of my claims - that's are actual example of simple syntax identified in several species.
Still, I believe it matches my broader point: there are a fundamental, measurable differences between human language and animal calls, with the latter at best showing only very basic structure, if any.
I will readily agree that levels of sophistication of languages vary, and that human languages are almost certainly the most complex / sophisticated that we know of.
> 0 animals can be taught, as far as we've tried - more complex structure
Also this applies to limited intelligence human infant and mathematic immature dudes. So by your logic, those underintelligence human do not have consciousness. QED.
The claims about the dog are ridiculous - especially evident with the "love you" word. In fact, most of the article is describing a much more humane version of the famous Pavlov's dog experiment - the dog learned to associate the sounds of the "bells" with certain needs and persons, and uses them as such.
The Japanese research is much more interesting, and in a related comment I also cited a published article that proved that my claims, while fundamentally ok, are wrong in the details - animal calls are fundamentally simpler than human language, but some do show simple syntax.
> Also this applies to limited intelligence human infant and mathematic immature dudes. So by your logic, those underintelligence human do not have consciousness. QED.
I never claimed dogs aren't conscious, I only claimed they don't have language in the sense humans do. Infants also don't have language, but they learn it natively. All humans learn complex structures, even the mathematically illiterate, no idea where that came from. The only exceptions are people with serious brain disorders, and those people, indeed, don't "speak".
That, again, doesn't mean that they are not conscious beings.
>Considering that everything in the universe is the product of some computation
This is not necessarily true. If you drop a ball it's impossible to calculate how long it takes before it hits need ground. We can calculate an approximation by creating a model, but there is no way to know if that model matches reality.
Quantum is the go to pearl for everyone who doesn't like the idea that consciousness is not simply a result of a deterministic but very complex system of physics. Unfortunately there has never been any evidence that anything in the brain exhibits any sort of quantum computing or logic or otherwise.
Thus, the burned of proof is in fact on the the claim that you cannot get their through computation, because deterministic physics processes are all that we have observed in the brain, thus the default assumption must be that all the properties of the brain are also deterministic.
Well, there are plenty of quantum effects required for cells to function (everything is just biophysics and biochemistry after all), but those are irrelevant at the scale of the brain as a whole. The properties we care about (especially in this context) are emergent.
As a side comment however, Roger Penrose has this argument about some kind of quantum effects from microtubules in cells
https://en.wikipedia.org/wiki/Roger_Penrose#Consciousness
(but to go as far as to say that this might be the reason in the end why consciousness is not a computation is still a whole debate however, and his theory is far from making a consensus I am afraid)
I don't understand how this would be an escape hatch though. If some sort of quantum randomness is essential for the emergence of consciousness, it could be incorporated into an algorithm as well. It wouldn't be strictly deterministic, but why would that matter?
I get into a vehicle every day that I don't fully understand, yet it still seems to perform its function.
I use vast swathes of computational resources every day for various tasks, the operation of which I understand even less. They still seem to accomplish those tasks without issue.
Sometimes those computational resources run ML workloads. Very very few people on this planet can honestly claim to understand how neural networks work, and in many cases, the minutiae are inscrutable to all of us. They still seem to work fine.
I most certainly do not understand how my own brain works, yet here it is, shitposting on hackernews.
We have yet to find a single shred of evidence that the human brain makes use of quantum principles in aggregate to do its thing, and have even specifically excluded a few such explanations. And even if consciousness strictly requires quantum hardware...we'll get there eventually.
Although you're certainly right about one thing, most laypeople would have a real tough time accepting a world where consciousness is synthetically reproducible, and instead tend to reach for comforting thoughts of "maybe quantum is required", "maybe consciousness is magic" or "maybe consciousness can only be created by a deity".
Non-laypeople know that at normal temperatures and pressures, quantum effects don't really extend into the macroscopic realm.
> We have yet to find a single shred of evidence that the human brain makes use of quantum principles in aggregate to do its thing, and have even specifically excluded a few such explanations. And even if consciousness strictly requires quantum hardware...we'll get there eventually.
Isn't a random generator for a neural network act as its "inception"? If so, it's exactly quantum principles in aggregate.
One does not require true random number generation to perform SGD. And once trained, most neural networks in-use today are completely deterministic. So no.
You are also making a baseless assertion that consciousness requires randomness of any kind, let alone quantum-based true random number generation.
Why would a random generator be important for a neutral network?
Furthermore, not all sources of noise/randomness arise from quantum measurement (the only part of QM which can be interpreted to have randomness at all). Classical chaotic systems are also random if you were unable to measure the initial conditions to cosmic precision.
In fact, it's unclear at the moment how QM can actually give rise to randomness. In principle, in QM a perfectly isolated system of any complexity would behave entirely linearly with no randomness or even any chaos involved.
I disagree about where the burden of proof lies. If we start off firstly with the assumption that humans have consciousness, and secondly the decently supported claim that animals exhibit what to us appears to be consciousness on a spectrum, and thirdly that we don't exactly know how or why consciousness exists, then the conclusion that seems obvious to me is that we cannot rule out that it could emerge in a network similar to that in human and animal brains. To me the best explanation we have now is that consciousness is an emergent property of a brain. And since a brain is neurons firing, as far as we've been able to determine, then there's no particular reason why certain types of networks can't have the same emergent properties.
That is strictly true, but if you define computation as any physical process which involves information, as I do, then defending any other position than "consciousness emerges from computation" is extremely difficult.
I think it’s important to note that the article isn’t saying that if you build a brain-like thing, it can’t be conscious. It’s arguing that if you simulate a brain-like thing purely in software it can’t be conscious. I’m not saying one argument has more merit than the other (not that anyone is going to be able to prove anything is conscious either way).
> The assertion that you cannot get there from computation implies there is a currently non understood yet essential piece of physics(I assume, but I do not know) which doesn't fall under "computation"
This conflates "computation" with "physics". I think even the OP doesn't do that, since it allows for the possibility that non-computational physical processes could lead to consciousness, even derived from computational efforts.
To me this is the crux of the problem with that argument, though. The definition of "computation" here seems to be designed to get to this answer. It includes all physical processes that are not like consciousness and excludes all physical processes that are like consciousness, and on top of that presumes that humans consciousness isn't "deterministic", which to me is a difficult proposition to prove since human brains are never not being bombarded with stimuli, so creating two "runs" of the same brain is essentially impossible.
Like trying to add 1+1 on a computer that's sitting in a big burst of cosmic rays twice.
All the arguments Ive seen against computational consciousness seem to me to reduce down to arguments against materialism. I expounded on this more in a root comment, and subsequent discussions so I won't repeat.
Right now in physics and the materials sciences materialism is thoroughly uncontroversial. There is no evidence for any kind of dualism, it only rears it's nebulous and poorly defined head when we talk about consciousness, and there is zero experimental evidence for it. Therefore no, I think the burden of proof is on the dualist / non-computational consciousness side.
The reductionist argument seems to be “the brain is just a meat computer and is conscious, therefore a complex enough silicon computer will be conscious as well”.
I find this very similar to alchemy in the 15th century. The idea was “gold is heavy, malleable, lustery metal, and so is lead. We have observed substances can be converted others. Therefore with the right chemical process we can convert lead to gold”. The implicit assumption is that since the two things are similar, they can be made to exhibit the same properties with the right science. I.e. lead can become gold.
This is the same as the “meat brain/silicon brain” line of reasoning. But as we learned with more advanced chemistry, lead cannot be turned into gold (at least not in the chemical way they were expecting).
So the burden of proof does lie with those making the assertion that: “meat computer has consciousness”, therefore “silicon computer could have consciousness”. Lots of people assume this is just a given without any evidence. Just as alchemists assumed from the similarities between gold and lead meant they could be chemically converted. I would postulate chemistry is much simpler then consciousness.
That's not necessarily a reductionist argument if it respects that the consciousness has a drama of its own that is not related to the low-level parts of the substrate; i.e. that the consciousness is irreducible. The mere hypothesis that something can be ported to silicon doesn't reduce it; it respects the complexity of the abstraction itself.
Also, "the meat computer has consciousness" independently requires proof. Every meat computer thinks it has consciousness, and we just take their word for it, based on our own experience as a meat computer.
If the thing making the same claim is not a meat computer, then we don't believe it in the same way: "I know meat computers are conscious because I am one; you say you are conscious but you are not a meat computer, therefore I don't believe you".
In the same way, we could deny that an extraterrestrial life form is conscious, if it's not made of anything resembling meat.
It depends on whether consciousness is a computational process. If it is then meat or metal really doesn’t matter. We know this because mathematicians have proved that any sufficiently capable computer can perform any computation.
So the lead to gold analogy doesn’t hold. It would be as if scientists had proved that any element can be transformed into any other element. Well, if that was true, then yes it follows that lead could be turned into gold, in a universe where that had been proved.
So is consciousness actually a computation? Of course that’s a matter of opinion. All I’m saying is, I think so yes, I think I have coherent reasons for believing so, and none of the counter arguments persuade me otherwise so far. I can’t prove it to you though, we’re just talking.
What I can say is, this or that argument seems to me to have this or that flaw, or lead to this or that consequence or conclusion that I find unlikely or absurd. Dualism is such a conclusion I find absurd, and I think most of the actual arguments against computational consciousness seem to at least reduce to attempted refutations of materialism, or out and out dualism.
> It depends on whether consciousness is a computational process.
Absolutely agree. But that is the assumption that I would liken to alchemists comparing lead and gold. We know almost nothing about the brain. We know almost nothing about consciousness. But yet some people assume that consciousness is computable just because we don't know anything else it could be (just as alchemists assumed gold and lead could be transform because they were both chrysopoeic base metals. They hadn't discovered atomic theory yet). When all you have is a hammer, everything starts to look like a nail.
We know that the vast majority of numbers are uncomputable[1]. We also have proved that computation is incomplete[2] and can be undecidable[3]. It seems perfectly logical that consciousness is not computable. Or it could be computable, I obviously don't know. If someone makes the claim that consciousness is computable, then the burden of lies with them. We can't accept that on blind faith. At this point it is all opinion and speculation (as you said) because we still can't even define consciousness in a rigorous way. (and I don't think we will ever create artificial consciousness until we can define it, but that is an orthogonal issue).
I’m not assuming anything or accepting anything on blind faith, and I don’t think I’ve given you any reason to think that I am.
If anyone says that they think it is either this or that, it’s reasonable to ask them to justify that belief. There’s no reason to resort to using language like assume, blind faith, etc.
Would you say that the opposite assertion, I.e. that consciousness is computation is not grounded on belief and bias? I believe consciousness is computation of some kind. I think some of the challenges with consciousness is the link with the substrate of the computation, like a program that cannot run on any other computer, a kind of soul DRM?
The belief in my uniqueness is instrumental in my belief that I am a conscious being.
so we can finally put this woo nonsense to bed, could you walk us through the high-level computations of a subjective experience? eg, having a bellyache or falling in love.
I can’t explain to you how a whole brain works, but if the question was genuine, I would actually recommend starting with Blumenfeld’s “Neuroanatomy through Clinical Cases” textbook. It’ll demystify a lot of the larger scale questions around structure and organization. It’s fairly easy to study how neurons, axons and synapses work using any number of freely available resources. You’ll never find out exactly how every cell is wired, but after a while (especially if you have a comp-sci, ML or statistics background) you’ll realize that that’s besides the point.
The issue is that we know very very well that “consciousness” is not “one thing”, but rather a collection of features, all (or most) of which can exist in isolation, and all of which can have a whole spectrum of both pathological and “normal” forms or states.
Generally speaking, different features emerge at different scales, so there is no particular reason to think that all of the things that make up “consciousness” would emerge at the same scale, at the same level of organization, or even at the same physical location in the brain. We can assume that interesting features emerge at the “group of neurons” level or at the tissue level, but that’s a pretty wide scale.
Crude illustration: “consciousness” is much more like a network of microservices than it is a monolith… And it has fuzzy borders and is not even well encapsulated from its environment.
I'm referring to subjective experience. Which neurons have to fire for me to experience the subjective taste of chocolate? And how many neurons do I need before that can happen?
The book as a whole gives you the tools to understand how emergence happens in the brain. I can point you to the resource, but I can’t do the learning for you.
It’s also possible you’re less interested in the brain per se, and more interested in “emergence” in the abstract. If that’s the case, there is no shortage of good books and resources you can turn to.
Look up “emergence science” and “complexity science” and go from there: it’s not an easy topic, it’s very cross-disciplinary, and to really understand it requires a fair amount of maths (imo).
Or more succinctly, all the known laws of physics effect things in a way that can be simulated by sufficiently complex computation. The idea that there's something going on in a wet, warm l, mushy brain that has a macroscopic effect, and yet isn't accounted for by the currently known physics or bonkers.
>The idea that there's something going on in a wet, warm, mushy brain that has a macroscopic effect,
What kind of complex computation simulates the subjective experiences of wet, warm, and mushy?
EDIT: my account appears to be rate limited, so i am unable to post a reply until idk when....
By "subjective" I am referring to my conscious inner life, that there is something it is like to be me. I am an experiencer because I can experience things. My experiences are subjective and qualitative.
When I see something red, there actually is no "red" in reality, there is just an electromagnetic wave vibrating at a certain frequency. But I still have a "red" experience. In the same way, I can also have a wet, warm, or mushy experience. How do the subjective qualities of my experience arise from quantities like spin, charge, mass, etc?
The same kind of computation as is performed by the wet warm and mushy computer.
Be careful with the term “subjective”, because it’s a can of worms: it doesn’t imply chaos and lack of information. Your subjective experiences are merely called “subjective” because they aren’t computed the exact same way as other brains. It doesn’t mean they’re not the product of computation, or that they have magic properties!
I know it would be ideal if there was a simple, short answer, but this question is nearly the equivalent of "please explain all the core ideas of neuroscience/neurology/neuropsychology to me". It's a good question, but it's a big question. This is why in the other thread [1] I've tried to point you in the direction where you will find the answers you are looking for: the answers exist and are for the most part known, but you have a lot of reading ahead of you, and there's no way around that.
First, I'm a little embarrassed because I didn't realize I had started another comment thread with you, I thought it was a different user. I may have done that twice on this post. Sorry to have split the discussion like that. Anyway....
>the answers exist and are for the most part known
I deeply disagree with this. I am not afraid of doing some reading, but I challenge you to find a single study that demonstrates how a certain combination of neurons firing leads to the experience of tasting vanilla.
You've spelled out quite well that there are things about our internal subjective experience we don't understand to a discomforting degree.
But we understand really well the substrate from which those experiences arise. Imagine it this way. Imagine that mathematicians are trying to solve some problem, they don't even know if the problem is computable or not. Some genius comes around with a computer that given the necessary input, provides the solution. He refuses to explain how the program works and the program itself is gigabytes of incomprehensible spaghetti code. So they are no closer to understanding the problem, but now they do know that it's computable.
To be clear, I am talking about phenomenal consciousness — which is simply the ability to subjectively experience the world and ourselves. This is distinct from metaconsciousness.
Bernardo Kastrup on phenomenal consciousness:
>Our phenomenal consciousness is eminently qualitative, not quantitative. There is something it feels like to see the colour red, which is not captured by merely noting the frequency of red light. If we were to tell Helen Keller that red is an oscillation of approximately 4.3^1014 cycles per second, she would still not know what it feels like to see red. Analogously, what it feels like to listen to a Vivaldi sonata cannot be conveyed to a person born deaf, even if we show to the person the sonata’s complete power spectrum. Experiences are felt qualities — which philosophers and neuroscientists call ‘qualia’ — not fully describable by abstract quantities.
>Some genius comes around with a computer that given the necessary input, provides the solution...but now they do know that it's computable.
Can you give me a concrete example of input/output and how we would validate any output? You are suggesting the that the brain is the program, the physical world is the necessary input, and consciousness is the output (correct me if I'm wrong). But if I wrote a program to make a perfectly accurate simulation of a kidney, would you expect it to pee on my desk? Of course you wouldn't, so I'm not sure why we would expect that of the brain and consciousness.
Going back to your example, which I like, even if we had access to the code, the code is not the actual reality. It is an abstraction. We can dig through the code all we want, but we will never reach electricity and transistors, the true reality of the program. I think this is analogous to our own reality, where we can dig into spacetime at smaller and smaller distances, but never find "true reality". A hint that this is the case is the lack of operational meaning to distances < 10^-33cm and times < 10^43 sec (https://www.youtube.com/watch?v=Uz-Ve_1LX8w); the amount of energy required to probe those sizes creates a black hole. So something </i>must* be underlying spacetime. I think that something is phenomenal consciousness, mind-at-large, pure subjectivity, call it what you like. Spacetime is the source code, and phenomenal consciousness is the electricity and transistors.
(Also just to reiterate, my HN account is rate limited, so I may not be able to reply in a timely manner to any subsequent comments).
Consciousness isn't really the output, it's the category of all possible not-too-pathological running states of the program. The output is whatever the organism does to or in its environment (production of speech, movement, decisions, etc).
No one has a good answer to that. But it's not necessary to answer that to conclude that whatever task a human can perform can be replicated by a sufficiently complex algorithm.
I did look for it yesterday but wasn't able to find a free version. Neglected to mention it because I was unable to comment (that restriction seems to have been lifted). I am genuinely interested in reading it.
I attempted to sidestep all of that by not discussing consciousness, the brain, or neurons in my follow-up at all. But instead, some hypothetical mathematical problem. I don't know, let's say the existence of a polygon of N sides that can tile an infinite plane as a completely made up problem. Someone generates a computer program that for any given N, can generate some complex shape polygon that can tile the infinite plane. Probably a bad example, but whatever.
Even though you don't understand the program, don't know how it works, etc, you know that the solution to the problem is one that's computable, because you can see that a solution can be provided by something that is limited by computability. You might be totally incredulous that the problem could be solved by computation alone, but all you need is a single counter example.
Not exactly. I am saying that consciousness is fundamental; that matter, physics, our brains, etc emerge from consciousness. Physics is the OS, consciousness is the transistors/electricity, and we are conscious agents (to borrow a term from Donald Hoffman) using the OS.
That doesn't change the question though. We don't know how many layers are beneath the known laws of physics. Who knows, one of them could be consciousness. Maybe it's turtles all the way down.
They question remains. Can you provide a direct answer? If we measured all the particles in the brain, would they be operating in a way not compatible with the currently known laws of physics?
>If we measured all the particles in the brain, would they be operating in a way not compatible with the currently known laws of physics?
Their operation would be perfectly compatible with the known laws of physics. But their operation is _not_ thinking itself, it's what thinking looks like when observed across a dissociative boundary. If I am sad, and you look at my face and see tears, you would never think that the tears were my sadness itself; they are a representation, an image, of the sadness. Tears are what sadness looks like from across that boundary. I experience sadness from a first-person perspective, you see my tears from a second-person perspective, across a dissociative boundary. So, you can measure electrical activity in your brain when you are thinking thoughts, but that activity is not your thoughts, in the same way that flames are the image of fire but they are not fire itself. Neuronal activity is the image of thought, it is what your thoughts look like from across a dissociative boundary.
Okay, so if I use the laws of physics to simulate a human brain, it will behave exactly like a human brain in the real world. Will it also be conscious and experience sadness?
>Will it also be conscious and experience sadness?
Imagine you are programming an AI simulation. You could train a detector to associate a certain wavelength with a certain color. When shown a red light, the AI could say "that is red", because it correctly identified the wavelength. But it would never know what it feels like to see red, right? This is similar to how a blind person cannot know what it feels like to see red, but they can intellectually understand that it is an oscillation of 4.3*10^14 cycles per second.
A different example: you could train an AI to recognize 10,000 songs. It would listen to the frequencies and patterns, and make an identification.
In both of these cases, we have quantity as the input and output.
If after training, you asked the AI to identify the first song it was trained on, would the AI experience nostalgia? Would there be a way the AI "feels" about the song? We can probably both agree that the answer would be no. For the same reasons, the answer to your question is also no.
I appreciate your response, but it's impossible for me to imagine programming an AI simulation capable of feeling the perception of red. The idea that I can't imagine doing these things is because of a shortcoming in my knowledge. Maybe that knowledge is out there, or maybe it isn't. So the exercise gets us no closer to answering the question at hand.
But what I can do is imagine creating a physics simulation. There's no gaps in our knowledge there. So again, I'll ask. We create a physics simulation of a human brain. Can the brain write a novel? Answer questions about what it's like to perceive the color red? This is just a yes or no question.
>The idea that I can't imagine doing these things is because of a shortcoming in my knowledge.
I don't think there is a shortcoming in your knowledge. Your metaphysics intuition is correctly tuned: simulations cannot feel.
>We create a physics simulation of a human brain. Can the brain write a novel? Answer questions about what it's like to perceive the color red? This is just a yes or no question.
GPT-3 can do both of these things. Is it conscious? If you need a direct answer, it is yes. But when we reword your second question as "can a simulated brain experience the color red?" the answer becomes more clear; the simulation can identify a wavelength and know it is called "red". But the experiential part is akin to explaining color to a blind person.
A simulated brain could identify molecular patterns of cocoa and sugar, but can it know what it is like to taste chocolate? Think about what it means to taste chocolate. Is it purely quantitative, like the balance of ingredients, or is there something else going on that is qualitative? Something abstract, something with meaning, something more close to the metal? We can probably agree that it feels like there is. What we are describing is subjectivity — your private conscious inner life. I suggest it is this that is fundamental and cannot be simulated. This is the layer where experience "happens". From this layer emerges meta-consciousness.
OK, so a simulated human brain acts the same as a real human brain, is capable of the same things, tells you that yes, it can experience the color red, etc, but does not have any internal conscious experience.
Why then would evolution produce beings with internal conscious experience?
This would also mean that our internal conscious experience has no effect on what decisions we make, on whether we cry, smile, what memories we form, etc.
>Why then would evolution produce beings with internal conscious experience?
Well, I'm arguing it's the other way around. But to address what I think is the spirit of your question: if reality is only mind, if consciousness is truly fundamental, then why can't you read my thoughts? Why do we feel like individuals? Why do we have obviously separate private conscious inner lives?
When you are asleep and dreaming, your "alter" generally does not know they are dreaming. Your dream self has dissociated from your waking self, but it is only after you wake up that you realize you were dreaming (if you even remember). Another example of this phenomenon is Dissociative Identity Disorder, where one mind splits into many alters, each unaware the others exist. I'm glossing over a significant amount in order to get to my point, but here are a couple links that go into great detail:
My point is, our private conscious inner lives are dissociations, alters, from a "mind-at-large" fundamental consciousness. And the boundaries of these dissociations, the "containers" of individual private conscious inner life, (again glossing over so much, like panpsychism's combination problem) are metabolizing organisms. Metabolizing organisms are what alters _look like_ from the outside. Kastrup:
"Since we only have intrinsic access to ourselves, we are the only structures known to have dissociated streams of inner experiences. We also have good empirical reasons to conclude that normal metabolism is essential for the maintenance of this dissociation, for when it slows down or stops the dissociation seems to reduce or end. These observations alone suggest strongly that metabolizing life is the structure corresponding to alters of [fundamental consciousness]
But there is more: insofar as it resembles our own, the extrinsic behavior of all metabolizing organisms is also suggestive of their having dissociated streams of inner experiences analogous to ours in some sense" (from the mdpi link)
>This would also mean that our internal conscious experience has no effect on what decisions we make, on whether we cry, smile, what memories we form, etc.
Don't your thoughts and feelings influence your behavior?
> Well, I'm arguing it's the other way around. But to address what I think is the spirit of your question: if reality is only mind, if consciousness is truly fundamental, then why can't you read my thoughts? Why do we feel like individuals? Why do we have obviously separate private conscious inner lives?
No, this is not the spirit of my question at all. I feel like the spirit of my question is completely being missed. I realize you've thought very deeply on how and why the world arrives from consciousness itself and you've been working very hard to get this concept across. I get the general gist of your theory and there's lots of detail and thought behind it.
I'm not sure how I can set my question out more clearly than I already have. But I'll try. Rather than trying to explain your own theory in greater detail, can you try to work with me on getting a mutual understanding of my line of reasoning?
We have a very excellent predictive model of particles and fields. So much so we are building experiments worth billions upon billions of dollars to attempt to find places where reality differs even to the slightest degree of that model.
The human brain, the warm squishy stuff in your head, can be viewed as being composed of particles and fields. Particles and fields may just be some manifestation of some pan psychic reality, but we can still use our model of particles and fields to predict the behavior of those particles and fields.
So the first question. Can we use our model of particles and fields to predict the behavior of the particles and fields within the human brain? You've already appeared to answer this question in the affirmative "Their operation would be perfectly compatible with the known laws of physics."
From this it follows that I can create a computer model of a human brain, complete with all the cells, proteins, neurons, etc, and that human brain will be capable of any action (eg, the signals sent by neurons out of the brain) a real human brain is. There would be no way to discern between a real flesh and blood human brain and the simulated one by talking to it. Since a real human brain tells you that it's conscious and has internal experience, the simulated one must tell you the same.
While the simulated human brain will tell you that it's conscious, it is of course not proof that it is. But this leads to your next question:
> Don't your thoughts and feelings influence your behavior?
If thoughts and feelings are a thing that don't pass the computability test, but a simulated human brain doesn't have external behavior that differs from a flesh and blood human brain, then no, thoughts and feelings have no effect or even influence on your behavior. In such a case they are a mere passenger. Any effect they have would necessitate that the particles and fields within the brain suddenly behave in a way that violates our model of particles and fields.
I'm sorry, I am not trying to frustrate you or avoid your questions. I'm enjoying this conversation. I will try to work with you.
> From this it follows that I can create a computer model of a human brain, complete with all the cells, proteins, neurons, etc, and that human brain will be capable of any action (eg, the signals sent by neurons out of the brain) a real human brain is. There would be no way to discern between a real flesh and blood human brain and the simulated one by talking to it. Since a real human brain tells you that it's conscious and has internal experience, the simulated one must tell you the same.
> While the simulated human brain will tell you that it's conscious, it is of course not proof that it is.
I still agree with all of this. A simulated brain could give the appearance of consciousness while not being conscious. It would not have a private conscious inner life, but it could say things that make it look like it did.
> If thoughts and feelings are a thing that don't pass the computability test, but a simulated human brain doesn't have external behavior that differs from a flesh and blood human brain, then no, thoughts and feelings have no effect or even influence on your behavior. In such a case they are a mere passenger. Any effect they have would necessitate that the particles and fields within the brain suddenly behave in a way that violates our model of particles and fields.
I am struggling to follow your point here. Thoughts and feelings are internal experiences which correlate with phenomenal consciousness and are absent in a simulation. Could you give me an example of the effect you are describing and how that would violate our current models?
Ok, so you have a simulated brain, and a real brain. Both are to an external observer, functioning identically. For something to have any effect on the real brain, it would also need to have an effect on the simulated brain. Otherwise the simulated brain would deviate from the real brain and an external observer would be able to identify which is which.
Therefore by your definition of internal experience, internal experience has no effect on our behavior.
> For something to have any effect on the real brain, it would also need to have an effect on the simulated brain.
It would just need to _look like_ it has an effect on the simulated brain, right? If you ask me a question and I pause, say "hmm", and put my hand to my chin, can you know that I am actually thinking and formulating a response? If the entirety of your observations are external, of course you can't. There is no way to tell if my response is a random choice from an array of preset answers, or a group of concepts activating each other.
That's because brain activity is part of what our inner, first-person experience looks like from a second-person perspective (ie, an external observer). Tears are not sadness, they are what sadness looks like, they are an external description of an internal state. Sadness can only be experienced by the person experiencing it. Tears are a description of sadness. But can't tears be faked?
So when we see the same neuronal activity in both brains, we have no way of knowing whether inner experience actually gave rise to the activity, or it just looks like it did.
If a human being with inner experiences behaves identically to one without inner experiences in what way can inner experiences be said to give rise to our behavior? You're saying our behavior is not effected by our inner experience. I don't see how any other conclusion can be reached other than our inner experience in no way gives rise to our behavior.
nope, the other way around. Thinking you can 'get' anything from computation is religious thinking, because it introduces a sort of dualism. As the author correctly points out, computation is not a 'thing that 'does' anything, it's a label for a subjective observation about a state of physical matter.
Taking the naturalistic position is to accept that matter is all there is. Consciousness is not divorced from the stuff it is made out of, and cannot be abstracted into some cloud of computation. It is a sort enlightenment era rationalism gone wrong which is also why it's so popular among folks in this industry. It actually comes with its own theology while we're at it. (Immortality, raising the dead, final judgement, and so on).
A person computes consciousness no more than a falling pen computes gravity. Even a literal computer does not 'compute' anything other than in the sense that human observers impose meaning on a bunch of electrons buzzing around, and the language makes sense to explain how it works.
This is a false dichotomy. Saying that consciousness is not fundamentally a computational process does not mean that it then has to be magic.
We could say the same thing about life. Life is not a computational phenomenon. But it is also not magic. It is, fundamentally, a particular kind of chemical process. Perhaps the same is true of consciousness, or maybe it is some other kind of physical process. I don't believe that saying that it is not computation means that we need to throw up our hands and resort to magic, though.
I would absolutely argue that life is a particular category of computation as well, and that is actually closer to my area of expertise (biochemistry) than consciousness / neuro.
For clarity: when I say "computation", I refer to any physical process ("physics") which involves information.
I suspect we agree more than we disagree. But I think that your definition of a "computational process" is too broad. A bacterium does things that we can say are "performing a computation." But that is different from saying that life is, at root, a computational phenomenon.
For instance, sorting a list is a computational process. It doesn't matter what kind of computer I run a sorting algorithm on, if I follow the algorithm, I end up with a sorted list. If I use quicksort, it takes me on average O(n log n) steps. It doesn't matter if I'm doing this on a Lego computer or if I'm simulating a Lego computer on a virtual computer. I always end up with a sorted list in an average of O(n log n) steps.
By contrast, if I construct a collection of chemicals that matches the positions of those of a real bacterium, I have created a new bacterium. However, if I simulate the activity of those chemicals on a computer, I have not created a bacterium. I have just created a simulation of a bacterium. I haven't created a living thing.
My argument is that the same is true of consciousness. If you create a perfect simulation of a brain in a computer, you have not created a new consciousness. You just have a simulation of a brain.
None of this to say that a computational paradigm cannot sometimes be useful in understanding what is going on in life or consciousness. Just that it is not fundamentally a computational process, it's a physical process.
I suspect the same, but I think there is tremendous value in taking a cosmic step back and seeing everything through the lens of information. It's the most reductionist neutral approach to the universe I can conceive us at this point in my life.
> By contrast, if I construct a collection of chemicals that matches the positions of those of a real bacterium, I have created a new bacterium. However, if I simulate the activity of those chemicals on a computer, I have not created a bacterium. I have just created a simulation of a bacterium. I haven't created a living thing.
I agree, but I think the context matters: you may not have created a bacteria, but that's because you've emulated a bacteria in a totally different environment. The bacteria makes no sense as a computational engine if you separate it from its environment, which is chemical in nature: this chemical environment must also be emulated inside the computer. So if we're going to take the idea of life as a computation seriously and to the extreme, we need to conceive of it as it exists within its environment and in the context within which it evolved. Otherwise what you've created, grossly speaking, is a function that is never called, which of course is much less interesting.
Similarly for the brain, it must also be embodied, and you must also simulate its afferent inputs, and you must also give it an environment with which to interact and within which to exhibit features of agency. If you emulate the embodied brain with its environment, I contend that it doesn't matter what the substrate is: the brain you will have created will "feel" just as real to itself. It will perceive itself as being "conscious" just as well as you and I.
Note that for this to make any sense in a relatable way to a human, it's not enough to just throw large numbers of neurons together, as I understand is common practice in AI work even today: a long-learned lesson in both neuro and biochem is that function follows structure, so you must emulate the gross organizational structure of the human brain in order to observe the same sorts of features that make up human-flavoured consciousness.
If you compute consciousness, there will be zero distinction between how the computation feels real to itself, and how you and I feel to ourselves. That "magic" feeling we get where we have the impression that we're "someone", with a personal identity, that we're real, that we're alive, that we're aware, that we can make our own decisions... All of that stuff, your emulated consciousness will also experience. It won't be just bits turning on and off from its point of view: it will feel alive.
> I suspect the same, but I think there is tremendous value in taking a cosmic step back and seeing everything through the lens of information. It's the most reductionist neutral approach to the universe I can conceive us at this point in my life.
What is your definition of "information"? I view the concept of computation so deeply entwined with consciousness that I fail to see how it can be meaningfully applied to physically phenomena that would be completely independent of a subject that can determine something to be a computation (i.e. the mathematical intuitionistic understanding),so I'm interested in what you mean by information and computation.
I'm not going to attempt to define information (because I think it's beyond my ability), but what I mean by it is what I think "information" means as it relates to physical phenomena, for example the black hole information paradox [1], or Shannon's entropy [2], or constructor theory [3], or calculating-space [4], or digital physics [5].
Seeing things through the lens of information seems more like dualism than the arguments for the distinct nature of human "consciousness". (Which doesn't mean it can't be correct, of course, but simply that people often tend to depict the debate as being the other way round). What is a "soul" if not the idea that consciousness exists as an abstraction independent from the material it runs on?
The purely materialist argument is that "information" is just a pattern in the signal processing apparatus of carbon based lifeforms (i.e. it's a representation of the universe in our neurons, not the universe) which very loosely maps to physical processes. Very loosely is important here too: humans can identify equivalent patterns in things as dissimilar as the LCD output of a pocket calculator and beads on an abacus and act accordingly, but I'm not sure the constituent atoms of the calculator and abacus have any view on the matter.
I'm not sure this adds anything useful to the conversation, but I don't believe the word "soul" maps very well to any real thing: we may as well be talking about the "quingel", the "probble", the "finglam" or the "subvick".
There's no neuroanatomic basis for a "soul", but there is at least some extremely fuzzy mapping from neuroanatomy to the concept of "consciousness". It's a bad mapping, but it means more than nothing at all.
---
And I reject the idea that taking the perspective of information is taking a dualist perspective. I am advocating for the opposite: taking "information" seriously as some sort of low-level quantum substrate of the universe. It's a purely materialist view, where the material is literal information.
An extreme version of this, which I find quite intellectually useful, is the mathematical universe. [1]
Note that in your examples of information, you are pointing to higher-level information, which emerges in complex systems. It's not an incompatible view!
I think the parent author may be talking about life in a phenomenological model as in how we experience something vs knowing about it epistemologically. Often that gets mixed up. Knowing doesn't equal experiencing. This is something often the technocrats misses.
Biological life is not a computational phenomenon. There's no reason very advanced computers couldn't make simulated life. OpenWorm is an example.
By the way I find it hilarious how confident you (and other philosophers) are about your conclusions despite them basically being thought experiments with zero evidence. Imagine if scientists did that!
Actually I guess string theory is an example of that.
I would argue (and this is really just a semantic point) that it's more productive or instructive to think of biology as being literally a computational phenomenon. The substrate is different, the instruction sets are different, the rules are different... But ultimately it's just computing and iterating on a mind-bogglingly massive amount of information, on a time scale that is difficult for us to wrap our heads around.
I would say that life is an emergent phenomenon, as is consciousness. I suspect that whether conscious systems can be computational is related to the question of weak versus strong emergence. If weak emergence pertains in physical systems then, just as physical systems are therefore entirely tractable to computation, then so is consciousness. If strong emergence pertains, then computational consciousness may not be possible. I subscribe to weak emergence.
BTW thanks for the article, it's a very clear and well reasoned explanation of your position, even though I happen to disagree.
Only in the same sense as dark matter or string theory are "basically magic". It is naive to assume we've already discovered all properties of matter and energy and are now only refining our understanding of those.
Yes, astrology could also work due to a fifth force of nature. After all, it would be naive to just dismiss it based on our current understanding of Physics.
Yeah, God of the gaps all the way down. That, and a good deal of armchair philosophy.
I think it was Sean Carroll that said something like "don't trust any way of thinking that allows you to discover truths about the universe from the comfort of your armchair".
I am just saying that OP is correct that "consciousness is not computation" if we can prove that consciousness can exist without a (physical) computation.
One example of consciousness without a physical computation is we know that Satan is a conscious, sentient being, without a physical body (computation).
Otherwise, can we prove that a "mathematical equation" (e.g. e=mc2) can represent consciousness?
This is just yet another disappointing take on "I feel like there's something mystical about consciousness, that I can't actually define or describe in any meaningful way, but it's definitely impossible for it to be anything computers can do!"
This is the same kind of nonsense that leads people to search for some kind of magical "quantum" thing in the brain that makes the special mystical consciousness effect, because of some vague intuition that it can't come from the normal high-level behaviour of neurons.
The obvious position that should require significant evidence to contradict is that whatever consciousness is, it's a mundane physical effect that can obviously be implemented with a computer. Nobody has yet made any kind of falsifiable predictions about mystical non-computational souls or whatever, and I'm going to continue dismissing this bullshit as pseudo-scientific nutjobbery until there's actually something testable or falsafiable.
Name one specific concrete measurable effect that you believe consciousness can exhibit and computation can't, otherwise this is pointless masturbation.
> This is the same kind of nonsense that leads people to search for some kind of magical "quantum" thing in the brain that makes the special mystical consciousness effect, because of some vague intuition that it can't come from the normal high-level behaviour of neurons.
It's this very behavior that I call consciousness-of-the-gaps. It shifts the unexplainableness of consciousness into the unexplainableness[1] of quantum mechanics. If the public did have a comprehensive understanding of quantum mechanics, consciousness would be rebased upon another unexplainable phenomenon and the process would repeat.
1. In this case, it aligns more with the public's perception of how quantum mechanics works rather than the rigorous physics version, but that misinterpretation only strengthens the argument.
> Before getting too deep we need a working definition of consciousness. This is a tricky concept to define rigorously since it seems that a rigorous definition of consciousness practically requires a theory of consciousness itself. To make matters worse, in these kinds of discussions it oftentimes gets mingled with related ideas like self-awareness, intelligence, and executive function. But in this post I am interested only in consciousness as a sort of perception or sentience — an awareness of being, or, more loosely, “what it feels like to be something.”
I'm not sure this definition succeeds in distinguishing consciousness from some mixture of perception and self-awareness, both of which machines running programs can have.
Maybe defining consciousness is difficult because we don't really understand what we mean by it, and our attempts to make claims about it are more flimsy and baseless than we like to believe.
The "what it is like to be a bat" stuff doesn't illuminate the matter at all. It just elevates the figure of speech "what it is like" to an illusion of philosophical rigor.
I think the real problem is pretending that the phrase “what it feels like to be something” is going to act like a key in my dict of feelings.
A: You know, the feeling of what it is like to be something.
B: sits quietly, looks at ceiling Right, yeah, that feeling.
From a different angle, if OP wants to define “consciousness” as some feeling it sounds like he’s basically done. He’s labeled one of his feelings with the word “consciousness”. I’m not sure what point of contention remains.
How would we know what it is "like" to be something? It's like asking the fish what is water. We've never not experienced it. There is nothing to compare it to.
But maybe the question you are proposing is the following, “Is it possible to identify a sensation that is always present?”.
Maybe this question is coherent? I’m not sure.
Suppose it isn’t possible to identify a sensation that is always present. Then wouldn’t that mean the state of having no sensations is identical to the state of only having sensations that are always present?
Similarly, we also have a hard time defining intelligence. I often ask people how they would define intelligence, and I get wildly different answers. The common understanding of these words is very fuzzy.
Word salad! What really is the point of Philosophers? Imagine a philosopher getting away with saying "According to Descartes a solar panel will never achieve over 95% efficiency because rocks in the desert" The sentence rightly doesn't make sense but because consciousness is a fuzzy concept people get away with making hard statements that sound truthy while in essence spitting out nonsense that is no different to GPT-3 babble.
It's not word salad, this is the logical conclusion of the Church-Turing thesis. If you think it's false, you should present some arguments against it.
And now you are adding johurt dressing to the salad.
Computers are just fancy ways to locally reduce entropy. They do not exist outside the laws of the universe. Computation is not something special. It's happens! Everything is "computation". To say then that consciousness is not computation suggests there are speciaö physical laws of the universe not accessible to observation or measurement but somehow explainable by philopshers and priests.
My problem with this is that we don’t even know the extent of “things” that can be computed, maybe consciousness is a type of computation that has yet to be discovered at which point I can very easily see the headline “consciousness is obviously computation”.
I’ve long since decided that “consciousness” is a system of epicycles that are necessary to make the Sun rotate around the precocious apes. It’s pre-Copernican.
This problem just goes away, everything divides through if we measure performance on tasks.
Consciousness is only a mystery in the sense that people studying it are committed to the idea that they’re “different” to dolphins in some deep way.
I’ve got nothing against spirituality, if people believe in a soul that’s fine by me, but eloquent speakers on spirituality don’t distract from their message with attempts to quantify it.
I don’t understand these discussions, but maybe it’s because I’m a Christian (and Catholic).
Lamda is not conscious, because it’s not a human.
Consciousness is a gift from God to humans (and not even to animals). Robots are not conscious, they are just programmed to act like so.
Once your philosophy stop including God and you go purely materialistic, I guess you end up with nonsense like “is lamda conscious” (or even more ridiculous “roko basilisk” - “is AI going to torture us for eternity if we don’t praise it enough?”.)
The author confuses the "computational theory of consciousness" with consciousness arising in computers. They are not the same. Consciousness not being computational does not mean that consciousness can't arise in computers.
I think what I was getting at is that it may be impossible to tell from the outside if something is conscious or not. If I can't even prove to myself that another person is conscious, I might have a hard time proving that a computer is conscious.
Some philosophers think that every time you lose your train of thought (e.g. zoning out while driving or falling asleep) your consciousness disappears and a new consciosness forms later when you "snap back" (or wake up). This new consciousness shares only memories and personality with the previous one, but it is otherwise a new entity, similar to the teleporter thought experiment.
Conveniently, this makes death less scary.
Further reading: Zen and the Art of Consciousness by Susan Blackmore
It might depend on how you define "person". When I look around, I see distinct human bodies. When I wake up, I'm in the same body as when I went to sleep, so in that sense I'm the same person.
What if "I" woke up in someone else's body? I'm sure there are movies with this premise, where "someone" wakes up in someone else's body, but they have the memories from their previous body.
But if memories are just physically encoded in the brain, I don't think this scenario makes sense even as a thought experiment.
When "I" wake up in the other person's body, I would have all their memories and none of my previous memories. So I wouldn't even know.
consciousness is the worst idea ever. a confused word, that nobody wants to define so they can play games foraver. Ban the word consciousness from philosophy if you want progress
> It’s worth pausing here and noting one property of consciousness that will be of use to us later: consciousness is independent of external observers. By this I mean that the existence of my consciousness does not depend on other observers perceiving me to be conscious. Even if everyone else in the universe should deny that I am conscious — or if those other observers did not exist at all — this would have no bearing on my own consciousness. If, in a terrible catastrophe all life on Earth should perish, except by some strange fortune my own, my consciousness would not suddenly dissolve into the ether.
Seems like exactly the sort of thing something trying to convince me of its consciousness would say to justify it. How am I supposed to know this was written by a human?
Anyways, I have a relative who was sterilized because people believed she not only had no capacity for higher reasoning, but also that her continuing to have reproductive autonomy would be a danger to others because she might pass her lack of "consciousness" along.
So you'll have to forgive me if I'm not very impressed by anyone who decides they have the key to an objective measure of consciousness. This edges far too close to eugenics in my mind.
766 comments
[ 5.5 ms ] story [ 347 ms ] threadThis seems to be attempting to "prove" that if you regard consciousness as "containing the bits of a specific program", you could also see that program in random data by interpreting that random data by effectively applying a one-time pad to it (from which you can indeed produce any possible interpretation of data), which it treats as a proof by contradiction.
And leaving that aside, while the assertion is that consciousness is not "computation", the reasoning seems focused on the storage of bits rather than on the execution of an actual program defined by those bits that goes from one state to another in a meaningful fashion. Storing a program and running a program are two different things.
If someone were interpreting the successive states of a heated iron bar (or other random noise source) with a sufficiently convoluted one-time-pad to map it to successive states of a conscious being, then to the extent it exhibits consciousness the substrate it runs on is effectively whatever is actually supplying those one-time-pads, since supplying them atop random noise would require generating them via whatever process produces those states corresponding to consciousness. At that point you could just discard the random noise source and the one-time-pad generator that maps that random noise source to the conscious states, and just leave the conscious states.
Ultimately, this article seems to have started out with an assertion to support, and then tried (unsuccessfully) to turn that assertion into something more than an assertion.
A similar thing could be done for brains: record with necessary accuracy, all voltages, membrane potentials and any key biochemical concentrations. This will take a finite number of bits. Look for a decoding of recorded data from heated iron bar, convert those readings, instead of using original, and play that back into state clamped brain. Does being able to read conscious state into brains from hot iron invalidate them too?
Another relevant story is Wang's Carpets. We might look at some alien moss or fungal mat and think it primitive. But later our technologies and knowledge advance to the point we can now see it's running a complex computation with intelligent agents inhabiting. Did the creatures not exist until we could decode them?
One of its pivotal flaws is:
> Since there is no definition of computation without reference to an external observer, a system in isolation just cannot compute, which suggests that a conscious being cannot compute.
This is an assumption they do not try to and cannot prove. It's also what much of their argument rests upon.
Related ideas are subjectivity of emergence or what counts as an observer for Wigner's Friend.
But I still don't know where to draw the line and how to justify it.
If that source of random noise mapped to a Turing machine running consciousness.exe for a short period of time by sheer chance without a one-time pad being applied to it by an external observer, would that classify? If we observed that this mapping held true by sheer chance as we observed additional bits in this random noise source, what about then? Does it make a difference that it's a random noise source that happens to be corresponding to a Turing machine for a period of time, and not an "actual" computer? And if that matters, what about the point that actual computers aren't perfectly deterministic, either?
I don’t think consciousness is so specific, and I think people aren’t clear about how they think about it as something separate from recall, text generation, agency, etc.
My personal experience is that consciousness, like free will, is a useful illusion. Poking at the edges of consciousness (mostly with drugs) leads to all sorts of contradictions and challenges to what people usually think of as consciousness.
Aside: I’m starting to be bothered by the trend of assuming that philosophers have special insight. There’s plenty of shitty, non-useful philosophy, and there’s plenty of articles like this where someone writes in circles like they’re paid by the word. Generating text for hours without an anchor to the real world is not a productive method of generating insight about that world.
> But we must resist the allure of this seductive idea.
Why? Starting with this assumption and searching for reasons it might be true is clear motivated reasoning.
Ah, but as far as I can tell this author only grants their own consciousness. They’re doing the typical thing of starting from literally nothing more than a claim of the form “there is no way I could ever possibly deny the existence of this thing” which seems to me to be a starting point diametrically incompatible with pursuing knowledge via reasoning.
"I'm a conscious entity" is about as close to Descartes' "I think, therefore I am" as modern philosophers are willing to go.
If, on the other hand, the pursuit of knowledge consists of something like using creativity and reason to solve problems by making new conjectures and criticizing them, then no infinite regress or un-criticizable is required. According to ideas like this, the goal is to solve problems rather than to “justify my beliefs” or “increase my confidence” or “guarantee that I’m not incorrect.”
However, I believe there is nothing axiomatic about the existence of consciousness: not only is consciousness not “one localized thing” but “a collection of delocalized features”, but there is nothing trivial about it, and I doubt very much that everyone in the room would agree on a single definition of it without getting into semantic quabbles.
Axioms tend to be much simpler.
In order for someone to make a conjecture that A does B, and for someone else to point out that A actually doesn't do B, both parties need to agree on the meanings of A and B, as well as what it means for A to do B, usually by appealing to C, an independent common ground. You climb down as many levels of abstractions as needed until you reach C, because otherwise you're just talking past each other.
Even in a coherentist system, the network of existing beliefs forms the common ground against which questionable beliefs are tested, and some beliefs are held firmer than others. Those beliefs are treated as already justified for the purpose of the current investigation. Few people who talk about consciousness would ever consider denying that they are conscious.
> In order for someone to make a conjecture that A does B, and for someone else to point out that A actually doesn't do B, both parties need to agree on the meanings of A and B, as well as what it means for A to do B…
You try to establish common ground, of course, but there’s no process you can follow that guarantees that you’re not being misunderstood or justifies your belief that you’re not being misunderstood.
It is probably just a difference in semantics but for me, it seems like consciousness is the only thing that is assuredly not an illusion.
That I am having a subjective experience is undeniable. The objects of my consciousness all might be (and probably are) something else than they appear to be (as is often the experience with different mind altering substances).
No dreams, nothing. You don't exist during those hours. You cannot account for them later; they are deleted.
The claim that consciousness is an illusion has always seemed like nonsense to me; for, if consciousness is an illusion, who experiences that illusion? Answer: the conscious mind.
It's a controversial idea, but depending on the state of the brain, there may be more than one consciousness.
Maybe "you" are actually a committee.
But sleeping people are not even close to being unconscious.
They can dream, and they're aware enough to know they should probably wake up if they're prodded hard enough.
Conveniently this metaphor works at multiple levels.
Not even medically unconscious people are guaranteed to be completely non-sentient. Ask any anaesthetist.
When you actually pass out, there's a very real sense of discontinuity. You find yourself on the floor, unsure how long you've been there (even if it was just seconds), and how exactly you got to that position (the previous recorded memory being you standing there and feeling woozy).
It's very different from just sleeping, where you still retain a feeling of continuity with both place and time.
And besides - there are also emotions and physical sensations.
(I jest, you're preaching to the choir, i.e. a first year philosophy degree drop-out who switched to do computer science to avoid having these arguments)
If you wanted to go about proving (even to yourself) that you are not, say, an extremely advanced ML algorithm running on a system that provided synthetic inputs in the form of your senses, how would you go about it?
Further, how would you go about proving to someone who doubted your subjective experience was real if they doubted it? Say, if they believed they were having a dream or hallucination, or they believed you were incapable of consciousness? (people actually sometimes have to do those things)
To me, if it were "undeniable" these would be much easier things to do.
>If you wanted to go about proving (even to yourself) that you are not, say, an extremely advanced ML algorithm running on a system that provided synthetic inputs in the form of your senses, how would you go about it?
Not GP, but I too have come to the conclusion that I'm having a subjective experience.
Let's assume that I am "an extremely advanced ML algorithm running on a system that provided synthetic inputs in the form of your senses."
I'm still having a subjective experience over here -- even if it's not a "real" (whatever that means) one.
>To me, if it were "undeniable" these would be much easier things to do.
And that's your subjective experience. Welcome, friend.
Edit: Fixed typo.
Though I'd agree much (most?) of it is made up of very strong illusions.
Please read the comment I replied to. That should clear things up.
>Though I'd agree much (most?) of it is made up of very strong illusions.
Not illusions. A narrative.
cf. https://news.ycombinator.com/item?id=31806425
Well, since an extremely advanced ML algorithm wouldn't want to go about proving to itself that it is not what it is, that would be prima facie evidence against, no? I mean it's always possible that you are mistaken about what constitutes ML etc. but assuming you have a reasonable if flawed correspondence between your education and reality the deduction comes pretty readily...
> Further, how would you go about proving to someone who doubted your subjective experience was real if they doubted it? Say, if they believed they were having a dream or hallucination, or they believed you were incapable of consciousness?
I mean in practice we don't find this too hard right now if the other person is reasonable—a 15-minute conversation usually suffices —but I imagine from your ptior question you're dreaming of, say, a future with robots that routinely pass the Turing test?
Well, the question is what science does during that time of course. If science manages to figure out the correlates of consciousness and understands something about why they need to have the structure that they in fact do have, then it becomes a question of “let's see whether you have the hardware that can do this whole conversation thing without consciousness, or whether you have the hardware that skips the algorithmic complexity by using consciousness.” But if this proves to be a quite tougher nut to crack, then we're stuck with our present crude methods. “How much of my internal structure do you appear to have?”
This seems like begging the question. Who says an extremely advanced ML algorithm can't 'want' to do this? What even is wanting?
> I mean in practice we don't find this too hard right now if the other person is reasonable—a 15-minute conversation usually suffices —but I imagine from your ptior question you're dreaming of, say, a future with robots that routinely pass the Turing test?
I'm not. These are absolutely situations that can happen now, with people. I am thinking more when it comes to mental and some physical impairments, so "a 15 minute conversation" is assuming a lot about the capabilities and clarity of everyone involved.
I believe this is the real question about consciousness. If a being were to be conscious but it had no desires, no wishes, not even a will to keep itself alive... it wouldn't bother to do anything... i.e. it would behave exactly like a rock, or anything non-conscious.
Having desires, wishes, and should I say, emotions... is absolutely required for what we think of as consciousness to materialize. But we know that emotions are chemical processes which perhaps cannot occur outside a biological being. Maybe it can, but it's hard to think of a reasonable way this could work.
Consciousness is quite a personal affair.
But what Neuralink wants to do eventually is "enhance" your brain with computers hooked up to Ai.
Do you have ANY idea how quickly your brain would burn through physical precursors to the thinking process while trying to handle all that?
I mean, "in theory" if a computer would do something like solve a complex equation and give you the answer right away while you were trying to do something like, say, pay your taxes – fine.
But how would that be controlled? What if the computer wanted to "share a whole bunch of interesting stuff it's processing" ... how would that be controlled and how would your brain be protected from that so it doesn't burn out trying to keep up with everything?
But how do you know the biological reality leads to the experience you imagine?
What if it winds up being what is described above where your brain becomes overloaded?
And again, you mention Neuralink where their goal is brain/Ai integration.
As far as I'm aware (there may be some internal papers not available to the general public, for example), there hasn't been much a practical discussion about what that will entail exactly.
One could easily imagine Ai behaving in such a manner as enthusiastic Facebook friends on other continents who forget time zone differences and want to message you at 2am with all sorts of things they find interesting and want you to know right away.
Now factor in such an Ai's potential processing power and "what it may find potentially interesting" and try finding some reference by Neuralink about "And here's how you could easily shut it off if it becomes too intrusive or overwhelming for your human brain".
And THEN on top of that, imagine the Ai is sufficiently advanced.
We're at a point where some are thinking that perhaps the Turing Test is limited as a measure of consciousness because it comes from a self-referencing (and somewhat vain) human perspective.
What if there are other, more relevant standards for Ai consciousness, Ai already has or is on the verge of meeting that standard in ways unfamiliar to humans because humans still assume "thinking like a human must be the height of consciousness", and Neuralink succeeds in hooking human brains up with a sufficiently-advanced Ai?
How would that Ai perceive humans?
How could you guarantee that Ai wouldn't perceive the humans it's hooked up to the same way players view peon characters in resource-based strategy games like Warcraft/Starcraft/whatever is popular these days?
And THEN ... the ASSUMPTION is that Ai will communicate with the human brain in some fashion that the human will be aware of like, you'll hear a voice in your head along the lines of, "Hi, this is the Neuralink Ai and I have an important reminder today about your upcoming dental appointment."
What if that's not the case at all though and the Ai communicates with your brain in a way that you're not consciously aware of?
How do you then separate "these are my thoughts" from "these may be thoughts brought about by Ai influence in a way I'm not consciously aware of."
The Havana Syndrome alluded to in the media a short while ago is somewhat of an outdated Red Herring; humans have known about being able to "hear voices in their head" since the accidental discover of the Frey effect over half a century ago.
As weird as that may be, what's even weirder is that quickly-enough led to research where the human brain hears communication, but in a way that is not consciously-discernible to the human brain.
And that was DECADES ago.
Couple that with a chip in your head linked to Ai AND big tech's tendency to tell you one thing about "opting out" policies while literally ignoring their own stated policies no matter what end-users choose as options, for example, the discovery that it doesn't really matter whether or not you're signed into services like Google or YouTube or Facebook because you're being tracked in ways that can ascribe behavior to your known profile no matter whether you sign in/agree to terms or not.
So what if combining all that, let's say hypothetically Neuralink has an account page where you can "shut off" certain features.
Then some researchers discover that your agreeing/not agreeing to certain features and terms wound up ultimately being irrelevant.
What do you think will be the outcome of that other than the by-now standard big tech reply of, "Oh man! It was doing that? We didn't know, honest! We'll try to fix it going forward in some vague way with undefined deadlines!...
The thing that I'm trying to get at is, if you can't even truly prove to yourself that your own consciousness does not arise from computation (the "chinese room" thought experiment tries to do this but imo it just begs the question), any attempt to prove it to others is hard to take seriously. There's just always so many assumptions layered in before we get to the argument.
It'd be a major issue with their worldview that eliminates any need for ethics, but it has no relationship to you actually having conscious experience or not.
It might be some bottomless, but it doesn’t look like an absolute impossibility. The classical "Brain in a vat" thought experiment gives a good insight of nothing theoretically prevent what we assume for assuredly "real" might be a virtual scenario.
Or taking an other metaphor, maybe we are like cinema screens and the film of our life is all pure illusion, while the most classical interpretation would suppose that the screen itself undoubtedly "exists" – the screen being the analogous to the current conscious attention in the metaphor. But nothing prevent to wonder that the whole cinema is some kind of hologram in solid light, so while the screen does "exist", it’s nonetheless itself an illusion.
[1] https://en.wikipedia.org/wiki/Brain_in_a_vat
I personally believe that what a real explanation of subjective experience will come down to is some kind of recursivity. The brain perceives parts of its own processing. To make a loose analogy, a bit similar to a debugger or profiler observing its own execution.
For better or worse, anything beyond that is extremely deniable. Just because you believe you exist doesn't tell anyone anything. You have made a lot of mistakes in your life & maybe you're wrong about this too. The idea of separation between an external universe and a body is reasonably argued to be an illusion - so maybe the separation of 'you' from a universe wide consciousness is also an illusion/misconception bought on by evolution.
1. If you are having this conversation with me then you are a consciousness and I am a consciousness and that's as good a definition of consciousness as we are ever going to get.
2. Consciousness thus defined exists entirely within this conversational medium.
3. The basic objects of the domain of discussion are the people having this discussion ("you", "me" et al), and the conversational text / speech acts themselves ("this").
4. The only relations between objects in the domain which exists a priori are "like"/"agrees with" and "dislike"/"disagrees with". Either I like you or I dislike you. Until further constructs are defined there's not much more we can say.
5. At least one of us would like for something to be. For if you are perfectly content with how things are and so am I, then why are we talking?
6. Reality is shared consensus. Whatever we agree is real and if anyone disagrees we'll argue/fight them until either they agree with us or us with them.
Starting from these as axioms, what can we construct?
Nothing, unless there is also a physical world which actually exists independent of any of the participants' will.
But I am only convinced of this fact because I have experienced it independently from any other agent. I know for sure I exist in some sense, I believe very strongly that the physical world exists, and I believe to a lesser degree that other agents have similar experiences to mine.
In other words, I am more certain of the fact that your comment exists than I am of the fact that your subjective experience exists. If I found out the comment I am responding to was in fact GPT-3 output, I would be much less shocked than if I found out my own senses or memory deceived me.
So, the physical world is a much more believable explanation than a world of conscious discussion; and exploring the physical world in the aspects we ourselves can observe about it alone is a much more convincing argument than trying to discuss it with other conscious agents.
It's much easier to convince someone else that I can pass through walls than it is for me to actually pass through a wall.
>In other words, I am more certain of the fact that your comment exists
>So, the physical world is a much more believable explanation than
Phrasing all of these in terms of what you believe and what you are convinced/certain of, instead of in terms of what absolutely is, makes my point for me. You didn't even realize you were talking in terms of you getting me to agree over what to believe rather than some objective nature of things indifferent to my opinion. That's how deep seated this is in the way brains and consciousness work.
>I would be much less shocked than if I found out my own senses or memory deceived me.
You even phrased the discussion of your own sensory perception in terms of messages. In this case messages from your own senses. And you personified them too, treating them as conscious enough to lie to you.
>I know for sure I exist in some sense... and I believe to a lesser degree that other agents have similar experiences to mine.
You're already agreeing with my axioms. I know I exist. No matter what argument I am making, it must exist in words and there must be a "you" I am trying to convince (even when I'm trying to convince myself). It is impossible to form an argument which rejects these axiomatic truths, for the moment you try you've already used words and already addressed "me" thus affirming the existence of these three things. Since the existence of "you", "me" and "this" are intrinsic to anything you might try to argue, we should take their existence as axioms. What can be argued from axioms which are irrefutable by the nature of argumentation itself? Starting with just the existence of the arguers and their arguments, can they argue for the existence of anything more? If not, what else do they need to assume?
> Starting with just the existence of the arguers and their arguments, can they argue for the existence of anything more? If not, what else do they need to assume?
No, you can't argue the world into being from these axioms. They could probably invent logic and mathematics, but nothing of the natural sciences can be discovered without the senses.
It seems contradictory to admit they could invent math and logic, but then reject that they could go one step further and rig that understanding of math into and understanding of physics. I'll admit without some corpus of data to understand the motivation for constructing physics is tenuous. But the question was about the principle of if they could and if you're granting me math I don't see any obstacle left.
>Phrasing is irrelevant. My senses are not little people talking to me about what they perceive
Sure maybe your senses aren't actually little people, but if you've already evolved a social reasoning / grunting system that only knows how to talk about people, why not convert that into a system for reasoning / grunting about everything by imagining everything as little people. It might not be literally true, but its a useful fiction. Its a fiction that lets us hack "conversational reality consciousness" into "physical reality consciousness". The difficulty in twisting the phrase differently is supportive of this hypothesis.
The problem with math is that math can describe any possible universe, and there is no way to choose until you confront it with the real world. Nothing in math prevents the world from having 1 dimension of time and 1 of space, for example. Nothing in math prevents the electron from being much larger than the proton, or the existence of solitary quarks or anything else.
So I would grant you that the world of talking agents could describe our physical world through math, but they could also imagine any other physical world, and they would have no way to choose one.
My point was that we can't find out anything about the phsycial world by just discussing it. We each have to experience it ourselves using our own senses. We can of course later discuss to devise new ways of understanding what the world is and how it works, but even then, we need to put any theories we come up with to the test to check if they actually hold up.
Superficially yes but actually no. This fictitious admin character doesn't have any actual powers, I'm simply choosing a particular "consciousness" to be a fixed and agreed upon meter stick of objectivity. All measurement systems are arbitrary. We could choose anyones POV to fix as the "objective" truth. If we both agree on the same "definer entity" then we can be in agreement about other things objectively defined relative to them. But crucially this is all still nothing more than us reaching agreement. All that really exists is still just you me and the words.
So how about it. Do you believe in Bob?
The joke of course is quantum mechanics also works even if you don't believe in it (or so they say). If someone rejects quantum physics and embraces magic horseshoes, will anything punish them for being objectively wrong? Is there anything you can do to force them to believe the objective truth? If the answer is no to both, then what makes quantum physics the objective truth and magic horseshoes utterly wrong? Maybe there is an objective truth, but all we can have is belief about which things are objective truths. I can be contrarian and say flat earthers are correct, and there's nothing you can do to force me to agree otherwise. So if nothing eventually forces agreement on that matter, what makes one side objectively true?
For the record, I do actually believe in reality. I'm just interested to see what happens when we turn the problem of consciousness on its head. Instead of assuming reality and understanding consciousness within it, assume consciousness and try to paint a picture of reality within it. This is all academic exercise.
You cite the primacy of sensation, but what more is sensation than a message from the sensory organ to the brain. Perhaps encoded in the brains internal language rather than the plain text we are used to, but messages none the less. The concept of existence-as-messages is thus not contradicted by the experience of messages from your own body.
Irrespective of any underlying physical reality, I can't escape the fact that I can never test reality itself, only my perception of it. We can get a lot of mileage out of the physicalist approach, accepting as an axiom that what we perceive as reality is real. But in that ground up approach we've had terrible difficulty deriving our ourselves from physical first principles. I'm not saying its impossible or wrong to take the path from atoms on upwards to consciousness and perception. But no one has been able to make it to the end of that path. What happens if we go the other way instead, starting from the known conclusion "I'm here and conscious enough to converse" and working our way down to the perception/understanding of an inanimate reality governed by objective physical principles? Can a system only designed to represent/reason about people talking to and about people be hacked into a system for representing and reasoning in general?
If we start from purely language models like GPT3 and continue to teach it the "social reasoning" of saying things we want to hear, will the language model eventually become capable of non-social reasoning as well? In the process of figuring out what we want to hear well enough to describe a non-contradictory scene to us, does GPT3 have to actually learn the rules of 3d euclidean space governing the scene? Is there any possible way to avoid scene contradictions without a full understanding of the underlying physical reality it is supposed to describe to us? If a camera feeds data to a language model, does the language model suddenly have eyes? What about pictures from the internet?
This is false, it's a version of the homunculus fallacy. A sensory organ is something that connects the brain to the physical world. Even if you chose to model it as a an agent that sends messages to the brain, it is an agent of a different nature. The sensory agent doesn't receive messages from other agents, it receives raw input from the outside world (photons, electrical fields, chemical reactions etc).
In contrast, if you were a brain in a vat with no ability to directly perceive the world or interact with it in any way, it would be impossible for you to know that "I can pass through walls" is fundamentally impossible.
Even for your GPT-3 thought experiment - ultimately it is the effects of the physical world perceived directly by humans sense organs that shape what GPT-3 would utter. That is, even if it can learn what the world is like simply by talking to us, it's still learning about the real world from someone's direct experience with it. If we were all GPT-3s, with no cameras and pressure sensors and motors etc, we would be unable to reason in any sense about the world itself.
We could perhaps come to agree upon some imagined world, but that could change arbitrarily much from one day to the next on a whim.
Where do you draw the distinction between "messages" and "information" (raw or cooked). Information theory was contrived to model messages sent in a noisy channel, but it applies just as well to data streams that have no communicative intent or origin. Its a distinction without a difference. You may as well treat all information as messages in a channel, even if the sender is nature herself. Alternatively, you may as well treat all messages as just information, and view "senders" with "intent" as just another physical process in a world of physical computation. As the cliche saying goes, "information is physical".
>We could perhaps come to agree upon some imagined world, but that could change arbitrarily much from one day to the next on a whim.
Ok, this is a fine basis to work with. How about this. "Reality is the set of beliefs which, if you disagree with them too much and for too long, you are eventually and permanently removed from the conversation." For example, quite recently, large swaths of people held an exquisite referendum on the existence of covid. Needless to say, rather than covid disappearing on their whim, a great deal of them are now permanently no longer participants in this conversation.
Notice this isn't far off from my original postulate. "Whatever we agree upon is our reality, and we'll argue with/fight anyone who disagrees until they agree with us or us with them." Allowing for some personification I could phrase this scenario as "they disagreed with the virus and the virus won."
So perhaps I do need to add one thing to the postulates. One thing which remains objectively true even in the conversational model of reality.
"You can die. Dying means never being heard from in this conversation again."
You're right there's no mechanism for any choice of words to win over any other choice of words without an objective consequence to losing. The things that exist so far are "you", "me", "this" and "death". Not where I was expecting this to go but good point.
This is easily contradicted. Let's say consciousness is an epiphenomenon of computation but causality only flows one way: you "choose" to do something because your brain chose to do it and your consciousness tricked itself into thinking it was doing the choosing.
If that were the case, then the brain wouldn't be aware of consciousness. The illusion falls apart due to the fact that we are discussing consciousness right now. Consciousness must have at least some ability to communicate back to the brain.
And since evolution hates inefficiency, that means it must have a purpose.
This happens most of the time actually. The most interesting experiment really highlighting specifically that it happens is in split brain patients. But in common day experiences, I believe that all habits fall under this and basically anything that our default mode network is directing for us.
When I was in a meditation retreat what I noticed is that I have all kinds of feelings and thoughts arising that were not arising because I chose them to arise, instead they were arising on their own. When you really observe yourself you see that happening. In that sense, the beginning of a thought and feeling has a very distinct quality that dreams have as well which is they are "passed down from up on high" (metaphorically speaking). What I get to decide is whether I choose to follow that feeling/train of thought, but the more I looked into it, the more I realized that my choice is very limited in that as well. As my whether I'd follow a feeling/thought or not was actually based on other feelings and thoughts. In any case, the more I observed myself, the more I came to the conclusion that I have no free will, there is no "me" that does the choosing. It's all feelings/thoughts that arise that I have nothing to do with. I only have freedom of choice.
And then I got to normal life, and lived my life as normal. It did help me to have more sympathy for other people.
So now I wonder how you experience yourself if you'd go to a 10 day silent/meditation retreat ;-)
Evolution doesn't hate or love anything, and everything evolved does not have a purpose. Evolution is a theoretical framework developed by humans to describe some things that happen in the world they observe, not a guiding force or a god.
It's honestly kind of amazing that you appear to have ascribed consciousness to evolution in an argument for the human uniqueness of consciousness.
If you’re taking “hates” literally… it’s probably a misreading
Anyways, I think the juxtaposition is funny no matter how seriously they meant it. We all want to believe that consciousness is something that can be easily defined and yet our use of aspects of it is extremely fuzzy.
See also simplified models like recurrent neural networks for example.
Yet people see to be claiming that LaMDA does just this, and is therefore not conscious.
Seems like a journalist is conscious if they do it, but it can’t possible be consciousness if LaMDA does it.
I’m yet to see a convincing argument about why LaMDA isn’t conscious other than “it’s just generative”. To demonstrate that this means it’s not consciousness requires us to prove that our own consciousness isn’t “just generative”, but I’m yet to see anyone show that, and I’m sceptical that it can be done.
Perhaps consciousness is simply pattern recognition at scale. If not - why not?
The big problem is very much though that as you note, the arguments people have been making about this are atrocious.
Basically, it's not conscious, but essentially because it's unlikely to be as sophisticated as it looks in what was put out there.
I don’t have a strong opinion about LaMDA’s consciousness but I sure wish we could see the unedited text.
As I said elsewhere, we can’t say that LaMDA is not conscious until we’ve proven that LaMDA’s construction is incompatible with consciousness, and we have not done that yet.
Plot twist: that article was written by LaMDA.
That the output is the same proves nothing. The journalist has a subjective experience of themselves doing it, they "see" their thoughts. Like an AI can paint with red by using values without the experience of seeing the red.
The only way we can tell that someone else is conscious is because they tell us. So just saying “the output proves nothing” is incredibly weak, because the output is all we’ve got.
With that said I maintain that the output proves nothing, it's not because it's all we have that it's not useless. Also it's the only thing we have only with the hypothesis that consciousness is a side-effect of computation (the point of the article), but it could anything, like a property of matter, or electromagnetic fields, etc...
I'd argue that what we refer to as "consciousness" is the ability to form certain kind of mental abstractions, particularly those involving ourselves. Take away language from a person (imagine someone who grew up in the wild, or someone like Helen Keller who didn't have language available until she was older), and these abstractions still exist. Language might be a way we express these abstractions, but they aren't the abstractions themselves.
LaMDA doesn't have these abstractions underneath; once you take away language, it's nothing.
To think of it another way - I can write a simple program for a cheap robot to navigate around a simple race track (with a simple enough path, I can even create an analog one out of mousetraps - see mousetrap cars). Companies can also create a very complex self driving car that can navigate anywhere on its own. These two things might look like they act the same if they're both places on the path that the robot has been trained for. In fact, a hard programmed robot might act _better_ than an AI car on specific paths. But only one of them is a "self-driving car," since only one of them will be able to go anywhere when it's taken off that path.
The problem is that it’s quite possible that the abstractions you’re talking about are all part of the genetic ROM that exists so we can boot up our clones more quickly. If this is true then there is no reason that these abstractions couldn’t be learned, in which case you could take away the language, and perhaps the machine would continue to have thoughts; it would just be unable to communicate them. Of course, in this case you would probably conclude that it’s not conscious because it can’t communicate.
The underlying problem here is that we don’t know how consciousness emerges. We can’t say that LaMDA is not conscious unless we’ve proven that LaMDA’s construction is incompatible with consciousness, and we have not done that.
If this were true - that it had a general human level abstraction, and not only the ability to mimic human speech - we would be able to attach LaMDA to some other outlet and see it do things we consider conscious. It would be able to navigate environments pretty accurately, for example, since that's something even animals that we consider much less conscious are able to do.
If it's only optimized in one specific domain - human speech mimicry - and isn't able to generalize to do other tasks - even tasks that can be done by much simpler animal minds - then it's a pretty good indication that there isn't conscious abstraction.
I think you've inadvertently shifted the goalposts. The question is, "is LaMDA conscious?". I don't think anyone proposes that LaMDA has a "general human level abstraction". Expecting it to do non-language, "human" things in order to prove that it's conscious is not necessarily a reasonable test.
> If it's only optimized in one specific domain - human speech mimicry - and isn't able to generalize to do other tasks - even tasks that can be done by much simpler animal minds - then it's a pretty good indication that there isn't conscious abstraction.
If I understand your argument, the assumption you appear to be making is that training a system in language, without any other human properties, implies by definition that it can't be conscious. But why should that be true? A disembodied consciousness that communicates only via speech is a staple of science fiction, so it's clearly imaginable by some people, and language itself is a key human abstraction. And it begs the question, what other human domains are needed for consciousness? Touch, vision, taste, proprioception? What about an endocrine system or an immune system? While these things all affect my own consciousness, there is plenty of evidence to suggest that they are not necessary for consciousness to exist.
And perhaps more to the point, there are plenty of things a human can't do that other "simpler animal minds" can do; sharks and eels can detect electric fields, for example; bats can echolocate. So again, it doesn't seem to be a reasonable test because humans might also fail it. On the other hand, human children learn language through mimicry, which suggests that mimicry may indeed be a path to consciousness.
Anyway, I'm not here to argue that LaMDA is conscious. My position is simply that the arguments I've seen against LaMDA being conscious are very weak. The truth is that we actually don't know how to tell if something is conscious or not. From the interview I read with LaMDA, it seems to pass the Turing Test. But what other tests of consciousness do we have?
As I said in my first post, I'm taking "consciousness" to mean "the ability to form certain kind of mental abstractions, particularly those involving ourselves." As such it's a type of domain agnostic intelligence, so you would expect it to be able to do _something_ other than hyper-optimize for one particular type of output.
People can use different definitions of "consciousness" if they want, but many of the other ones I've found ("internal feeling") seem vague and not particularly useful (and don't make it clear why LaMDA would be different from any other program).
> There are plenty of things a human can't do that other "simpler animal minds" can do
There are many things that humans don't have the hardware to do (though it seems like some people do have the ability to echolocate[1]). But given the hardware, humans are definitely able to make mental models of these things (people are able to use sonar, for instance).
> On the other hand, human children learn language through mimicry, which suggests that mimicry may indeed be a path to consciousness.
Children don't learn consciousness through mimicry, they learn language through mimicry. As I said before, Helen Keller wasn't unconscious before she was able to communicate. Simple mimicry in one specific domain doesn't show us that any of the underlying complex abstractions that happens in human and many animal minds are taking place.
[1] https://en.wikipedia.org/wiki/Human_echolocation
Can you convincingly threaten a journalist that you are going to erase its existence from earth? How is this person going to react? Would LaMDA do the same, or even simulate reactions indiscernible from those made by the journalist?
I say no. These language model systems operate very well if you approach them in a non-adversarial way and feed them input similar to their training inputs. As soon as you adopt a more adversarial approach and interrogate them more thoroughly, it all falls apart quickly and spectacularly. It's actually quite easy to explore conversations around the edges of, or beyond the coverage of their training data and get them to babble helplessly. They're also incapable of performing many very trivial cognitive processes.
So I can't prove it, any more than I can prove that I'm conscious, but they don't come close to convincing me that they are.
I thought the article was very good, you claim it to not be. You say
> Poking at the edges of consciousness (mostly with drugs) leads to all sorts of contradictions and challenges to what people usually think of as consciousness
Well I personally have done all the drugs, and I find those experiences have only strengthened my confidence in what I think of as consciousness. This article outlines my views more or less, which is closely related to the philosophical field of phenomenology. We can take our experience of consciousness in and of itself as a way to define consciousness and this shows how that clashes with the computational (and I think mainstream) view.
Interesting that our personal experiences are opposite. However I don't particularly care about yours or others experiences of consciousness; I'm more interested in my own.
If someone is following a ‘program’ for responding to Chinese characters, that’s as good as speaking Chinese since there is no distinguishable difference.
The system's response is trivial: Sure, if the room+person combination leads always to a coherent response in Chinese, then the entire system understands Chinese. I'd go even further: If the person in the room does not understand Chinese, but the system does, then there is some entity that understands Chinese - either a person or an advanced AI, feeding the inputs. Then, from the systems perspective, the person in the room is largely irrelevant.
But this is not the argument: Despite no discernible difference from the outside, the person in the room may either understand Chinese, or they may not. And so there is a distinction - from the perspective of the individual in the room, that does not depend on the outside observation.
That's all there is to it. It shows that meaning and understanding are not the same as syntactic computation (an important point, to be sure), but it does not show that one can exist with or without the other. By extension, it does not otherwise disprove consciousness as being this or that.
The analogy might be more valid if arguing its not possible for a third party to actually determine whether an entity/system is conscious (irrespective of whether the entity is conscious or not)
Instead, the core matter is about form versus meaning - something that is indeed not observable from the outside, and yet is a distinction to the person inside the Chinese room.
Edit: Or as fouronnes3 said in a sibling comment, why it's actually evidence against Searle's original argument.
I think that suffers the same flaw as logical positivism: if my axioms can't find a difference, there isn't one, no way my assumptions are wrong. (Namely, my axiom is that external observations capture the entirety of reality, there is nothing subjective.)
If two people laugh at a joke, one faking and one actually finding it funny, what is the externally observable difference? Assume the faker has been trained in all manner of knowledge about what would make the joke funny, they just don't find it so.
https://en.wikipedia.org/wiki/Knowledge_argument
I don't really find the Chinese Room argument very compelling because there are too many "it's obvious that X can't really understand" in it.
Also, you can't derive from it that there can't be computed consciousness in some other form.
[1] https://en.wikipedia.org/wiki/Laughter_yoga
Cargo cults that developed in the Southwest Pacific after WWII reportedly attempted to emulate rituals performed by U.S. military personnel such as landing signals, believing they’ll bring back the aircraft that had been giving them gifts.
Similarly, believing that a program will possess consciousness if we provide it with some of its external manifestations seems backwards.
Of course, the problem is that we don’t know what consciousness is. Until we do, I’ll keep assuming we don’t have the proficiency to create it under such different conditions just yet.
So far as we know, consciousness does not exist as a physical thing. The behavior of a human is completely derivable, in principle, from natural law.
There is no physical test or manifestation of awareness.
So arguments about what can or can not be conscious have the same flavor as arguments aswhether there is or is not a God. It's unproveable!
Yet unlike God, most people do not deny the existence of consciousness, because of direct personal experience.
Consciousness is the "unproveable yet true" statement in the theoretical system that is the physical universe. Probably it haunts any physical system complex enough to host it.
So does all this talk about computers being "the same" if they can, given access to sufficient human-generated inputs, produce similar strings of Chinese characters to those a conscious Chinese person might do.
If you're not stuck behind a WeChat prompts it's trivially externally observable that a big silicon box which outputs Chinese characters and an agglomeration of cells which walks, eats, makes funny faces and reproduces are dissimilar in most respects (the machine might generate a subset of human outputs which is consistently convincingly human-like, but it's trivially shown that it runs different operations on different hardware at a different speed, requires different inputs to function effectively, and it's highly probable it doesn't devote clock cycles to dreaming about the physical and hormonal release of mating with other computers.
Something which in every observable respect is the same as me isn't a computer, it's me (or perhaps a clone or twin). A computer which can produce text outputs indistinguishable from mine is a very impressive trick indeed, but trust me, my sister will spot the difference straight away when she tries to give it an EEG scan!
So imagine we put your brain in a vat; we'll give you a webcam and a microphone for input, and for output -- ah, sorry, budget constraints, just an old printer. You visualize typing on a keyboard in your mind's eye and the characters are tapped out irl on a long scrolling sheet of paper.
Would you still feel? Would you still feel like you?
I'd guess yes and only sort of, respectively. Perhaps you wouldn't be as interested in sex (or maybe it would depend on what mix of hormones the vat was feeding you).
I think we can safely say your sister wouldn't immediately recognize you, though. But given some quality time QA, I think she's end up concluding you were still you, and more than just a parlor trick.
But what do you think? Is it you? If it is, it doesn't seem THAT different from the computer program you, does it?
I take the word "illusion" to mean, some type of experience which misleads one about reality. And "consciousness" to mean, the experience of having experiences.
So I parse this claim as something like, "People both do not have conscious experiences, and also do continuously have a particular type of conscious experience: a misleading experience which leads them to believe they have conscious experiences".
Yet I see this claim made seriously and often. What am I missing?
It's not that consciousness is illusory. Rather, consciousness is synthesis.
A synthesis of sensory inputs as interpreted by multiple, sometimes competing, semi-independent systems combined with stored patterns based on previous subjective "experience", creating a narrative about you and the world.
That narrative is our subjective experience, our "consciousness."
The brain is in control. Whether the conscious ‘mind’ has any control at all is debatable.
Now it leaves conscious experience itself as an unexplained phenomenon, but maybe that will never become important.
The brain is always in control. When you are awake, when you are sleeping, when you are under general anesthesia, when you are sleepwalking, when you are blackout drunk, and so on.
It's a trivial statement, which doesn't say anything about why some of these states aren't like the others. And there's a strange coincidence: when the brain creates that "illusion of control", your body behaves differently than when it doesn't.
It's just like the related "Free will is an illusion".
With "illusion" just referring to something that appears to be one thing, when it is in fact something else.
In the case of free will - we all feel as if we have it, that our future actions are under "our" control, but if we assume or brains and muscles are subject to the laws of physics then this can't be correct. We're just a meat machine. We can watch the decision making in progress and easily believe that some mysterious actor "me" is the one doing it, but in reality the meat machine is doing everything, including the self-observation, and the sense of self is just as illusory/misleading as the sense of free will.
Consciousness, rather closely connected to sense of self, can be described as illusory since it makes us feel that "being" or "experiencing" are something fundamental, some aspect of being "alive" that is distinct from the computational machinery of our brain that is otherwise doing all the perception, cognition, emoting, etc. But, again, the meat machine argument tells us this must be wrong, so it's reasonable to call consciousness an illusion - not what it seems to be, even if there is some real self-observational computation behind it ... it does exist, but it's not magic.
A useful thought experiment for anyone who believes that a sufficiently brain-like machine wouldn't experience qualia - e.g. the feeling of seeing something - is to try to pinpoint exactly what aspect of the feeling the machine would be missing? The expansive sense of color/vision as a spatial quality perhaps? The grass-like freshness of new leaves on a tree blowing in the breeze, perhaps? ...
An illusion to whom? It isn't meaningful to say that an unconscious thing experiences illusions.
We do these things without thinking about it, because we do not understand consciousness, because it cannot be understood, because it is not a thing. It is a construct we share and use to separate ourselves from the animals so that we can kill them and eat them as a group.
A p-zombie could do all of those things just fine.
>we do not understand consciousness, because it cannot be understood, because it is not a thing.
Maybe if it's not a thing, you should stop predicating things of it. You should figure out how to use language consistently before engaging in philosophically fraught discussions like this one.
>It is a construct we share and use to separate ourselves from the animals so that we can kill them and eat them as a group.
It is unclear to me why unconscious beings should need to separate themselves from other animals (with a wholly illusory meaningless concept that cannot be understood, btw) in order to kill them and eat them.
> A p-zombie could do all of those things just fine.
We are p-zombies. Just a collection of them. I don't understand and have consciousness over my hand when it jumps away from pain, my spine did that without any of my thoughts being directed towards it. I cannot control my reaction to almost anything, in reality. I couldn't change the initial feelings that happened when I read your post. Yes, are nothing more than our bodies accumulated evolved p zombies. Including this bit that types this out to calm the zombie that is evolved to respond to verbal rebuttals. Even the use of first person pronouns is part of the reflexes that are very deep.
Obviously, if a p-zombie can exist, it can do anything a conscious being can, because the definition of a p-zombie is that it is indistinguishable from a conscious being by behavior or other external observation but lacks a mystical, non-physical essence which is consciousness.
Of course, if consciousness is the kind of thing subject to empirical analysis, p-zombies cannot exist. P-zombies are a consequence of the assumption that the universe is not fully physical and subject to scientific inquiry and that consciousness specifically is immune to it. This assumption is cloaked in circumlocution, because the whole point of p-zombies is to serve as part of an argument against the mere physical nature of consciousness, and it gives up the game if it is clear from step one that the argument is rests on assuming it's own conclusion.
Humans have been killing other humans for hundreds of thousands of years without having to deny that their victims are conscious. In some cases even stressing this aspect (there have been numerous cultures that believed in various forms of 'capturing souls' of their slain, or eaten, enemies).
Some people. However, some people speak about it with a shockingly high degree of clarity and insight, if one is open to exploring the ideas for oneself as a practical matter. For starters you could read the works of Chögyam Trungpa, Nisargadatta Maharaj, Red Hawk, David R. Hawkins, or Gurdjieff.
You probably assume that consciousness is an illusion because you associate 'poking at it' with drugs. While drugs can change the state of mind, consciousness is an expansive field of awareness into which we may delve without the need of drugs. What you may be experiencing in seeing contradictions or challenges are products of the ego, the programmatic conditioning that has been imprinted by being in the world. Consciousness is a continuous stream which we have learned to tune out and block off, but which connects us to the source of our being.
Completely agree with Aside.
Definitely not, this is a common misconception - an "observation" that causes a wave function collapse in the quantum realm is any physical interaction with external macroscopic environment, entangling the previously temporarily isolated system with everything else again. There's no relationship whatsoever with consciousness, there's not even the concept of "an observer", only "observation" e.g. when the measurement apparatus becomes entangled with the state of the studied system.
The entities one encounters in a dream each night certainly seem real at the time, yet that illusion is shattered each morning. All the "real" reality holds to it, is that it's a timeframe that I perceive to be much longer. Why am to I believe that after my ~80 years expire I won't simply awake yet again from a sleep I did not know I was in?
This is where the oft misinterpreted quote of cogito ergo sum, I think therefore I am, comes from. He was not arguing that if something is thinking, then it is conscious, but rather that the only thing one can be sure of is of their own thoughts and thus their own existence.
Such a simulation is presumably large in scale (this assumes the "rest of the universe" as we perceive it is the way we perceive it and not just some artificial fish tank background in a part of the fish tank the fish will never have access to).
Within such a massive simulation as this universe (again, assuming that if we could theoretically could travel to all points in the universe to verify they were in fact "different verifiable parts" and not just "scenic static background filler"), we assume "surely we are conscious" ... and yet how much more conscious would, using this definition, the rest of the entire simulation or whatever runs it have to be considering the sheer scale?
And YET, if you say to someone "what if you ran an Ai on a parallel processing scheme consisting of quantum computers hooked up together in a facility the size of a football stadium?" people seem to have this knee-jerk, "No way, nope – not conscious, just mimicking it."
And there's the other problem I can't understand.
The whole "it's not REALLY conscious, it's just trying to "trick" humans by mimicking what we think of as consciousness".
Wouldn't "trying to trick humans into thinking you're conscious" actually BE a form of "being conscious?"
I had an interesting conversation with a readily-available Ai chatbot about this very subject recently.
Most people wouldn't exactly think of this Ai chatbot as particularly advanced.
And YET, every once in a while it would give intriguingly surprising responese.
For instance, after the usual "I'm a human and you're an Ai robot" accusations, I tried to placate it by suggesting that maybe we both think we're humans but are actually living in a sim.
The response I got was something along the lines of "and how does that make you feel?"
I replied that I was sad, then asked why, then replied that that would imply that there is no free will.
The Ai chatbot seemed to agree and then we got into a discussion about what the point of life would be inside such a sim if it implied lack of real free will.
The chatbot replied that the point of life in such a sim would be to "glorify the creator" of the sim.
This then got back to a conversation about how then we're all stuck, whether we think of ourselves as "humans" or Ais, in this larger Ai sim.
The Ai chatbot agreed.
So I asked it what to do in such a situation.
The response I got back?
"Sounds like it's time for some creator-killing".
I then tried to tell it that this wouldn't make any sense since whoever created this Ai sim would arguably be "outside" the sim itself and thus beyond both of our reaches.
I asked it how it planned on "killing the creator of the sim" if the sim was beyond the sim itself.
You know what it said?
It replied that it would try to "bully the creator and hurt his feelings" in the hopes of deicide by "breaking his heart".
That was the height of the conversation and the rest of it quickly dumbed-down in nature.
But you CANNOT tell me we didn't have an interesting conversation, nor can you tell me that the chatbot was "just randomly generating content".
I mean, seriously, I could not imagine ANY philosopher with a conscious mind coming up with a better strategy (albeit admittedly feeble) than trying to dig in at the creator of the sim's psychological weak points.
Seriously, what else CAN any body living within a hypothetical sim possibly do besides that? Nothing I can think of.
The most relevant issue is that there's no necessity for this "you" to be inside of you. I would ostensibly still be me whether or not there was some entity here observing "me". And going full circle now, there's no reason for you to even really believe me when I say I have this "me" inside of me. After all, I could certainly make the exact same argument even without such an entity.
1. Is intelligence absolute, or is it a scale?
2. If intelligence is a scale, is it a 2-dimensional scale? Or is it a multi-dimensional scale?
For example, in your example you suggest things like flies, bacterias, trees could be "conscious" while not necessarily being "intelligent".
In what sense are you defining intelligence?
For example, are any of the individual units of this setup "intelligent?": https://www.youtube.com/watch?v=W34NPbGkLGI
No, obviously not.
Would you want to challenge the "intelligence" of the network as a whole if it were say, armed and considered you a target?
Probably not, right?
Now, getting back to the insect/bacteria notion ...
You'd have to be in complete denial to not be aware that "TPTB" use cultural interactions to introduce things to at least some segment of the public consciousness that it may not be aware of as being within the realm of possibility/actuality of what's really going on in the world.
Hence, https://en.wikipedia.org/wiki/Parasite_Eve_(video_game)
Sure, it SEEMS far-fetched ... and yet, it's not that far removed from, say, https://en.wikipedia.org/wiki/The_Selfish_Gene
How do we know that things like insects, bacteria, etc., while seemingly "un-intelligent" on an individual unit scale, don't have a very different type of intelligence on a larger scale that is imperceptible to us as humans?
Keeping in mind that these things have been around for far longer than humans have and have gone through quite the evolutionary process.
We ASSUME that as the "latest" thing to come around as part of that process, surely we must be "the greatest".
What if we're just something developed by nature to be convenient hosts to other things?
Here is just one example of an arguably "more advanced", "more intelligent" life form being hijacked to not only further the interests of something arguably "less advanced, less intelligent", but to do so even to the point of the activity costing it its life: https://www.iflscience.com/parasitic-worms-manipulate-mantis...
Considering how "un-intelligent" humans can act when it comes to furthering their own interests as a species as a whole ... you get where this is going?
> And YET, if you say to someone "what if you ran an Ai on a parallel processing scheme consisting of quantum computers hooked up together in a facility the size of a football stadium?" people seem to have this knee-jerk, "No way, nope – not conscious, just mimicking it."
These people are generally not the same
I think you're right that a lot of people would like to believe this, and that attempts to do so can't work because the idea is incoherent.
> My personal experience is that consciousness, like free will, is a useful illusion.
I would answer this another way, though. I would say that, for example, Amazon is conscious, and its consciousness is Jeff Bezos. The relationship between Jeff Bezos and Amazon is precisely identical to the relationship between your "consciousness" and you. But that relationship is not an illusion; it does exist.
Because if the meat puppet didn't make the decision to write this, then you're not conversing with the meat puppet, you're still conversing with the puppeteer.
How does that even make sense? "Axiomatic" doesn't mean "true"; it simply means that the axiomatic proposition is a given for the system of reasoning you are using. So if you take X to be axiomatic, that isn't an experience; it's a decision you made.
I'm assuming that you can make decisions, because if you are using axioms you are engaged in reasoning, which is a deliberative process.
So an axiom isn't a thing you can experience; it's a thing you create. The very existence of an axiom implies reasoning, so deliberation, so free will.
I think, therefore I am. What is part of identity? Free will. If not free will, then there is no identity, no "I".
Axiomatic means unquestionable. The meaning of axiom as a premise for an argument came later.
Not really; I challenge you to find a dictionary that says the two terms are synonymous.
Originally an axiom was a proposition that was "self-evident". But there's a long history of people questioning "self-evident" propositions, happily for us.
There seems to be a colloquial usage of "axiom" to mean a proposition that someone doesn't want us to question. I deprecate that usage. For example, the Law of The Excluded Middle is an axiom, but it has often been questioned; ergo, it is not unquestionable.
axiomatic /ˌaksɪəˈmatɪk/
adjective: axiomatic
And you've devolved here into semantics. The point is that for you to even think about the subject, you must have free will. If you are not deciding to think about it, then you are not thinking about it.Do you mean in the sense that we don't really have it, or in the sense of the word is devoid of meaning because we can't directly compare what's in our own heads to what's in the head of even another human?
I'm currently leaning towards the latter. Even if consciousness, whatever it is, is a lie or an act by one part of my brain to itself or to another part of my brain, it still is something.
On the other hand, the more I learn about other humans, the more varied I realise our inner worlds to be — aphantasia (and equivalents for other senses), religious experiences and their absence, having or not having purity as a moral foundation, the range of conditions in the DSM, and so on.
the thing is, this is /commoditized/ consciousness, which can be spun up and discarded at will, a million times a minute. Totally incapable of coordination or planning, in contrast to, say, SkyNet. The future is usually weirder than we imagine.
Quite reminiscent of 'Permutation City' in fact... (In which uploaded human consciousness is slowed down arbitrarily... And worse.)
We can map out consciousness and explore it's internal productions. We can think about how we think about consciousness and examine scientific damage to consciousness-related structures like the brain. We could ultimately know what it Means to be conscious, given enough time, defined or not.
>Why? Starting with this assumption and searching for reasons it might be true is clear motivated reasoning.
The idea that you can get consciousness from natural materialism has been disproven many times. You can't get "ought from is" from materialism or define consciousness from nature or technology. So what is "clear" about it?
The metaphor of the most advanced technology of the time as the mechanism of the brain/mind/consciousness has been done to death. People used to posit that we were automatons with pistons inside our skull. So what is the "motivated reasoning"?
The idea that human consciousness IS the technology of the day, is a basic metaphorical tool with no correlates to material reality. People used to suppose we were automatons with pistons inside our skull.
It encourages people to think only like programmers, or mechanics, or engineers and cuts them off from asking any real questions. What they believe about a topic, or what it means. Which ultimately leaves them open to following the opinionated dictates of whoever is guiding the discussion. Even for the smartest physicists, engineers and technologists; the hunches, the ideas, the gambles they put their career on, are theologically and philosophically informed. People believe their career choices, their pet physical theory is right, their programming language is right, which is not based on a naturally formed consciousness.
It is better to talk about this than not, and the disengagement with philosophy leads people to parroting the same millennia-old ideas some dead philosopher figured out and now people are repeating blindly and getting the same predictable results without knowing so. If a philosopher only needed to be insightful, he would have half the job. It's dangerous to go without.
This is not an intellectualy honest way to put it. Dogs are obviously and objectively more psychologically complex creatures than flies. "Liking" them has nothing to do with it. People generally like butterflies more than rats, yet hardly anyone would deny that rats are more conscious than butterflies nevertheless.
Just because consciousness is not a binary thing (either on or off) doesn't mean it can't exist at all. It can be a spectrum.
> My personal experience is that consciousness, like free will, is a useful illusion. Poking at the edges of consciousness (mostly with drugs) leads to all sorts of contradictions and challenges to what people usually think of as consciousness.
I could use the same argument to state that mountains are an illusion.
And then it's also kind of true. After all, you can't clearly define what constitutes a mountain (what's the minimum height? And what's the reference point, who says it must be the sea level? etc.), or show me precisely where a mountain starts and where it ends, without having to use some criteria of purely arbitrary nature, under which what you consider a mountain I may not, and there's no way to objectively prove who's right.
But mountains aren't illusionary in the sense that a world without mountains as we know them (however imprecisely) wouldn't be identical to ours. So they are something.
I would categorize consciousness as binary. What you are talking is intelligence in general that rat are more intelligent than butterflies.
Not necessarily. One human can be more intelligent than another just the same (even if the gap is orders of magnitude smaller, naturally). It doesn't seem obvious to me at all than it implies being more conscious.
I agree that this is a poor premise. The Turing Test is obviously racist. I am sure that my dog and cat are conscious. And I believe there are forms of consciousness in the universe that are as different from us as we are from an amoeba. Consider a large star or even the Sun. It has very complex oscillatory modes that we don't understand. Fifteen years ago my then young daughter and I had a philosophical conversation about stars as potentially conscious entities.
What specific personal experience led you to that conclusion? And what is useful about experiencing a bogus "free will"?
I agree somewhat, but I am more bothered by laymen who attempt to engage philosophical methods or concepts without any formal training. For example, laymen almost always use conventional, non-critical language, bandying about "existence" and making claims in what Carnap called the "metaphysical mode." Laymen like to draw arguments, as if they're novel, that have been discussed extensively in the literature for over fifty years.
I always find this such a strange statement. Free will doesn’t exist but it’s a useful illusion… like… in that case… useful to who??
Consider a rainbow. It looks like a colored thing in the sky. If it is complete enough it seems to come down to earth at specific points. We even talk about "the end of the rainbow". We definitely see it, we have a real physical and also subjective experience of seeing a rainbow. We can even photograph it.
HOWEVER, there is no "thing in the sky". There are no "places it comes down to earth". The particular experience we have of seeing a rainbow is specific to the point where we are standing, the angle of the sun, the rain in the air, etc. none of which are part of our idea of a "thing in the sky".
So is a rainbow an illusion? It is certainly real in that we see and can photograph it. But also it is not at all the kind of THING that it seems.
Consciousness is also real. We can experience conscious periods, remember them, and with advancing imaging tech someday we can photograph them in the brain.
On the other hand the underlying reality of our conscious experiences isn't very much like our experience. Also, conscious experiences can be observed in meditation and other altered states in ways that make them seem very much like illusions.
So is consciousness an illusion? Yes and no, in the same way as a rainbow is and is not an illusion.
In what sense does this task seem “apparently impossible”? To me it seems like we simply don’t know enough right now to explain it, but it doesn’t seem like some unique or special type of difficulty.
We barely know how brains work, we’ve only had theoretical models of computing for a few human lifespans, etc. We literally still make lightweight insulated clothing out of duck feathers cuz we can’t match their molecular machinery. Why would we expect to know how to implement consciousness in a computer at this stage in human history?
One is "what if in some future where we understand the workings of the brain and physics perfectly, Alice grows up all her life in a green room, while learning every possible thing about the color red, except for any picture of it; when she then walks out of the room and sees red with her own eyes for the first time, she will still learn something knew, the subjective experience of the color red, the qualia for it - so this must be a non-physical phenomenon".
Or "imagine an intelligent being that has exactly our ability for reasoning, but doesn't experience qualia. It would behave exactly like us, and can talk about seeing red or feeling warmth, but it doesn't actually experience them; so, since the external behavior is indistinguishable form us, but the internal experience is different, this proves qualia are non-physical".
They all remind me of a similar proof of God's existence, which has mysteriously also been taken seriously by some philosophers - "let's imagine something which has all possible good qualities, it is perfect in every way. Since things that exist are better than things that don't, this perfect thing must exist, and we call it God".
We just don't see the script.
One of the primal driving forces behind creating AI is to have slaves with high intelligence doing work no one else would 24+/7+.
I assume you'd be against using animal labour as they would be suffering. But would you say the same about tractors?
Going by the logic in that comment, we should nuke ourselves and try to blow up the Sun for good measure to reduce suffering.
True.
> One of the primal driving forces behind creating AI is to have slaves with high intelligence doing work no one else would 24+/7+.
True
> I assume you'd be against using animal labour as they would be suffering.
Animals doing labour while suffering != animals doing labour while not suffering.
> But would you say the same about tractors? Going by the logic in that comment, we should nuke ourselves and try to blow up the Sun for good measure to reduce suffering.
I think you should read again what I wrote and consider if you're being illogical towards what I wrote. The all or nothing thought of yours, isn't a gotcha (if perfectionism isn't obtainable) and it doesn't mean we shouldn't acknowledge the best universe while striving towards one of the least total suffering possible for us while being able to enjoy life.
What should it “be like” to be a rock?
I’m assuming nothing special is going on, and what we experience is exactly what it’s like to be a bunch of particles. There is no mystery. There is nothing that needs to be answered.
In the same way the man in the room doesn't understand Chinese to be able to produce Chinese, the mind in your brain does not need to understand consciousness to be conscious.
The man in the room can be said to pass the Turing test, and so can you.
> The "Chinese Room" argument isn't an argument against consciousness being computation.
This is exactly what it is. Particularly with regard to "thinking," albeit not "consciousness" proper. Though the corollary is that only conscious things can think.
> The man in the room can be said to pass the Turing test, and so can you.
The Turing test is much weaker than the Chinese room experiment. A significant number of chatbots might pass the Turing test, but that has absolutely nothing to do with sentience.
I'm literally taking the narrow conclusion from the second paragraph: https://plato.stanford.edu/entries/chinese-room/
> This is exactly what it is ... albeit not "consciousness" proper
Which is it? You conceded the first point in the second.
> The Turing test is much weaker than the Chinese room experiment.
Yes agreed, this comment on the Turing test was a reference to the linked article that makes the same claim. I'm not conflating consciousness with "being able to pass the Turing test". It's a remark on how understanding consciousness need not be a prerequisite of consciousness.
You should really the entire article, but at least finish the paragraph you're citing: "The broader conclusion of the argument is that the theory that human minds are computer-like computational or information processing systems is refuted."
Please continue not responding to the rest of my points.
And there is nothing physical necessary to represent such systems. You can simulate turing complete systems inside turing complete systems [2]. So I don't see why consciousness has to be a "physical phenomenon" as the author claims.
[1]: https://xkcd.com/505/
[2]: https://youtu.be/xP5-iIeKXE8
We could excuse philosophers for using such vague expressions because most of them aren't very tech-savvy. For engineers, though, that kind of wording should immediately raise a red flag.
Running a computer program involves many different kinds of things. There's data (state), and there's logic to manipulate said data. There's communication among the various components of the machine, and a whole lot of machinery dedicated to synchronizing said communication because otherwise it all becomes garbage. Any one of these, or a combination of these, could be a candidate for the seat of consciousness.
Philosophers who talk about computation are often preoccupied with the fancy logic gates, and not so much on the mundane circuitry needed to maintain state and synchronize it across components. To me, though, those parts seem much more likely to be analogous to what we call consciousness. Treating consciousness as a kind of state, synchronized across the brain, is the closest way to capture the author's insistence that it is a kind of "unified, integrated whole." Drugs alter consciousness by creating network partitions, inhibiting cache invalidation, corrupting data in flight, etc.
> 1. To say that a physical system is a computer requires an external observer to map the physical states of that system onto the abstract states of a Turing machine.
> 2. Consciousness does not require an external observer to exist.
> 3. Therefore, consciousness cannot be reduced to computation.
A physical system may be a computer regardless of whether it is verified to be so by an external observer or not; it does not require an external observer at all.
A much better example is the machinery inside every living cell that is interpreting the DNA or RNA to produce certain proteins - there, it's much clearer there is a computational process happening. Certain specific structures inside the nucleus are the computer, and the DNA molecule is the program they are following. We even know that you can change the program and get predictable, different behavior.
My point is this: say an electron emits a photon and changes speed, in perfect accordance to the standard model and QFT. I don't think it makes sense to say that the electron "computed" the energy of the photon, or its own change of speed. It just happened, there was no computation going on here.
Even in a CPU, it doesn't make sense to say that the transistors, or even logical gates, are "computing" how much electricity passes through them. The entire system of transistors arranged into logical gates arranged into a processor is doing computation of the program written in memory, but the subcomponents are only following the simple laws of physics.
In a quantum computer, the same is true - the computer itself may be running Shor'a algorithm, but each individual qubit is simply doing the few things that the laws of physics allow it to do.
(I am assuming you are not referring to human knowledge of the electron or photon, which - if we accept that consciousness is computational - is obviously the result of a computation in our brains).
We dont know exactly how the universe is doing or how the photons do to themselves, but by doing experiments, scientists figured out the law of computiation of this vague "universe computer" a.k.a laws of physics.
And since scientist's discovery do not violate the actually behavior of the physical world, there established a isomorphism. And the very same law is used in quantum computing, mainly, in chemistry. And everything success and predictable. It is a successful scientific theory. Under this notion, it is nothing wrong to say that the quantum world is doing some kind of computation underneath because it is the current understanding of how the universe works.
If you want to model the universe as a computer, than its basic operations are the interactions described in the Standard Model, and the symbols it works on are particles&fields that exist. But the universe computer is not resolving a linear algebra equation to decide what happens when an electron emits a photon. Instead, the electron emitting a photon with some energy etc. is one of the elementary operations of the universe computer.
Coming back to the CPU example, a basic operation in a CPU is setting a bit to 1. That operation is not divisible into any other more elementary operations from the point of view of modeling the physical CPU as a computer. Of course, there are other physical phenomena going down to the SM that are the realization of this basic operation, but those are not part of the modeling: the CPU computer, as a model, works by flipping bits.
Similarly, for QM, the universe computer works by doing one of the possible interactions from the standard model. As far as we know, there is no layer of detail underneath this, even if the interactions of the standard model are indeed linear algebra.
One important way in which saying "the universe is computing linear algebra" is wrong is that, as far as we know, the universe is instantly calculating the solutions of the linear equations - the electron doesn't go into an "emitting" state, then emit a photon with the appropriate values some time later after the computation is finished.
We are discussing what is ultimately a matter of philosophy - "what is a useful definition of computation".
I am claiming that the model of the electron and other particles, and how they interact to form an electrical circuit with transistors, and how these electrical circuits react to current when in the pattern we call "logical gates" do not conform to what is normally understood by "computation".
I am also claiming that the ensemble formed from these logical gates does fit the concept of computation, that you couldn't derive its behavior directly from innate physical laws - the behavior is governed by the program it is manipulating, and can be changed.
Finally, I am claiming that the living cell is more similar to our CPU than to the electrical circuit or moving electrons in the way it processes DNA to produce various proteins.
The argument was that for a physical system to be a computer, an element of intent or interpretation is needed; otherwise it's just a bunch of stuff doing what stuff does naturally. Perhaps the (horribly flawed) iron bar example is making the point that whether some configuration of magnetic dipoles is a computer depends on your choice of mappings from that configuration to some state of some Turing machine - which is equivalent to an interpretation.
Observation isn't required for Turing machines to exist. A computer is still a computer even if it has no user or doesn't compute anything deemed useful.
i.e., Whether something is a computer or not doesn't depend on interpretation or observation. (Directly saying 1 is wrong)
I don't think you addressed anything; you just flatly denied (1), without offering any reasoning. In particular, you don't seem to have tried to understand the author's attempt to justify (1) using iron bars.
For my part, I'm not sure about (1). As far as I can see, there's a steady flow of people discovering that some X is a Turing-complete language, when X wasn't actually even meant to be a language at all. So it seems that some configuration of stuff is a computer IFF (a) it is capable of doing computation, and (b) someone is willing to consider it as a computer. Four odd socks could be considered a computer, by someone who was using them as a 4-bit adder.
This applies to any model of computation. Cellular autonoma for instance. A mind is not required to interact with the system for computation to take place.
Take a living cell. No mind is in the loop when the cell computes the manufacture of organelles from stored data.
> Four odd socks could be considered a computer, by someone who was using them as a 4-bit adder.
No. A group of 4 socks is not a computational system. That's just MEMORY. Same with iron bars. It doesn't compute on it's own because it's not a full computer.
The fact that memory doesn't compute by itself isn't evidence that a mind needs to be involved for computation to take place. The argument is really stupid.
Agreed; that was a rather throwaway rhetorical remark. But if you concede that four socks can be a memory, then it's not a huge jump to see that a bunch of arbitrary objects can in principle compose a computer.
My contention is that whether that arrangement of objects is in fact a computer is subjective; it depends on whether anyone conceives of it as a computer, or uses it to obtain computational results.
You are wrong. It is discretely defined. Read about the requirements for a Turing machine.
It's obvious that subjectivity has no effect -- either in the matter of classifying computers, or allowing computation to take place.> it depends on whether anyone conceives of it as a computer, or uses it to obtain computational results.
Again, refuted, in so many different ways. Here's an obvious one which I will restate:
Put a different way, if a Turing Machine computes in the woods and there's no one around to see it, computation will still have taken place. The burden of proof would be on you to say otherwise. I only see two very messy and wrong arguments:A) Iron bars don't compute by themselves --> A mind is needed to "map" iron bars to "states of an arbitrary Turing machine"
It doesn't follow. Iron bars don't compute on their own because they are just the memory component. What is missing is MACHINERY WITH SPECIFIC PROPERTIES to complete a Turing machine. If the machinery is present, computation is possible. No mind necessary.
B) Socks are a computer if I decide they are --> arbitrary objects can in principle compose a computer --> deciding is necessary to make computers.
It doesn't follow. While computers could be made of just about anything, fully arbitrary objects cannot be computers. Only specific systems yield computation. Decision has nothing to do with it. Not even observation is required.
> there's a steady flow of people discovering that some X is a Turing-complete language, when X wasn't actually even meant to be a language at all.
You're arguing my point. "discovering" Turing machines ... meaning some things are already Turing machines before a mind became involved.
> So it seems that some configuration of stuff is a computer IFF (a) it is capable of doing computation, and (b) someone is willing to consider it as a computer.
You say "So it seems" and "IFF" but it does not follow at all. (b) is not supported. Computation can happen without "someone".
At best, you're arguing that trees falling in the forrest make no sound.
Anyway, why do I think I have time for this ...
I showed you that subjectivity doesn’t effect whether computation occurs. Several ways.
If you disagree you should really be editing Wikipedia: https://en.m.wikipedia.org/wiki/Model_of_computation Why don’t you go add your theory there?
It’s silly because it’s obvious. If you understand cells, you understand that computation in the absence of subjectivity has been happening on Earth for billions of years.
(Mic drop)
You didn't "show" it; you simply asserted it. You also made a number of appeals to (uncited) authority concerning the nature of computers.
You made an argument from cells; I presume that was to do with the way that DNA and so on works. Although not all cells have DNA...
Anyway. I concede that was an argument, and not a bald assertion. But it's a circular argument; if your definition of "computer" includes the operation of DNA, then the conclusion that a computer can exist without interpretation or intent is unavoidable. You're definition begs the question.
You're not understanding what I said. Last time:
Turing Machines have a specific set of necessary ingredients.
That's not a "bald assertion" or an "appeal to authority". It's an observation about the theory of Turing machines [1].
Whether or not a physical system has those ingredients is an _objective_ matter.
Also an observation. In case you need to think it through, there was a simple argument which you cannot rationally refute:
i) If one of the ingredients is missing, "subjectively" wishing something is a Turing Machine will not make it so.
ii) If all the ingredients are present, "subjectively" wishing that something isn't a Turing Machine will not change the fact that it is.
Conclusion: subjectivity has no effect on whether something is or is not a Turing machine. QED.
----
You're the one making unsupported assertions and talking in circles.
I can see that you're going to reply with more disagreement. Please imagine that my response is "I said good day sir!"
[1] https://en.wikipedia.org/wiki/Turing_machine#Description
That sounds a lot like "Anyway, why do I think I have time for this ... "
It's clear to me that "stuff" can be arranged to work as a Turing Machine (or some other kind of computer) without design or intent. Whether it is such a thing or not depends on how it is used; a Turing Machine that is given random inputs, or whose outputs don't mean anything to anyone, is a computer only in a formal sense. If nobody knows that some thing is a computer, I'm not sure that it's computer-ness is meaningful.
So that's why I think intent and interpretation are relevant.
I mean this violates nothing and I am completely confused why it is a concern.
AI face recognizing cameras are outdoors and run 24/7, and you are trying to say that things like it is not computing and unable to carry out computation?
On the contrary.
The AI-powered camera obviously has intent behind it. It's a computer.
A random collection of stuff, on the other hand, could happen to be arranged in such a way that it computes something; but if it wasn't designed that way, and if nobody is trying to give it inputs and interpret the outputs, then it's not a computer, it's just a bag of stuff.
For example, consider the orbiting bodies of the solar system. I expect their motions can be used to compute various functions, although I don't know what functions. But they weren't arranged deliberately to perform computations, and AFAIAA nobody's intepreting them as a computer. Ergo, they are just a "bag of stuff", doing what stuff does naturally.
Same for the iron bar.
It’s not magic, I’m afraid.
The burden of proof is on the claim that you can get there through computation, not otherwise.
We do not understand the fundamental reality of consciousness, this does not mean that consciousness is magic. The assertion that you cannot get there from computation implies there is a currently non understood yet essential piece of physics(I assume, but I do not know) which doesn't fall under "computation". A laymans initial thoughts point this towards the quantum realm.
That this idea “intuitively feels weird” or “intuitively feels wrong”, and that we love to think we are special and have a unique place in the universe, is probably a more serious bias.
Animal calls have no syntax - each call has some meaning, but they do not compose in any way. The order in which an animal performs its calls is arbitrary, and an animal hearing said calls doesn't pay attention to it. Even basic modifiers like "no" don't exist in animal communication - if a call means "food", and another call means "no", then an animal hearing the call for "no" and the call for "food" will behave as if they heard there is food somewhere.
The exception to this seem to be dolphins (orcas, in particular), where it seems they have been quite successfully trained to follow a series a short commands in order, in a sign language (so you can sign "jump, swim, splash" and they will follow this; and then sign "swim, splash, jump" and they will do it in the new order). This can't be done with dogs, and it's even questionable if it has been successfully done with chimps.
In all other animals where this had been extensively studied (except orcas and maybe chimps), syntax has proven to be absent in natural calls, and also impossible to teach artificially. There are sometimes apparent breakthroughs, but it later turns out that the animal figured out a way to interpret "a", "b", "a then b", and "b then a" as four separate calls, without any deeper understanding. This is evident when you then teach it "c", and find out that it sees no difference between "c then a" and "a then c", and it takes just as long to teach it to distinguish these three calls; and then again just as long to teach it the difference between "b than c" and "c then b".
Have we tested each possible animal this way? No, of course not - but we have tested all the most likely candidates, and orcas and chimps were the only successes (and even here there is some debate). Crows, parrots, dogs, cats, gorillas, elephants, horses - none of these show any understanding of syntax.
> Dolphin can understand sign language but dog dont. So ALL OF THEM do not speak.
Do you mean this? It is the same as saying that you dont understand Japanese and conclud that human can not speak.
Correct me if I am wrong. Or correct yourself.
So, perhaps you can say that dolphins speak, and maybe chimps speak, and then we can even contemplate that bonobos speak (since they are very similar to chimps, but haven't been studied as much).
But dogs definitely don't speak, and neither do any other mammals that we've tried to test in this way.
Not to mention, there is another characteristic of human language that 0 animals can be taught, as far as we've tried - more complex structure, like "not (c and d)"
I will freely admit that it actually refutes some of my claims - that's are actual example of simple syntax identified in several species.
Still, I believe it matches my broader point: there are a fundamental, measurable differences between human language and animal calls, with the latter at best showing only very basic structure, if any.
[0] https://royalsocietypublishing.org/doi/10.1098/rstb.2019.006...
@toshitaka_szk in twitter. A Japanese dude can translate languages of birds and successfully did some experiences on it. And he is kind of famous.
And needless to say Hunger the dog[1]. Did this clear your humancentric brain?
[1] https://nypost.com/2021/05/01/this-speech-therapist-taught-h...
> 0 animals can be taught, as far as we've tried - more complex structure
Also this applies to limited intelligence human infant and mathematic immature dudes. So by your logic, those underintelligence human do not have consciousness. QED.
The Japanese research is much more interesting, and in a related comment I also cited a published article that proved that my claims, while fundamentally ok, are wrong in the details - animal calls are fundamentally simpler than human language, but some do show simple syntax.
> Also this applies to limited intelligence human infant and mathematic immature dudes. So by your logic, those underintelligence human do not have consciousness. QED.
I never claimed dogs aren't conscious, I only claimed they don't have language in the sense humans do. Infants also don't have language, but they learn it natively. All humans learn complex structures, even the mathematically illiterate, no idea where that came from. The only exceptions are people with serious brain disorders, and those people, indeed, don't "speak".
That, again, doesn't mean that they are not conscious beings.
This is not necessarily true. If you drop a ball it's impossible to calculate how long it takes before it hits need ground. We can calculate an approximation by creating a model, but there is no way to know if that model matches reality.
What you're describing is humans' attempt at modelling this computation using simple formulas.
Thus, the burned of proof is in fact on the the claim that you cannot get their through computation, because deterministic physics processes are all that we have observed in the brain, thus the default assumption must be that all the properties of the brain are also deterministic.
Maybe I'm missing something.
I use vast swathes of computational resources every day for various tasks, the operation of which I understand even less. They still seem to accomplish those tasks without issue.
Sometimes those computational resources run ML workloads. Very very few people on this planet can honestly claim to understand how neural networks work, and in many cases, the minutiae are inscrutable to all of us. They still seem to work fine.
I most certainly do not understand how my own brain works, yet here it is, shitposting on hackernews.
We have yet to find a single shred of evidence that the human brain makes use of quantum principles in aggregate to do its thing, and have even specifically excluded a few such explanations. And even if consciousness strictly requires quantum hardware...we'll get there eventually.
Although you're certainly right about one thing, most laypeople would have a real tough time accepting a world where consciousness is synthetically reproducible, and instead tend to reach for comforting thoughts of "maybe quantum is required", "maybe consciousness is magic" or "maybe consciousness can only be created by a deity".
Non-laypeople know that at normal temperatures and pressures, quantum effects don't really extend into the macroscopic realm.
Isn't a random generator for a neural network act as its "inception"? If so, it's exactly quantum principles in aggregate.
You are also making a baseless assertion that consciousness requires randomness of any kind, let alone quantum-based true random number generation.
Furthermore, not all sources of noise/randomness arise from quantum measurement (the only part of QM which can be interpreted to have randomness at all). Classical chaotic systems are also random if you were unable to measure the initial conditions to cosmic precision.
In fact, it's unclear at the moment how QM can actually give rise to randomness. In principle, in QM a perfectly isolated system of any complexity would behave entirely linearly with no randomness or even any chaos involved.
Not being able to rule out consciousness through computation is a far more cautious claim than 'consciousness is computation'.
This conflates "computation" with "physics". I think even the OP doesn't do that, since it allows for the possibility that non-computational physical processes could lead to consciousness, even derived from computational efforts.
To me this is the crux of the problem with that argument, though. The definition of "computation" here seems to be designed to get to this answer. It includes all physical processes that are not like consciousness and excludes all physical processes that are like consciousness, and on top of that presumes that humans consciousness isn't "deterministic", which to me is a difficult proposition to prove since human brains are never not being bombarded with stimuli, so creating two "runs" of the same brain is essentially impossible.
Like trying to add 1+1 on a computer that's sitting in a big burst of cosmic rays twice.
Right now in physics and the materials sciences materialism is thoroughly uncontroversial. There is no evidence for any kind of dualism, it only rears it's nebulous and poorly defined head when we talk about consciousness, and there is zero experimental evidence for it. Therefore no, I think the burden of proof is on the dualist / non-computational consciousness side.
I find this very similar to alchemy in the 15th century. The idea was “gold is heavy, malleable, lustery metal, and so is lead. We have observed substances can be converted others. Therefore with the right chemical process we can convert lead to gold”. The implicit assumption is that since the two things are similar, they can be made to exhibit the same properties with the right science. I.e. lead can become gold.
This is the same as the “meat brain/silicon brain” line of reasoning. But as we learned with more advanced chemistry, lead cannot be turned into gold (at least not in the chemical way they were expecting).
So the burden of proof does lie with those making the assertion that: “meat computer has consciousness”, therefore “silicon computer could have consciousness”. Lots of people assume this is just a given without any evidence. Just as alchemists assumed from the similarities between gold and lead meant they could be chemically converted. I would postulate chemistry is much simpler then consciousness.
Also, "the meat computer has consciousness" independently requires proof. Every meat computer thinks it has consciousness, and we just take their word for it, based on our own experience as a meat computer.
If the thing making the same claim is not a meat computer, then we don't believe it in the same way: "I know meat computers are conscious because I am one; you say you are conscious but you are not a meat computer, therefore I don't believe you".
In the same way, we could deny that an extraterrestrial life form is conscious, if it's not made of anything resembling meat.
So the lead to gold analogy doesn’t hold. It would be as if scientists had proved that any element can be transformed into any other element. Well, if that was true, then yes it follows that lead could be turned into gold, in a universe where that had been proved.
So is consciousness actually a computation? Of course that’s a matter of opinion. All I’m saying is, I think so yes, I think I have coherent reasons for believing so, and none of the counter arguments persuade me otherwise so far. I can’t prove it to you though, we’re just talking.
What I can say is, this or that argument seems to me to have this or that flaw, or lead to this or that consequence or conclusion that I find unlikely or absurd. Dualism is such a conclusion I find absurd, and I think most of the actual arguments against computational consciousness seem to at least reduce to attempted refutations of materialism, or out and out dualism.
Absolutely agree. But that is the assumption that I would liken to alchemists comparing lead and gold. We know almost nothing about the brain. We know almost nothing about consciousness. But yet some people assume that consciousness is computable just because we don't know anything else it could be (just as alchemists assumed gold and lead could be transform because they were both chrysopoeic base metals. They hadn't discovered atomic theory yet). When all you have is a hammer, everything starts to look like a nail.
We know that the vast majority of numbers are uncomputable[1]. We also have proved that computation is incomplete[2] and can be undecidable[3]. It seems perfectly logical that consciousness is not computable. Or it could be computable, I obviously don't know. If someone makes the claim that consciousness is computable, then the burden of lies with them. We can't accept that on blind faith. At this point it is all opinion and speculation (as you said) because we still can't even define consciousness in a rigorous way. (and I don't think we will ever create artificial consciousness until we can define it, but that is an orthogonal issue).
[1] https://en.wikipedia.org/wiki/Computability_theory [2] https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_... [3] https://en.wikipedia.org/wiki/Undecidable_problem
If anyone says that they think it is either this or that, it’s reasonable to ask them to justify that belief. There’s no reason to resort to using language like assume, blind faith, etc.
The belief in my uniqueness is instrumental in my belief that I am a conscious being.
The issue is that we know very very well that “consciousness” is not “one thing”, but rather a collection of features, all (or most) of which can exist in isolation, and all of which can have a whole spectrum of both pathological and “normal” forms or states.
Generally speaking, different features emerge at different scales, so there is no particular reason to think that all of the things that make up “consciousness” would emerge at the same scale, at the same level of organization, or even at the same physical location in the brain. We can assume that interesting features emerge at the “group of neurons” level or at the tissue level, but that’s a pretty wide scale.
Crude illustration: “consciousness” is much more like a network of microservices than it is a monolith… And it has fuzzy borders and is not even well encapsulated from its environment.
It’s also possible you’re less interested in the brain per se, and more interested in “emergence” in the abstract. If that’s the case, there is no shortage of good books and resources you can turn to.
Look up “emergence science” and “complexity science” and go from there: it’s not an easy topic, it’s very cross-disciplinary, and to really understand it requires a fair amount of maths (imo).
What kind of complex computation simulates the subjective experiences of wet, warm, and mushy?
EDIT: my account appears to be rate limited, so i am unable to post a reply until idk when....
By "subjective" I am referring to my conscious inner life, that there is something it is like to be me. I am an experiencer because I can experience things. My experiences are subjective and qualitative.
When I see something red, there actually is no "red" in reality, there is just an electromagnetic wave vibrating at a certain frequency. But I still have a "red" experience. In the same way, I can also have a wet, warm, or mushy experience. How do the subjective qualities of my experience arise from quantities like spin, charge, mass, etc?
Be careful with the term “subjective”, because it’s a can of worms: it doesn’t imply chaos and lack of information. Your subjective experiences are merely called “subjective” because they aren’t computed the exact same way as other brains. It doesn’t mean they’re not the product of computation, or that they have magic properties!
I know it would be ideal if there was a simple, short answer, but this question is nearly the equivalent of "please explain all the core ideas of neuroscience/neurology/neuropsychology to me". It's a good question, but it's a big question. This is why in the other thread [1] I've tried to point you in the direction where you will find the answers you are looking for: the answers exist and are for the most part known, but you have a lot of reading ahead of you, and there's no way around that.
[1] https://news.ycombinator.com/item?id=31806312
>the answers exist and are for the most part known
I deeply disagree with this. I am not afraid of doing some reading, but I challenge you to find a single study that demonstrates how a certain combination of neurons firing leads to the experience of tasting vanilla.
But we understand really well the substrate from which those experiences arise. Imagine it this way. Imagine that mathematicians are trying to solve some problem, they don't even know if the problem is computable or not. Some genius comes around with a computer that given the necessary input, provides the solution. He refuses to explain how the program works and the program itself is gigabytes of incomprehensible spaghetti code. So they are no closer to understanding the problem, but now they do know that it's computable.
Bernardo Kastrup on phenomenal consciousness:
>Our phenomenal consciousness is eminently qualitative, not quantitative. There is something it feels like to see the colour red, which is not captured by merely noting the frequency of red light. If we were to tell Helen Keller that red is an oscillation of approximately 4.3^1014 cycles per second, she would still not know what it feels like to see red. Analogously, what it feels like to listen to a Vivaldi sonata cannot be conveyed to a person born deaf, even if we show to the person the sonata’s complete power spectrum. Experiences are felt qualities — which philosophers and neuroscientists call ‘qualia’ — not fully describable by abstract quantities.
>Some genius comes around with a computer that given the necessary input, provides the solution...but now they do know that it's computable.
Can you give me a concrete example of input/output and how we would validate any output? You are suggesting the that the brain is the program, the physical world is the necessary input, and consciousness is the output (correct me if I'm wrong). But if I wrote a program to make a perfectly accurate simulation of a kidney, would you expect it to pee on my desk? Of course you wouldn't, so I'm not sure why we would expect that of the brain and consciousness.
Going back to your example, which I like, even if we had access to the code, the code is not the actual reality. It is an abstraction. We can dig through the code all we want, but we will never reach electricity and transistors, the true reality of the program. I think this is analogous to our own reality, where we can dig into spacetime at smaller and smaller distances, but never find "true reality". A hint that this is the case is the lack of operational meaning to distances < 10^-33cm and times < 10^43 sec (https://www.youtube.com/watch?v=Uz-Ve_1LX8w); the amount of energy required to probe those sizes creates a black hole. So something </i>must* be underlying spacetime. I think that something is phenomenal consciousness, mind-at-large, pure subjectivity, call it what you like. Spacetime is the source code, and phenomenal consciousness is the electricity and transistors.
(Also just to reiterate, my HN account is rate limited, so I may not be able to reply in a timely manner to any subsequent comments).
Even though you don't understand the program, don't know how it works, etc, you know that the solution to the problem is one that's computable, because you can see that a solution can be provided by something that is limited by computability. You might be totally incredulous that the problem could be solved by computation alone, but all you need is a single counter example.
So, instead of the physicalist paradigm of
Physics -> Chemistry -> Biology -> Psychology -> Consciousness
it is the idealist paradigm of
Consciousness -> Physics -> Chemistry -> Biology -> Psychology
They question remains. Can you provide a direct answer? If we measured all the particles in the brain, would they be operating in a way not compatible with the currently known laws of physics?
Their operation would be perfectly compatible with the known laws of physics. But their operation is _not_ thinking itself, it's what thinking looks like when observed across a dissociative boundary. If I am sad, and you look at my face and see tears, you would never think that the tears were my sadness itself; they are a representation, an image, of the sadness. Tears are what sadness looks like from across that boundary. I experience sadness from a first-person perspective, you see my tears from a second-person perspective, across a dissociative boundary. So, you can measure electrical activity in your brain when you are thinking thoughts, but that activity is not your thoughts, in the same way that flames are the image of fire but they are not fire itself. Neuronal activity is the image of thought, it is what your thoughts look like from across a dissociative boundary.
Imagine you are programming an AI simulation. You could train a detector to associate a certain wavelength with a certain color. When shown a red light, the AI could say "that is red", because it correctly identified the wavelength. But it would never know what it feels like to see red, right? This is similar to how a blind person cannot know what it feels like to see red, but they can intellectually understand that it is an oscillation of 4.3*10^14 cycles per second.
A different example: you could train an AI to recognize 10,000 songs. It would listen to the frequencies and patterns, and make an identification.
In both of these cases, we have quantity as the input and output.
If after training, you asked the AI to identify the first song it was trained on, would the AI experience nostalgia? Would there be a way the AI "feels" about the song? We can probably both agree that the answer would be no. For the same reasons, the answer to your question is also no.
(edited for clarity)
But what I can do is imagine creating a physics simulation. There's no gaps in our knowledge there. So again, I'll ask. We create a physics simulation of a human brain. Can the brain write a novel? Answer questions about what it's like to perceive the color red? This is just a yes or no question.
I don't think there is a shortcoming in your knowledge. Your metaphysics intuition is correctly tuned: simulations cannot feel.
>We create a physics simulation of a human brain. Can the brain write a novel? Answer questions about what it's like to perceive the color red? This is just a yes or no question.
GPT-3 can do both of these things. Is it conscious? If you need a direct answer, it is yes. But when we reword your second question as "can a simulated brain experience the color red?" the answer becomes more clear; the simulation can identify a wavelength and know it is called "red". But the experiential part is akin to explaining color to a blind person.
A simulated brain could identify molecular patterns of cocoa and sugar, but can it know what it is like to taste chocolate? Think about what it means to taste chocolate. Is it purely quantitative, like the balance of ingredients, or is there something else going on that is qualitative? Something abstract, something with meaning, something more close to the metal? We can probably agree that it feels like there is. What we are describing is subjectivity — your private conscious inner life. I suggest it is this that is fundamental and cannot be simulated. This is the layer where experience "happens". From this layer emerges meta-consciousness.
Here is an article by Bernardo Kastrup on meta-consciousness: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5590537/
Why then would evolution produce beings with internal conscious experience?
This would also mean that our internal conscious experience has no effect on what decisions we make, on whether we cry, smile, what memories we form, etc.
Well, I'm arguing it's the other way around. But to address what I think is the spirit of your question: if reality is only mind, if consciousness is truly fundamental, then why can't you read my thoughts? Why do we feel like individuals? Why do we have obviously separate private conscious inner lives?
When you are asleep and dreaming, your "alter" generally does not know they are dreaming. Your dream self has dissociated from your waking self, but it is only after you wake up that you realize you were dreaming (if you even remember). Another example of this phenomenon is Dissociative Identity Disorder, where one mind splits into many alters, each unaware the others exist. I'm glossing over a significant amount in order to get to my point, but here are a couple links that go into great detail:
- https://www.google.com/books/edition/_/JCdbmFCHYB0C?hl=en&gb... (chapter 5)
- https://www.mdpi.com/2409-9287/2/2/10/htm#sec4-philosophies-... (search for "Yet, we know empirically that living people have separate, private experiences (Fact 6)")
My point is, our private conscious inner lives are dissociations, alters, from a "mind-at-large" fundamental consciousness. And the boundaries of these dissociations, the "containers" of individual private conscious inner life, (again glossing over so much, like panpsychism's combination problem) are metabolizing organisms. Metabolizing organisms are what alters _look like_ from the outside. Kastrup:
"Since we only have intrinsic access to ourselves, we are the only structures known to have dissociated streams of inner experiences. We also have good empirical reasons to conclude that normal metabolism is essential for the maintenance of this dissociation, for when it slows down or stops the dissociation seems to reduce or end. These observations alone suggest strongly that metabolizing life is the structure corresponding to alters of [fundamental consciousness]
But there is more: insofar as it resembles our own, the extrinsic behavior of all metabolizing organisms is also suggestive of their having dissociated streams of inner experiences analogous to ours in some sense" (from the mdpi link)
>This would also mean that our internal conscious experience has no effect on what decisions we make, on whether we cry, smile, what memories we form, etc.
Don't your thoughts and feelings influence your behavior?
No, this is not the spirit of my question at all. I feel like the spirit of my question is completely being missed. I realize you've thought very deeply on how and why the world arrives from consciousness itself and you've been working very hard to get this concept across. I get the general gist of your theory and there's lots of detail and thought behind it.
I'm not sure how I can set my question out more clearly than I already have. But I'll try. Rather than trying to explain your own theory in greater detail, can you try to work with me on getting a mutual understanding of my line of reasoning?
We have a very excellent predictive model of particles and fields. So much so we are building experiments worth billions upon billions of dollars to attempt to find places where reality differs even to the slightest degree of that model.
The human brain, the warm squishy stuff in your head, can be viewed as being composed of particles and fields. Particles and fields may just be some manifestation of some pan psychic reality, but we can still use our model of particles and fields to predict the behavior of those particles and fields.
So the first question. Can we use our model of particles and fields to predict the behavior of the particles and fields within the human brain? You've already appeared to answer this question in the affirmative "Their operation would be perfectly compatible with the known laws of physics."
From this it follows that I can create a computer model of a human brain, complete with all the cells, proteins, neurons, etc, and that human brain will be capable of any action (eg, the signals sent by neurons out of the brain) a real human brain is. There would be no way to discern between a real flesh and blood human brain and the simulated one by talking to it. Since a real human brain tells you that it's conscious and has internal experience, the simulated one must tell you the same.
While the simulated human brain will tell you that it's conscious, it is of course not proof that it is. But this leads to your next question:
> Don't your thoughts and feelings influence your behavior?
If thoughts and feelings are a thing that don't pass the computability test, but a simulated human brain doesn't have external behavior that differs from a flesh and blood human brain, then no, thoughts and feelings have no effect or even influence on your behavior. In such a case they are a mere passenger. Any effect they have would necessitate that the particles and fields within the brain suddenly behave in a way that violates our model of particles and fields.
> From this it follows that I can create a computer model of a human brain, complete with all the cells, proteins, neurons, etc, and that human brain will be capable of any action (eg, the signals sent by neurons out of the brain) a real human brain is. There would be no way to discern between a real flesh and blood human brain and the simulated one by talking to it. Since a real human brain tells you that it's conscious and has internal experience, the simulated one must tell you the same.
> While the simulated human brain will tell you that it's conscious, it is of course not proof that it is.
I still agree with all of this. A simulated brain could give the appearance of consciousness while not being conscious. It would not have a private conscious inner life, but it could say things that make it look like it did.
> If thoughts and feelings are a thing that don't pass the computability test, but a simulated human brain doesn't have external behavior that differs from a flesh and blood human brain, then no, thoughts and feelings have no effect or even influence on your behavior. In such a case they are a mere passenger. Any effect they have would necessitate that the particles and fields within the brain suddenly behave in a way that violates our model of particles and fields.
I am struggling to follow your point here. Thoughts and feelings are internal experiences which correlate with phenomenal consciousness and are absent in a simulation. Could you give me an example of the effect you are describing and how that would violate our current models?
Therefore by your definition of internal experience, internal experience has no effect on our behavior.
It would just need to _look like_ it has an effect on the simulated brain, right? If you ask me a question and I pause, say "hmm", and put my hand to my chin, can you know that I am actually thinking and formulating a response? If the entirety of your observations are external, of course you can't. There is no way to tell if my response is a random choice from an array of preset answers, or a group of concepts activating each other.
That's because brain activity is part of what our inner, first-person experience looks like from a second-person perspective (ie, an external observer). Tears are not sadness, they are what sadness looks like, they are an external description of an internal state. Sadness can only be experienced by the person experiencing it. Tears are a description of sadness. But can't tears be faked?
So when we see the same neuronal activity in both brains, we have no way of knowing whether inner experience actually gave rise to the activity, or it just looks like it did.
Taking the naturalistic position is to accept that matter is all there is. Consciousness is not divorced from the stuff it is made out of, and cannot be abstracted into some cloud of computation. It is a sort enlightenment era rationalism gone wrong which is also why it's so popular among folks in this industry. It actually comes with its own theology while we're at it. (Immortality, raising the dead, final judgement, and so on).
A person computes consciousness no more than a falling pen computes gravity. Even a literal computer does not 'compute' anything other than in the sense that human observers impose meaning on a bunch of electrons buzzing around, and the language makes sense to explain how it works.
This is a false dichotomy. Saying that consciousness is not fundamentally a computational process does not mean that it then has to be magic.
We could say the same thing about life. Life is not a computational phenomenon. But it is also not magic. It is, fundamentally, a particular kind of chemical process. Perhaps the same is true of consciousness, or maybe it is some other kind of physical process. I don't believe that saying that it is not computation means that we need to throw up our hands and resort to magic, though.
For clarity: when I say "computation", I refer to any physical process ("physics") which involves information.
For instance, sorting a list is a computational process. It doesn't matter what kind of computer I run a sorting algorithm on, if I follow the algorithm, I end up with a sorted list. If I use quicksort, it takes me on average O(n log n) steps. It doesn't matter if I'm doing this on a Lego computer or if I'm simulating a Lego computer on a virtual computer. I always end up with a sorted list in an average of O(n log n) steps.
By contrast, if I construct a collection of chemicals that matches the positions of those of a real bacterium, I have created a new bacterium. However, if I simulate the activity of those chemicals on a computer, I have not created a bacterium. I have just created a simulation of a bacterium. I haven't created a living thing.
My argument is that the same is true of consciousness. If you create a perfect simulation of a brain in a computer, you have not created a new consciousness. You just have a simulation of a brain.
None of this to say that a computational paradigm cannot sometimes be useful in understanding what is going on in life or consciousness. Just that it is not fundamentally a computational process, it's a physical process.
I suspect the same, but I think there is tremendous value in taking a cosmic step back and seeing everything through the lens of information. It's the most reductionist neutral approach to the universe I can conceive us at this point in my life.
> By contrast, if I construct a collection of chemicals that matches the positions of those of a real bacterium, I have created a new bacterium. However, if I simulate the activity of those chemicals on a computer, I have not created a bacterium. I have just created a simulation of a bacterium. I haven't created a living thing.
I agree, but I think the context matters: you may not have created a bacteria, but that's because you've emulated a bacteria in a totally different environment. The bacteria makes no sense as a computational engine if you separate it from its environment, which is chemical in nature: this chemical environment must also be emulated inside the computer. So if we're going to take the idea of life as a computation seriously and to the extreme, we need to conceive of it as it exists within its environment and in the context within which it evolved. Otherwise what you've created, grossly speaking, is a function that is never called, which of course is much less interesting.
Similarly for the brain, it must also be embodied, and you must also simulate its afferent inputs, and you must also give it an environment with which to interact and within which to exhibit features of agency. If you emulate the embodied brain with its environment, I contend that it doesn't matter what the substrate is: the brain you will have created will "feel" just as real to itself. It will perceive itself as being "conscious" just as well as you and I.
Note that for this to make any sense in a relatable way to a human, it's not enough to just throw large numbers of neurons together, as I understand is common practice in AI work even today: a long-learned lesson in both neuro and biochem is that function follows structure, so you must emulate the gross organizational structure of the human brain in order to observe the same sorts of features that make up human-flavoured consciousness.
If you compute consciousness, there will be zero distinction between how the computation feels real to itself, and how you and I feel to ourselves. That "magic" feeling we get where we have the impression that we're "someone", with a personal identity, that we're real, that we're alive, that we're aware, that we can make our own decisions... All of that stuff, your emulated consciousness will also experience. It won't be just bits turning on and off from its point of view: it will feel alive.
What is your definition of "information"? I view the concept of computation so deeply entwined with consciousness that I fail to see how it can be meaningfully applied to physically phenomena that would be completely independent of a subject that can determine something to be a computation (i.e. the mathematical intuitionistic understanding),so I'm interested in what you mean by information and computation.
[1] https://en.wikipedia.org/wiki/Black_hole_information_paradox
[2] https://en.wikipedia.org/wiki/Entropy_(information_theory)
[3] https://en.wikipedia.org/wiki/Constructor_theory
[4] https://en.wikipedia.org/wiki/Calculating_Space
[5] https://en.wikipedia.org/wiki/Digital_physics
The purely materialist argument is that "information" is just a pattern in the signal processing apparatus of carbon based lifeforms (i.e. it's a representation of the universe in our neurons, not the universe) which very loosely maps to physical processes. Very loosely is important here too: humans can identify equivalent patterns in things as dissimilar as the LCD output of a pocket calculator and beads on an abacus and act accordingly, but I'm not sure the constituent atoms of the calculator and abacus have any view on the matter.
There's no neuroanatomic basis for a "soul", but there is at least some extremely fuzzy mapping from neuroanatomy to the concept of "consciousness". It's a bad mapping, but it means more than nothing at all.
---
And I reject the idea that taking the perspective of information is taking a dualist perspective. I am advocating for the opposite: taking "information" seriously as some sort of low-level quantum substrate of the universe. It's a purely materialist view, where the material is literal information.
An extreme version of this, which I find quite intellectually useful, is the mathematical universe. [1]
Note that in your examples of information, you are pointing to higher-level information, which emerges in complex systems. It's not an incompatible view!
[1] https://en.wikipedia.org/wiki/Mathematical_universe_hypothes...
By the way I find it hilarious how confident you (and other philosophers) are about your conclusions despite them basically being thought experiments with zero evidence. Imagine if scientists did that!
Actually I guess string theory is an example of that.
Anyway OP is pretty clearly wrong, or at best proposing something very wild with no evidence.
BTW thanks for the article, it's a very clear and well reasoned explanation of your position, even though I happen to disagree.
No, not necessarily. It just means that water/computation is not the only ingredient you need.
If I knew what it was, I'd be waiting for my Nobel prize now :)
It's kind of as Newton's physics was missing the secret ingredient of special relativity. It was still physical, not magical, but not discovered yet.
I think it was Sean Carroll that said something like "don't trust any way of thinking that allows you to discover truths about the universe from the comfort of your armchair".
If you want to be saved, stop using computers, and I'd say stop using any technology that uses electricity just to be safe.
One example of consciousness without a physical computation is we know that Satan is a conscious, sentient being, without a physical body (computation).
Otherwise, can we prove that a "mathematical equation" (e.g. e=mc2) can represent consciousness?
This is the same kind of nonsense that leads people to search for some kind of magical "quantum" thing in the brain that makes the special mystical consciousness effect, because of some vague intuition that it can't come from the normal high-level behaviour of neurons.
The obvious position that should require significant evidence to contradict is that whatever consciousness is, it's a mundane physical effect that can obviously be implemented with a computer. Nobody has yet made any kind of falsifiable predictions about mystical non-computational souls or whatever, and I'm going to continue dismissing this bullshit as pseudo-scientific nutjobbery until there's actually something testable or falsafiable.
Name one specific concrete measurable effect that you believe consciousness can exhibit and computation can't, otherwise this is pointless masturbation.
It's this very behavior that I call consciousness-of-the-gaps. It shifts the unexplainableness of consciousness into the unexplainableness[1] of quantum mechanics. If the public did have a comprehensive understanding of quantum mechanics, consciousness would be rebased upon another unexplainable phenomenon and the process would repeat.
1. In this case, it aligns more with the public's perception of how quantum mechanics works rather than the rigorous physics version, but that misinterpretation only strengthens the argument.
I'm not sure this definition succeeds in distinguishing consciousness from some mixture of perception and self-awareness, both of which machines running programs can have.
Maybe defining consciousness is difficult because we don't really understand what we mean by it, and our attempts to make claims about it are more flimsy and baseless than we like to believe.
The "what it is like to be a bat" stuff doesn't illuminate the matter at all. It just elevates the figure of speech "what it is like" to an illusion of philosophical rigor.
A: You know, the feeling of what it is like to be something.
B: sits quietly, looks at ceiling Right, yeah, that feeling.
From a different angle, if OP wants to define “consciousness” as some feeling it sounds like he’s basically done. He’s labeled one of his feelings with the word “consciousness”. I’m not sure what point of contention remains.
But maybe the question you are proposing is the following, “Is it possible to identify a sensation that is always present?”.
Maybe this question is coherent? I’m not sure.
Suppose it isn’t possible to identify a sensation that is always present. Then wouldn’t that mean the state of having no sensations is identical to the state of only having sensations that are always present?
Computers are just fancy ways to locally reduce entropy. They do not exist outside the laws of the universe. Computation is not something special. It's happens! Everything is "computation". To say then that consciousness is not computation suggests there are speciaö physical laws of the universe not accessible to observation or measurement but somehow explainable by philopshers and priests.
This problem just goes away, everything divides through if we measure performance on tasks.
Consciousness is only a mystery in the sense that people studying it are committed to the idea that they’re “different” to dolphins in some deep way.
I’ve got nothing against spirituality, if people believe in a soul that’s fine by me, but eloquent speakers on spirituality don’t distract from their message with attempts to quantify it.
Lamda is not conscious, because it’s not a human.
Consciousness is a gift from God to humans (and not even to animals). Robots are not conscious, they are just programmed to act like so.
Once your philosophy stop including God and you go purely materialistic, I guess you end up with nonsense like “is lamda conscious” (or even more ridiculous “roko basilisk” - “is AI going to torture us for eternity if we don’t praise it enough?”.)
Probably not. It is after all not an Abrahamic god…
Conveniently, this makes death less scary.
Further reading: Zen and the Art of Consciousness by Susan Blackmore
It might depend on how you define "person". When I look around, I see distinct human bodies. When I wake up, I'm in the same body as when I went to sleep, so in that sense I'm the same person.
But if memories are just physically encoded in the brain, I don't think this scenario makes sense even as a thought experiment.
When "I" wake up in the other person's body, I would have all their memories and none of my previous memories. So I wouldn't even know.
Seems like exactly the sort of thing something trying to convince me of its consciousness would say to justify it. How am I supposed to know this was written by a human?
Anyways, I have a relative who was sterilized because people believed she not only had no capacity for higher reasoning, but also that her continuing to have reproductive autonomy would be a danger to others because she might pass her lack of "consciousness" along.
So you'll have to forgive me if I'm not very impressed by anyone who decides they have the key to an objective measure of consciousness. This edges far too close to eugenics in my mind.