Imagine that I visit a cloning facility that can flawlessly duplicate a human, memories and all. The techs there put me in a medical gown and sedate me, and a while later I wake up in an empty room.
Without any additional information, I have no way to determine if I am the clone or not. I know only that there is another me in the building somewhere, having the exact same thoughts as I am having right now.
In fact, I have no way to know if the cloning was even successful or not. There is no way for me to distinguish from this room if I am the clone, if the other copy was disintegrated, or if no copy was made at all. The room I'm in could be on Mars for all I know.
Does it make sense for me to worry about the fate of the other potential copy if I know absolutely that nothing of my essence has been lost either way?
You are placed in a box. Moments later, you are told, "We have successfully made a copy of you. We are sending it home now. You must be disposed of."
Will you allow them to dispose of you?
This is the question being posed, not whether a copy will have no idea if it is the original. The point is that it isn't relevant if one is a copy. No one was moved, it's only that a second person now exists and killing either is murder of a unique person.
(Again, uniqueness is not a question of whether these people will think or react to situations in the same way, but rather that there are two different conciousnesses at play.)
I think the fact that the original is kept around (and asked a question) might be complicating things.
For example, what if you were taken apart, your molecules were sent to Mars, and then you were reassembled exactly as they were? Would that be continuous consciousness?
If so, how is that different from the cloning option if your clone were assembled on Mars and the original were completely destroyed at the very same instant? (Or maybe the original were destroyed just an instant earlier.)
Yes, and if we actually had a way to disassemble and reassemble humans, the reassembled beings would tell you that the experiment works and their consciousness goes back to their first memories as a child. But the disassembled person would have been destroyed and therefore unable to share their opinion on the matter.
Basically our reasoning about identity is poorly evolved for duplicates. The closest we can come to feeling comfortable with it is going to sleep and waking back up. I think mostly because we're just used to doing that (since otherwise going to sleep and waking back up is similar to dying and coming back slightly different).
Was an interesting book overall - I'd recommend it.
Maybe I'm the odd one out, but if that was the original plan that I had intended, I wouldn't object to that (given that it was painless) - after all, minus that conversation the clone would be me "in spirit" if you will.
If I didn't consent to the cloning, that would complicate things. Likewise, my trust in the cloning process would proably affect my decision.
Or put it another way, let's say we live in a software simulation. Do you care whether our sysadmins live migrate our simulation over to a new Alien Web Services instance?
Sure, as much as it makes sense to worry about the fate of any other independent human being - and perhaps even more so, if you feel more affinity with your copies than you do with random strangers.
I wouldn't use a facility that killed a stranger at random every time I used it, and I sure as hell wouldn't use a facility if it killed a living person who was identical to me.
Frankly, I don't see what your 'essence' has to do with it. Killing people is morally wrong.
What about it, though? For the sake of argument, I'm not taking a position on whether it is or isn't wrong, I'm just looking for the "why". Ending a consciousness without either a defense motive or the permission of that consciousness (as in, violating the consciousness's right to life)? Depriving the world of a unique being? Causing damage to the human race by needlessly ending a human life? Release and/or destruction of a soul (or some other supernatural-ish argument)?
"Killing people is morally wrong" works for all of those reasons right now (and others, depending on your favored philosophy), but some of them are invalidated if we assume that we can make an exact copy of a person at a given point in time.
Violating the consciousness's right to life. I could be missing something, of course, but whether we can make an exact duplicate of a person at a given point in time doesn't seem relevant to this.
I'm curious why the answers to questions of this genre usually seem obvious to me, but clearly aren't obvious for everyone. I'm not implying any value judgement either way - perhaps I'm just a dullard. But I do wonder what the difference in mindset is.
You go to sleep every night. When you wake up the next day, how do you know you're the same person as yesterday? What does that mean?
Now consider if you didn't have memories of the past. What would make you "you"? The body would be the same but your amnesia would change things, eh?
Now what if you knew next week you would lose your memories for sure, but this week you can go on an amazing vacation. Does it matter where you go? It matters to your current self but to your future self.
Now think about sacrificing pleasure now so you can live a longer life. Or feel good about your life at 80.
Now think about the afterlife. What if you are reincarnated with no memories of the past? We already kind of established that "reincarnated you" wouldn't care about current you. But will current you care about reincarnated you?
Consider this: if a "future you" doesn't remember being "current you" how are they really different than a random person with a similar body who also has amnesia?
In short, long term, is it all about the memories?
What is the self preservation instinct about?
Finally the big one... if there were no conscious observers at all, in what sense would anything exist? That is, if I described to you a universe that you can never detect or deduce must exist from any observations in this one, how would saying "that universe EXISTS be different than saying it DOESN'T EXIST? It seems the word itself requires concious observers. Thus, the concept of consciousness and existence may be two sides of the same amazing conundrum.
You know you're the same person as yesterday because without you, there is no knowing. A clone is no more you than an identical twin is you. What makes you you is the ability to ask the question in the first place. If you were dead, there would be no question.
You know based on your ability to ask the question that you currently are you and not a clone of yourself, but you don't know that the person yesterday was you and not some other person that you were cloned from.
but you don't know that the person yesterday was you and not some other person that you were cloned from.
The parent did not say anything about this and neither did I. Of course you cannot know whether you are a clone or not. My point is that if a clone is made of you and you are killed, you will be dead, you won't be the clone.
Because when I die (or anybody else dies), the universe ends for me (or them). The clone is just a different person who can't tell that he's not me. But he is him.
The only thing you can really know for sure is that you're still alive. When you're dead, you know longer have the ability to know.
What a weird question. You're always you, until you die. Even if you became totally paralyzed, blind, deaf, etc. (total sensory deprivation) you'd still be you. All that is you is just the voice in your head, the ability to ask the question, and to think about an answer.
As for what happens to your body or anything else in the world? You can't ever get rid of those doubts. The only thing you can know without a doubt is that a doubter exists. Cogito ergo sum, as Descartes wrote.
In context of the thought experiment there can be more than one person with equal claim to be the person that you believe yourself to be. For example, given the postulated copying technology there could be Two or ten or a hundred people with the same claim 91 you to be today's version of who you remember yourself to be. The only thing that makes one person's claim better than another's is a perceived continuity of physical substrate. But even that perceived continuity is at least partly a fiction, since the physical substrate isn't really the Same from one day to the next, or even from one moment to the next. Bits of it are always being swapped out, lost and replaced. It's a valid question to ask, in that context how do you know when you wake up that you're the same person who went to sleep? Why do you think you are the same person, as opposed to an entirely new person who happens to have inherited a particular batch of ingredients of memory for assembling a story about who you are?
My argument is that you don't really know other people, you only know yourself. Clones don't even have to be physical replicas of people, they need only be entities or systems which can pass the Turing test. The fact that a clone may think it is me makes zero difference to my friends who have no way of knowing and it makes zero difference to me because I can clearly see that it is a separate person from me.
Now, you could contrive a thought experiment where you clone me in such a way that neither I nor the clone know which one of us is the clone, but then, how is that any different in principle from babies switched at birth in a hospital?
Your memories don't define you, since they may be false. The only thing that defines you is your consciousness in the present. Everything else is I/O, and subject to interpretation and trust.
My point is that you don"t know yourself, either. You know in any given moment whatever self you have constructed from the sensations, emotions, inferences, and memories available to you. It "s conventional to assume that there is continuity with the selves that you remember, and that"s the default common sense position to take--a sort of pragmatic shorthand--but we don"t actually know that the continuity is real.
In fact, we do know that human memory is spectacularly unreliable, and, moreover, that it is a virtuoso liar, very accomplished at convincing us that we vividly remember things that never actually happened. It's proficient at rearranging memories and sensory impressions into ad hoc stories that we uncritically accept as true records of events.
From this point of view it seems a little over optimistic to me to say that we know ourselves, or to say that I know that the me of today is "the same person" (whatever I might mean by that) as the me of yesterday or last week or 1965.
It's just a throwaway snide remark about expecting metaphysics to have concrete answers for anything (It's not even coherent; if the air molecules don't vibrate without an observer, there probably isn't a tree or a forest either).
(metaphysics is still an interesting and useful exercise, but the state of not having an answer to a particular question isn't much more of a real conundrum than not having asked the question in the first place. I mean, when you are hungry, eating feels pretty good regardless of whether the food exists or not.)
I personally try to resolve this quandary in my head by defining my consciousness not as a single state at an instant in time, but rather a pattern over a time continuity - just as mass and matter occupies a continuum of space instead of a single point.
Going even deeper, from one moment to another - how do you know that you're you? After all, you're not _exactly_ the same, since each moment brings new experiences, ever so slightly changing your memories. Quantum entanglement, perhaps? If not, aren't we a collection of particles with a certain probability distribution that we're able to influence ourselves?
(Maybe this whole discussion is less interesting to Buddhists trained in not-self thinking. :)
There are a few problems I see with this. Each individuals perspective is limited. That is only reason why your 'clone' might not know or care about those things.
As third party observers in that situation we would know that one is a 'copy' and we (or at least I) would care (and even object) to the killing of either one. From my perspective I can see that there are two distinct unique individuals having different thoughts saying different things and doing different things. Its when the 'clones' are still alive that you can tell that you have two different people now. Since they occupy separate space they will have different perspectives and experiences and different lives.
> From my perspective I can see that there are two distinct unique individuals having different thoughts saying different things and doing different things.
Can you though? Assuming the cloning process works, and assuming they wake up in identical rooms, why would they think, say, or do anything differently?
Yes, because it wouldn't be possible to keep the environment "identical" for very long, so they would diverge.
Thinking about it more... I don't even think it is remotely possible to control the "initial conditions" perfectly enough (upon waking up), so they would diverge instantly. Sure, they might act very similarly, but it would quite clear that they are separate entities.
Another similar thought experiment is what happen if we cut ourselves laterally exactly in half and then fill back in the missing half on both parts. In the case of both halves, we are continuing an existing consciousness.
We have some data on this from split-brain patients. These are people whose corpus callosum---a bundle of fibers linking their left and right cerebral hemispheres---is cut. Such people start to act as if they are two individuals. Each hemisphere only has access to half of the visual field, and due to a quirk of how language is lateralized (typically left hemisphere only) they can make conflicting verbal and nonverbal reports of ambiguous stimuli: saying one thing yet writing another.
So given this, it's not really fair to say that there is a unitary state to split. Each half would get a different state (perhaps "synchronized" at the time of the split), and only one half would be able to talk about the experience.
The author seems to believe continuous consciousness is necessary for identity. I hope he never falls asleep. If he does that means he is dead, and tomorrow morning an usurper with his memories will wake up in his bed.
The author of the article is asking a bit of a different question than the the authors of the articles he cites. The cited are more about "what is consciousness" in the context of can a computer be. The author is asking "what am I" That's a little harder isn't it?
He's asking if he got copied instantly and sent to Mars which one of the copies would still be me? Well, they both would wouldn't they? Each having a continuous experience leading up to the point they were copies and stepped out of the machine.
The fun part is that, barring quantum hooey that the articles bring up, there seems to be no scientific reason to doubt the physical possibility of this eventually occurring. If you can get copied, and both copies are you, what happens to the underpinnings of most world religions? Delightfully subversive thought right there.
"And what of the immortal soul in such transactions? Can this machine transmit and reattach it as well? Or is it lost forever, leaving a soulless body to wander the world in despair?"
Look at any photograph or work of art. If you could duplicate exactly the first tiny dot of color, and then the next and the next, you would end with a perfect copy of the whole, indistinguishable from the original in every way, including the so-called "moral value" of the art itself. Nothing can transcend its smallest elements.
It's less a fallacy than a fantasy. It's a bizarre universalist solipsism: A perception that all people are is what is important for you to consider, carried to the most absurd conclusion of thinking that what you are is only what is important for others to consider.
Therefore, if you duplicate a process, delete the old and copy the new one in its place, it's the same process.
This is totally acceptable to me if there is a proof somewhere that there is no such thing as subjective experience, but then it makes no sense to talk about "consciousnesses" as separable anyway, and the entire discussion is pointless.
edit: If Peapod suddenly goes out of business and Amazon seamlessly picks up my grocery deliveries, is Amazon Peapod?
The author's "fallacy" isn't so much a fallacy as much as it is one particular (and rather unsurprising) opinion on how to answer the questions that Aaronson and Hanson have asked.
In this sense, the author fundamentally misses the point of what Aaronson and Hanson have written. Aaronson and Hanson seem to be primarily interested in identifying questions and exploring the problems that arise when trying to give simple answers to those questions.
The fact that the author is able to quickly and cleanly resolve his personal belief about how these questions should be answered doesn't indicate that the questions are easy to answer or even that there is an obviously correct answer. At least any more than a Christian's belief in God cleanly settles the question of whether God exists.
I think what throwawaysocks is saying is that your refutation doesn't really add value to the discussion. I agree with you, that my clone's consciousness is not my own consciousness, but what are the deeper implications?
Particularly when we start to consider AI, and its not your own human consciousness but rather an artificial one, does our notion of "murder" change? Then they question, is true "cloning" even possible? (Aaronson uses quantum uncertainty to say 'no', but Hanson claims 'yes' given total understanding of the brain.)
Thanks for your article, I'm personally on your side but this whole debate has really made me think.
I came to conclusion that torturing Sims is indeed morally wrong. Now it's not that much wrong, because they are not very self-aware, but if they were more self-aware, it would be very wrong. So yeah, killing clones or even virtual replicas willy-nilly is wrong.
On the other hand, I would have no problem with teleportation-via-cloning. I don't have a problem with being unconscious every night, after all (kudos to http://existentialcomics.com/comic/1 for pointing that out to me). Also I have no problem, every day when I go to work, to sit for half an hour in a huge metal structure which is propelled by enormous speed through a tunnel. What if it crashed, and I would be smashed to death? It's pretty much the same question the article asks, what if some incident caused the clone not being recreated successfully? I simply trust the people who built it and maintain it, and I know many people use it with no issues, so why not use it?
Yes and no. It's not wrong to just stop the simulation forever. Torturing or repeatedly killing a virtual being for fun is wrong, but turning off ("killing") a virtual being for the purpose of it being turned off is acceptable.
However, it can be wrong to erase their universe, if that potentially destroys something interesting the virtual beings have created.
In other words, the intent matters. Killing a mouse for fun is immoral, killing it to research cancer is not.
It's an interesting problem, but I don't feel like this page accomplished much besides calling a position he disagreed with a fallacy and using unexplained appeals to emotion.
>To refute this, let’s conduct a thought experiment. Pretend that you can copy a human brain. There are ten copies of me. They are all individually conscious — perfect replicas that only diverge after the point when replication happened. Is it okay to kill five of these copies? No, of course not! Each one is a self-aware, intelligent mind, human in everything but body. The identicalness doesn’t change that.
The things that scare me about death are the ideas that my specific goals would be left unpursued, that my responsibilities would be abandoned, my friends would lose my friendship, and that my memories and experiences would lose their significance upon the world.
If an alien came to Earth with a perfect matter-duplicater (that I have reason to fully trust), duplicated me such that I'm not even sure which is the original, then remembers that he's not allowed to use it on humans, and explains he needs to painlessly euthanize and disassemble one of me, I wouldn't be particularly shaken up by it. Sure, I'd feel I was teased with a cool toy and possibility, but the only injury that has happened is in a way equivalent to a few seconds of memory loss. I've done worse to myself by drinking. There were no big life experiences lost or goals left unpursued.
---
Going a little further... if I were an AI or could somehow copy myself, I imagine I would frequently make short-lived copies to work together with to accomplish my goals. I as my copies would be told that I'm going to be removed in a few days; I think I could handle the idea of knowing the surviving-me is going to be amnesic to my several days of (probably barely)unique experiences in a world much closer to our goals.
In a world where everyone or all AIs could do the above, I imagine the minds who did the above would become much more successful in nearly all endeavors. Choosing not to do so would be choosing to fail, unless the strategy was made illegal. I struggle to fault the strategy morally given that it would be something done by a single consenting individual to themselves, though it does still sound a little weird to my instincts about how survival is supposed to work. If we encountered otherwise similar aliens which already worked that way, would we judge them, or only consider it an abomination when humans did it to themselves?
I'm a bit disappointed that I haven't seen much sci-fi explore concepts like this.
The long-running space opera comic "Schlock Mercenary" actually has treatments of this topic that are frequently clever (and humorous). As example, here [http://www.schlockmercenary.com/2004-08-23] is an instance of an AI making copies of itself to act as the smart guidance in warship-killing high-energy munitions. If the munition survives impact, it can be refurbished and used again; if not, the AI shard dumps a 'gestalt' of new experiences back to the core before the munition breaks up.
Since the AI copies are crafted by the master AI, they don't generally exhibit divergence from its will or an independent survival instinct.
This is something I've thought about frequently, and I completely agree. The "purpose" of my existence would be maintained, so why would I care which one you choose?
We all do this without thought on a daily basis with files on our filesystem. A particular document encodes a set of information; if we duplicate that document, we now have two identical copies of the same information. By the original author's argument, they are unique and distinct, because they occupy different blocks on our storage medium! But common sense makes it clear that which one we delete doesn't matter, because they serve the same purpose - provided that we preserve at least one copy.
(To those that argue that the file system metaphor is flawed, because consciousness is "different" or "special" in some way, I reference the inverse of Clarke's Third Law: Any sufficiently understood magic is indistinguishable from technology.)
Now, if those documents continue to evolve and change over time, such that they no longer match, now you are left with a version control problem, and one that seems to be nigh-impossible to solve with a consciousness. In my opinion, this is a situation worth avoiding if we can, as it becomes very sticky very quickly, since the uniqueness of the experiences produces an infinite gradient that makes deciding on an appropriate cutoff of termination very difficult.
As for the continuity of thought, we certainly don't see falling asleep at the end of the day as the "death" of our current consciousness, or waking up in the morning as the "birth" of our new consciousness, despite the (perceived) discontinuity. So why would your source or target self perceive it that way?
Fortunately, regardless of how you feel about the whole "continuity of thought" argument, the solution is very simple - users would simply not be required to make use of these new products/services. Don't want to get faxed to Mars? No problem! Users are not currently required to make copies of files, or transmit files, or make backups of files, after all.
Anyway, that was longer than I had intended. Hopefully someone finds that somewhat interesting, although I suppose it's really all moot anyway until we're even remotely close to the technology required to implement this.
You are begging the question. Who is "I" in this case, when there are two beings in existence? You are presuming that "you" exists as a singular thing, even when there are two physical bodies. But that is no longer true the moment that you are cloned. There are now two separate entities that diverge slowly, as even a slight difference in location will mean different inputs to your senses and different events occurring that change your respective paths.
MILD SPOILER: If you could make short-lived copies of yourself, you should probably keep a close eye on them. After all, you're clever, they're clever. You're motivated to keep existing, so are they.
This is kind of the point I'm arguing against. I'm sure many people would have struggles with their copies, but I think someone disciplined and with the right (well, useful) definitions of self and the meanings of survival internalized could avoid the issue entirely, and accept their short-livedness if they know another instance of themselves is going to continue.
Given the choice between having one copy disassembled or having both disassembled, I'd choose to have one copy survive, whether or not it was the "I" copy. Both copies would be "me" as far as could be measured (ignoring the last few minutes, where I'm assuming that our experiences would be similar but not identical).
Given the choice of having both of "us" survive though, I'd rather not have either life terminated. I'd be more bothered by the destruction of one of us than by the chance that I'd be the one who chooses to start a new life from scratch (or whatever we decided as our solution). Death is death, whether or not there's a substantially-identical copy. One consciousness would end.
This doesn't deserve the label "fallacy"; the only thing I got is that the author disagrees about the implications of restarting a mind somewhere else. Well, great, but you need to explain substantively why you think that is, not simply disagree with it.
But even then, you should only label the flawed premise -- the one you're refuting to make this point -- as the fallacy, not the conclusion.
What's worse, the author's disagreement has to deal with the (almost) established "consciousness branching" that happens all the time. If you accept the universality of the Schrödinger equation, then your body is branching into different parts of the wavefunction, yet (each of) you obviously internally perceive it as the same consciousness.
The no-cloning theorem together with chaos theory suggests that cloning is fundamentally impossible. Mostly people will respond to this by saying "the brain isn't a quantum computer", which is almost certainly true, but what is actually required for mind uploading is "the brain's behavior is not influenced by quantum phenomena at all", which is even more certainly false.
Yet we're still obsessed with mind uploading. Why?
I guess it's because, while we accept that life seems to have some "atman", a sense of identity, and quantum mechanics allows this in a very weak sense, it makes no sense that our perceptions would be at all influenced by quantum mechanics. Using the no-cloning theorem to resolve the problem of mind uploading seems like cheating, and we're all pretty sure that souls don't exist, so we tend to think that anyone who looks for something like a soul in fundamental physics must be pushing an agenda. But this latter reasoning is a fallacy; the well is poisoned when anyone who suggests that quantum mechanics might derail computational neuroscience must believe in some weird magical revisionism of ESP or hypercomputations occurring in the brain or whatever.
But there's an easier culprit hiding in plain sight: the tendency of people to talk more about things they think they can understand, than things where they run a serious risk of being proven flat wrong. I could be flat wrong (but this is not my first rodeo); many posters in this thread as well as the blogger cannot. Loss aversion strikes again.
I don't understand why the author of the article believes that it's impossible to clone consciousness.
If you made an exact copy of yourself, then you would be able to control both bodies simultaneously and you would also receive sensory feedback from both bodies at the same time.
The main caveat is that these sensory experiences will be partitioned across the two bodies - You (your consciousness) will receive sensory information from both bodies simultaneously but you won't be able to synchronize that information across the partition (unless you made both of your bodies talk to each other).
Just imagine that all creatures on earth were born with only half a body split down the middle; one eye, one ear, one nostril, one arm, etc... As (half) a human, you would probably think that two half-bodies sharing a single mind is impossible... Maybe, as (full) humans, when we think about two bodies sharing a single mind, we also fall into the same psychological trap.
It's true that comparing two half-bodies vs two full-bodies isn't a perfect comparison because in real life, the right and left halves of our brains are connected and do exchange some information... But studies show that some people have been able to live with only half a brain (or half of the brain severely damaged) - Both hemispheres do experience a certain level of isolation (partition) from each other; so a lot of the information is not actually shared across both hemispheres and yet you still experience it. So this does lend some credence to the notion that a single consciousness (or at least a part of that single consciousness) CAN exist across a physical partition.
So the two-body scenario is just a slightly more extreme version of the two-halves scenario which we all subjectively experience on a daily basis.
Imagine waking up in a strange facility. There's a tray in front of you with a pill and a label that says "eat me."
In World A, the drug causes amnesia--after 12 hours, you'll fall asleep, and completely forget the preceding 12 hours.
In World B, the drug is a suicide pill--after 12 hours, you will fall asleep and die painlessly. However, just before you woke up, a "backup" was uploaded. And upon your death, this backup will be restored.
It seems to me that the subjective experience(s) in both cases would be the same. But we consider the latter "death" and the former "too much to drink."
There is no fallacy. Everyone's here who is intrigued by this question but dissapointed with the rather simple and unresolving reply in this blog post, should watch Moon directed by Duncan Jones (David Bowie's son who recently directed Warcraft.) Questions like these don't have simple answers. I'm rather dissapointed with the hand wave of emotion the post author ascribes to. But it is bringing the topic up, so I am happy for that.
I wonder, what does quantum physics have to do with human experience and brain operation?
It's a very persistent idea floating around, despite contradicting everything we know about how a brain operates.
More on topic, what he misses is that "consciousness" is a null word. It does not connect to any real phenomenon, and merely a result of a human being unable to imagine it's own non-existence, thus producing the long list of things like anima, spirit soul, consciousness and so on.
Just to throw out a few references, since they seem unmentioned so far:
The Ship of Theseus is an ancient recognition of a similar problem.
Greg Egan has written two stories that deal with copy-vs-continuity of consciousness: Diaspora and Permutation City. Both start with the premise "what if we could digitize our mind?" but go in different directions.
Consider another version: An exactly similar human being to you is discovered to exist on another planet. They have your memories, your aspirations, your habits, etc. They were not created as a copy of you. This person was born to parents, just like you, and lived the same life that you did. They are reading this post on HN right now.
Surely in this scenario the answer to "is it OK to kill one of you now?" is "no".
So why is this different to some people than a synthetically-created clone?
>Surely in this scenario the answer to "is it OK to kill one of you now?" is "no".
I don't know about that.
I'm not sure about this intuition pump. If they are truly identical to me, then the world they live in must also me identical to mine or we would have diverged. So if we discovered Earth-2, then Earth-2 would simultaneously discover us (or Earth-3 etc). No actions can cause divergence, so it's not possible for anyone on any Earth to only kill one of you. It would be like trying to kill your reflection.
If we're talking about some kind of god who is outside the mirror/chain Earth system, and has the ability to break symmetry as they see fit, it's still not clear-cut. In effect, from such a perspective, there is only one Earth (albeit with copies) until changes are made. It's not even clear that deleting one of the several identical copies of the whole Earth is wrong. Moreover, deciding to kill someone on one of several heretofore identical Earth's and seeing how they diverge is informationally the same as spawning a new Earth - an act of creation rather than destruction. Nothing is lost - that person still lives their life on an Earth somewhere - but now new lives will be led, new experiences, new sadness but also somewhere new joy.
I think it's pretty clear though that our morality of "no killing" is based pretty strongly upon the assumption that there's only one of someone, they can only be killed once, and it's forever. Start violating these assumptions and it breaks down quickly.
Only if you assume the finality of death makes anything less good. If you assume every conscious being has inherent value then the entire argument has to go another way which isn't much better, but it can be characterized as a viable alternative if perfect copies of any one person could be created down to the Planck constants.
This whole discussion sheds a bit of light on the inevitable difficulties one encounters when attempting to remain unflinchingly dedicated to a purely materialistic philosophical worldview.
My point is essentially to encourage open-mindedness about that which we don't (and possibly never can) fully understand from a materialist (as opposed to a philosophical) perspective. :)
One's position in space-time is an inextricable component of "the self". Even in theory, a perfectly replicated individual has already fundamentally diverged from the original just by virtue of its existence as a duplicate (since duplication implies a distinct position in space-time). Ultimately, what I'm saying is that the idea of a "perfect duplicate" is something of a contradiction, and any theory of mind that attempts to define one's "homunculus" in terms of a physical brain state is in some ways moot because although our human senses might indicate that we are dealing with two identical individuals we are actually dealing with two fundamentally distinct beings, despite the apparent similarities.
William Thomas "Tom" Riker was a result of a transporter accident in 2361 that created two William Thomas Rikers, genetically indistinguishable from each other, with personality and memories identical up to the point of the duplication. One of the duplicates continued to be known as William Riker. The other chose to use his middle name and be known as Thomas Riker.
Thomas and William clashed almost immediately due to the resentment each felt towards the other. The eight years of living different lives made them entirely different men – William evolved into a cautious and duty-driven officer while Thomas remained impulsive and reckless. As a result of conflicts with William – who was by that time a superior officer (a commander) – Lieutenant Riker decided to leave the Enterprise. He chose to use his middle name, Thomas, to distinguish himself from his counterpart. Captain Jean-Luc Picard managed to get Thomas a posting on the USS Gandhi where he could continue his Starfleet career. Upon Thomas' departure, William seemed to have accepted Thomas' existence and gave him his trombone as a parting gift. (TNG: "Second Chances")
I would, yes, though I'd be careful about drawing comparisons in this regard between computer programs (as we know them today) and the phenomenon that we call consciousness, since the framework of computation deliberately extricates the essence of a program from the physical phenomena that underpins its existence.
Okay, but then you're just moving the debate over a few feet rather than addressing the core point: can a human likewise be abstracted away to some substrate-independent program such that we lose nothing by deleting some running copies? Can we build a box that gives us the same answers Bill Gates would, and causes us to deem it "another Bill Gates" without confusion, modulo having a body and physical access to the world?
> can a human likewise be abstracted away to some substrate-independent program such that we lose nothing by deleting some running copies
Oh. Well, assuming our current model of computation, I think a "substrate-independent" human program is essentially non-human by definition (that is to say, lacking the particular qualities of 'consciousness' that we commonly ascribe to humans), so I'd conclude that "nothing is lost" insomuch as "nothing existed" in the first place. In theory, it seems plausible that we could create a program that could consistently pass a "Bill Gates" Turing test, but it is easy for a programmer to imagine how such a program could exist without actually being conscious.
Since there isn't a computer that seems conscious at this time, the idea of machine consciousness is supported by thought experiments. Here's one old chestnut: "What if you replaced your neurons one by one with neuron-sized and shaped substitutes made of silicon chips that perfectly mimicked the chemical and electric functions of the originals? If you just replaced one single neuron, surely you'd feel the same. As you proceed, as more and more neurons are replaced, you'd stay conscious. Why wouldn't you still be conscious at the end of the process, when you'd reside in a brain-shaped glob of silicon? And why couldn't the resulting replacement brain have been manufactured by some other means?"
91 comments
[ 3.1 ms ] story [ 152 ms ] threadPersonal identity is a deep philosophical problem - see Derek Parfit's 'Reasons and Persons'.
Without any additional information, I have no way to determine if I am the clone or not. I know only that there is another me in the building somewhere, having the exact same thoughts as I am having right now.
In fact, I have no way to know if the cloning was even successful or not. There is no way for me to distinguish from this room if I am the clone, if the other copy was disintegrated, or if no copy was made at all. The room I'm in could be on Mars for all I know.
Does it make sense for me to worry about the fate of the other potential copy if I know absolutely that nothing of my essence has been lost either way?
You are placed in a box. Moments later, you are told, "We have successfully made a copy of you. We are sending it home now. You must be disposed of."
Will you allow them to dispose of you?
This is the question being posed, not whether a copy will have no idea if it is the original. The point is that it isn't relevant if one is a copy. No one was moved, it's only that a second person now exists and killing either is murder of a unique person.
(Again, uniqueness is not a question of whether these people will think or react to situations in the same way, but rather that there are two different conciousnesses at play.)
For example, what if you were taken apart, your molecules were sent to Mars, and then you were reassembled exactly as they were? Would that be continuous consciousness?
If so, how is that different from the cloning option if your clone were assembled on Mars and the original were completely destroyed at the very same instant? (Or maybe the original were destroyed just an instant earlier.)
Let's say your molecules were taken apart... and that's it.
That's called "dying".
I'm not sure how to reconcile this with the nature of consciousness, but it certainly makes for an interesting thought experiment.
https://www.quora.com/How-long-does-it-take-for-most-of-the-...
This particular one reminds me of the teleporter example which is in this excerpt: https://www.getflashnotes.com/waking-up-by-sam-harris/ (just control-f for teleporter).
Basically our reasoning about identity is poorly evolved for duplicates. The closest we can come to feeling comfortable with it is going to sleep and waking back up. I think mostly because we're just used to doing that (since otherwise going to sleep and waking back up is similar to dying and coming back slightly different).
Was an interesting book overall - I'd recommend it.
If I didn't consent to the cloning, that would complicate things. Likewise, my trust in the cloning process would proably affect my decision.
Or put it another way, let's say we live in a software simulation. Do you care whether our sysadmins live migrate our simulation over to a new Alien Web Services instance?
I wouldn't use a facility that killed a stranger at random every time I used it, and I sure as hell wouldn't use a facility if it killed a living person who was identical to me.
Frankly, I don't see what your 'essence' has to do with it. Killing people is morally wrong.
What about it, though? For the sake of argument, I'm not taking a position on whether it is or isn't wrong, I'm just looking for the "why". Ending a consciousness without either a defense motive or the permission of that consciousness (as in, violating the consciousness's right to life)? Depriving the world of a unique being? Causing damage to the human race by needlessly ending a human life? Release and/or destruction of a soul (or some other supernatural-ish argument)?
"Killing people is morally wrong" works for all of those reasons right now (and others, depending on your favored philosophy), but some of them are invalidated if we assume that we can make an exact copy of a person at a given point in time.
Violating the consciousness's right to life. I could be missing something, of course, but whether we can make an exact duplicate of a person at a given point in time doesn't seem relevant to this.
I'm curious why the answers to questions of this genre usually seem obvious to me, but clearly aren't obvious for everyone. I'm not implying any value judgement either way - perhaps I'm just a dullard. But I do wonder what the difference in mindset is.
You go to sleep every night. When you wake up the next day, how do you know you're the same person as yesterday? What does that mean?
Now consider if you didn't have memories of the past. What would make you "you"? The body would be the same but your amnesia would change things, eh?
Now what if you knew next week you would lose your memories for sure, but this week you can go on an amazing vacation. Does it matter where you go? It matters to your current self but to your future self.
Now think about sacrificing pleasure now so you can live a longer life. Or feel good about your life at 80.
Now think about the afterlife. What if you are reincarnated with no memories of the past? We already kind of established that "reincarnated you" wouldn't care about current you. But will current you care about reincarnated you?
Consider this: if a "future you" doesn't remember being "current you" how are they really different than a random person with a similar body who also has amnesia?
In short, long term, is it all about the memories?
What is the self preservation instinct about?
Finally the big one... if there were no conscious observers at all, in what sense would anything exist? That is, if I described to you a universe that you can never detect or deduce must exist from any observations in this one, how would saying "that universe EXISTS be different than saying it DOESN'T EXIST? It seems the word itself requires concious observers. Thus, the concept of consciousness and existence may be two sides of the same amazing conundrum.
You know based on your ability to ask the question that you currently are you and not a clone of yourself, but you don't know that the person yesterday was you and not some other person that you were cloned from.
The parent did not say anything about this and neither did I. Of course you cannot know whether you are a clone or not. My point is that if a clone is made of you and you are killed, you will be dead, you won't be the clone.
> When you wake up the next day, how do you know you're the same person as yesterday?
> You know you're the same person as yesterday
What defines "you"? The matter you're made of?
Is it a particular arrangement of that matter? (Would you care if your neurons were replaced one by one, while keeping the same arrangement?)
Is it your memories, your thoughts, your inclinations?
Or is it something else?
The only thing you can really know for sure is that you're still alive. When you're dead, you know longer have the ability to know.
As for what happens to your body or anything else in the world? You can't ever get rid of those doubts. The only thing you can know without a doubt is that a doubter exists. Cogito ergo sum, as Descartes wrote.
Now, you could contrive a thought experiment where you clone me in such a way that neither I nor the clone know which one of us is the clone, but then, how is that any different in principle from babies switched at birth in a hospital?
Your memories don't define you, since they may be false. The only thing that defines you is your consciousness in the present. Everything else is I/O, and subject to interpretation and trust.
In fact, we do know that human memory is spectacularly unreliable, and, moreover, that it is a virtuoso liar, very accomplished at convincing us that we vividly remember things that never actually happened. It's proficient at rearranging memories and sensory impressions into ad hoc stories that we uncritically accept as true records of events.
From this point of view it seems a little over optimistic to me to say that we know ourselves, or to say that I know that the me of today is "the same person" (whatever I might mean by that) as the me of yesterday or last week or 1965.
(metaphysics is still an interesting and useful exercise, but the state of not having an answer to a particular question isn't much more of a real conundrum than not having asked the question in the first place. I mean, when you are hungry, eating feels pretty good regardless of whether the food exists or not.)
I personally try to resolve this quandary in my head by defining my consciousness not as a single state at an instant in time, but rather a pattern over a time continuity - just as mass and matter occupies a continuum of space instead of a single point.
(Maybe this whole discussion is less interesting to Buddhists trained in not-self thinking. :)
As third party observers in that situation we would know that one is a 'copy' and we (or at least I) would care (and even object) to the killing of either one. From my perspective I can see that there are two distinct unique individuals having different thoughts saying different things and doing different things. Its when the 'clones' are still alive that you can tell that you have two different people now. Since they occupy separate space they will have different perspectives and experiences and different lives.
Can you though? Assuming the cloning process works, and assuming they wake up in identical rooms, why would they think, say, or do anything differently?
Thinking about it more... I don't even think it is remotely possible to control the "initial conditions" perfectly enough (upon waking up), so they would diverge instantly. Sure, they might act very similarly, but it would quite clear that they are separate entities.
Would you be OK killing off one of the halves?
So given this, it's not really fair to say that there is a unitary state to split. Each half would get a different state (perhaps "synchronized" at the time of the split), and only one half would be able to talk about the experience.
He's asking if he got copied instantly and sent to Mars which one of the copies would still be me? Well, they both would wouldn't they? Each having a continuous experience leading up to the point they were copies and stepped out of the machine.
The fun part is that, barring quantum hooey that the articles bring up, there seems to be no scientific reason to doubt the physical possibility of this eventually occurring. If you can get copied, and both copies are you, what happens to the underpinnings of most world religions? Delightfully subversive thought right there.
-- Sister Miriam Godwinson, "We Must Dissent"
-- CEO Nwabudike Morgan, "The Ethics of Greed"
Therefore, if you duplicate a process, delete the old and copy the new one in its place, it's the same process.
This is totally acceptable to me if there is a proof somewhere that there is no such thing as subjective experience, but then it makes no sense to talk about "consciousnesses" as separable anyway, and the entire discussion is pointless.
edit: If Peapod suddenly goes out of business and Amazon seamlessly picks up my grocery deliveries, is Amazon Peapod?
In this sense, the author fundamentally misses the point of what Aaronson and Hanson have written. Aaronson and Hanson seem to be primarily interested in identifying questions and exploring the problems that arise when trying to give simple answers to those questions.
The fact that the author is able to quickly and cleanly resolve his personal belief about how these questions should be answered doesn't indicate that the questions are easy to answer or even that there is an obviously correct answer. At least any more than a Christian's belief in God cleanly settles the question of whether God exists.
Particularly when we start to consider AI, and its not your own human consciousness but rather an artificial one, does our notion of "murder" change? Then they question, is true "cloning" even possible? (Aaronson uses quantum uncertainty to say 'no', but Hanson claims 'yes' given total understanding of the brain.)
Thanks for your article, I'm personally on your side but this whole debate has really made me think.
On the other hand, I would have no problem with teleportation-via-cloning. I don't have a problem with being unconscious every night, after all (kudos to http://existentialcomics.com/comic/1 for pointing that out to me). Also I have no problem, every day when I go to work, to sit for half an hour in a huge metal structure which is propelled by enormous speed through a tunnel. What if it crashed, and I would be smashed to death? It's pretty much the same question the article asks, what if some incident caused the clone not being recreated successfully? I simply trust the people who built it and maintain it, and I know many people use it with no issues, so why not use it?
However, it can be wrong to erase their universe, if that potentially destroys something interesting the virtual beings have created.
In other words, the intent matters. Killing a mouse for fun is immoral, killing it to research cancer is not.
>To refute this, let’s conduct a thought experiment. Pretend that you can copy a human brain. There are ten copies of me. They are all individually conscious — perfect replicas that only diverge after the point when replication happened. Is it okay to kill five of these copies? No, of course not! Each one is a self-aware, intelligent mind, human in everything but body. The identicalness doesn’t change that.
The things that scare me about death are the ideas that my specific goals would be left unpursued, that my responsibilities would be abandoned, my friends would lose my friendship, and that my memories and experiences would lose their significance upon the world.
If an alien came to Earth with a perfect matter-duplicater (that I have reason to fully trust), duplicated me such that I'm not even sure which is the original, then remembers that he's not allowed to use it on humans, and explains he needs to painlessly euthanize and disassemble one of me, I wouldn't be particularly shaken up by it. Sure, I'd feel I was teased with a cool toy and possibility, but the only injury that has happened is in a way equivalent to a few seconds of memory loss. I've done worse to myself by drinking. There were no big life experiences lost or goals left unpursued.
---
Going a little further... if I were an AI or could somehow copy myself, I imagine I would frequently make short-lived copies to work together with to accomplish my goals. I as my copies would be told that I'm going to be removed in a few days; I think I could handle the idea of knowing the surviving-me is going to be amnesic to my several days of (probably barely)unique experiences in a world much closer to our goals.
In a world where everyone or all AIs could do the above, I imagine the minds who did the above would become much more successful in nearly all endeavors. Choosing not to do so would be choosing to fail, unless the strategy was made illegal. I struggle to fault the strategy morally given that it would be something done by a single consenting individual to themselves, though it does still sound a little weird to my instincts about how survival is supposed to work. If we encountered otherwise similar aliens which already worked that way, would we judge them, or only consider it an abomination when humans did it to themselves?
I'm a bit disappointed that I haven't seen much sci-fi explore concepts like this.
Since the AI copies are crafted by the master AI, they don't generally exhibit divergence from its will or an independent survival instinct.
We all do this without thought on a daily basis with files on our filesystem. A particular document encodes a set of information; if we duplicate that document, we now have two identical copies of the same information. By the original author's argument, they are unique and distinct, because they occupy different blocks on our storage medium! But common sense makes it clear that which one we delete doesn't matter, because they serve the same purpose - provided that we preserve at least one copy.
(To those that argue that the file system metaphor is flawed, because consciousness is "different" or "special" in some way, I reference the inverse of Clarke's Third Law: Any sufficiently understood magic is indistinguishable from technology.)
Now, if those documents continue to evolve and change over time, such that they no longer match, now you are left with a version control problem, and one that seems to be nigh-impossible to solve with a consciousness. In my opinion, this is a situation worth avoiding if we can, as it becomes very sticky very quickly, since the uniqueness of the experiences produces an infinite gradient that makes deciding on an appropriate cutoff of termination very difficult.
As for the continuity of thought, we certainly don't see falling asleep at the end of the day as the "death" of our current consciousness, or waking up in the morning as the "birth" of our new consciousness, despite the (perceived) discontinuity. So why would your source or target self perceive it that way?
Fortunately, regardless of how you feel about the whole "continuity of thought" argument, the solution is very simple - users would simply not be required to make use of these new products/services. Don't want to get faxed to Mars? No problem! Users are not currently required to make copies of files, or transmit files, or make backups of files, after all.
Anyway, that was longer than I had intended. Hopefully someone finds that somewhat interesting, although I suppose it's really all moot anyway until we're even remotely close to the technology required to implement this.
> I wouldn't be particularly shaken up by it
You are begging the question. Who is "I" in this case, when there are two beings in existence? You are presuming that "you" exists as a singular thing, even when there are two physical bodies. But that is no longer true the moment that you are cloned. There are now two separate entities that diverge slowly, as even a slight difference in location will mean different inputs to your senses and different events occurring that change your respective paths.
MILD SPOILER: If you could make short-lived copies of yourself, you should probably keep a close eye on them. After all, you're clever, they're clever. You're motivated to keep existing, so are they.
This is kind of the point I'm arguing against. I'm sure many people would have struggles with their copies, but I think someone disciplined and with the right (well, useful) definitions of self and the meanings of survival internalized could avoid the issue entirely, and accept their short-livedness if they know another instance of themselves is going to continue.
Given the choice between having one copy disassembled or having both disassembled, I'd choose to have one copy survive, whether or not it was the "I" copy. Both copies would be "me" as far as could be measured (ignoring the last few minutes, where I'm assuming that our experiences would be similar but not identical).
Given the choice of having both of "us" survive though, I'd rather not have either life terminated. I'd be more bothered by the destruction of one of us than by the chance that I'd be the one who chooses to start a new life from scratch (or whatever we decided as our solution). Death is death, whether or not there's a substantially-identical copy. One consciousness would end.
But even then, you should only label the flawed premise -- the one you're refuting to make this point -- as the fallacy, not the conclusion.
What's worse, the author's disagreement has to deal with the (almost) established "consciousness branching" that happens all the time. If you accept the universality of the Schrödinger equation, then your body is branching into different parts of the wavefunction, yet (each of) you obviously internally perceive it as the same consciousness.
Yet we're still obsessed with mind uploading. Why?
I guess it's because, while we accept that life seems to have some "atman", a sense of identity, and quantum mechanics allows this in a very weak sense, it makes no sense that our perceptions would be at all influenced by quantum mechanics. Using the no-cloning theorem to resolve the problem of mind uploading seems like cheating, and we're all pretty sure that souls don't exist, so we tend to think that anyone who looks for something like a soul in fundamental physics must be pushing an agenda. But this latter reasoning is a fallacy; the well is poisoned when anyone who suggests that quantum mechanics might derail computational neuroscience must believe in some weird magical revisionism of ESP or hypercomputations occurring in the brain or whatever.
But there's an easier culprit hiding in plain sight: the tendency of people to talk more about things they think they can understand, than things where they run a serious risk of being proven flat wrong. I could be flat wrong (but this is not my first rodeo); many posters in this thread as well as the blogger cannot. Loss aversion strikes again.
If you made an exact copy of yourself, then you would be able to control both bodies simultaneously and you would also receive sensory feedback from both bodies at the same time.
The main caveat is that these sensory experiences will be partitioned across the two bodies - You (your consciousness) will receive sensory information from both bodies simultaneously but you won't be able to synchronize that information across the partition (unless you made both of your bodies talk to each other).
Just imagine that all creatures on earth were born with only half a body split down the middle; one eye, one ear, one nostril, one arm, etc... As (half) a human, you would probably think that two half-bodies sharing a single mind is impossible... Maybe, as (full) humans, when we think about two bodies sharing a single mind, we also fall into the same psychological trap.
It's true that comparing two half-bodies vs two full-bodies isn't a perfect comparison because in real life, the right and left halves of our brains are connected and do exchange some information... But studies show that some people have been able to live with only half a brain (or half of the brain severely damaged) - Both hemispheres do experience a certain level of isolation (partition) from each other; so a lot of the information is not actually shared across both hemispheres and yet you still experience it. So this does lend some credence to the notion that a single consciousness (or at least a part of that single consciousness) CAN exist across a physical partition.
So the two-body scenario is just a slightly more extreme version of the two-halves scenario which we all subjectively experience on a daily basis.
In World A, the drug causes amnesia--after 12 hours, you'll fall asleep, and completely forget the preceding 12 hours.
In World B, the drug is a suicide pill--after 12 hours, you will fall asleep and die painlessly. However, just before you woke up, a "backup" was uploaded. And upon your death, this backup will be restored.
It seems to me that the subjective experience(s) in both cases would be the same. But we consider the latter "death" and the former "too much to drink."
(http://www.overcomingbias.com/2016/04/is-forgotten-party-dea... got me thinking in these terms--it's by Robin Hanson, who is one of the examples from the article.)
It's a very persistent idea floating around, despite contradicting everything we know about how a brain operates.
More on topic, what he misses is that "consciousness" is a null word. It does not connect to any real phenomenon, and merely a result of a human being unable to imagine it's own non-existence, thus producing the long list of things like anima, spirit soul, consciousness and so on.
The Ship of Theseus is an ancient recognition of a similar problem.
Greg Egan has written two stories that deal with copy-vs-continuity of consciousness: Diaspora and Permutation City. Both start with the premise "what if we could digitize our mind?" but go in different directions.
Surely in this scenario the answer to "is it OK to kill one of you now?" is "no".
So why is this different to some people than a synthetically-created clone?
I don't know about that.
I'm not sure about this intuition pump. If they are truly identical to me, then the world they live in must also me identical to mine or we would have diverged. So if we discovered Earth-2, then Earth-2 would simultaneously discover us (or Earth-3 etc). No actions can cause divergence, so it's not possible for anyone on any Earth to only kill one of you. It would be like trying to kill your reflection.
If we're talking about some kind of god who is outside the mirror/chain Earth system, and has the ability to break symmetry as they see fit, it's still not clear-cut. In effect, from such a perspective, there is only one Earth (albeit with copies) until changes are made. It's not even clear that deleting one of the several identical copies of the whole Earth is wrong. Moreover, deciding to kill someone on one of several heretofore identical Earth's and seeing how they diverge is informationally the same as spawning a new Earth - an act of creation rather than destruction. Nothing is lost - that person still lives their life on an Earth somewhere - but now new lives will be led, new experiences, new sadness but also somewhere new joy.
I think it's pretty clear though that our morality of "no killing" is based pretty strongly upon the assumption that there's only one of someone, they can only be killed once, and it's forever. Start violating these assumptions and it breaks down quickly.
(IMHO)
http://memory-alpha.wikia.com/wiki/Thomas_Riker
William Thomas "Tom" Riker was a result of a transporter accident in 2361 that created two William Thomas Rikers, genetically indistinguishable from each other, with personality and memories identical up to the point of the duplication. One of the duplicates continued to be known as William Riker. The other chose to use his middle name and be known as Thomas Riker.
Thomas and William clashed almost immediately due to the resentment each felt towards the other. The eight years of living different lives made them entirely different men – William evolved into a cautious and duty-driven officer while Thomas remained impulsive and reckless. As a result of conflicts with William – who was by that time a superior officer (a commander) – Lieutenant Riker decided to leave the Enterprise. He chose to use his middle name, Thomas, to distinguish himself from his counterpart. Captain Jean-Luc Picard managed to get Thomas a posting on the USS Gandhi where he could continue his Starfleet career. Upon Thomas' departure, William seemed to have accepted Thomas' existence and gave him his trombone as a parting gift. (TNG: "Second Chances")
Oh. Well, assuming our current model of computation, I think a "substrate-independent" human program is essentially non-human by definition (that is to say, lacking the particular qualities of 'consciousness' that we commonly ascribe to humans), so I'd conclude that "nothing is lost" insomuch as "nothing existed" in the first place. In theory, it seems plausible that we could create a program that could consistently pass a "Bill Gates" Turing test, but it is easy for a programmer to imagine how such a program could exist without actually being conscious.
There are a lot of 'memories' stored in the body, and the brain and body (and endocrine system) are inseparably intertwined in my opinion.
[1] https://en.wikipedia.org/wiki/Embodied_cognition
Reminds me of the Ship of Theseus but with the human body/brain.
http://www.jaronlanier.com/aichapter.html