Ask HN: Do you think machine consciousness is possible?

71 points by zoba ↗ HN
(Read the last bit to skip the story)

I'm planning on going to grad school to study AI because I think it is very interesting. For a very long time it has just seemed natural to me that computer scientists would eventually discover a way to make computers appear as intelligent as humans. Since it wasn't done yet, I wanted to work on this problem. I had no thoughts of solving it, but perhaps help it along.

However, I recently had the scary idea that machine consciousness may not be possible. I've thought this before, however this time it really hit me and scared me some. Considering I'd like to devote much of my resources to the problem, I'm now a little concerned that it may all be a waste. I'd prefer not to waste my life on something that turns out like the phlogiston theory.

Therefore, because it may bring good discussion and for my own benefit I'm asking:

Do you think machine consciousness (or at least something that looks like it) is possible? If not on current computer architecture, which "new lead" in computation do you think will allow it?

For extra credit: Do you think the Church-Turing thesis (anything that is computable is computable by a Turing machine) indicates that machine consciousness is possible?

188 comments

[ 1.4 ms ] story [ 176 ms ] thread
There have been many threads recently about this exact theme, http://www.google.com/search?q=site%3Anews.ycombinator.com+a... should turn up some results.

My personal take on this is that I'm really not sure.

There are some pretty clever people here that are sure it is possible, 20 years or less.

There are others (of which I'm one) that think it may be possible but either devilishly hard compared to what has been achieved to date or beyond our abilities, figure at least 20 years, probably much more, if ever.

And then there are those that think that it is impossible.

I'm an absolute nobody when it comes to stuff like this but it interests me greatly. When I was 15 or so I envisioned a world about 2 decades away where computers could be taught. We're 30 years down the line from that point and we're still programming computers more or less the same as back then.

But that does not mean that things can't change overnight, and who knows, maybe you're just the guy for the ticket and you will be the one to crack this nut.

AI will be achieved -- over a period of decades and by brute force. At the end, I can't say whether you'll have machine consciousness or not, but you'll have something that is indistinguishable from it. And that's good enough.

What we call consciousness is probably very closely tied with having a physical body perceiving things it the outside world. So I think for a long time there will be differences between machine and man, but machines will eventually win out and become vastly superior to people. I just wouldn't count on it in your lifetime.

The really interesting question is: if we can quantify consciousness, what happens when we create something that's more aware and conscious than we are? Would we be considered sentient by a being that thinks a thousand times faster and in hundreds of thought-trains, lives for a million years, and can converse millions of ways simultaneously at bandwidths millions of times greater than speech?

We would be like insects to something like that, and it's not such a far-fetched idea or that far off.

Perhaps, but (to the best of our apparent knowledge) the difference between "doggy brain" and "human brain" isn't really that the "human brain" is orders of magnitude faster at thinking "doggy thoughts"; it appears to be at least as much a qualitative difference in what it does as in the amount it can do in a given unit time...and the moreso if you start comparing, say, "gecko brain" to "human brain".

Agree with your general thrust but it's the qualitative change that's the more interesting, as it's perhaps unknowable (in the way that your dog Fido will never understand most of your thoughts, no matter how patiently you explain them).

Doubt it, but then I'm not holding my breath for a personal helicopter either.
I've never heard a good reason why it wouldn't be possible. But even supposing that it isn't, AI research is not wasted. It is clear that products related to AI (natural language processing, computer vision, game opponents, etc) can be exceedingly useful even though they're not sufficient to produce a conscious AI. So don't worry!

My personal belief is that the reason progress in AI has been so slow is that AI optimists vastly underestimated the computing power needed to produce good AI, and when our computers are finally fast enough (20-30 years perhaps, and the most important metric is probably not FLOPS but memory bandwidth) AI will actually not be that hard to achieve.

No one knows what human consciousness really is or if it even applies to other animals.

Having said that, consciousness is not a requirement for intelligence. It would interesting, and plausible enough for self-improving AI as smart as a human to be developed without addressing the question of consciousness.

For extra credit: No there is not enough evidence.

Here is my opinion on this matter: It is possible to simulate the universe, but it is not possible to be the universe.

Simulation is different from being, just as predicting the weather is different from manipulating the weather. And simulating consciousness on a computer is different from being conscious.

Consciousness isn't a unique body in our universe, it's a state. (so far as we can guess)
I am dead certain that it would NOT be possible to achieve consciousness in a computer program.

On the other hand, I know if I ever had to debate this question with Daniel Dennett or Eliezer Yudkowsky or any other capable person who believes there's nothing special about "consciousness" I would lose the debate.

Attributing a special meaning to consciousness, on the surface, is just as illogical as believing in religion- Neither seems to have a valid defense, as far as I can tell. Therefore, I find it disquieting that I DON'T believe in religion but DO feel so certain that computers can't be conscious.

Once you figure out the reason for this apparent inconsistency in my belief system, please let me know :)

OK here's an argument for why you could be right.

Whatever it is that distinguishes consciousness for what computers can possible do is also the thing that makes you realise that computers can't be conscious. If you could logically write the reason down to 'prove it' then it would also be something that you could implement in a computer. So the reason, whatever it is, has lie somewhat outside of specifiable logic.

Just in case this is somehow original. I'm designating this 'thing' as a 'dejb' and calling the whole thing "dejb's theory/proposition/whatever". Although it actually probably just a restatement of Godel's incompleteness theorem. Also I actually believe that computers can be concsious.

I like dejb's theory! I'll have to give it some thought.
Searle is an atheistic (I think - at least he's nonreligious) philosopher who agrees with you. He argued against "functionalism" - which is pretty much the stance that most here are taking - that consciousness is a complex adaptive process, nothing "magical".

However, he's just playing with your intuition, the same way you are, and his logic (though not his certainty, either, from the bit I've read) is flawed. You should read up on his "Chinese room" thought experiment yo learn why.

Searle is an atheistic (I think - at least he's nonreligious) philosopher who agrees with you. He argued against "functionalism" - which is pretty much the stance that most here are taking - that consciousness is a complex adaptive process, nothing "magical".

Now that someone brought up the S-word, I can't help but paste in one of my favorite Searle quotes (http://sss.stanford.edu/others/johnsearle/), which to me very clearly indicates how much of a mental and logical struggle it is (and how much flubbery is required!) to attempt to justify his position:

'Could a machine think?' My own view is that only a machine could think, and indeed only very special kinds of machines, namely brains and machines that had the same causal powers as brains. And that is the main reason strong AI has had little to tell us about thinking, since it has nothing to tell us about machines. By its own definition, it is about programs, and programs are not machines. Whatever else intentionality is, it is a biological phenomenon, and it is as likely to be as causally dependent on the specific biochemistry of its origins as lactation, photosynthesis, or any other biological phenomena. No one would suppose that we could produce milk and sugar by running a computer simulation of the formal sequences in lactation and photosynthesis, but where the mind is concerned many people are willing to believe in such a miracle because of a deep and abiding dualism: the mind they suppose is a matter of formal processes and is independent of quite specific material causes in the way that milk and sugar are not.

Even if the guy's a freaking idiot (which he is), you've gotta appreciate the way he tries to turn the "dualist" label against the people that are most opposed to dualism, while that's almost exactly what he's arguing for. He would have made a great politician; unfortunately, he's made his career arguing about something that he doesn't really understand, which is a damn shame, I hate to see so much mental energy spent fighting a lost battle...

I tend to agree with you, though not with so much certainty. I think the question of consciousness is fundamentally unanswerable, and I think a lot of people like to use that fact to construct arguments about how consciousness and simulation of consciousness are the same thing because they are scientifically indistinguishable. However just because a falsifiable theory can not be constructed does not mean there isn't an objective truth.

This need for all beliefs to have a scientific basis is a bit pathological in my opinion. You shouldn't feel bad for believing something "illogical" if the only reason it is illogical is Occam's Razor or scientific intractability.

My reasoning for this is that it should be obvious to everyone that our world and experiences are increasingly shaped by ideas in modern times. Even though physics can explain the mechanics of the universe in great detail, the way ideas affect our individual experiences and society as a whole has very little to do with their ability to be objectively classified as true or false. By forcing a scientific perspective you are limiting yourself from the benefits of philosophies and ideas that are beyond the realm of science.

I'm not saying that a belief about artificial consciousness is beneficial one way or the other, but just that you shouldn't be chided into dismissing your own opinion just because it does not rest on firm logical conclusions based on established scientific evidence. The other side is quite wobbly as well.

I agree with you in that consciousness does really see ill defined. But I don't think that would stop a computer program from achieving consciousness.

Even with consciousness being so ill defined I'd say that there are a few points that we can agree on. For one, even if peoples definitions and expectations of consciousness differ, we can all find examples of life that we agree are conscious.

The boundary may be really fuzzy. We might disagree whether animals are conscious or brain-dead/comatose are conscious. But we could probably find a case that we agreed on.

And once we assume that I would ask you whether a computer program would ever be able to pass the Turing test and convince us that it was conscious. If a computer program would be able to pass I'd argue that it was conscious, since consciousness seems to me to be totally subjectively defined.

If you say that a program would not be able to pass, I might agree with you intuitively, but I would question it intellectually. Because I've always believed that there are cases that are definitely perceived as conscious by other people. I would be shocked if someone truly believed that the person next to me is somehow not conscious. So the next question I would ask is whether consciousness resides and arises from something that can be explained scientifically. Or to put it simply: Can we describe,now or in the future, the workings of the brain? If so, I really don't think we would need a scientific explanation for any epiphenomenon such as consciousness, all we would need to is recreate those initial conditions and we should be able to mimic consciousness.

If you say that we won't be able to mimic the workings of the human brain I'd simply say that you are making the statement that there is physical phenomena that can never ever be explained by science.

If you say that even if we recreate a human brain, we will never mimic consciousness, then I'd reply that you are saying that my personal definition of consciousness not affected and defined by other peoples definition of consciousness.

I think that might be why you see an inconsistency between attributing a special meaning to consciousness and and religion. We instinctively act like the human race can find common ground on the definition of consciousness. We don't instinctively act like we can find common ground on religious beliefs.

If we assume that humans are conscious, then yes, it is possible for a machine to be conscious. The only arguments categorically differentiating humans from electronic machines are religious in nature.

I define consciousness as having an internal model of the world that includes yourself, as well as your own though processes (at a lower degree of fidelity). This says how you compute things, not what you are computing, so it is orthogonal to Church-Turing.

Whether it is achieved will depend on economic forces. I don't see much economic value in making conscious computers, or things which seem to be down that line. So I expect consciousness will come out of pure research (perhaps within a corporation, like IBM), well after computers have exceeded the raw processing power needed.

Because they will be so different from humans, it will have to demonstrate a significantly higher degree of consciousness than a human needs to in order for most people to be comfortable with the term.

Wouldn't you agree that it feels awfully "weird" to be a living, breathing human? Doesn't it seem to be beyond what can be expressed by mere computer algorithms?

I realize it is not a defensible argument to say that the human experience is just to "weird" to be computable- I wish I had a better argument for defending my position...

Nonetheless, I find it hard to understand people, such as yourself, that are probably exposed to this same "weirdness" in their heads as I am and yet are so confident it is merely an illusion created by a sufficiently complicated computation.

Weird compared to what? I have no experience being anything other than a living, breathing human. Thomas Nagle wrote up the classic "qualia" argument you're making as "What is it like to be a Bat?" (here: http://www.clarku.edu/students/philosophyclub/docs/nagel.pdf..., it seems persuasive: that there is a certain subjective quality to consciousness & conscious experience— it must feel like something to be a bat, or a person, or whatever? Hofstadter's and Dennett's book, The Mind's I (http://www.amazon.com/Minds-Fantasies-Reflections-Self-Soul/...) is an interesting introduction to this kind of stuff, the authors have a lot of fun @ qualia's expense.
The "weirdness" (or whatever you want to call the enormous explanatory gap between our mental lives and inanimate matter) deserves an answer. It's not something to sweep under the rug as "subjective" or "unscientific" or "poorly defined".

I agree, though, Hofstadter and Dennett have done an extraordinary job of devising such an answer. There is lots more work to be done, of course.

The "weirdness" (or whatever you want to call the enormous explanatory gap between our mental lives and inanimate matter) deserves an answer. It's not something to sweep under the rug as "subjective" or "unscientific" or "poorly defined".

But what if the answer to the "weirdness" question is, in fact, that it's just an ill-formed question? What if it's just an illusion, and your brain is tricked into seeing something magical about itself where there is nothing there?

Apart from a mystical explanation, I cannot imagine any satisfying answer to the question (i.e. one that doesn't leave you feeling uneasy like Hofstadter's answer does to most people), which usually means that there's something wrong with what we're asking, not with how we're trying to answer it.

I would be satisfied with a good explanation of why the question is ill-formed, how the illusion comes about, etc.

I really don't think the question is ill-formed, though. There is a big explanatory gap. Denying that is simply dishonest. It would be like, if you don't know why a bicycle is easier to balance when it's moving, pretending that there is nothing to explain. "Oh, it's just bicycle parts."

"Wouldn't you agree that it feels awfully "weird" to be a living, breathing human?"

From my perspective, being a computer would feel a lot weirder.

I think drastically spoken, feelings are just firing neurons. I don't think there is really anything special about (apart from the degree of complexity of the human body). If a computer has an algorithm that says

if(user hasn't typed anything for 14 days) lonely = true

then it has feelings, too. It might sound absurd, but only because it looks so simple. But imagine an enourmously complex program, and the information "lonely" trickles through it. There might be a routine somewhere

if lonely = true connect_irc_channel(#depression)

and so on and so on. From a certain level of complexity onwards, it won't be so obvious anymore, and we won't be able to prevent feeling that the computer really feels.

Wouldn't you agree that it feels awfully "weird" to be a living, breathing human?

No - it's just normal. I think it would be weirder if it wasn't some kind of computation, if I was a unique spirit out of several tens of billions that appeared from some unknowable place and origin, or whatever else 'i' might be.

Doesn't it seem to be beyond what can be expressed by mere computer algorithms?

I have no reason to think it is - I haven't felt what it's like to be a quicksort or a face recognition algorithm for comparison, but I certainly feel like a heap of evolved feedback loops at times - when I fear the dark in a room I know, when I catch sight of living shapes where there are none, when I feel fight-or-flight at certain caller ID numbers, when I desire things that I also don't want, when I feel judgements of people based on some trivial detail.

Nonetheless, I find it hard to understand people, such as yourself, that are probably exposed to this same "weirdness" in their heads as I am and yet are so confident it is merely an illusion created by a sufficiently complicated computation.

What else could it be? There isn't anywhere else it could reasonably be that we know of. You're either suggesting something unknowable (e.g. magic) or some kind of cruel joke (i.e. like a quine is a program that prints it's own source code, we could be a consciousness that sees the world except for the blind spot around the part that allows us to see how we work - and a cover over where that would be).

I'm not confident that we are 'just' a computation, but I'm fairly confident that I am in accordance with the laws of physics (including any we don't know yet) and that places limits on what's possible with the amount of matter in my head, the energy input and output, the sensory input and output bandwidths, the timescales involved, the known behavioural results of varying localities of brain damage, etc.

Besides, what do you mean illusion? Consciousness is not 'fake'.

There is one nonreligious factor that significantly differentiates humans from machines: that humans evolved, while machines were built by humans. There's an argument to be made that humans cannot understand consciousness well enough to build it, because consciousness is our only tool for doing that.

Of course, this only applies to intentional creation of consciousness. It says nothing about the distinct possibility that we could create a consciousness by accident.

But we know how to get the effects of evolution... on steroids: unsupervised, feedback/effect based learning (generic algorithms, NNs, etc.)

We can already generate big enough NNs that we can understand them in general (some areas get "specialised" in some way and more active in some conditions), and we know that the result is what we expect - but there's no way someone will take a look at the weights / resulting model and tell you what it does. I think we can create consciousness intentionally, but not understanding the process completely.

Evolution does not have a goal in mind.
That doesn't mean it can't be used to achieve a goal.

Selectively breeding cows for increased milk production allows evolution to find a way or ways of making that happen. The breeder doesn't need to worry about hormones or glands. Evolution doesn't need to worry about the breeder's motives, why are only the high milk producers producing offspring?

No, but it is driven by external pressures, which we could apply to artificial consciousness by hacking it :) Ok, artificial selection - but no reason we couldn't game it to be genetic.
A computer-based consciousness would also be a result of evolution (albeit in a different way than things we know usually) being the byproduct of something that an evolved species created to help itself adapt.

Sometimes, when I'm getting overlogical, I see machine intelligence as being pretty much our destiny (i.e. that singularity thing) and that organic life may be very much obsolete once it gets into its own.

As organic life was to the universe before it, machine life will be to us.

Evolution is not directed by a consciousness, so you cannot loosely use the term "evolution" to describe the DEVELOPMENT of computer-based consciousness.

ADDED: but yes, extended phenotype and all that jazz.

Just because biological evolution was not directed by intelligence does NOT mean you can't use evolution to describe a process of gradual improvement based on intelligent improvements rather than random changes and natural selection. Both are evolution, just in different domains, using different methods. (Your complaint sounds sort of like saying reading on a computer shouldn't be called reading because there is no actual printing involved in producing it.)
humans evolved, while machines were built by humans.

The design of machines evolves all the time.

You are assuming that there is no innate quality of human organic compounds and processes that differentiates us from electrical components.

It may very well be the case that this is either true or false. We simply don't have enough evidence.

And given the fact that we are discovering new properties of matter and organic reactions all the time, there is a bias towards this being false.

Bah....I call bullshit. This is pure anthropomorphism. Humans think they are the shit, but in fact they are only story-telling animals (which does give us an evolutionary advantage, incidentally. We are not limited in our information transfer inter-generationally by genes alone.) We are limited by the same physics as the chips we make. This innate quality you speak of is pure vapor. Even if humans were somehow able to become mentats, we'd still be limited by the tenants of information theory and what is computable. The fact that human intelligence is emergent leads me to believe that machine intelligence will be the same, albeit very different than a simian mammal's intelligence. Fish are smarter than we are at swimming. Think ants.
humans are unique in the universe among all life forms and inanimate objects, are more than the sum of their physical parts, have a connection with something larger than the universe, and though in an insignifcant corner of an insignificant galaxy have an eternal significance.
See, there goes that story-telling and thinking we're "the shit" again. Thanks for providing a concrete example.
Humans have a connection with something larger than the universe? What does that even mean?

From the biological point of view, the simplest species is AS SUCCESSFUL as the most complex species, since both are still alive and procreating.

Why do humans have eternal significance? To whom?

There's no way to prove we're unique in the universe. At most you could say the known universe, but even then you're going to have a hard time convincing people of that.
At most you could say the known universe

Known to who? I don't like this very self-referencing way of thinking. Very similar to the discovery of America... people were already in America way before Columbus came and "discovered" it. (although apparently he didn't even realize he hadn't arrived to India)

My peeps, it is not necessary to give negative karma for someone expressing an honest and inoffensive opinion you don't agree with.
How about an opinion that's so muddled that it's not even wrong?
It's akin to having a group of people sketching on a large piece of paper, trying to build on each other's marks to create an accurate representation of a scene, and someone comes in a scribbles all over it saying "but I see scribbles! All pencil marks are valid! Don't be so limited!".

He/She's allowed to have such an opinion, but this discussion is trying for a particular feel and that isn't contributing helpfully to it.

How are we unique? Do you know there is no other life in the universe similar to us? Can you prove there is no other life in the universe similar(or the same) as us? What do we have a connection to? God? Am I less significant because I don't feel this connection, because I don't feel humans have eternal significance?
@nwatson - That is a religious belief. It may not be a conventional religious belief, but it is a religious belief. Not to dismiss all religions (although, as an atheist, I tend to), but more to note that the GGP had pointed out that he assumed no religious belief.
Bah....I call bullshit. This is pure anthropomorphism. Humans think they are the shit, but in fact they are only story-telling animals

The poster you are responding did not claim that humans are the shit. They merely pointed out the fact that we have so far proven incapable of proving that we aren`t said shit. Now, you could argue that such a hypothesis could only be falsified by the construction of a machine consciousness. That, however, is orthogonal to the fact that it remains possible that there is in fact some bizarre quality of the universe or the human race that makes machine consciousness impossible. Not a terribly scientific position to take, perhaps, but a perfectly sound philosophical one.

yeah perhaps I drew him into this one, but "innate quality of human organic compounds" seemed like a bit of human elitism to me. To which I say "hey buddy, just because we are (apparently) the most dominant species on the planet (which I also am not too certain I agree with), doesn't mean we are the end all be all, or worse, somehow different than all the animalia we happen to try to place ourselves "above" or something. Trust me, when humans wear out their welcome here (which seems to be coming with great alacrity with regard to cosmic timescales), the insects will be more than happy to eat our corpses and continue on happily without us. We are adaptable, but not THE MOST adaptable organism on the planet. And regarding the philosophical position business, I also call bollocks, as there was not one rational argument presented to back his premise regarding the "innate quality of organic human compounds". Is the human neuron somehow magically different than a chimp's neuron? Methinks not, i.e. it is particularly unsound to somehow make our molecules different than any other organisms molecules simply because we are human.

No, human intelligence is an emergent quality, and I fathom that even our massive representations of humanity's information (akin to what Google is compiling) will soon begin to exhibit interesting qualities of its own once it becomes complex enough to exhibit perhaps interesting, unanticipated emergent qualities (in fact, if it didn't I would be absolutely shocked). Many strange and unpredictable (or at the very least unanticipated) things arise from even the simplest of "complex systems" (Conway's game of life and some of Wolfram's automata), much less the wonderful systems detailed by our individual neural mappings and our individual genomes. (q.v. http://en.wikipedia.org/wiki/Emergent_behavior)

People always think that some human will "write" an AI like HAL or the like, but it is much more likely that nature will roll its own AI once we have made a comfy enough nest for it to germinate. After all, isn't that how we got here? (from an evolutionary biologists standpoint anyhow...)

And again, sorry for being contentious. I just despise the "religious" argument (even if there is no "named religion" being expounded... Let's keep Ockham's Razor at the ready here....)

There is a difference between being the most dominant species on the planet and being composed of living cells, resulting in breathing, aging, dieing, and reproducing. An organic thing should not be looked at in the same light as something that is not.
Why do you assume that our artificial intelligence will not be organic? Look to the rise of wetware, my friend.
Then we'll make machines out of "human organic compounds" to exploit those processes.

We're machines. That we may not have made a machine of the same class ourselves is an implementation detail. Maybe a large one, but still not really an argument against making conscious intelligent machines. Biology can do it, sooner or later so can we.

We are biological. So if we do it, it is really just a phenotypical expression of our genes.

Perhaps the disagreement is about when do machines (keeping in mind they're mechanical by definition) cross over to becoming more and more like biological entities (replication). And once they do, is it still coherent to think of them as machines as we use the term today. Or would we then think machines have been elevated to the biological level.

This is certainly a good point. At least I somewhen crossed the line when I just started to see things as input-output-devices with more or less complicated algorithms in the middle.

The human has a hideous complicated algorithm in there, involving a live-long history, internal feedback and reflection and arbitrary side constraints.

A spider for example is much simpler. 'If the net shakes, walk where it shakes and eat.' (Certainly, + a batch of regulations to be able to walk and sense, but the point stands).

However, imo at a certain point, a computer program passes the complexity of, say, a spider. Just consider modern compilers. These things are of baffling complexity and do things inside no human can imagine in details :) Or, imagine data mining software, or even just very complicated, security aware network guards. All of these softwares are very, very complicated in their input-output-behaviour, and even though they are not as complicated as a human, they certainly can compete with a spider, at least for me.

And exactly this view is what caused some pretty nasty discussions for me, since some people are just not crossing the line of 'everything is an input-output device of different forms' and they stand hard on the ground that machines and animals are different, because they are machines and animals (and some go ridiculous ways, 'god made animals, humans made machines', and whatsoever, not even some 'but the spider might be more complicated than FOO, because, which whould be a nice discussion :) ).

So, overall, I, as a person who is working hard on being a tolerant, non-racist person (which is really hard), don't see a reason to exclude the possibility of machines and robots being conscious, just because 'they are electrical and not organic' (which has a ring of "he is black, he CANNOT do science" to me. Sorry if I just offended a lot of people).

Are you saying it will be possible someday to kill a human being, disassemble all its parts, reassemble them and bring that person back to life?
Are you saying its not possible? I don't know if I agree with Freaky, but I learned a long time ago to be hesitant saying anything is impossible.
Er.. no, I'm saying if there's something magically special about our organic composition required to make a conscious being, then we can build machines out of the same organic compounds. If you want to get those compounds by disassembling people, well.. I guess waste not want not, but it isn't really what I had in mind ;)

"Consciousness is not computable" just means we need to build a new class of machine. Whether it's a blob of organic jelly in a box, or an IC with currently unknown structures on it to exploit certain handwavy quantum processes, or whatever; it just makes the task more complex, it doesn't make it impossible.

Considering we've already simulated the basic building blocks of our brain and understand all the basic molecules involved, I'd say that we already have a very strong indication that we can eventually have a machine conciseness similar to our own.

The only two physical process that we know of that are fundamentally impossible to compute with our computing model are quantum computations and chaotic systems. And for chaotic systems we most certainly can simulate them in a way that the output has all the correct properties as far as we know. It's more that we can't reliably do prediction in such systems due to finite precision.

The simple fact is that computers are already our superiors on many tasks. And those tasks are not simple either.

You are assuming that there is no innate quality of human organic compounds and processes that differentiates us from electrical components.

It may very well be the case that this is either true or false. We simply don't have enough evidence.

It's true, we can't prove it one way or the other at the moment, but that usually just means that Occam's razor should guide our speculations and assumptions. Since the brain seems like all it's doing is performing some computations, why would we ever assume that its true function is to do something more than that?

And given the fact that we are discovering new properties of matter and organic reactions all the time, there is a bias towards this being false.

There's a good chance that the way the brain achieves its computations is indeed a bit more complex than, for instance, an artificial neural network, absolutely. But that's an implementation detail, and there's certainly no reason to assume that there's no computational model that can account for it - biologists are very close to having full working models of individual neurons already. Further, it's highly likely that the brain does its job in a biologically convenient way, not a logically convenient one, and I'd give good odds that there are a lot of logical simplifications that could be made to end up with a cleaner architecture that performs the exact same tasks.

>I define consciousness as having an internal model of the world that includes yourself, as well as your own though processes (at a lower degree of fidelity). This says how you compute things, not what you are computing, so it is orthogonal to Church-Turing.

Do you have any proof or any testing that can be done to validate that?

He's defining a term. He doesn't need proof - though you're welcome to dispute his definition.
Thats what I am doing. Of course I did not use the right words, but I assume my intent was clear.
I guess the question we're really asking is this: is the human brain a Universal Turing Machine? Everything we've learned so far points to yes (our study of neural networks has revealed a possible mathematical structure which is Turing Complete), therefore my vote is with the 'yay's.

P.S. I am an atheist, therefore I do not believe there is anything metaphysical about the human brain. Maybe there is something special about our mushy carbon structure -- doesn't matter. In that case, we'll just build our artificial brain with wetware. Hardware is hardware is hardware (and physics is physics). Eventually we'll get an artificial neural net as complex as our biological one. The real question is software...

Aren't humans also differentiated from machines in that the latter are designed, while humans are not designed?

Machines are just an expression of humans' extended phenotypical effects on their environment.

Is that actually a quantifiable distinction? Does how something is constructed have any affect on it's properties? (assuming equivalent precision of the tools involved)

Building a car by hand versus building one with a modern robotic assembly line both yield the same output.

Construction implies design.

Evolved entities (read: complex entities) are not designed.

Ok, but how is a constructed object and a ... (randomly? naturally? what ever word you want to use) evolved object quantitatively different?

One could follow all the steps involved in getting a human from his evolutionary ancestor to his current state by using a pair of magic tweezers to make each mutation in the genome happen at the right time and in the right way. Could you tell the difference based on the result?

Yes, you can easily tell the difference between entities that have evolved by natural selection, and constructed entities.

(BTW, mutation is random, but evolution is not a random process).

Two example quantitative differences (although, I'm not sure why the specific exclusion of qualitative differences):

1) The human eye, for example, has flaws (that are solved in other similarly evolved eyes) that an engineer laughs at - if you were building a human eye, you wouldn't make the same "mistakes". The deviation from the "better" version is measurable.

2) Evolved entities exist because of procreation. And their only reason for existence is to assist genetic material to replicate. Constructed objects reproduce exactly 0 times and have 0 genetic (or other replicator material).

The human eye, for example, has flaws (that are solved in other similarly evolved eyes) that an engineer laughs at

What are those flaws, if you happen to know? I'm curious.

An example of something that joubert is referring to is the fact that photoreceptors (rods and cones) in the human eye are actually located behind the retinal ganglial cells, nerve fibers, and capillaries. So light has to pass through a layer of tissue before being detected. One consequence of this is the blind spot.

An engineer would probably try a different ordering.

In his 1986 book, The Blind Watchmaker, Dawkins comments:

Any engineer would naturally assume that the photocells would point towards the light, with their wires leading backwards towards the brain. He would laugh at any suggestion that the photocells might point away, from the light, with their wires departing on the side nearest the light.

Yet this is exactly what happens in all vertebrate retinas. Each photocell is, in effect, wired in backwards, with its wire sticking out on the side nearest the light. The wire has to travel over the surface of the retina to a point where it dives through a hole in the retina (the so-called "blind spot") to join the optic nerve.

This means that the light, instead of being granted an unrestricted passage to the photocells, has to pass through a forest of connecting wires, presumably suffering at least some attenuation and distortion (actually, probably not much but, still, it is the principle of the thing that would offend any tidy-minded engineer). I don't know the exact explanation for this strange state of affairs. The relevant period of evolution is so long ago.

You often come across odd stuff like this in machinery. The usual reason is that it made fabrication (or access for repair) easier. Engineering is all about tradeoffs, the "right" design is just one of them.
That's really fascinating--thanks for elaborating.

That said, this isn't, for me, a strong refutation of design. Perhaps the designer's purpose was to create creatures with imperfections such as these, with the higher purpose of communicating something deeper.

Or, is there any way of knowing that we won't some day discover there is a very good reason for this?

I think we are arguing at cross purposes here. I'm not arguing what we call evolution doesn't lead to machines very different from the ones that we might chose to engineer ourselves. Rather, I am arguing that, as far as I can tell, I could, given enough time, money and energy, construct a living being using methods very different from natural selection and get the same result. In other words, I don't see why there is anything special about the path taken (some paths may require less energy though. ;-) )

Therefore, I don't consider designing vs evolving as a good way to separate machines capable of consciousness from machines not capable of consciousness.

There is something very special about the path taken (although this phrase is a little misleading) in evolution - not from a design perspective, but from a result perspective - evolution by natural selection is neither a random process, nor a goal-directed design.

Organisms (and by extension, or by reason) are alive because their ancestors have been lucky enough to survive long enough to have offspring.

Note, I'm not arguing that machines cannot be capable of consciousness.

But I am saying that constructed machines are necessarily distinguishable from organisms primarily because they're designed with a goal in mind (unlike organisms - these aren't designed but are the result of the co-operation of genes into higher levels of complexity, team work that has happened to be useful to the survival of the replicating matter - DNA, RNA, or possibly other such material elsewhere in the universe).

I would posit that because of the evolved nature of organisms, there may be flaws in their consciousness (e.g. through chemical imbalance, irrational deduction etc.). It is more likely that machines that are designed for intelligence & consciousness would be gifted (or cursed, depending on your convictions) with perfection, rationality, normalcy, as attributes, or at least their designers would attempt that.

So the the nature of their consciousness would be qualitatively different from that of organisms.

The more I think and learn about evolution as something abstract (IE something that can exist in principle only, happens to be embodied in biology and could be embodied elsewhere), the more I think that evolution is creation. At the very least it is so similar to "creativity" that any discussion on the topic would probably quickly degrade to boring semantic argument.

From this I think there are two potentially interesting products:

- Evolution by natural selection may be usable as an engine of machine consciousness. Some variant of evolution may be at the core of our own consciousness.

- Evolution by natural selection is a mechanical process that we can "take apart" and understand relatively easily. It also happens to be the engine of the process that created species. Happens to be. Even if the evolution of species had never happened, evolution would still exist in abstract. This put it in very good position to be discovered. Perhaps there are equally powerful concepts waiting to be discovered. Perhaps one of them is at the core of our own creativity. Perhaps one of them can be at the core of machine consciousness.

Eliezer made what I think was a similar point in posts at Overcoming Bias and Less Wrong; he also pointed out that evolution is really stupid. If we can understand the result of an evolved process, we should, using intelligence, be able to do much better. The key is understanding.
I'm not sure I agree. For several reasons:

a) The phenomena of qualia remains unexplained. This has no bearing on making something that can mimic consciousness, but does become significant if one wants the real thing. The last argument on this: http://news.ycombinator.com/item?id=766462

b) QM may be intrinsically linked to consciousness. (see Roger Penrose, _The Emperor's New Mind_) Of course, this a mute point given the QM computing seems to be around the corner.

The fact that qualia are an inherently internal experience means that there can be no way to communicate them to anyone else. We are willing to accept that the other minds problem isn't a problem because we believe they are similar to us. This is why I say that machine intelligence would have to display a very high level of consciousness for us to accept it as conscious. Humans get a pass.

I discount Penrose's argument because he offers no phenomena that actually require his QM effects to explain and offers no plausible evolutionary path by which human minds could have evolved the mechanisms relying on QM. Do other primates have QM effects in their brains? What do these explain, given that the current models of the brain that we have are capable of explaining everything that we see? (Though we don't yet have the practical capability to simulate an entire complex brain and suitable io.)

I know virtually nothing about CS, which may be an advantage in seeing the sense in your thinking in abstract.

If we assume that humans are conscious & material, without bothering to define either, we at least know it is possible to have consciousness embodied in something material. At least in theory we could figure out how human consciousness works and replicate it. There may be easier ways, but this is at least one theoretical possibility.

You could in the same way have inferred that humans could figure out a way of creating flight from the fact that it exists in nature.

What if as Julian Jaynes says consciousness is a social creation rather than a biological one? Machine social interaction would be tremendously different and could be incapable of sustaining consciousness.
Why couldn't the machine partake in the social construct? It could be programmed in.
About those economic forces - I'm betting on games to get there first. With Creatures and Sims there have been already two examples that the public is absolutely keen on getting software that features lifelike behaviour. And the wide distribution of games also means that games can get access to a lot of distributed processing power beating even supercomputers. Also game AI programmers are probably the one with the most practical experience because they have actually to produce stuff that works.
That brings up interesting ethical issues -- could you play a FPS game knowing the bad guys were 'conscious'?
I play Multiplayer FPS even though I know my opponents are conscious. Doesn't prevent me from shooting them because I know I only kill their avatar and not themselves. Same with AI - I shoot at an avatar I won't destroy it's code.

The ethical issues will rather be the moment when AI's are clever enough that they can be teached. What will humans teach them? Personally I believe the best protection from abuse will be to get that learning process in the public (as compared to AI's learning from companies or military for specific purposes). So my own long term target are distributed virtual worlds in which AI's improve by getting passed around. A single computer user might still teach them bad stuff, but I hope some sort of selection based on many people watching and exchanging bots will get the best possible results even though completely preventing abuse won't be possible.

Code is not the avatar - it is the knowledge and memories (probably written in form of a big blob of neural net coefficients) that makes a concious being. Each time the game is restarted there would be new beings in it (unless their data is saved between games somewhere).

Erasing knowledge and memories of such AI would be analog to killing, and it will become problem, because throwing out data is easier, and writing you enemies to DVD after each game of Quake 10 becouse of moral issues will be problematic after some time.

But we can't even successfully predict how proteins will fold (only about 70% are predicted correctly) and we presume we know all about chemistry and physics. Yet what we model, and what we observe, are quite different.

If true machine consciousness is possible, it's a lot further off than we would like to think.

which "new lead" in computation do you think will allow it?

I don't know if hardware is holding back progress in AI, especially with distributed computing; it is more likely a question of software and programming. I think in order for a computer to exhibit a reasonable facsimile of human intelligence, it will have to do more than just run programs written by humans. It would have to have the ability to write and rewrite its own code.

Are humans at a place where we can write and rewrite our own DNA?
The computer equivalent of human DNA would have more to do with hardware than code. When I talk about computers rewriting code, I'm talking more about rewriting patterns of thought processing. While humans can't rewrite DNA on the fly, we can definitely teach ourselves new ways of approaching problems.
Hardware is holding back progress in AI. Well, I can only speak for game AI, but given that game AI even has to work with simpler worlds and can fake many things I suppose it's not much different in other AI areas. And game AI programming is a lot about programming against the limited available resources. You mention for example the ability to rewrite own code, but what you think about might be more about unsupervised learning in general. I occasionally did experiments in that area - and so far always stopped them after a while due to lack of processing power. I don't say that there isn't also a need to improve the software side, but missing processing power was at least so far the thing that was always blocking me more.

Just to give you one example. Think of bots in shooters. Humans have no problem browsing the whole screen in realtime and evaluating the data. But try even just doing the simplest image processing in realtime with a typical resolution and you will already find out how hard that is. So bots can't do that. Instead they cheat and shoot a few single visibility lines and even those are already targeted because bots can know where opponents are and so they only do for example linechecks to certain areas like head, body, arm, legs and try to find out if that line is blocked. And even that might already be too slow with many opponents so you have reduce it further and shoot only a certain number of lines each frame. And there is no processing yet - this is only about getting the simple information of - is there something or not. This is how slow computers still are. You could certainly do a little more with giving a single bot the whole processing power of your computer, but with the increasing complexity of game worlds the needed processing power grows rather fast (current addition of simple solid-body physics in games makes AI programming already way harder). For AI tasks computers are still very slow.

I wouldn't bet on machine consciousness happening in your lifetime. But consciousness is a cool enough thing that solving 0.001% of the problem is useful too. Machine vision, collaborative filtering, machine learning, all of these are attacking a tiny subpart of the consciousness problem, but they're still useful.

Don't worry that AI will turn out like phlogiston. The journey will yield its own rewards, and plenty of partial success will also be extremely valuable.

(comment deleted)
Let's address the easy part:

Do you think the Church-Turing thesis (anything that is computable is computable by a Turing machine) indicates that machine consciousness is possible?

Only if you assume that consciousness is a computation, which is assuming everything.

I normally try to resist this topic, but what you're saying here tugs at my heart-strings:

I'm now a little concerned that it may all be a waste. I'd prefer not to waste my life on something that turns out like the phlogiston theory.

Consider the total failure of algorithmic approaches to even begin, even pathetically, to replicate anything recognizable as consciousness. Were the people working on it dumber than you?

Try to find some way to control for the geek fantasy factor, in yourself and others, before deciding what to do.

(By the way, now I'm curious: what was it that led you to "the scary idea that machine consciousness may not be possible"?)

I've thought of it several times, but it wasn't until recently that it actually kind of scared me. It was just a surprise thought that arose as I was once again thinking about the topic.

It probably scared me this time and not others because I'm very stressed over the GRE, and grad school applications (namely: where the heck should I apply), and how me telling grad schools "Oh, I'd like to study machine consciousness" will go over. I could be wrong, but I'm concerned they wont take me seriously...so I've been trying to think of something that sounds more acceptable.

My advice, if you are considering a PhD, is to pick your school based on your potential research advisor more than the department or the school in general. Contact him or her before you apply and tell them your plans of study and research interests and then go from there.
We have no definition of consciousness, so it's impossible to say whether machine consciousness is possible, or whether we have it already.
I think the definition of consciousness is implied as whatever we humans are experiencing right now. So for a computer to be truly realizing its own existence, it would have to be modeled after ourselves.
The problem is you have to believe what the computer tells you when it says it has consciousness.

(In the same way that I have to believe you when you say you have consciousness).

Then what's the point? Why would we want to make a machine modeled after humans, which are not particularly well adapted for doing any of the things that we want done in the modern world, when a machine designed to perform practical tasks (like a computer) would be much better?

We shouldn't assume that we humans are the best at everything. We shouldn't assume that we humans are the best at anything, really.

Some philosopher said this sometime back regarding Penrose's theory: (I can't find the original so might be a bit different) "Quantum Computation in microtubules is as explanatory as magical pixie dust in the brain"

We can substitute pretty much anything for the first noun phrase: " X is as explanatory as magical pixie dust in the brain "

You may perhaps be confusing "definition" with "understanding".

My dictionary defines consciousness thus:

consciousness |ˈkän ch əsnəs| noun the state of being awake and aware of one's surroundings : she failed to regain consciousness and died two days later. • the awareness or perception of something by a person : her acute consciousness of Mike's presence. • the fact of awareness by the mind of itself and the world : consciousness emerges from the operations of the brain.

====

We may not know HOW consciousness comes about, but we do have a definition of what consciousness is.

awake(p): not in a state of sleep; completely conscious; "lay awake thinking about his new job"; "still not fully awake"

And now we're in an infinite loop.

rabidsnail was probably talking about a "formal definition" not a short description in a dictionary. The key to artificial intelligence is to develop a mathematical model of intelligence/consciousness. Something like "An intelligent being is a 5-tuple ..." With this model we can calculate intelligent responses/development and a Turing machine can calculate. Therefore one can implement an AI on a computer.

If there is no such model or we can't find it for some reason we won't develop an AI.

Does one really need a mathematical model in order to claim one has a "definition". Isn't a mathematical model's role to "understand" something (and hence predict)?
I think it is possible that some area's are going about the problem wrong way. With current hardware attempting to simulate the brain as it works I don't think is the viable option.

I seem to have the problem also that we are using the thing we are trying to simulate to work out how to simulate it. From our perspective the problem could be almost impossible.

I look at consciousness fairly oddly:

I believe consciousness is an emergent property of a complex system; it is simply the nature of the universe that complex systems exhibit consciousness. I'm defining "complex system" as any system whose outputs/results affect the inputs/possible states. If I think something, the possibility space for my next thought is dependent upon my previous thought, and so on.

I think artificial consciousness research will progress, through computer simulation, to the point where a "real" consciousness emerges from a sufficiently complex simulation. The hard part will be mapping its inputs/outputs into human-compatible form; success will probably occur by accident at first. But when we can do this we'll be able to talk to a totally simulated consciousness through the prism of it being another "person".

At this point more research/thought will be put into the nature of consciousness itself, and how to connect with other-than-human consciousnesses. We'll use the experience of bridging communication with artificial consciousness to successfully communicate with naturally-occuring consciousnesses associated with other complex systems (the earth, a tree, the galaxy, etc). It sounds a little insane, but I totally think this is within the realm of possible in our lifetimes.

Of course, that's all based on the notion that consciousness is an emergent property of a complex system, and not something entirely unique or bestowed by higher powers or whatever.

I agree that simulation is the key to a concious AI system. If or when we ever succeed in simulating a human mind to a close enough degree of comparability, it is almost a given that the system will be self concious.

There are some problems simulating the human brain that would also have to be addressed even once we can create a working system, such as the AI being a bit of a blank slate, like a infant or a coma patient.

I see the whole process as having to follow a path similar to this:

1. A breakthrough in computing power, something capable of simulating very accurately small areas of space, this means perfectly representing ridiculously complicated chemical reactions and some natural laws.

2. Succeeding in creating a software environment to execute these simulations within.

3. A breakthrough in mapping an existing person, some sort of scan that creates a mathematically provable perfect (or close enough) representation of an area in space. Like some humans mind or possibly their entire body until the subject of the scan can be simplified on the computer. Sort of like taking a photo and then cropping off the body. The above simulation environment may be what is used to provide the simulated inputs and outputs to the head, like the CNS and cardiovascular system. Not to mention the inputs to the eyes and other senses. Sort of like a virtual head in a jar.

4. So far we would have a conscious system, but it would be a copy of a pre-existing being. The next step would to be to somehow, ethically, re-write this being. This would provide a learning challenge with the goal of simplifying and modularising the human brain. Such as hacking language areas, input nerves, the reliance on virtual blood and sustenance and most importantly the memory.

The final product of this important stage is the most simple and easily tweak-able simulation of the human brain that could be used by all researchers and eventually commercial applications. If all these virtual brains are the same or comparable, this isolates the memory as a way to load in or edit what is essentially... people. The creepiest analogy may be the best, they will be like swappable save game files, executing in virtual machines (the hacked brains) that operate within another virtual machine (dare I say it, a super-simple matrix of sorts)

5. We may never reach anywhere near this far along the process due mainly to ethical reasons that cannot be overcome with mere ingenuity. But if we do, the next step is compressing all this down further and further until we have the most simple possible (perhaps provable somehow) implementation of a mind that does not require all the layers of virtualization.

God it's easy to get caught up in this stuff. I hold this prediction on my fingertips in hopes that any developments may blow it away so I can re-evaluate and make a new one.

Aren't you also (perhaps only almost) defining complex systems as those that have consciousness? I.e. your argument for the one is dependent on the other and vice versa?
hence the robots who think for themselves in science fiction. and the backwardsly un-emergent Alzheimers when circuits blow fun to think about
I agree with this. I think consciousness "happens" when you cross a certain complexity threshold. For argument's sake let's say this happens around 100 billion interconnections. Cross this threshold and consciouness "bursts" into being.

What is consciousness? Well, for me a big part of it is the "movie screen" of my vision. Consciousness is sort of like this movie I can watch, with the understanding that the hands that go into the field of vision are my own- "I" can control them and my body's position in the field. If I want the field to change, I can make the movie look somewhere else. My consciousness also includes the full mapping of the body, including its pain and pleasure states, temperature, hunger, etc etc.

Language is a large part of consciousness, in particular, English. Integrating language into my consciousness was pretty catastrophic- it was like formatting my conscsiouness with a new files system. Apparently it erased all memories previous to that because I can't remember anything earlier than that. English both enables and limits the scope of what I can think. It has a lot of ambiguities and is only moderately good at feelings, and emotions. But hey, it has tenses which vaguely relate to this sensation of passing time that is relevant to my consciousness and a significant portion of reality can successfully be communicated to me by any other consciousness running the Englihs module. If I wanted to be more precise I suppose I could convery my consciousness to LOGLAN or some other language built on predicate calculus rather than germanic and latin grammar base.

In addition to all this, my consciousness stores a lot of memories, skills, ideas, and memes. I have models for basic mathematical concepts, as well as skill sets such as how to operate a motor vehicle. I also have prejudices and preferences. My consciousness is limited by this- if I were not familiar with certain philosophical memes, such as "social contracts", my consciousness would not be able to understand advantages of a national healthcare plan.

Emotions are also part of my consciousness. There is happiness, a "feeling" of well-being when dopmanine, electricity, magnetism, radio waves, and more are flowing rather well. I am likely to be happy when the majority of my body and psyche's needs are met and I am pleasantly engaged and using enough, but not too much, of my energy bandwidth. Anger is a territorial response I would feel in my consciousness when I feel my phsyical or psychological territories are being threatened- it is a territory defense response. Sadness is a tension I feel when there is cognitive dissonance between an idealized vision I have of how things should be. (Tears of sadness if things are worse than I feel they should be, happiness if they are better.) Sadness may also be felt due to various social rejections as my organic ape-mind has a great deal of hardware devoted to social interactions. Pain is experienced when my nerves detect entropy is increasing in my wetware.

As for communicating with other sentiences, this seems inevitable as more species approach and cross Threshold. If humans are at 100 billion+ (or whatever Threshold requires), there are many speces which must be very close. Some of the more advanced marine mammals such as dolphins and whales are apparently very close, let's say 98 billion interconnections, and they even have some of the advanced cerebrocortical structures we think of as giving us our higher civilized thought structures. Elephants, chimps and other greater apes, and maybe dogs, cats, pigs, and some birds seem to be climbing above the 70 billion mark.

If we could figure out how to artificially enhance brains with more interconnections, both human and animal, we could probably "boost" several near-sentient animals across the line by nano-creating more neural interconnections. Energy would flow across the new connections and they would "burst" into their own version of movie-screen of consciousness. It would be fun having other sentiences to talk to. I think we as humans are fairly l...

It's not all or nothing: even if true AI would not be achieved in your lifetime, there would still be loads of useful things with the stuff you learn.

As for the question, I am sure that it is possible (except I take issue with the word "consciousness" - what is it supposed to mean?).

In short: yes.

I think it's painfully clear that sometime in the coming years humanity will reach the pinnacle of its scientific achievement with the advent of an artificially intelligent machine: one that is able to think and reason and is self-aware. Technology is moving at such a rapid pace and in the right direction that this is just the next logical step.

In order for this to occur, however, significant advances must be made in fields outside of technology; e.g. quantum computing will probably be a huge stepping stone, and for that to come to fruition we must first fully prove quantum mechanics.

"In order for this to occur, however, significant advances must be made in fields outside of technology; e.g. quantum computing will probably be a huge stepping stone, and for that to come to fruition we must first fully prove quantum mechanics."

I recently heard about the term "yak shaving". This seems to be a good example.

A point of clarification should be made here... Are you speaking of machine consciousness behaving similar to our own?

Human's consciousness is but one form, in my opinion. We are a complicated biological machine seeking the fulfillment of our existence; how is that any different than a machine seeking the fulfillment of its existence?

I believe the money is in self-evolving circuits and programming; allowing the machine to mold what defines its existence based on external parameters - and overtime, based on internal parameters (will to change).

Try this thought experiment. Imagine there was a replicator that could take an exact copy of your current atom states and replicate them somewhere else while keeping you perfectly intact. Doesn't have to be physical - could be replicated by a computer program.

Now here's the thing - you would still feel like you, looking out of your eyes. The other versions of you would be distinctly separate to you - not you.

That bit that makes you feel you are you - that's what some would call consciousness and others would call a soul.

Imagine your replicate is put in a world without oxygen. It would very fast feel rather different from you despite being identical. You assume consciousness is an inside state which can be copied. But it could as well be a process in constant need of an external influence which might refuse to connect to your replica. Like the difference between standing and flowing water - watched only in one instant they might seem identical. And while the solution would certainly be to copy the world as well, as long as we can't define consciousness we can't really say for sure how much we would have to replicate.
Where did you go to school? It sounds very much like you have a symbolic background. There are many approaches and people are attacking them from different angles. Here are just a couple of resources worth looking at off the top of my head:

Christof Koch at CalTech (hi, virgil) http://www.klab.caltech.edu/~koch/

Larry Yaeger & John Beggs http://www.indiana.edu/~rcapub/v30n2/mindmade.shtml

Of course, Douglas Hofstadter http://www.cogsci.indiana.edu/

Koch, Churchland, etc. speak on consciousness: http://mitworld.mit.edu/video/342

I am at NC State (graduating in May), and my research adviser does have a symbolic background (I'm pretty sure)...very good read on your part!

Thanks very much for the links. I am stressing over where to apply, hopefully these will help some.

(comment deleted)
Hey guys, maybe human consciousness is nothing else but an "illusion" provided by the matching of information between what we perceive through our senses and what we have stored in our brains? Of course, that "matching" process is the big problem to crack. Could we hypothesize that when babies are born they aren't aware because they don't have a 'minimum threshold' of information in their brains that is required to enable the 'match' being produced by their input senses? Or from the other spectrum, say, advanced patients with Alzheimer disease lose their self-awareness because their memories are destroyed and the 'match' needed to trigger the 'illusion' of consciousness is disrupted ?
As a meta-comment, notice that threads like these usually result in many people posting lengthy first-level comments, and not doing much replying and discussing. (Compared to other HN threads, of course.)
what about when the brain is powered by machines? Does that count as a computer being self-aware?
Sounds like you should be going to the Singularity Summit to talk to other people who devote their lives to this issue.

Nick Bostrom's Simulation Argument details the most obvious probabilistic implications of substrate independence in consciousness: http://www.simulation-argument.com/

The most blatantly obvious indicator that consciousness is substrate-independent: We are DNA-based life forms. DNA stores information. It's program code stored in molecules. You are the product of the code of your parents. If for some bizarre reason we find out that we HAVE to use DNA to create other conscious systems, we will still have the ability to do exactly that. Not "machine" in the sense of being composed of metal, but certainly "machine" in the sense of not being the immediate product of natural selection.

David Chalmers' work should be particularly relevant to you, and you will find him at the Singularity Summit this year. http://en.wikipedia.org/wiki/David_Chalmers

Even if you don't live to see machine consciousness as a reality, the only other pursuits that might compare are anti-aging and intelligence enhancement research. If you're not going to create something that can figure out how to give you an indefinite self-contiguous narrative, you have to support its creation or face certain death.

I'm guessing you already have your CS undergrad or will have it soon, and you're interested in AI, so that seems the natural choice. I'd say you're overdecided if that's what you want to study.

Thanks for pointing me to this. I had heard of the Singularity University, but not the summit.

Hopefully they have videos posted online of the event...right now two months rent is not available to spend on a conference, very unfortunately.

Consciousness is a feature of our external world/ Universe. We should formulate theories and test it experimentally. We should describe it in terms of basic components. Unfortunately, this is not how AI has been treating consciousness. AI treats consciousness as an art/engineering problem rather than rigorous science. But, there is hope : Journal of Experimental and Theoretical AI http://www.tandf.co.uk/journals/tf/0952813X.html

Current theories say consciousness is caused by a specific mode of representation/ computation. Like good scientists and rational thinkers we should submit everything to rigorous testing/ proof. Let us not please take things at face value.

[Or we can take the easy route and live in a bubble :) ]

Of course it's possible, everything you can imagine is possible, given enough time and resources. But is it probable?

But I think the question you should be asking yourself, if you're wondering how to spend your life, isn't if machine consciousness is probable, possible, whatever. You should be asking, what do I want to do with MY consciousness? If you want to work on these problems, then do it. Doesn't matter if it happens in your lifetime or not. If you enjoy it then give it a try. Worst case is you spend your time doing what you enjoy. Best case is... well, machine consciousness I suppose.

Besides, you should always be thinking big. The person I admire the most has a small metal plaque that sits on her desk. It says "what would you attempt if you knew you couldn't fail". I happen to think this is the most powerful sentiment I've ever encountered.