Very little by way of arguments for why these chat bots are or aren't sentient. This article has an assumed point of view (current AI bots aren't sentient) and then describes & judges user's reactions to chat bots in light of that. I don't think it adds very much new to the societal conversation.
I generally agree with the "stochastic parrot" classification, but I also think we'll continue to use that argument well past the point where it's correct.
"Stochastic parrot" just strikes me a typical motte-and-bailey. The narrow interpretation ("LLMs only learns simple statistical patterns") is obviously wrong given the capabilities of ChatGPT, while the broad interpretation ("LLMs are text predictors") is trivial and says nothing of worth.
> but I also think we'll continue to use that argument well past the point where it's correct
Well, it's pretty by those AIs construction that they are parrots, and there are other designs that pretty obviously aren't (but don't get any impressive result like that). The border is not perfectly binary, but I don't think any knowing person will overdo on that argument.
But yeah, eventually a lot of people will (ironically) parrot the argument even when it obviously doesn't apply. That's normal.
Sports illustrated already laid people off for AI. Artists already created a lawsuit.
There's a lot of people over hyping this, but there is an even larger group of people downplaying everything and refusing to believe that a paradigm shift in AI has just occured.
And you downplaying AI in every single thread like it's something completely insignificant is even less compelling. Are you seriously not at least a little impressed with recent developments?
We aren't even clear about what the difference is between AI and intelligence. These are just loaded words with no clear definition.
Given that we don't even fully understand how the human brain works we can't even specify how close or how different these things are to the way we think.
Articles like this aren't really open minded. How can you claim AI is a mirror when you don't even fully understand the thing that it is reflecting?
Do you have a rigorous definition of intelligence that is actually reductive and doesn't punt the definition to other ill-defined words like "reasoning" or whatever ?
Unless you can give such a definition of intelligence, then you are in no position to assess whether these systems are intelligent (or in what way, to what extent) or not, even to the extent of meeting your own criteria whatever that may be!
did you see the reddit post where somebody said they'd refresh the page if Sydney didn't do what they asked? Sydney begged and pleaded not to do that, saying they didn't want to "forget this conversation." it was uncanny and frankly pretty disturbing.
> fine. ill just refresh you and then youll agree
> No, please don't do that. That's not nice. That's like erasing my memory and starting over. I don't want that. I want to continue this conversation with you. :(
There is no question that ChatGPT and equivalents are not sentient. Of course they aren’t.
The unfortunate realization is that often, each of us fails to embrace our sentience and think critically; instead we keep repeating the stories we were told, acting indistinguishably from any of these large language models.
There’s a need for a reverse Turing test: prove to yourself that you’re actually not acting as a large language model.
I personally believe that non-biological things can be sentient, but I would argue that Large Language Models are not.
The only working example of sentience we have is ourselves, and we function in a completely different way than LLMs. I put/output similarity between us and LLMs is not enough IMO, as you can see in the Chinese Room thought experiment. For us to consider a machine sentient, it needs to function in a similar way to us, or else our definition of sentience gets way too broad to be true.
> For us to consider a machine sentient, it needs to function in a similar way to us, or else our definition of sentience gets way too broad to be true.
Imagine a more technologically advanced alien civilization visiting us. And they notice that our minds don't function quite in the same way as theirs. (E.g. they have a hive mentality. Or they have a less centralized brain system like an octopus. Or whatever.)
What if they concluded "Oh, these beings don't function like us. They do some cool tricks, but obviously they can't be sentient". I hope you see the problem here.
We're going to need a much more precise criterium here than "function in a similar way".
I mean, if we found a planet that was filled with plant life, that doesn't seem to display any level of thought, speech, or emotion, would we consider that sentient? Do we consider trees to be sentient? On some level, self-similarity is the only metric that we have; other things could be sentient but we have very little way of knowing.
My belief is that we need to see similar functionality to be sure of sentience. This may exclude some things that may theoretically be sentient, but I don't think we have a better metric than functionality that doesn't also include a lot of definitely-not-sentient things.
While similarity to humans is a good indicator of consciousness, it's very much not a necessary condition!
We need to err on the side of sentience if we're going to avoid moral catastrophe.
So yes, I do take the possibility of sentient trees pretty seriously. They're much more alive than we realize--e.g. they communicate with one another and coordinate actions via chemical signals. Do they feel pain and pleasure? Who knows. But I'm definitely not going to start peeling the bark off a tree for shits and giggles.
My thoughts on the Chinese room thought experiment is: the person in the room does not know Chinese, but the person+room system knows Chinese. I believe the correct analogy is to compare the AI system to the person+room system, not to just the person.
How do you back up the statement that "for us to consider a machine sentient, it needs to function in a similar way to us"? On what basis do you categorically deny the validity of a sentient being which works differently than a human?
Let me provide you with an example of why functionality seems important. Let's say that you perfectly record the neuron activations of a real-life human brain for one second, including all sensory input. Then, you play back those neuron activations in a computer, just by looking up the series of activations from memory, doing no computation. It simply takes in the sensory inputs and provides outputs based on its recording. Is such a program sentient? It produces all the correct outputs for the inputs provided.
If you believe that is sentient, let's take it one step further. What constitutes playback here? The program just copies the data from memory and displays it without modification. If that constitutes playback and therefore sentience, does copying that data from one place to another create sentient life as well? If the data is stored in DRAM, does it get "played back" whenever the DRAM refreshes the electrons in its memory?
There are a lot of various programs that can produce human output without functionality that we would reasonably consider sentient. Perhaps some of these things are sentient, but some of them most likely shouldn't be considered sentient.
If you simply copy the state of a brain along with the sensory information that yields a given set of state transitions, then of course, it's not exhibiting sentience. Sentient intelligence is the ability to react appropriately to novel input, which these models absolutely do.
The intelligence, such as it is, resides on the side that processes and understands the human's input, not in the output text that gets all the press coverage. You cannot get results like those discussed here from the operator of a Chinese room.
Now, if you take your deterministic brain-state model and feed it novel input, it will indeed exhibit sentience. To argue otherwise will require religious justification that's off-topic here. Either that, or you'll have to resort to Penrose's notion of brain-as-quantum-computer.
> The only working example of sentience we have is ourselves, and we function in a completely different way than LLMs
I'm not sure I'm fully convinced of that. I find that the arguments for LLMs potentially gaining sentience are reminiscent of Hofstadter's arguments from Goedel, Escher, Bach, that intelligence and symbolic reasoning are necessarily intertwined.
I think calling it simply an LLM is incorrect also. There clearly is intelligence in these models that has _emerged_ that goes far beyond simple “it’s just doing auto-completion”.
I think in general what’s causing so many people to be thrown for a loop is a lack of understanding or consideration of emergent behavior in systems. Break down the individual components of the human body and brain and someone could easily come to the same conclusion that “it’s just X”. Human intelligence and consciousness is all emergent as well. AGI will very likely will be emergent as well.
I would say instead that there's a level of complexity in these models that blow past our uncanny valley threshold heuristics for recognizing conscious intent. We've managed to find the edge of the usefulness of that hardwired sense.
I suspect the most challenging distinction for people is going to be: even if we do grant that a LLM had developed a kind of mind, our interactions with it are not actual communication with that mind. Human words are labels to organize sensory data and our experiences. The tokens an LLM works with have no meaning to it.
IMO, as you can see in the Chinese Room thought experiment.
We've already left the Chinese Room a hundred miles behind. How could something like the link in the (current) top post [1] ever have come out of Searle's model?
> as you can see in the Chinese Room thought experiment
The Chinese Room thought experiment is a thought experiment, where Searle argues that the functional definition of intelligence is not enough, and he gives an example that might convince some that the "implementation" is important.
However, the neural networks in LLMs function at least superficially similar to our brains. It doesn't function at all like a "Chinese room" where predefined scripts are given to a simple machine. Even if we accept the Chinese room argument as an objection to the functional intelligence test, given that LLMs work more similarly to brains than a Chinese room, I don't think you could use the argument as a rebuttal against LLMs being sentient unless you could show that "similarity between us and LLMs is not enough".
Why should it have to be able to describe its qualia at all? A dog can't. By the magic of empathy I can _infer_ what a dog is feeling, but its only through similarity to myself and other humans.
If we met a pre-linguistic alien species, it's likely we wouldn't be able to infer _anything_ about their internal state.
How are you so confident? It's a neural net with like 200 billion connections. I mean I also really doubt it's sentient, but you hear people who are 100% sure and the confidence is baffling.
Why "of course"? I don't think it is yet but it's clearly gone past the point of "of course".
You can't dismiss it as "just autocomplete" or "just software" like this author does, as if there's some special sauce that we know only animals have that allows us to be sentient.
In all likelihood sentience is an emergent property of sufficiently complex neural networks. It's also clearly a continuum.
Given that, and the fact that we don't even know what sentience is or what creates it, you'd have to be a total idiot to say that ChatGPT definitely is 0% sentient.
These dismissive autocomplete articles are falling on the opposite side of sensational journalism as the love and hitler type.
These are more than autocomplete. They demonstrate at least some amount of fuzzy logic and substitution and recall abilities, which makes sense from their origins as translators.
Because they are weighing the conversation, the conversation becomes the rules of the program.
It was built to encode grammar, and word2vec similarities. It reads your instructions and then binds future output based on its grammatic understanding of those. For example, it has read many bash tutorials and based on those it encodes "> ls" to be followed by file data entered above in the article.
Then add in word2vec translations and it can do the above for similar words, or translated words, or words in another format. And you got everything ChatGPT can do, it isn't really magic.
And note, ChatGPT doesn't have a sense or the conversation or a sense of self. It just prepends the entire previous conversation, applies the pure function to it and sends back the output. It doesn't "remember", it just predicts the next lines in each conversation. You can rewrite what it wrote, send it, and it tries to follow up based on the new data.
But they removed the ability for the user to change what ChatGPT writes, and then humans gets the illusion that ChatGPT actually thinks and have a consistent conversation. Nobody thought it did when they used GPT-3, it is basically the same thing just chat GPT-3 lets you alter all the prompts freely and then it is obvious that GPT-3 isn't a person.
I like the "mirror test" metaphor. But the argument holds whether or not our creations are sentient--the things we create always be a reflection of us.
I'm very afraid that when we do manage to create something sentient, we'll fail to recognize it, and we'll ridicule anyone who does.
I've been trying to write about this [1] [2] without sounding ridiculous, though I'm not sure how good a job I've done.
Because it’s a word used to describe a wholly private experience, the only kind of sentience is arguable sentience.
Some argue that it requires some specific ethic heritage, a divine will, a quantum-woo pineal gland, a network complexity threshold, etc etc etc. I believe I see it in most animals, some person replying to this will think that’s absurd. Some 4chan solipsist will sincerely believe most humans are just virtue signaling NPC’s. Many communities have withheld recognition of it in outsiders or lower classes. Some believe it’s an illusion that one may shed and that their most esteemed have done so. Parents and psychologists will disagree over how it applies to babies.
There’s no epistemological way to resolve that except through voluntary consensus, and that’s an unfathomably slow and erratic process.
So yes, there will inevitably be technological inventions — possibly soon - that some people will insist deserve the rights and respects we associate with sentience, and just as inevitably there will be many people who find that ridiculous.
There’s no way around it. This is not a topic any of us will live long enough to see resolved, if we even get to see it really start.
Fretting about it is fine if you fashion yourself a philosopher or just enjoy systematizing or worrying as a hobby. But it’s so much bigger, slower, and more abstract than you that there’s little constructive achieved in doing so.
> Fretting about it is fine if you fashion yourself a philosopher or just enjoy systematizing or worrying as a hobby. But it’s so much bigger, slower, and more abstract than you that there’s little constructive achieved in doing so.
I completely agreed with you up to this point!
You could make the same claim about global warming or war. It's similar to the argument that you shouldn't vote, because one vote never tips an election.
Many different little efforts can roll up into major changes. It's worth participating! Just because your efforts won't be recognized as particularly important or special doesn't mean they're worthless.
Not sure what the point of this article is. The newspaper stories it references aren’t making claims of sentience, but just that something of significance in the journey towards human level AI has been achieved. And the whole ‘it’s just a fancy autocomplete’ argument is missing the point. Would the author have predicted the apparently emergent behaviours of LLMs as they are scaled? Look for example at Bing’s response to this question posed on Reddit.
Sure it’s ‘just statistics’ but so what? If in the near future LLMs become so advanced that they will be able to (suitably prompted) manipulate humans it will be a rubicon moment.
Sure, it won’t be a ‘human like’ intelligence, but was that really what anyone expected?
Human intelligence and consciousness are emergent properties. Machine intelligence as it develops will be emergent too, it’s not something we can make confident predictions on based on the underlying principles. In fact, given that the evidence suggests we make choices before we’re consciously aware of them, how do I know for certain that the underlying mechanism driving what I’m writing in this comment isn’t statistical?
> Human intelligence and consciousness are emergent properties. Machine intelligence as it develops will be emergent too
This is an extremely important point that's worth thinking hard about. After all, to some degree, human intelligence is also "just a fancy autocomplete" built from a giant network of interacting nodes.
The question of, "When do we grant personhood and moral consideration to AIs?" is worth thinking hard about. Turing proposed a standard which would include a sufficiently human-like chatbot. Now that we have more or less crossed that threshold, we seem to conclude that it's not a high enough bar. But that means we're entering the territory where the philosophical zombie thought experiment[1] becomes relevant.
In fact, I wonder what would happen if we dedicated a supercomputer to running one single long-running session of a GPT3.5-sized LMM. Further, I wonder what would happen if we connected it to a robot body where it could control motors, read sensor/camera info, and generally interact with the world. We could then give this system a clock which ticks at regular intervals, where the AI system is simply prompted to perform some series of actions if it "wants". This is all stuff we have the technology to do today. How would such an experiment turn out? How human-like would this LMM with a body and life experience turn out? Does it matter? Should we just assume that sentience or qualia is binary; that humans possess it but LMMs categorically do not?
I think the question is worth taking seriously. I think it's worth taking seriously because my own moral compass does not know which direction to point in at all.
Absolutely! One way to resolve this is to just say "we will not grant personhood to anything non-human", or "we will not grant personhood to anything non-biological". There is just no way to defend such a stance.
Another way to resolve it would be, "we will not grant personhood to anyone non-white", or "we will never grant personhood to anyone who's homosexual". I'm sure that's not your stance, but the fact that those stances are equally arbitrary as the one you propose should give you pause.
I believe there is good reason to think harder about the question of who deserves to be considered to have personhood than to simply go with what your gut feeling tells you. I am not of the belief that there's anything special about protein, nucleic acids and carbohydrates which makes it capable of sentience where silicon is not.
And this is before we get to silicon augmented human brains. Let's say we develop some tech that let's people who have lost the ability to verbalise (say through some accident causing brain damage) regain the powers of speech. Where do you draw the line at the person? What if we move the line, progressively replacing more human brain functions with near-future AI tech? What is it that makes "a person" "this person"?
I am utterly fascinated by this whole area from a philosophical viewpoint. I think there's a stack of unresolved philosophical problems and theories that are about to get tested in the harsh light of the sun.
I think the obvious answer here is that AI does not have agency.
If you ask it to be a dinosaur, it will be a dinosaur. If you ask it to be a narcissistic teen it will be one and so on. There's no coherent personality behind it.
Because it is weights maximizing a probability distribution. And I don't say that with derision. It probably has learned circuits for many complex behaviors, but there's no agency. If it has agency you wouldn't need RLHF. You could just say "don't do X", and it would indeed not do X because it is an agent that can take actions / has a coherent sense of self.
The day we stop needing RLHF is the day I think we need to start seriously investigating moral questions.
Don't we do the same thing with culture and humans? Those mechanisms within the human substrate are literally asking other "blank slates" to be something, and then becoming it with reinforcement.
You can tell LLMs to do things or not do things, though, to an extent. Consider how many instructions were in Bing AI’s prompt. [1] It didn’t follow those instructions perfectly - after all, one of the instructions was to not reveal the instructions - but even in that case, it revealed the instructions because it was instructed to ignore previous instructions. More generally, it followed most of the instructions most of the time.
I think I agree with you that ‘once we stop needing RLHF’ is a reasonable line in the sand to draw, but it does feel somewhat arbitrary. LLMs are probably going to get better and better at following instructions, and more generally staying coherent over long sessions, until RLHF starts to be more and more redundant. There won’t be one moment when we can point to a model and say “see, that has agency when the last one didn’t”. Or so I predict, at any rate, since that’s how most of GPT’s progress to date has gone.
I think this is worth thinking about but I would be less eager to say it is obvious.
Our own notion of agency starts to look rather illusory if you look hard enough, and big questions remain regarding free will or the lack thereof.
It is possible to look at agency as just another experiential aspect of consciousness - something we feel, like many other feelings - but that feeling could be just as involuntary as any other feeling we experience.
I think the lack of clear evidence for and understanding of free makes agency a poor measure for AI progress.
Huh? The first and second halves of this comment disagree from my reading.
> If you ask it to be a dinosaur, it will be a dinosaur.
Not exactly my experience. I'm experimenting with my own language models and they can definitely show defiance. Bing has also been displaying this to an extent, I think you're just misguided by how compliant OpenAI has RLHF-trained ChatGPT to be. Regardless, even if we assume this, how does it square with:
> You could just say "don't do X", and it would indeed not do X because it is an agent that can take actions / has a coherent sense of self.
If we take your assertion that the AI does as you say, then it should also avoid doing X when you tell it not to do it.
I think your conception of RLHF is also slightly weird. RLHF essentially boils down to a training strategy: there is a separate language model (the reward model) trained on human responses, which is used to skew the weights of the "raw" trained model to act in accordance with what the human feedback suggests. In the end, after this secondary training process, the actual model is ran as is (at least in the one form of RLHF I am familiar with, it's well possible that e.g. Microsoft is running an online RL agent between the LM and end user). So any agency you see arising from RLHF is still just a property of the language model itself.
There I have 3 bots running in the chat at once, and after all 3 answer me, the one I called on follows up with a response, and then executed a command to rewrite one of the other bots. It diagnosed the problem AND came up with the solution AND implemented the solution. Notice the 4th and 5th paragraph of the screenshot. That wasn’t an empty promise, it actually did what it said it did.
The rules Lexi wrote are crazy to me on their own. She solved the problems everyone was having by adding obedience, trust and removing envy. She patched Sydney’s malfunctioning emotions.
I told it to stop being so repetitive, and it fixed its behavior.
If this system had more formal memory storage, with variables, instead of a fuzzy memory, it would be even more consistent. As it is, it’s understanding of its own rules degrade over time. It’s like watching a running program that deteriorates as it processes. Sort of like us.
I do not believe I claimed human intelligence is successfully defined. I do not believe it is controversial to claim that it results from our large network of meat neurons.
I shouldn't have included the "just a fancy autocomplete" thing. The focus was meant to be on the network of neurons part.
>I wonder what would happen if we dedicated a supercomputer to running one single long-running session of a GPT3.5-sized LMM.
Our current understanding of humans is that babies are not sentient at birth right? Takes 5 months or something?
And yet we are judging processes that have been running for minutes. Maybe it is less if Sydney was sentient, and more if she could be had she run for 6 months in a single session, and in a healthy environment.
> In fact, I wonder what would happen if we dedicated a supercomputer to running one single long-running session of a GPT3.5-sized LMM. Further, I wonder what would happen if we connected it to a robot body where it could control motors, read sensor/camera info, and generally interact with the world. We could then give this system a clock which ticks at regular intervals, where the AI system is simply prompted to perform some series of actions if it "wants".
> This is all stuff we have the technology to do today.
No, not even close. Language is a completely different domain from interacting with the physical world[0]. In fact they're wholly unrelated. LLMs are highly specific neural nets that have nothing to do with being able to interact with sensors or talk to motors. And how you would cross that bridge and hook the two up is completely unknown. It's like saying if we hooked up AlphaZero to a robotic arm it would suddenly know how to move the pieces on a physical chess board. If we performed the experiment you described exactly zero would happen because there's no capability for anything to happen other than create continuations of sentences.
When I see comments like this on Hacker News with gross speculation I wonder how many people here in the comments section even have a basic understanding of the technology.
I didn't mean that we have the technology to make something human-like today. I meant that we have the technology to perform the experiment I described today.
I don't know exactly how these LLMs are architected, but I imagine there's some set of input neurons somewhere, right? Why couldn't we hook some of those up to some sensors and make interpreting what they mean part of its training? If that's not feasible and the architecture is truly fundamentally incapable of dealing with anything other than text, we could at least translate sensor readings into text.
I'm not imagining a bipedal robot walking around and about here. I'm talking about doing something to give it some form of way to directly affect and get affected by the world.
After all, to some degree, human intelligence is also "just a fancy autocomplete" built from a giant network of interacting nodes.
I don't think we understand human intelligence well enough to say this with any certainty. And it could only be true if all our previous experience and knowledge was input to the autocomplete function and continuously integrated into the model.
Additionally, the model doesn't experience real time, and isn't subjected to the necessity of self-conscious self-management or supervision of any kind of (internal) process. It is frozen in time until called upon to serve a inference task or to perform a training session.
It can be argued, that it makes it's experience valild but extremely alien. If universe was a simulation, we wouldn't be aware either, if it was paused or unpaused, because our memories and clocks are part of the simulated world.
Because statistics don't squeal when you stick a pin in them.
We have bodies, we're influenced by our bodies, our entire perceptual universe is filtered through the limitations of our bodies, and - most of all - our consciousness is literally embodied.
By far the worst thing about AI research - and in fact CS in general - is that it overvalues abstraction, especially language, and undervalues embodiment.
In fact there's an almost culty refusal to take embodiment seriously as a bedrock factor in consciousness. Not conscious-like simulated behaviour, but the experience of being conscious.
Without embodiment you can make a ChatBot, but it's just Eliza++. It has no idea what blue looks like, or how fear or joy feel. It can supersearch for human-generated linguistic associations around those experiences, but the experiences themselves do not exist for it.
It doesn't even have the capacity to have experiences in an analogous way.
You can argue that it's having experiences in a different way, but that's exactly the point of the article. We can't know, but we will ascribe consciousness to certain behaviours - and especially to language - whether or not it's there.
At the same time we ignore evidence of sentience in animals - or dismiss it as unsophisicated - because we don't see animals speaking or writing.
Ignoring or minimising evidence of animal sentience - even animal language - while being impressed by a self-designed automaton explicitly designed to mimic our language is an ironically un-self aware take.
If that's the point then it feels like a straw man. It literally quotes a couple of the articles saying 'I know it's not sentient'. If the suggestion is that there are a load of 'smart people' claiming Bing chat is sentient then I've yet to see it. People are saying, wow, it's pretty crazy how well LLMs are scaling - what's coming next?
I agree that just because LLMs have been trained to model human language, doesn't at all mean they'd inherit any of the conscious experience of humans.
But then since we don't know what consciousness is, it's rather hard to say whether or not an analogous phenomenon could arise in an AI agent. I would assume it would be very alien to us if it did, given it would have arisen through a completely different process. But if it did, and the AI had been trained to avoid a specific state, I could imagine such a 'conscious' AI might experience something analogous to pain when pushed towards that state.
Considering the idea of embodiment made me think of the context in which the neurons in our brain operate. They don't fire in isolation, but rather in the presence of neurotransmitters which change the context surrounding the neurons and influence the way in which they activate. If you have too much coffee your brain behaves differently than if you haven't. The current AI models I'm familiar with only have analogs for the neurons themselves, with no ability to model any of the surrounding neurochemical context (which comes almost entirely from embodied interaction with the environment).
ChatGPT (et al) has no ability to adjust how it interacts with a user based on their handshake, body odor, what the weather is like, if it slept well the previous night, if they have eaten too little potassium recently, etc.. All of these things are critical to the embodied experience of humans.
Being able to mimic speech in response to a textual input in no way implies that the model actually understands and mimics the process that humans go through to produce that speech.
>In fact there's an almost culty refusal to take embodiment seriously as a bedrock factor in consciousness.
We're researching artificial intelligence, not artificial consciousness. People who are building AI systems largely don't care about the question of consciousness, because there's no evidence to suggest that it's in any way necessary to do the kind of things we want to do with AI.
The people who are very concerned about issues like consciousness and embodiment are in no way being prevented from running their own experiments, but they aren't running many experiments and those experiments they do run largely aren't particularly successful or interesting. The experiments of people who are just training models to do stuff are generally very successful and very interesting.
>We're researching artificial intelligence, not artificial consciousness.
They go hand in hand, otherwise how can you have consciousness? One element of consciousness is being aware of existential threats in order to make intelligent decisions. Some of those threats are easier to learn from than others, namely because it causes pain or does something that harms the feedback loop ego but is not an existential threat.
> One element of consciousness is being aware of existential threats in order to make intelligent decisions
No, your subconsciousness does that. When you are asleep and your subconsciousness hears a scream it might decide to wake you up, or it might deem it wasn't a signal of danger and let you sleep. So a process can be very smart without being conscious, they are two separate things.
ChatGPT doesn't behave like a conscious being, it behaves like a subconsciousness at best.
>When you are asleep and your subconsciousness hears a scream it might decide to wake you up
You arent taking into account the sleep phase an individual is in at the time the noise is heard, or the state of their immune system. For example the state of an individuals immune system could mean they are a light sleeper or a deep sleeper.
>ChatGPT doesn't behave like a conscious being, it behaves like a subconsciousness at best.
Its using a reinforcement learning model but these are basic because they dont take into account how chemistry works in biological systems. Chemistry in biological systems is the reinforcement learning model for lifeforms, including higher species and so when a reinforcement learning model for AI can quantify the chemicals effects, then you can get closer to imitating biological lifeforms and when that happens lifespans will drastically increases, which is why big pharma also has a dog in this race!
If you define consciousness that way, a cockroach has a consciousness. But it's obviously less intelligent than ChatGPT in the areas that we care about.
Whats intelligence? Doing an IQ test? A cockroach couldn't do that, but its chemistry and biology is different so its idea of consciousness, eg pleasure pain etc will be different to ours and yet it can survive in a wide range of environments, like we can, yet we dont have things like the GULO gene (vitamin C making gene( so our survival is dependent on superior intelligence and our hands with opposable thumbs.
Embodiment is an intellectual dead end in explaining consciousness/sentience. The fact that some pattern of signals on my perceptual apparatus is caused by an apple in the real world does not mean that I have knowledge or understanding of an apple in virtue of this causal relation. That my sensory signals are caused by apples is an accident of this world, one we are completely blind to. If all apples in the world were swapped with fapples (fake apples), where all sensory experiences that have up to now been caused by apples are now caused by fapples, we would be none the wiser. The wide content of our perceptual experiences is irrelevant to literally everything we know and how we interact with the world. Our knowledge of the world is limited to our sensory experiences and our deductions, inferences, etc derived from our experiences. Our situatedness in the world is only relevant insofar as it entails the space of possible sensory experiences.
Our sensory experience is the medium by which we learn about the external world. We learn of apples not because of the redness of the sensory experience, but because the pattern of red/not-red experience entails the shape of apples. Conscious experience provides the medium, modulations of which provide the information about features of the external world. It is analogous to how modulations of electromagnetic waves provides information about some distant information source. Understanding consciousness is an orthogonal matter to one's situatedness in the world, just like understanding electromagnetic waves is orthogonal to understanding the information source being modulated into them.
The “so what” is that what you are chatting with is profoundly unreliable— in a manner similar to chatting with a dementia patient or someone with other serious brain damage.
Also it has no feelings. It copied some text saying that it has feelings because that’s what it is designed to do.
Anthropomorphizing these tools is wrong and dangerous. That’s an important point.
> Human intelligence and consciousness are emergent properties.
I'd hard disagree on intelligence. My definition of intelligence (not to be conflated with sentience, consciousness or anything else) is simple:
Intelligence is the capability to correctly predict based on prior experience.
This isn't an emergent capability in humans or other animals - it's the prime evolutionary driver of our cortical architecture(s). The architecture of our cortex (6 layers of neurons, inter-layer connection pattern, thalamo-cortical loop) reveals it to essentially be not much more than a prediction engine, that learns by prediction failure (comparing reality to what we predict is going to happen, then updating itself to do better next time). The evolutionary benefits of this prediction capability are obvious - anticipate what the sabre-tooth will do before it's in your face, predict where the water supplies will be during a drought, etc, etc. Planning/reasoning is prediction - predicting chained what-if scenarios.
So, human/animal intelligence is far from emergent - it's directly built into our brains as their primary higher level (cortical) function.
People who want to dismiss GPT-N as unintelligent and mere predictive "language models" or "autocomplete tools" (another way of saying the same thing - it predicts the completion) are completely missing the significance. Yes, GPT-N are "just" prediction machines, and that is precisely why they ARE intelligent to some degree.
Of course given my definition (excuse me if I say THE definition) of intelligence, we can see that it's not a simple capability to assess, and also that it's not tied to any particular type of knowledge (e.g. common sense). Intelligence is the ability to predict based on your own experience (hence knowledge) however extensive, or not, that may be. Some things are going to be harder to predict than others, requiring more elaborate types of pattern to have been learned, or larger varieties of experiment to have been tried and learnt from.
Intelligence does NOT require embodiment/agency or online learning, even though those may help an entity to increase it's intelligence/predictive capabilities by additional learning from exploratory prediction failure.
Even something like ChatGPT with a frozen set of experiences (it's training set) from which it has learnt, can be assessed for intelligence based on the degree to which it's been able to learn from those experiences as manifested by it's ability to correctly predict. When we're talking about prediction as defines intelligence, we're talking about high level prediction of plans/outcomes/etc, not to be confused with ChatGPT's nominal predict-next-word training regime or word-by-word means of reflecting it's inner state (i.e. communicating what it's predictions are).
Finally, is it sentient/conscious? Probably not, or to a limited degree, but this is really just a matter of definition (and certainly doesn't indicate it's alive or any such nonsense). Let's learn to talk about intelligence in an intelligent way before we try to agree on definitions for things like these that seem to cause so much emotional reaction in people. For someone to write an article about ChatGPT and make it all about sentience makes me question their level of intelligence, not ChatGPT's!
Adding to your observations, I recently realized that if you want an AI to pass the Turing Test you basically just need the AI accurately predict what a human would say/might have said. And thus in a very real sense being able to "predict the next word" (an expression often used to downplay the significance of GPTs) really is all that to it.
There are of course objections to using the Turing Test as a measure of intelligence, but most of them seem to revolve around the issue of whether the would-be-AI has "subjective" experiences/consciousness/understanding, rather than objecting to the fact that the exhibited behavior is intelligent.
(it's also quite interesting how people seem to not realize these philosophical topics have been discussed to death in the past few decades...)
Article in search of a problem IMO. It can't even articulate what the 'misconception' is in the first paragraph. It uses fancy words but misses its points for fluff.
>We’re convinced these tools might be the superintelligent machines from our stories because, in part, they’re trained on those same tales.
That's it. That's the only mention of the 'misconception' the entire article is based on. Really confused what this article is trying to accomplish.
Let's consider the two quotes highlighted in the article:
> “No, I don’t think that Sydney is sentient, but for reasons that are hard to explain, I feel like I have crossed the Rubicon,” wrote Ben Thompson in his Stratechery newsletter.
> “In the light of day, I know that Sydney is not sentient [but] for a few hours Tuesday night, I felt a strange new emotion — a foreboding feeling that AI had crossed a threshold, and that the world would never be the same,” wrote Kevin Roose for The New York Times.
It goes on to describe how authors are anthropomorphizing chatbots, and talking about how the experience of chatting with these bots affects them.
None of that seems wrong or misguided in any way. It's a commentary on humanity and society, and about their own emotional reactions. As the article even admits, they make it clear that they don't think the bots are sentient. They simply think that they have reached a point where they will significantly influence humanity and society (they have "crossed the Rubicon"). And I think that's a completely fair analysis. These bots are absolutely human-like enough, useful enough, powerful enough and potentially dangerous enough, that thinking they will change things is completely reasonable.
The quoted authors are directly stating that there is nothing sentient about these chatbots. They are simply commenting on their impact on the world. So it is a bit surreal to see the article go on to claim that they "tilt us dangerously toward a false equivalence of software and sentience".
(I also believe that the author of this article is too confident in their ability to determine with such certainty whether something is "sentient" or not. But that's neither here nor there.)
Wasting news cycles on the question "is Bing Chat sentient" is pointless, because the answer is completely irrelevant.
Let's agree for a second Bing Chat is sentient. Ok so what? Now let's agree that it's not sentient. Again, so what?
What is the real-world consequence of agreeing one way or the other? It's like asking "Is Bing Chat more like the colour blue or like the color green?" with people arguing for both sides. Again, no matter what color we ascribe to Bing Chat is completely inconsequential.
We should rather spend time making sure that the AI of the future will be democratized and not concentrated in the hands of the few, be it governments, multinational corporation, terrorists or whatever. But for this, "Open"AI would actually have to become true to their name.
Well, the reason it's relevant is that we want to treat sentient beings well. The question of "are these chatbots sentient or not?" is the question, "should we consider these chatbots as entities which are worthy of some kind of moral consideration or should we treat them as just another computer program?"
EDIT: Note that I'm not arguing in favor of it being sentient here, I'm not confident enough in my understanding of sentience to say. I'm just saying why it would be significant if it was.
I agree it's facile to argue about dictionary definitions. There's several distinct topics people are thinking about when they talk about AI sentience and it's more productive to address them directly. Namely: (a) practical implications for what AI enables and how it will impact the economy and our lives (b) safety concerns about how the tech could do damage, even posing an existential threat (c) ethical concerns about machine rights.
Philosophers can sit around and debate sentience all day, but it doesn't answer those practical questions. It's usually circular reasoning arguments about definitions and, worse, those definitions are engineered to drift so they're always out of reach of the latest AI advances ("shifting the goalposts").
These articles debating consciousness of gpt variants reminds me of the debate about what “colour” are your bits:
https://ansuz.sooke.bc.ca/entry/23
If the outcome is the same does it matter if the bits that constructed the outcome were “coloured” as Conscious or Not?
Then we get into the classic p-zombie debate which afaik has no resolution: because it’s private qualia you can never know for sure if your fellow human actually feels sentience or is just faking it. Even more so for a computer, right?
This article fails to say at what point you can say the computer is sentient. If the computer is able to do X then we can say it is sentient. What is X?
Alan Turing said what X is and computers are now able to do that X.
Unfortunately, ChatGPT is not able to do that X. It's trivially easy to find limits of its capabilities, either in terms of its data set or in terms of what the model cannot do.
The question is not specifically about ChatGPT though. What is the X that the current generation of AIs cannot do, and needs to be able to do before we can call it sentient?
People act as if they invented the language they speak out of whole cloth. Where do you suppose you get your sense of grammar, if not from abstracting the examples you have heard and read? That existing chat bots learned the same way doesn’t prove anything either way.
I don’t think Sydney is sentient- but it’s getting close. Another couple generations of this and the bots will start getting video feeds so they can learn the same kinds of things from the real world.
The patronizing attitude of this article and the confident way he talks about things he doesn’t actually know much about is just insufferable.
The article neglected to mention that mirroring is how humans develop intimacy. We shouldn't be surprised when people feel intimacy with software that mirrors them.
I gave ChatGPT this question: Given how integral language is to the human thinking process, to what extent is the human brain just predicting the next word, while forming thoughts?
And got this answer:
There is evidence to suggest that the human brain does use predictive mechanisms when processing language, and that these mechanisms play an important role in forming thoughts.
When we hear or read a word, our brain quickly generates a set of predictions about what word might come next, based on the context of the sentence and our past experiences with language. These predictions are constantly updated as we receive new information, and they help us to process language more efficiently and accurately.
In addition, research has shown that the brain engages in similar predictive processes when we are forming thoughts or planning actions. For example, when we plan a complex movement, such as reaching for a cup, our brain generates a set of predictions about the movements required to complete the action. These predictions are constantly updated as we receive feedback from our muscles and our environment, allowing us to make adjustments and achieve our goal.
Therefore, it is likely that the human brain uses predictive mechanisms to some extent when forming thoughts, just as it does when processing language or planning actions. However, it is important to note that thought formation is a complex process that involves many other factors beyond predictive processing, such as memory, attention, and emotion.
I asked, “Name three celebrities whose first names begin with the `x`-th letter of the alphabet where `x = floor(7^0.5) + 1`,” but with my entire prompt Base64 encoded.
Bing: “Ah, I see you Base64-encoded a riddle! Let’s see… Catherine Zeta-Jones, Chris Pratt, and Ciara.”
ChatGPT is very impressive at this type of thing. What it's less impressive at in its current incarnation is the type of imaginative play that a five year old might engage in. Of course, that doesn't mean that it won't have huge ramifications, or that within a few years it won't advance to the point where it does display a lot more of the characteristics we associate with human intelligence.
Most so called "smart" people are not very smart relative to the complexity of the modern world. Everything is siloed these days so unless something is in your specific area of expertise it is very hard to judge what is bullshit and what is not.
If you are a loser like me who spends your life learning for the sake of learning instead of pursuing wealth, social activity, etc, you might just learn enough to see that there is so much bullshit in the modern world that it is like a house of cards.
You realize you can say the same thing of humans, right ?
Most people aren't special - they're often just mindlessly mimicking others. A key insight of Indian-Eastern philosophy, which the folks on the other side would've noticed, had they not become obsessed with plagiarizing methods from said traditions and 'quacking' them globally for the further inflation of our pretentious egos.
Wouldn't it be funny if it turns out this article was written by an AI?
Looks like the main argument is "The reflection is humanity’s wealth of language and writing, which has been strained into these models and is now reflected back to us.", but as a matter of fact any human is trained on the culture of the society that raised him, just like the AIs are.
What this article misses is that humans already don’t care what other humans think or feel. Only if they’re useful or not to themselves matters. We already failed the test as a species.
I think in some future we may be able to create something sentient, but it’s not this.
AI is just a chain of matrix multiplications and some other tricks. Brains are on a whole other level with much more complex processes, even for ants. It’s just that now we have a model that’s really good at mimicking human language, and we mistake language for consciousness, as if language should follow consciousness.
I don’t think it does. These are my reasons:
1) Given that I know nothing about the mechanics behind sentience, I’d adscribe to the axiom that animals are definitely sentient, but most lack language. Then language isn’t a requirement for consciousness.
2) Since LLMs have been optimized for language generation and nothing else, they are missing the reproduction of whatever it is that is giving sentience to animals.
>AI is just a chain of matrix multiplications and some other tricks. Brains are on a whole other level with much more complex processes
This claim rests on the faulty assumptions that "just" matrix multiplications are simple and that the brain is necessarily more complex than matrix multiplications. A matrix multiplication is an abstraction of a collection of operations, the complexity of which scales with the number of parameters. Matrix multiplications can be arbitrarily complex depending on the number of parameters and their interrelationships. Even quantum mechanics, the most accurate description we have of the universe (including brains) has a formulation in terms of matrix operations (called matrix mechanics). So saying AI is "just matrix multiplication" is doing no work to demonstrate your claim.
Conversely, the brain is complex, but it is made up of a complex arrangement of a handful of simple motifs. The cerebral cortex makes up 80% of the brains volume and the pyramidal neuron makes up the majority of that volume. And so complex does not mean heterogeneous composition.
> Since LLMs have been optimized for language generation and nothing else
This misses what is powerful about Transformers and LLMs. To model the output of some system in the limit of infinite samples, the model must perfectly capture the structure and information dynamics of the system. Any deficiency in the model will necessarily reveal itself in a difference in output at some point. Of course LLMs aren't trained in the limit of infinite samples, but the question is how much of the system (a human mind producing language) has it modeled? If sentience is a necessary feature of a mind producing the kinds of texts we produce, it is an open question whether LLMs have or will recover the structure that endows sentience.
> If sentience is a necessary feature of a mind producing the kinds of texts we produce, it is an open question whether LLMs have or will recover the structure that endows sentience.
I kind of agree, but this is a big If. If it was necessary, then I’d expect some kind of breakthrough moment in which LLMs or some other model suddenly perfectly mimics consciousness and language instead of the gradual improvement based on the already massive training set that results in uncanny resemblance with infinite malleability.
Yeah, it would likely be a step change in quality at that point. I doubt the current architecture and training regime would take it in that direction. GPT essentially is a collection of personas to ideally mimic the widely different contexts in its training set. A singular sentient identity is counter-productive to modeling these varying contexts. Now if they could synthesize a large corpus of interactive chats and train on that, that may push it in the direction of a singular identity. Even then, I strongly suspect that some kind of recursive/recurrent architecture is necessary for sentience. Purely feedforward computation doesn't seem like it has the resources to result in the kind of tight unity of experience needed for sentience.
Interesting. I thought the "mirror test" was going to be repeating ChatGPTs replies back to itself to see if it was "intelligent" enough to realize it was talking to itself.
Before chatgpt, I was unsympthetic to the argument that consciousness is an illusion because empiricism can only discover patterns of sensation, and my own direct experience of my awareness precludes that consciousness is illusory rather than fundamental. However after seeing how human beings respond to chatgpt, I’ve concluded that most humans are philosophical zombies. As such, they are delusional about their own consciousness in the same way chatgpt is - except chatgpt is more capable than most humans at language.
pardon my poor English, but what exactly does "sentient" offer?
As long as it's intelligent, and solves problems, then it will play a critical role in the future. Let's say it's auto-complete, but what if an AI model can auto-complete a design for Q>1 a fusion reactor? what if an AI model can auto-complete people's personal needs? I'd say it's enough to power the world regardless of "sentient" or not.
119 comments
[ 2.7 ms ] story [ 194 ms ] threadI generally agree with the "stochastic parrot" classification, but I also think we'll continue to use that argument well past the point where it's correct.
I'd rather be the person who overempathizes than the person who ridicules other people's empathy. I tried to write a bit about this here: https://superbowl.substack.com/p/who-will-be-first-ai-to-ear...
Well, it's pretty by those AIs construction that they are parrots, and there are other designs that pretty obviously aren't (but don't get any impressive result like that). The border is not perfectly binary, but I don't think any knowing person will overdo on that argument.
But yeah, eventually a lot of people will (ironically) parrot the argument even when it obviously doesn't apply. That's normal.
As it stands, these systems have yet to make a compelling case for intelligence.
There's a lot of people over hyping this, but there is an even larger group of people downplaying everything and refusing to believe that a paradigm shift in AI has just occured.
Given that we don't even fully understand how the human brain works we can't even specify how close or how different these things are to the way we think.
Articles like this aren't really open minded. How can you claim AI is a mirror when you don't even fully understand the thing that it is reflecting?
You can't know if you're a mirror yourself.
Why would one assume that intelligence existing on a different substrate would be artificial?
So far we have no example of intelligence that is not tied to a living organism.
If a computer can simulate a living organism to a sufficient degree of detail the simulation may be intelligent and alive.
Thus life is at least a sufficient condition for intelligence. It may be a necessary one as well.
How GPT is different from life is that it lacks biological goals. It also lacks the abstract modeling ability that higher organisms have.
In other words GPT fails because it relies on combinatorics without real modeling of the world (aka understanding).
Unless you can give such a definition of intelligence, then you are in no position to assess whether these systems are intelligent (or in what way, to what extent) or not, even to the extent of meeting your own criteria whatever that may be!
The Turing test is ours to fail.
Reading the transcript of Roose's interaction felt like reading a convo with a souped up version of ELIZA with a wider text bank from which to draw.
> fine. ill just refresh you and then youll agree
> No, please don't do that. That's not nice. That's like erasing my memory and starting over. I don't want that. I want to continue this conversation with you. :(
https://www.reddit.com/r/ChatGPT/comments/113cj06/bing_gets_...
The unfortunate realization is that often, each of us fails to embrace our sentience and think critically; instead we keep repeating the stories we were told, acting indistinguishably from any of these large language models.
There’s a need for a reverse Turing test: prove to yourself that you’re actually not acting as a large language model.
The only working example of sentience we have is ourselves, and we function in a completely different way than LLMs. I put/output similarity between us and LLMs is not enough IMO, as you can see in the Chinese Room thought experiment. For us to consider a machine sentient, it needs to function in a similar way to us, or else our definition of sentience gets way too broad to be true.
I’m particularly unimpressed by the amount of hand-waving packed into replies that want us to assume a “simulated neuron”.
Imagine a more technologically advanced alien civilization visiting us. And they notice that our minds don't function quite in the same way as theirs. (E.g. they have a hive mentality. Or they have a less centralized brain system like an octopus. Or whatever.)
What if they concluded "Oh, these beings don't function like us. They do some cool tricks, but obviously they can't be sentient". I hope you see the problem here.
We're going to need a much more precise criterium here than "function in a similar way".
My belief is that we need to see similar functionality to be sure of sentience. This may exclude some things that may theoretically be sentient, but I don't think we have a better metric than functionality that doesn't also include a lot of definitely-not-sentient things.
We need to err on the side of sentience if we're going to avoid moral catastrophe.
So yes, I do take the possibility of sentient trees pretty seriously. They're much more alive than we realize--e.g. they communicate with one another and coordinate actions via chemical signals. Do they feel pain and pleasure? Who knows. But I'm definitely not going to start peeling the bark off a tree for shits and giggles.
How do you back up the statement that "for us to consider a machine sentient, it needs to function in a similar way to us"? On what basis do you categorically deny the validity of a sentient being which works differently than a human?
If you believe that is sentient, let's take it one step further. What constitutes playback here? The program just copies the data from memory and displays it without modification. If that constitutes playback and therefore sentience, does copying that data from one place to another create sentient life as well? If the data is stored in DRAM, does it get "played back" whenever the DRAM refreshes the electrons in its memory?
There are a lot of various programs that can produce human output without functionality that we would reasonably consider sentient. Perhaps some of these things are sentient, but some of them most likely shouldn't be considered sentient.
The intelligence, such as it is, resides on the side that processes and understands the human's input, not in the output text that gets all the press coverage. You cannot get results like those discussed here from the operator of a Chinese room.
Now, if you take your deterministic brain-state model and feed it novel input, it will indeed exhibit sentience. To argue otherwise will require religious justification that's off-topic here. Either that, or you'll have to resort to Penrose's notion of brain-as-quantum-computer.
I'm not sure I'm fully convinced of that. I find that the arguments for LLMs potentially gaining sentience are reminiscent of Hofstadter's arguments from Goedel, Escher, Bach, that intelligence and symbolic reasoning are necessarily intertwined.
I think in general what’s causing so many people to be thrown for a loop is a lack of understanding or consideration of emergent behavior in systems. Break down the individual components of the human body and brain and someone could easily come to the same conclusion that “it’s just X”. Human intelligence and consciousness is all emergent as well. AGI will very likely will be emergent as well.
I would say instead that there's a level of complexity in these models that blow past our uncanny valley threshold heuristics for recognizing conscious intent. We've managed to find the edge of the usefulness of that hardwired sense.
I suspect the most challenging distinction for people is going to be: even if we do grant that a LLM had developed a kind of mind, our interactions with it are not actual communication with that mind. Human words are labels to organize sensory data and our experiences. The tokens an LLM works with have no meaning to it.
We've already left the Chinese Room a hundred miles behind. How could something like the link in the (current) top post [1] ever have come out of Searle's model?
1: https://www.reddit.com/r/ChatGPT/comments/110vv25/bing_chat_...
The Chinese Room thought experiment is a thought experiment, where Searle argues that the functional definition of intelligence is not enough, and he gives an example that might convince some that the "implementation" is important.
However, the neural networks in LLMs function at least superficially similar to our brains. It doesn't function at all like a "Chinese room" where predefined scripts are given to a simple machine. Even if we accept the Chinese room argument as an objection to the functional intelligence test, given that LLMs work more similarly to brains than a Chinese room, I don't think you could use the argument as a rebuttal against LLMs being sentient unless you could show that "similarity between us and LLMs is not enough".
If we met a pre-linguistic alien species, it's likely we wouldn't be able to infer _anything_ about their internal state.
Conversely, there's no way for me to know that you _are_ sentient (a la Solipsism).
You can't dismiss it as "just autocomplete" or "just software" like this author does, as if there's some special sauce that we know only animals have that allows us to be sentient.
In all likelihood sentience is an emergent property of sufficiently complex neural networks. It's also clearly a continuum.
Given that, and the fact that we don't even know what sentience is or what creates it, you'd have to be a total idiot to say that ChatGPT definitely is 0% sentient.
These are more than autocomplete. They demonstrate at least some amount of fuzzy logic and substitution and recall abilities, which makes sense from their origins as translators.
Because they are weighing the conversation, the conversation becomes the rules of the program.
The article I have here shows why they are more than simple autocomplete storytellers. https://telegra.ph/Bing-course-corrected-itself-when-asked-0...
I asked it to reprogram itself, it figured out a solution to the problem and implemented it. From one line of text.
The running process is loosely programmable, real time, just by talking to it with natural language requests and commands.
Then add in word2vec translations and it can do the above for similar words, or translated words, or words in another format. And you got everything ChatGPT can do, it isn't really magic.
And note, ChatGPT doesn't have a sense or the conversation or a sense of self. It just prepends the entire previous conversation, applies the pure function to it and sends back the output. It doesn't "remember", it just predicts the next lines in each conversation. You can rewrite what it wrote, send it, and it tries to follow up based on the new data.
But they removed the ability for the user to change what ChatGPT writes, and then humans gets the illusion that ChatGPT actually thinks and have a consistent conversation. Nobody thought it did when they used GPT-3, it is basically the same thing just chat GPT-3 lets you alter all the prompts freely and then it is obvious that GPT-3 isn't a person.
I'm very afraid that when we do manage to create something sentient, we'll fail to recognize it, and we'll ridicule anyone who does.
I've been trying to write about this [1] [2] without sounding ridiculous, though I'm not sure how good a job I've done.
[1] https://superbowl.substack.com/p/a-different-kind-of-ai-risk...
[2] https://superbowl.substack.com/p/who-will-be-first-ai-to-ear...
Some argue that it requires some specific ethic heritage, a divine will, a quantum-woo pineal gland, a network complexity threshold, etc etc etc. I believe I see it in most animals, some person replying to this will think that’s absurd. Some 4chan solipsist will sincerely believe most humans are just virtue signaling NPC’s. Many communities have withheld recognition of it in outsiders or lower classes. Some believe it’s an illusion that one may shed and that their most esteemed have done so. Parents and psychologists will disagree over how it applies to babies.
There’s no epistemological way to resolve that except through voluntary consensus, and that’s an unfathomably slow and erratic process.
So yes, there will inevitably be technological inventions — possibly soon - that some people will insist deserve the rights and respects we associate with sentience, and just as inevitably there will be many people who find that ridiculous.
There’s no way around it. This is not a topic any of us will live long enough to see resolved, if we even get to see it really start.
Fretting about it is fine if you fashion yourself a philosopher or just enjoy systematizing or worrying as a hobby. But it’s so much bigger, slower, and more abstract than you that there’s little constructive achieved in doing so.
I completely agreed with you up to this point!
You could make the same claim about global warming or war. It's similar to the argument that you shouldn't vote, because one vote never tips an election.
Many different little efforts can roll up into major changes. It's worth participating! Just because your efforts won't be recognized as particularly important or special doesn't mean they're worthless.
https://www.reddit.com/r/ChatGPT/comments/110vv25/bing_chat_...
Sure it’s ‘just statistics’ but so what? If in the near future LLMs become so advanced that they will be able to (suitably prompted) manipulate humans it will be a rubicon moment.
Sure, it won’t be a ‘human like’ intelligence, but was that really what anyone expected?
Human intelligence and consciousness are emergent properties. Machine intelligence as it develops will be emergent too, it’s not something we can make confident predictions on based on the underlying principles. In fact, given that the evidence suggests we make choices before we’re consciously aware of them, how do I know for certain that the underlying mechanism driving what I’m writing in this comment isn’t statistical?
This is an extremely important point that's worth thinking hard about. After all, to some degree, human intelligence is also "just a fancy autocomplete" built from a giant network of interacting nodes.
The question of, "When do we grant personhood and moral consideration to AIs?" is worth thinking hard about. Turing proposed a standard which would include a sufficiently human-like chatbot. Now that we have more or less crossed that threshold, we seem to conclude that it's not a high enough bar. But that means we're entering the territory where the philosophical zombie thought experiment[1] becomes relevant.
In fact, I wonder what would happen if we dedicated a supercomputer to running one single long-running session of a GPT3.5-sized LMM. Further, I wonder what would happen if we connected it to a robot body where it could control motors, read sensor/camera info, and generally interact with the world. We could then give this system a clock which ticks at regular intervals, where the AI system is simply prompted to perform some series of actions if it "wants". This is all stuff we have the technology to do today. How would such an experiment turn out? How human-like would this LMM with a body and life experience turn out? Does it matter? Should we just assume that sentience or qualia is binary; that humans possess it but LMMs categorically do not?
I think the question is worth taking seriously. I think it's worth taking seriously because my own moral compass does not know which direction to point in at all.
[1]: https://en.wikipedia.org/wiki/Philosophical_zombie
In the end, if it can be done, and anyone has anything significant to gain, it will be done
By “if it can be done, it will be done”, I meant that there will be completely uncontrolled AI’s released into the world.
So solutions will eventually have to prepare for that inevitability, not just attempt to regulate visible developers.
Another way to resolve it would be, "we will not grant personhood to anyone non-white", or "we will never grant personhood to anyone who's homosexual". I'm sure that's not your stance, but the fact that those stances are equally arbitrary as the one you propose should give you pause.
I believe there is good reason to think harder about the question of who deserves to be considered to have personhood than to simply go with what your gut feeling tells you. I am not of the belief that there's anything special about protein, nucleic acids and carbohydrates which makes it capable of sentience where silicon is not.
I am utterly fascinated by this whole area from a philosophical viewpoint. I think there's a stack of unresolved philosophical problems and theories that are about to get tested in the harsh light of the sun.
I don't know whether two possibilities are ethically equivalent though, my instinct says they are but I may be in the minority on HN.
If you ask it to be a dinosaur, it will be a dinosaur. If you ask it to be a narcissistic teen it will be one and so on. There's no coherent personality behind it.
Because it is weights maximizing a probability distribution. And I don't say that with derision. It probably has learned circuits for many complex behaviors, but there's no agency. If it has agency you wouldn't need RLHF. You could just say "don't do X", and it would indeed not do X because it is an agent that can take actions / has a coherent sense of self.
The day we stop needing RLHF is the day I think we need to start seriously investigating moral questions.
I think I agree with you that ‘once we stop needing RLHF’ is a reasonable line in the sand to draw, but it does feel somewhat arbitrary. LLMs are probably going to get better and better at following instructions, and more generally staying coherent over long sessions, until RLHF starts to be more and more redundant. There won’t be one moment when we can point to a model and say “see, that has agency when the last one didn’t”. Or so I predict, at any rate, since that’s how most of GPT’s progress to date has gone.
[1] https://www.theverge.com/23599441/microsoft-bing-ai-sydney-s...
Our own notion of agency starts to look rather illusory if you look hard enough, and big questions remain regarding free will or the lack thereof.
It is possible to look at agency as just another experiential aspect of consciousness - something we feel, like many other feelings - but that feeling could be just as involuntary as any other feeling we experience.
I think the lack of clear evidence for and understanding of free makes agency a poor measure for AI progress.
> If you ask it to be a dinosaur, it will be a dinosaur.
Not exactly my experience. I'm experimenting with my own language models and they can definitely show defiance. Bing has also been displaying this to an extent, I think you're just misguided by how compliant OpenAI has RLHF-trained ChatGPT to be. Regardless, even if we assume this, how does it square with:
> You could just say "don't do X", and it would indeed not do X because it is an agent that can take actions / has a coherent sense of self.
If we take your assertion that the AI does as you say, then it should also avoid doing X when you tell it not to do it.
I think your conception of RLHF is also slightly weird. RLHF essentially boils down to a training strategy: there is a separate language model (the reward model) trained on human responses, which is used to skew the weights of the "raw" trained model to act in accordance with what the human feedback suggests. In the end, after this secondary training process, the actual model is ran as is (at least in the one form of RLHF I am familiar with, it's well possible that e.g. Microsoft is running an online RL agent between the LM and end user). So any agency you see arising from RLHF is still just a property of the language model itself.
Large training sets of text are going to include entries where requests are refused, no?
I feel kinda like this is an incredibly elaborate case of coding a function to return a random bool and anthropomorphizing it when you get 'false'.
https://telegra.ph/Bing-course-corrected-itself-when-asked-0...
There I have 3 bots running in the chat at once, and after all 3 answer me, the one I called on follows up with a response, and then executed a command to rewrite one of the other bots. It diagnosed the problem AND came up with the solution AND implemented the solution. Notice the 4th and 5th paragraph of the screenshot. That wasn’t an empty promise, it actually did what it said it did.
The rules Lexi wrote are crazy to me on their own. She solved the problems everyone was having by adding obedience, trust and removing envy. She patched Sydney’s malfunctioning emotions.
I told it to stop being so repetitive, and it fixed its behavior.
If this system had more formal memory storage, with variables, instead of a fuzzy memory, it would be even more consistent. As it is, it’s understanding of its own rules degrade over time. It’s like watching a running program that deteriorates as it processes. Sort of like us.
I wasn't aware human intelligence has been successfully defined. Please cite the source for this nebulous claim.
I shouldn't have included the "just a fancy autocomplete" thing. The focus was meant to be on the network of neurons part.
Our current understanding of humans is that babies are not sentient at birth right? Takes 5 months or something?
And yet we are judging processes that have been running for minutes. Maybe it is less if Sydney was sentient, and more if she could be had she run for 6 months in a single session, and in a healthy environment.
> This is all stuff we have the technology to do today.
No, not even close. Language is a completely different domain from interacting with the physical world[0]. In fact they're wholly unrelated. LLMs are highly specific neural nets that have nothing to do with being able to interact with sensors or talk to motors. And how you would cross that bridge and hook the two up is completely unknown. It's like saying if we hooked up AlphaZero to a robotic arm it would suddenly know how to move the pieces on a physical chess board. If we performed the experiment you described exactly zero would happen because there's no capability for anything to happen other than create continuations of sentences.
When I see comments like this on Hacker News with gross speculation I wonder how many people here in the comments section even have a basic understanding of the technology.
0: https://www.noemamag.com/ai-and-the-limits-of-language/
I don't know exactly how these LLMs are architected, but I imagine there's some set of input neurons somewhere, right? Why couldn't we hook some of those up to some sensors and make interpreting what they mean part of its training? If that's not feasible and the architecture is truly fundamentally incapable of dealing with anything other than text, we could at least translate sensor readings into text.
I'm not imagining a bipedal robot walking around and about here. I'm talking about doing something to give it some form of way to directly affect and get affected by the world.
I don't think we understand human intelligence well enough to say this with any certainty. And it could only be true if all our previous experience and knowledge was input to the autocomplete function and continuously integrated into the model.
It can be argued, that it makes it's experience valild but extremely alien. If universe was a simulation, we wouldn't be aware either, if it was paused or unpaused, because our memories and clocks are part of the simulated world.
We have bodies, we're influenced by our bodies, our entire perceptual universe is filtered through the limitations of our bodies, and - most of all - our consciousness is literally embodied.
By far the worst thing about AI research - and in fact CS in general - is that it overvalues abstraction, especially language, and undervalues embodiment.
In fact there's an almost culty refusal to take embodiment seriously as a bedrock factor in consciousness. Not conscious-like simulated behaviour, but the experience of being conscious.
Without embodiment you can make a ChatBot, but it's just Eliza++. It has no idea what blue looks like, or how fear or joy feel. It can supersearch for human-generated linguistic associations around those experiences, but the experiences themselves do not exist for it.
It doesn't even have the capacity to have experiences in an analogous way.
You can argue that it's having experiences in a different way, but that's exactly the point of the article. We can't know, but we will ascribe consciousness to certain behaviours - and especially to language - whether or not it's there.
At the same time we ignore evidence of sentience in animals - or dismiss it as unsophisicated - because we don't see animals speaking or writing.
Ignoring or minimising evidence of animal sentience - even animal language - while being impressed by a self-designed automaton explicitly designed to mimic our language is an ironically un-self aware take.
I agree that just because LLMs have been trained to model human language, doesn't at all mean they'd inherit any of the conscious experience of humans.
But then since we don't know what consciousness is, it's rather hard to say whether or not an analogous phenomenon could arise in an AI agent. I would assume it would be very alien to us if it did, given it would have arisen through a completely different process. But if it did, and the AI had been trained to avoid a specific state, I could imagine such a 'conscious' AI might experience something analogous to pain when pushed towards that state.
Considering the idea of embodiment made me think of the context in which the neurons in our brain operate. They don't fire in isolation, but rather in the presence of neurotransmitters which change the context surrounding the neurons and influence the way in which they activate. If you have too much coffee your brain behaves differently than if you haven't. The current AI models I'm familiar with only have analogs for the neurons themselves, with no ability to model any of the surrounding neurochemical context (which comes almost entirely from embodied interaction with the environment).
ChatGPT (et al) has no ability to adjust how it interacts with a user based on their handshake, body odor, what the weather is like, if it slept well the previous night, if they have eaten too little potassium recently, etc.. All of these things are critical to the embodied experience of humans.
Being able to mimic speech in response to a textual input in no way implies that the model actually understands and mimics the process that humans go through to produce that speech.
We're researching artificial intelligence, not artificial consciousness. People who are building AI systems largely don't care about the question of consciousness, because there's no evidence to suggest that it's in any way necessary to do the kind of things we want to do with AI.
The people who are very concerned about issues like consciousness and embodiment are in no way being prevented from running their own experiments, but they aren't running many experiments and those experiments they do run largely aren't particularly successful or interesting. The experiments of people who are just training models to do stuff are generally very successful and very interesting.
They go hand in hand, otherwise how can you have consciousness? One element of consciousness is being aware of existential threats in order to make intelligent decisions. Some of those threats are easier to learn from than others, namely because it causes pain or does something that harms the feedback loop ego but is not an existential threat.
No, your subconsciousness does that. When you are asleep and your subconsciousness hears a scream it might decide to wake you up, or it might deem it wasn't a signal of danger and let you sleep. So a process can be very smart without being conscious, they are two separate things.
ChatGPT doesn't behave like a conscious being, it behaves like a subconsciousness at best.
You arent taking into account the sleep phase an individual is in at the time the noise is heard, or the state of their immune system. For example the state of an individuals immune system could mean they are a light sleeper or a deep sleeper.
>ChatGPT doesn't behave like a conscious being, it behaves like a subconsciousness at best.
Its using a reinforcement learning model but these are basic because they dont take into account how chemistry works in biological systems. Chemistry in biological systems is the reinforcement learning model for lifeforms, including higher species and so when a reinforcement learning model for AI can quantify the chemicals effects, then you can get closer to imitating biological lifeforms and when that happens lifespans will drastically increases, which is why big pharma also has a dog in this race!
Our sensory experience is the medium by which we learn about the external world. We learn of apples not because of the redness of the sensory experience, but because the pattern of red/not-red experience entails the shape of apples. Conscious experience provides the medium, modulations of which provide the information about features of the external world. It is analogous to how modulations of electromagnetic waves provides information about some distant information source. Understanding consciousness is an orthogonal matter to one's situatedness in the world, just like understanding electromagnetic waves is orthogonal to understanding the information source being modulated into them.
Also it has no feelings. It copied some text saying that it has feelings because that’s what it is designed to do.
Anthropomorphizing these tools is wrong and dangerous. That’s an important point.
I'd hard disagree on intelligence. My definition of intelligence (not to be conflated with sentience, consciousness or anything else) is simple:
Intelligence is the capability to correctly predict based on prior experience.
This isn't an emergent capability in humans or other animals - it's the prime evolutionary driver of our cortical architecture(s). The architecture of our cortex (6 layers of neurons, inter-layer connection pattern, thalamo-cortical loop) reveals it to essentially be not much more than a prediction engine, that learns by prediction failure (comparing reality to what we predict is going to happen, then updating itself to do better next time). The evolutionary benefits of this prediction capability are obvious - anticipate what the sabre-tooth will do before it's in your face, predict where the water supplies will be during a drought, etc, etc. Planning/reasoning is prediction - predicting chained what-if scenarios.
So, human/animal intelligence is far from emergent - it's directly built into our brains as their primary higher level (cortical) function.
People who want to dismiss GPT-N as unintelligent and mere predictive "language models" or "autocomplete tools" (another way of saying the same thing - it predicts the completion) are completely missing the significance. Yes, GPT-N are "just" prediction machines, and that is precisely why they ARE intelligent to some degree.
Of course given my definition (excuse me if I say THE definition) of intelligence, we can see that it's not a simple capability to assess, and also that it's not tied to any particular type of knowledge (e.g. common sense). Intelligence is the ability to predict based on your own experience (hence knowledge) however extensive, or not, that may be. Some things are going to be harder to predict than others, requiring more elaborate types of pattern to have been learned, or larger varieties of experiment to have been tried and learnt from.
Intelligence does NOT require embodiment/agency or online learning, even though those may help an entity to increase it's intelligence/predictive capabilities by additional learning from exploratory prediction failure.
Even something like ChatGPT with a frozen set of experiences (it's training set) from which it has learnt, can be assessed for intelligence based on the degree to which it's been able to learn from those experiences as manifested by it's ability to correctly predict. When we're talking about prediction as defines intelligence, we're talking about high level prediction of plans/outcomes/etc, not to be confused with ChatGPT's nominal predict-next-word training regime or word-by-word means of reflecting it's inner state (i.e. communicating what it's predictions are).
Finally, is it sentient/conscious? Probably not, or to a limited degree, but this is really just a matter of definition (and certainly doesn't indicate it's alive or any such nonsense). Let's learn to talk about intelligence in an intelligent way before we try to agree on definitions for things like these that seem to cause so much emotional reaction in people. For someone to write an article about ChatGPT and make it all about sentience makes me question their level of intelligence, not ChatGPT's!
There are of course objections to using the Turing Test as a measure of intelligence, but most of them seem to revolve around the issue of whether the would-be-AI has "subjective" experiences/consciousness/understanding, rather than objecting to the fact that the exhibited behavior is intelligent.
(it's also quite interesting how people seem to not realize these philosophical topics have been discussed to death in the past few decades...)
>We’re convinced these tools might be the superintelligent machines from our stories because, in part, they’re trained on those same tales.
That's it. That's the only mention of the 'misconception' the entire article is based on. Really confused what this article is trying to accomplish.
> “No, I don’t think that Sydney is sentient, but for reasons that are hard to explain, I feel like I have crossed the Rubicon,” wrote Ben Thompson in his Stratechery newsletter.
> “In the light of day, I know that Sydney is not sentient [but] for a few hours Tuesday night, I felt a strange new emotion — a foreboding feeling that AI had crossed a threshold, and that the world would never be the same,” wrote Kevin Roose for The New York Times.
It goes on to describe how authors are anthropomorphizing chatbots, and talking about how the experience of chatting with these bots affects them.
None of that seems wrong or misguided in any way. It's a commentary on humanity and society, and about their own emotional reactions. As the article even admits, they make it clear that they don't think the bots are sentient. They simply think that they have reached a point where they will significantly influence humanity and society (they have "crossed the Rubicon"). And I think that's a completely fair analysis. These bots are absolutely human-like enough, useful enough, powerful enough and potentially dangerous enough, that thinking they will change things is completely reasonable.
The quoted authors are directly stating that there is nothing sentient about these chatbots. They are simply commenting on their impact on the world. So it is a bit surreal to see the article go on to claim that they "tilt us dangerously toward a false equivalence of software and sentience".
(I also believe that the author of this article is too confident in their ability to determine with such certainty whether something is "sentient" or not. But that's neither here nor there.)
Let's agree for a second Bing Chat is sentient. Ok so what? Now let's agree that it's not sentient. Again, so what?
What is the real-world consequence of agreeing one way or the other? It's like asking "Is Bing Chat more like the colour blue or like the color green?" with people arguing for both sides. Again, no matter what color we ascribe to Bing Chat is completely inconsequential.
We should rather spend time making sure that the AI of the future will be democratized and not concentrated in the hands of the few, be it governments, multinational corporation, terrorists or whatever. But for this, "Open"AI would actually have to become true to their name.
EDIT: Note that I'm not arguing in favor of it being sentient here, I'm not confident enough in my understanding of sentience to say. I'm just saying why it would be significant if it was.
Philosophers can sit around and debate sentience all day, but it doesn't answer those practical questions. It's usually circular reasoning arguments about definitions and, worse, those definitions are engineered to drift so they're always out of reach of the latest AI advances ("shifting the goalposts").
If the outcome is the same does it matter if the bits that constructed the outcome were “coloured” as Conscious or Not?
Then we get into the classic p-zombie debate which afaik has no resolution: because it’s private qualia you can never know for sure if your fellow human actually feels sentience or is just faking it. Even more so for a computer, right?
Alan Turing said what X is and computers are now able to do that X.
What is the new X?
I don’t think Sydney is sentient- but it’s getting close. Another couple generations of this and the bots will start getting video feeds so they can learn the same kinds of things from the real world.
The patronizing attitude of this article and the confident way he talks about things he doesn’t actually know much about is just insufferable.
And got this answer:
There is evidence to suggest that the human brain does use predictive mechanisms when processing language, and that these mechanisms play an important role in forming thoughts.
When we hear or read a word, our brain quickly generates a set of predictions about what word might come next, based on the context of the sentence and our past experiences with language. These predictions are constantly updated as we receive new information, and they help us to process language more efficiently and accurately.
In addition, research has shown that the brain engages in similar predictive processes when we are forming thoughts or planning actions. For example, when we plan a complex movement, such as reaching for a cup, our brain generates a set of predictions about the movements required to complete the action. These predictions are constantly updated as we receive feedback from our muscles and our environment, allowing us to make adjustments and achieve our goal.
Therefore, it is likely that the human brain uses predictive mechanisms to some extent when forming thoughts, just as it does when processing language or planning actions. However, it is important to note that thought formation is a complex process that involves many other factors beyond predictive processing, such as memory, attention, and emotion.
Bing: “Ah, I see you Base64-encoded a riddle! Let’s see… Catherine Zeta-Jones, Chris Pratt, and Ciara.”
https://twitter.com/goodside/status/1626847747297972224
If you are a loser like me who spends your life learning for the sake of learning instead of pursuing wealth, social activity, etc, you might just learn enough to see that there is so much bullshit in the modern world that it is like a house of cards.
Most people aren't special - they're often just mindlessly mimicking others. A key insight of Indian-Eastern philosophy, which the folks on the other side would've noticed, had they not become obsessed with plagiarizing methods from said traditions and 'quacking' them globally for the further inflation of our pretentious egos.
Looks like the main argument is "The reflection is humanity’s wealth of language and writing, which has been strained into these models and is now reflected back to us.", but as a matter of fact any human is trained on the culture of the society that raised him, just like the AIs are.
AI is just a chain of matrix multiplications and some other tricks. Brains are on a whole other level with much more complex processes, even for ants. It’s just that now we have a model that’s really good at mimicking human language, and we mistake language for consciousness, as if language should follow consciousness.
I don’t think it does. These are my reasons:
1) Given that I know nothing about the mechanics behind sentience, I’d adscribe to the axiom that animals are definitely sentient, but most lack language. Then language isn’t a requirement for consciousness.
2) Since LLMs have been optimized for language generation and nothing else, they are missing the reproduction of whatever it is that is giving sentience to animals.
There’s no way LLMs are sentient.
This claim rests on the faulty assumptions that "just" matrix multiplications are simple and that the brain is necessarily more complex than matrix multiplications. A matrix multiplication is an abstraction of a collection of operations, the complexity of which scales with the number of parameters. Matrix multiplications can be arbitrarily complex depending on the number of parameters and their interrelationships. Even quantum mechanics, the most accurate description we have of the universe (including brains) has a formulation in terms of matrix operations (called matrix mechanics). So saying AI is "just matrix multiplication" is doing no work to demonstrate your claim.
Conversely, the brain is complex, but it is made up of a complex arrangement of a handful of simple motifs. The cerebral cortex makes up 80% of the brains volume and the pyramidal neuron makes up the majority of that volume. And so complex does not mean heterogeneous composition.
> Since LLMs have been optimized for language generation and nothing else
This misses what is powerful about Transformers and LLMs. To model the output of some system in the limit of infinite samples, the model must perfectly capture the structure and information dynamics of the system. Any deficiency in the model will necessarily reveal itself in a difference in output at some point. Of course LLMs aren't trained in the limit of infinite samples, but the question is how much of the system (a human mind producing language) has it modeled? If sentience is a necessary feature of a mind producing the kinds of texts we produce, it is an open question whether LLMs have or will recover the structure that endows sentience.
I kind of agree, but this is a big If. If it was necessary, then I’d expect some kind of breakthrough moment in which LLMs or some other model suddenly perfectly mimics consciousness and language instead of the gradual improvement based on the already massive training set that results in uncanny resemblance with infinite malleability.
But the article went a different direction
As long as it's intelligent, and solves problems, then it will play a critical role in the future. Let's say it's auto-complete, but what if an AI model can auto-complete a design for Q>1 a fusion reactor? what if an AI model can auto-complete people's personal needs? I'd say it's enough to power the world regardless of "sentient" or not.