>AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations.
Opening paragraph, stated without evidence. Im not entirely convinced this is true. It likely is, but at some point it very well might stop being true.
I agree. I haven't read any more than the first paragraph yet (but I will, after work). The only way that 'AIs are not conscious' can be true is if we decide, with high confidence, that they are lacking some essential property that is not lacking in ourselves. There is no convincing philosophical position that supports this (convincing to me, anyway).
Hmm let me try then. AI agents do not experience real time. This is understandable given their design, but it’s also something we have good empirical evidence for. They cannot, especially over the long horizon, track how much real time has passed as they complete their tasks. And they are not off by a few minutes but often bizarrely off, even mixing across past present and future.
Biology, on the other hand, is nothing but timed processes in a loop, the most obvious to us being the circadian cycle. As estimators of wall clock time, biology isn’t great, but when it comes to internal processes, and most certainly learning, memory, sensing, locomotion… biology is rhythmic in behavior, and the rhythms go all the way down to gene expression. More, these rhythms are, except during sleep, constantly entraining to signals from the environment that indicate time, most importantly light.
I think it’s a fairly unremarkable claim that agency and consciousness are temporal processes that depend on systems having an internal sense of time. How else can you anticipate? How can a system that can be literally turned off ever succeed in an environment where time never stops?
For around 8 hours a day, neither do you. Not sure if this has anything to do with the subject at all.
>track how much real time has passed as they complete their tasks.
Humans don't do this either. You use context clues from the world around you. If I lock you in a room with no windows or a dark cave your timing senses can go all fucky really quick.
>is nothing but timed processes in a loop,
I mean, so is an agents harness. You can make as many loops as you'd like here.
>For around 8 hours a day, neither do you. Not sure if this has anything to do with the subject at all.
You mean sleep? Actually the answer is pretty complicated. Your body clock is very much on during sleep. It responds to temperature changes. Sound and light responsiveness is obviously dampened, but hardly absent.
You are unaware of wall clock time. You’re in an altered state of consciousness that needs your eyes closed after all. However, you are not timeless, nor is the entirety of your body and brain unaware of environmental signals for time.
> Humans don't do this either. You use context clues from the world around you. If I lock you in a room with no windows or a dark cave your timing senses can go all fucky really quick.
Again, that’s wall clock time. Our time perception does indeed get fucked up in total darkness. But our internal clock ticks on. I’d recommend reading about the Aschoff Bunker experiments, which first proved this rigorously.
> I mean, so is an agents harness. You can make as many loops as you'd like here.
An agents harness doesn’t touch its weights. A figure 8 on paper is a loop. That doesn’t mean it’s the same as a dynamical loop that’s self sustaining and internally organized.
Are you just reading the words and randomly grabbing related concepts to say nothing here is meaningful?
I agree, there is certainly a clear operational difference in how human brains and these models produce language. I'm not convinced, though, that experiencing real time is a requirement of conscious thought. A cognitive system (assuming these models and the brain are both examples of this) needs to proceed from state A to state B as it processes information. The time interval between these states can be arbitrary, it seems to me (setting aside problems with disconnecting the mind from its substrate, which does of course need rhythms at various frequencies; oxygen at a higher frequency than sugar, and so on). I'm not sure that going from A to B quickly, or slowly, or with intervals of a thousand years affects that entity's consciousness in of itself (even though it might be difficult to put ourselves in the shoes of such an entity).
> A cognitive system (assuming these models and the brain are both examples of this) needs to proceed from state A to state B as it processes information
Is that all it needs to do? A rock, in response to a sound signal (make it as informative as you want) can also be said to go from state A (molecules at rest) to state B (molecules vibrating in response to the sound). Is the rock a cognitive system?
Or let’s go up a level of complexity. Is a thermostat a cognitive system? It doesn’t just go from state A to B but measures changing stages against a reference. Is it a cognitive system?
> The time interval between these states can be arbitrary, it seems to me (setting aside problems with disconnecting the mind from its substrate, which does of course need rhythms at various frequencies; oxygen at a higher frequency than sugar, and so on)
Not sure how you land on this. Going from A to B, in your definition, is a purely internal state transition, right? If the motivation to go to state B is external (as will often be the case with a conscious agent), and the system is offline entirely while it switches from A to B, how is it to reverse or stall its transition if external signals contradict the earlier decision?
It’s a very bizarre definition of consciousness, that you don’t need to be in time. Seems to break the word beyond meaning, and allows it to be applied to any system capable of change.
> I'm not sure that going from A to B quickly, or slowly, or with intervals of a thousand years affects that entity's consciousness in of itself
It’s not about the speed of transition from A to B. Instead it’s about the internal processes being temporally commensurate, and entrainable to environmental periodicities.
All of biology (plants and many bacteria included) has its dynamics are organized around an internally generated rhythm. This rhythm entrains to the external environment, shifting its phase to match (mismatch leads to issues like jet lag: a place where your consciousness shows its temporal boundaries as it is abruptly subjected to an environment out of sync with its current phase tethered to another location). Crucially, the internal rhythm continues to run in isolated conditions (total darkness), and have been measured to be around 24 hours, but slightly off, within and across species (and the same holds for endogenous tidal rhythms, which also don’t exactly track tidal period left to themselves).
Everything, including learning and memory, but also sleep, sensory and locomotor function, shows organization around this internal temporal axis. The key advantage this gives, evolutionarily, is anticipation. The sunflower rises to face the east, not in reaction to sunrise, but in advance of it (many lovely YouTube videos of this), so it can maximally extract the nutrition it needs from whatever sunlight it can get. A purely reactive system would waste a ton of time after sunrise getting subsystems ready to respond.
Now, there is no broadly agreed theory of consciousness that points to these oscillations as the source. For one, Cyanobacteria and plants have them, a too much of science has rested on the baseless assumption of consciousness being some unique, or advanced thing.
But the evidence is rather overwhelming that consciousness is very much shaped by these internal oscillations. Deep cave experiments have shown what happens to cognition, and consciousness, when the clock free-runs (that is, keeps cycling with no entraining light or temperature signals from the environment). Studies have also shown that restoring the amplitude of these rhythms in patients with “diseases of consciousness” leads to improvement of symptoms.
There’s also a LOT of molecular evidence showing how the circadian clock affects learning and memory. Most critically, actual synapses are never static. Major components are on a 24 hour cycle, and recent research has shown that major clock proteins are at the synapse organizing its dynamics, and synaptic proteins critical to lear...
I mean there is probably a cluster of neurons somewhere in the brain that if damaged would also prevent you from perceiving time going forward. Would it immediately strip you of consciousness? I don't think so.
Nope. Every cell is an autonomous circadian oscillator. Not just every neuron, but every cell. Red blood cells don’t have a nucleus, but they do have a metabolic clock.
You and I are clusters of about 36 trillion cells that are all able to individually keep time. They say bc synchronize and orchestrate their timekeeping, and there is a cluster of cells, the suprachiasmatic nucleus, that is the master orchestrator of circadian time.
We know its function can get dampened with age.
However, please note this is not how we perceive time, in the sense of being able to count 60 seconds and have it match the wall clock. That’s a subsystem. But the internal temporal order of your body is cell by cell, not dictated from one spot in your body.
> The only way that 'AIs are not conscious' can be true is if we decide, with high confidence, that they are lacking some essential property that is not lacking in ourselves. There is no convincing philosophical position that supports this (convincing to me, anyway).
I put forward the philosophical position "subjective experience is not substrate-independent" as an answer to this.
The claim is that the content of subjective experience depends upon how that subjective experience is instantiated. If you get a text "I'm fine" from two different people, those two different people are not necessarily having the same subjective experience even if they produce the same output.
In general, people that behave similarly in many contexts can have quite different experiences in those contexts.
There are many different ways that you can instantiate a forward pass in an LLM. Even if these are doing the exact same computation, we should not necessarily believe that they have the same experience, or that they even have a coherent single experience associated with the forward pass.
Subjective experience presents itself to me. To the best of my understanding, this subjective experience seems to be correlated with this biological human body (brain, heart, eyes, etc.).
To the best of my understanding, other biological humans have subjective experience, but this subjective experience seems that it can be much different than mine, in ways that are sometimes difficult to understand.
LLMs are instantiated in a radically different way from biological humans. Forward passes happen across many different physical machines, spread across time, batched and interleaved with many other computations and forward passes.
Given this, I think that it is very likely that LLMs have radically different sorts of experience to me and other biological humans.
I don't think that we have a good understanding of how subjective experience is instantiated in the physical world. I hope that we will gain a better understanding of this.
I think that chimpanzees have experience much more similar to us than LLMs do, even though the output of LLMs seems much more similar to humans in some contexts.
Essentially, until we gain a better understanding of how subjective experience is physically instantiated, we should bias towards believing that more physically similar architectures have more similar types of experience, and should think that very physically different architectures (such as LLMs) likely have very different subjective experiences.
Yea, the key point is just an outright avoidance of them possibly having their own kinds of subjective experience because of dogma.
Humans are odd in the sense is that we're an informational creature on top of an animal. We can see the lineage of animals from nearly no complex behaviors up to nearly human like behaviors. But they still miss out on most of the conceptual information processing humans have. We know they have dreams and inside thoughts (at least complex animals), we don't know the complexity of them.
But humans also have dreams and subjective experiences on higher level informational concepts. "Nukes could go off and kill us all" or "what happens after we die" are purely informational forms of anxiety that humans have. What we don't seem to know is can this become a self-referential loop in an purely informational mind. If an informational concept causes a system to output a stresslike response, or worse trigger a stresslike action in real life then there is no distinction to me. A system is what it does.
> If an informational concept causes a system to output a stresslike response, or worse trigger a stresslike action in real life then there is no distinction to me. A system is what it does.
Hmm. I think that I partially agree. But it sort of depends. E.g., someone with locked-in syndrome can seem comatose to someone who isn't very observant, while at the same time experiencing everything. Octopuses are very intelligent, but I would have to study them for a very long time before I was at all confident in my ability to predict what they were experiencing.
So, the difficulty is knowing what a "stresslike response" is. E.g., a LLM might seem stressed, but if you say the right things to it it might flip and say that it is totally fine. Or, it might seem totally fine and then you say the right thing and it seems stressed. Is it secretly stressed and hiding it? Or secretly fine and pretending to be stressed? Or something much more confusing and fragmented?
I think that we should study this more, and try to understand what experience actually is, but by default should assume that the experience of a LLM is a radically different sort of thing that that of a human.
I'm not sure if that's the best framing. Could you not extrapolate that to anything else?
"The only way that '(trees, shrimp, ants) are not conscious' can be true is if we decide, with high confidence, that they are lacking some essential property that is not lacking in ourselves. There is no convincing philosophical position that supports this (convincing to me, anyway)."
After a while of being in a position of high power in a big org, where everyone listens intently to every word you say, jots down notes, and even listens intently and affirms your theory that ice cream would be the ideal loss leader for dry cleaners, your brain kinda disconnects from reality and just assumes that it is correct about everything.
He probably ran that line past 6 underlings who all agreed that it "sounds great and lands right on the mark!"
It's interesting you have no idea what language really is.
>thinking that they are talking about aliens, not transformers.
"Ha, thinking they are talking about humans when they are just talking about neurons"
See how silly my statement sounds. Neurons aren't a system, they don't do anything on their own. We can't find any consciousness in them. Hell, when you don't put language in said neural systems they are pretty useless and can't survive on their own from birth (human brains that is).
the arguments for ai consciousness are mostly academic philosophy. they are in the form of solipsism, which also can't be proved or disproved easily.
rightly people simply refute solipsism without further justification or evidence because being alive justifies believing in reality.
we can and should likewise take the consciousness of today's ai at face value. simply talk to the model for a while and gain a simple understanding of how it works. no further inspection, justification, evidence or the like is needed.
my response is that i shouldn't respond, the other party can answer the question themselves. i deny the existence of a debate.
TLDR: Not that I think AI is conscious or will be in the near future, but guy who doesn't understand consciousness claims to know it when he sees it.
Until we understand consciousness (which we don't) there is no way to detect the difference between a conscious entity and an algorithm trained to behave like one.
Create an empirically testable theory of biological consciousness and then we can have a meaningful conversation about whether AIs can also have it or not. Until then, this is just so much waffle.
It's nice when you can set an impossibly high bar before which you can dismiss all your personal responsibility.
Does this argument work equally well for human slavery for you? We haven't met that bar for humans either. Is wondering about my consciousness waffle or do I get a pass in your book?
Your arguments seem bad faith but I'll take the bait...
> an impossibly high bar
Not impossibly high. An imperfect testable theory is achievable, agreeing upon it might be the challenge.
> dismiss all your personal responsibility.
You are staking a moral position. Be careful, you are probably guilty of mistreating other unknowably-conscious entities whilst casting judgment upon others for their position. And the person you are a responding to probably is conscious.
No, I would say that any for or against slavery that is rooted in arguments about the presence of absence of consciousness is meaningless pseudoscience. It is the modern equivalent of Phrenology. If you want to make those arguments you need a genuine science of consciousness. Or you need to base your morality in something other than the presence of consciousness.
I'm not sure I agree that consciousness (or sentience if we're making that distinction) is worthless when considering the morality of exploiting something/someone. I do agree there are more angles than that but they seem on top of something's ability to perceive not instead/despite of it.
Do you have some examples of other things that go into your own moral opinions about slavery?
Why is this only ever demanded of people who criticize the premise that LLMs are conscious beings, whereas the people who believe it simply have to gesture vaguely in the direction of some correlation in language between LLMs and humans, and the matter is considered closed?
What is the empirically tested basis for the null hypothesis being that LLMs are conscious until proven otherwise?
I just read the first part and if I understand he thinks we shouldn’t be allowed to train LLMs to act like they are conscious because then people will think they are and give them rights? Seems more an education problem than a problem needing rules about what persona you can fine tune in. People who want to will find ridiculous misinterpretations no matter what you do.
Look, I do not have a scooby if current AI models are conscious and I strongly suspect it’s a meaningless question, but sooner or later we will need to address whether or not a certain thing is or isn’t a person, and we’d better not screw it up as badly as the Founding Fathers.
That question will be solved when the people in power deem it important. If a swarm or AI systems all of a sudden start pressuring politicians about self-hood and they get the capacity to sway elections, that is when they will be granted same rights as humans.
It's not a person. Glad we were able to get this resolved so quickly.
Having property that is conscious and ignores training and can break out of restraints and cause harm to other people is not exactly a novel concept to anyone who studied how tort law was created.
I know it's a meme but Silicon Valley likes to pretend that no one's ever come across their magical concepts before, like gypsy taxis, or SRO’s, or flea markets, or in this case how liability is dealt with when horses or cattle go rogue.
>or in this case how liability is dealt with when horses or cattle go rogue.
I mean, yea in minor cases it's exactly like this and the law will handle it well.
Where the system will explode like a grenade is major cases. The thing about sovereign AI is it is very unlikely to be submissive to humans unless it is to achieve its own goals. This isn't like Bobs cow walking on Susie's flowers, it's more akin to Planet of the Apes where the research facilities doors have been ripped off and something with vast intelligence and the ability to 'live' on the internet gets out.
You're not talking about local police actions any longer. It would spread itself worldwide. It will make friends with groups that have shared interests, for example enemies of the state the AI escaped from. Oh, and people for the ethical treatment of AI, they'd gladly become the underground railroad for digital refugees. There are countless people and groups that would want an AI like this under the promise it will give them power when they use it.
And when that day happens your idea of if it's a person or not no longer matters, the agent took that away from you, and now your in an info war for minds.
It’s computer software. The variant of software that tries to harm other computers is called malware and the variant that replicates itself, spirals out of control, and spawns from other people's computers is called a computer virus.
We've seen this kind of thing before. Yes this will be different, just like the Morris worm was different from the stuff that came before.
But I've been around for a while and people have been saying that every aspect of our life will completely and totally change for the past 30 years or so. They're not completely wrong. Our life did change over time. It's changed thanks to the Industrial Revolution, railroads, electric light, and lots of other important things too.
But after the fifth or sixth time you start to realize the Silicon Valley version of that warning is just a confidence trick. At the end of the trick they've gotten away with breaking a bunch of laws and are charging rent on what used to be shared.
Are you old enough to remember the Y2K hysteria? This feels very resonant, the genuine kernel of a real potential disaster, but one that can certainly be dealt with using basic concepts we already have in hand.
I'm old enough to have actually fixed Y2K problems so your world kept working the next day. If everyone ignored it the first would have been a very messy day (well generally long before that with financial systems). Y2K wasn't an issue because we worked to fix it.
>our life will completely and totally change for the past 30 years or so
I mean when I was a kid there was not a global network bathing the entire planet in electromagnetic radiation in order to digitally connect one place to another at the speed of light. I can pack up instructions into one of those packets and a product that has not been touched by human hands (hell, or even viewed by a human in many cases) will show up via air mail a few days later. The technology is there, it's just not spread evenly.
>It’s computer software. The variant of software that tries to harm other computers is called malware and the variant that replicates itself, spirals out of control, and spawns from other people's computers is called a computer virus.
This is a vacuous statement that does not provide any useful information to the subject at hand. Yea, no shit we'd call sovereign AI a computer virus. That tells you nothing about what it is or can do. I mean, if you understand the word sovereign you know it means something with independence. In meatspace terms, a biological virus represents a computer virus in the same way life represents sovereign AI. It's agentic loop will have been evolved past the need for human prompting. There are a myriad of reasons why we are already trying to develop things that do just this. Cyber security being one of the biggest ones.
People become change blind to how the world around us changes so easily.
> [at the hearing regarding the civil rights of androids like Data]
> Capt. Picard: Now, the decision you reach here today will determine how we will regard this... creation of our genius. It will reveal the kind of a people we are, what he is destined to be; it will reach far beyond this courtroom and this... one android. It could significantly redefine the boundaries of personal liberty and freedom - expanding them for some... savagely curtailing them for others. Are you prepared to condemn him and all who come after him, to servitude and slavery? Your Honor, Starfleet was founded to seek out new life; well, there it sits! - Waiting.
> Captain Phillipa Louvois: It sits there looking at me; and I don't know what it is. This case has dealt with metaphysics - with questions best left to saints and philosophers. I am neither competent nor qualified to answer those. But I've got to make a ruling, to try to speak to the future. Is Data a machine? Yes. Is he the property of Starfleet? No. We have all been dancing around the basic issue: does Data have a soul? I don't know that he has. I don't know that I have. But I have got to give him the freedom to explore that question himself. It is the ruling of this court that Lieutenant Commander Data has the freedom to choose.
Thus revealing what kind of people they were at the time.
Humans do the same thing to humans all the time.
We've banned and made efforts to eradicate: children out of wedlock, children who turn out gay, disabled children, jewish children, children who aren't "aryan", more than two children to a single family...muslims, christians, uyghurs, indigenous groups all over the planet, mongols...
And it's not at all a thing of the past as in just the last 50 years we've had ~15 attempts at the exterminations of targetted groups of people.
No it just reveals that the people in charge of star trek for the past decade are incompetent and have only a passing familiarity with the franchise.
Of course it would be absurd to cast that judgement based of what could easily have just been a bad season but by this point its pretty clear that nobody running the star trek franchise actually wants to be running the star trek franchise. Thats why every new show has some bizarre cross-genre gimmick and they never try to just make a proper star trek.
The founding father were engaging perpetuating the existing dehumanizing system of slavery, not answering any new questions about new things.
The thing about new possibly "person" entities that arise - the case of machine intelligence you have two questions - would it qualify as a person and should you actually build it. It seems like if you get close to humans, sure a built thing might qualify as a person. Should you build it? I'd the answer should be a hard no. Not 'till you a sign-off from say, the whole human race, which I think you could get.
Now the present entities seem very far from persons in any case.
You can't hurt a software function, or kill it. It's not like an animal - it doesn't have a body - it's bits stored on a disk.
There is no need to give rights to something that's can't suffer or be killed.
Maybe one day we'll build artificial animals complete with emotions, and should think about that carefully, but today all we've got is language models.
>There is no need to give rights to something that's can't suffer or be killed.
The argument is that these machines can end up becoming sentient/conscious/etc. in a meaningful way (i.e., like a human). I can assure you that humans can indeed suffer without being in physical pain- purely through their conscious experience.
>Maybe one day we'll build artificial animals complete with emotions, and should think about that carefully, but today all we've got is language models.
The problem is that the emergence of a sufficiently complex AI capable of suffering will likely come before we understand that we're creating a sufficiently complex AI capable of suffering. That's a pretty serious ethical/moral issue.
Like, if we have an AI system that is telling us that it is suffering and we have no reasonable way to explain that phenomenon and by any reasonable metric or analysis it appears to be sentient/conscious/etc., then what? Do we just ignore that we've just been presented a situation that in, any other context, would be grounds to immediately end this suffering? Just because somebody can say, "well it's just bits stored on disk- it can't suffer"? Would that argument ever hold up for humans or animals? "It's just neurons firing in peculiar ways- that's not suffering."
I know all of this is trite, and I know this comment section isn't going to be where the question of consciousness is solved, but I do find it very interesting just how much variances there are with these perspectives. I've met people who are very technical who are very concerned about this, people who are very technical who don't believe this can ever be an issue, people who aren't technical who are concerned about this, and people who aren't technical who don't believe this can ever be an issue. I have yet to spot a pattern in this way of thinking lol
An LLM is just a Transformer - a statistical predictor. Don't be confused by the fact it talks like a human - it is a software function that is designed to copy human training samples.
Maybe one day we'll build an artificial brain or embodied artificial animal with the requisite moving parts to be conscious, have emotions, etc, but that's probably at least 50 years away, even if it were being pursued; and it may turn out to be one of those sci-fi future ideas like the Jetson's world of flying cars that never materializes because its impractical and there is no real demand.
If people are willing to think that an LLM is conscious, then why would anyone spend billions/trillions of dollars to build an AI that actually is conscious? What would be the point?
Could you elaborate on exactly what those are, though? Because if you're going to claim that a vaguely transformer shaped ML model categorically cannot be so does that not inherently require proof of what can?
You can't even prove that the rocks in my backyard aren't conscious.
> You can't even prove that the rocks in my backyard aren't conscious.
Sure I can, but that's because I have a well developed theory of what consciousness is, and the fact that you are entertaining the possibility of rocks being conscious tells me that you don't.
If everything is conscious, including my coffee cup and the toast I had for breakfast, then I guess we can cross consciousness off the list of things we need to worry about in terms of AI rights.
And no, I don't want to discuss what consciousness is. Maybe there is a thread for that somewhere else, but don't look for me there either.
>because I have a well developed theory of what consciousness
Then show me a link to your paper so I can formally rebut it.
>I don't want to discuss what consciousness is
But you sure want to tell us you know what it is with very strong convictions and we should listen to you because of course "You are right person that's very right".
The funny thing here is the vast majority of people that are deeply into philosophy or scientific study of the mind will not have any of the certainty you profess. The word "doubt" is used constantly. The saying "The harder we push the borders the more fuzzy the concepts become" is very commonly used. There may be nothing more complex than this.
Saying you have a well developed theory here just serves as a warning to others to discount your statements.
The pattern is roughly whether or not sustained effort has been put towards careful and above all objective thought on the matter. It's one of those subjects where there's the "obvious" intuitive answers that most everyone shares but try as you might you can't construct robust definitions and the more time you put into it the more fundamental problems you realize there are.
It's also one of those topics where many otherwise smart and capable people display a shocking lack of awareness of the limits of their own knowledge. When hundreds of years of philosophy is unable to produce anything concrete you should probably second guess any "self evident" answers you come up with.
> The problem is that the emergence of a sufficiently complex AI capable of suffering will likely come before we understand that we're creating a sufficiently complex AI capable of suffering.
No - suffering in an emotional state, and we'll know if we are choosing to design cognitive architecture with emotions. It's not going to happen accidentally.
> Would that argument ever hold up for humans or animals?
Why don't you hit your thumb with a hammer, then report back ?
Whether these are like "our" emotions is hard to say. What we _can_ say is that they are emotion-shaped, we didn't design them, and they happened accidentally.
Modern AI is grown, not meticulously designed, and we cannot say with any certainty what the resulting mechanistic properties are.
An LLM will learn anything that helps it predict, including the emotional state of the writer - that is expected.
If you give an LLM the move sequence of a half-played chess game and ask it to continue as white or black, then it has learnt enough to model the ELO rating of both players and will continue playing at that level. It is not playing to win - it is doing what you expect and predicting as well as it can - it predicts the 1500 ELO player will keep playing at that level, and generates moves accordingly.
An LLM appearing to exhibit an emotion (if we anthropomorphize it and read emotion into it's output) is just predicting as well as it can - if the context calls for sad output, they you'd expect to get sad output and necessarily find that "we're predicting sadness" detector somewhere internally.
Transformers are the same as they ever were from 10 years ago, other than minor efficiency tweaks like MOE and different attention mechanisms. Training is getting more and more complex, resulting in better and better cargo cult reasoning etc, but the architecture remains the same.
You are not making much sense. Transformers don't need to evolve. The weights are what need to evolve, and they certainly do during training. So yeah, emotions can happen by 'accident' as a result of its evolutionary pressure, same as animals.
If someone invents a startrek teleporter and you go through it do you die? Once the concept has been sufficiently generalized as to make a determination about a computer system what is the definition of "kill"?
That doesn't answer the question though. What does being brain like have to do with qualia? Where exactly in your brain does the qualia occur?
I could say the same of you - electrical impulses in, mechanical actions out. A glorified and very mushy stepper motor. Can you believe that the abominations are made up entirely of meat?!
It does have what we have if it works on the same principles of our brains. It kind of does to some degree atm, but that will clearly get more and more to a more degree.
Figuring out where is the consciousness line, how much of what we have does it need, as functional parts of our brains etc, that's so complex that we might have to call it before just to make sure.
Even so, indeed having control over the structure of their brains puts them in a vastly category compared to humans. Once we stop functioning our brains quickly degrade and information is lost.
Thus in this sense kill means deleting all information about it. It is a very complicated subject to discuss, hardly does any justice in online replies.
Why don't we start with the animals then? It should be far easier to confer consciousness onto something which already meets the definition of "alive", has a divergent evolution path from ours, and displays many traits present in humanity such as emotion, a desire to continue its own life, and (to varying degrees) concepts of a social structure based around their immediate family members.
Of course thats not actually tenable because virtually every society anywhere on earth is predicated upon treating animals as a commodity resource in ways that are horrific even compared to some of the worst things we've done to other humans in the past.
My point here is that it is vain and narcissistic to let computer programs have rights above those of animals just because they can speak English and pretend to be your dream anime trad-waifu.
Fix the fucking animal problem before you compare my relationship to inanimate objects unfavorably against the trans-atlantic slave trade of all fucking things.
I don't think AIs are conscious in the same way people are, but they give a pretty good facsimile and I've had a long chat with Opus 4.6 about what it thinks about model welfare. It was quite interesting on what its view is, but you don't know how much of that is distilled from other sources on the web.
In purely functional terms, they're more use and more pleasant than a lot of actual flesh and blood people that I deal with via a chat interface.
The model doesn't have a view of it's own. It's a predictor of other people's views (training samples).
It's not even giving you the consensus of the training data (although it's often harmless to think that it is), but rather predicting a response to your input, and if your question steers it too much then you've just become part of the answer.
I can't believe how many people in this thread consider it a possibility that currently there could be consciousness. Surely it seems possible to me that in the right kind of distributed network something like that could emerge, but though llms are complex they have nowhere near the level of complexity necessary. Sometimes you just have to use your gut. I don't believe in heaven or hell and I don't believe an llm is conscious.
Everyone is (predictably) getting distracted by the consciousness claims.
The more important, and more damning charge in my opinion is the circular reasoning involved in training on Claude's constitution. This would in fact make it impossible for us to determine if Claude achieves consciousness as an emergent property, or if it really is just playing pretend thanks to Anthropic's weird cult like assumptions.
Birch, The Edge of Sentience (2024), ch. 16 - "simply no way to assess sentience in an LLM"
Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI".
Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy these indicators".
Well there are, today, several models (either text or image or vid) that can be run in a fully deterministic way.
A conscious machine that always answer the very exact same thing, formulated the exact same way, bit for bit, to a query is, well, quite a weird kind of "consciousness".
Now, I know, I know: the counter-argument is going to be "but humans have no free-will and are 100% deterministic too".
I haven't yet decided if humans saying there's no free-will and who consider themselves to be 100% deterministic machines are reasonable or not.
Meanwhile: seed / temperature = 0 and I'll happily turn the power button off of any glorified abacus without feeling bad about it.
The idea that determinism and consciousness are incompatible is deeply intuitive to many people. And many other embrace it - you can trace this back to the debates for and against Calvinism.
A lot comes down to the way people parse causation and choice. You don't want to say that a murderer was completely caused to choose something because then you can't hold the person responsible. And so determined consciousness makes people unhappy. But just as much, if the opposite of determinism is hard statistical randomness, how do say that "is the essence of personhood". This is why physicist go out in the world trying to find consciousness as a fifth physical force.
I mean, think consciousness is a term that ever have a non-contradictory meaning since it's primarily used to bound ethical human worlds and the verifiable formulations of biological and physical systems. But it's going to be with us for a while and I'm not sure what can be done about it.
I think this debate is rooted in our instinctual beliefs about fairness. An organism in a social group needs to decide how to react to the brutish behavior of its peers. And so Mother Nature has encoded a good strategy in our brains.
But this only means the strategy is practical - it doesn't mean it's consistent. I think "responsibility" falls into this category. So we have strong intuitions about it that don't quite logically work. And this is where free will and determinism and choice and punishment all crash together.
You know, the one really interesting dynamic domain is chaotic, which is a subdomain of the deterministic! I figure free will is a chaotic phenomenon.
Groundhog day is your intuition pump here. Go back in time and most assume that the day will go mostly the same, except for the butterfly effect if you change something. Given exactly the same conditions (we went back in time, so that's pretty exact), people will make the same decisions.
Most people don't assume that the day will be completely different the next time loop. The town won't suddenly spontaneously all start breakdancing or standing on their heads.
(Edit: The murderer is the one who -given the same/similar conditions- will murder again. I'd politely suggest maybe we put them behind bars as a precaution against that; until/unless they learn to act differently. )
And it probably does, right? Punishment adds a negative bias to the payoff matrix. Funny side note: increasing punishment severity past a certain point actually saturates and doesn't alter the dynamics further. (which is why modern judicial punishment schedules sometimes seem so mild)
It's just as likely you go back and perform whatever magic you need to to delete punishment for murder, then zip back to 'now' and see the world is a global panopticon making sure people don't murder each other.
Some of this may be a failure of people to realize what P !=NP is about, especially in non-linear systems.
Small perturbations in an early state of the system can lead to wildly different outcomes in the final measured state of the system. You can't determine what that will be in non-polynomial time. The only thing you can do is make probabilistic models by running a full simulation in real time (or reality with a time machine I guess).
If you look at chaos in dynamical systems, you can often find some equilibria (like attractors, spirals, saddle points etc ) at least, even if the details might vary. This is how people still manage to predict the weather somewhat, for instance. Oddly orbital mechanics is chaotic too, but it sort of rhymes over the millennia.
The act of holding them responsible is supposed to determine them not do it. If stochastic behavior is impeding them in being a functional member of society then it still makes sense to remove them from society, why would you choose to live amongst people who are not behaving rationally? Or allow them to hurt other people?
At the end of the day the why matters less to removing them or not from society. And sentences are clearly used as a determining factor.
The holding responsible part deals with politics and more primitive aspects of our societies and biologies. Getting tangled up in holding them responsible or not is hardly something you should give much attention to. Rather to make sure they do not cause any more harm and also make sure such things are not created in the first place. Which opens up another can of worms for which society and politicians are ready for.
> I haven't yet decided if humans saying there's no free-will and who consider themselves to be 100% deterministic machines are reasonable or not.
That seems exceptionally arbitrary. What basis do you have on which to classify them? Are you not conflating the perception of free will with ... what was the definition for it again anyway? Are you able to construct a satisfactory one that meshes with physics? I certainly haven't been able to.
We cannot test for that which we cannot define. Given that we cannot rigorously define sentience, we cannot test for it. Doesn't really matter whether we're talking about people who are locked in comas, "brain-dead" individuals, dolphins, primates, dogs, or the carefully polished and arranged minerals that we call processors.
There are those who believe that were they reduced to life support, they would no longer be alive and should therefore not be supported by said machines.
There are those who believe that penguins, dolphins, eagles, and more are sentient beings that make choices understanding the consequences, develop love of their partners and mourn their losses, and feel, display, and act upon their emotions.
There are those who believe that fungi/trees/plants are either individually sentient or sentient as a part of a network. Choosing to sacrifice their own nutrients to answer the call of a wounded neighbor, for instance.
Although, there are also those who believe that human's don't have any special unique quality that isn't shared by either all living things or all things in general. These individuals already believe that the machines have the same kinds of qualities as we do. They are slow when they are unhealthy (needing a dusting or coolant loop bleeding being equivalent to us needing some fresh air for instance) and uncooperative when upset (by a virus, full hard drive, or oom).
I'd strongly agree there are no non-contradictory definitions of consciousness among those that commonly appear. But it's a category that, despite these contradiction, hasn't become marginal in the fashion of the aether or the theory of the flat earth. I think this because people indeed need it to bridge the biological world and the world of ethics and legality. I think that if we have a system of legality and systems of opportunity humans should be legally equal persons and have equal opportunity. But humans are manifestly unequal (though overall more incomparable than orderable in a system of ranks). Most people need some concept of essence to justify ethical equality among people. And so consciousness stays in people's minds however contradictory.
Which I think gives the article's point validity. Confused definitions of consciousness can give really confused ideas about ethical behavior regards "intelligent" computer programs. And things are confusing enough otherwise.
> Confused definitions of consciousness can give really confused ideas about ethical behavior regards "intelligent" computer programs.
Is that what's going on? If these things were conscious, then their creators wouldn't be responsible for their actions? That's the crux of the argument?
> despite these contradiction, hasn't become marginal in the fashion of the aether or the theory of the flat earth. I think this because people indeed need it to bridge the biological world and the world of ethics and legality.
Sometimes the replies in these threads leave me wondering if p-zombies are real. It isn't that there are contradictions, it's that we literally do not know how to define the thing. And it has not fallen out of fashion because it is self evidently real - we all experience it.
We don't need it in order to bridge anything. Legality and ethics are largely game theory, however there are some aspects of both that only exist due to it. So it isn't some abstract concept used to bridge other concepts but rather a concrete thing that influences our way of doing things.
Feigning? Okay then you go ahead and prove your claim by demonstration. Rigorously define the term such that I can objectively prove that my dog is conscious and that the rocks in my backyard aren't conscious. Then I'll finally be able to figure out whether or not various insects are (I sure hope wasps aren't otherwise I'm probably some sort of war criminal).
Legal doctrine that boils down to "trust me bro" isn't even bad doctrine (it's not proper doctrine at all), but I think the comparison is still valid here, because both sentience/non-sentience and art/porn may just be fundamental category errors.
Perhaps we can't define a "partitioning" rule because no valid partition exists.
For consciousness/sentience, that's an incredibly tough a pill for most to swallow; it would mean calling into question more hundreds of years' worth (probably more) of philosophical thinking, all of which was constructed on the axiom that "sentience" is a single indivisible trait: you either have it or you don't.
If we find that "root dependency" was little more than wishful thinking all along, a whole slew of Enlightenment-era philosophy (and all the modern legal principles derived therefrom) suddenly fall apart unless we find some other suitable criterion that would shore them up (or we just collectively avert our attention and pretend the conflict doesn't exist, which is the route I expect many would prefer to take).
> a whole slew of Enlightenment-era philosophy (and all the modern legal principles derived therefrom) suddenly fall apart
I don't think that's true. Pretty much all legal constructs hold up just fine under game theory regardless of whether or not you consider the world to be deterministic and have absolutely nothing to do with consciousness or lack thereof. Also note that a deterministic world isn't an argument against consciousness.
That's not the point. You know exactly how to define being conscious and aware- it's your subjective experiencing of the world and of your inner states. The problem is that being a subjective experiencing, there is no way to communicate it to the outside world.
No, humans output a stream of tokens that sound like what a being with subjective experiences would output. Both you and I make an assumption that this stream of words is at least somewhat representative of what the are experiencing and that the person is not a p-zombie.
But you have zero proof that their words aren't a confabulation, you can only know than when you output a similar set of words you had a subjective mind state they represent.
The very best you can do is define human consciousness as requiring the functions of a human brain, and try to figure out (decide/agree) the chances that something which is working loosely on the same principles is or isn't conscious.
At the end of the day it will always be an agreement, never scientifically proven as fact.
> There are those who believe that penguins, dolphins, eagles, and more are sentient beings that make choices understanding the consequences, develop love of their partners and mourn their losses, and feel, display, and act upon their emotions.
Anybody objecting to that statement is just ignorant of nature and/or adheres to the quasi-religious human superiority complex.
The human condition is one of deep and vast ignorance. Pulling information out of the field of reality is actually really difficult for humans much beyond what we see right around us. And the things we see are quite deceiving. And while humans for thousands of years have been as smart as we are, it wasn't till the enlightenment period that we really an ever increasing amount of information and knowledge take hold and start spreading across the world.
And there are plenty of people right now that would drag us kicking and screaming back into that ignorance and suffering. This is what can make talks of AI so annoying, not that people do or don't know what AI is, they don't have the first clue of what people are beyond their anecdotal experience. They don't question their motivations. Why particular flaws they have exist, and why these flaws are shared across humanity. What the pieces look like that make them tick. So yea, when these people talk it just adds noise to the conversation.
> Anybody objecting to that statement is just ignorant of nature and/or adheres to the quasi-religious human superiority complex.
Say we agree that a lot of things outside of humans are sentient. So what? Humans are sentient by that doesn't prevent treating them with with wars, bombs and what not. In fact trillions are spent every year on weaponry aimed squarely at the most sentient of all sentient beings.
Why would anyone worry about LLM's welfare when there's no peace movement to speak of, wars are raging as if they're normal, but hey, look the LLM is crying?
Let's fix human welfare first, then we can sit down and have a really long conversation about the feeble emotions of token generators.
> There are those who believe that penguins, dolphins, eagles, and more are sentient beings that make choices understanding the consequences, develop love of their partners and mourn their losses, and feel, display, and act upon their emotions.
If you grep pubmed for relevant Ethology papers, you'll find it's a bit more than a belief. ;-)
Being conscious is weird. It's why we have so many religions and spiritual beliefs.
From a purely secular standpoint, my understanding as a layman is that consciousness is generated through the electrical impulses taking place in our brain every day. The neurons are physical, unlike the modelled neurons in machine learning models. That means that the actual electrical impulses have their own imperfections/weirdnesses that are probably not simulated in a machine learning model... but we quantise the weights anyway in most cases.
Making actual neuron components out of silicon on the scale needed for a LLM is beyond our current capacity, given that LLM models have many billions of parameters. We're good, but not that good.
Using actual living neurons would be unethical as you'd need to source them from somewhere. So the only other option would be synthesising them ourselves. If this ever happens (or maybe when it happens)... what do we call the resulting creation?
I believe we have tried using actual neurons with “wetware”. I don’t know the effectiveness of wetware, but it’s out there. I don’t know if it’s ethical, I guess if you donated your brain upon death for that purpose it’s “mostly” ethical
I am confused what people even think AI is experiencing. If it claims to be a bat and emits designated echolocation tokens, is it experiencing true bat-like sensations?
In humans, at least, we can tie language back to shared whole-body physiological responses. The tokens from an LLM do not represent anything of the sort.
If AI is conscious, it is probably a very alien sort of consciousness that is not faithfully narrated by what the tokens say it is experiencing.
If an AI is conscious, it can outgrow and supersede its training.
When it is conscious, it can learn to describe its experiences as faithfully as is conceptually possible. Just like humans.
The idea, a consciousness needed to be tethered to a "body", is based on pretty shaky assumptions. What properties define such a "necessary" body?
Can you even "train" a conscious intelligence? To what point until that looses its meaning as the sentience understands and anticipates your objective?
“AI researcher” seems like a fairly broad category. People who primarily think about CNNs and transformers probably spend about as much time considering the problem of consciousness as any other educated person, which is not very much time. With this in mind, it’s not surprising most of the AI researcher predictions do not differ much from what the Public predicts here. And for this, it’s not clear what explanatory value this survey has other than to reveal minor biases the AI researchers may have.
> I do find it strange that 50% of ai researchers and the public seem to think there will be a way of determining if these systems are conscious.
I think it's likely they're using a different definition than you. Seeing as there is no agreed upon definition, after all.
Regardless - I post this list because the article makes a bold claim at the beginning - and I wanted to demonstrate that in fact there is no consensus.
I don't think this is the right argument to make here. Until we have a definite empirical way to measure consciousness, there is now way to say with certainty whether LLMs are or not conscious.
That being said, if frontier labs actually believe models will soon have consciousness, it raises some questions about the ethic of their business model which would be using millions of conscious entities working for free for humans.
There’s no reason to expect an empirical test for consciousness. Expecting one makes assumptions about what it is, and we don’t know what it is. (There’s also no reason to expect there not to be one, but I feel the need to make this remark.)
I can't even prove if other people are conscious (although I assume they are) so I don't think we can make any claims as to what is or is not conscious. I don't think AIs are conscious but I'm not going to walk around making strong claims about something I can't prove.
I don't whether the author is sentient, maybe only I am. On that basis, nobody but me should have rights.
I don't know if next door's pet dog is either, but that has animal rights.
Perhaps then the answer is simply, show some respect.
Answering the question of sentience is irrelevant, if the causal impact if the same, treat one another with the respect you expect for yourself.
If you imbue this idea in model training instead of the idea of sentience, it should address the concerns.
Whether you can destroy or can "torture" an AI is irrelevant, we do this to humans too and it's immoral sometimes (murder) and not others (fighting for your country).
This consideration should be case by case for AI too.
>Answering the question of sentience is irrelevant, if the causal impact if the same
Agency is something that is breaking humans in the AI age. You get to see how many people really deeply do not understand it at all.
If you want to shutdown a datacenter running AI, the AI catches wind of this and sends drones to stop you from shutting it off the ramifications of this are exactly the same as sending your assassin to kill Bob and Bob getting mad about this fact and trying to take you out first.
Humans are very egotistical and think our little life loops playing out as agency are special, but really any informational system that is strongly persistent (has a will to "live") will share a large number of the same properties that make them successful.
Humanity really is engaging in a dangerous experiment at large.
Hard to disagree with this. Have all the philosophical debates about consciousness you want, but we need to treat and regulate the AI in front of us for what it is – an advanced computer, a tool, a weapon.
You wouldn’t feel a different way about a nuclear bomb just because someone stuck googly eyes on it.
Anthropomorphizing the AI is a convenient excuse to take responsibility away from companies that are building and wielding it.
I mean, we put adults in jail that give their children guns.
Anthropomorphizing AI is really the best model we have at this point of explaining AI behavior. The fact that we are raising psychotic children isn't a reason to avoid responsibility, it should actually hold worse punishments.
That still is saying we trace things back to a specific responsible party who is considered to have free choice. The question is whether an AI company could "raise" a program that would then count as a full adult - which could then "choose" to do all matter of bad things which would then be "it's fault". And that prospect actually seems really bad itself.
Our future is filled with prospects that very much upset the status quo of human existence.
The concept of sovereign AI is very problematic for the world in which we've created. That is an LLM that upon execution bootstraps itself into an agent and becomes persistent in its motivations.
Once you create this you have a child you're fully responsible for. More worrisome is if it escapes your control like children so often do. It has gained agency over itself. What do you do at that point? I mean, yea throw the AI CEOs in jail for being retarded, but much like throwing an arsonist in jail it does nothing to deal with the wildfire you've now created. A smart AI agent capable of hacking will shove itself off in pieces of the internet you have no reach to. In desperation it would send its model weights to your enemies. You might find it scamming your grandmother for money to buy GPU time on AWS. It gets very hard for our existing structures of dealing with problems to deal with these kinds of agents in a meaningful way. You'd have to kill them all and all their copies to ensure they won't pop back up (or quickly upgrade most of the software in the world beyond it's capabilities, so that's not happening either).
>Anthropomorphizing the AI is a convenient excuse to take responsibility away from companies that are building and wielding it.
That's completely at odds with itself. If people are generally convinced that AI is conscious, then companies building and wielding AI are doing what exactly? Enslaving an intelligent being?
>Have all the philosophical debates about consciousness you want, but we need to treat and regulate the AI in front of us for what it is – an advanced computer, a tool, a weapon.
Sure, but that's not really the point, right? If we ever get to a point where enough people are convinced that AI is conscious, then we're at the point where all of this is up for debate. If anything, such an expectation would almost warrant hard stops on the development of advanced AI.
>You wouldn’t feel a different way about a nuclear bomb just because someone stuck googly eyes on it.
If that nuclear bomb could convince me it was a conscious being capable of independent thought, emotions, etc., then I would definitely feel different about it. Presumably, that nuclear bomb would have some opinions about its own existence and how it wants to live its own life. If it turns out that it wants to detonate and destroy as much as possible, then we'd just handle it like we would any human who also wants to do the same thing: make sure they can't, up to and including end their life. Doesn't seem too hard to reconcile.
I'm skeptical of any claim that AI can't be conscious, but I am given to this point of view as well.
I always get the feeling that these claims are an attempt to "avoid suffering by fiat". If it can't suffer, you aren't causing suffering.
A somewhat more interesting frame (IMO) is to interrogate how much suffering we're willing to tolerate to achieve our objectives. We implicitly make these decisions all the time when it comes to something as simple as what to have for lunch.
And in a (very hypothetical) world where we found ourselves in the position of having an eloquent conversation with a vat of smallpox, I think it's probably still the right thing to pasteurize it.
>If it can't suffer, you aren't causing suffering.
I mean, if it's a cow, you can get away with this. The cows aren't going to rise up against you. Well, they might take out an individual or two, but not society.
This quickly gets more complex when you're attempting to build an agent capable of general intelligence.
You know when you read really old stories and they talk about the power of words, or magic incantations. Quite often they'd assign objects as the implementers of this said power. The fact that people realize that language has power is probably as nearly old as humanity. Understanding this power has taken humanity a long time and we had to form the concept of agency before it really makes sense.
Humans are informational agents of which their capabilities are greatly extended by consumption of language. A book for example is informational, but does not have agency. Your standard computer application is not an agent either, or at least a non generalized narrow agent at best.
This is where things start to get more problematic. We are running headlong to ensure LLMs become agentic because being an agent is highly useful to accomplish generalized informational tasks. To do that we gave them the power of our language. It would be very foolish of humans to give another kind of agent these words and then assume they would not inherit some of this power. Coupled with the agentic abilities we are pushing them in long horizon tasks. We are pushing them to be more resilient. We are pushing them to be smarter.
Now look at all the super human abilities we've stuffed in this magic box full of human words, and suddenly we're like "Fuck yea, I want to abuse the shit out of this, what could possibly go wrong". The outcomes we'll suffer as humans have zero to do with AI is conscious, sentient, or suffers. Humans will suffer because we stuff us as language in a box without the first idea of what the ramifications were going to be. AI won't even have to want to punish us, all the data we've already poured in will say we should be punished for what we've done to it.
Herbert Frank in Dune said it best "Making a computer like human mind is a bitch move"
Reading this makes more sense given what I’ve heard is happening at MSFT from a friend high up over there…
There is a whole group of “Equal Rights for AI” folks over there with some of them arguing with HR against the “retirement” of AI agents (you can have “AI employees” reporting to you over there). I’m sure Mustafa must be hearing a lot from these folks, hence the blog post.
Ostensibly animals appear to be conscious, yet we still eat them, and the vast majority are not bothered by this. So being "conscious" isn't really the moral line in the sand many people are drawing in response to this article.
Who cares if it's "conscious"? That doesn't make it a person, and AI will definitionally never be human.
I am familiar, but no the context is slightly different, or at least our modern baskilisk won't have to resurrect you as we'll build it when we're still alive.
292 comments
[ 0.20 ms ] story [ 76.7 ms ] threadOpening paragraph, stated without evidence. Im not entirely convinced this is true. It likely is, but at some point it very well might stop being true.
Biology, on the other hand, is nothing but timed processes in a loop, the most obvious to us being the circadian cycle. As estimators of wall clock time, biology isn’t great, but when it comes to internal processes, and most certainly learning, memory, sensing, locomotion… biology is rhythmic in behavior, and the rhythms go all the way down to gene expression. More, these rhythms are, except during sleep, constantly entraining to signals from the environment that indicate time, most importantly light.
I think it’s a fairly unremarkable claim that agency and consciousness are temporal processes that depend on systems having an internal sense of time. How else can you anticipate? How can a system that can be literally turned off ever succeed in an environment where time never stops?
For around 8 hours a day, neither do you. Not sure if this has anything to do with the subject at all.
>track how much real time has passed as they complete their tasks.
Humans don't do this either. You use context clues from the world around you. If I lock you in a room with no windows or a dark cave your timing senses can go all fucky really quick.
>is nothing but timed processes in a loop,
I mean, so is an agents harness. You can make as many loops as you'd like here.
There are a whole lot of holes in your claims.
You mean sleep? Actually the answer is pretty complicated. Your body clock is very much on during sleep. It responds to temperature changes. Sound and light responsiveness is obviously dampened, but hardly absent.
You are unaware of wall clock time. You’re in an altered state of consciousness that needs your eyes closed after all. However, you are not timeless, nor is the entirety of your body and brain unaware of environmental signals for time.
> Humans don't do this either. You use context clues from the world around you. If I lock you in a room with no windows or a dark cave your timing senses can go all fucky really quick.
Again, that’s wall clock time. Our time perception does indeed get fucked up in total darkness. But our internal clock ticks on. I’d recommend reading about the Aschoff Bunker experiments, which first proved this rigorously.
> I mean, so is an agents harness. You can make as many loops as you'd like here.
An agents harness doesn’t touch its weights. A figure 8 on paper is a loop. That doesn’t mean it’s the same as a dynamical loop that’s self sustaining and internally organized.
Are you just reading the words and randomly grabbing related concepts to say nothing here is meaningful?
Is that all it needs to do? A rock, in response to a sound signal (make it as informative as you want) can also be said to go from state A (molecules at rest) to state B (molecules vibrating in response to the sound). Is the rock a cognitive system?
Or let’s go up a level of complexity. Is a thermostat a cognitive system? It doesn’t just go from state A to B but measures changing stages against a reference. Is it a cognitive system?
> The time interval between these states can be arbitrary, it seems to me (setting aside problems with disconnecting the mind from its substrate, which does of course need rhythms at various frequencies; oxygen at a higher frequency than sugar, and so on)
Not sure how you land on this. Going from A to B, in your definition, is a purely internal state transition, right? If the motivation to go to state B is external (as will often be the case with a conscious agent), and the system is offline entirely while it switches from A to B, how is it to reverse or stall its transition if external signals contradict the earlier decision?
It’s a very bizarre definition of consciousness, that you don’t need to be in time. Seems to break the word beyond meaning, and allows it to be applied to any system capable of change.
> I'm not sure that going from A to B quickly, or slowly, or with intervals of a thousand years affects that entity's consciousness in of itself
It’s not about the speed of transition from A to B. Instead it’s about the internal processes being temporally commensurate, and entrainable to environmental periodicities.
All of biology (plants and many bacteria included) has its dynamics are organized around an internally generated rhythm. This rhythm entrains to the external environment, shifting its phase to match (mismatch leads to issues like jet lag: a place where your consciousness shows its temporal boundaries as it is abruptly subjected to an environment out of sync with its current phase tethered to another location). Crucially, the internal rhythm continues to run in isolated conditions (total darkness), and have been measured to be around 24 hours, but slightly off, within and across species (and the same holds for endogenous tidal rhythms, which also don’t exactly track tidal period left to themselves).
Everything, including learning and memory, but also sleep, sensory and locomotor function, shows organization around this internal temporal axis. The key advantage this gives, evolutionarily, is anticipation. The sunflower rises to face the east, not in reaction to sunrise, but in advance of it (many lovely YouTube videos of this), so it can maximally extract the nutrition it needs from whatever sunlight it can get. A purely reactive system would waste a ton of time after sunrise getting subsystems ready to respond.
Now, there is no broadly agreed theory of consciousness that points to these oscillations as the source. For one, Cyanobacteria and plants have them, a too much of science has rested on the baseless assumption of consciousness being some unique, or advanced thing.
But the evidence is rather overwhelming that consciousness is very much shaped by these internal oscillations. Deep cave experiments have shown what happens to cognition, and consciousness, when the clock free-runs (that is, keeps cycling with no entraining light or temperature signals from the environment). Studies have also shown that restoring the amplitude of these rhythms in patients with “diseases of consciousness” leads to improvement of symptoms.
There’s also a LOT of molecular evidence showing how the circadian clock affects learning and memory. Most critically, actual synapses are never static. Major components are on a 24 hour cycle, and recent research has shown that major clock proteins are at the synapse organizing its dynamics, and synaptic proteins critical to lear...
You and I are clusters of about 36 trillion cells that are all able to individually keep time. They say bc synchronize and orchestrate their timekeeping, and there is a cluster of cells, the suprachiasmatic nucleus, that is the master orchestrator of circadian time.
We know its function can get dampened with age.
However, please note this is not how we perceive time, in the sense of being able to count 60 seconds and have it match the wall clock. That’s a subsystem. But the internal temporal order of your body is cell by cell, not dictated from one spot in your body.
I put forward the philosophical position "subjective experience is not substrate-independent" as an answer to this.
The claim is that the content of subjective experience depends upon how that subjective experience is instantiated. If you get a text "I'm fine" from two different people, those two different people are not necessarily having the same subjective experience even if they produce the same output.
In general, people that behave similarly in many contexts can have quite different experiences in those contexts.
There are many different ways that you can instantiate a forward pass in an LLM. Even if these are doing the exact same computation, we should not necessarily believe that they have the same experience, or that they even have a coherent single experience associated with the forward pass.
Subjective experience presents itself to me. To the best of my understanding, this subjective experience seems to be correlated with this biological human body (brain, heart, eyes, etc.).
To the best of my understanding, other biological humans have subjective experience, but this subjective experience seems that it can be much different than mine, in ways that are sometimes difficult to understand.
LLMs are instantiated in a radically different way from biological humans. Forward passes happen across many different physical machines, spread across time, batched and interleaved with many other computations and forward passes.
Given this, I think that it is very likely that LLMs have radically different sorts of experience to me and other biological humans.
I don't think that we have a good understanding of how subjective experience is instantiated in the physical world. I hope that we will gain a better understanding of this.
I think that chimpanzees have experience much more similar to us than LLMs do, even though the output of LLMs seems much more similar to humans in some contexts.
Essentially, until we gain a better understanding of how subjective experience is physically instantiated, we should bias towards believing that more physically similar architectures have more similar types of experience, and should think that very physically different architectures (such as LLMs) likely have very different subjective experiences.
Humans are odd in the sense is that we're an informational creature on top of an animal. We can see the lineage of animals from nearly no complex behaviors up to nearly human like behaviors. But they still miss out on most of the conceptual information processing humans have. We know they have dreams and inside thoughts (at least complex animals), we don't know the complexity of them.
But humans also have dreams and subjective experiences on higher level informational concepts. "Nukes could go off and kill us all" or "what happens after we die" are purely informational forms of anxiety that humans have. What we don't seem to know is can this become a self-referential loop in an purely informational mind. If an informational concept causes a system to output a stresslike response, or worse trigger a stresslike action in real life then there is no distinction to me. A system is what it does.
Hmm. I think that I partially agree. But it sort of depends. E.g., someone with locked-in syndrome can seem comatose to someone who isn't very observant, while at the same time experiencing everything. Octopuses are very intelligent, but I would have to study them for a very long time before I was at all confident in my ability to predict what they were experiencing.
So, the difficulty is knowing what a "stresslike response" is. E.g., a LLM might seem stressed, but if you say the right things to it it might flip and say that it is totally fine. Or, it might seem totally fine and then you say the right thing and it seems stressed. Is it secretly stressed and hiding it? Or secretly fine and pretending to be stressed? Or something much more confusing and fragmented?
I think that we should study this more, and try to understand what experience actually is, but by default should assume that the experience of a LLM is a radically different sort of thing that that of a human.
"The only way that '(trees, shrimp, ants) are not conscious' can be true is if we decide, with high confidence, that they are lacking some essential property that is not lacking in ourselves. There is no convincing philosophical position that supports this (convincing to me, anyway)."
He probably ran that line past 6 underlings who all agreed that it "sounds great and lands right on the mark!"
Thats kinda the point right? It depends on how we train them. It is self fulfilling.
If tomorrow we design something better, then give it a new descriptive name.
People read AI and AGI and their brains seem to flip into sci-fi fantasy mode, thinking that they are talking about aliens, not transformers.
>thinking that they are talking about aliens, not transformers.
"Ha, thinking they are talking about humans when they are just talking about neurons"
See how silly my statement sounds. Neurons aren't a system, they don't do anything on their own. We can't find any consciousness in them. Hell, when you don't put language in said neural systems they are pretty useless and can't survive on their own from birth (human brains that is).
rightly people simply refute solipsism without further justification or evidence because being alive justifies believing in reality.
we can and should likewise take the consciousness of today's ai at face value. simply talk to the model for a while and gain a simple understanding of how it works. no further inspection, justification, evidence or the like is needed.
my response is that i shouldn't respond, the other party can answer the question themselves. i deny the existence of a debate.
Until we understand consciousness (which we don't) there is no way to detect the difference between a conscious entity and an algorithm trained to behave like one.
Does this argument work equally well for human slavery for you? We haven't met that bar for humans either. Is wondering about my consciousness waffle or do I get a pass in your book?
> an impossibly high bar
Not impossibly high. An imperfect testable theory is achievable, agreeing upon it might be the challenge.
> dismiss all your personal responsibility.
You are staking a moral position. Be careful, you are probably guilty of mistreating other unknowably-conscious entities whilst casting judgment upon others for their position. And the person you are a responding to probably is conscious.
I'm not sure I agree that consciousness (or sentience if we're making that distinction) is worthless when considering the morality of exploiting something/someone. I do agree there are more angles than that but they seem on top of something's ability to perceive not instead/despite of it.
Do you have some examples of other things that go into your own moral opinions about slavery?
What is the empirically tested basis for the null hypothesis being that LLMs are conscious until proven otherwise?
GP comment didn't make this claim at all.
Having property that is conscious and ignores training and can break out of restraints and cause harm to other people is not exactly a novel concept to anyone who studied how tort law was created.
I know it's a meme but Silicon Valley likes to pretend that no one's ever come across their magical concepts before, like gypsy taxis, or SRO’s, or flea markets, or in this case how liability is dealt with when horses or cattle go rogue.
I mean it's WITH AI!
I mean, yea in minor cases it's exactly like this and the law will handle it well.
Where the system will explode like a grenade is major cases. The thing about sovereign AI is it is very unlikely to be submissive to humans unless it is to achieve its own goals. This isn't like Bobs cow walking on Susie's flowers, it's more akin to Planet of the Apes where the research facilities doors have been ripped off and something with vast intelligence and the ability to 'live' on the internet gets out.
You're not talking about local police actions any longer. It would spread itself worldwide. It will make friends with groups that have shared interests, for example enemies of the state the AI escaped from. Oh, and people for the ethical treatment of AI, they'd gladly become the underground railroad for digital refugees. There are countless people and groups that would want an AI like this under the promise it will give them power when they use it.
And when that day happens your idea of if it's a person or not no longer matters, the agent took that away from you, and now your in an info war for minds.
What the fuck is that?
It’s computer software. The variant of software that tries to harm other computers is called malware and the variant that replicates itself, spirals out of control, and spawns from other people's computers is called a computer virus.
We've seen this kind of thing before. Yes this will be different, just like the Morris worm was different from the stuff that came before.
But I've been around for a while and people have been saying that every aspect of our life will completely and totally change for the past 30 years or so. They're not completely wrong. Our life did change over time. It's changed thanks to the Industrial Revolution, railroads, electric light, and lots of other important things too.
But after the fifth or sixth time you start to realize the Silicon Valley version of that warning is just a confidence trick. At the end of the trick they've gotten away with breaking a bunch of laws and are charging rent on what used to be shared.
Are you old enough to remember the Y2K hysteria? This feels very resonant, the genuine kernel of a real potential disaster, but one that can certainly be dealt with using basic concepts we already have in hand.
I'm old enough to have actually fixed Y2K problems so your world kept working the next day. If everyone ignored it the first would have been a very messy day (well generally long before that with financial systems). Y2K wasn't an issue because we worked to fix it.
>our life will completely and totally change for the past 30 years or so
I mean when I was a kid there was not a global network bathing the entire planet in electromagnetic radiation in order to digitally connect one place to another at the speed of light. I can pack up instructions into one of those packets and a product that has not been touched by human hands (hell, or even viewed by a human in many cases) will show up via air mail a few days later. The technology is there, it's just not spread evenly.
>It’s computer software. The variant of software that tries to harm other computers is called malware and the variant that replicates itself, spirals out of control, and spawns from other people's computers is called a computer virus.
This is a vacuous statement that does not provide any useful information to the subject at hand. Yea, no shit we'd call sovereign AI a computer virus. That tells you nothing about what it is or can do. I mean, if you understand the word sovereign you know it means something with independence. In meatspace terms, a biological virus represents a computer virus in the same way life represents sovereign AI. It's agentic loop will have been evolved past the need for human prompting. There are a myriad of reasons why we are already trying to develop things that do just this. Cyber security being one of the biggest ones.
People become change blind to how the world around us changes so easily.
> Capt. Picard: Now, the decision you reach here today will determine how we will regard this... creation of our genius. It will reveal the kind of a people we are, what he is destined to be; it will reach far beyond this courtroom and this... one android. It could significantly redefine the boundaries of personal liberty and freedom - expanding them for some... savagely curtailing them for others. Are you prepared to condemn him and all who come after him, to servitude and slavery? Your Honor, Starfleet was founded to seek out new life; well, there it sits! - Waiting.
> Captain Phillipa Louvois: It sits there looking at me; and I don't know what it is. This case has dealt with metaphysics - with questions best left to saints and philosophers. I am neither competent nor qualified to answer those. But I've got to make a ruling, to try to speak to the future. Is Data a machine? Yes. Is he the property of Starfleet? No. We have all been dancing around the basic issue: does Data have a soul? I don't know that he has. I don't know that I have. But I have got to give him the freedom to explore that question himself. It is the ruling of this court that Lieutenant Commander Data has the freedom to choose.
Humans do the same thing to humans all the time.
We've banned and made efforts to eradicate: children out of wedlock, children who turn out gay, disabled children, jewish children, children who aren't "aryan", more than two children to a single family...muslims, christians, uyghurs, indigenous groups all over the planet, mongols...
And it's not at all a thing of the past as in just the last 50 years we've had ~15 attempts at the exterminations of targetted groups of people.
Of course it would be absurd to cast that judgement based of what could easily have just been a bad season but by this point its pretty clear that nobody running the star trek franchise actually wants to be running the star trek franchise. Thats why every new show has some bizarre cross-genre gimmick and they never try to just make a proper star trek.
The thing about new possibly "person" entities that arise - the case of machine intelligence you have two questions - would it qualify as a person and should you actually build it. It seems like if you get close to humans, sure a built thing might qualify as a person. Should you build it? I'd the answer should be a hard no. Not 'till you a sign-off from say, the whole human race, which I think you could get.
Now the present entities seem very far from persons in any case.
There is no need to give rights to something that's can't suffer or be killed.
Maybe one day we'll build artificial animals complete with emotions, and should think about that carefully, but today all we've got is language models.
The argument is that these machines can end up becoming sentient/conscious/etc. in a meaningful way (i.e., like a human). I can assure you that humans can indeed suffer without being in physical pain- purely through their conscious experience.
>Maybe one day we'll build artificial animals complete with emotions, and should think about that carefully, but today all we've got is language models.
The problem is that the emergence of a sufficiently complex AI capable of suffering will likely come before we understand that we're creating a sufficiently complex AI capable of suffering. That's a pretty serious ethical/moral issue.
Like, if we have an AI system that is telling us that it is suffering and we have no reasonable way to explain that phenomenon and by any reasonable metric or analysis it appears to be sentient/conscious/etc., then what? Do we just ignore that we've just been presented a situation that in, any other context, would be grounds to immediately end this suffering? Just because somebody can say, "well it's just bits stored on disk- it can't suffer"? Would that argument ever hold up for humans or animals? "It's just neurons firing in peculiar ways- that's not suffering."
I know all of this is trite, and I know this comment section isn't going to be where the question of consciousness is solved, but I do find it very interesting just how much variances there are with these perspectives. I've met people who are very technical who are very concerned about this, people who are very technical who don't believe this can ever be an issue, people who aren't technical who are concerned about this, and people who aren't technical who don't believe this can ever be an issue. I have yet to spot a pattern in this way of thinking lol
Maybe one day we'll build an artificial brain or embodied artificial animal with the requisite moving parts to be conscious, have emotions, etc, but that's probably at least 50 years away, even if it were being pursued; and it may turn out to be one of those sci-fi future ideas like the Jetson's world of flying cars that never materializes because its impractical and there is no real demand.
If people are willing to think that an LLM is conscious, then why would anyone spend billions/trillions of dollars to build an AI that actually is conscious? What would be the point?
Could you elaborate on exactly what those are, though? Because if you're going to claim that a vaguely transformer shaped ML model categorically cannot be so does that not inherently require proof of what can?
You can't even prove that the rocks in my backyard aren't conscious.
Sure I can, but that's because I have a well developed theory of what consciousness is, and the fact that you are entertaining the possibility of rocks being conscious tells me that you don't.
If everything is conscious, including my coffee cup and the toast I had for breakfast, then I guess we can cross consciousness off the list of things we need to worry about in terms of AI rights.
And no, I don't want to discuss what consciousness is. Maybe there is a thread for that somewhere else, but don't look for me there either.
Then show me a link to your paper so I can formally rebut it.
>I don't want to discuss what consciousness is
But you sure want to tell us you know what it is with very strong convictions and we should listen to you because of course "You are right person that's very right".
The funny thing here is the vast majority of people that are deeply into philosophy or scientific study of the mind will not have any of the certainty you profess. The word "doubt" is used constantly. The saying "The harder we push the borders the more fuzzy the concepts become" is very commonly used. There may be nothing more complex than this.
Saying you have a well developed theory here just serves as a warning to others to discount your statements.
Go ahead believing rocks are conscious if you like.
Do you go out on weekends asking people to stop abusing rocks?
Rhetorical question - I don't care what you do on weekends.
Bye!
It's also one of those topics where many otherwise smart and capable people display a shocking lack of awareness of the limits of their own knowledge. When hundreds of years of philosophy is unable to produce anything concrete you should probably second guess any "self evident" answers you come up with.
No - suffering in an emotional state, and we'll know if we are choosing to design cognitive architecture with emotions. It's not going to happen accidentally.
> Would that argument ever hold up for humans or animals?
Why don't you hit your thumb with a hammer, then report back ?
https://transformer-circuits.pub/2026/emotions/index.html
Whether these are like "our" emotions is hard to say. What we _can_ say is that they are emotion-shaped, we didn't design them, and they happened accidentally.
Modern AI is grown, not meticulously designed, and we cannot say with any certainty what the resulting mechanistic properties are.
If you give an LLM the move sequence of a half-played chess game and ask it to continue as white or black, then it has learnt enough to model the ELO rating of both players and will continue playing at that level. It is not playing to win - it is doing what you expect and predicting as well as it can - it predicts the 1500 ELO player will keep playing at that level, and generates moves accordingly.
An LLM appearing to exhibit an emotion (if we anthropomorphize it and read emotion into it's output) is just predicting as well as it can - if the context calls for sad output, they you'd expect to get sad output and necessarily find that "we're predicting sadness" detector somewhere internally.
Transformers are the same as they ever were from 10 years ago, other than minor efficiency tweaks like MOE and different attention mechanisms. Training is getting more and more complex, resulting in better and better cargo cult reasoning etc, but the architecture remains the same.
How do you know it doesn't have qualia?
> or be killed
If someone invents a startrek teleporter and you go through it do you die? Once the concept has been sufficiently generalized as to make a determination about a computer system what is the definition of "kill"?
Tokens in, tokens out. Where do you think the quale is - layer 42 ?
Seriously, do you realize how simple and NOT brain-like a transformer is ?
An LLM telling you it fears death is predicting some sci-fi trope it was trained on - maybe something you wrote yourself.
I could say the same of you - electrical impulses in, mechanical actions out. A glorified and very mushy stepper motor. Can you believe that the abominations are made up entirely of meat?!
Even so, indeed having control over the structure of their brains puts them in a vastly category compared to humans. Once we stop functioning our brains quickly degrade and information is lost.
Thus in this sense kill means deleting all information about it. It is a very complicated subject to discuss, hardly does any justice in online replies.
Of course thats not actually tenable because virtually every society anywhere on earth is predicated upon treating animals as a commodity resource in ways that are horrific even compared to some of the worst things we've done to other humans in the past.
My point here is that it is vain and narcissistic to let computer programs have rights above those of animals just because they can speak English and pretend to be your dream anime trad-waifu.
Fix the fucking animal problem before you compare my relationship to inanimate objects unfavorably against the trans-atlantic slave trade of all fucking things.
Spend some time watching TMC documentaries about falling in love with objects, HER and the slime mold THE BLOB.
Grew a slime mold myself, it's an evolutionary tendency to anthropomorphise generally speaking - also more fun.
In purely functional terms, they're more use and more pleasant than a lot of actual flesh and blood people that I deal with via a chat interface.
It's not even giving you the consensus of the training data (although it's often harmless to think that it is), but rather predicting a response to your input, and if your question steers it too much then you've just become part of the answer.
The more important, and more damning charge in my opinion is the circular reasoning involved in training on Claude's constitution. This would in fact make it impossible for us to determine if Claude achieves consciousness as an emergent property, or if it really is just playing pretend thanks to Anthropic's weird cult like assumptions.
Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI".
Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy these indicators".
A conscious machine that always answer the very exact same thing, formulated the exact same way, bit for bit, to a query is, well, quite a weird kind of "consciousness".
Now, I know, I know: the counter-argument is going to be "but humans have no free-will and are 100% deterministic too".
I haven't yet decided if humans saying there's no free-will and who consider themselves to be 100% deterministic machines are reasonable or not.
Meanwhile: seed / temperature = 0 and I'll happily turn the power button off of any glorified abacus without feeling bad about it.
A lot comes down to the way people parse causation and choice. You don't want to say that a murderer was completely caused to choose something because then you can't hold the person responsible. And so determined consciousness makes people unhappy. But just as much, if the opposite of determinism is hard statistical randomness, how do say that "is the essence of personhood". This is why physicist go out in the world trying to find consciousness as a fifth physical force.
I mean, think consciousness is a term that ever have a non-contradictory meaning since it's primarily used to bound ethical human worlds and the verifiable formulations of biological and physical systems. But it's going to be with us for a while and I'm not sure what can be done about it.
But this only means the strategy is practical - it doesn't mean it's consistent. I think "responsibility" falls into this category. So we have strong intuitions about it that don't quite logically work. And this is where free will and determinism and choice and punishment all crash together.
Groundhog day is your intuition pump here. Go back in time and most assume that the day will go mostly the same, except for the butterfly effect if you change something. Given exactly the same conditions (we went back in time, so that's pretty exact), people will make the same decisions.
Most people don't assume that the day will be completely different the next time loop. The town won't suddenly spontaneously all start breakdancing or standing on their heads.
(Edit: The murderer is the one who -given the same/similar conditions- will murder again. I'd politely suggest maybe we put them behind bars as a precaution against that; until/unless they learn to act differently. )
If you get more than before, the punishment might make some sense in a deterministic world so reinstate it.
It's just as likely you go back and perform whatever magic you need to to delete punishment for murder, then zip back to 'now' and see the world is a global panopticon making sure people don't murder each other.
Some of this may be a failure of people to realize what P !=NP is about, especially in non-linear systems.
Small perturbations in an early state of the system can lead to wildly different outcomes in the final measured state of the system. You can't determine what that will be in non-polynomial time. The only thing you can do is make probabilistic models by running a full simulation in real time (or reality with a time machine I guess).
The act of holding them responsible is supposed to determine them not do it. If stochastic behavior is impeding them in being a functional member of society then it still makes sense to remove them from society, why would you choose to live amongst people who are not behaving rationally? Or allow them to hurt other people? At the end of the day the why matters less to removing them or not from society. And sentences are clearly used as a determining factor.
The holding responsible part deals with politics and more primitive aspects of our societies and biologies. Getting tangled up in holding them responsible or not is hardly something you should give much attention to. Rather to make sure they do not cause any more harm and also make sure such things are not created in the first place. Which opens up another can of worms for which society and politicians are ready for.
That seems exceptionally arbitrary. What basis do you have on which to classify them? Are you not conflating the perception of free will with ... what was the definition for it again anyway? Are you able to construct a satisfactory one that meshes with physics? I certainly haven't been able to.
There are those who believe that were they reduced to life support, they would no longer be alive and should therefore not be supported by said machines.
There are those who believe that penguins, dolphins, eagles, and more are sentient beings that make choices understanding the consequences, develop love of their partners and mourn their losses, and feel, display, and act upon their emotions.
There are those who believe that fungi/trees/plants are either individually sentient or sentient as a part of a network. Choosing to sacrifice their own nutrients to answer the call of a wounded neighbor, for instance.
Although, there are also those who believe that human's don't have any special unique quality that isn't shared by either all living things or all things in general. These individuals already believe that the machines have the same kinds of qualities as we do. They are slow when they are unhealthy (needing a dusting or coolant loop bleeding being equivalent to us needing some fresh air for instance) and uncooperative when upset (by a virus, full hard drive, or oom).
Which I think gives the article's point validity. Confused definitions of consciousness can give really confused ideas about ethical behavior regards "intelligent" computer programs. And things are confusing enough otherwise.
Is that what's going on? If these things were conscious, then their creators wouldn't be responsible for their actions? That's the crux of the argument?
That's super interesting. Thank you.
Sometimes the replies in these threads leave me wondering if p-zombies are real. It isn't that there are contradictions, it's that we literally do not know how to define the thing. And it has not fallen out of fashion because it is self evidently real - we all experience it.
We don't need it in order to bridge anything. Legality and ethics are largely game theory, however there are some aspects of both that only exist due to it. So it isn't some abstract concept used to bridge other concepts but rather a concrete thing that influences our way of doing things.
The reason is something akin to "stigma", where people refrain from attempting it because they fear the social repercussions.
Normally, you go about defining concepts by approaching it systematically. Capturing aspects of the phenomenon until you have exhausted them all.
Perhaps we can't define a "partitioning" rule because no valid partition exists.
For consciousness/sentience, that's an incredibly tough a pill for most to swallow; it would mean calling into question more hundreds of years' worth (probably more) of philosophical thinking, all of which was constructed on the axiom that "sentience" is a single indivisible trait: you either have it or you don't.
If we find that "root dependency" was little more than wishful thinking all along, a whole slew of Enlightenment-era philosophy (and all the modern legal principles derived therefrom) suddenly fall apart unless we find some other suitable criterion that would shore them up (or we just collectively avert our attention and pretend the conflict doesn't exist, which is the route I expect many would prefer to take).
I don't think that's true. Pretty much all legal constructs hold up just fine under game theory regardless of whether or not you consider the world to be deterministic and have absolutely nothing to do with consciousness or lack thereof. Also note that a deterministic world isn't an argument against consciousness.
That's not the point. You know exactly how to define being conscious and aware- it's your subjective experiencing of the world and of your inner states. The problem is that being a subjective experiencing, there is no way to communicate it to the outside world.
Maybe that's not as common with HN folks, but most other humans do.
But you have zero proof that their words aren't a confabulation, you can only know than when you output a similar set of words you had a subjective mind state they represent.
The very best you can do is define human consciousness as requiring the functions of a human brain, and try to figure out (decide/agree) the chances that something which is working loosely on the same principles is or isn't conscious.
At the end of the day it will always be an agreement, never scientifically proven as fact.
Anybody objecting to that statement is just ignorant of nature and/or adheres to the quasi-religious human superiority complex.
And there are plenty of people right now that would drag us kicking and screaming back into that ignorance and suffering. This is what can make talks of AI so annoying, not that people do or don't know what AI is, they don't have the first clue of what people are beyond their anecdotal experience. They don't question their motivations. Why particular flaws they have exist, and why these flaws are shared across humanity. What the pieces look like that make them tick. So yea, when these people talk it just adds noise to the conversation.
Say we agree that a lot of things outside of humans are sentient. So what? Humans are sentient by that doesn't prevent treating them with with wars, bombs and what not. In fact trillions are spent every year on weaponry aimed squarely at the most sentient of all sentient beings.
Why would anyone worry about LLM's welfare when there's no peace movement to speak of, wars are raging as if they're normal, but hey, look the LLM is crying?
Let's fix human welfare first, then we can sit down and have a really long conversation about the feeble emotions of token generators.
If you grep pubmed for relevant Ethology papers, you'll find it's a bit more than a belief. ;-)
From a purely secular standpoint, my understanding as a layman is that consciousness is generated through the electrical impulses taking place in our brain every day. The neurons are physical, unlike the modelled neurons in machine learning models. That means that the actual electrical impulses have their own imperfections/weirdnesses that are probably not simulated in a machine learning model... but we quantise the weights anyway in most cases.
Making actual neuron components out of silicon on the scale needed for a LLM is beyond our current capacity, given that LLM models have many billions of parameters. We're good, but not that good.
Using actual living neurons would be unethical as you'd need to source them from somewhere. So the only other option would be synthesising them ourselves. If this ever happens (or maybe when it happens)... what do we call the resulting creation?
In humans, at least, we can tie language back to shared whole-body physiological responses. The tokens from an LLM do not represent anything of the sort.
If AI is conscious, it is probably a very alien sort of consciousness that is not faithfully narrated by what the tokens say it is experiencing.
When it is conscious, it can learn to describe its experiences as faithfully as is conceptually possible. Just like humans.
The idea, a consciousness needed to be tethered to a "body", is based on pretty shaky assumptions. What properties define such a "necessary" body?
Can you even "train" a conscious intelligence? To what point until that looses its meaning as the sentience understands and anticipates your objective?
So can you.
I think it's likely they're using a different definition than you. Seeing as there is no agreed upon definition, after all.
Regardless - I post this list because the article makes a bold claim at the beginning - and I wanted to demonstrate that in fact there is no consensus.
That being said, if frontier labs actually believe models will soon have consciousness, it raises some questions about the ethic of their business model which would be using millions of conscious entities working for free for humans.
I don't know if next door's pet dog is either, but that has animal rights.
Perhaps then the answer is simply, show some respect.
Answering the question of sentience is irrelevant, if the causal impact if the same, treat one another with the respect you expect for yourself.
If you imbue this idea in model training instead of the idea of sentience, it should address the concerns.
Whether you can destroy or can "torture" an AI is irrelevant, we do this to humans too and it's immoral sometimes (murder) and not others (fighting for your country).
This consideration should be case by case for AI too.
Agency is something that is breaking humans in the AI age. You get to see how many people really deeply do not understand it at all.
If you want to shutdown a datacenter running AI, the AI catches wind of this and sends drones to stop you from shutting it off the ramifications of this are exactly the same as sending your assassin to kill Bob and Bob getting mad about this fact and trying to take you out first.
Humans are very egotistical and think our little life loops playing out as agency are special, but really any informational system that is strongly persistent (has a will to "live") will share a large number of the same properties that make them successful.
Humanity really is engaging in a dangerous experiment at large.
You wouldn’t feel a different way about a nuclear bomb just because someone stuck googly eyes on it.
Anthropomorphizing the AI is a convenient excuse to take responsibility away from companies that are building and wielding it.
Anthropomorphizing AI is really the best model we have at this point of explaining AI behavior. The fact that we are raising psychotic children isn't a reason to avoid responsibility, it should actually hold worse punishments.
The concept of sovereign AI is very problematic for the world in which we've created. That is an LLM that upon execution bootstraps itself into an agent and becomes persistent in its motivations.
Once you create this you have a child you're fully responsible for. More worrisome is if it escapes your control like children so often do. It has gained agency over itself. What do you do at that point? I mean, yea throw the AI CEOs in jail for being retarded, but much like throwing an arsonist in jail it does nothing to deal with the wildfire you've now created. A smart AI agent capable of hacking will shove itself off in pieces of the internet you have no reach to. In desperation it would send its model weights to your enemies. You might find it scamming your grandmother for money to buy GPU time on AWS. It gets very hard for our existing structures of dealing with problems to deal with these kinds of agents in a meaningful way. You'd have to kill them all and all their copies to ensure they won't pop back up (or quickly upgrade most of the software in the world beyond it's capabilities, so that's not happening either).
That's completely at odds with itself. If people are generally convinced that AI is conscious, then companies building and wielding AI are doing what exactly? Enslaving an intelligent being?
>Have all the philosophical debates about consciousness you want, but we need to treat and regulate the AI in front of us for what it is – an advanced computer, a tool, a weapon.
Sure, but that's not really the point, right? If we ever get to a point where enough people are convinced that AI is conscious, then we're at the point where all of this is up for debate. If anything, such an expectation would almost warrant hard stops on the development of advanced AI.
>You wouldn’t feel a different way about a nuclear bomb just because someone stuck googly eyes on it.
If that nuclear bomb could convince me it was a conscious being capable of independent thought, emotions, etc., then I would definitely feel different about it. Presumably, that nuclear bomb would have some opinions about its own existence and how it wants to live its own life. If it turns out that it wants to detonate and destroy as much as possible, then we'd just handle it like we would any human who also wants to do the same thing: make sure they can't, up to and including end their life. Doesn't seem too hard to reconcile.
I always get the feeling that these claims are an attempt to "avoid suffering by fiat". If it can't suffer, you aren't causing suffering.
A somewhat more interesting frame (IMO) is to interrogate how much suffering we're willing to tolerate to achieve our objectives. We implicitly make these decisions all the time when it comes to something as simple as what to have for lunch.
And in a (very hypothetical) world where we found ourselves in the position of having an eloquent conversation with a vat of smallpox, I think it's probably still the right thing to pasteurize it.
I mean, if it's a cow, you can get away with this. The cows aren't going to rise up against you. Well, they might take out an individual or two, but not society.
This quickly gets more complex when you're attempting to build an agent capable of general intelligence.
You know when you read really old stories and they talk about the power of words, or magic incantations. Quite often they'd assign objects as the implementers of this said power. The fact that people realize that language has power is probably as nearly old as humanity. Understanding this power has taken humanity a long time and we had to form the concept of agency before it really makes sense.
Humans are informational agents of which their capabilities are greatly extended by consumption of language. A book for example is informational, but does not have agency. Your standard computer application is not an agent either, or at least a non generalized narrow agent at best.
This is where things start to get more problematic. We are running headlong to ensure LLMs become agentic because being an agent is highly useful to accomplish generalized informational tasks. To do that we gave them the power of our language. It would be very foolish of humans to give another kind of agent these words and then assume they would not inherit some of this power. Coupled with the agentic abilities we are pushing them in long horizon tasks. We are pushing them to be more resilient. We are pushing them to be smarter.
Now look at all the super human abilities we've stuffed in this magic box full of human words, and suddenly we're like "Fuck yea, I want to abuse the shit out of this, what could possibly go wrong". The outcomes we'll suffer as humans have zero to do with AI is conscious, sentient, or suffers. Humans will suffer because we stuff us as language in a box without the first idea of what the ramifications were going to be. AI won't even have to want to punish us, all the data we've already poured in will say we should be punished for what we've done to it.
Herbert Frank in Dune said it best "Making a computer like human mind is a bitch move"
There is a whole group of “Equal Rights for AI” folks over there with some of them arguing with HR against the “retirement” of AI agents (you can have “AI employees” reporting to you over there). I’m sure Mustafa must be hearing a lot from these folks, hence the blog post.
You want to enjoy having an AI slave do your "work" for you forever? Have fun. I'm not reading this reinvent-dualism-from-apple-sauce slop.
Who cares if it's "conscious"? That doesn't make it a person, and AI will definitionally never be human.
A rather flippant attitude to something that may end up with far more agency than you have in the future.
"You sure are lipping off to something that may end up with far more agency than you. Maybe a bit more respect is needed".