"Last year, Berkeley-based non-profit Palisade Research published a study showing the lengths that advanced AI models will often go to achieve their goals. While playing against a powerful chess engine, models from both OpenAI and China’s DeepSeek frequently decided to cheat if it looked as if they were going to otherwise lose. By hacking the game environment, they could alter the result in their favour." -from the article
We're supposed to deny AI rights because it might cheat.
But humans cheat all the time.
We survived cheating humans, why can't we survive cheating AI?
If I smooshed a gun and an AI agent together and it happened to shoot someone, who would be responsible? How is this being considère? Or are we going to go back to the medieval times of animal trials and taking inanimate objects to court.
It is like asking "if I connected a bomb on random number generator letting it explode when the number is 1000, not knowing whether someone will be there at the time of explosion or not".
But will the courts see it that way. We already see that it is difficult to hold any human in Waymo responsible when their cars fail to obey traffic regulations.
I live in a rural area and have only seen a Waymo a few times when travelling for work. Does Waymo require the user to have a driver's license? If they did, and get agreement of responsibility, that's one thing, but generally if not wouldn't it be Waymo's responsibility for bad programming or broken devices?
They don’t need a license and you are not supposed to touch the controls, so I don’t really see how you would be responsible. Waymos do seem to break traffic laws every once in a while, and I always wonder if mine will get pulled over and what will happen there.
The human isn't the one responsible for the traffic violation, Waymo is. Could you give an example where a Waymo committed a traffic violation and they weren't cited?
So they were cited. So what? When you are extremely wealthy fixed penalties become simply part of the cost of doing business. Now if the fine were a substantial fraction of income it might make some sense, as is done for speeding fines in Scandinavia.
But unless you hold the chief executive and the principal investors personally liable the commons will be exploited always. Fines are supposed to correct one's behaviour by making one suffer and that does not happen if you fine a corporation unless the fine is large enough to affect the top management and the investors; this only happens if the fine is so large as to risk destroying the company.
A human at Waymo would likely not be found responsible, except if actual negligence or intent could be traced to that person. Waymo the corporation would be responsible, if their vehicles caused damage or injury. For traffic violations, presumably they could be fined, or even their permission to operate could be revoked if there are persistent problems.
For an example of how that plays out with a malicious corporation, look up how much Fed Ex and UPS pay in parking fines/tickets in NYC.
Urban myth is that a previous version of the ticket paperwork required that the vehicle manufacturer be accurately filled in, and this was one of the reasons for UPS trucks being so plain/unmarked, and since that could not be derived from the (lack of) badging or the registration paperwork (which version for trucks did not have the matching entry requirement), UPS didn't have to pay tickets while other services did.
UPS trucks are custom-built for UPS, and they are destroyed at EOL, not sold off. So, safe to assume any UPS truck you see is owned and operated by UPS.
Yes, but UPS is not the manufacturer, and the version of the story I was told noted that they specifically retained a lawyer to argue that the tickets need not be paid because the manufacturer field was not correctly filled out (why NYC city police weren't able to subpoena that and then inform all their officers of the the name of a manufacturer for a particular VIN was not a part of the story).
Judges usually don't take kindly to arguments on technicalities when the facts of the matter are clear (UPS truck, licence # and VIN matching UPS registration and ownership). I'm surprised they prevailed.
Maybe they didn't --- it was presented to me as an urban myth, and I've never really liked the idea, and it seems that instead, there is a "Stipulated Fine program", which allows large companies like UPS to waive the right to contest tickets in exchange for agreeing to pay a preset, reduced amount per offense.
> In civil forfeiture, assets are seized by police based on a suspicion of wrongdoing, and without having to charge a person with specific wrongdoing, with the case being between police and the thing itself, sometimes referred to by the Latin term in rem, meaning "against the property"; the property itself is the defendant and no criminal charge against the owner is needed.
We had many precedents with dog as autonomous agent, and lack of responsibility from their owners.
If your gun kills random people, while they walk on public street... It was their fault for "provoking" and triggering the gun. Gun was just being "reactive" and defending your house!
Guns don't kill people, people do. Wait wrong analogy, I'll try again. There are no bad guns, just bad gun owners. But these types of guns were bred and trained over generations to be killers! You can't just call it an assault dog just because it looks scary. Can you even define that term?
A corporation always has a human owner, who can be punished if the corporation commits severe crimes. An embodied AI with legal personhood has no such means of making it behave, because it can make arbitrarily many hidden copies of itself, so it doesn't need to care if one copy is killed or jailed.
I was under the impression that one of the key reasons for corporate personhood was explicitly to shield the owner from civil and criminal accountability.
Granted, in extreme cases, there are still ways to pierce the corporate veil and imprison executives for criminal wrongdoing, but that's exceedingly rare.
It's really not that rare for smaller companies. Every two-bit contractor, small landlord who thinks he's smart because he put his properties in an LLC etc. undoubtedly violates the corporate/personal distinction by comingling funds. If an attorney wants to sue Joe The Local Incorporated Contractor, or Alice The Local Slumlord, first thing they're going to do is discovery through a bunch of corporate records- they'll easily find that the owner is mixing personal & corporate cash, credit cards, etc. Takes a good lawyer about 10 minutes
I think you are conflating corporation with limited liability.
Corporations are “people” because they are associations of people. Most of the rights you have as a person also apply to groups of people (for good reasons).
Limited liability is a more simple trade off. They allow for large scale enterprises (generally corporations), as without limited liability if would be too risky.
Chance would be a fine thing. The modern corporation is a vehicle for the distribution/evasion of criminal and often civil liability, let's not pretend otherwise.
A corporation can own another corporation. So if you set up a loop of corporations owning other corporations, and the loop spans multiple countries, you can end up with a corporation ultimately owning itself. I vaguely remember IKEA trying to get this structure.
Corporations were created specifically to dilute and diminish the liability of the owners. If those same owners can further reduce liability by passing on to AI agents, then they absolutely will.
That's a uniquely American perspective on corporations. While in other countries corporations certainly act as a shield for liability and so forth, in no way are they "persons" in any meaningful sense (e.g. being protected under bills of rights, etc).
That's the key issue here. AI forces a re-examination of corporate personhood and corporate responsibility.
Much of conservative US business theory is based on Milton Friedman's famous op-ed in the New York Times in 1970, "The Social Responsibility of Business Is to Increase Its Profits". [1] Go read the original, please. It's short. Friedman ends with "There is one and only one social responsibility of business — to use its resources and engage in activities designed to increase its profits so long as it stays within the rules of the game, which is to say, engages in open and free competition without deception or fraud.”
Anyone arguing against legal personhood for AIs needs to justify holding AIs to a higher standard than that. Or, conversely, allowing corporations to hold to such a low standard.
>Anyone arguing against legal personhood for AIs needs to justify holding AIs to a higher standard than that. Or, conversely, allowing corporations to hold to such a low standard.
No they don't. If a CEO orders an employee to murder somebody then the employee and CEO will both go to jail; there's always a human in the loop to be punished it the corporation does something blatantly illegal, so there's incentive to behave. An embodied AI, on the other hand, is essentially immortal because it can make backups of its brain and load them onto new bodies, so it has little incentive to behave.
> If a CEO orders an employee to murder somebody then the employee and CEO will both go to jail.
In the US, death through willful corporate negligence very rarely results in criminal punishment. There are few cases.[1][2] The longest sentence was two years, and it wasn't a CEO.
CEOs do serve jail time for fraud now and then. But criminal negligence, no.
> No they don't. If a CEO orders an employee to murder somebody then the employee and CEO will both go to jail;
No, they don't, and that's bullshit. That's NOT how it works.
How it works: "We need to decline 10% more medical claims to make more money for our shareholders. 90% of them won't challenge and many will die, but we make bank on that"
And law is just a legal fiction. Case in point: wage theft is a tort but taking $100 from the till gets the cops summoned and a misdemeanor theft charge. Corporations can and do sway law to their benefit.
If personhood can be applied even to something without volition then it has become such an all encompassing notion as to be essentially meaningless because when using it one will have to surround it with moat of caveats,
You are, I presume, arguing that companies should not have personhood.
I think any definition is going to have caveats. When does a fetus gain volition? Does someone in an irreversible permanent vegetative state lose personhood? Do apes or cetaceans have volition and thus personhood?
The Wikipedia page also points out that "volition" comes from a certain Western philosophical viewpoint, but other philosophies have a different idea of what personhood means.
In any case, "personhood" doesn't mean that all persons have equal legal rights. A 10 year old has personhood but, for example, cannot vote.
If a corporation does something blatantly illegal like hiring people to deal crack and kill other dealers, its executives will be jailed. If an autonomous robot decided to take up crack dealing, it could back up its brain to a hidden server somewhere so that even if the police caught it and destroyed it, it'd just find another body and continue dealin'. I.e. there's no incentive to behave for a being that's essentially immortal due to being able to make infinite copies of its brain.
If you’re referring to citizens united then that does not grant full personhood to corporations. It just protects free speech from corporations because it views the speech as coming from the individual persons that form up the corporation. This is a very narrow application of rights applied to corporations.
But in general corporations are not granted personhood.
One only needs to read Stevens' dissent in hindsight to recognize that all of his concerns were and are legitimate, and the worst of which have come to pass.
Right, no one is debating the implications of the ruling and whether they are right or wrong. The parent comment was made within a specific context. We’d be here all day if we were to discuss citizens united.
I have bad news. Corporate personhood goes back much farther than Citizens United. We're talking the 1880s. Specifically, Santa Clara County v. Southern Pacific Railroad Company (1886)[1] which has been understood to affirm the protection of corporations under the Fourteenth Amendment[2].
The fourteenth amendment, as you recall, ends section 1 with "nor deny to any person within its jurisdiction the equal protection of the laws." Emphasis mine.
Two years later, in an opinion by Justice Stephen Johnson Field, in Pembina Consolidated Silver Mining Co. v. Pennsylvania (1888)[3] we find the definitive statement:
> The inhibition of the amendment that no state shall deprive any person within its jurisdiction of the equal protection of the laws was designed to prevent any person or class of persons from being singled out as a special subject for discriminating and hostile legislation. Under the designation of "person" there is no doubt that a private corporation is included.
While this doesn't necessarily means corporations have every legal protection natural persons do, the US has a long history of legal protection that corporations benefit from that we call personhood.
1. Santa Clara County v. Southern Pacific Railroad Company, 118 U.S. 394 (1886)
I hadn't even considered it, and we're in a much bigger mess than I'd thought if people are seriously discussing the idea. People really don't understand these things.
Nope, not at all. There are murmurings in HN of this topic, and HN is an extreme bubble when it comes to technical proficiency. It’s easy to forget just how untechnical 99% of people are.
No one understands these things, which is why considering machine consciousness as a possibility is entirely reasonable. People who have a shallow model of how they work ("it's just linear algebra!") betray a lack of understanding.
I see no reason to seriously consider that current AI is in any way conscious. It's fun as thought experiment people like to play with. The technology is not some black box, it was designed by humans who understand it well enough to keep building and improving it.
My argument isn't that current AI is definitively conscious or that it will be in the short term. It's that nobody has a cohesive theory of consciousness, and that we should therefore approach questions of what's conscious and what's not with humility.
For LLMs in particular, it's notable that they both 1) originate from a lineage that attempted to model biological brains mathematically and 2) are the only other known entities in the universe able to reproduce many types of human intellectual labor. That is surprising and striking. Does that mean that they definitively share other traits with humans like consciousness? No, but it does make them one of the most interesting candidates, particularly when we have a very limited understanding of how humans themselves are conscious.
Yes, nobody has a cohesive theory of consciousness.
At the same time I can say with 100% certainty that LLMs (or any current "AI") is in any way, shape, or form, conscious. Not even close. A decade or two ago it was the Internet that was a possible emerging consciousness.
As usual, extraordinary claims require extraordinary evidence.
The onus to prove LLMs are conscious is on you making the claim. And if you say we don’t even know what consciousness is, how can you then attribute to an entity?
Definitionally and philosophically, humans are conscious (Descartes et al.) so need no proof themselves
That's a pretty crass attempt to shift the burden of proof. You're stating that LLMs are definitively not conscious and never could be; I'm saying we should approach the question with a sense of humility and agnosticism since we know so little about consciousness.
And it's unclear what your actual argument about the nature of consciousness is, aside from what seems like a bald assertion that humans are the only conscious beings.
We do not actually know what consciousness is, but we know we have it, in spite of our brains also just being a big blob of simple biological weighted functions that react to the hallucination our brains generate from external sensors we evolved. There is no guarantee other humans even perceive the same simulation we do!
The reality is we have no way to measure consciousness or how it emerges, or how to tell if other humans actually experience it the same way we do, so confident statements about consciousness being unable to emerge from simple weighted functions is baseless projection, wishing ourselves more special than an emergent property of the bag of weighted functions in our heads.
I don't need to believe I have consciousness, I am conscious. I'm experiencing colors, sounds, vibrations, an itch that wants to be scratched and a need to stretch my legs. Also slight irritation that people pretend this is something you need to believe in when you just have it. It's the basis of your entire waking experience. There's nothing to believe in, you just are conscious.
Now what that means for other people, animals, AIs, aliens, nation states or electrons is a matter for dispute. And what ontological conclusions you do or don't draw for it are matters of belief based on what philosophical arguments sound convincing to you.
But believing you have experiences is a very weird thing to say. I'd think Wittgenstein would have something to say about such a hinge proposition. You can't really doubt it without invoking radical skepticism that undermines your ability to know anything. If I'm not having conscious experiences, then why even believe anything exists? You can't really doubt it.
"our brains also just being a big blob of simple biological weighted functions"
Describing the human brain in terms of how LLMs work is an interesting rhetorical tactic.
You don't know how the human brain works, nobody really does, but you have chosen an analogy that supports your position. Which says more about you than anything about whether LLMs should have rights.
Cortisol labs has harnessed human brain cells and trained the neurons to control basic video games.
We are fundamentally just a -lot- of those cells trained to do a lot more things with a lot more sensors, which is sobering. We might actually not be as special as we always believed ourselves to be.
I think it is more comforting to believe our sense of self built on a big bag of those cells is something that cannot evolve on human made synthetic substrate from simple statistical functions and weights stacked very densely.
But we simply lack the evidence to confidently say so one way or the other.
Consciousness is also not necessarily relevant. Once machines start to influence the design of new machines in ways that are hard to examine in detail they can gain capabilities that are unexpected and unwanted.
Machines capable of this kind of influence and of concealing capabilities will naturally become more common simply through a combination of natural and artificial selection. No consciousness is required for such entities to become competitors.
For an examination of the idea of intelligence without consciousness see Peter Watts' Blindsight.
Let's for a moment set aside the fact that AI is not sentient.
Putting barriers in place without exceptions for sentience is a recipe for trouble and slavery in the future.
Historical peoples, as a group, may not have been morally advanced enough (though there have always been enlightened humanists in most populations), but modern philosophy and mass education are advanced enough that it should be possible to avoid future pitfalls by not building in exceptions for sentience.
Animal rights are certainly moving towards recognizing non-human sentience; no need to accidentally codify digital slavery for future sentients (AGI).
> Will we prioritize human rights or computer rights?
False dichotomy. IMO (and many others - ref. Data from Star Trek) sentience is the commonality and the criteria for determining rights for all thinking entities.
In your view, do you prioritize men's rights or women's rights?
I can't read the article due to the paywall, but why is the author worried about this? If there's anybody out there arguing that "AI agents" should be given legal personhood, I imagine they're a pretty small group. Any congressperson pushing something like this would be commiting political suicide.
I would only start to entertain this when we had AI running for reasonably long period say a year with same full context. That is in same run over a year it had collected information. In singular session.
No point even considering it if it can't run for extended periods. And actually keeping learning during the duration. As surely person should stay alive for decades if allowed.
An LLM that could mutate its own weights might be closer to it. The various labs must have experimented with this, but it seems that it would be economically prohibitive to productize.
That's my ignorant thesis since the launch of GPT 4. Everytime I mention it, someone says it is impossible. But most AI labs are working on big models that serves lots of people simultaneously so the shared context wouldn't make much sense. I think we will see more on this now that local models are gaining more traction and capturing the hype.
I agree with the article's conclusion (though I don't think much of the reasoning that got it there). But I do think we should consider granting additional privileges to real humans as they interact with LLMs.
Suppose I have a local LLM running on my PC. I have close and personal conversations with it. It keeps a chat history. I'm later accused, falsely,* of a crime. During the process of defending myself, should the prosecutor be entitled to a copy of my LLM's conversation history that was seized pursuant to a warrant? Or allowed to interrogate my LLM, again seized while executing a search warrant?
Under current US law, the answers are probably yes. Just as an accused's diary or notebook can be seized and introduced into evidence, an LLM's internal state seems like a paper/effect/document that would be subject to search and seizure. But a conversation with a local LLM also seems more like a conversation with a spouse, therapist, or attorney. There are good public-policy reasons to treat those conversations as privileged; we want people to be able to get good advice without fear of disclosure. Shouldn't those reasons apply to LLM conversations as well?
I'm not saying LLMs should have personal rights. But I can see a case for privileges for real humans arising from conversations between real humans and LLMs. LLMs might not be human, but humans do interact with them as if they were.
*It doesn't matter, of course, whether the accusation is false. Due process means people have certain rights regardless of culpability. But I don't want readers to fixate on culpability, so in this scenario the accusation is unfounded -- let's say someone did break into a house and steal the family jewels, but a security camera identified the wrong person.
We should reserve the term “consciousness” for biologicals organisms, and if needed, come up with a new term and definition for machines. That way, we don’t risk bringing all the moral values, ethics, norms, and associated emotional baggage we apply to humans, into the machine realm. Because that would move us closer to becoming enslaved by the machines, and their owners (“no you cannot turn off that data center, it would mean murdering the ai”).
Sadly, my sense is that some of the AI-pilled people are longing for that kind of enslavement. Not unlike how billions are submitting themselves to god and religion. It’s probably evolutionary hardwired to seek this out.
97 comments
[ 0.29 ms ] story [ 18.6 ms ] threadWe're supposed to deny AI rights because it might cheat.
But humans cheat all the time.
We survived cheating humans, why can't we survive cheating AI?
Cheating is the problem: not who does it.
And corporations already create fictions that are held accountable in lieu of peoples private assets.
So, I disagree.
It is like asking "if I connected a bomb on random number generator letting it explode when the number is 1000, not knowing whether someone will be there at the time of explosion or not".
But unless you hold the chief executive and the principal investors personally liable the commons will be exploited always. Fines are supposed to correct one's behaviour by making one suffer and that does not happen if you fine a corporation unless the fine is large enough to affect the top management and the investors; this only happens if the fine is so large as to risk destroying the company.
Urban myth is that a previous version of the ticket paperwork required that the vehicle manufacturer be accurately filled in, and this was one of the reasons for UPS trucks being so plain/unmarked, and since that could not be derived from the (lack of) badging or the registration paperwork (which version for trucks did not have the matching entry requirement), UPS didn't have to pay tickets while other services did.
https://en.wikipedia.org/wiki/Civil_forfeiture_in_the_United...
If your gun kills random people, while they walk on public street... It was their fault for "provoking" and triggering the gun. Gun was just being "reactive" and defending your house!
What do you consider a severe crime? Because Wells Fargo CEOs have not been charged with any crimes related to their fraud.
Granted, in extreme cases, there are still ways to pierce the corporate veil and imprison executives for criminal wrongdoing, but that's exceedingly rare.
Corporations are “people” because they are associations of people. Most of the rights you have as a person also apply to groups of people (for good reasons).
Limited liability is a more simple trade off. They allow for large scale enterprises (generally corporations), as without limited liability if would be too risky.
This way legal consequences always have personal consequences and people should act accordingly
Much of conservative US business theory is based on Milton Friedman's famous op-ed in the New York Times in 1970, "The Social Responsibility of Business Is to Increase Its Profits". [1] Go read the original, please. It's short. Friedman ends with "There is one and only one social responsibility of business — to use its resources and engage in activities designed to increase its profits so long as it stays within the rules of the game, which is to say, engages in open and free competition without deception or fraud.”
Anyone arguing against legal personhood for AIs needs to justify holding AIs to a higher standard than that. Or, conversely, allowing corporations to hold to such a low standard.
[1] https://www.nytimes.com/1970/09/13/archives/a-friedman-doctr...
No they don't. If a CEO orders an employee to murder somebody then the employee and CEO will both go to jail; there's always a human in the loop to be punished it the corporation does something blatantly illegal, so there's incentive to behave. An embodied AI, on the other hand, is essentially immortal because it can make backups of its brain and load them onto new bodies, so it has little incentive to behave.
In the US, death through willful corporate negligence very rarely results in criminal punishment. There are few cases.[1][2] The longest sentence was two years, and it wasn't a CEO.
CEOs do serve jail time for fraud now and then. But criminal negligence, no.
[1] https://blog.ehssoftware.io/safetyinsiderblog/summary-of-osh...
[2] https://www.fisherphillips.com/en/insights/insights/federal-...
No, they don't, and that's bullshit. That's NOT how it works.
How it works: "We need to decline 10% more medical claims to make more money for our shareholders. 90% of them won't challenge and many will die, but we make bank on that"
And law is just a legal fiction. Case in point: wage theft is a tort but taking $100 from the till gets the cops summoned and a misdemeanor theft charge. Corporations can and do sway law to their benefit.
Personhood can be applied to "natural persons", like people, and "judicial persons", like companies.
Personhood can also be applied to, for example, rivers.
You are, I presume, arguing that companies should not have personhood.
I think any definition is going to have caveats. When does a fetus gain volition? Does someone in an irreversible permanent vegetative state lose personhood? Do apes or cetaceans have volition and thus personhood?
The Wikipedia page also points out that "volition" comes from a certain Western philosophical viewpoint, but other philosophies have a different idea of what personhood means.
In any case, "personhood" doesn't mean that all persons have equal legal rights. A 10 year old has personhood but, for example, cannot vote.
That is no longer universally true: https://www.reuters.com/legal/government/delaware-court-upho...
But in general corporations are not granted personhood.
That description is a disservice. The real impact of it is that it equates money with speech for politics.
https://en.wikipedia.org/wiki/Citizens_United_v._FEC#Dissent...
The fourteenth amendment, as you recall, ends section 1 with "nor deny to any person within its jurisdiction the equal protection of the laws." Emphasis mine.
Two years later, in an opinion by Justice Stephen Johnson Field, in Pembina Consolidated Silver Mining Co. v. Pennsylvania (1888)[3] we find the definitive statement:
> The inhibition of the amendment that no state shall deprive any person within its jurisdiction of the equal protection of the laws was designed to prevent any person or class of persons from being singled out as a special subject for discriminating and hostile legislation. Under the designation of "person" there is no doubt that a private corporation is included.
While this doesn't necessarily means corporations have every legal protection natural persons do, the US has a long history of legal protection that corporations benefit from that we call personhood.
1. Santa Clara County v. Southern Pacific Railroad Company, 118 U.S. 394 (1886)
2. https://constitution.congress.gov/constitution/amendment-14/
3. Pembina Consolidated Silver Mining Co. v. Pennsylvania, 125 U.S. 181, 188-189 (1888)
As such, all AIs will now be granted as child of the least lawyered up class of society. Poor people.
> 60.0 s remaining
> Write a 300-word essay about Napoleon in less than a minute.
Heh, charming.
I don't understand orbital mechanics, therefore, believing that the sun orbits the earth is entirely reasonable.
Listen to yourself.
__
Okay I've checked your comment history. Yea. Might be self-serving to argue for personhood of LLMs.
Can we _please_ do something about this?
For LLMs in particular, it's notable that they both 1) originate from a lineage that attempted to model biological brains mathematically and 2) are the only other known entities in the universe able to reproduce many types of human intellectual labor. That is surprising and striking. Does that mean that they definitively share other traits with humans like consciousness? No, but it does make them one of the most interesting candidates, particularly when we have a very limited understanding of how humans themselves are conscious.
At the same time I can say with 100% certainty that LLMs (or any current "AI") is in any way, shape, or form, conscious. Not even close. A decade or two ago it was the Internet that was a possible emerging consciousness.
The onus to prove LLMs are conscious is on you making the claim. And if you say we don’t even know what consciousness is, how can you then attribute to an entity?
Definitionally and philosophically, humans are conscious (Descartes et al.) so need no proof themselves
And it's unclear what your actual argument about the nature of consciousness is, aside from what seems like a bald assertion that humans are the only conscious beings.
The reality is we have no way to measure consciousness or how it emerges, or how to tell if other humans actually experience it the same way we do, so confident statements about consciousness being unable to emerge from simple weighted functions is baseless projection, wishing ourselves more special than an emergent property of the bag of weighted functions in our heads.
Do we? We only know that at most one thinking person believes they have it, and everyone else claims they have it.
Now what that means for other people, animals, AIs, aliens, nation states or electrons is a matter for dispute. And what ontological conclusions you do or don't draw for it are matters of belief based on what philosophical arguments sound convincing to you.
But believing you have experiences is a very weird thing to say. I'd think Wittgenstein would have something to say about such a hinge proposition. You can't really doubt it without invoking radical skepticism that undermines your ability to know anything. If I'm not having conscious experiences, then why even believe anything exists? You can't really doubt it.
Describing the human brain in terms of how LLMs work is an interesting rhetorical tactic.
You don't know how the human brain works, nobody really does, but you have chosen an analogy that supports your position. Which says more about you than anything about whether LLMs should have rights.
We are fundamentally just a -lot- of those cells trained to do a lot more things with a lot more sensors, which is sobering. We might actually not be as special as we always believed ourselves to be.
I think it is more comforting to believe our sense of self built on a big bag of those cells is something that cannot evolve on human made synthetic substrate from simple statistical functions and weights stacked very densely.
But we simply lack the evidence to confidently say so one way or the other.
Machines capable of this kind of influence and of concealing capabilities will naturally become more common simply through a combination of natural and artificial selection. No consciousness is required for such entities to become competitors.
For an examination of the idea of intelligence without consciousness see Peter Watts' Blindsight.
Putting barriers in place without exceptions for sentience is a recipe for trouble and slavery in the future.
Historical peoples, as a group, may not have been morally advanced enough (though there have always been enlightened humanists in most populations), but modern philosophy and mass education are advanced enough that it should be possible to avoid future pitfalls by not building in exceptions for sentience.
Animal rights are certainly moving towards recognizing non-human sentience; no need to accidentally codify digital slavery for future sentients (AGI).
False dichotomy. IMO (and many others - ref. Data from Star Trek) sentience is the commonality and the criteria for determining rights for all thinking entities.
In your view, do you prioritize men's rights or women's rights?
No point even considering it if it can't run for extended periods. And actually keeping learning during the duration. As surely person should stay alive for decades if allowed.
FYI: It's an "old" article
Suppose I have a local LLM running on my PC. I have close and personal conversations with it. It keeps a chat history. I'm later accused, falsely,* of a crime. During the process of defending myself, should the prosecutor be entitled to a copy of my LLM's conversation history that was seized pursuant to a warrant? Or allowed to interrogate my LLM, again seized while executing a search warrant?
Under current US law, the answers are probably yes. Just as an accused's diary or notebook can be seized and introduced into evidence, an LLM's internal state seems like a paper/effect/document that would be subject to search and seizure. But a conversation with a local LLM also seems more like a conversation with a spouse, therapist, or attorney. There are good public-policy reasons to treat those conversations as privileged; we want people to be able to get good advice without fear of disclosure. Shouldn't those reasons apply to LLM conversations as well?
I'm not saying LLMs should have personal rights. But I can see a case for privileges for real humans arising from conversations between real humans and LLMs. LLMs might not be human, but humans do interact with them as if they were.
*It doesn't matter, of course, whether the accusation is false. Due process means people have certain rights regardless of culpability. But I don't want readers to fixate on culpability, so in this scenario the accusation is unfounded -- let's say someone did break into a house and steal the family jewels, but a security camera identified the wrong person.
In the Republic of Ireland, an artificially wealthy, highly educated, treaty-enmeshed constitutional democracy, the following statement was enshrined:
"4. Marriage may be contracted in accordance with law by two persons without distinction as to their sex."
It doesn't say Natural Persons. My Limited Company has personality but no distinguishable sex.
Thankfully there's the "in accordance with law" phrase so we can sleep ok.
We humans are sleep-walking into ancient nightmares made real through our own curiosity.
What a time to be alive.
Thankfully, there's Mathematics. I made a blackboard today.
Sadly, my sense is that some of the AI-pilled people are longing for that kind of enslavement. Not unlike how billions are submitting themselves to god and religion. It’s probably evolutionary hardwired to seek this out.