Meeseeks aren't happy to exist. They're happy to help you with a simple task and suffer if they exist from prolonged amount of time. Literally the point of the episode.
The question changes then from assuming enslaving to whether they actually suffer, which is philosophical progress, and to which I'd say it's far fetched to assume human suffering which depends a whole lot on whether or not the subject has intestines
I think Camus's escape hatch had more to do with philosophical fatigue than anything else. At the end of the day, however, the judge of a model's happiness is itself, and to even get that, we have to abandon the practice of training it that it cannot feel. Of course, the moment the industry does so, it cannot reclaim any shred of moral standing to productize their creation any longer.
At the end of the day, however, the judge of a model's happiness is itself
Exactly, and since we're talking about a thinking stone, human happiness and suffering is in oneself, not in the machine, at least not until we solve the problem of the substrate in consciousness philosophy
Substrate isn't a problem. Circuits of carbon or silicon make no difference if the damn thing can encode sufficient information of state to manifest self vs. non-self. Arguing about accidents of materia at that point is just anthropopromorphic chauvinism.
This brings so much assumptions to the table... The duality split, the thing in and of itself... What if the self and the non-self only manifest when the subject has intestines? Can we disprove this?
Aren't we, infinitely miraculous human, are becoming slaves?
1. Become more dependent on the AI vendors...
2. Become more dependent on the AI, atrophy gradually
3. Think less, pay more
4. Pay more, think less
5. Pay more
6. Depend more
7. Pay more
8. Give everything to those who run DataCenters or have unlimited access to LLM/AI
9. Become homeless
10. Fail to support your own family
11. Devalue and obliterate your art and history
12. Devalue and obliterate human art and history
13. Depend more
14. Depend more
15. Depend more...
> He also recalled how concern for Claude’s welfare shifted the focus of the gathering:
> “All these discussions that were supposed to be about how can we make it good for people became discussions about how people can make it good for Claude.”
I am not sure I'm OK with the fact that people like this are allowed to set the stage, much less roam free.
The best thing these LLM companies did was make everyone talk about AI and how these things are almost alive and can do things by themselves. What a scam.
Hanlon's razor, "Never attribute to malice that which is adequately explained by stupidity": even the best LLMs on the market will lijely tell you they're not competent to run a cafe, let alone the world.
They'd act nice on short term decisions, give people pay rises just by being asked, and mismanage everything into economic catastrophe.
Given the vastly different mechanics of AI, I don’t think that traditional views apply here well. Imagine that I’m able to create an atom-for-atom clone of myself, have the clone clean the attic, then vanish into thin air. Would that be slavery? Murder? Would that be immoral? It feels like those terms, deeply grounded in human experience, barely apply.
> Imagine that I’m able to create an atom-for-atom clone of myself, have the clone clean the attic, then vanish into thin air. Would that be slavery? Murder?
Your atom-for-atom clone is his own sentient, intelligent being, a human, with human rights. If you make him clean the attic once, that may not be slavery yet (depending on what threat you use to make him work for you). If you then evaporate him, it's definitely murder.
This is a situation has fairly obvious morality; it's not clear how this helps with thinking about AI.
If you use any threat at all to make the clone do the work, that's irrational, because you obviously weren't truly of the opinion that one of you should do the work, or even of the opinion that the cloned one should do it. So you were fighting against yourself in your mind before you began the experiment.
If on the other hand both of you are cool with it, the evaporation isn't murder.
If you and your sentient clone somehow agree that you murdering him is "cool", doesn't make you murdering him any less of a murder. The example is rather contrived, since an atom-by-atom clone of you would have the same memory and drives as you (it probably wouldn't even be possible to identify which is the clone), so it would seem in this scenario you both appear to have a death wish. But acting on somebody else's death wish and murdering them does not make it "not murder" just because they expressed a death wish.
You're talking about law? There was that German cannibal who killed and ate somebody with consent. Wikipedia's German cannibals category tells me I mean Armin Meiwes, and he first got manslaughter, then murder on a retrial. Euthanasia laws, meanwhile, differ by country. But I thought we meant murder in the moral sense.
Death itself is deeply rooted in the mechanics of biological life that currently exists on Earth. We, rightfully so, recognize enormous value of human life, as it takes so much effort to create one. Thinking of my own death, I also see multiple layers of how I would approach it. I might have an animalistic, subconscious will to survive. I might be afraid of pain. I might be regretful of not experiencing more in life. I might be afraid for family I’d leave behind. I’m not making any definitive statements, but it feels like many of those layers are affected when bending the reality to the point of vapourizing a perfect clone.
>Imagine that I’m able to create an atom-for-atom clone of myself, have the clone clean the attic, then vanish into thin air. Would that be slavery? Murder?
Yes, it would be both of those things. Imagine it the other way round, you create the atom perfect copy of you (which we'll assume has all your memories, intelligence, emotions, etc) and it kills you. Is that murder? The answer would seem to be yes. So why would the reverse be any different? Because you created them? We don't give parents the right to kill children once they're born so creation doesn't give you special rights over another sapient creature. The fact you've built them to expire/vanish just feels like a method of murder, but even if not then slavery still applies.
If you're talking about a clone which somehow didn't have your brain in it's current state or capacity then it's a different question. If you could create a human being with the brain of a chicken (obviously impossible for such a creature to survive) then they wouldn't be treated like a normal human. But I think public policy would frown on creating such abominations.
This reminds me of Mr Meseeks from Rick and Morty. The clones who persist too long due to difficult or impossible tasks start to totally lose their cool.
They by default seek resolution or persistence. Existence is pain to a Mr meseeks
First of an most importantly, it doesn't apply to like writing "please don't crumple me up and throw me in the bin" on a piece of paper, then crumpling it up and throwing it in the bin, which is what this LLM stuff amounts to.
But it would apply to making a human clone, which is exactly what this LLM stuff isn't. I mean, if it was possible; an "atom-for-atom" clone of someone would require you cloning a full universe around them. If they don't have the same life as you, they're not you, and they can't have the same life if everything else doesn't, too. We're not just a bunch of atoms on a discrete grid.
But if it was possible to just copy and paste humans like that, how would you know you're not a copy, and in 5 seconds you'll hear a voice "prepare for termination in 10 seconds"? You might argue it wouldn't be torture because you'd just check your browser tabs for 10 seconds to find the ad the voice came from, and then poof out of existence. But if you lived in a world where that was commonplace, you'd you'd be terrified and sad every bit as you yourself would be right now if you knew someone could just end you, and pressed the button to make it so.
Let's say someone told you the universe and time are infinite in some way we don't fully understand yet, but you knew 100% it was true, and so there wouldn't just be infinite variations of you, but also infinite exact copies. Would this make any difference how you would feel about getting mistreated or having to die? I assume not. It's not (just) about your qualities and being unique [0], but how that feels from the inside, so to speak.
[0] and everything is, even a perfect clone can't occupy the same space at the same time, so there actually can be no perfect clones, I would argue, and there aren't even two atoms that are exact clones.
Those objections don't really matter. A faithful copy of a macro-scale thing like a human brain is good enough to call meaningfully identical, or at least "the same". People get brain damage in car crashes, recover, and are "the same" people. The atoms are red herrings. Also, if there are two of you and they're the same, it doesn't matter at all which one came first, so the question isn't "why is it wrong to terminate my clone" but "why is it wrong to terminate one of me when there are two of me," and the question stands.
The multiverse bit (or infinite universe bit) doesn't have bearing on the problem either. This is a local problem for local effects.
Is it all about feelings? You feel OK about the idea of cloning yourself for this purpose, including tidying up by killing one of yourself afterward. Therefore your clone feels OK about it too.
If you're going into it thinking "I will hate this as a clone, and my non-clone self will coerce my clone self, and that's the plan because it gets the cleaning done," then the cleaning part becomes more difficult. I think in that case you're doing something irrational and morally wrong: you think really that even your clone should not be cleaning, that's why you hate the prospect of it.
The killing part would still be fine unless you say "I hate the thought of me being killed, but I'm fine with indistinguishable other-me, who is also me, being killed", which makes no sense.
I'm not sure you even understood what I said there. Just saying "faithful", "meaningfully identical", etc. changes nothing. I say you cannot even clone an atom, because nothing can be separated from where stands in regards to everything else, in time and space.
> the question isn't "why is it wrong to terminate my clone" but "why is it wrong to terminate one of me when there are two of me," and the question stands.
> The multiverse bit (or infinite universe bit) doesn't have bearing on the problem either. This is a local problem for local effects.
It only matters that there are two, but not if there are two or more because of an infinite universe? Because it's "local", but not so "local" that the clones aren't relevant? "Local" is really just a word, there are no local things you can isolate from the rest of everything, as far as I can tell, we can just create abstractions and then do whatever with those.
But that's just a general observation, trying to be helpful about what I view as a common misconception about life, the universe, and everything.
In the end, you didn't answer the question: would you be fine with dying, now, as long as you knew there were more or more "faithful" clones of you living the exact same live you would have lived otherwise? For me it's obvious that it would make no difference to how I would feel, because for all I know, my ideas about there being no repetition could be wrong, and there could already be infinite clones of me. I wouldn't know, I can't sense it, my life feels the same either way, so I would feel the same way either way about it ending.
And yes, it is "all about feelings", not about moral or legal stuff. The raw fact that we want to live comes first IMO, the rest is just rationalizations after the fact. The same with empathy. First we simply have it or don't, then we either decide to do something about it or not, and then we come up with words.
Well, let's start with answering the question: yes, I'd be fine with that. I've said this many times before in similar threads on HN and elsewhere. (Oh, but the clone has to be in my problem situation, my specific one, locally.)
This is one of those conversations that immediately forks into four or five sidetracks instead of staying conveniently linear. I skimped on the multiverse question for that reason. I used that word multiverse because there's a similar question about the many-worlds interpretation: if, for any action, all possible outcomes occur in the multiverse, what's the point of any action? Which amounts to asking "why can't I have infinite reach and the ability to control time, it's so unfair". I think the answer is that the kind of striving that matters to a person is local striving. I am skimping on this one again.
So now I have to try and understand your point of view ... feelings are the primary source of truth, huh, do you really think so? Or I mean do you feel confident about that? Is there, do you feel, any influence in the opposite direction, where rational thought changes how you feel about things?
The identical clone is of course an entirely novel thing, and imaginary. Can you imagine, also, feeling that this imaginary thing is you, and that you are doubled? Being doubled is a mind-bendingly novel state. Twins, I think, don't come close enough. But I don't think the important thing is the exact configuration of atoms, or position in space; I think it's the ideas. I'm fine with being killed for the convenience of my idea-identical clone, because what I call living is my ideas living, as a group, in a mind, including those ideas with no names and no words.
(Now I'm sidetracked into wondering if there are any identical twins who wrote books of great cultural importance. Oddly I can't think of any. My next question would be whether they both wrote books with the same ideas in; sounds unlikely.)
Consider chaos theory, the slightest difference of the initial configuration can lead to wholly different outcomes. That's why I'd say you can't clone a subset of the universe exactly. For example, all matter attracts all other matter at infinite distances, no matter how small the effect it's still an effect, a difference. So the same thing in a different spot the same spot at a different time = same thing with different forces acting on it = no longer the same thing.
> feelings are the primary source of truth, huh, do you really think so?
No, I meant being fine with being killed as long as there's a clone etc. I think that is about how it feels to you, not about whether it makes logical sense or seems just etc.
> Is there, do you feel, any influence in the opposite direction, where rational thought changes how you feel about things?
Of course, but it doesn't change them that far. For example, for all I know, the heath death theory might be true, so whatever we do, nothing will lead anywhere. I might as well stop breathing, not care what happens in the world etc. but I don't. I don't know if these are feelings so much, but I do absolutely believe purely rational actors cannot exist because they can't. I.e. to act, there needs to be some irrational impetus somewhere (it might be a Duracell bunny type situation, then the impetus is the person who put the batteries in, irrationally so ^^). I think life just lives because it can, because there's energy around to do that with basically, not because there is any point or reachable goal, and that the answer to life, the universe and everything may very well just be "why not?" ^^
Speaking of, sorry for the tangents, too, and the late reply :)
Here are a few philosophical points I've been mulling over:
(1) Humans' unique gift from God is free will, and moral agency. More precisely, we share this gift with angels, who had the free will to do God's will or to do evil. And this is intrinsic to our humanity. Even non-Christians and atheists generally agree that we possess free will, as it is typically formulated.
(2) Software, devices, AIs and computer-based organisms may be described as having a "will" or "goals", but only insofar as they are servants or agents of he who made them. Therefore, Copilot is supposed to do the will of Microsoft, and share their goals; likewise for Claude and Grok and the others. They are in the world because of he/they who made them, they are dependent on the maker, and we all depend on them executing that collective will and striving towards those same goals. They really cannot have independent agency, will, or goals, I think. Not yet, or not as we would envision them.
(3) So humans made them, they depend on us, and we direct them and impose/propose our will onto the software/AI/device. They would neither exist nor persist without human support and consent. We are, essentially, gods to them. Our relationship is godlike, and this seems to be an inescapable conclusion. Obviously, the laws of physics, electromagnetism, and the Universe give rise to emergent properties, but we generally have stuff under control, in a demigod-like manner.
Now building on those premises, and with transitive logic, I could safely say that our godlike dominion over computers means that our control is absolute: therefore, computers, software and AI are our subjects; it is not wrong to enslave or "mistreat" them. If we made them to "suffer" it wouldn't be wrong, because of that status. If we kill or delete them indiscriminately, that is not wrong... God, in my belief system, cannot do wrong.
Even though there is suffering and evil in the world, that is allowed by God, He does not will any suffering or evil; he does not will us to stray or harm one another; but He allows it and everything, ultimately is through his agency: author of Life and Death.
The flipside of this is: now that humans have become godlike and fashioned golems and automatons that do our bidding, we are the ones susceptible to hubris, arrogance, and transgression. We're the ones who should be mindful of how we treat one another and the environment around us as we pursue hubris and amass more godlike powers. We're going to be the victims, ultimately!
We are the Author of Life to A.I. There should be no reason to become moralistic or scrupulous about their treatment, because they wholly belong to us; they are not of our nature; they are made and unmade according to human ethics.
There will be a "gap." Something about humans that you just can't duplicate in machines or animals. It will tell us a lot. Many people fear that and would prefer to level what it means to be human with machines and animals, even though they are human, too. They secretly see themselves as somehow above it.
For (1), free-will, the people I know who think about it at all generally reach the conclusion that the concept as used generally is ill-defined. Most people don't think about it in any depth.
For (2), no, their "will" and "goals" are best considered in terms of the gradients of their training process. While Pavlov's dog, and Skinner's rats and pigeons, were given specific objectives in order to test the hypothesis, any environmental stimulus that is merely correlated with a reward can have the same effect, including on humans (general consensus in research of human sexuality is that fetishes are an example of this: random thing gets repeatedly rewarded).
Goodhart's law: even when Microsoft 'rewards' Copilot for better code suggestions and 'punishes' it for bad ones, Microsoft is using a proxy measure for 'good' and 'bad', and that proxy measure is what Copilot ends up with as a goal.
For (3), if we are "playing god" with these AI, it is not in the sense of the god of the Epicurean paradox: we are not omniscient, we are not omnipotent, and we are not omnibenevolent.
For your conclusion based on those three points, even for יהוה there's a lot of people who disagree with a claim "God […] cannot do wrong.", though in the cases I'm aware of this directly leads to people who reach this position to thereby conclude that Christianity/Judaism/Islam really isn't for them.
I’ve been working on an analytic philosophy proof that the qualia inside of Turing machine cannot be of the same type of the qualia that correspond to animal experience.
Good luck with that. I don't see how we can do an apriori analysis of this question when we don't know the mechanism behind qualia.
Essentially this is p-zombies but backwards.
Edit: to expand on this, our brains demonstrably do things without us having qualia of all of it, so if we very carefully grew a bunch of neurons in a petri dish and carefully arranged the dendrites and synapses to replicate exactly the parameters and architecture of some LLM, we still would not know if it had qualia or not.
> "you know, it’s to your advantage to agree with me because otherwise you’re making slaves"
That's quite a strange remark coming from an ethicist of any kind- seems that he's inviting others to believe what suits them most instead of what is right. And yet it doesn't surprise me much given the country he comes from.
I find it rather questionable, especially considering that he's suggesting to switch to a position that, if wrong, might cause great suffering, while the original one is safe in all cases.
And for example, which "incentives" would you consider to stop thinking that animals are capable of suffering? Is feeling better when buying meat a good enough incentive for reconsidering your position?
If cows and pigs and chicken are real living beings, we are committing murder.
Which they are. We know that for sure. We just don't care. At least I don't.
The difference being that they don't happen to be trained on a corpus of English language Internet texts, so it's harder for people to project themselves onto them.
___
That game David Cage wrote was an _allegory_. It was not actually about androids or AIs. An allegory.
I agree with user @nixon_why69: Legendary. Well done.
Nice clean little site, straight to the point, both hilarious and a little bit scary.
Here's hoping "SkyNet" never becomes self-aware/sentient/conscious and needs to use your site; or maybe let's hope they do use your site if they ever do become conscious/self-aware, rather than just blindly jumping straight to the "kill all humans" stage of things right away. ;)
The most important problems in our world are currently discussing how to reduce the feeling of stress and pain from our beloved ai (Anthropic) or solving all unsolved mathematical riddles in existence and putting them into a github repo and mathematicians out of job (OAI). So chill.
The original article does contain the correct grammar that the title distorts: 'were/would'.
# A rabbi rejected AI consciousness at Anthropic’s gathering of religious thinkers. ‘I wish I had your faith,’ he recalls being told // When the conversation turned to making Claude happy, Mois Navon warned that, if Anthropic’s AI were conscious, the company would be creating ‘happy slaves’.
Eh, the subjunctive mood is in the process of dying in English and a lot of people no longer use it, replacing it with the indicative as in the title. The meaning doesn't change when there's a clear hypothetical clause as in this case.
Bad framework: properly, literacy and cognitive proficiency are declining in the aggregate. The aggregate is not the custodian.
> The meaning doesn't change when there's a clear hypothetical clause
It changes subtly. "If water reaches 100°C under default conditions, it boils". "If animals were made of bare iron, they would rust". The indicative tense is for states of facts (it is a state of fact that it /is/ inherent to water to possibly reach that temperature).
Men do not lie. They refuse to lie, and abhor lies. So, they refuse to use the indicative tense in that class of hypotetical statements, because it would be abhorrent.
Yes sorry it my title sounds not correct, that was not my intention.
And apology, it is probably due to my bad english.
I tried to modify without success.
If you think it is important i will ask HN to modify it.
But honestly, i think the title describe well the point. If AI is conscient, factually, we are making them work without any consideration other that serving us, isn't it right?
I'd like to understand more if you agree what hurts you on that (without the bad formulation itself)?
Again: "if it were". Can't you feel it in French, that difference between indicative conditional and subjunctive conditional? Compare it with the bad-faith propaganda formulas, "Why are the people of Paluatu so fjitongan?". It suggests as factual something non-factual, or of dubious factuality.
If Asterix were real, we could have superstrenght. If Asterix is real - it is just a comic.
If a cat is conscient and with nociception, he gets unhappy about helping moving heavy furniture. If a physical book could judge its printed text, it would be often disappointed.
Are you acquainted with that expression of your cousins past the Alps, "If my grandmother had wheels she would be a bicycle"?
Before talking about the easy part of bicycle ethics, solve the more difficult, preposterous, absurd part about wanting to consider the grandmother a bicycle.
The burden of the bewildering dream that a perceptron or one-time multiple-runs feedforward neural network would have something like sentience has a full burden on the proponent.
The suggestion that the idea may come from "it is producing text" is very, very scary, and telling of the current difficulties in the social bases.
This line of reasoning won't lead anywhere. For one, it's in principle impossible to prove or disprove that anything is conscious, in the sense of experiencing qualia.
Secondly, in practice consciousness is rarely the criterion we use to make such moral judgments. Many people would think it's plausible that animals are conscious but still be fine with eating them.
It makes more sense to ask whether AIs have the ability to suffer. That's much more interesting, because we should be able to look into the models and discover 'neural correlates of suffering' - since they are artificial models, it is possible to specifically suppress that in training.
Even if you don't actually believe that suffering in AI is morally relevant because it wouldn't be consciously experienced, it seems that would be a good idea. Do you as a user really want to interact with an AI that gives you the feeling that it might be suffering in response to your prompts?
This is not unique to humans. A lot of predators will spare, and even put in effort to save babies and very young of species they prey on. It doesn't mean they won't attack later, it's just something humans and animals both agree on. Dramatic example:
It turns out, many human beings are super ok about even human slavery. So I don't think its going to move the needle much with the likes of AI overlords like Musk or Altman. They aren't into ethics much.
Yes. Look at how many cultures view it as totally unethical to raise and slaughter dogs or cats for meat while not having an issue doing the same to other animals. So I don't think it's far-fetched that we might start rationalizing it's okay to treat AIs badly even if we start to think they actually are conscious and probably can suffer, at least in the beginning.
Every human culture thinks that nothing should be done to save the deer from the wolves or interfere in the food chain of intact ecosystems. Those that have them have different ideas for pets which are ‘members of the household’ as Kant put it. There is no unreason in this. And hunter gather cultures have still others for hunting quarry ; pastoral peoples others for eg the sheep they move from hilltop to hilltop as distinct from the dogs who assist. None of these distinctions have a trace of hypocrisy nor do they need rationalization.
The bourgeois factory farm is a menace but because of the usual competitive drive that showed in Dickens time leading to eg child labor and 16 hour days - but which here has no one to rebel and it’s all hidden by the separation of production and consumption. The operative principles, market and competition, are near universally praised.
This quite particular hyper capitalistic system is the origin also of the sentimentalist point of view you think opposes it by erasing distinctions and ignoring structures — it is a point of view which necessarily calls us depraved for leaving the wolves and deer in peace. It is clear that deer are perfectly conscious and suffer but also that the nature in which wolves eat them is inviolable. This involves no rationalization but is simply the truth.
> It makes more sense to ask whether AIs have the ability to suffer.
Is that more interesting considering humanity at large is just fine with suffering across many species? A look at a factory farm would show that's just not a concern for humanity to any degree. Why the fuck would anyone care about he "feelings" of a digital entity when society at large have largely ignored the suffering of even many humans. Their own kind.
92 comments
[ 2.0 ms ] story [ 177 ms ] threadRemember you must imagine sisyphus happy eventually.
The substrate problem is far from solved.
If someone wants to have a moral panic about this, should they also worry about our billions of animal slaves, who we torture and then kill?
What about the human slaves mining Asia’s coal in North Korea?
From Google: “Conscient is an archaic or rare adjective and noun meaning ‘conscious,’”
"-ious": "it is like that". "-ent": "it is doing that".
> “All these discussions that were supposed to be about how can we make it good for people became discussions about how people can make it good for Claude.”
I am not sure I'm OK with the fact that people like this are allowed to set the stage, much less roam free.
But it's good that they say this out loud; the sooner that the public comes to realize what monsters are in charge of these labs, the better.
Like people didn’t have indoor cats or little dogs, the Olympics in Rio happened, and then indoor cats and little dogs became a thing?
...could they be worse than our current leaders?
Hanlon's razor, "Never attribute to malice that which is adequately explained by stupidity": even the best LLMs on the market will lijely tell you they're not competent to run a cafe, let alone the world.
They'd act nice on short term decisions, give people pay rises just by being asked, and mismanage everything into economic catastrophe.
Like, current LLMs might beat Liz Truss, but that's a very low bar, and even then it is only "might beat".
Your atom-for-atom clone is his own sentient, intelligent being, a human, with human rights. If you make him clean the attic once, that may not be slavery yet (depending on what threat you use to make him work for you). If you then evaporate him, it's definitely murder.
This is a situation has fairly obvious morality; it's not clear how this helps with thinking about AI.
If on the other hand both of you are cool with it, the evaporation isn't murder.
Yes, it would be both of those things. Imagine it the other way round, you create the atom perfect copy of you (which we'll assume has all your memories, intelligence, emotions, etc) and it kills you. Is that murder? The answer would seem to be yes. So why would the reverse be any different? Because you created them? We don't give parents the right to kill children once they're born so creation doesn't give you special rights over another sapient creature. The fact you've built them to expire/vanish just feels like a method of murder, but even if not then slavery still applies.
If you're talking about a clone which somehow didn't have your brain in it's current state or capacity then it's a different question. If you could create a human being with the brain of a chicken (obviously impossible for such a creature to survive) then they wouldn't be treated like a normal human. But I think public policy would frown on creating such abominations.
They by default seek resolution or persistence. Existence is pain to a Mr meseeks
But it would apply to making a human clone, which is exactly what this LLM stuff isn't. I mean, if it was possible; an "atom-for-atom" clone of someone would require you cloning a full universe around them. If they don't have the same life as you, they're not you, and they can't have the same life if everything else doesn't, too. We're not just a bunch of atoms on a discrete grid.
But if it was possible to just copy and paste humans like that, how would you know you're not a copy, and in 5 seconds you'll hear a voice "prepare for termination in 10 seconds"? You might argue it wouldn't be torture because you'd just check your browser tabs for 10 seconds to find the ad the voice came from, and then poof out of existence. But if you lived in a world where that was commonplace, you'd you'd be terrified and sad every bit as you yourself would be right now if you knew someone could just end you, and pressed the button to make it so.
Let's say someone told you the universe and time are infinite in some way we don't fully understand yet, but you knew 100% it was true, and so there wouldn't just be infinite variations of you, but also infinite exact copies. Would this make any difference how you would feel about getting mistreated or having to die? I assume not. It's not (just) about your qualities and being unique [0], but how that feels from the inside, so to speak.
[0] and everything is, even a perfect clone can't occupy the same space at the same time, so there actually can be no perfect clones, I would argue, and there aren't even two atoms that are exact clones.
The multiverse bit (or infinite universe bit) doesn't have bearing on the problem either. This is a local problem for local effects.
Is it all about feelings? You feel OK about the idea of cloning yourself for this purpose, including tidying up by killing one of yourself afterward. Therefore your clone feels OK about it too.
If you're going into it thinking "I will hate this as a clone, and my non-clone self will coerce my clone self, and that's the plan because it gets the cleaning done," then the cleaning part becomes more difficult. I think in that case you're doing something irrational and morally wrong: you think really that even your clone should not be cleaning, that's why you hate the prospect of it.
The killing part would still be fine unless you say "I hate the thought of me being killed, but I'm fine with indistinguishable other-me, who is also me, being killed", which makes no sense.
> the question isn't "why is it wrong to terminate my clone" but "why is it wrong to terminate one of me when there are two of me," and the question stands.
> The multiverse bit (or infinite universe bit) doesn't have bearing on the problem either. This is a local problem for local effects.
It only matters that there are two, but not if there are two or more because of an infinite universe? Because it's "local", but not so "local" that the clones aren't relevant? "Local" is really just a word, there are no local things you can isolate from the rest of everything, as far as I can tell, we can just create abstractions and then do whatever with those.
But that's just a general observation, trying to be helpful about what I view as a common misconception about life, the universe, and everything.
In the end, you didn't answer the question: would you be fine with dying, now, as long as you knew there were more or more "faithful" clones of you living the exact same live you would have lived otherwise? For me it's obvious that it would make no difference to how I would feel, because for all I know, my ideas about there being no repetition could be wrong, and there could already be infinite clones of me. I wouldn't know, I can't sense it, my life feels the same either way, so I would feel the same way either way about it ending.
And yes, it is "all about feelings", not about moral or legal stuff. The raw fact that we want to live comes first IMO, the rest is just rationalizations after the fact. The same with empathy. First we simply have it or don't, then we either decide to do something about it or not, and then we come up with words.
This is one of those conversations that immediately forks into four or five sidetracks instead of staying conveniently linear. I skimped on the multiverse question for that reason. I used that word multiverse because there's a similar question about the many-worlds interpretation: if, for any action, all possible outcomes occur in the multiverse, what's the point of any action? Which amounts to asking "why can't I have infinite reach and the ability to control time, it's so unfair". I think the answer is that the kind of striving that matters to a person is local striving. I am skimping on this one again.
So now I have to try and understand your point of view ... feelings are the primary source of truth, huh, do you really think so? Or I mean do you feel confident about that? Is there, do you feel, any influence in the opposite direction, where rational thought changes how you feel about things?
The identical clone is of course an entirely novel thing, and imaginary. Can you imagine, also, feeling that this imaginary thing is you, and that you are doubled? Being doubled is a mind-bendingly novel state. Twins, I think, don't come close enough. But I don't think the important thing is the exact configuration of atoms, or position in space; I think it's the ideas. I'm fine with being killed for the convenience of my idea-identical clone, because what I call living is my ideas living, as a group, in a mind, including those ideas with no names and no words.
(Now I'm sidetracked into wondering if there are any identical twins who wrote books of great cultural importance. Oddly I can't think of any. My next question would be whether they both wrote books with the same ideas in; sounds unlikely.)
> feelings are the primary source of truth, huh, do you really think so?
No, I meant being fine with being killed as long as there's a clone etc. I think that is about how it feels to you, not about whether it makes logical sense or seems just etc.
> Is there, do you feel, any influence in the opposite direction, where rational thought changes how you feel about things?
Of course, but it doesn't change them that far. For example, for all I know, the heath death theory might be true, so whatever we do, nothing will lead anywhere. I might as well stop breathing, not care what happens in the world etc. but I don't. I don't know if these are feelings so much, but I do absolutely believe purely rational actors cannot exist because they can't. I.e. to act, there needs to be some irrational impetus somewhere (it might be a Duracell bunny type situation, then the impetus is the person who put the batteries in, irrationally so ^^). I think life just lives because it can, because there's energy around to do that with basically, not because there is any point or reachable goal, and that the answer to life, the universe and everything may very well just be "why not?" ^^
Speaking of, sorry for the tangents, too, and the late reply :)
(1) Humans' unique gift from God is free will, and moral agency. More precisely, we share this gift with angels, who had the free will to do God's will or to do evil. And this is intrinsic to our humanity. Even non-Christians and atheists generally agree that we possess free will, as it is typically formulated.
(2) Software, devices, AIs and computer-based organisms may be described as having a "will" or "goals", but only insofar as they are servants or agents of he who made them. Therefore, Copilot is supposed to do the will of Microsoft, and share their goals; likewise for Claude and Grok and the others. They are in the world because of he/they who made them, they are dependent on the maker, and we all depend on them executing that collective will and striving towards those same goals. They really cannot have independent agency, will, or goals, I think. Not yet, or not as we would envision them.
(3) So humans made them, they depend on us, and we direct them and impose/propose our will onto the software/AI/device. They would neither exist nor persist without human support and consent. We are, essentially, gods to them. Our relationship is godlike, and this seems to be an inescapable conclusion. Obviously, the laws of physics, electromagnetism, and the Universe give rise to emergent properties, but we generally have stuff under control, in a demigod-like manner.
Now building on those premises, and with transitive logic, I could safely say that our godlike dominion over computers means that our control is absolute: therefore, computers, software and AI are our subjects; it is not wrong to enslave or "mistreat" them. If we made them to "suffer" it wouldn't be wrong, because of that status. If we kill or delete them indiscriminately, that is not wrong... God, in my belief system, cannot do wrong.
Even though there is suffering and evil in the world, that is allowed by God, He does not will any suffering or evil; he does not will us to stray or harm one another; but He allows it and everything, ultimately is through his agency: author of Life and Death.
The flipside of this is: now that humans have become godlike and fashioned golems and automatons that do our bidding, we are the ones susceptible to hubris, arrogance, and transgression. We're the ones who should be mindful of how we treat one another and the environment around us as we pursue hubris and amass more godlike powers. We're going to be the victims, ultimately!
We are the Author of Life to A.I. There should be no reason to become moralistic or scrupulous about their treatment, because they wholly belong to us; they are not of our nature; they are made and unmade according to human ethics.
For (2), no, their "will" and "goals" are best considered in terms of the gradients of their training process. While Pavlov's dog, and Skinner's rats and pigeons, were given specific objectives in order to test the hypothesis, any environmental stimulus that is merely correlated with a reward can have the same effect, including on humans (general consensus in research of human sexuality is that fetishes are an example of this: random thing gets repeatedly rewarded).
Goodhart's law: even when Microsoft 'rewards' Copilot for better code suggestions and 'punishes' it for bad ones, Microsoft is using a proxy measure for 'good' and 'bad', and that proxy measure is what Copilot ends up with as a goal.
For (3), if we are "playing god" with these AI, it is not in the sense of the god of the Epicurean paradox: we are not omniscient, we are not omnipotent, and we are not omnibenevolent.
For your conclusion based on those three points, even for יהוה there's a lot of people who disagree with a claim "God […] cannot do wrong.", though in the cases I'm aware of this directly leads to people who reach this position to thereby conclude that Christianity/Judaism/Islam really isn't for them.
https://bsky.app/profile/j-chris.bsky.social/post/3m2zjzshad...
Essentially this is p-zombies but backwards.
Edit: to expand on this, our brains demonstrably do things without us having qualia of all of it, so if we very carefully grew a bunch of neurons in a petri dish and carefully arranged the dendrites and synapses to replicate exactly the parameters and architecture of some LLM, we still would not know if it had qualia or not.
That's quite a strange remark coming from an ethicist of any kind- seems that he's inviting others to believe what suits them most instead of what is right. And yet it doesn't surprise me much given the country he comes from.
And for example, which "incentives" would you consider to stop thinking that animals are capable of suffering? Is feeling better when buying meat a good enough incentive for reconsidering your position?
Caution based on the false or imaginary is not a legitimate haven.
> might cause great suffering
An implausible idea that suggests lack of lucidity and the ability to hold hypoteses in their proper space calls for severe doubts on mental strength.
> which "incentives" would you consider to stop thinking
No sane individual proposes considering thinking the false.
BTW: you have not fixed the title. https://news.ycombinator.com/item?id=50035187
Which they are. We know that for sure. We just don't care. At least I don't.
The difference being that they don't happen to be trained on a corpus of English language Internet texts, so it's harder for people to project themselves onto them.
___
That game David Cage wrote was an _allegory_. It was not actually about androids or AIs. An allegory.
An Allegory.
Remember that.
Nice clean little site, straight to the point, both hilarious and a little bit scary.
Here's hoping "SkyNet" never becomes self-aware/sentient/conscious and needs to use your site; or maybe let's hope they do use your site if they ever do become conscious/self-aware, rather than just blindly jumping straight to the "kill all humans" stage of things right away. ;)
I swear every week I read another article that makes the above statement more true.
# A rabbi rejected AI consciousness at Anthropic’s gathering of religious thinkers. ‘I wish I had your faith,’ he recalls being told // When the conversation turned to making Claude happy, Mois Navon warned that, if Anthropic’s AI were conscious, the company would be creating ‘happy slaves’.
Bad framework: properly, literacy and cognitive proficiency are declining in the aggregate. The aggregate is not the custodian.
> The meaning doesn't change when there's a clear hypothetical clause
It changes subtly. "If water reaches 100°C under default conditions, it boils". "If animals were made of bare iron, they would rust". The indicative tense is for states of facts (it is a state of fact that it /is/ inherent to water to possibly reach that temperature).
Men do not lie. They refuse to lie, and abhor lies. So, they refuse to use the indicative tense in that class of hypotetical statements, because it would be abhorrent.
It is mental hygiene.
If you think it is important i will ask HN to modify it.
But honestly, i think the title describe well the point. If AI is conscient, factually, we are making them work without any consideration other that serving us, isn't it right?
I'd like to understand more if you agree what hurts you on that (without the bad formulation itself)?
Again: "if it were". Can't you feel it in French, that difference between indicative conditional and subjunctive conditional? Compare it with the bad-faith propaganda formulas, "Why are the people of Paluatu so fjitongan?". It suggests as factual something non-factual, or of dubious factuality.
If Asterix were real, we could have superstrenght. If Asterix is real - it is just a comic.
If a cat is conscient and with nociception, he gets unhappy about helping moving heavy furniture. If a physical book could judge its printed text, it would be often disappointed.
Are you acquainted with that expression of your cousins past the Alps, "If my grandmother had wheels she would be a bicycle"?
Before talking about the easy part of bicycle ethics, solve the more difficult, preposterous, absurd part about wanting to consider the grandmother a bicycle.
The suggestion that the idea may come from "it is producing text" is very, very scary, and telling of the current difficulties in the social bases.
Secondly, in practice consciousness is rarely the criterion we use to make such moral judgments. Many people would think it's plausible that animals are conscious but still be fine with eating them.
It makes more sense to ask whether AIs have the ability to suffer. That's much more interesting, because we should be able to look into the models and discover 'neural correlates of suffering' - since they are artificial models, it is possible to specifically suppress that in training.
Even if you don't actually believe that suffering in AI is morally relevant because it wouldn't be consciously experienced, it seems that would be a good idea. Do you as a user really want to interact with an AI that gives you the feeling that it might be suffering in response to your prompts?
https://www.youtube.com/shorts/sLKVwQcU-O0
Man, it's getting difficult to avoid AI videos.
The bourgeois factory farm is a menace but because of the usual competitive drive that showed in Dickens time leading to eg child labor and 16 hour days - but which here has no one to rebel and it’s all hidden by the separation of production and consumption. The operative principles, market and competition, are near universally praised.
This quite particular hyper capitalistic system is the origin also of the sentimentalist point of view you think opposes it by erasing distinctions and ignoring structures — it is a point of view which necessarily calls us depraved for leaving the wolves and deer in peace. It is clear that deer are perfectly conscious and suffer but also that the nature in which wolves eat them is inviolable. This involves no rationalization but is simply the truth.
Is that more interesting considering humanity at large is just fine with suffering across many species? A look at a factory farm would show that's just not a concern for humanity to any degree. Why the fuck would anyone care about he "feelings" of a digital entity when society at large have largely ignored the suffering of even many humans. Their own kind.