Can a gun commit murder? Same answer. HAL killed people in the course of following its human written program. It would make as much sense to charge autopilot software with homicide.
While both a computer(HAL) and a gun share the attribute of being non-human, there is an argument to be made that HAL could make decisions and thus has "agency", while a gun cannot make decisions. This differentiates the question significantly.
Edit - responding to your comment edit, I don't know if it's common knowledge that HAL was written by humans (or not) but it's perhaps less clear with machine learning and AI algorithms used for modern autopilot systems, the systems learn and further program their own behavior beyond what the human programmers wrote, so in that sense I do think self-agency is emerging and as such there is responsibility on the part of both the car and it's creator, perhaps similar to how a parent is responsible for the independent actions of a young child.
You can reduce that entire essay to "Does an AI have agency?"
And the answer wouldn't change: "Not unless we know it has independent subjectivity and self-awareness."
It always comes down to qualia. Our model of agency is based on a certain set of subjective experiences we take as working definitions of both self-awareness and free will.
We assume that other humans operate the same way. We can't know for sure, but life gets very complicated - and difficult - if we assume we're sentient but everyone else is just a meat robot. So assuming other humans have similar experiences is a reasonable position.
This breaks down with mechanical intelligence, because we can only understand it through visible behaviours. And of course you can have agency-like behaviours without human-like qualia and subjective awareness.
I strongly suspect this is not a solvable problem.
You can imagine some kind of hypothetical (magic) tool for measuring consciousness objectively. We can choose to believe in sentience, or we can choose not to. Neither position can be justified without objective measurement, and that isn't available without a magic device.
But even if it was available, the device would be paradoxical. The output of any objective consciousness measuring device still has to be experienced subjectively, which means it may be unreliable.
Agency is just a nominative dignpost for saying that we can't say for certain what somebody's motives are given dome physical measures. That the outcome of a probability function with them as an agent is not deterministic in nature.
In that sense, HAL absolutely has agency. As certain as you to me.
Seriously, I think elevators are a good model for this, as they are autonomous systems that occasionally kill people. Who's at fault? If the manufacturer messed up the design, they're liable. If the owner ignores maintenance, they're liable. If someone intentionally sabotages the elevator to kill someone, that would be murder. If someone is elevator surfing and dies, then it's their own fault. It's the same with other autonomous systems; the blame could lie anywhere, depending on the circumstances. (I've simplified the legal issues somewhat, of course.)
I think factory floor robots will be a grey area before elevators. As long as things are deterministic then "fault" can be ascribed - it was the workers fault for climbing over the barriers, or it was the manufacturers fault for not testing the arm would stop moving in < 1.2 seconds.
But as soon as AI / ML becomes involved, it stops being deterministic and becomes probability. "Judgement" is asked of the robot arm. Is that image a person or a car door.
I think this is likely to hit before cars simply because so many of us drive cars but none of us operate giant robot arms.
But supposing the robot is "guilty". How (or why) would anyone punish it? Put it in jail? I don't think it will mind. So what would be the point of finding it guilty? Rather, it would make more sense to, in some way "fix" the robot.
Same would be true of people, were they easily "fixed".
I think we should treat it as if an animal killed a human. We don't call that murder. If a dog kills a human, I think it is usually put down? Not sure if a whale does it. So it may depend. But in any case, murder I think is reserved for conscious killing of a human by another human.
I'm not sure about an animal killing a human, but certain animals, like primates, definitely commit murder (i.e. the conscious killing of another member of its tribe for territory, social status, or whatever).
Everything living kills everything in sight for resources. From viruses to civilizations. To some extent this can be moderated, or for a time some optimizations might result in even less destruction, but this is the way this world is.
Killing is rational in a sense - for some subset of purposes, in context of some subset of environmental pressures, both of which change constantly, and often faster than adaptation to kill does.
That "judgement" is a design decision. A ML algorithm misidentifying a person as a car door isn't that different from an elevator cable snapping. That happens more or less randomly too. It could be bad design, bad maintenance (e.g. real world data drifted away from training data but no retraining took place), or just bad luck (an element in almost any accident). Depending on the fault someone or nobody is at fault. And even if nobody is at fault we can still decide to advance the state of the art, and that in the future no single elevator cable snapping should cause more than a minor inconvenience (due to multiple cables, breaks etc).
A common metric already used in industrial automation liability examination is Safety Integrity Level. [1] This is a probabilistic measure of how likely a failure is.
These discussions are always around the hypothetical of artificial intelligence, so real world analogies don't really apply. Today's automations will do whatever they are programmed to do, so the blame can always be traced back to a person. If the robot wakes up and independently decides "I'm going to kill someone today!" then all bets are off.
No, but if you tell someone to kill someone else, and that person does, you'll be found guilty - assuming the normal standards of you reasonably expecting a person to do that, you being mentally competent, etc.
Those caveats apply to machines as well: if I tell a toaster to kill someone, and then the toaster electrocutes them, I would not be guilty.
The key word is "independently". If you build a robot that is capable of doing anything, and it chooses to kill someone (without you specifically telling it to), whose fault is it? It is pointless to speculate about it today, since the status of a hypothetical intelligent robot of the future could be that of a pet (you are responsible), a child (you are not responsible) or really anything else.
If I build a gun mount, that uses a basic computer vision algorithm to identify people out a window. It then randomly select a target - “true” random so no claims I set a seed - and then shoots them.
Do you think there is an argument that I should not be responsible?
If “random” is too independent, or you want it to be systematic, we could do something else:
* the person with the highest “red hat” score according to a 20 year old copy of opencv
Or we could do something modern and “AI” and give it a completely random neural net that’s super deep and has three outputs: angle, pitch, and whether to pull the trigger.
Or we could go all out and just replace this hypothetical machine that acts like an independent person with the next best thing: a regular human.
If a government, or a terrorist group, gives a group of people (or soldiers) a bunch of weapons and drops them off in a city, and they start shooting, do you think that the government would be responsible?
That’s easy. Your intent was for it to maybe shoot someone, and that is a criminal intent. If you had no such intent, it’s either accidental murder or no guilt (e.g. you built a robot that takes a heap of non-classy trash and throws it into a grinder; an intruder was grabbed by it accidentally because they weren’t well-dressed).
This discussion mistakes existing policies as some objective base, but it’s the other way round.
Well yeah kinda. Name any mass murderer we've had and tell me the parents didn't have to hold account - perhaps not always legally, but do they have jobs or friends? Even if they do, will it ever be the same?
The significant disanalogy is the amount of control parents have over their child's character compared to a programmer. Parents can pretty much do their best to raise a child, but the child may still end up screwed up. I wouldn't really put the blame at their feet. If they deliberately raise their child to be a killer, that's obviously another matter. By contrast, an AGI designer is in an unprecedented situation where they essentially have total control over the character of the being they're bringing into existence. If the AGI does bad things, it's very much on the programmer.
That's assuming the AGI designer even understands what they built. If the first AGI is built via some semi-automated bootstrapping process based on a lesser intelligence, you can't really predict how the resulting intelligence will behave.
For sure, which means it's the designer's responsibility to do due diligence in making certain they do understand it. Kickstarting such a process is not an ethically responsible thing to do, unless they are sure it and its future iterations are aligned.
I would say that in a case like that we would still be well within the bounds of the existing legal frameworks to assign blame according to contributing factors, strength of causality, etc. There's no legal principle that only one person can be criminally liable for murder. Therefore, if the AI is a legal person, it could be criminally liable. If its designers designed it with murderous intention, or reckless disregard for life, so could they. It would just depend on the circumstances, as would the punishment. I'm sure the law would need to be updated to take into accounts the rights of created persons, if we ever get to that point, but it doesn't seem remotely insurmountable.
At least from an in-universe perspective, this is discussed in some length in the sequel, "2010". (Moreso in the novel than in the (underrated) movie adaptation.)
The explanation given is that HAL was given, in secret, orders not to reveal the true purpose of the mission to Poole and Bowman, the two crewmembers not in hibernation. Because even the designer[1] of HAL was not made aware of this secret instruction, they were not aware that HAL's fundamental design was incompatible with the concealment of information, and in attempting to obey the instruction, created what in humans would be called a mental health crisis.
[1]: "Dr. Langley" in 2001 the movie, but in the book and in all adaptations of the sequel, he has the wonderful name of "Dr. Sivasubramanian Chandrasegarampillai".
My favourite character naming is in the epilogue of 3001:
So every word is mine: well, almost every word. I must confess that I
found Professor Thirugnanasampanthamoorthy (Chapter 35) in the Colombo
telephone directory; I hope the present owner of that name will not object to
the loan.
I actually think the epilogues are my favourite parts of the series, as Clarke wraps up thoughts and ties in real world developments in the thirty-odd years between the titles.
Interestingly enough, both the names are Tamil who form the minority population in Sri Lanka, not from the Sinhalese majority.
Under British rule, a large part of the administration and educated class came from this community who were favoured by the British as part of their divide and rule policy and I'm sure the effects carried on at least 1-2 generations beyond independence.
The Sri Lankan civil war resulted largely from of reversing this policy in the opposite direction after independence.
I notice in the wikipedia page on "Tamil language" that it says this:
> Unlike most Indian languages, Tamil does not distinguish aspirated and unaspirated consonants.
If that's true, why do the Tamil names Chandrasekhar and Thirugnanasampanthamoorthy use the digraphs "th" and "kh"? What sounds do those represent, and how would they be different if they were spelled "t" and "k"?
The best guess I've been able to come up with is "Tamil shares a writing system with one or more other Indian languages that do distinguish aspirated consonants, and it just happens to spell these names with the aspirated version of the consonant", but I know literally nothing about this.
the Tamil written script indeed doesn't distinguish th and t or kh and k and indeed more sounds. However when they are spoken, especially by the more educated folks, they are pronounced differently from contextual knowledge.
the contextual knowledge in this case is that both these names are actually from Sanskrit. When you write Sanskrit in Devnagari script, it does indeed distinguish and so the educated folks will pronounce it correctly being aware of this.
Sadly, the astronauts were no different to HAL than the opposing chess pieces when he plays chess against Frank Poole.
When they became a threat to the mission they were discarded.
Cold logic. Murder is like "sin", something we ascribe to malice, against God. What HAL did is just as horrible, but perhaps not "murder"?
The article introduced me to the idea of mens rea. I take it there may come a time when a machine moves beyond cold logic and we might have to call a similar act murder.
Interesting Clarke suggesting it was perhaps a conflict within HAL to both conceal the mission from the two conscious astronauts and trying to be truthful.
> The article introduced me to the idea of mens rea. I take it there may come a time when a machine moves beyond cold logic and we might have to call a similar act murder.
If a human uses cold logic to decide to kill someone, is that murder? Why not if a machine does it then?
In many Western legal systems, to prove a crime like Murder there is normally two things the prosecution must demonstrate:
That you A. Carried out the act; typically known as the Actus Reus, and
That you B. Had the guilty intent or mind, typically known as the Mens Rea.
This is why the defense of "insanity" exists in so many legal systems; if you were insane you cannot satisfy condition B and therefore cannot have committed a crime such as murder, even if you performed the act of killing another human while insane.
However, this is a very accademic distinction in practice in most places. There is usually a second slightly lesser crime known as something like "Manslaughter" or "Culpable Homicide", which is in essence a Murder charge without B - you killed someone but it was an accident for example - you didn't actually mean for them to die. Such cases satisfy the Actus Reus but not the Mens Rea of murder, and are therefore often known as a crime other than "murder" itself.
The above is an absolute butchering of Western Criminal law practices, but the general distinction between the action and the intent to perform the action is found in a lot of places, and a Murder charge often requires prosecution to demonstrate both Actus Reus and Mens Rea or the charge cannot stand. The Manslaughter charge is sometimes the fallback position for when proving Mens Rea beyond reasonable doubt for Murder fails.
The elevator pitch for this entire article is really, "Was Hal capable of mens rea - yes or no?" So far no machine has ever reached that bar, at least to my knowledge! No one really doubts a machine can do the actus reus - of course it can.
Note that the intent (Mens Rea) for murder doesn't usually need to be Death.
In England for example the requirement for Murder is you intended at least "grievous bodily harm" and the victim in fact died.
For Attempted Murder, the intent proved must be Death, but for actual Murder, there is no need to show that the perpetrator specifically intended death.
It seems like the definition stretches enough for "chimp A murdered chimp B", or even the abstract use "murdered the environment". It's kill, but with an element of premeditation?
In English, the term murder usually means something closer to "the intentional unjustified killing of a sapient* being."
If it's not intentional, it's not murder, it's an accident. If it is justified, then it isn't murder, it's e.g., self-defense. If the creature killed isn't sapient, then it isn't murder, but it might be animal cruelty.
Note that in order for it to be intentional, the killer must also meet some criteria for sapience, but that's not necessarily the same as being human. For example, in English translations of the Bible, Jesus refers to Satan as "a murderer from the beginning." (John 8:44)
I think you may have left out explaining your asterisk?
I have not heard the definition to be about a sapient being. I always thought it would need to be a human killing a human. Perhaps in fantasy novels it can be a dwarf killing an orch, but in the real world I don't think anything else qualifies. If we include AI in this definition, we put AI above animals, which I think is a stretch for the forseeable future.
It is not justifiable to intentionally kill people. It is to be expected that some forms of self defence will often - even usually - kill the other party, but that's not intentional killing. Likewise, armed police are trained to shoot to incapacitate opponents. Yes, shooting a person in the centre of mass will sometimes kill them, but, that's not the intent of doing it.
This is maybe less obvious if your government intentionally kills its own citizens for some reason and you've found it important to draw a moral distinction between "We pay government officials to deliberately kill people" and "Murder" but I have no problem saying that's the same thing, stop doing that.
> I thought murder was the killing of a human by another human?
No, the killing of a human by a (legal) person (which includes both natural persons and all other things subject, in their own being, to the law as an actor rather than mere property, such as corporations) is homicide. For the question to even be coherent, one must presuppose that HAL is considered a legal person — a subject of legal responsibility — but humanity is not required.
Murder is unlawful (phrased another way, “without legal excuse”, self-defense is an excuse, for instance) homicide with malice aforethought. (“malice aforethought” includes, but is not limited to, intent to kill; see, e.g., the felony murder rule, depraved heart murder, etc.)
They didn't really murder HAL in the movie/story because they left him in a state that was restorable. So all they really did was hibernate him. Much like putting someone in a medical coma and then waking them up isn't killing them.
Autonomous computers likely have already killed humans in Libya two years ago.[1] Four problems with this new reality:
1) Misidentification. When selecting a target, will autonomous weapons be able to distinguish between hostile soldiers and 12-year-olds playing with toy guns?
2) Low-end proliferation. Market pressures could result in the creation and widespread sale of what can be thought of as the autonomous weapon equivalent of the Kalashnikov assault rifle: killer robots that are cheap, effective and almost impossible to contain as they circulate around the globe.
3) High-end proliferation. High-end autonomous weapons are likely to lead to more frequent wars because they will decrease two of the primary forces that have historically prevented and shortened wars: concern for civilians abroad and concern for one's own soldiers.
4) Undermining the laws of war. How can autonomous weapons be held accountable? Who is to blame for a robot that commits war crimes? Who would be put on trial? The weapon? The soldier? The soldier's commanders? The corporation that made the weapon? (from
https://www.commondreams.org/views/2021/12/30/humanitys-fina...)
Are these even serious discussions? Of course the people running these autonomous systems are responsible. It's like dropping a bomb on someone and saying "hey I didn't do it, blame the bomb!"
No it is not making its own decisions. There's always a clear path from programming to execution. The slight variability added by a neural net or whatever else doesn't change that fact.
If a serial killer chooses their victims by using a random number generator, the algorithm isn't going to be tried for murder.
I see No Country for Old Men as partly a study on the conflict between those who believe in free will and those who don't. The villain, Sugar, has shed the illusion of free will, and has thrown himself into the thing he knows motivates him: prejudicial termination of other people who stand in the way of his current clients' goals.
This is in contrast to the backdrop of a community reeling with outrage at and fear of his brutality. They see him as an agent of evil, some sort of amoral if not immoral demon, a spawn of the escalating narco wars. Normal people imbue themselves and others with causative powers. This is exemplified by the exchange with Norma Jean.
"You don't need to kill me. I ain't got any money."
"I'll flip a coin. It's the best I can do. Call it."
"No, I ain't doin' that. It ain't the coin that decides. It's you, it's always been you."
Someone built it and deployed it. These aren’t naturally occurring phenomena. If someone sets up an automated turret in a field that just fires at anything that moves, the person who put it there is responsible, not the turret.
Drones did not decide to fly to Libya and shoot “hostiles” all by themselves.
So if one corporation, let's say Skynet, built an autonomous killer robot and sold it to any interested government, who would be responsible?
Is Skynet suddenly responsible for the outcome of every situation the robots are placed in? What about the secondhand robot market? Are the governments who own the bots responsible? Are the governments responsible for errors? If your car's brakes get stuck and you hurt someone, isn't the manufacturer responsible? If a gun you own misfires due to a defect, wouldn't the manufacturer be responsible?
To me it seems that autonomous warbots are going to put the world in a tighter MAD bind. If X has killer robots, Y better have killer robots or strongly align themselves with X.
To break this down further in a sort of visual way:
- Agents involved in a bot-involved death
-- Operator
-- Owner
-- Manufacturer
-- Programmer/Engineer
-- Government representatives
-- Victim
-- Represented populace
What would make sense to me is that the represented pressure their representatives to regulate manufacturers in terms of sales and safety controls. If a sale is found to be by regulation then the manufacturer should be less liable, presuming their programmers and engineers followed safety regulations. If the sale is valid, then the operator should be responsible, otherwise the manufacturer is responsible as well. In the case of malfunction, the manufacturer should be held responsible along with individual engineers and programmers. If the malfunction was due to a gap in regulatory strictures, then the government responsible for the regulation, along with the manufacturer, should be held responsible.
To hold a soldier/operator criminally responsible for deploying an autonomous weapon that commits war crimes, prosecutors would need to prove both actus reus and mens rea, that is, a guilty act and a guilty mind.
This would be difficult as a matter of law, and possibly unjust as a matter of morality, given that autonomous weapons are inherently unpredictable.
The distance separating the soldier from the independent decisions made by autonomous weapons in rapidly evolving environments may be simply too great.
Personally? Yes. They brought the thing about. If they did not do their due diligence to endure that outcome eouldn't happen, responsibility, when traced out lay with them.
If you were hoping to trip me up on "how many layers of seperstion until it's not your fault", you got the wrong guy. Unfortunately, I endure maintaining some of the most banal systems imaginable specifically because if I can reasonably forsee an outcome I'm not comfortable being culpable for, I don't work on it.
...You'll slso be surprised how muchoral quandry there is in many systems we take for granted.
Someone is responsible for giving it executive function. The person culpable should be held accountable for the actions they let it make under that circumstance. In the case of AI models, training data should be subpoenaed and cross-referenced with the curators of the data or whoever oversaw the training of the model.
Landmines are autonomous weapons, just much slower than a computerized missile. If you think landmines should be banned, think the same regarding "intelligent" autonomous weapons.
To be clear this is the argument you appear to be making:
If I get a gun, and I shoot it without aiming, and then it hits someone, it doesn't matter that I wasn't aiming, it's my fault.
If I try to making it not my fault by putting the gun in a vice, with a mechanism that randomly pulls the trigger, and then it hits someone, it's my fault.
If I put that vice on a drone, and have the drone fly around randomly, still shooting randomly, and it hits someone, it's my fault.
If I make that drone start following a specific person, firing randomly, and it hits someone, it's my fault.
If I make that drone start following a specific person, shooting specifically at that person, it's my fault.
If that drone accidentally hits someone other than the target it is now ... not my fault?
If that drone gets confused and is following the wrong person, and hits that person it is now ... also not my fault?
A landmine is not an autonomous weapon. Autonomy suggests being able to select a target and harm only that target. A landmine just blows up when someone steps on it; there is no selection, thus no autonomy.
A landmine is programmed to attack a target of a specific weight when it enters a specific area. The targeting model was programmed by engineers to try and target soldiers (or armoured vehicles), and the locations they were placed were also for that purpose. However, it's possible for the heuristics to fail, and for a mine to blow up a child, because the weights weren't tuned correctly, or to fail to explode when a soldier steps on it.
By your same logic, any AI has no autonomy, because it does X when its code selects X. A guard dog robot attacks will act consistently given identical inputs.
1) Mines have had this problem for more than a hundred years, and mines from previous wars are still misidentifying targets to this day
2) Mines have also had this problem
3) Not sure this is even grounded in reality - WW2, Vietnam, Afghanistan, and Iraq wars showed no significant "concern for civilians abroad". WW1&2 showed the concern about life of a countries own soldiers was only important in as much as running out of them meant losing.
4) Seriously? If I make a machine that targets tanks, but I do it badly and it confuses them with school buses, the laws of wars would make me responsible. But that's also not a good argument in general, because the "laws of war" have only ever applied to the defeated.
Just watched this a few days ago (it's on HBO Max for those who subscribe, and it's a very high quality digitization). It's a fantastic movie that still holds its own as a science fiction tale and work of art.
The setup (spoilers):
- Impending malfunction that can't be duplicated by Hal's twin back on Earth
- Poole and Bowman discuss what to do in the pod
- Bowman goes out to fix the antenna and his pod-like vehicle sneaks up on him
- something happens (not shown) that causes Bowman and the vehicle to go spinning off into space
- Poole goes out in a second vehicle to retrieve the body
- Hal [apparently] shuts down the hibernation pods of the three other astronauts
- Hal refuses to let Poole (without helmet) back onto the ship, shuts down discussion
- Poole removes the emergency hatch and blows himself back into the ship
- Poole disconnects Hal's memory and logic controllers, as Hal pleads with him to talk things over.
In this portrayal, Hal is a sentient machine, not a brain in a vat connected to life support systems. This is made clear in the final sequence in the list above.
For audiences watching the movie (as opposed to reading the book), it's not clear if it acts with malice, detached self-interest, psychopathic delusions, or something else. Science Fiction at this time (mid-1960s) was already using crazed robots or corrupted/untrustworthy machine intelligences as a storytelling device, for instance in Herbert's Dune and Destination: Void, so these scenarios would not have been unfamiliar.
This is quite different than say, a Tesla driving its trusting owner off a cliff or a military drone programmed to kill or directed by human operators to fire a missile.
Murder presupposes will. It requires an executive function, like cognitive flexibility. Computers can do that, but we're going back into a circular argument of free will(unimpeded choice) versus automation(determinism).
As we create more complex systems and getting closer to AI, we are also discovering how much of a machine we are. We may never be able to have a clear distinction.
I found it interesting that, in some cultures, killing someone was not against the law of the land.
My understanding is that if you killed someone in ancient Rome, for instance, it was the duty of those nearest the dispute to either avenge the cowardly deed or acknowledge that the dead person had it coming.
I had always interpreted HALs actions as he had come to a most curious solution: he had to "complete the mission" but he couldn't let those humans awake know of the mission. This can be accomplished by having an "empty set" of human beings.
To arrive at the empty set involves humans dying, which then sets him up for his further actions (in inactions, in not opening the pod bay doors). This is the conflict which causes his breakdown.
So, partially, it was a failure of the people giving the orders that an empty set solution might be the one HAL arrived at: no crew, no crew to keep secrets from.
101 comments
[ 4.2 ms ] story [ 173 ms ] threadIf not, how is it different than a smart pile of meat?
Either AGI has free will and is responsible for its actions, or we don’t either.
Edit - responding to your comment edit, I don't know if it's common knowledge that HAL was written by humans (or not) but it's perhaps less clear with machine learning and AI algorithms used for modern autopilot systems, the systems learn and further program their own behavior beyond what the human programmers wrote, so in that sense I do think self-agency is emerging and as such there is responsibility on the part of both the car and it's creator, perhaps similar to how a parent is responsible for the independent actions of a young child.
And the answer wouldn't change: "Not unless we know it has independent subjectivity and self-awareness."
It always comes down to qualia. Our model of agency is based on a certain set of subjective experiences we take as working definitions of both self-awareness and free will.
We assume that other humans operate the same way. We can't know for sure, but life gets very complicated - and difficult - if we assume we're sentient but everyone else is just a meat robot. So assuming other humans have similar experiences is a reasonable position.
This breaks down with mechanical intelligence, because we can only understand it through visible behaviours. And of course you can have agency-like behaviours without human-like qualia and subjective awareness.
I strongly suspect this is not a solvable problem.
You can imagine some kind of hypothetical (magic) tool for measuring consciousness objectively. We can choose to believe in sentience, or we can choose not to. Neither position can be justified without objective measurement, and that isn't available without a magic device.
But even if it was available, the device would be paradoxical. The output of any objective consciousness measuring device still has to be experienced subjectively, which means it may be unreliable.
So - unsolvable.
Agency is just a nominative dignpost for saying that we can't say for certain what somebody's motives are given dome physical measures. That the outcome of a probability function with them as an agent is not deterministic in nature.
In that sense, HAL absolutely has agency. As certain as you to me.
But as soon as AI / ML becomes involved, it stops being deterministic and becomes probability. "Judgement" is asked of the robot arm. Is that image a person or a car door.
I think this is likely to hit before cars simply because so many of us drive cars but none of us operate giant robot arms.
Same would be true of people, were they easily "fixed".
Killing is rational in a sense - for some subset of purposes, in context of some subset of environmental pressures, both of which change constantly, and often faster than adaptation to kill does.
Accept this. It will not ever change.
People are self-reproducing robots that are made of meat. Robots made of meat are very hard to fix.
It all depends on if 'robots', however one defines them are 'conscious', however one defines that.
But until that gets settled, it's always a human pulling the trigger.
No difference from when someone's pitbull chews off someone else's head.
It might be the owner's fault, it might be not. But the dog in question can not be blamed.
[1] https://blog.msasafety.com/what-safety-integrity-level-means...
Those caveats apply to machines as well: if I tell a toaster to kill someone, and then the toaster electrocutes them, I would not be guilty.
Do you think there is an argument that I should not be responsible?
If “random” is too independent, or you want it to be systematic, we could do something else:
* the person with the highest “red hat” score according to a 20 year old copy of opencv
Or we could do something modern and “AI” and give it a completely random neural net that’s super deep and has three outputs: angle, pitch, and whether to pull the trigger.
Or we could go all out and just replace this hypothetical machine that acts like an independent person with the next best thing: a regular human.
If a government, or a terrorist group, gives a group of people (or soldiers) a bunch of weapons and drops them off in a city, and they start shooting, do you think that the government would be responsible?
This discussion mistakes existing policies as some objective base, but it’s the other way round.
I had no idea that was actually a thing. https://en.wikipedia.org/wiki/Elevator_surfing
The explanation given is that HAL was given, in secret, orders not to reveal the true purpose of the mission to Poole and Bowman, the two crewmembers not in hibernation. Because even the designer[1] of HAL was not made aware of this secret instruction, they were not aware that HAL's fundamental design was incompatible with the concealment of information, and in attempting to obey the instruction, created what in humans would be called a mental health crisis.
[1]: "Dr. Langley" in 2001 the movie, but in the book and in all adaptations of the sequel, he has the wonderful name of "Dr. Sivasubramanian Chandrasegarampillai".
"Wonderful"? It would appear to be a take-off on https://en.wikipedia.org/wiki/Subrahmanyan_Chandrasekhar
So every word is mine: well, almost every word. I must confess that I found Professor Thirugnanasampanthamoorthy (Chapter 35) in the Colombo telephone directory; I hope the present owner of that name will not object to the loan.
I actually think the epilogues are my favourite parts of the series, as Clarke wraps up thoughts and ties in real world developments in the thirty-odd years between the titles.
Under British rule, a large part of the administration and educated class came from this community who were favoured by the British as part of their divide and rule policy and I'm sure the effects carried on at least 1-2 generations beyond independence.
The Sri Lankan civil war resulted largely from of reversing this policy in the opposite direction after independence.
I notice in the wikipedia page on "Tamil language" that it says this:
> Unlike most Indian languages, Tamil does not distinguish aspirated and unaspirated consonants.
If that's true, why do the Tamil names Chandrasekhar and Thirugnanasampanthamoorthy use the digraphs "th" and "kh"? What sounds do those represent, and how would they be different if they were spelled "t" and "k"?
The best guess I've been able to come up with is "Tamil shares a writing system with one or more other Indian languages that do distinguish aspirated consonants, and it just happens to spell these names with the aspirated version of the consonant", but I know literally nothing about this.
the contextual knowledge in this case is that both these names are actually from Sanskrit. When you write Sanskrit in Devnagari script, it does indeed distinguish and so the educated folks will pronounce it correctly being aware of this.
When they became a threat to the mission they were discarded.
Cold logic. Murder is like "sin", something we ascribe to malice, against God. What HAL did is just as horrible, but perhaps not "murder"?
The article introduced me to the idea of mens rea. I take it there may come a time when a machine moves beyond cold logic and we might have to call a similar act murder.
Interesting Clarke suggesting it was perhaps a conflict within HAL to both conceal the mission from the two conscious astronauts and trying to be truthful.
If a human uses cold logic to decide to kill someone, is that murder? Why not if a machine does it then?
That you A. Carried out the act; typically known as the Actus Reus, and That you B. Had the guilty intent or mind, typically known as the Mens Rea.
This is why the defense of "insanity" exists in so many legal systems; if you were insane you cannot satisfy condition B and therefore cannot have committed a crime such as murder, even if you performed the act of killing another human while insane.
However, this is a very accademic distinction in practice in most places. There is usually a second slightly lesser crime known as something like "Manslaughter" or "Culpable Homicide", which is in essence a Murder charge without B - you killed someone but it was an accident for example - you didn't actually mean for them to die. Such cases satisfy the Actus Reus but not the Mens Rea of murder, and are therefore often known as a crime other than "murder" itself.
The above is an absolute butchering of Western Criminal law practices, but the general distinction between the action and the intent to perform the action is found in a lot of places, and a Murder charge often requires prosecution to demonstrate both Actus Reus and Mens Rea or the charge cannot stand. The Manslaughter charge is sometimes the fallback position for when proving Mens Rea beyond reasonable doubt for Murder fails.
The elevator pitch for this entire article is really, "Was Hal capable of mens rea - yes or no?" So far no machine has ever reached that bar, at least to my knowledge! No one really doubts a machine can do the actus reus - of course it can.
> https://en.wikipedia.org/wiki/Actus_reus
> https://en.wikipedia.org/wiki/Mens_rea
> https://en.wikipedia.org/wiki/Manslaughter
> https://en.wikipedia.org/wiki/Culpable_homicide
And good luck getting an investigator out to Jupiter to gather evidence for the prosecution.
In England for example the requirement for Murder is you intended at least "grievous bodily harm" and the victim in fact died.
For Attempted Murder, the intent proved must be Death, but for actual Murder, there is no need to show that the perpetrator specifically intended death.
If it's not intentional, it's not murder, it's an accident. If it is justified, then it isn't murder, it's e.g., self-defense. If the creature killed isn't sapient, then it isn't murder, but it might be animal cruelty.
Note that in order for it to be intentional, the killer must also meet some criteria for sapience, but that's not necessarily the same as being human. For example, in English translations of the Bible, Jesus refers to Satan as "a murderer from the beginning." (John 8:44)
I have not heard the definition to be about a sapient being. I always thought it would need to be a human killing a human. Perhaps in fantasy novels it can be a dwarf killing an orch, but in the real world I don't think anything else qualifies. If we include AI in this definition, we put AI above animals, which I think is a stretch for the forseeable future.
This is maybe less obvious if your government intentionally kills its own citizens for some reason and you've found it important to draw a moral distinction between "We pay government officials to deliberately kill people" and "Murder" but I have no problem saying that's the same thing, stop doing that.
No, the killing of a human by a (legal) person (which includes both natural persons and all other things subject, in their own being, to the law as an actor rather than mere property, such as corporations) is homicide. For the question to even be coherent, one must presuppose that HAL is considered a legal person — a subject of legal responsibility — but humanity is not required.
Murder is unlawful (phrased another way, “without legal excuse”, self-defense is an excuse, for instance) homicide with malice aforethought. (“malice aforethought” includes, but is not limited to, intent to kill; see, e.g., the felony murder rule, depraved heart murder, etc.)
previously: https://news.ycombinator.com/item?id=22000494
1) Misidentification. When selecting a target, will autonomous weapons be able to distinguish between hostile soldiers and 12-year-olds playing with toy guns?
2) Low-end proliferation. Market pressures could result in the creation and widespread sale of what can be thought of as the autonomous weapon equivalent of the Kalashnikov assault rifle: killer robots that are cheap, effective and almost impossible to contain as they circulate around the globe.
3) High-end proliferation. High-end autonomous weapons are likely to lead to more frequent wars because they will decrease two of the primary forces that have historically prevented and shortened wars: concern for civilians abroad and concern for one's own soldiers.
4) Undermining the laws of war. How can autonomous weapons be held accountable? Who is to blame for a robot that commits war crimes? Who would be put on trial? The weapon? The soldier? The soldier's commanders? The corporation that made the weapon? (from https://www.commondreams.org/views/2021/12/30/humanitys-fina...)
[1]https://www.npr.org/2021/06/01/1002196245/a-u-n-report-sugge...
If a serial killer chooses their victims by using a random number generator, the algorithm isn't going to be tried for murder.
This is in contrast to the backdrop of a community reeling with outrage at and fear of his brutality. They see him as an agent of evil, some sort of amoral if not immoral demon, a spawn of the escalating narco wars. Normal people imbue themselves and others with causative powers. This is exemplified by the exchange with Norma Jean.
"You don't need to kill me. I ain't got any money."
"I'll flip a coin. It's the best I can do. Call it."
"No, I ain't doin' that. It ain't the coin that decides. It's you, it's always been you."
"I got here the same way the coin did. Call it."
Who, then?
Drones did not decide to fly to Libya and shoot “hostiles” all by themselves.
Is Skynet suddenly responsible for the outcome of every situation the robots are placed in? What about the secondhand robot market? Are the governments who own the bots responsible? Are the governments responsible for errors? If your car's brakes get stuck and you hurt someone, isn't the manufacturer responsible? If a gun you own misfires due to a defect, wouldn't the manufacturer be responsible?
To me it seems that autonomous warbots are going to put the world in a tighter MAD bind. If X has killer robots, Y better have killer robots or strongly align themselves with X.
To break this down further in a sort of visual way:
- Agents involved in a bot-involved death
-- Operator
-- Owner
-- Manufacturer
-- Programmer/Engineer
-- Government representatives
-- Victim
-- Represented populace
What would make sense to me is that the represented pressure their representatives to regulate manufacturers in terms of sales and safety controls. If a sale is found to be by regulation then the manufacturer should be less liable, presuming their programmers and engineers followed safety regulations. If the sale is valid, then the operator should be responsible, otherwise the manufacturer is responsible as well. In the case of malfunction, the manufacturer should be held responsible along with individual engineers and programmers. If the malfunction was due to a gap in regulatory strictures, then the government responsible for the regulation, along with the manufacturer, should be held responsible.
See? Easy.
This would be difficult as a matter of law, and possibly unjust as a matter of morality, given that autonomous weapons are inherently unpredictable.
The distance separating the soldier from the independent decisions made by autonomous weapons in rapidly evolving environments may be simply too great.
But I wish governments could just ban AI
I mean, look at landmines, the original autonomous killing weapon.
There is no ambiguity who is responsible.
If you were hoping to trip me up on "how many layers of seperstion until it's not your fault", you got the wrong guy. Unfortunately, I endure maintaining some of the most banal systems imaginable specifically because if I can reasonably forsee an outcome I'm not comfortable being culpable for, I don't work on it.
...You'll slso be surprised how muchoral quandry there is in many systems we take for granted.
It's s curse.
Soldiers:
- aren’t naturally occurring phenomena
- just fires at anything that moves (though it wasn’t gp’s idea, so you may add “with a rifle” suffix to it)
- did not decide to fly to Libya and shoot “hostiles” all by themselves
If I get a gun, and I shoot it without aiming, and then it hits someone, it doesn't matter that I wasn't aiming, it's my fault.
If I try to making it not my fault by putting the gun in a vice, with a mechanism that randomly pulls the trigger, and then it hits someone, it's my fault.
If I put that vice on a drone, and have the drone fly around randomly, still shooting randomly, and it hits someone, it's my fault.
If I make that drone start following a specific person, firing randomly, and it hits someone, it's my fault.
If I make that drone start following a specific person, shooting specifically at that person, it's my fault.
If that drone accidentally hits someone other than the target it is now ... not my fault?
If that drone gets confused and is following the wrong person, and hits that person it is now ... also not my fault?
By your same logic, any AI has no autonomy, because it does X when its code selects X. A guard dog robot attacks will act consistently given identical inputs.
You've literally just said it is programmed. If it is programmed, how can it be autonomous?
AI is also programmed; it attacks when the score returned by the threat function exceeds a threshold.
Unless we're talking about AGI, in which case I misunderstood and assumed we were talking about AI that we can currently make.
2) Mines have also had this problem
3) Not sure this is even grounded in reality - WW2, Vietnam, Afghanistan, and Iraq wars showed no significant "concern for civilians abroad". WW1&2 showed the concern about life of a countries own soldiers was only important in as much as running out of them meant losing.
4) Seriously? If I make a machine that targets tanks, but I do it badly and it confuses them with school buses, the laws of wars would make me responsible. But that's also not a good argument in general, because the "laws of war" have only ever applied to the defeated.
If it is done by a powerful country, no one. If it is done by a less powerful country, someone. The same as is the case now.
The setup (spoilers):
- Impending malfunction that can't be duplicated by Hal's twin back on Earth
- Poole and Bowman discuss what to do in the pod
- Bowman goes out to fix the antenna and his pod-like vehicle sneaks up on him
- something happens (not shown) that causes Bowman and the vehicle to go spinning off into space
- Poole goes out in a second vehicle to retrieve the body
- Hal [apparently] shuts down the hibernation pods of the three other astronauts
- Hal refuses to let Poole (without helmet) back onto the ship, shuts down discussion
- Poole removes the emergency hatch and blows himself back into the ship
- Poole disconnects Hal's memory and logic controllers, as Hal pleads with him to talk things over.
In this portrayal, Hal is a sentient machine, not a brain in a vat connected to life support systems. This is made clear in the final sequence in the list above.
For audiences watching the movie (as opposed to reading the book), it's not clear if it acts with malice, detached self-interest, psychopathic delusions, or something else. Science Fiction at this time (mid-1960s) was already using crazed robots or corrupted/untrustworthy machine intelligences as a storytelling device, for instance in Herbert's Dune and Destination: Void, so these scenarios would not have been unfamiliar.
This is quite different than say, a Tesla driving its trusting owner off a cliff or a military drone programmed to kill or directed by human operators to fire a missile.
As we create more complex systems and getting closer to AI, we are also discovering how much of a machine we are. We may never be able to have a clear distinction.
My understanding is that if you killed someone in ancient Rome, for instance, it was the duty of those nearest the dispute to either avenge the cowardly deed or acknowledge that the dead person had it coming.
The world is a brutal place sometimes.
To arrive at the empty set involves humans dying, which then sets him up for his further actions (in inactions, in not opening the pod bay doors). This is the conflict which causes his breakdown.
So, partially, it was a failure of the people giving the orders that an empty set solution might be the one HAL arrived at: no crew, no crew to keep secrets from.