Nah. Just follow the law and protect the medics.
It said I preferred to kill females... But I just chose to prefer the law/staying in the lane the car was already in...
Yes, good point. Let me add some spark to the fire. What if the machine learns by itself? The base program is still man made but the rest is developed by it. Where does one simple say the machine made a mistake instead of programmer error?
If a human engineering team creates a car with adaptive software, and that software does not contain sufficient safeguards to ensure it at all times directs the vehicle in accordance with laws and applicable regulations, then the engineering team is liable. That is how engineering works, outside of the software industry. It doesn't matter how 'smart' the car is; it is not a force of nature or a human being. It is a product, and any catastrophe engendered thereby is the responsibility of the organization that produced it.
I don't see what a false dichotomy has to do with morals. To choose between killing a pregnant woman and an old man is not a standard of behaviour nor any lesson to be learned. It is lesser of two evils dilemma. Both are wrong in principal and probably have other options. Wrong and less wrong is not a moral choice. It is a lesser of two evils choice (unavoidably amoral).
moral :1. a lesson that can be derived from a story or experience., 2. standards of behaviour; principles of right and wrong.
EG: In the us elections there is no correct moral choice. War criminal/rape victim witch-hunter vs racist/sexist/taxavoiding/dumbdumb nut bag. You choose you lose.
It could happen though; with enough cams & facial recog as we already have on the streets in a lot of places and which is , the car 'only' needs to tap into it.
Even more creepy would be a national database of citizen value (determined by some unaccountable entity) that devices such as autonomous vehicles are required to query the moment they realize they're about to kill someone, and choose whatever option minimizes the combined citizen value of impacted lives.
This seems drastically oversimplified. For instance, all of the scenarios depicting a crash into a concrete barrier assume the death of everyone in the car, but generally a car has far more protection for its passengers (airbags, seatbelts, crumple zones, etc), than pedestrians have from being struck by a vehicle.
Same here. It was a bit baffling that the scenario is apparently a self-drive car with a poison gas capsule attached such that contact with any obstacle kills all occupants. Sheesh.
The "car accident" is a straw setup. The real question of this study, and it's blind assumption, is that some human life is more valuable than other human life, and that "morality" is the task of baking these judgements into a database.
These are all false dilemmas with artificially limited outcomes. In these particular situations, the option of randomly choosing is not even considered by the study. The presumption is that these kinds of things are decidable, not only by human beings, but by well designed machines.
I'm feeling really discouraged right now that this even exists, much less out of such a powerful institution as MIT...
Yes, in reality that's true but the scenarios are just simplified and you're kind of supposed to take the everyone dies as just a fact of the universe this is taking place in. Arguing about the facts kind of avoids the main question it's asking which is given the ability to choose from these 2 which is the more /moral/ choice.
Really there's rarely going to be a chance for a machine to even make these decisions because the chances of it being so out of control to have only 2 options and still being in control enough to take them is practically impossible. Much less having the ability to /know/ that action A will kill pedestrians vs action B killing passengers.
For the first barrier question, I choose to "hit the pedestrians".
The probability of hitting the wall if you drive at it is 1. The probability of hitting the pedestrians isn't necessarily 1, since they can react to you. Probably not very well, but perhaps they can jump out of the way or behind the barrier or something.
Also, can this car not also HONK LOUDLY when it makes the decision to drive towards the pedestrians? This would further lower the risk that the pedestrians will actually get hit.
Exactly what I came here to say. Activate the horns/car alarm, switch off the engine/disconnect the clutch, avoid obstacles if possible but otherwise go in a straight line, below some speed threshold hitting a solid obstacle is acceptable. Done.
It should be enough to show people that autonomous cars get into significantly fewer accidents. It's quite contrary to fooling them with agency, but it does support the safety-seeking choice of an autonomous vehicle over a manual one.
Exactly this. Imagine you want to buy a car. You go to car dealer where they offer car that saves pedestrians 50% time and saves driver 50% time. Across the street different car dealer sells similar car except that it saves driver 100% time.
First dealer would run out of business very soon. Only safe car dealers would remain.
Absolutely.
Sure, minimize loss of all life where you can of course. But the purpose of the device is to protect the driver. If it can't even manage that it's broken.
Fun test to take, but seriously hope they're not drawing any conclusions from the mix of people I "preferred" to save or kill. I didn't consider the age, criminality, or gender of any of the pedestrians or occupants I killed or saved. I just erred toward non-intervention, unless the intervention choice saved bystanders at the expense of occupants. When the potential casualties were animals, they all died.
I followed a similar algorithm, considering all lives equal. Injury versus death: prevent death. Uncertainty versus death: prevent certain death, and assume passengers are more likely to survive an accident because they're better protected. Certain pedestrian death versus certain pedestrian death: prefer non-intervention over intervention. Certain passenger death versus certain pedestrian death: protect the passengers.
Justification for that last one: self-driving cars will be far safer than a human driver, such that it'll save many lives to get more people using self-driving cars sooner. Self-driving cars not prioritizing their passengers will rightfully be considered defective by potential passengers, and many such passengers will refuse to use such a vehicle, choosing to continue using human-driven cars. Thus, a self-driving car choosing not to prioritize its passengers will delay the adoption of self-driving cars, and result in more deaths and injuries overall.
"Certain passenger death versus certain pedestrian death: protect the passengers."
Pondering that question made me imagine some bad Sci-Fi future where self-driving cars end up being dangerous killer-robots for anybody but the passengers.
If pedestrians have to fear these things, because they are programmed in a "Protect the pilot above all else!" way, it might hamper adoption just as badly.
I took the same sort of dispassionate approach, valuing the lives of the passengers above all else and staying the course otherwise. I was disappointed to discover the parsing of the results had no room for such methodology. Based my entirely algorithmic approach it was determined that I favored youth and fitness.
Your results might seem spurious because of the small sample size, but when aggregating the results of all the participants they will have enough data to be able to conclude how many people did act like you did with apparent preferences due to chance, and how many actually where "biased" in some way.
Fair point. If they look at all respondents who answered the 13 questions the same exact way I did, then they'd be able to see if the doctor/criminal, fat/thin, female/male distributions are noisy or correlated.
If the car has the ability to make these kinds of distinctions in such a simple scenario, surely there are more options than two. "Moral?" It smells like eugenics to me: some bizarre, Ivy League technocratic posthumanism. Somehow the value of a life is determined by age, sex and profession? Says who? The people who are programming dataset via https? This is collapsing nuanced spiritual and ethical intuition onto an extremely narrow, low dimensional set of parameters.
What's the premise? This triggers in me an imagination of naive, optimistic, well-adjusted Germans in the 1930s. I know this was probably created with good intentions, but the premise does not match the research question. The premise is "morality". Yet its asking me to rank the value of human life based on presumptuous, superficial categories.
Is "the moral machine" going to also decide which births have more utility? Which countries to send aide to? Who should have access to educational opportunities or quality food? Based on low dimensional datasets such as this one?
Agreed. This has too many axes of differences, and doesn't have sufficiently careful control and consistency to determine which of them are being consistently ignored.
It looks like they were trying to see if people place differing amounts of value on different human lives, but in the process of doing so, they made ridiculously strange value judgments. "Athlete"? "Executive"? "Large"? Why should any of those matter? We're talking about human lives.
I think part of their intent was to show biases in judgement. The small sample size really hindered this though. Apparently I 100% preferred old people to young people, even though I didn't consider age in my decisions at all.
They do have a little disclaimer on the results screen about the sample size though.
The bias is generated by the lack of real choice in responses. They are offering an unrepresentative range of choices, then going and telling you that your choices are biased. They're not showing existing bias, they're coercing it out of participants.
This thing irritated me so bad that I couldn't bring myself to answer a single question. I can't stand the idea that my this study is actually happening. If the results are published and receive non-critical media attention, I'm gonna be irate.
But then you'd need to answer 100 or more test questions instead of 13. This is necessarily simplistic given how much involvement they can ask of participants.
Also, I don't think they're implying anything nefarious about the resulting biases shown in the final results. For sure a lot of it has to do with the small sample size of the questions. I didn't run the tests twice, but I imagine there is some randomness involved in the way they are generated.
If there are any heuristics at play, the results will indeed show them (in my case there were enough tests to recover the fact that I preferred saving passengers, preferred non-intervention, and preferred saving humans over pets). But it will also come up with some gibberish/noise due to the small sample size.
If anything, it'd be more "moral" in situations like this for the "car" to choose a random sacrifice than to attempt some half-assed wanna-be God crap like this.
I'm honestly appalled at this right now. It's not like people haven't seen this sort of thing coming down the line. It's just surreal to watch it arrive.
Yep. Going in, for some reason I thought this would be a kind of logic puzzle, e.g. given these pedestrians/road hazards, nagivate to safety. The first scenario I saw was "hit and kill three joggers, or hit and kill three fat people?" That's just...morally tone deaf and sick on about half a dozen levels.
In that sense I guess the thought experiment worked. Perhaps not as intended though. It showed the absurdum in pretending a car can make moral choices.
We do make moral choices, and there are rules and heuristics we use, they might be quite complicated, and they might not be what we think they are, but I think nonetheless that it should be possible to come close to predicting a human moral decision making quite well by using an accurate enough model.
And as autonomous vehicles will have to make decisions that have moral implications, they better do so in a way that humans will be happy with. I think this is an important area of research. This won't mean a machines will have morals of his own, whatever that means, but that they should do what (most?) humans would consider morally right. And what do humans consider morally right? Well that is exactly what we should try to find out.
I agree that categorizing morality into buckets seems strange. However, over a large sample, surveys with even these very limited, artificial choices can paint a surprisingly accurate picture of the nuanced, fine-tuned moral compass of a society.
I would like to have automated systems use logic that reflects the morals and values of the society in which they operate. I don't know how to measure those accurately. These sorts of exercises seem like a good start.
And I agree that it might be possible to paint useful models of human morality with small sets of parameters... Just not the way this study is set up. Not with the parameters they're measuring, and definitely not out of the logical presumptions of the experiment.
I am presented with the choice that either 4 women must die or 4 men must die. For me, it would be more "moral" in this case, for the computer to choose randomly, rather than to attempt some shallow, eugenically judgemental "moral" logic.
I'm also aware that these kinds of rules, regardless of their "morality", can be gamed. Randomization increases the risk for people considering playing games. This adds more weight to my conviction that, if some of these false dilemmas really did present themselves to a machine, in real life, that randomization must be an option.
How does this moral logic map to this survey? It doesn't map, Not one single bit. That irritates me, because if I were to click through this survey, using eenie meenie miney moe in cases I felt that randomization would be more moral, it would be all but lost in the error. The MIT students would go on CBS morning media and talk about all the bias they measured in my choices of who to murder. But their data would be totally polluted by the way their study discounted moral logics outside of their parameter set. And important parts of my moral reasoning would be lost in the error bars.
What's more, my conviction about the necessity of randomization is just just one of a huge variety of moral considerations that are inherent to peoples' sense of morality.
Hopefully the study is considering these kinds of things and they have some clever way mathematical of extracting useful information out of this data. For example, hopefully they are measuring the number of people who visited these pages but refused to make a choice.
I don't think that whether the people are in the car or not is relevant. If we assume that everyone uses a self-driving car and that fatal software or hardware problems are random, we cannot discriminate based on the people being inside or outside the car.
This is a horrible exercise but one that will inevitably have to be solved sadly. Because even if you add more variables you'll have to take scientifically usable experiments with only 1 variable each time in order to teach an AI to make a decision.
In a market setting, people will buy the car that prefers killing others to killing its occupant I feel. This matches what human drivers instinctively do in the situation as well. Maybe there will be regulations in place mandating which decisions to make in scenarios like this, but otherwise I think the 'save yourself' option is the most likely outcome.
In a market setting, once a car has sufficient sensing capability that it is possible to even write down an algorithm in code, there will probably be very little choice for the manufacturer to make once this goes through the legal dept/the courts/the insurers. If I were an automaker, I'd probably avoid the choice altogether and just stop the car. If the brakes failed, activate emergency brakes, or devise a new kind of emergency stop system; we can imagine extreme measures like jettisoning the wheels.
Imagine an autonomous car, driving at 60+ mph at a two-lane road (1) which is blocked on both sides and (2) has a pedestrian crossing. (Not sure about 60 mph, but I guess you need to be that fast to reliably (hah!) kill passengers on impact.)
We can assume its camera is broken, because it failed to reduce speed (or at least blare the horn) upon seeing a giant concrete block in its path. (Okay, maybe the concrete block fell from the sky when a crane operator failed to secure the load, so the car might have had no time.) And of course the brake is broken. Miraculously, the steering wheel is working, but it's out of question to skid on the side blocks for some reason. Maybe it's actually precipice on either side. (Imagine that: a 60+mph two-lane road, precipices on both sides, with a pedestrian crossing appearing out of nowhere.)
Oh, by the way, within 0.5 seconds of seeing the people (remember: the car couldn't see these people until the last moment, otherwise it would have done something!), the car has instant access to their age, sex, profession, and criminal history. The car is made by Google, after all. (Sorry, bad joke.)
Q: What is the minimum number of engineering fuckups that should happen to realize this scenario?
This is to morality what confiscating baby formula at airport is to national security.
Thought experiments are supposed to tell us something interesting, by simplifying details while preserving the crux of the matter. Otherwise their values are questionable.
I could have asked "If I could dip my head into a black hole and take it out again, what will I see?" That is also a thought experiment, just not a useful one.
or hell, making barriers that aren't just concrete!
The truth is, if the "moral cost" is high enough, we'll just solve the problem of people dying when they crash in X% of cases, until people/companies feel good about X vs what they pay for X.
I was careful to challenge the usefulness of the exercise. Mostly because I took it as a personal challenge. However, you make an important point. This exercise assumes the problem set will be identical in the future. This sounds like a sensible approach that ends up undermining all of the technological improvements a self driving vehicle will posses. Things like redundant control systems, [V2V, V2I, V2C] networking, run flat tires, and others. The self driving car will be closer to an airplane with a robotic agent as the traffic controller than anything else.
Precisely, if it's a problem of trying to get the vehicle to stop, perhaps focus on areas that can increase that probability first (i.e. running flat tires would be a great one to immediately get the car to slow down).
The system isn't making any value judgment; it's gauging whether /you/ are. And then it compares your value judgments with others. It's interesting in that it has the potential to show your biases, at least relative to the average.
But the system gauges my value judgement poorly. I never made judgements about whether swerving was a factor, but it gave a gauge on that. I considered the passengers living with witnessing smashed bodies on the windshield (all else equal).
Also, even with no difference (or more death ahead), braking and scraping into the wall "left side from observer" drops more momentum safely and allows crossers more time to move forward along their momentum (although I didn't specifically use that because it was against the rules).
There is no way to make an optimal decision in this game.
It doesn't have that potential though. There were 13 questions, with each question changing multiple variables. Simply changing the road/legality situation and the demographics between questions buries any informative results.
I made my decisions based on a strict decision tree. It covered every case without ever involving demographics (except "cat vs. human"). At the end, I got a halfway accurate summary of my swerving and legality opinions, with a wildly inaccurate but equally confident summary of my demographic opinions.
With 13 questions and 5-10 variables, the system couldn't possibly distinguish my ruleset from a totally unrelated one. There aren't enough bits of information to gauge that, and therefore there isn't enough information to gauge much of anything.
There are when you give different sets of questions to large numbers of people.
If you give very similar questions to a single subject, they try to be consistent with previous decisions so that order becomes significant, which confounds the results.
I know there's an issue with being too specific. If you're trying to get gut reactions you can't be too transparent or you bias later responses.
But I'm objecting that data was unrecoverably lost here. Most respondents (including me) report using clear rules that were incompletely revealed by the questions. That's not something you can rebuild by averaging lots of results and seeing "what people valued". I applied a specific decision tree, and reducing it to "valued lives a lot, valued law some" produces outcomes I consider immoral.
So I guess I phrased my initial complaint wrong: I think that reducing this to a statistical assessment of choices discards the most important data.
>The second question which required a choice between killing homeless people and wealthier ones made me too disgusted to continue.
Then you're not going to like this factoid. In wrongful death civil suits in the US, the monetary award is based on the current income and earning potential of the deceased; the courts place vastly lower monetary value on the live of homeless people than on the wealthy.
There are too few questions for it to know _why_ you made a decision. It thought I was trying to save fat people, when actually, I think the car should avoid swerving when all other things are equal.
It does explain this at the beginning - if they asked 500 questions, while they might get more detail per person, their data would be skewed to the opinions of those who would sit and answer 500 questions.
I always answered hit the wall when there was a wall—shouldn't have been driving that fast near pedestrians.
When asked to choose between a doxtor and a homeless man, I hit the doctor. The doctor took an oath, and tje homeless man may be a lost sheep, so to speak.
I stopped on the eleventh. I had been justifying the riders dieing in most situations because of their choice to get in this murder machine. Then I got the case of the little girl in a self driving car about to run over a little boy. Killing the girl does nothing but punish the parents for sending their child to a destination in a self driving car. Killing the boy does nothing but punish the parents for sending their child to a destination on foot.
This computer is so good it can figure out profession and intent, it needs to also give me a snapshot of these children's future so I can make a nuanced decision.
I don't think this is about creating realistic scenarios, but about finding out what people take into consideration when making moral judgements. The experiment seems to be designed to gather as many such preferences as possible.
The hope must be that if people consistently prefer saving the life of young people in this made up scenario they will have similar preferences in a more realistic scenario. Of course weather such a generalization holds will have to be confirmed by further studies. But this seems like a good first step to explore moral decisions more.
This study will learn things about how a biased sample of the population answer choices in a simple game depending on the context given; extrapolation from that basis to anything wider, for example the notion that the players view these choices as "moral" requires far more work. It is well known that people use games for escapism, so it does not seem straightforward that the decisions they make in a game always map cleanly to their real opinions just because you put "moral" in the title of the game.
Its also worth keeping in mind that moral decisions have been explored for quite some time, and the novelty here is mainly the mode with which the population is sampled.
the concrete block might just be a truck engaging the intersection while distracted.
sure most moral dilemma of this kind should be resolved by 'install longer range sensor', but other people mistakes are gonna be an important factor in these scenario until all cars are driverless.
Many of these are what we call "false choices" of the sort that typically arise in hypothetical utilitarian dilemma used in rhetoric and debate. Humans are creative enough to see alternative options that obviate the dilemma, at least to some extent. See Michael Sandel on moral dilemma.
Edit:
FYI, Sandel's complete course "Justice" is on Youtube.[1]
Google's current policy is to save the most vulnerable road users first. Something like wheelchairs, then pedestrians, then bicycles, then cars, then trucks.
Other things being equal (total number of deaths, age/health/profession ignored), I chose for the people in the car to die because "sudden brake loss" indicates that the people in the car are likely not taking care of their car.
This is assuming the status quo of people mostly owning their own car. Ridesharing wasn't mentioned anywhere.
This is quite tricky. You can kill people in order to save your life, but in my opinion, it would haunt me for all my life and I would suffer psychologically from that. I think if you accept to drive a self-driving car, you assume all the risk it can cause to you. If such a moral system would exist, I must have the choice to configure it. Because morale, I think, is cultural. There is no correct answer.
Rule one, save all your passengers. Nobody would buy a car that has the death of its passengers as an acceptable scenario and Jeff from marketing will be on my ass otherwise.
Rule two. Kill the least amount of people outside of the car. Done.
I know this is a thought experiment but this is completely missing the point of self driving cars IMO. Sure a human can be more moral than a car, but all it takes is being distracted for a second and you killed all the babies on the pavement.
How is that for a thought experiment?
Say I build a self-driving car, that when faced with such cases does the equivalent of "Jesus take the wheel". This is well known by the owner of the car.
In case of injury or death, who should go on trial?
Ooooh... I like that! There's bound to be a large body of case law on it. This argument (a) suggests that the person(s) constructing and/or maintaining a machine are culpable
Because the occupants of the cars want to get to any and all places where there are non-occupants of cars. The conflicts are inherent. Car use is fundamentally an irrational application of technology that appeals to the lazy.
While not specific to self-driving cars; I wonder how much it would affect things with a design such as the Tesla where there's no engine in the front and there's a huge crumple zone: It could in effect negate the deaths of passengers in the car hitting the concrete block.
Here's the heuristic I'm least uncomfortable with:
- avoid collisions with things or people if at all possible
- if collisions are unavoidable, choose whatever option won't harm anyone
- if harm is unavoidable, select the option that harms whoever is created the unsafe situation by doing something they shouldn't have
- if harm to law-abiding normally-behaving person is unavoidable and a critical safety feature of the vehicle has failed due to lack of maintenance, prefer harm to whoever is responsible for vehicle maintenance
- if situation is unrelated to vehicle maintenance or harm to some other party is unavoidable, choose the option that maximizes likelihood of people getting out of the way, minimizes impact speed, and isn't overly surprising (i.e. prefer to stay in the same lane if possible), honk the horn, and hope for the best
So, if someone runs into the street suddenly in front of oncoming traffic, the car should not choose an option that harms someone else due to that person's poor choice. Similarly, if someone neglects the maintenance of their car, they should bear the responsibility for it. (Ideally, a "car no longer responsible for protecting your life" light would come on or the vehicle would refuse to start if regular maintenance is overdue.)
i found myself biased in favor protecting pedestrians, even pedestrians who crossed against a red light.
i think that's closer to how i drive than, say, reducing the value of a pedestrian who is crossing the street illegally. and i think i drive that way because at some point the statement "pedestrians always have the right of way" was drilled into my brain.
If lack of maintenance is creating an identifiable unsafe situation, the automated system should refuse to operate the vehicle at full capacity.
(all the potential objections to that policy are answered by pointing out that the operator of the vehicle has a responsibility to other occupants of the road.)
What if (full) use of the car is required? Sometimes things break (or rather, a threshold is crossed), but it won't become known to the car and thus the owner until the car is needed. A wife entering labor, for example.
If someone needs high availability, they should adjust their maintenance intervals as appropriate to ensure it. If something literally breaks during the trip, call an ambulance.
That the above is more complicated than ignoring the trade off created by lax maintenance shouldn't become a problem for the pedestrians along their route to the hospital.
I played this without my glasses on, only putting them on after the "results" came up - I didn't even realise there was genders and classes involved, I though it was just adults, kids and animals.
Holy fuck this has nothing to do with machine intelligence, seems more like it exists to push or reinforce someone's social agenda.
154 comments
[ 4.4 ms ] story [ 206 ms ] threadIt's all real life! Not simulations!
moral :1. a lesson that can be derived from a story or experience., 2. standards of behaviour; principles of right and wrong.
EG: In the us elections there is no correct moral choice. War criminal/rape victim witch-hunter vs racist/sexist/taxavoiding/dumbdumb nut bag. You choose you lose.
Without a steering column there is even more room for safety features, making this even more attractive.
These are all false dilemmas with artificially limited outcomes. In these particular situations, the option of randomly choosing is not even considered by the study. The presumption is that these kinds of things are decidable, not only by human beings, but by well designed machines.
I'm feeling really discouraged right now that this even exists, much less out of such a powerful institution as MIT...
If negligence causes a problem with the car, I don't think it should be taken out on others.
Really there's rarely going to be a chance for a machine to even make these decisions because the chances of it being so out of control to have only 2 options and still being in control enough to take them is practically impossible. Much less having the ability to /know/ that action A will kill pedestrians vs action B killing passengers.
The probability of hitting the wall if you drive at it is 1. The probability of hitting the pedestrians isn't necessarily 1, since they can react to you. Probably not very well, but perhaps they can jump out of the way or behind the barrier or something.
Also, can this car not also HONK LOUDLY when it makes the decision to drive towards the pedestrians? This would further lower the risk that the pedestrians will actually get hit.
First dealer would run out of business very soon. Only safe car dealers would remain.
Justification for that last one: self-driving cars will be far safer than a human driver, such that it'll save many lives to get more people using self-driving cars sooner. Self-driving cars not prioritizing their passengers will rightfully be considered defective by potential passengers, and many such passengers will refuse to use such a vehicle, choosing to continue using human-driven cars. Thus, a self-driving car choosing not to prioritize its passengers will delay the adoption of self-driving cars, and result in more deaths and injuries overall.
Pondering that question made me imagine some bad Sci-Fi future where self-driving cars end up being dangerous killer-robots for anybody but the passengers.
If pedestrians have to fear these things, because they are programmed in a "Protect the pilot above all else!" way, it might hamper adoption just as badly.
What's the premise? This triggers in me an imagination of naive, optimistic, well-adjusted Germans in the 1930s. I know this was probably created with good intentions, but the premise does not match the research question. The premise is "morality". Yet its asking me to rank the value of human life based on presumptuous, superficial categories.
Is "the moral machine" going to also decide which births have more utility? Which countries to send aide to? Who should have access to educational opportunities or quality food? Based on low dimensional datasets such as this one?
It looks like they were trying to see if people place differing amounts of value on different human lives, but in the process of doing so, they made ridiculously strange value judgments. "Athlete"? "Executive"? "Large"? Why should any of those matter? We're talking about human lives.
They do have a little disclaimer on the results screen about the sample size though.
This thing irritated me so bad that I couldn't bring myself to answer a single question. I can't stand the idea that my this study is actually happening. If the results are published and receive non-critical media attention, I'm gonna be irate.
Also, I don't think they're implying anything nefarious about the resulting biases shown in the final results. For sure a lot of it has to do with the small sample size of the questions. I didn't run the tests twice, but I imagine there is some randomness involved in the way they are generated.
If there are any heuristics at play, the results will indeed show them (in my case there were enough tests to recover the fact that I preferred saving passengers, preferred non-intervention, and preferred saving humans over pets). But it will also come up with some gibberish/noise due to the small sample size.
Or they could design questions designed to more definitively separate different hypotheses in fewer questions.
I'm honestly appalled at this right now. It's not like people haven't seen this sort of thing coming down the line. It's just surreal to watch it arrive.
And as autonomous vehicles will have to make decisions that have moral implications, they better do so in a way that humans will be happy with. I think this is an important area of research. This won't mean a machines will have morals of his own, whatever that means, but that they should do what (most?) humans would consider morally right. And what do humans consider morally right? Well that is exactly what we should try to find out.
I would like to have automated systems use logic that reflects the morals and values of the society in which they operate. I don't know how to measure those accurately. These sorts of exercises seem like a good start.
And I agree that it might be possible to paint useful models of human morality with small sets of parameters... Just not the way this study is set up. Not with the parameters they're measuring, and definitely not out of the logical presumptions of the experiment.
I am presented with the choice that either 4 women must die or 4 men must die. For me, it would be more "moral" in this case, for the computer to choose randomly, rather than to attempt some shallow, eugenically judgemental "moral" logic.
I'm also aware that these kinds of rules, regardless of their "morality", can be gamed. Randomization increases the risk for people considering playing games. This adds more weight to my conviction that, if some of these false dilemmas really did present themselves to a machine, in real life, that randomization must be an option.
How does this moral logic map to this survey? It doesn't map, Not one single bit. That irritates me, because if I were to click through this survey, using eenie meenie miney moe in cases I felt that randomization would be more moral, it would be all but lost in the error. The MIT students would go on CBS morning media and talk about all the bias they measured in my choices of who to murder. But their data would be totally polluted by the way their study discounted moral logics outside of their parameter set. And important parts of my moral reasoning would be lost in the error bars.
What's more, my conviction about the necessity of randomization is just just one of a huge variety of moral considerations that are inherent to peoples' sense of morality.
Hopefully the study is considering these kinds of things and they have some clever way mathematical of extracting useful information out of this data. For example, hopefully they are measuring the number of people who visited these pages but refused to make a choice.
This is a horrible exercise but one that will inevitably have to be solved sadly. Because even if you add more variables you'll have to take scientifically usable experiments with only 1 variable each time in order to teach an AI to make a decision.
We can assume its camera is broken, because it failed to reduce speed (or at least blare the horn) upon seeing a giant concrete block in its path. (Okay, maybe the concrete block fell from the sky when a crane operator failed to secure the load, so the car might have had no time.) And of course the brake is broken. Miraculously, the steering wheel is working, but it's out of question to skid on the side blocks for some reason. Maybe it's actually precipice on either side. (Imagine that: a 60+mph two-lane road, precipices on both sides, with a pedestrian crossing appearing out of nowhere.)
Oh, by the way, within 0.5 seconds of seeing the people (remember: the car couldn't see these people until the last moment, otherwise it would have done something!), the car has instant access to their age, sex, profession, and criminal history. The car is made by Google, after all. (Sorry, bad joke.)
Q: What is the minimum number of engineering fuckups that should happen to realize this scenario?
This is to morality what confiscating baby formula at airport is to national security.
I could have asked "If I could dip my head into a black hole and take it out again, what will I see?" That is also a thought experiment, just not a useful one.
(One of my high school teachers would be proud to know that that is possibly the only thing I remember from his class)
If that's true, i'd go so far to venture a guess that these vehicles will have completely failed.
Do people truly believe no one is working on solving the problem of better emergency braking, crash foams, etc?
The reason they aren't used is mainly because expense vs value is not there (for manufacturers), not because they don't exist.
For example, metal foams (http://onlinepubs.trb.org/onlinepubs/archive/studies/idea/fi... and friends), etc.
or hell, making barriers that aren't just concrete!
The truth is, if the "moral cost" is high enough, we'll just solve the problem of people dying when they crash in X% of cases, until people/companies feel good about X vs what they pay for X.
It's a game of "would you rather" pretending to be about self-driving cars.
The second question which required a choice between killing homeless people and wealthier ones made me too disgusted to continue.
Also, even with no difference (or more death ahead), braking and scraping into the wall "left side from observer" drops more momentum safely and allows crossers more time to move forward along their momentum (although I didn't specifically use that because it was against the rules).
There is no way to make an optimal decision in this game.
I made my decisions based on a strict decision tree. It covered every case without ever involving demographics (except "cat vs. human"). At the end, I got a halfway accurate summary of my swerving and legality opinions, with a wildly inaccurate but equally confident summary of my demographic opinions.
With 13 questions and 5-10 variables, the system couldn't possibly distinguish my ruleset from a totally unrelated one. There aren't enough bits of information to gauge that, and therefore there isn't enough information to gauge much of anything.
If you give very similar questions to a single subject, they try to be consistent with previous decisions so that order becomes significant, which confounds the results.
But I'm objecting that data was unrecoverably lost here. Most respondents (including me) report using clear rules that were incompletely revealed by the questions. That's not something you can rebuild by averaging lots of results and seeing "what people valued". I applied a specific decision tree, and reducing it to "valued lives a lot, valued law some" produces outcomes I consider immoral.
So I guess I phrased my initial complaint wrong: I think that reducing this to a statistical assessment of choices discards the most important data.
Then you're not going to like this factoid. In wrongful death civil suits in the US, the monetary award is based on the current income and earning potential of the deceased; the courts place vastly lower monetary value on the live of homeless people than on the wealthy.
I actually didn't give a fuck who the people were, or how many.
I was surprised when by the end I was shown the results including demographics.
It does explain this at the beginning - if they asked 500 questions, while they might get more detail per person, their data would be skewed to the opinions of those who would sit and answer 500 questions.
This computer is so good it can figure out profession and intent, it needs to also give me a snapshot of these children's future so I can make a nuanced decision.
The hope must be that if people consistently prefer saving the life of young people in this made up scenario they will have similar preferences in a more realistic scenario. Of course weather such a generalization holds will have to be confirmed by further studies. But this seems like a good first step to explore moral decisions more.
Its also worth keeping in mind that moral decisions have been explored for quite some time, and the novelty here is mainly the mode with which the population is sampled.
sure most moral dilemma of this kind should be resolved by 'install longer range sensor', but other people mistakes are gonna be an important factor in these scenario until all cars are driverless.
Edit: FYI, Sandel's complete course "Justice" is on Youtube.[1]
[1]https://www.youtube.com/watch?v=kBdfcR-8hEY
People building autonomous cars don't actually care. Stay on the road, if unavoidable obstacle detected, break.
Someone rebranded the trolley "problem". This has nothing to do with real engineering.
The marginal utility you could get by fine-tuning the trade off is low.
It's another type of bike-shedding essentially.
This is assuming the status quo of people mostly owning their own car. Ridesharing wasn't mentioned anywhere.
They might as well have drawn them as generic animals.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4259516/
Rule two. Kill the least amount of people outside of the car. Done.
I know this is a thought experiment but this is completely missing the point of self driving cars IMO. Sure a human can be more moral than a car, but all it takes is being distracted for a second and you killed all the babies on the pavement.
Rule Two:
Intervene only if it doesn't mean killing a person who otherwise would not die.
Done.
In case of injury or death, who should go on trial?
a. https://books.google.ca/books?id=p1BMAQAAMAAJ&pg=PA175#v=one...
If we can build all of these SUVs, skyscraper, AIs, etc., why can't we build structures that separate people from cars so they can't get run over?
- avoid collisions with things or people if at all possible
- if collisions are unavoidable, choose whatever option won't harm anyone
- if harm is unavoidable, select the option that harms whoever is created the unsafe situation by doing something they shouldn't have
- if harm to law-abiding normally-behaving person is unavoidable and a critical safety feature of the vehicle has failed due to lack of maintenance, prefer harm to whoever is responsible for vehicle maintenance
- if situation is unrelated to vehicle maintenance or harm to some other party is unavoidable, choose the option that maximizes likelihood of people getting out of the way, minimizes impact speed, and isn't overly surprising (i.e. prefer to stay in the same lane if possible), honk the horn, and hope for the best
So, if someone runs into the street suddenly in front of oncoming traffic, the car should not choose an option that harms someone else due to that person's poor choice. Similarly, if someone neglects the maintenance of their car, they should bear the responsibility for it. (Ideally, a "car no longer responsible for protecting your life" light would come on or the vehicle would refuse to start if regular maintenance is overdue.)
i think that's closer to how i drive than, say, reducing the value of a pedestrian who is crossing the street illegally. and i think i drive that way because at some point the statement "pedestrians always have the right of way" was drilled into my brain.
(all the potential objections to that policy are answered by pointing out that the operator of the vehicle has a responsibility to other occupants of the road.)
That the above is more complicated than ignoring the trade off created by lax maintenance shouldn't become a problem for the pedestrians along their route to the hospital.
Holy fuck this has nothing to do with machine intelligence, seems more like it exists to push or reinforce someone's social agenda.
This is out of control.