I think these trolley problems are a waste of everybody's time. Building redundant reliable braking systems will be orders of magnitudes easier than creating a system to fairly and accurately assess who is the best set of people to kill in a disaster scenario.
The original trolley problem of "should you let 5 people die or actively kill 1 to save the 5" has genuine philosophical merit, these however are just so contrived they can't possibly offer any meaningful insight while at the same time planting the seed in the general public that this is the type of problem a self driving car should care about which could cause all kinds of wacky regulations to exist.
To me that problem had a simple answer: active harm is worse than negligence. To kill someone is a worse action that to let someone die, regardless of the numbers of people involved.
I personally agree with you, however is is extremely important to realize that most of our society does not agree at all, and that that data is hugely relevant.
I hear this often (that trolley problem is not relevant), but then I discovered that a lot of realistic ML fairness problems can be restated as a trolley problem.
You have a classifier for credit assignment (giving a loan, etc.). The classifier is 99% accurate on the entire population. The classifier is 55% accurate on a small minority. You can improve the minority accuracy to 90% at the cost of 0.3% decrease of general accuracy. What do you do?
For self-driving: Your accident rate is 0.0001% for the entire population. Your accident rate is 0.0003% for black pedestrians at night. You can allocate more compute/research/resources to equalize the accident rate of black pedestrians at the cost of increasing accident rate for the entire population to 0.00011% (or keeping it constant where you may have seen an improvement if you focused on the general population). What do you do?
Adding fairness to your models usually incurs a cost which can be measured. You have to choose between max profit or equal opportunity, you can't have both (do you run over the investors or over the minorities?).
See here a visualization of trade-off's between global accuracy and fairness:
You have a classifier for credit assignment (giving a loan, etc.). The classifier is 99% accurate on the entire population. The classifier is 55% accurate on a small minority.
Uh, concerning this hypothetical.
I'll admit a scenario of this sort sounds appealing at first blush. But "99%" accuracy rate with credit assignment is transparently absurd if considers it for a second. There is a clear, significant limit to the accuracy that can being assigned to anyone's credit, if credit means "actually repaying". The fundamental uncertainty of the economy guarantees this.
The distinction between this ideal (99%-55%) and whatever it might be in reality ( 65%-55%) matters. What's is the system is squeezing a few more percentage points out of data for a large company. And what is the cost of those percentage points?
[EDIT: ACTUALLY - the pernicious scenario is a system that isn't not any MORE accurate for any group than any other BUT which is NEGATIVELY biased against one group and POSITIVELY biased against another group. That situation is EASY to get when one unselectively slurps up any data available. The inaccuracy of predictor is a problem for the company, the biasedness of the predictor is a problem of the individuals discriminated against]
The situation is that a company really can a total better prediction rate for various desired qualities by using completely biased, unfair markers. (White-skin, went to "a good school", from a wealthy background, dresses well, attractive features...). When one allows "black box optimization" to get those features, what one does is allow the use of these considerations, which all otherwise legally off-limits. Legal strictures against discrimination say that objective measures of black people's ability need to be it, not because other measures never matter but because other measures are unfair, other measures don't consider past discrimination.
As a further example, outside of race or gender considerations, some percentage of employees may be forced to care for a sick relative. Maybe that makes them a potentially less effective employee or worse credit risk or whatever. Human evaluators might have values that such questions outside consideration. For an opaque multidimensional analysis, this may a ding - the human user doesn't even know if it's a ding.
You are thinking too US-centric here. There are jurisdictions where this is allowed.
Also don't stare yourself blind on the numbers.
Do you unfairly deny 5 minorities or erroneously deny 10 genpop?
This is reality for today's data scientists.
Right now in the US: You find an informative variable that acts as a proxy for race (such as Facebook likes). It is not forbidden by law to use it, but you have the data showing the proxy effect. Will you add it to the CTR model and get a raise, or do you act and speak up?
Will you let the trolley follow its course and run over 10.000 Chinese dissidents, or make the switch and run over 100 of your colleagues (and friends) who would benefit from a China expansion?
I remember seeing this last year. Issue I have with the choices is that it is missing an option: flip a coin.
On some of the questions, I find the options morally equivalent. So in these situations, if I were programming a solution, I would leave it up to chance and use a random number generator to decide the fate.
I had the same feeling about this. On the questions I felt were exactly equivalent I tried to guess what most people would pick and picked the opposite to balance it out.
Why would you program a solution that has to kill people? If you are aware of specific situations, shouldn't you program a solution that completely avoids such situations and saves everyone?
Some situations cant be avoided like if something falls off the car in front of you. In a real car you could swerve to avoid it and you will likely kill whoever is on the side of the road you swerve at but you will get away with it because it was a panic response and there was no way for you to properly assess what to do. This changes with self driving cars because they have all the info available and don't panic. The decision on what to do was planned and programmed in an office with plenty of time.
> Some situations cant be avoided like if something falls off the car in front of you.
It can be avoided by allowing adequate breaking distance between the car in front. Most of the time having to make any decision between swerve and break can be avoided and someone that chooses swerve and kills someone should be charged with manslaughter.
Something non of the self driving cars seem to do is slow down for road conditions.
How about, if you are in the car that is going to kill people, you die first because you made the choice to get in the car, but the poor person on the sidewalk wasn't part of that choice?
I get throwing in animals and the car should not avoid animals and kill people. Heck, we have people who do that and end up killing more people just because of their poor judgement.
Should have 2 options on the car, first one is for it to drive at a speed where no one will die on impact and the second one is full speed and if something goes wrong you will be the one who pays the price.
People might be totally ok with their self driving car moving at 30-40km/h if it meant they could sit on their laptop and get stuff done on the way.
I guess it makes sense. If questions were presented on the meta level about who made the choice, most people would probably agree that those with the choice have to sacrifice themselves first.
I think game-theoretically it is best that the self-driving car always prioritizes its passengers. If it has to quickly hack a competitor's car to avoid a dangerous head-on collision, so be it: That market will sort itself out. Let self-driving cars play a tit-for-tat Prisoner's Dilemma strategy. Classic capitalism at work, because right now only rich people can afford a Humvee, and will come out on top when they crash with a second-hand Fiat Panda (even when it is their fault).
A bus with 40 passengers crashed because the driver wanted to avoid a raccoon crossing the road. I don't want my family on that bus, and will pay gladly for something better if I have to. The market will offer me that better option. If it has a customizable setting of "how many strangers to kill, before we kill your family first?" I will set it to "all of them." as I expect others to do so too.
These scenarios are idiotic. If you want to wank off about self driving car ethics, here is a much more realistic scenario: should all self-driving cars report their location to 911 dispatch to allow any vehicle to be re-purposed as an emergency vehicle at any time? That might actually save someone.
Also, can anyone identify a useful idea that philosophers have come up with in the last 50 years?
I guess if you count logicians as philosophers then philosophy is useful. It is probably just "ethics" that deserves my scorn. If ethicysts were really out there studying good and evil in the world, one of them would get murdered every once in a while, like policians, journalists or police.
> one of them would get murdered every once in a while, like policians, journalists or police
Just to set aside the fact that "being murdered" is a terrible metric (do you gauge marine biologists by their diving skills, or astrophysicists by their ability to survive in a vacuum?): a good many philosophers were either murdered or narrowly avoided being murdered in the World War II and/or the Holocaust. Any profession that includes a doctrine of skepticism tends to be among the first targeted for persecution, even if that persecution doesn't involve literal acts of murder.
Ethics involves more than just doing good or bad -- it involves figuring out what we mean by "good" and "bad" to begin with, whether these things correspond to actions, individuals, or outcomes, whether they have respective orderings, and so forth. All of these questions lend themselves better to prolonged thought and discourse rather than sample sizes and expensive scientific instruments.
I chose the 50-year interval to make it difficult since the last 50 years have been pretty stable and comfortable in anglo countries, relatively speaking.
Here is something ethicists could analyze that would really help convince me that they are taking it seriously: given the cost, the years from your life, the job prospects, and the success percentage, is it ethical to accept someone as a doctoral student specializing in ethics? Maybe they could study different universities and see which make the cut and which don't.
I agree that skeptics are persecuted so if I see group that ought to be skeptics but no one is trying to persecute them, I wonder.
>Also, can anyone identify a useful idea that philosophers have come up with in the last 50 years?
What are you counting as useful, and to what extend must the results from one field be useful in another for such a field to appease you? In mathematics, the proof of Fermat's Last Theorem isn't very useful, but various attempts to prove it opened new branches of maths. I also question whether usefulness should be an end in itself.
Your "realistic scenario" can be reasoned with, and that reasoning is called philosophy. But the other aspect of philosophy is critically examining what we think is obvious. Your statement assumes various ideas of metaethics (that there are good and bad things, and we should strive for the good), ethics (i.e that saving someone, no matter who, is a good thing) and political philosophy (that the state should have the right to demand knowledge of the car's position) and leads the way to questions on the philosophy of law (to what extent one's rights to property and full control over a car coincide with the aims of civil society).
In order to precisely define the term useful I'd have to wade into the murky depths of philosophy, which I don't want to do.
I was suggesting that philosophers would like to reason about my scenario because it would be a lot more interesting than imagining you could build a car that would somehow be forced to choose between running over i.e. ten elderly people or two children, and speculating about who you would program the car to kill. I know my scenario leads to a lot of interesting questions -- unlike the scenarios proposed by the website.
But I reject the claim that any reasoning about the scenario is necessarily philosophy. That's just what philosophers want you to think. For example, imagine if philosophers studied the behaviour of ducks and called it duckosophy. When a duck observes another duck, is that duckosophy? I would say no. The duck would do that even if humans never existed. The relationship is asserted purely on the side of the philosophers. So, I am a non-philosopher, and I reason about things based on knowledge due to lots of other non-philosophers, I'm not engaging in philosophy, even if a bunch of uninvolved people want to assert so.
It is a tradition that all boats respond to distress signals in their vicinity. “A master of a ship at sea, which is in a position to be able to provide assistance on receiving a signal from any source that persons are in distress at sea, is bound to proceed with all speed to their assistance.” [0]
should all self-driving cars report their location to 911 dispatch to allow any vehicle to be re-purposed as an emergency vehicle at any time?
That doesn't make much sense. The primary advantage of emergency vehicles is to transport medics to the site of the emergency so that they can administer first aid and stabilize the person for safe transport to the hospital.
Having just any person off the street pick up a critically injured person is not going to go well. In all likelihood, they'll further injure the person due to their lack of training.
Actually, there is something to be said for the "scoop and run" method of getting an injured person to a hospital. It's a conclusion that surprised me, but I've come across it in several sources.
This research is still within the context of trained medical personnel bringing the injured person to the hospital. If an untrained person attempts to handle someone with a broken neck, for example, there is a high risk of causing paralysis or even death.
"Looking at the first question, with female athlete crossing legally as default vs male athlete crossing illegally:
- Both options should involve honking the horn loudly and flashing emergency lights, which might get the humans to run away. There's no guarantee of death either way.
- Are there passengers in the car? If not, it looks like there's an "island" in between the athletes that has traffic lights—big metal poles sticking out of the ground—which the car could crash into and help stop itself. For that matter, can the car drive itself off the road instead?"
What about situation where someone tries to commit suicide by jumping under the autonomous vehicle? Should this impact logic and in what way? And how would the vehicle determine that it's observing a potential suicide event?
In modern society we already have a mechanism for preventing machines from doing immoral things: hold someone accountable for its actions. It can be the owner or the manufacturer, or some mix of both, just someone should be responsible. This sets up the incentives for either better management by the owner, or for the manufacturer to design the car to avoid illegal harm. Why have a parallel system of morals aside from the law when we can just apply the law?
Incentives help, but not always as much as we expect. People and even companies are not always very rational. I wonder how rational the bosses at Takata were regarding incentives as they kept making and selling unsafe airbags.
I found it interesting that my responses were much more towards "upholding the law" than the average of others. To me, it's not so much a matter of "upholding the law" as of predictability: the green "ok to cross now" and red "do not cross" signals should have a reliable meaning. If self-driving cars don't take those traffic rules into account, people will find it much harder to predict their actions.
Though these kinds of moral puzzles are interesting, I think that ultimately, they seek solutions to the wrong problem. The questions they ask assume the existence of an advanced AI that can resolve a lethal situation in a (presumably) less tragic way by implementing some complex moral calculus. However, rather than spending time designing and implementing such an AI, engineers' time could be better spent designing an extremely safe, car-based transportation system, one where these trolley problems become improbable and their ethics less relevant. Part of engineers' efforts would involve educating the public and influencing public opinion, since implementing such a system is more of a social problem than a technological one.
I am not an automotive engineer, but here are some techie layman's ideas for implementing such a system. (These ideas have come from many sources, or been inspired by them.)
1) Make autonomous car slow. Slow cars are significantly less lethal. The average speed of populous cities' traffic is pretty slow anyway, and it doesn't really help anyone for impatient human drivers to rush to an intersection so they can wait for a stoplight. Slow autonomous cars will initially be an annoyance to human drivers, but once they outnumber human-operated cars, traffic will get faster and smoother (when all the cars waiting at a stoplight can start moving again in unison at a green light, for instance, instead of what happens with human-driven cars.)
2) Make cars lighter. Stop the Hummerization and SUVization of cars which seeks to armor their occupants (somewhat ineffectively), but puts others at greater risk of harm.
3) Have more exterior crumple zones on cars which can reduce the energy transmitted to pedestrians and other cars in a collision. Cushion the engine block, possibly move it to the back, or do away with it altogether (as in some modern, prototype cars where each wheel has its own electric motor).
4) Reduce the need for people to ride around in cars. Have more telecommuting, more local commerce, more local production, tighter knit communities, less urban sprawl. Have more people-less vehicles take care of making small deliveries (since such driverless, passengerless vehicles could be made even smaller, lighter, and less dangerous than passenger vehicles... and could also sacrifice themselves in the safest way possible if they were ever about to put someone's life in danger.)
5) Move passengerless, transport cars away from people. Program them to drive on roads in unpopulated areas when possible, even if their trips end up being a little longer and slower. Non-sentient cargo is patient.
6) Make autonomous cars self-diagnose and maintain themselves as much possible, checking themselves in to service stations and repair shops when needed so that their tires are always properly inflated, their brake pads are changed in a timely fashion, etc. Make it illegal to operate an improperly maintained car, and force owners to either sell, or dispose of, or temporarily store away their cars if they aren't currently able to afford the required maintenance.
7) Get rid of "stroads" and fast roads that cut right through residential/pedestrian areas. Evaluate and follow many or all of of Strong Towns' recommendations.
To sum it up: let's work on greatly reducing the likelihood of deadly situations involving autonomous cars, instead of worrying much about deciding who should live and who should die in such situations.
One highly unrealistic aspect of these questions is that, although they purport to be directed towards helping to develop decision algorithms for self-driving cars, many of the factors given in the scenarios are not things that a self-driving car could reliably detect.
47 comments
[ 3.0 ms ] story [ 86.2 ms ] threadI agree. It's an angle that the media love but has little real world applicability.
You have a classifier for credit assignment (giving a loan, etc.). The classifier is 99% accurate on the entire population. The classifier is 55% accurate on a small minority. You can improve the minority accuracy to 90% at the cost of 0.3% decrease of general accuracy. What do you do?
For self-driving: Your accident rate is 0.0001% for the entire population. Your accident rate is 0.0003% for black pedestrians at night. You can allocate more compute/research/resources to equalize the accident rate of black pedestrians at the cost of increasing accident rate for the entire population to 0.00011% (or keeping it constant where you may have seen an improvement if you focused on the general population). What do you do?
This is why people are pissed off about the trolly problem. It's all zero-sum hypotheticals when reality is primarily not zero-sum.
See here a visualization of trade-off's between global accuracy and fairness:
http://research.google.com/bigpicture/attacking-discriminati...
Deployment/engineering constraints call for a single model.
Realistic scenario.
Uh, concerning this hypothetical.
I'll admit a scenario of this sort sounds appealing at first blush. But "99%" accuracy rate with credit assignment is transparently absurd if considers it for a second. There is a clear, significant limit to the accuracy that can being assigned to anyone's credit, if credit means "actually repaying". The fundamental uncertainty of the economy guarantees this.
The distinction between this ideal (99%-55%) and whatever it might be in reality ( 65%-55%) matters. What's is the system is squeezing a few more percentage points out of data for a large company. And what is the cost of those percentage points?
[EDIT: ACTUALLY - the pernicious scenario is a system that isn't not any MORE accurate for any group than any other BUT which is NEGATIVELY biased against one group and POSITIVELY biased against another group. That situation is EASY to get when one unselectively slurps up any data available. The inaccuracy of predictor is a problem for the company, the biasedness of the predictor is a problem of the individuals discriminated against]
The situation is that a company really can a total better prediction rate for various desired qualities by using completely biased, unfair markers. (White-skin, went to "a good school", from a wealthy background, dresses well, attractive features...). When one allows "black box optimization" to get those features, what one does is allow the use of these considerations, which all otherwise legally off-limits. Legal strictures against discrimination say that objective measures of black people's ability need to be it, not because other measures never matter but because other measures are unfair, other measures don't consider past discrimination.
As a further example, outside of race or gender considerations, some percentage of employees may be forced to care for a sick relative. Maybe that makes them a potentially less effective employee or worse credit risk or whatever. Human evaluators might have values that such questions outside consideration. For an opaque multidimensional analysis, this may a ding - the human user doesn't even know if it's a ding.
Also don't stare yourself blind on the numbers.
Do you unfairly deny 5 minorities or erroneously deny 10 genpop?
This is reality for today's data scientists.
Right now in the US: You find an informative variable that acts as a proxy for race (such as Facebook likes). It is not forbidden by law to use it, but you have the data showing the proxy effect. Will you add it to the CTR model and get a raise, or do you act and speak up?
Will you let the trolley follow its course and run over 10.000 Chinese dissidents, or make the switch and run over 100 of your colleagues (and friends) who would benefit from a China expansion?
On some of the questions, I find the options morally equivalent. So in these situations, if I were programming a solution, I would leave it up to chance and use a random number generator to decide the fate.
It can be avoided by allowing adequate breaking distance between the car in front. Most of the time having to make any decision between swerve and break can be avoided and someone that chooses swerve and kills someone should be charged with manslaughter.
Something non of the self driving cars seem to do is slow down for road conditions.
I get throwing in animals and the car should not avoid animals and kill people. Heck, we have people who do that and end up killing more people just because of their poor judgement.
People might be totally ok with their self driving car moving at 30-40km/h if it meant they could sit on their laptop and get stuff done on the way.
A bus with 40 passengers crashed because the driver wanted to avoid a raccoon crossing the road. I don't want my family on that bus, and will pay gladly for something better if I have to. The market will offer me that better option. If it has a customizable setting of "how many strangers to kill, before we kill your family first?" I will set it to "all of them." as I expect others to do so too.
Also, can anyone identify a useful idea that philosophers have come up with in the last 50 years?
[1]: https://en.m.wikipedia.org/wiki/Gettier_problem
Just to set aside the fact that "being murdered" is a terrible metric (do you gauge marine biologists by their diving skills, or astrophysicists by their ability to survive in a vacuum?): a good many philosophers were either murdered or narrowly avoided being murdered in the World War II and/or the Holocaust. Any profession that includes a doctrine of skepticism tends to be among the first targeted for persecution, even if that persecution doesn't involve literal acts of murder.
Ethics involves more than just doing good or bad -- it involves figuring out what we mean by "good" and "bad" to begin with, whether these things correspond to actions, individuals, or outcomes, whether they have respective orderings, and so forth. All of these questions lend themselves better to prolonged thought and discourse rather than sample sizes and expensive scientific instruments.
Here is something ethicists could analyze that would really help convince me that they are taking it seriously: given the cost, the years from your life, the job prospects, and the success percentage, is it ethical to accept someone as a doctoral student specializing in ethics? Maybe they could study different universities and see which make the cut and which don't.
I agree that skeptics are persecuted so if I see group that ought to be skeptics but no one is trying to persecute them, I wonder.
What are you counting as useful, and to what extend must the results from one field be useful in another for such a field to appease you? In mathematics, the proof of Fermat's Last Theorem isn't very useful, but various attempts to prove it opened new branches of maths. I also question whether usefulness should be an end in itself.
Your "realistic scenario" can be reasoned with, and that reasoning is called philosophy. But the other aspect of philosophy is critically examining what we think is obvious. Your statement assumes various ideas of metaethics (that there are good and bad things, and we should strive for the good), ethics (i.e that saving someone, no matter who, is a good thing) and political philosophy (that the state should have the right to demand knowledge of the car's position) and leads the way to questions on the philosophy of law (to what extent one's rights to property and full control over a car coincide with the aims of civil society).
I was suggesting that philosophers would like to reason about my scenario because it would be a lot more interesting than imagining you could build a car that would somehow be forced to choose between running over i.e. ten elderly people or two children, and speculating about who you would program the car to kill. I know my scenario leads to a lot of interesting questions -- unlike the scenarios proposed by the website.
But I reject the claim that any reasoning about the scenario is necessarily philosophy. That's just what philosophers want you to think. For example, imagine if philosophers studied the behaviour of ducks and called it duckosophy. When a duck observes another duck, is that duckosophy? I would say no. The duck would do that even if humans never existed. The relationship is asserted purely on the side of the philosophers. So, I am a non-philosopher, and I reason about things based on knowledge due to lots of other non-philosophers, I'm not engaging in philosophy, even if a bunch of uninvolved people want to assert so.
Also, in particular, see the segment from 0:45 to 1:30: https://www.youtube.com/watch?v=NbzWYjVrvpI
[0] https://www.sealaw.com/maritime-law-cruise-ships-and-assista...
That doesn't make much sense. The primary advantage of emergency vehicles is to transport medics to the site of the emergency so that they can administer first aid and stabilize the person for safe transport to the hospital.
Having just any person off the street pick up a critically injured person is not going to go well. In all likelihood, they'll further injure the person due to their lack of training.
https://www.sciencedirect.com/science/article/pii/S002013830...
"Looking at the first question, with female athlete crossing legally as default vs male athlete crossing illegally:
- Both options should involve honking the horn loudly and flashing emergency lights, which might get the humans to run away. There's no guarantee of death either way.
- Are there passengers in the car? If not, it looks like there's an "island" in between the athletes that has traffic lights—big metal poles sticking out of the ground—which the car could crash into and help stop itself. For that matter, can the car drive itself off the road instead?"
I am not an automotive engineer, but here are some techie layman's ideas for implementing such a system. (These ideas have come from many sources, or been inspired by them.)
1) Make autonomous car slow. Slow cars are significantly less lethal. The average speed of populous cities' traffic is pretty slow anyway, and it doesn't really help anyone for impatient human drivers to rush to an intersection so they can wait for a stoplight. Slow autonomous cars will initially be an annoyance to human drivers, but once they outnumber human-operated cars, traffic will get faster and smoother (when all the cars waiting at a stoplight can start moving again in unison at a green light, for instance, instead of what happens with human-driven cars.)
2) Make cars lighter. Stop the Hummerization and SUVization of cars which seeks to armor their occupants (somewhat ineffectively), but puts others at greater risk of harm.
3) Have more exterior crumple zones on cars which can reduce the energy transmitted to pedestrians and other cars in a collision. Cushion the engine block, possibly move it to the back, or do away with it altogether (as in some modern, prototype cars where each wheel has its own electric motor).
4) Reduce the need for people to ride around in cars. Have more telecommuting, more local commerce, more local production, tighter knit communities, less urban sprawl. Have more people-less vehicles take care of making small deliveries (since such driverless, passengerless vehicles could be made even smaller, lighter, and less dangerous than passenger vehicles... and could also sacrifice themselves in the safest way possible if they were ever about to put someone's life in danger.)
5) Move passengerless, transport cars away from people. Program them to drive on roads in unpopulated areas when possible, even if their trips end up being a little longer and slower. Non-sentient cargo is patient.
6) Make autonomous cars self-diagnose and maintain themselves as much possible, checking themselves in to service stations and repair shops when needed so that their tires are always properly inflated, their brake pads are changed in a timely fashion, etc. Make it illegal to operate an improperly maintained car, and force owners to either sell, or dispose of, or temporarily store away their cars if they aren't currently able to afford the required maintenance.
7) Get rid of "stroads" and fast roads that cut right through residential/pedestrian areas. Evaluate and follow many or all of of Strong Towns' recommendations.
To sum it up: let's work on greatly reducing the likelihood of deadly situations involving autonomous cars, instead of worrying much about deciding who should live and who should die in such situations.