It was a test EV with level two autonomy (driver is always in control) features that are currently in production vehicles. It sounds like the test driver's family was in the car as well.
I hope everyone is as well as can be. Having media attention like this focused on an apparent driving error must be terrible.
The autonomous driving label was probably on the car to comply with a German privacy law.
That doesn't work for me. The car was clearly "self driving" when it crossed the lane line. Even at very slow speeds that can be deadly because we're talking about something that can happen in less than a second or two.
You cannot tell someone "this car can drive itself but you must be ready to take control at all times because if it fails it's your fault" while knowing that a failure can happen in less than a second.
From what I've found you're more than twice as likely to get into a wreck when using this tech.
> The car was clearly "self driving" when it crossed the lane line
Because it's impossible the driver was driving it manually and made a mistake?
It's entirely possible that this was indeed a testing incident (although late afternoon and family with the driver doesn't sound like it, could still be), but we do not know that yet.
You'll have to forgive me for not taking an insurance company's uncited number at face value.
> Stats estimate that per million miles driven, there are 9.1 self-driving car accidents, whereas there are only 4.1 crashes per million miles driven with human-operated vehicles.
What stats? Where are these stats?
Also the type/severity of accident matters here as well which further muddies the water. All that said I'm not willing to accept the status quo just because the new tech has caused a few deaths (5 according to the link you provided) since the status quo is 30-40K/year [0]. Obviously the important stat is a per-capita-type number that gives us "deaths per car of a given automation level, 0-5".
I agree that source is lacking. I did find this though, and it says "Verified Tesla Autopilot Death" are at 15 now, but the total deaths from Tesla involved crashes are at 290 since 2013. So you may be more likely to die from being smacked by a Tesla than being in one using their autopilot feature:
Level 2 automation doesn't typically switch lanes.
I'd wager 10-to-1 odds this was an active mistake by the driver. It is possible, however, that the car was in lane-following mode and got horribly confused (in which case, it is still a mistake by the driver, just different).
> You cannot tell someone "this car can drive itself but you must be ready to take control at all times because if it fails it's your fault" while knowing that a failure can happen in less than a second.
You can; you only need to be powerful enough to dilute responsibly for failures enough to get away with it. It is self evident that the margin for error is too small to rely on human intervention. It is absurd to expect humans to maintain the equivalent of manual driving situational awareness at all times. But when you're hell bent on selling automated driving you lose the capability of grasping these things.
Level 2 is the equivalent of Tesla's Autopilot. The driver is not always in control, but they must always be monitoring and be ready to immediately assume control.
(In many ways, L2 is just about the dumbest imaginable level of automation)
This is just wrong. Innocents being injured or killed while testing this tech is more akin to a negligent murder via technology than an "accident".
Many years ago the 3M company created a magnetic tape for the lane lines on roads and a device to detect those for automobiles, along with a gizmo that would vibrate the driver's seat on the side the vehicle was getting too close to crossing the lane lines.
That could help prevent these kinds of incidents, but cameras trying to detect painted lane lines have many obvious fail points and one of them is keeping the car in it's lane. This incident proves that once again.
You cannot tell a human "this car can drive itself but you need to be ready to take control at all times" because that's a clear oxymoron. It either works, or it doesn't, and if it doesn't it's potentially deadly, and it only takes a second to become deadly, and humans cannot react to that failure 100% of the time. That's been proven too many times already.
> Innocents being injured or killed while testing this tech is more akin to a negligent murder via technology than an "accident".
I'd like to see us treat autonomous road safety the way we treat aircraft safety, where even minor incidents are thoroughly investigated and often lead to industry-improving changes. Their approach clearly works because despite the fact that we hurl hundreds of thousands of people through the air every day in millimetre thick metal tubes, it's the safest way to travel.
Leaving autonomous driving to the whims of the empathy-deficient utilitarianist VCs of the world, and barely regulating their work at all, is really not ok.
That is what I am trying to say. It is not feasible. There are less airplanes or pilots compared to all cars and drivers out there. You can't have the same standard
If our goal is truly to work towards Level 5 autonomy, then by definition most accidents couldn't involve human error other than a person hurling themselves in front of such a vehicle.
> “The vehicle has a level 2 driving assistance system that is already incorporated in production vehicles today and which can support the driver on demand,” the company said. “With level 2 vehicles the driver always retains responsibility.”
> BMW added that the vehicle was required to be marked as a test car for data protection purposes, because it was recording footage.
This was not self-driving to start with and could end up being 100% operator error. Though even if it was self-driving (again, it wasn't) I'm nowhere near ready to call it "negligent murder via technology". It's terrible when self-driving tech causes accidents like this but opponents of self-driving seems all too comfortable ignoring the other crashes that happen daily and on top of that there is no way to know how many accidents have been prevented by self-driving/driver assist systems. At the end of the day I care a lot more about the net-difference in car deaths, not the causes of only ones that involved "technology" (obviously accidents per-capita, by car automation type, should be taken into account here). Should we also get rid of drive-by-wire? Since that's tech that can fail? Where does this end?
The question was not really a quantitative one. The purpose of asking that question is to determine the mindset of the person answering.
Safety systems don't generally get built with an MVP mindset, but that seems to be what is happening with autonomous cars and it's concerning to me on multiple fronts.
1 less death/accident than cars with only human drivers per capita. And level 1+ automation must be in separate buckets.
For example:
9 accidents out of 100 cars on the road in some level of driver automation (bucketed per-level) is acceptable if there were 10 accidents of 100 cars on the road with only a human driver over the same timespan (probably some kind of rolling average).
We should absolutely investigate the causes of driverless car (at any level) crashes and penalize companies if they are at fault and the issue was foreseeable but a lot people seem to think the only acceptable answer is “0 accidents” which is just absurd. Don’t let good be the enemy of great.
I'd like to invite you to consider that with the (I would say) utilitarian view that one less death is an acceptable MVP, you are implicitly saying that you'd be ok with products being sold in the full knowledge that they would autonomously kill tens of thousands of people per year.
While it may be true that perfect is the enemy of good, there is also such a thing as setting the bar too low.
> you are implicitly saying that you'd be ok with products being sold in the full knowledge that they would autonomously kill tens of thousands of people per year.
Which is the status quo for both cars and a bunch of other industries (pharma, tabaco, alcohol, fast food, the list goes on). Nothing is ever perfect (though many things don't have "death" as a potential failure case) but you are only looking at the downside and ignoring the lives that would be saved.
I don't think many families think "Well our Bobby died in a car accident that would have been preventable by technology but even if he had access to it thousands of other people would still die even while using that tech so we are ok with him dying".
> Which is the status quo for both cars and a bunch of other industries
The key difference here is that alcohol and tobacco do not autonomously kill people. We can feel safe blaming deaths caused by them on the individuals who purchased and used those products. Same with automotive deaths. As soon as the blame goes from diffuse individuals to a single locus, the torts will come out in full-force. If a self-driving car kills 10% as many people per vehicle-mile as the average individual driver, then the company making it will get sued out of existence.
I don't think the answer is to not allow autonomous cars to be sold, I think the answer is to engineer self-driving cars with the same kind of rigour that the aircraft industry uses, rather than the kind of rigour that startups use.
If you don't accept that cars may be sold that cause slightly fewer deaths than the Ford F-150, what's your argument for accepting that the Ford F-150 is sold?
(Or, if you will, substitute any other concrete existing model of car for the one I named.)
We accept that human drivers are not perfect and are going to cause thousands of deaths per year with their vehicles.
It's a whole different ballgame to accept that computers are going to choose to kill thousands of people per year, and it is not much comfort to know that it's slightly fewer thousands of people. I think this is especially true when the response to such deaths is not "we will investigate thoroughly and ensure this never happens again".
This is not a logically consistent argument, but humans are not logically consistent beings.
Hum, It depends. In absolute terms, yes, of course. But 9 accidents by machines among 100 drivers aren't necessarily an improvement in all aspects over 10 accidents caused by humans among 100 drivers.
In the first case, we "remove" people at random from a pool of drivers good and bad, so we have more percentage of bad drivers after all. In the second we remove much more bad drivers than good drivers so is much more safe for the rest. Humans can learn from past experiences and apply creative solutions to complex problems, machines are slower in this sense.
It's basically a space race between companies to build a self driving car, except nobody knows what they're doing and they're willing to put as many lives on the line as necessary. We have Elon Musk and his bullshit to thank for this.
Billions of people travel in cars. Those lives are on the line every day. One million people die in car crashes each year. The amount of injuries to people and property is huge. Preventing most of this carnage is a major goal of these self driving car initiatives. How many deaths from someone not following explicit instructions of always being ready to take over from the beta version of some self driving car system would you accept to save 1/2 a million lives per year? One, ten, one hundred, one thousand, more?
>That could help prevent these kinds of incidents, but cameras trying to detect painted lane lines have many obvious fail points and one of them is keeping the car in it's lane. This incident proves that once again.
Hold on a second. Lane-assist is a perfectly fine feature that is obviously meant to be used in conjunction with an alert driver with their hands on the wheel. That is typically dubbed 'Level 1' of autonomous driving, similar to adaptive cruise control. This accident wasn't related to that, but instead to a 'Level 2' autonomous driving where there is some level of driving automation.
Have a BMW X5 with this auto-steer nonsense and have had several incidents where it's abruptly turned the wheel, where if I weren't grabbing it it would have caused an accident. Ended up disabling the assistant system.
Two years ago an rental Audi A2 almost crashed me several times into the tunnel wall on the right side. It was a rainy night and sometimes when I drove into the tunnel, the car steared really hard right.
I wonder why they were testing this if it's exactly the same as the production system? Because if I read it correctly that's what they're saying now. That makes no sense. There must be something beta in there.
At least it sounds the thing was chock full of cameras and telemetry recorders so it shouldn't be hard to find out exactly what happened, and prevent it in the future.
I would hesitate to refer to being in a condition where you are watching a machine, ready to grab it the moment it tries to kill you, as "easy", but I understand how you're using the word.
48 comments
[ 4.1 ms ] story [ 118 ms ] threadhttps://www.autoevolution.com/news/bmw-ix-with-an-autonomous...
It was a test EV with level two autonomy (driver is always in control) features that are currently in production vehicles. It sounds like the test driver's family was in the car as well.
I hope everyone is as well as can be. Having media attention like this focused on an apparent driving error must be terrible.
The autonomous driving label was probably on the car to comply with a German privacy law.
That doesn't work for me. The car was clearly "self driving" when it crossed the lane line. Even at very slow speeds that can be deadly because we're talking about something that can happen in less than a second or two.
You cannot tell someone "this car can drive itself but you must be ready to take control at all times because if it fails it's your fault" while knowing that a failure can happen in less than a second.
From what I've found you're more than twice as likely to get into a wreck when using this tech.
Because it's impossible the driver was driving it manually and made a mistake?
It's entirely possible that this was indeed a testing incident (although late afternoon and family with the driver doesn't sound like it, could still be), but we do not know that yet.
Literally not a scrap of proof that is the case here and yet you are willing to state it as fact.
> From what I've found you're more than twice as likely to get into a wreck when using this tech.
Source? This smells like FUD.
https://getjerry.com/questions/how-many-fatalities-have-been...
> Stats estimate that per million miles driven, there are 9.1 self-driving car accidents, whereas there are only 4.1 crashes per million miles driven with human-operated vehicles.
What stats? Where are these stats?
Also the type/severity of accident matters here as well which further muddies the water. All that said I'm not willing to accept the status quo just because the new tech has caused a few deaths (5 according to the link you provided) since the status quo is 30-40K/year [0]. Obviously the important stat is a per-capita-type number that gives us "deaths per car of a given automation level, 0-5".
[0] https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/...
https://www.tesladeaths.com/
I'd wager 10-to-1 odds this was an active mistake by the driver. It is possible, however, that the car was in lane-following mode and got horribly confused (in which case, it is still a mistake by the driver, just different).
You can; you only need to be powerful enough to dilute responsibly for failures enough to get away with it. It is self evident that the margin for error is too small to rely on human intervention. It is absurd to expect humans to maintain the equivalent of manual driving situational awareness at all times. But when you're hell bent on selling automated driving you lose the capability of grasping these things.
(In many ways, L2 is just about the dumbest imaginable level of automation)
Except that it literally helps 100000s of people make their drives more enjoyable and is perfectly fine in a in many situations.
Many years ago the 3M company created a magnetic tape for the lane lines on roads and a device to detect those for automobiles, along with a gizmo that would vibrate the driver's seat on the side the vehicle was getting too close to crossing the lane lines.
That could help prevent these kinds of incidents, but cameras trying to detect painted lane lines have many obvious fail points and one of them is keeping the car in it's lane. This incident proves that once again.
You cannot tell a human "this car can drive itself but you need to be ready to take control at all times" because that's a clear oxymoron. It either works, or it doesn't, and if it doesn't it's potentially deadly, and it only takes a second to become deadly, and humans cannot react to that failure 100% of the time. That's been proven too many times already.
I'd like to see us treat autonomous road safety the way we treat aircraft safety, where even minor incidents are thoroughly investigated and often lead to industry-improving changes. Their approach clearly works because despite the fact that we hurl hundreds of thousands of people through the air every day in millimetre thick metal tubes, it's the safest way to travel.
Leaving autonomous driving to the whims of the empathy-deficient utilitarianist VCs of the world, and barely regulating their work at all, is really not ok.
> “The vehicle has a level 2 driving assistance system that is already incorporated in production vehicles today and which can support the driver on demand,” the company said. “With level 2 vehicles the driver always retains responsibility.”
> BMW added that the vehicle was required to be marked as a test car for data protection purposes, because it was recording footage.
This was not self-driving to start with and could end up being 100% operator error. Though even if it was self-driving (again, it wasn't) I'm nowhere near ready to call it "negligent murder via technology". It's terrible when self-driving tech causes accidents like this but opponents of self-driving seems all too comfortable ignoring the other crashes that happen daily and on top of that there is no way to know how many accidents have been prevented by self-driving/driver assist systems. At the end of the day I care a lot more about the net-difference in car deaths, not the causes of only ones that involved "technology" (obviously accidents per-capita, by car automation type, should be taken into account here). Should we also get rid of drive-by-wire? Since that's tech that can fail? Where does this end?
In your view, what is the MVP for self-driving car safety, in terms of the number of crashes/deaths per day?
Safety systems don't generally get built with an MVP mindset, but that seems to be what is happening with autonomous cars and it's concerning to me on multiple fronts.
somewhere around 1 death per hundred million miles seems reasonable.
For example:
9 accidents out of 100 cars on the road in some level of driver automation (bucketed per-level) is acceptable if there were 10 accidents of 100 cars on the road with only a human driver over the same timespan (probably some kind of rolling average).
We should absolutely investigate the causes of driverless car (at any level) crashes and penalize companies if they are at fault and the issue was foreseeable but a lot people seem to think the only acceptable answer is “0 accidents” which is just absurd. Don’t let good be the enemy of great.
While it may be true that perfect is the enemy of good, there is also such a thing as setting the bar too low.
Which is the status quo for both cars and a bunch of other industries (pharma, tabaco, alcohol, fast food, the list goes on). Nothing is ever perfect (though many things don't have "death" as a potential failure case) but you are only looking at the downside and ignoring the lives that would be saved.
I don't think many families think "Well our Bobby died in a car accident that would have been preventable by technology but even if he had access to it thousands of other people would still die even while using that tech so we are ok with him dying".
The key difference here is that alcohol and tobacco do not autonomously kill people. We can feel safe blaming deaths caused by them on the individuals who purchased and used those products. Same with automotive deaths. As soon as the blame goes from diffuse individuals to a single locus, the torts will come out in full-force. If a self-driving car kills 10% as many people per vehicle-mile as the average individual driver, then the company making it will get sued out of existence.
(Or, if you will, substitute any other concrete existing model of car for the one I named.)
It's a whole different ballgame to accept that computers are going to choose to kill thousands of people per year, and it is not much comfort to know that it's slightly fewer thousands of people. I think this is especially true when the response to such deaths is not "we will investigate thoroughly and ensure this never happens again".
This is not a logically consistent argument, but humans are not logically consistent beings.
In the first case, we "remove" people at random from a pool of drivers good and bad, so we have more percentage of bad drivers after all. In the second we remove much more bad drivers than good drivers so is much more safe for the rest. Humans can learn from past experiences and apply creative solutions to complex problems, machines are slower in this sense.
Hold on a second. Lane-assist is a perfectly fine feature that is obviously meant to be used in conjunction with an alert driver with their hands on the wheel. That is typically dubbed 'Level 1' of autonomous driving, similar to adaptive cruise control. This accident wasn't related to that, but instead to a 'Level 2' autonomous driving where there is some level of driving automation.
Ahem, the Airline industry already found out why this not true. Automation always requires more training not less.
Meatbags do it all the time.
Two people die in a car accident every minute [0] and nobody cares. (Actually, two and a half, but that's just too gory.)
[0] https://www.who.int/news-room/fact-sheets/detail/road-traffi...
I wonder why they were testing this if it's exactly the same as the production system? Because if I read it correctly that's what they're saying now. That makes no sense. There must be something beta in there.
At least it sounds the thing was chock full of cameras and telemetry recorders so it shouldn't be hard to find out exactly what happened, and prevent it in the future.
that being said you have to pay attention and you have to be ready to intervene. it doesn't try to kill me often but it does happen occasionally
such a shame for those to die this way