"This is the first known fatality in just over 130 million miles where Autopilot was activated," states a post on Tesla's corporate website. "Among all vehicles in the U.S., there is a fatality every 94 million miles. Worldwide, there is a fatality approximately every 60 million miles. It is important to emphasize that the NHTSA action is simply a preliminary evaluation to determine whether the system worked according to expectations."
From what I understand, the autopilot can only be used on highways. I would suspect that highway travel is much safer per mile in the first place. Are we comparing highway driving with autopilot to regular mixed city/highway driving?
Also, those averages are over the entire car population, with a median age of around ten years and an average crash safety rating relatively far below that of a Model S.
Highways are the most dangerous environment. High speed combined with cross traffic. City streets don't have enough speed usually for crashes be lethal.
It's common to use the term 'freeway' or 'Interstate' (meaning Interstate Highway/Freeway) to refer to controlled access highways in the US.
There's lots of highways in the US that are just 2 lanes, one in each direction, with a speed limit of 55mph and cross traffic from smaller roads that should stop first before turning on the highway in either direction. These are what we usually mean with the word 'highway'.
I wonder what the driver was doing when the vehicle crashed. Tesla says that the driver is ultimately responsible for controlling the vehicle but then provides a system that is likely result in the driver not paying attention--sleeping, reading, or otherwise being distracted. How can Tesla and the governments that license drivers resolve this situation?
Both Mercedes and Volvo have publicly stated they are taking full responsibility for any accidents which occur during use of their semi-autonomous systems. Tesla needs to step up, or it will hurt their public image.
Have you seen a 2016 Mercedes E-class? It has autonomous systems on essentially the same level as a Tesla, just without the flashy "Autopilot" branding. The new S-class coming out next year is going to have a lot more autonomous features on top of that, including Lidar (which Tesla lacks).
> Turns out, the 2017 E-Class is just playing coy when it comes to self-driving capabilities. It is the first production car to receive an autonomous driving license in Nevada.
So the technical system (hardware+software) is as good as or better than a Model S, but Mercedes believe it is irresponsible to call that Autopilot and pretend like the car is fully autonomous, so they've put in restrictions like requiring some tiny driver input once every 60 seconds so it knows you're paying attention.
I think you mean the 2017 E-class, they just started appearing at dealers in CA, and I'm excited to go test-drive one and see how much better it is than my 2014 E-class.
That said, it's my understanding that DISTRONIC still disengages if you take your hands off the wheel for more than a short amount of time, same as the 2014 model, and I haven't seen Mercedes make any announcement that the 2017 model will be so good that they're gonna assume liability for accidents.
All bark, no bite. It's great that there's competition in the autonomous tech, but I'm very skeptical of either Mercedes or Volvo standing behind that promise any time soon now.
While I agree there is a huge danger with semi-autonomous systems as drivers become disengaged, Volvo has already publicly announced they will take full legal responsibility for any crashes when their cars are in autonomous mode.
...of which they have 0 on the market so far. Let's see how well that announcement holds up in the face of reality when they actually put cars on the market.
I'm not so sure that the driver being 100% responsible is ridiculous with this particular iteration of technology. I have not experienced the Telsa system. However, IF the following from the blog post is true:
>...Additionally, every time that Autopilot is engaged, the car reminds the driver to “Always keep your hands on the wheel. Be prepared to take over at any time.” The system also makes frequent checks to ensure that the driver's hands remain on the wheel and provides visual and audible alerts if hands-on is not detected. It then gradually slows down the car until hands-on is detected again.
THEN, this seems a design that both envisions and forces a driver to keep their hands on the wheel and stay focused. This does not seem to be a design that directs, encourages, or allows a driver to engage the system then take a nap.
At some point in the future, with fully autonomous driving, yes, there must be some legal framework wherein the car manufacturer is responsible. This piece of technology does not seem to fall within that framework.
Probably looking down at phone or the like. I don't buy that any sky would prevent an attentive driver from seeing a big rig perpendicular to them.
I've had doubts about autopilot after it was hastily released and I've seen some unnerving anecdotes on Tesla groups about it. I think Tesla needs to keep Elon in check a bit more...
"the highway perpendicular" it says. Neither noticed it "against a brightly lit sky".
I take that as the truck was crossing the street (from a side street) in front of the car, but because of the sun blinding the sensors and the driver, the car didn't stop.
Relevant section:
In a separate crash on May 7 at 3:40 p.m. on U.S. 27 near the BP Station west of Williston, a 45-year-old Ohio man was killed when he drove under the trailer of an 18-wheel semi.
The top of Joshua Brown’s 2015 Tesla Model S vehicle was torn off by the force of the collision. The truck driver, Frank Baressi, 62, Tampa was not injured in the crash.
The FHP said the tractor-trailer was traveling west on US 27A in the left turn lane toward 140th Court. Brown’s car was headed east in the outside lane of U.S. 27A.
When the truck made a left turn onto NE 140th Court in front of the car, the car’s roof struck the underside of the trailer as it passed under the trailer. The car continued to travel east on U.S. 27A until it left the roadway on the south shoulder and struck a fence. The car smashed through two fences and struck a power pole. The car rotated counter-clockwise while sliding to its final resting place about 100 feet south of the highway. Brown died at the scene.
Charges are pending.
Oh that was much better, the Verge one was sparse and the app I was using basically passed it through Readability so I had a hard time visualizing the accident. Thank you!
My sympathies to the people who built the systems that make Autopilot possible. Even knowing it was statistically bound to happen eventually, this has to weigh heavily from the "damn, I should've thought of this once-in-a-130-m-miles corner case" perspective.
Unlike many other fatal crashes, however, they presumably now have data from the corner case they can use to regression test every future release of the system.
> What we know is that the vehicle was on a divided highway with Autopilot engaged when a tractor trailer drove across the highway perpendicular to the Model S. Neither Autopilot nor the driver noticed the white side of the tractor trailer against a brightly lit sky, so the brake was not applied.
Failing to detect the white side of a tractor against a brightly lit sky is something I can see camera/image based sensors struggling with, but not LIDAR based sensors.
This is going to be a major morality issue with self driving cars.
What if a deer crosses the road? Will the car hit the deer to save the life of the driver? Avoid the deer to avoid an accident? Or save the deer and run into the ditch, possibly harming the driver?
Unfortunately, this technology is not capable of making those decisions and if ultimately the driver is responsible, then maybe we should just keep it that way then.
"Or save the deer and run into the ditch, possibly harming the driver?"
This is not even a slight morality issue. The person is more important than the deer.
"Unfortunately, this technology is not capable of making those decisions"
Of course it is. Program the machine so that its primary concern is protection of human life. Which should be its secondary concern (protection of property or protection of deer) is a more interesting question. But running into the ditch to save the deer while threatening harm to the driver is simply not an option.
There was some discussion, but Tesla has a forward facing radar, but it is mounted lower than the camera and trailers of certain height have been shown to avoid it's detection window.
So, speculatively, it is a double failure:
1) Trailer was high enough to avoid radar
2) Trailer was light enough/low contrast to avoid detection by the camera
Upside of working on "boring CRUD apps that won't change the world": your edge-cases won't result in fatalities.
I hope Tesla learns something from this, from the outside looking in - they appear to have a cavalier, can-do attitude (nothing wrong with the latter) towards safety engineering. Release beta self-driving software to production vehicles? Yeah, why not?
It is possible he was expecting the Model S to auto-brake as it normally does, and possibly realized when it was too late and panicked/tried to swerve. But I'm just speculating, I'm waiting to hear the outcome of the NHTSA investigation.
I'd have thought the auto-brake would be expected to react faster than a human driver in that situation. So if you saw such an event and the car wasn't already braking you'd hit the brakes, not sit around waiting to find out if it handled it or not.
I think the problem might have been that the middle of the trailer is not at ground level but at some hight where the lidar data is ignored or not there.
> In the blog post, Tesla reiterates that customers are required to agree that the system is in a "public beta phase" before they can use it
I absolutely disagree with this, and it should not be used as a 'get out' clause from Tesla. If you work with non-technical people on technical issues on a day to day basis you'll understand why - non-technical people literally don't understand what stuff like this means. They'll read it, then say 'Oh but they installed it in my car anyway so it must be safe', and use it anyway.
Beware clickbait and intentional inflammatory posturing. I noticed The Verge selectively quoted Tesla without directly linking to the source (although it was part of Elon's tweet):
"It is important to note that Tesla disables Autopilot by default and requires explicit acknowledgement that the system is new technology and still in a public beta phase before it can be enabled. When drivers activate Autopilot, the acknowledgment box explains, among other things, that Autopilot “is an assist feature that requires you to keep your hands on the steering wheel at all times," and that "you need to maintain control and responsibility for your vehicle” while using it. Additionally, every time that Autopilot is engaged, the car reminds the driver to “Always keep your hands on the wheel. Be prepared to take over at any time.” The system also makes frequent checks to ensure that the driver's hands remain on the wheel and provides visual and audible alerts if hands-on is not detected. It then gradually slows down the car until hands-on is detected again."
So the key here is that Tesla autopilot is a driver assist but is positioned such that the driver needs to remain alert and able to assume control of the vehicle at any moment. So when it is said, "Neither Autopilot nor the driver noticed..." it is critical to note that ultimately the driver failed to control their vehicle. Personally, I find this reasonable.
> The system also makes frequent checks to ensure that the driver's hands remain on the wheel and provides visual and audible alerts if hands-on is not detected. It then gradually slows down the car until hands-on is detected again
I don't care much about the user agreement, but this should mean that the driver had their hands on the wheel when this happened. Does we know if it requires both hands on the wheel, or can you placate the car with one hand while you read reddit with the other?
Seeing as I have seen videos of drivers sleeping in traffic with autopilot engaged, I imagine there may be a way to evade/circumvent this check entirely.
So your question is sound, and I don't know the answer...but saying "A distracted driver ran into a truck trailer" is less catchy.
Imagine Carmaker X has a secret switch on their cars that a driver may pull to completely disable their breaks. The switch has no utility, eg it's not there for use by service technicians. It's only there so you can see how far you can go while driving without breaks.
Would it make any difference if the switch came with a big angry red sticker saying "DON'T PULL THIS SWITCH UNLESS YOU KNOW WHAT YOU'RE DOING"?
It's a setting that can get you killed. There's no other reason to have it other than to risk getting killed (presumably because some people enjoy this kind of thing). Warning or no warning makes no difference at all. There's no justification for having a thing like that on anyone's car.
They can put anything they want in the EULA it doesn't means it would stand up scrutiny in the court of law.
Autopilot features are in a grey area now since they are very new but Tesla is still responsible regardless of what they claim to be.
Seatbelts and Airbags were also in "beta phase" at some point, and more likely than not some people got killed by a deploying airbag in it's early days or by a seatbelt with a shoddy release function before everything was standardized but you could still sue for compensation in those incidents as well.
Seatbelts and airbags are safety features designed to protect you during an accident, as in given an accident has already happened.
"Autopilot" is a system that takes control of the car away from the driver. As such there's no comparison to safety features and in fact it's rather the opposite: it's a system that makes driving a lot more dangerous than it normally is.
(Unless the system is much better at driving than the human driver, which it most definitely is not).
An airbag accidentally deploying while driving could lead to loss of control of the vehicle, there have been such cases. It is not given that an accident has already happened.
I can see why Volvo criticized Tesla for rolling out a system that's not fully autonomous. If drivers are willfully distracted because they think the car will handle any situation, that's a very dangerous place to be in. Though it's not clear what the driver was doing in this situation, so it's possible that this would have been a fatal crash in any other car without autopilot as well.
A company's "public beta phase" system named "Autopilot" just drove a customer head-first into a trailer and killed them, at least in part due to inadequate sensor hardware that seems unlikely a software update can remedy. Tesla really ought to take a step back and consider the damage they may do to their brand if they start killing customers and blaming the dead person for having trusted a Tesla product.
70 comments
[ 1.3 ms ] story [ 166 ms ] threadThere's lots of highways in the US that are just 2 lanes, one in each direction, with a speed limit of 55mph and cross traffic from smaller roads that should stop first before turning on the highway in either direction. These are what we usually mean with the word 'highway'.
If not, comparing auto pilot deaths with average deaths per mile is disingenuous.
I also wonder if that would encourage more risky behaviour from users or less.
Actually, it appears to be nowhere near the level of a Tesla.
http://mashable.com/2016/06/16/2017-mercedes-benz-e-class-fi...
> Turns out, the 2017 E-Class is just playing coy when it comes to self-driving capabilities. It is the first production car to receive an autonomous driving license in Nevada.
So the technical system (hardware+software) is as good as or better than a Model S, but Mercedes believe it is irresponsible to call that Autopilot and pretend like the car is fully autonomous, so they've put in restrictions like requiring some tiny driver input once every 60 seconds so it knows you're paying attention.
That said, it's my understanding that DISTRONIC still disengages if you take your hands off the wheel for more than a short amount of time, same as the 2014 model, and I haven't seen Mercedes make any announcement that the 2017 model will be so good that they're gonna assume liability for accidents.
All bark, no bite. It's great that there's competition in the autonomous tech, but I'm very skeptical of either Mercedes or Volvo standing behind that promise any time soon now.
It's a vaporware promise.
>...Additionally, every time that Autopilot is engaged, the car reminds the driver to “Always keep your hands on the wheel. Be prepared to take over at any time.” The system also makes frequent checks to ensure that the driver's hands remain on the wheel and provides visual and audible alerts if hands-on is not detected. It then gradually slows down the car until hands-on is detected again.
THEN, this seems a design that both envisions and forces a driver to keep their hands on the wheel and stay focused. This does not seem to be a design that directs, encourages, or allows a driver to engage the system then take a nap.
At some point in the future, with fully autonomous driving, yes, there must be some legal framework wherein the car manufacturer is responsible. This piece of technology does not seem to fall within that framework.
I've had doubts about autopilot after it was hastily released and I've seen some unnerving anecdotes on Tesla groups about it. I think Tesla needs to keep Elon in check a bit more...
> [...] a tractor trailer drove across the highway perpendicular to the Model S.
https://www.teslamotors.com/blog/tragic-loss
I take that as the truck was crossing the street (from a side street) in front of the car, but because of the sun blinding the sensors and the driver, the car didn't stop.
Relevant section: In a separate crash on May 7 at 3:40 p.m. on U.S. 27 near the BP Station west of Williston, a 45-year-old Ohio man was killed when he drove under the trailer of an 18-wheel semi. The top of Joshua Brown’s 2015 Tesla Model S vehicle was torn off by the force of the collision. The truck driver, Frank Baressi, 62, Tampa was not injured in the crash. The FHP said the tractor-trailer was traveling west on US 27A in the left turn lane toward 140th Court. Brown’s car was headed east in the outside lane of U.S. 27A. When the truck made a left turn onto NE 140th Court in front of the car, the car’s roof struck the underside of the trailer as it passed under the trailer. The car continued to travel east on U.S. 27A until it left the roadway on the south shoulder and struck a fence. The car smashed through two fences and struck a power pole. The car rotated counter-clockwise while sliding to its final resting place about 100 feet south of the highway. Brown died at the scene. Charges are pending.
Here is a google maps view of the accident location: https://goo.gl/maps/SSKyoxhoaxp
http://imgur.com/gallery/aWDK0y9/new
> What we know is that the vehicle was on a divided highway with Autopilot engaged when a tractor trailer drove across the highway perpendicular to the Model S. Neither Autopilot nor the driver noticed the white side of the tractor trailer against a brightly lit sky, so the brake was not applied.
Failing to detect the white side of a tractor against a brightly lit sky is something I can see camera/image based sensors struggling with, but not LIDAR based sensors.
[1] https://www.teslamotors.com/blog/tragic-loss
What if a deer crosses the road? Will the car hit the deer to save the life of the driver? Avoid the deer to avoid an accident? Or save the deer and run into the ditch, possibly harming the driver?
Unfortunately, this technology is not capable of making those decisions and if ultimately the driver is responsible, then maybe we should just keep it that way then.
This is not even a slight morality issue. The person is more important than the deer.
"Unfortunately, this technology is not capable of making those decisions"
Of course it is. Program the machine so that its primary concern is protection of human life. Which should be its secondary concern (protection of property or protection of deer) is a more interesting question. But running into the ditch to save the deer while threatening harm to the driver is simply not an option.
So, speculatively, it is a double failure:
1) Trailer was high enough to avoid radar
2) Trailer was light enough/low contrast to avoid detection by the camera
I hope Tesla learns something from this, from the outside looking in - they appear to have a cavalier, can-do attitude (nothing wrong with the latter) towards safety engineering. Release beta self-driving software to production vehicles? Yeah, why not?
I absolutely disagree with this, and it should not be used as a 'get out' clause from Tesla. If you work with non-technical people on technical issues on a day to day basis you'll understand why - non-technical people literally don't understand what stuff like this means. They'll read it, then say 'Oh but they installed it in my car anyway so it must be safe', and use it anyway.
https://www.teslamotors.com/blog/tragic-loss
The full context of the agreement:
"It is important to note that Tesla disables Autopilot by default and requires explicit acknowledgement that the system is new technology and still in a public beta phase before it can be enabled. When drivers activate Autopilot, the acknowledgment box explains, among other things, that Autopilot “is an assist feature that requires you to keep your hands on the steering wheel at all times," and that "you need to maintain control and responsibility for your vehicle” while using it. Additionally, every time that Autopilot is engaged, the car reminds the driver to “Always keep your hands on the wheel. Be prepared to take over at any time.” The system also makes frequent checks to ensure that the driver's hands remain on the wheel and provides visual and audible alerts if hands-on is not detected. It then gradually slows down the car until hands-on is detected again."
So the key here is that Tesla autopilot is a driver assist but is positioned such that the driver needs to remain alert and able to assume control of the vehicle at any moment. So when it is said, "Neither Autopilot nor the driver noticed..." it is critical to note that ultimately the driver failed to control their vehicle. Personally, I find this reasonable.
I don't care much about the user agreement, but this should mean that the driver had their hands on the wheel when this happened. Does we know if it requires both hands on the wheel, or can you placate the car with one hand while you read reddit with the other?
So your question is sound, and I don't know the answer...but saying "A distracted driver ran into a truck trailer" is less catchy.
Would it make any difference if the switch came with a big angry red sticker saying "DON'T PULL THIS SWITCH UNLESS YOU KNOW WHAT YOU'RE DOING"?
It's a setting that can get you killed. There's no other reason to have it other than to risk getting killed (presumably because some people enjoy this kind of thing). Warning or no warning makes no difference at all. There's no justification for having a thing like that on anyone's car.
Autopilot features are in a grey area now since they are very new but Tesla is still responsible regardless of what they claim to be.
Seatbelts and Airbags were also in "beta phase" at some point, and more likely than not some people got killed by a deploying airbag in it's early days or by a seatbelt with a shoddy release function before everything was standardized but you could still sue for compensation in those incidents as well.
"Autopilot" is a system that takes control of the car away from the driver. As such there's no comparison to safety features and in fact it's rather the opposite: it's a system that makes driving a lot more dangerous than it normally is.
(Unless the system is much better at driving than the human driver, which it most definitely is not).
I'm happy to let you all beta test this one for me.