Its just a matter of time before the injured and dead count starts increasing. Tesla needs to take their responsibility and disable AutoPilot before more people die because of its severe faults and flaws.
This is a deficiency in several automatic emergency braking systems [1]. Tesla extracts as much signal from front facing radar as they can, but edge cases abound.
Marketplace or Planet Money had a piece a few weeks ago that mentioned the hardest problem was consumer education about these safety systems. People are inherently not interested in learning about the life safety systems in their cars. They expect it to “just work”. Whose failing is it if warnings clearly state that isn’t the case?
“The car companies don't hide the fact that today's AEB systems have blind spots. It's all there in the owner's manuals, typically covered by both an all-encompassing legal disclaimer and explicit examples of why the systems might fail to intervene. For instance, the Camry's AEB system may not work when you're driving on a hill. It might not spot vehicles with high ground clearance or those with low rear ends. It may not work if a wiper blade blocks the camera. Toyota says the system could also fail if the vehicle is wobbling, whatever that means. It may not function when the sun shines directly on the vehicle ahead or into the camera mounted near the rearview mirror.”
Without doubt it is Tesla’s fault, due to how they have named and marketed the feature. Even if they fixed the name, they have created a system that encourages people to disengage when it is not safe to do so. You can’t make a system inherently unsafe and absolve responsibility by adding a warning.
I don’t think so. My understanding is that this is completely intended (and documented) behavior in Teslas and all other makes with this feature. Correct me if I’m wrong, but no adaptive cruise control systems perform full emergency breaking or emergency swerving when traveling at high speeds.
I drive a 2018 VW Golf Variant (Station Wagon) and the Adaptive Cruise Control from VW also has this problem. I regularly use it for city driving from traffic light to traffic light which means that I basically only have to steer a little, but if I'm driving towards a stationary car waiting in front of a red light, it will not detect the car until it starts to move.
> Can any other Traffic-Aware Cruise Control (TACC) stop in such a case? (Real question, I don't know).
I haven't found conclusive evidence, but the lack of news around other TACCs crashing into stationary objects is an indicator it might be a Autopilot specific issue.
> My current understanding is that Tesla's TACC is the best on the market. Correct?
Define best. It definitely has a lot of features few or no other systems have; on the other hand, it's a test program. It's officially Beta-Software, given to drivers to test and identify issues. So from a feature-set it's probably the best you can buy - but it turns you into a Test-Driver rather than a user of a product.
> I haven't found conclusive evidence, but the lack of news around other TACCs crashing into stationary objects is an indicator it might be a Autopilot specific issue.
IMO, I think this may actually not be true. Due to Tesla's popularity, and the amount of investors that have stated that they were shorting TSLA, I do think that media have a tendency to report more news about such accidents just because a Tesla was involved. If a Tesla crashes somewhere, the first question probably is "Was Autopilot enabled?", while with most other car crashes, it's just a car crash.
So I don't think that lack of news about other cars with TACC means that they don't suffer from similar issues. (TBH I wouldn't be able to name any other cars with such an advanced TACC either).
This has been true for a long time. Radar can't differentiate between a wall, a stopped car, or a dead skunk. Heck, it can get confused with a bridge's shadow on a bright day. Thus autopilot has to be configured to deal with these situations: stop or run into them. The vast majority of the situation you don't want the car to stop, as that will lead to a more dangerous situation and increase the chance of a collision.
It's one of the many situations that show that "autopilot", particularly Level 4 and Level 5, have a very, very long way to go. Also, as the reddit thread discusses the psychology of autopilot, this is why I don't want self-driving cars yet. Humans are horrendous at paying attention when they aren't directly in control. Airlines have been dealing with this problem for decades. Tesla has even started calling their services "driver assist" instead of autopilot to help combat this problem.
And yes, many people use autopilot without a problem every day, but you have to remember that it only takes one single miss to kill you (see: driver killed by ramming into a lane divider on autopilot).
> Radar can't differentiate between a wall, a stopped car, or a dead skunk
"Radar" absolutely can.
What can't differentiate between these things is the radar in Teslas that was intended for use in adaptive cruise control. There are any number of not insanely expensive radars that could do this job perfectly fine. Tesla just decided to go "full self driving" with this rather bad radar because "humans can do it with vision only!!!1!!!1!"
I don't the specifics of Tesla's radar implementation but what I have seen of the accidents usually seem like a textbook examples of incorrectly implemented clutter filtering. It's not an easy problem to solve in practice (re: cheaply), but known solutions have existed for decades.
The radar module in Teslas uses the same filtering strategy as most adaptive cruise control radar modules, which is "if relative velocity to object is nearly the opposite of the car's velocity, throw it out."
Right, sounds like they drop a bandpass filter on the down-converted Doppler spectrum. It's the simplest way to "solve" the clutter problem by just getting rid of everything stationary.
Automotive radar is clearly not up to the task of differentiating stopped cars from signs at a range far enough away to brake safely, it's true. In theory if you could fit a narrow beam phased array radar on a car it could do the job but that isn't something at all in the cards. But self driving cars with lidar have enough precision to deal with this problem even if they do cost more.
Rather than showing a problem with level 4 or level 5 automony I'd argue that this, and the psychology of driving, show that level 2 and level 3 are too dangerous since they require constant driver attention without constant driver engagement. We should just abandon the idea of having those autonomy level for cars and go straight to 4 and 5. That's what Waymo and other responsible self-driving development teams are doing and it looks like it'll work.
A better approach would be to focus on making more and more improvements to level 1 autonomy. The driver would be hands-on all the time, but the computer would intervene as necessary to prevent collisions. We already have some of that with forward collision prevention systems, blind spot warning, stability control, etc. Just keeping improving those systems by adding more sensors and authority over more vehicle controls.
Level 4+ autonomy is only going to be viable on a few limited roads for decades to come. The AI and sensor challenges are tremendous. We shouldn't let that long-term goal distract us from real safety improvements that could be realized relatively quickly with minimal technical risk.
Can someone with more computer vision knowledge explain why we can't teach a machine learning algorithm to detect stationary cars.
I have seen other Tesla demo videos where it clearly detects stationary cars. It seems ridiculous to say that it is a shortcoming of camera when our own two eyes do it.
It's not that it either always detects it or never detects it with video so much as that it doesn't detect it nearly as frequently as a human would. By contrast radar is pretty much always going to see things that aren't stationary.
But it would seem to me like that is just a matter of needing more training, or maybe more processing power.
At some point with enough training data it will detect a stopped car and react much more quickly than a human would. And even if it's not perfect, all it has to do is beat a human to be worthwhile.
It's not so easy, because you don't just have to detect stopped cars, you have to reject things which aren't stopped cars. False positive rejection appears to be something both Tesla and Uber are struggling with. (In the Arizona fatal accident, the Uber vehicle had Uber's own emergency braking system disabled because it was throwing up too many false positives.)
If there was a drug which reliably blocked memory consolidation, like a more polite and controlled benzo blackout, maybe humans could act as their own self driving cars
This not news. This has existed from the first iteration of Tesla Autopilots. That is why there where crashes with stationery vehicles for Tesla.
This is shortcoming for using only cameras or any vehicle that has adaptive cruise control.
The problem is that the front camera cannot distinguish from the road and a stationery vehicle. For that, they are the same and they ignore stationary obstacles. If they where taken into consideration, then they would not work at all.
This is where Lidar comes in but at its current price, or at price that Tesla was designing its system it was too expensive.
I am not entirely sure that pure cameras will ever be enough
This is something I don't understand. If you have more than 1 camera, you can run an edge detection algorithm on the images from each camera and use simple trig to build up a 3d model of what is in front of the car. This can easily distinguish between a shadow or road kill, vs a large object in the way. Is there a technical reason they don't do this (such as too much processing power required)?
You are trivialising the problem. Getting reliable data from stereo images is still far from solved, especially in realtime. (Also, the best currently available techniques are likely too computationally expensive to run on the compute that Tesla has put in the car).
There could be situations where this does not work accurately and that's when problems occur. If you watch closely you can even see it sometimes in this video.
Couldn't a secondary camera help more? Matching the images is not trivial and may take a while, but you could extract spatial information. Maybe a laser or a flash projector works better, but I imagine these systems to be susceptible to disturbances like reflections, especially when multiple cars are driving around.
Probably a long way still until we have real self-driving cars...
Not sure why you are downvoted. My Subaru does a pretty good job of detecting both moving and stationary objects, using only stereo cameras. It does have its limitations, and I wouldn't trust it to be a level 4 or 5 auto-auto without additional sensors. But it sure blows away Tesla when it comes to little things like detecting parked fire engines and highway barriers.
What you describe is specifically related to emergency braking. None of these systems claim to prevent collisions at high speeds through emergency braking. But they will significantly reduce the impact. No, I haven't tested that personally, and I don't plan to, so no actual "experience" there.
However, there are other aspects of EyeSight I do have experience with at highway speeds.
- Collision alert. It's an extra set of eyes to keep an eye out for trouble. When I'm checking over my shoulder before a lane change, I can take a little more time and look more carefully, knowing that it is watching the road ahead. It has alerted me for things like a car slowing suddenly or squeezing in front of me.
- Adaptive cruise. This helps by reducing load on me while driving and maintaining a safe following distance. So I can focus less on the car ahead and pay more attention all around.
Tip for users of EyeSight and other systems:
Always remember that like every other system out there it's not perfect and has limitations. Don't get complacent. After 5+ years of driving with it, I'll still keep my foot ready to brake, even in situations I'm confident it can handle. Occasionally I'll give in to the urge to brake only to find that it has started braking by the time my foot hits the pedal. But I'd rather react too soon than wait too long.
The issue is that tesla packaged up a radar designed for adaptive cruise control as part of their "autopilot." Maybe there is some disclaimer somewhere that says autopilot will happily run into stopped cars, but I don't know that most drivers of Teslas are aware of this.
A LIDAR array is a requirement for automated driving. It can operate in some conditions that radar cannot. lidar, radar and computer vision should all be used in concert for best effect and Tesla knows this, but chooses/chose not to in order to meet a price point. That choice may have been the right one, I don't know.
Hacker news (and me too, to be honest) focuses on Tesla, but there are a lot of other players doing this and doing it better IMO.
This is rather useful.. If I notice a Tesla behind me, I'll do my best not to come to an abrupt complete stop. Sounds like if I slow down gradually, it will give it enough data to slow down as well.
Perhaps someone who drives a car with a different radar could tell us if any other real world systems (actually installed on mass market vehicles right now) does better.
Tesla's system is not only not "autopilot" (it will happily plow into stationary vehicles), it's not even the best adaptive cruise control on the market for dealing with stationary vehicles: https://www.caranddriver.com/features/a24511826/safety-featu....
In a recent Car & Driver test, Subaru beat Tesla in every single scenario involving fast approach to a stationary/slow-moving target (although all collided with the target at closing speeds >= 50 mph).
Is this just because it's hard to distinguish using visual data when it's a 2-D image on the road (from the sun, intentional paint, a spill etc) and when its a 3-D object sitting upright on the road?
54 comments
[ 3.7 ms ] story [ 32.6 ms ] threadMy current understanding is that Tesla's TACC is the best on the market. Correct?
Marketplace or Planet Money had a piece a few weeks ago that mentioned the hardest problem was consumer education about these safety systems. People are inherently not interested in learning about the life safety systems in their cars. They expect it to “just work”. Whose failing is it if warnings clearly state that isn’t the case?
“The car companies don't hide the fact that today's AEB systems have blind spots. It's all there in the owner's manuals, typically covered by both an all-encompassing legal disclaimer and explicit examples of why the systems might fail to intervene. For instance, the Camry's AEB system may not work when you're driving on a hill. It might not spot vehicles with high ground clearance or those with low rear ends. It may not work if a wiper blade blocks the camera. Toyota says the system could also fail if the vehicle is wobbling, whatever that means. It may not function when the sun shines directly on the vehicle ahead or into the camera mounted near the rearview mirror.”
[1] https://www.caranddriver.com/features/a24511826/safety-featu...
Tesla should be responsible and stop marketing it.
Otherwise you're begging legislators to write ham-handed laws, begging courts to set awful precedents.
Either way, they are absolutely poisoning the well.
Here’s an an article from last year:
https://www.wired.com/story/tesla-autopilot-why-crash-radar/
https://www.caranddriver.com/news/a15342750/2018-volvo-xc60-...
I haven't found conclusive evidence, but the lack of news around other TACCs crashing into stationary objects is an indicator it might be a Autopilot specific issue.
> My current understanding is that Tesla's TACC is the best on the market. Correct?
Define best. It definitely has a lot of features few or no other systems have; on the other hand, it's a test program. It's officially Beta-Software, given to drivers to test and identify issues. So from a feature-set it's probably the best you can buy - but it turns you into a Test-Driver rather than a user of a product.
IMO, I think this may actually not be true. Due to Tesla's popularity, and the amount of investors that have stated that they were shorting TSLA, I do think that media have a tendency to report more news about such accidents just because a Tesla was involved. If a Tesla crashes somewhere, the first question probably is "Was Autopilot enabled?", while with most other car crashes, it's just a car crash.
So I don't think that lack of news about other cars with TACC means that they don't suffer from similar issues. (TBH I wouldn't be able to name any other cars with such an advanced TACC either).
It's one of the many situations that show that "autopilot", particularly Level 4 and Level 5, have a very, very long way to go. Also, as the reddit thread discusses the psychology of autopilot, this is why I don't want self-driving cars yet. Humans are horrendous at paying attention when they aren't directly in control. Airlines have been dealing with this problem for decades. Tesla has even started calling their services "driver assist" instead of autopilot to help combat this problem.
And yes, many people use autopilot without a problem every day, but you have to remember that it only takes one single miss to kill you (see: driver killed by ramming into a lane divider on autopilot).
"Radar" absolutely can.
What can't differentiate between these things is the radar in Teslas that was intended for use in adaptive cruise control. There are any number of not insanely expensive radars that could do this job perfectly fine. Tesla just decided to go "full self driving" with this rather bad radar because "humans can do it with vision only!!!1!!!1!"
It works pretty much as designed.
Rather than showing a problem with level 4 or level 5 automony I'd argue that this, and the psychology of driving, show that level 2 and level 3 are too dangerous since they require constant driver attention without constant driver engagement. We should just abandon the idea of having those autonomy level for cars and go straight to 4 and 5. That's what Waymo and other responsible self-driving development teams are doing and it looks like it'll work.
Level 4+ autonomy is only going to be viable on a few limited roads for decades to come. The AI and sensor challenges are tremendous. We shouldn't let that long-term goal distract us from real safety improvements that could be realized relatively quickly with minimal technical risk.
I have seen other Tesla demo videos where it clearly detects stationary cars. It seems ridiculous to say that it is a shortcoming of camera when our own two eyes do it.
At some point with enough training data it will detect a stopped car and react much more quickly than a human would. And even if it's not perfect, all it has to do is beat a human to be worthwhile.
This is where Lidar comes in but at its current price, or at price that Tesla was designing its system it was too expensive.
I am not entirely sure that pure cameras will ever be enough
There could be situations where this does not work accurately and that's when problems occur. If you watch closely you can even see it sometimes in this video.
Probably a long way still until we have real self-driving cars...
https://www.iihs.org/iihs/sr/statusreport/article/53/3/2
Have you had any experience with eyesight helping at highway speeds?
What you describe is specifically related to emergency braking. None of these systems claim to prevent collisions at high speeds through emergency braking. But they will significantly reduce the impact. No, I haven't tested that personally, and I don't plan to, so no actual "experience" there.
However, there are other aspects of EyeSight I do have experience with at highway speeds.
- Collision alert. It's an extra set of eyes to keep an eye out for trouble. When I'm checking over my shoulder before a lane change, I can take a little more time and look more carefully, knowing that it is watching the road ahead. It has alerted me for things like a car slowing suddenly or squeezing in front of me.
- Adaptive cruise. This helps by reducing load on me while driving and maintaining a safe following distance. So I can focus less on the car ahead and pay more attention all around.
Tip for users of EyeSight and other systems: Always remember that like every other system out there it's not perfect and has limitations. Don't get complacent. After 5+ years of driving with it, I'll still keep my foot ready to brake, even in situations I'm confident it can handle. Occasionally I'll give in to the urge to brake only to find that it has started braking by the time my foot hits the pedal. But I'd rather react too soon than wait too long.
("dumb" as in limited in scope and not trying to drive the car as a whole.)
Every radar cruise control system comes with a warning that it can't detect stopped traffic. This seems like a case of RTFM.
https://www.caranddriver.com/features/a24511826/safety-featu...
Hacker news (and me too, to be honest) focuses on Tesla, but there are a lot of other players doing this and doing it better IMO.
Perhaps someone who drives a car with a different radar could tell us if any other real world systems (actually installed on mass market vehicles right now) does better.
In a recent Car & Driver test, Subaru beat Tesla in every single scenario involving fast approach to a stationary/slow-moving target (although all collided with the target at closing speeds >= 50 mph).