I wonder if it is imprudent to create too many vehicles that have partial self-driving capability before we have ones that can do 100% of it. Instead of to the point where you can take your attention away from the road, but not to the point where it can reliably avoid accidents on your behalf. Too many people aren't wise enough to use the partial autonomous technology without abusing it.
There are warning placards in several places in your car saying such.
Sometimes an airbag can cause more injury than the accident.
A seatbelt may trap you in a sinking car.
But overall, airbags and seatbelts have significantly reduced traffic injuries and fatalities.
Autopilot saves lives. Autopilot used incorrectly can kill people. But overall, Autopilot has significantly reduced the number of accidents involving Teslas.
So do we take airbags away because using them incorrectly might hurt a child? Are they a safety feature or marketing value?
This is a common misinterpretation. The data only shows that TACC + Autosteer reduced crashes by 40% vs TACC alone. There is no comparison to complete driver control.
Tesla’s data shows autopilot (TACC + Autosteer) to be 50% safer than human driver control. While I have yet to see an independent study verifying that, the evidence is compelling. Humans are terrible drivers.
> "The probability of having an accident is 50 per cent lower if you have Autopilot on," said Musk, speaking at an energy conference in Oslo, Norway. "Even with our first version, it's almost twice as good as a person."
> The results are from Tesla's first generation of Autopilot, which owners have already used to drive over 47 million miles since it launched in October 2015.
Okay, so the problem there is that it isn't an apples to apples comparison. The 'miles driven while using autopilot' basically only contain very nice driving conditions, either slow moving congestion, wide open roads, no miles with snow surely, no lane changing, etc. But the 'miles driven without autopilot' contain all those things, quick lane changes and snow and rain and city driving and parking lot driving, basically all the scenarios where accidents are very likely. It's pure selection bias, it says nothing about the state of Autopilot and Musk likes it that way.
I think Tesla has enough telemetry and miles driven to come up with some idea of whether or not miles traveled with Autopilot on have more or fewer deaths than cars in general.
I don't know what the answer is, but there's certainly evidence.
Maybe they could take some ideas from the rail industry, which have been dealing with this issue for quite some time now.
Engineers are required to to make some kind of input to verify that they're still alert, or the train stops[1] if they haven't done anything in X seconds.
Obviously you wouldn't want the car to just stop in the middle of the road, but there could be something like that designed.
Tesla does have a sensor in the wheel that "feels" for resistance on the wheel. In my experience, it's more than sensitive enough to be able to detect a hand being on the wheel, but is easily defeated by idiots.
I'd imagine Tesla wasn't thinking about how the driver alertness feature could be countered unfortunately. Perhaps we'll see Tesla adapting the eye tracking tech that Cadillac Hypercruise is using.
Never knew about the Cadillac. Very cool, thanks for calling it out.
I wonder how to truly tell if a driver is attentive? The Cadillac indicates it uses head tracking technology. I wonder if that is sufficient? Seems you probably need even eye-tracking: an attentive driver keeps looking around or something.
All smart-assist cars are required to sense if the drivers are actually maintaining their hands of the wheel. I don't know if they are to disengage or merely to annoy the user, but Tesla is notorious for skirting these regulations.
People have been safely and intelligently using cruise control for... decades? It is a partial automation solution that reduces half of what you have to manage when controlling the car (how fast you're going).
I suspect the primary problem is not the partial autonomous technology, but the marketing which is convincing people they can trust it. Nobody ever told you your cruise control means you don't have to pay attention, so very few[1] incidents happen due to misuse of it.
First and foremost, Tesla should never have been allowed to release "Autopilot" being called that. From the very name it provides a false impression of what it can do.
[1]I know you can find one. Humans are remarkably stupid.
It seems like an apt name to me; it pilots your car automatically. The only detail omitted is how often it pilots you into plainly visible large objects at full speed.
It's true that people may be in particular need of a refresher, since such features could be perceived as more than advanced cruise control. It should be strongly emphasized to all drivers that we need to take full control whenever we see any exceptional conditions, from hazards, obstructions, pedestrians or wildlife in or entering the road, to poor visibility, and even when passing or being passed by small vehicles.
So, you're saying that Tesla is wrong, without even bothering to collect any data about whether or not Tesla owners are confused about the capabilities of the system?
As long as there is a human variable I severely doubt we will every get to a system that is 100%. The raw truth of it is to make a system that can do it close to 100% you need data. Tesla has chosen to put a semi-autonomous system into the hands of consumers in order to generate the massive amounts of data necessary to achieve full automation. Other companies have chosen to log a crazy amount of hours using technicians to generate the same set of data, both have and will continue to experienced crashes as they work toward making the system better.
Tesla I think has done a fantastic job at striking a balance between getting something into the market the generate real-world data and mitigating the risks that come with it. If this car really did crash into the fire truck at around 65 MPH then it did a pretty good job at keeping everyone safe.
I think I would be hard pressed to find any technological or mechanical innovation that did not initially involve risk or have an accident associated with it. I've seen more examples where a Autopilot has done the right thing and avoided an accident than I have read stories of it making a mistake and causing an accident.
Correct, "safe" was a poor choice of word. Alive and relatively unscathed according to the article is what I was going for. Thanks for the clarification.
Data is insufficient. The most famous polling miss in history relied on 2.4 million responses (~1.8% of the population) to conclude in 1936 that the Republicans would win the presidential election. What matters is the quality of your data, and if it is sufficient to draw reasonable conclusions from.
I don't see any evidence that Tesla has more, or less for that matter, of the necessary high-quality data that is needed for self-driving cars. The fact that real people are driving those cars continuously, potentially generating reams of data, is pretty immaterial, since the majority of that data is going to be useless (let's face it, 10 million hours of driving in the same conditions isn't better than 1 million hours of driving).
Yeah, the quote about the NTSB on a similar accident is telling: "the National Transportation Safety Board ruled that Autopilot contributed to the crash by allowing lengthy disengagement from the driving process."
In my view, driving is already boring enough that it's insufficiently attention-grabbing for safety's sake. That's why, e.g., the "naked streets" model is removing lane marking. Decreasing (false) certainty forces more cognitive engagement, making things safer: https://www.theguardian.com/commentisfree/2016/feb/04/remova...
If anything, for the decades [1] while we're waiting for full automation, I'd like to see the equivalent of Threat Image Projection. It's the technology for luggage scanning where they put images of fake guns, bombs, etc in the images of real luggage. If the operator doesn't press the "threat" button they are alerted that they slipped up. It brings the frequency of important events up into the range where humans can pay attention. It'd be great to have the same thing for cars, so that AR images of deer, cyclists, children, and fire trucks appear; if you don't attempt to take evasive action, you lose points. [2]
Would it not be better engineering to have the roads driving the cars, as opposed to having the cars drive the roads?
If the roads had sensors in the "bots dotts" and would track and steer all the volume of traffic - isnt that better than all the infantile-AI-cars doing it?
Do you really think there will ever be a 100% driverless car? I find it highly unlikely. And the paradox is, the better self-driving cars get, the more trouble driver is in when they are forced to take over. Drivers will be lacking basic driving skill, and essentially will get stuck on snowy/gravel/flooded roads.
Will this turn out to be another "brightly lit sky" autopilot bug, like the 2016 semi truck incident?
https://www.tesla.com/blog/tragic-loss "Neither Autopilot nor the driver noticed the white side of the tractor trailer against a brightly lit sky, so the brake was not applied"
Tesla, to save money, is trying to build autopilot from regular video cameras in visible spectrum. EDIT: Some models still contain other sensors, but Musk has been trying to go with purely optical for a long time.
It's pretty granular, almost like a flight recorder. They've defended themselves against similar claims before by publishing details from the car's logs.
Yeah, though some of their defenses have been a little ... on the nose.
"Autopilot was not active at the time of collision, according to logs"... because it was deactivated by hard manual braking a moment before, because Autopilot was about to cause a hard collision...
> The California Highway Patrol and Culver City Fire Department confirmed the southbound Tesla had struck the fire truck, but could not immediately confirm whether the vehicle had been on Autopilot.
It sounds like it's not 100% confirmed that Autopilot was at fault. The first sentence of the article also reads "reportedly on 'Autopilot'"
We'll know soon enough based on stored vehicle telemetry. Experience suggests that the right to privacy is the first thing to go in the event of a newsworthy crash in a Tesla.
The more I think about it, the more I think that there should be an outward signal, such as a special light, that a car is in self-driving mode. We already require signals when changing lanes, reversing, and for electric cars to be audible at low speeds (starting in 2019). Those nearby shouldn’t have to guess if a car is under computer control.
Aggressively cut in whenever they want with impunity and zero consequences? Self driving cars will always avoid obstacles where possible and never suffer road rage.
The only thing that keeps a check on the impulses of the most aggressive human drivers is the unpredictability of other human drivers.
Ha ha, except a Model S on AutoPilot isn't really being driven by the computer, it's just automating a greater portion of the tasks performed by a human driver.
The equally interesting question is what you think computer drivers should do differently when driving near a car that is under computer control.
For example, if your computer control car knows that the car behind is also computer controlled, it could be more aggressive on the brakes to avoid a forward collision without causing a rear collision.
Tomorrow, the situation might be that a computer controlled car could see that the car behind is tailgating and the car in the next lane is computer controlled — then determine the safest action would be to swerve into the next lane rather than brake suddenly...
There's no way to avoid the trolley problem as long as you're developing self-driving cars. You can wilfully ignore the trolley problem and let the algorithm do whatever it does, but that doesn't make it go away.
And I would dispute that emergency braking suddenly while being tailgated is less "crazy shit" than swerving into the next lane where your presence will, with an exceptional degree of confidence, be immediately detected and compensated for.
To clarify, the scenario in my mind is one that would be bold but not dangerous; something a good human driver might do with 360° situational awareness. In this instance it could be the difference between swerving to miss an obstacle or a multiple-car collision — even if your self driving car can stop in time, the tailgater is likely to slam into you, which in turn slams you into the obstacle.
Or to change the situation a bit, if the "tailgating" car happened to be a self driving car with an aggressive profile, that could swing the decision back towards braking. Point is, knowing that a car is being driven by a computer can aid decisions made by other cars.
Self-driving mine trucks have indicator lights for just this reason. Green indicates the truck is under the control of a human driver, flashing orange indicates the truck is under automatic control, and flashing red indicates the truck has a fault and will not move.
I have been thinking about this more and more on my commute every day. Commuting up and down US101, I see so many Tesla's driving on the left lane, setting their autopilot to 65mph, just blocking traffic. There needs to be a visible indicator on the outside.
By the way, one can speculate there are interesting ways to trigger beeps inside these Teslas coasting on the left lane.
Just driving on Autopilot at the speed of traffic but at a safe stopping distance is enough to prompt a minority of assholes to aggressively tailgate and then pass and insert themselves into that empty space in front. Happens most days.
The reason is often because what you think is the speed of the traffic is actually the speed of the asshole in front of you that's going much slower than he could on a congested route.
It doesn't take a single car to reduce the speed of a lane. It takes exactly two. One to slow it down, and the other one to stop the others from letting the guy in the front know that he is going too slow.
I’m aware what speed of traffic means. The actual reason for the phenomenon I described is that there are a lot of hyper-aggressive low-skilled drivers out there. Note that “skill” in this context means driving in a way that optimizes for keeping yourself and your fellow commuters alive so we can all go home to our families; not an ability to weave through traffic and teach slow drivers a thing or two about blocking lanes.
You know I take issue with Tesla on these incidents. They're trying to have it both ways. They market it as 'autopilot' to appeal to the dream of complete self-driving but if an accident actually happens it can never be the fault of autopilot. After all it's required for the driver to be paying attention so if they didn't override autopilot it's obvious they weren't paying attention. Really I'd like to see the government to step in and force Tesla to market their product more honestly.
Okay, again I'm not disagreeing with you, but those questions are about as leading as you can get. Ask a portion of the population "Did you know that meiosis is the process through which cells divide?" and you'll get a lot of yes's.
I do not own one, but I am on the Tesla Reddit pretty regularly. It gets a regular stream of posts from people about how often it tries to deviate them into turn lanes or other unexpected behavior.
To be clear, these are from people who really like the feature and find it useful, but they are really aware that it has to be watched just like if they were 100% manipulating the controls.
Agreed. I am a big fan of Tesla and I used to make excuses for them on this but I think it’s b.s. they didn’t rename it after the incident where the guy died. Cool marketing names do not outweigh being totally honest and up front.
If I made rat poison that looked like candy and named it “Yummy Candy Rat Poison” and some kid died coz they ate it, I’d deserve some blame regardless of parents keeping poison away from kids.
It isn't called full self driving, it is called auto-pilot. In aviation, auto-pilot is a system used to control the trajectory of an aircraft without constant 'hands-on' control by a human operator being required. Autopilots do not replace human operators but assist them in controlling aircraft, allowing them to focus on broader aspects of operations.
Well context is important, the people who operate planes know exactly what autopilot in a specific plane means, the people who operate cars are generally aware that there are cars.
Tesla state that autopilot will "match speed to traffic conditions, keep within a lane, automatically change lanes without requiring driver input, transition from one freeway to another, exit the freeway when your destination is near, self-park when near a parking spot and be summoned to and from your garage."
I believe the problem lies in "Enhanced" and "Full self driving hardware". While this is an important point and is probably true, there should probably be bigger warnings that there is currently no full self driving software, and therefore, no full self driving.
Not really. I am one of the people that actually use it. They have some warnings for sure, but you should probably note I also wrote the first comment in this thread.
The thing is that almost nobody knows what an Autopilot actually does. Even here on Hacker News you see comments to the effect of "why do we still need pilots, planes take off fly and land by themselves!"
That's my point - the public perception (even among a technical crowd) is way off from the reality. People think an "Autopilot" does way more than it actually does.
I personally don’t agree, people know the pilots have a real role in the plane among even my non-tech friends, but it’s just anecdotal vs your anecdotal so it doesn’t make either of us right or wrong.
This is the same argument people use every time, and that Tesla uses. The general public does not know that. It's an irresponsible name for the system and they know it.
I don't think the naming is the problem. No matter what you call it, the fact that it works 99% of the time gives people confidence to continue using it, up until it doesn't work, and then there's no time to react. In a car, you generally have some space and time in front of you, but you usually have no lateral margin of error, and that assymmetry is what makes it different from planes and boats.
I couldn’t agree more. It baffles me how Tesla can get away with such blatant false advertising (it’s anything but “autopilot” in any layman sense), especially in light of there being multiple incidents of bad accidents which showed them at fault.
There's lots of autopilot systems that smash into shit, sailboats are one of them, you can put your sailboat on autopilot and it's going to smash into stuff.
Samething with an airliner, autopilot will not avoid another aircraft.
They exist. But collision avoidance is far from a necessary retirement. E.g. my father in law's private single engine. Autopilot is basically cruise control, but you know...for a pilot
TCAS in a plane is like Automatic Emergency Braking (AEB) in a car. It's active 100% of the time in cars, not just when you're using super-cruise-control or other automation.
Of course TCAS is always enabled while in flight and is technically unrelated to autopilot. That doesn’t negate the point that many Airbus planes will automatically follow RAs when on autopilot if that is enabled by the flight crew.
The existence of one that avoids some aircraft in some situations does not negate the fact that there are aircraft autopilot systems which do not.
Also TCAS enabled autopilots will still smash into other planes. TCAS isn’t magic plane detection AI it’s a radio signal, you can turn it off and not all aircraft have it.
Just Incase you were going to use the same logical fallacy just because an A380 has TCAS doesn’t mean all planes do.
Also airbus planes have some automatic flight control systems that override pilot input, those systems have CAUSED planes to smash into stuff. So even on an aircraft autopilot / automatic controls still cause preventable accidents.
From Elon's interview, he mentioned that 'autopilot' was a name akin to the flight autopilots. Pilots are never expected to get their eyes off the controls.
This crash is unfortunate and should be investigated. But autopilot naming is something I don't think is the problem.
> Pilots are never expected to get their eyes off the controls.
Pilots do so all the time, because airplanes aren't flying in a congested 2D space. Airspace is mostly empty, with very large distances between planes and features, allowing for plenty of early warning.
And an airplane's "autopilot" has nothing to do with Tesla's Autopilot. Their purpose and working mechanisms are completely different.
The aircraft autopilot is pretty much the same thing. It steers on the designated lane, maintaining attitude, and relies on the pilot to be aware of conditions.
It operates the controls but relies on the human operator for supervision. Pilots talk about “flying in the magenta” and exactly the same thing applies to Tesla Autopilot.
Believing that autopilot means something different is the bullshit attitude that has to change.
I think the main difference is that, on an airplane, a momentary (less than 5 seconds) lapse in judgment isn’t going to down the plane. Pilots know when they have to be heavily engaged — during takeoff and landing — and when to let autopilot take over. Here, if you zone out for a few seconds at any point in your trip you could easily kill yourself and others.
A lapse of judgement is potentially going to cost lives in a plane too. You have an overly romanticised notion of air safety if you believe otherwise. Pilot Error is what they call it, including euphemisms such as “controlled flight into terrain” meaning the pilot was in control and doing everything wrong.
There are two people flying the plane, with rigorous training and controlled work hours. Air safety, as they say, is no accident.
An aircraft autopilot is not designed to automatically avoid collisions with other traffic or animals. It's purpose is only to maintain course, speed, and altitude. In VFR conditions while autopilots are useful continuous scanning of your surroundings is necessary.
Meanwhile a Tesla "autopilot" is designed to drive in traffic with other vehicles.
The aircraft autopilot can also fly a route and handle everything from takeoff to landing, but pilots don’t rely on it to do so.
Tesla Autopilot can keep pace with surrounding traffic and stay between marked lines but does not recognise stationary obstacles.
They are very much the same in that they automate the mundane parts of the operation of the vehicle. They are also the same in that they do not know how to handle conditions outside their operating envelope.
Yes I was aware of TCAS. There are some cool technologies out there. I am pretty sure out of all aircraft with autopilots, the amount with this technology is extraordinarily low. Especially if you are considering only personal aircraft and not ones designed to carry over 1,000 passengers.
The topic of conversation in this thread is about the system that the general population thinks about when they hear “airline autopilot”. Most folks would naturally think of commercial passenger planes and their (believed) capabilities.
Just a quick note - most of the larger aircraft with autopilots are most often flown under IFR (Instrument Flight Rules), which means that there is someone else (Air Traffic Controllers) managing all of the routing and collision avoidance for them.
Even pilots in IFR conditions are doing little more than maintaining a particular altitude and heading.
This means that a aircraft and vehicle autopilot have very different requirements. They are not, at all, doing the exact same things.
Which is silly, since pilots can absolutely take their eyes off the controls at cruising altitude and not slam into something. If, for some reason, the system disengages they would have ample time to take control of the plane. That is absolutely not true with an automobile, where in some situations, an action must be taken in milliseconds.
> From Elon's interview, he mentioned that 'autopilot' was a name akin to the flight autopilots.
Then he failed, as they are completely unrelated.
> Pilots are never expected to get their eyes off the controls.
This is also false. They need to be alert, but they most certainly do not need to be as attentive as keeping their hands on the controls unlike with Tesla.
Unlike the sibling comment below alluded too, many commercial airplane autopilot systems are advanced enough to land the plane completely on auto as well as some even automatically respond to TCAS alerts to avoid nearby aircraft.
I do not believe Tesla’s are equipped with the necessary hardware to scientifically show beyond a reasonable doubt they assisted in preventing accidents.
Part of the problem is that whether an accident would have occurred is partially dependent on the human driver. While some accidents are unavoidable and some are completely preventable, there's a gray area in between those that depends on driver skill to a degree.
Tesla has not been counting collisions caused by the driver having to brake hard because the autopilot was about to happily slam into another vehicle/object/person? Tesla's summation of those is that they were not a failing of Autopilot, because it was not active at the moment of impact. That's ... disingenuous ... at best.
And then:
"It would have avoided this accident [by braking, steering]", and studiously saying nothing about the future, "... but it would still have been in a collision 0.42 seconds later".
For all the collisions it would have avoided, there's another subset where it would only have "delayed" the collision. But that won't be mentioned. Because it doesn't fit the narrative.
Source? I've seen people speculate about the first, but just speculation. And that Google/Waymo does the same thing. I don't think it's true for either: public companies get sued for lies like that, and the NTSB is not stupid.
Couldn’t agree more. People who argue for Elon’s meaning of auto-pilot are wrong. Unless 100% of Tesla drivers are 100% aware of the limitations calling it auto-pilot is dangerous.
And yet when we get stuck in thought and don’t think about where we’re going and drive to our office on the weekend by accident we say “oh, I got stuck in autopilot”
These accidents are not people mistaking the purpose of autopilot, they are people slipping into false confidence that the car will handle the driving for them because it has in the past.
Most commercial airline autopilots in use today could land the plane on auto if desired by the flight crew, so no, they don’t “just maintains a heading and an altitude”.
I think we all know that such technology exists. That is why I specifically said "in the traditional sense", to point out that I'm talking about what the word was invented for.
Autopilot can refer to everything between a "wing leveler", and a full-on automated landing system. So no, even in aviation there is no standard definition of what autopilot does.
Emergency services want autonomy in being able to shut the freeway down.
How about PUT THE FUCKING AUTOPILOT SYSTEM IN THE EMERGENCY vehicles -- such that THEY can report to ANY auto driving car "SLOW DOWN we are doing emergency services [HERE]"
And then you get both -- The emergency services vehicles should be freaking beacons to any self/auto driving car.
To assume that this was short-sighted on either side is correct. Tesla should have been thinking about this, emergency services should have been thinking about this.
Why doesnt Tesla take their Semi model and make a Fire-Engine model????
I'm pretty confident that no system (beyond the visual and audio cues currently on emergency vehicles) is ready for implementation. Car mesh/broadcast networking is a tempting idea, but I'm not sure any of us are ready for it.
> Why doesnt Tesla take their Semi model and make a Fire-Engine model????
The automation in Tesla's vehicles is, and should be, designed around modern conditions, not a utopia. Otherwise disable all those features until that utopia arrives.
Or would we say collisions with pedestrians are okay until such pedestrians all have autonomous human beacons (made by Tesla, of course)?
Tesla autopilot is more capable than an airplane autopilot. An airplane on autopilot will crash into stationary objects in its path, by design. Thus, airplane autopilots require full pilot attention as well.
This thread happens with every Tesla crash, by the way:
1. Tesla crashes
2. Driver says autopilot was engaged
3. Long debate on HN about exact meaning of word "autopilot"
4. 9 times out of 10, Tesla log says autopilot wasn't on before the crash, or that obstacle was visibly in the path for more than enough time for the driver to take action (usually half in the lane and half out)
The other one time out of 10 the driver says the Tesla suddenly accelerated and drove through most of a strip mall.
I can explain. For about the first five minutes of the comment's life, the first sentence used a snippy tone. I got the message and removed the snippy part.
In boats we have had autopilot for decades. It takes the boat on a straight path to a point via GPS only. You must pay attention and correct course for obsticals in your path. Simple, logical and efficient.
But you can hand off complete control to a co-pilot in a plane while you go use the restroom, etc. With autopilot you are supposed to be there to monitor it.
People who actually drive Tesla cars see a lot more information than people who see Tesla marketing. Market surveys of Tesla owners say that they aren't confused about it requiring driver attention.
Also, requiring the driver to pay attention on what the automation do is a bit unrealistic.
It's really difficult to stay focus and (mostly) do nothing at the same time. Monitoring automation is boring. Also switching from a passive mode to an active mode in few seconds in case of an emergency might be really hard.
I'm not sure the Tesla "autopilot" is designed in a way that really keeps the driver in the loop and active enough.
I guess it's not all bad news - a collision with a stationary object at 65mph and the driver was able to walk away
> Because of the force of the impact, firefighters advised the Tesla driver that he should be taken for a medical evaluation, but he showed no significant injuries and refused treatment,
You know how stoplights are in tune with emergency vehicle flashing lights? I think the auto pilot systems need to get more in tune with those systems as well. An alert needs to fire when it detects an emergency.
I've heard of people not turning off their cruise control when going through an accident area. Boneheads. This is no different - just involving the thing after cruise control.
Because most modern vehicles are made to the same crash safety standards and would likely have a similar outcome?
The Tesla isn't magical, made of adamantium, or such.
Not to sound overly snarky, but as someone who has been a firefighter / paramedic for the best part of a decade, I've certainly seen my fair share of MVAs up close.
Just the other day I was using the Tesloop [1] service from near LAX and the driver was praising the autopilot feature. I thought it was really neat at the time, but I would be much more hesitant about it now :/
As a current Tesla owner and frequent autopilot user I can attest to this being a scenario where autopilot has not worked for me - traveling at high speed and approaching a stopped vehicle from a distance. I now always take it off autopilot when this occurs because it will not brake in time.
I'm actually surprised by this. Isn't this something which the numerous brake-assist systems are supposed to handle easily? The radar should see far enough and Doppler should tell the relative speed.
From a technical perspective—why would Tesla not have taken more care to address this scenario? It seems like low hanging fruit in comparison to other challenges that autopilot has to be able to handle.
I’ve wondered this myself - it’s a great system and I’ve really enjoyed using it once I learned what it’s good for and the limitations, but this does seem like a glaring issue. My best guess based on the AP display, is that it doesn’t see the stopped vehicle out far enough to initiate braking.
They've previously said they detect stationary objects and purposefully ignore them. I think with the intention of not stopping for foliage or something.
I get the impression that these incidents (if this is the case again) are due to Tesla being confident in their camera-based approach (with mid-range radar). Other manufacturers seem to be emphasizing hardware to a greater extend, with Audi even having lidar in the new A8.
They did address it, starting with software 8.0. See http://www.tesla.com/blog/upgrading-autopilot-seeing-world-r... . Chances are that evidence will show that the vehicle detected the fire truck, gave audible warning, and then started braking. That's all if the driver had not disabled forward collision warning and automatic emergency braking. They are standard safety features and unless disabled are active during manual driving as well as when Autopilot is active. I don't work for Tesla, but I am very familiar with this type of technology and Tesla vehicles' owner manual descriptions of operation.
Same here. There is rarely a day that I don't have to disengage because of a car moving into my lane a little too close and Autopilot doesn't see it. It's great in bumper to bumper traffic, but at highway speed there are too many blindspots.
Just curious what action is needed to disengage autopilot, is it simply a matter of grabbing steering wheel, or does a button need to be pressed? Is there a "hybrid" mode where it remains in autopilot but with option to manually adjust speed and steering?
Good they remembered the "slows car" feature. Perhaps fines for autopilot incidents should be greater than if a human makes the mistake. Car maker pays a sort of penalty tax every time their AI screws up. Hitting a fire truck is unforgivable. Emergency brake feature can't detect a big truck? That's back to drawing board stuff right there.
"The California Highway Patrol and Culver City Fire Department confirmed the southbound Tesla had struck the fire truck, but could not immediately confirm whether the vehicle had been on Autopilot."
As I understand it (and please correct me if I'm wrong) the automatic emergency braking (AEB) system is separate from the autosteer system on a Tesla. It seems to me that this kind of incident is a result of AEB failure, not of autosteer failure.
I wonder how Tesla's AEB stacks up against similar systems on other vehicles? There are a lot of vehicles on the road now with automatic braking, but we don't see every single frontal collision on such cars reported as some kind of systemic failure.
Tesla's AEB likely triggers earlier than many others. See this example http://youtu.be/FadR7ETT_1k . The vehicles ahead were moving, but after http://www.tesla.com/blog/upgrading-autopilot-seeing-world-r... was implemented in software 8.0, I believe the same was possible for already-stopped vehicles, too. We'll find out in the accident report, and perhaps directly from Tesla after they retrieve the vehicle's data.
Firefighter here. This kind of collision happens all the time -- although it's usually a manually-driven car. This is why we park the BRT* behind the accident and we try to stay in front of the truck. A moving 4000-lb car is no match for a stationary 30,000-lb fire truck.
This is worrying. The Florida case was understandable. I fail to understand how a big red vehicle with flashing lights could've been missed by the Vision system of Tesla.
I take issue with people who insist on abrogating responsibility. The Tesla tells you to keeps your hands on the wheel.
Pilot training covers techniques for avoiding automation dependency (aka letting autopilot kill you and all tour passengers). Perhaps we need the same for car drivers: if the car was as good a driver as you, it wouldn’t give you the option to drive.
To me Tesla autopilot accidents are like airliner crashes: they lead to a safer system, one that in the future will avoid the same mistake again. Compare that to human-operated vehicles: we keep getting into the same accidents over and over. The fact that autopilot data gets sent back to the factory, is analyzed and results are fed back into the the production line is a game changer that will help Tesla keep ahead of the competition a while to come. Right now the rate and importance of autopilot system interations are an order of magnitude higher than other auto-safety evolution from the past.
183 comments
[ 2.2 ms ] story [ 226 ms ] threadAirbags also kill babies if used incorrectly.
There are warning placards in several places in your car saying such.
Sometimes an airbag can cause more injury than the accident.
A seatbelt may trap you in a sinking car.
But overall, airbags and seatbelts have significantly reduced traffic injuries and fatalities.
Autopilot saves lives. Autopilot used incorrectly can kill people. But overall, Autopilot has significantly reduced the number of accidents involving Teslas.
So do we take airbags away because using them incorrectly might hurt a child? Are they a safety feature or marketing value?
There is no evidence of this.
http://bgr.com/2017/01/19/tesla-autopilot-crash-safety-stati...
Time will tell as they ramp production.
> "The probability of having an accident is 50 per cent lower if you have Autopilot on," said Musk, speaking at an energy conference in Oslo, Norway. "Even with our first version, it's almost twice as good as a person."
> The results are from Tesla's first generation of Autopilot, which owners have already used to drive over 47 million miles since it launched in October 2015.
I don't know what the answer is, but there's certainly evidence.
Sorry, I don't get it. How do you use something incorrectly when by definition you aren't really using it at all? It seems like an oxymoron.
Engineers are required to to make some kind of input to verify that they're still alert, or the train stops[1] if they haven't done anything in X seconds.
Obviously you wouldn't want the car to just stop in the middle of the road, but there could be something like that designed.
[1] https://www.quora.com/If-a-train-driver-falls-asleep-what-ha...
I'd imagine Tesla wasn't thinking about how the driver alertness feature could be countered unfortunately. Perhaps we'll see Tesla adapting the eye tracking tech that Cadillac Hypercruise is using.
https://www.dailydot.com/debug/tesla-orange-hack/
I wonder how to truly tell if a driver is attentive? The Cadillac indicates it uses head tracking technology. I wonder if that is sufficient? Seems you probably need even eye-tracking: an attentive driver keeps looking around or something.
I suspect the primary problem is not the partial autonomous technology, but the marketing which is convincing people they can trust it. Nobody ever told you your cruise control means you don't have to pay attention, so very few[1] incidents happen due to misuse of it.
First and foremost, Tesla should never have been allowed to release "Autopilot" being called that. From the very name it provides a false impression of what it can do.
[1]I know you can find one. Humans are remarkably stupid.
It's true that people may be in particular need of a refresher, since such features could be perceived as more than advanced cruise control. It should be strongly emphasized to all drivers that we need to take full control whenever we see any exceptional conditions, from hazards, obstructions, pedestrians or wildlife in or entering the road, to poor visibility, and even when passing or being passed by small vehicles.
More than half a century in mass-market automobiles. https://en.wikipedia.org/wiki/Cruise_control#History
Tesla I think has done a fantastic job at striking a balance between getting something into the market the generate real-world data and mitigating the risks that come with it. If this car really did crash into the fire truck at around 65 MPH then it did a pretty good job at keeping everyone safe.
I think I would be hard pressed to find any technological or mechanical innovation that did not initially involve risk or have an accident associated with it. I've seen more examples where a Autopilot has done the right thing and avoided an accident than I have read stories of it making a mistake and causing an accident.
I believe your definition of keeping everyone safe differs from the common usage.
I don't see any evidence that Tesla has more, or less for that matter, of the necessary high-quality data that is needed for self-driving cars. The fact that real people are driving those cars continuously, potentially generating reams of data, is pretty immaterial, since the majority of that data is going to be useless (let's face it, 10 million hours of driving in the same conditions isn't better than 1 million hours of driving).
In my view, driving is already boring enough that it's insufficiently attention-grabbing for safety's sake. That's why, e.g., the "naked streets" model is removing lane marking. Decreasing (false) certainty forces more cognitive engagement, making things safer: https://www.theguardian.com/commentisfree/2016/feb/04/remova...
If anything, for the decades [1] while we're waiting for full automation, I'd like to see the equivalent of Threat Image Projection. It's the technology for luggage scanning where they put images of fake guns, bombs, etc in the images of real luggage. If the operator doesn't press the "threat" button they are alerted that they slipped up. It brings the frequency of important events up into the range where humans can pay attention. It'd be great to have the same thing for cars, so that AR images of deer, cyclists, children, and fire trucks appear; if you don't attempt to take evasive action, you lose points. [2]
[1] E.g., iRobot founder Rodney Brooks says not before mid 2030s: http://rodneybrooks.com/my-dated-predictions/
[2] I am, of course, kidding.
This type of accident in Level 2 systems is never going to be zero.
If the roads had sensors in the "bots dotts" and would track and steer all the volume of traffic - isnt that better than all the infantile-AI-cars doing it?
Will this turn out to be another "brightly lit sky" autopilot bug, like the 2016 semi truck incident?
https://www.tesla.com/blog/tragic-loss "Neither Autopilot nor the driver noticed the white side of the tractor trailer against a brightly lit sky, so the brake was not applied"
https://www.tesla.com/blog/upgrading-autopilot-seeing-world-...
"Autopilot was not active at the time of collision, according to logs"... because it was deactivated by hard manual braking a moment before, because Autopilot was about to cause a hard collision...
It sounds like it's not 100% confirmed that Autopilot was at fault. The first sentence of the article also reads "reportedly on 'Autopilot'"
Isn't that usually because the owner lied about using Autopilot?
The only thing that keeps a check on the impulses of the most aggressive human drivers is the unpredictability of other human drivers.
For example, if your computer control car knows that the car behind is also computer controlled, it could be more aggressive on the brakes to avoid a forward collision without causing a rear collision.
Tomorrow, the situation might be that a computer controlled car could see that the car behind is tailgating and the car in the next lane is computer controlled — then determine the safest action would be to swerve into the next lane rather than brake suddenly...
And I would dispute that emergency braking suddenly while being tailgated is less "crazy shit" than swerving into the next lane where your presence will, with an exceptional degree of confidence, be immediately detected and compensated for.
To clarify, the scenario in my mind is one that would be bold but not dangerous; something a good human driver might do with 360° situational awareness. In this instance it could be the difference between swerving to miss an obstacle or a multiple-car collision — even if your self driving car can stop in time, the tailgater is likely to slam into you, which in turn slams you into the obstacle.
Or to change the situation a bit, if the "tailgating" car happened to be a self driving car with an aggressive profile, that could swing the decision back towards braking. Point is, knowing that a car is being driven by a computer can aid decisions made by other cars.
By the way, one can speculate there are interesting ways to trigger beeps inside these Teslas coasting on the left lane.
It doesn't take a single car to reduce the speed of a lane. It takes exactly two. One to slow it down, and the other one to stop the others from letting the guy in the front know that he is going too slow.
Please drive safely.
The really scary ones are the ones that own it, know about the limitations, and intentionally bypass the safeguards.
https://insideevs.com/tesla-autopilot-survey-results-98-unde...
To be clear, these are from people who really like the feature and find it useful, but they are really aware that it has to be watched just like if they were 100% manipulating the controls.
https://www.dailydot.com/debug/tesla-orange-hack/
If I made rat poison that looked like candy and named it “Yummy Candy Rat Poison” and some kid died coz they ate it, I’d deserve some blame regardless of parents keeping poison away from kids.
https://carbuzz.com/news/teens-take-tesla-model-s-for-joyrid...
I believe the problem lies in "Enhanced" and "Full self driving hardware". While this is an important point and is probably true, there should probably be bigger warnings that there is currently no full self driving software, and therefore, no full self driving.
Samething with an airliner, autopilot will not avoid another aircraft.
Wrong! There are autopilots which integrate TCAS into the autopilot (A380 among them).
Source: http://www.airbus.com/newsroom/press-releases/en/2009/08/eas...
Tesla Autopilot is a combination of both steering and throttle control.
TCAS sometimes will and it doesn’t matter if autopilot is on/off.
Also TCAS enabled autopilots will still smash into other planes. TCAS isn’t magic plane detection AI it’s a radio signal, you can turn it off and not all aircraft have it.
Just Incase you were going to use the same logical fallacy just because an A380 has TCAS doesn’t mean all planes do.
Also airbus planes have some automatic flight control systems that override pilot input, those systems have CAUSED planes to smash into stuff. So even on an aircraft autopilot / automatic controls still cause preventable accidents.
This crash is unfortunate and should be investigated. But autopilot naming is something I don't think is the problem.
Pilots do so all the time, because airplanes aren't flying in a congested 2D space. Airspace is mostly empty, with very large distances between planes and features, allowing for plenty of early warning.
And an airplane's "autopilot" has nothing to do with Tesla's Autopilot. Their purpose and working mechanisms are completely different.
The airplane analogy is pure bullshit.
It operates the controls but relies on the human operator for supervision. Pilots talk about “flying in the magenta” and exactly the same thing applies to Tesla Autopilot.
Believing that autopilot means something different is the bullshit attitude that has to change.
Warren Van Der Burch gave a classic presentation, Children of Magenta: https://youtu.be/pN41LvuSz10
There are two people flying the plane, with rigorous training and controlled work hours. Air safety, as they say, is no accident.
Meanwhile a Tesla "autopilot" is designed to drive in traffic with other vehicles.
Tesla Autopilot can keep pace with surrounding traffic and stay between marked lines but does not recognise stationary obstacles.
They are very much the same in that they automate the mundane parts of the operation of the vehicle. They are also the same in that they do not know how to handle conditions outside their operating envelope.
The A380 begs to differ...
http://www.airbus.com/newsroom/press-releases/en/2009/08/eas...
Even pilots in IFR conditions are doing little more than maintaining a particular altitude and heading.
This means that a aircraft and vehicle autopilot have very different requirements. They are not, at all, doing the exact same things.
Then he failed, as they are completely unrelated.
> Pilots are never expected to get their eyes off the controls.
This is also false. They need to be alert, but they most certainly do not need to be as attentive as keeping their hands on the controls unlike with Tesla.
Unlike the sibling comment below alluded too, many commercial airplane autopilot systems are advanced enough to land the plane completely on auto as well as some even automatically respond to TCAS alerts to avoid nearby aircraft.
I do not believe Tesla’s are equipped with the necessary hardware to scientifically show beyond a reasonable doubt they assisted in preventing accidents.
Tesla has not been counting collisions caused by the driver having to brake hard because the autopilot was about to happily slam into another vehicle/object/person? Tesla's summation of those is that they were not a failing of Autopilot, because it was not active at the moment of impact. That's ... disingenuous ... at best.
And then:
"It would have avoided this accident [by braking, steering]", and studiously saying nothing about the future, "... but it would still have been in a collision 0.42 seconds later".
For all the collisions it would have avoided, there's another subset where it would only have "delayed" the collision. But that won't be mentioned. Because it doesn't fit the narrative.
These accidents are not people mistaking the purpose of autopilot, they are people slipping into false confidence that the car will handle the driving for them because it has in the past.
To be fair, an autopilot, in the traditional sense, just maintains a heading and an altitude.
Tesla wants autopilot
Emergency services want autonomy in being able to shut the freeway down.
How about PUT THE FUCKING AUTOPILOT SYSTEM IN THE EMERGENCY vehicles -- such that THEY can report to ANY auto driving car "SLOW DOWN we are doing emergency services [HERE]"
And then you get both -- The emergency services vehicles should be freaking beacons to any self/auto driving car.
To assume that this was short-sighted on either side is correct. Tesla should have been thinking about this, emergency services should have been thinking about this.
Why doesnt Tesla take their Semi model and make a Fire-Engine model????
Difficult to blame the emergency vehicles when the semi-autonomous car fails to detect something that stands out so much.
The automation in Tesla's vehicles is, and should be, designed around modern conditions, not a utopia. Otherwise disable all those features until that utopia arrives.
Or would we say collisions with pedestrians are okay until such pedestrians all have autonomous human beacons (made by Tesla, of course)?
This thread happens with every Tesla crash, by the way:
1. Tesla crashes
2. Driver says autopilot was engaged
3. Long debate on HN about exact meaning of word "autopilot"
4. 9 times out of 10, Tesla log says autopilot wasn't on before the crash, or that obstacle was visibly in the path for more than enough time for the driver to take action (usually half in the lane and half out)
The other one time out of 10 the driver says the Tesla suddenly accelerated and drove through most of a strip mall.
And what are stats of accidents on Auto-pilot vs without? If less, I'd say it's a success.
Have you ever driven a Tesla?
It's really difficult to stay focus and (mostly) do nothing at the same time. Monitoring automation is boring. Also switching from a passive mode to an active mode in few seconds in case of an emergency might be really hard.
I'm not sure the Tesla "autopilot" is designed in a way that really keeps the driver in the loop and active enough.
> Because of the force of the impact, firefighters advised the Tesla driver that he should be taken for a medical evaluation, but he showed no significant injuries and refused treatment,
I've heard of people not turning off their cruise control when going through an accident area. Boneheads. This is no different - just involving the thing after cruise control.
Why am I not already driving a Tesla?
The Tesla isn't magical, made of adamantium, or such.
Not to sound overly snarky, but as someone who has been a firefighter / paramedic for the best part of a decade, I've certainly seen my fair share of MVAs up close.
[1] https://tesloop.com/
It is not allowed in Japan (where I live) because of the draconian vehicle inspection program, but I guess in some US states it is more liberal.
So was it on autopilot or not.
I wonder how Tesla's AEB stacks up against similar systems on other vehicles? There are a lot of vehicles on the road now with automatic braking, but we don't see every single frontal collision on such cars reported as some kind of systemic failure.
*Big Red Truck
Pilot training covers techniques for avoiding automation dependency (aka letting autopilot kill you and all tour passengers). Perhaps we need the same for car drivers: if the car was as good a driver as you, it wouldn’t give you the option to drive.
https://youtu.be/pN41LvuSz10