I was expecting a catch. Perhaps a play on words or something.
Nope. Just an actual Tesla colliding with an actual jet.
What’s surprising to me is that it keeps moving post collision… seems odd that it can meet with unexpected resistance and not trigger some sort of emergency measure. (eg. stopping)
I used to be bullish about Tesla’s “only cameras” approach, and I still think they’ll crack FSD in some form, but I am increasingly wondering if there needs to be additional sensors for an abundance of extra safety around collision avoidance.
Does an airplane provide substantially more resistance than trying to roll through a decent sized pothole without the benefit of much momentum?
That airplane is small enough it probably gets dragged around by hand when it's not chocked. The chocks small airports tend to use are pretty small to fit under the wheel fairings many small planes have. I can see it pushing the aircraft around by the tail.
Fair enough. So judging by resistance alone may not be enough….
Needs additional forward facing sensors then? (For both long and short distance object detection?)
Earlier on in Autopilot/FSD beta testing it was easier to overlook edge cases like this but I think we’re several years into this now and I’m starting to feel that at this stage it should be almost impossible for these cars to be driving into stationary objects.
I’m a proponent of what Tesla are doing but they need to solve this ASAP now, whether by hardware or software, before it erodes too much more confidence (especially as it’s all too easy for people to extrapolate from these documented edge cases and believe these vehicles to be less safe than they actually are).
As this is unfinished software which should have extremely high safety requirements: I would very much like for it to be stuck on minor bumps and whatnot. In fact, I would like it to stop moving the car, sound a very loud alarm, and immediately notify the operator on multiple channels if anything even remotely out of the ordinary happens. Tesla should devote as many resources to detecting abnormal conditions as they are on making the car drive in the first place.
The owners here are alpha testing a complex and dangerous system and they should expect nothing less.
While I do agree with your skepticism, how would stopping immediately work on the highway? There are often bumps and ground shifts on the highway. Don't want to stop automatically then.
> The owners here are alpha testing a complex and dangerous system and they should expect nothing less.
This is the part of Tesla's business that upsets me. Their marketing department is being allowed to do what marketing departments do, rather than biting the bullet and letting it be known that this is a complex, dangerous, young, and known to fail system.
There are simple decisions here that should be made but weren't, e.g. "keep the product code-name until it is out of beta"; nothing on a current model Tesla deserves being called "self-driving" in any way.
My understanding is that a human is //supposed// to be observing the vehicle the entire time that smart summon is in use, hence why the end-user has to hold down a button on the app the entire time the vehicle is in motion.
My understanding is that smart summon operates on a completely different set of rules than navigate on city streets, even though they might share some common functionality, like object detection and mapping.
As a baseline it's supposed to avoid running into objects. It doesn't seem to be able to do this relatively simple task very well even at 5mph, which is why you need to watch it like a hawk. Basically nobody sane ever uses it at all after they've tested it a few times.
It runs into the tail of the plane, which is sticking out over the place it wants to travel, and pretty high up (it appears to collide with the top of the windshield).
It's understandable why it may not be designed to check that situation, as it is not one you'd normally find in a parking lot (unless maybe a parking lot full of semi trailers).
I guess they assume they don’t have to check for high objects because if it’s in a height restricted parking garage when smart summon is activated, then by definition it must have fit past any restrictions originally on the way in.
It is absolutely not understandable or excusable! This is a company who claims that they have self-driving technology at or above the level of human safety. Driving into something that your car can't fit under isn't an edge case. It's just baseline, minimum viable product.
Tesla is going to do more harm to the prospects of self-driving cars than anyone. Their "test in production" attitude is going to not only cost (more) lives but also force the government to strictly regulate the tech delaying the other companies that are acting responsibly.
> It's understandable why it may not be designed to check that situation, as it is not one you'd normally find in a parking lot (unless maybe a parking lot full of semi trailers).
Sounds pretty similar to the accidents involving trailers, where the top of the cars were sheared off because it didn't detect anything there.
> Neither Brown nor the car braked for a tractor-trailer, which had turned left in front of the Tesla and was crossing its path. Brown’s Tesla also went beneath the trailer and its roof was sheared off. After that crash Tesla CEO Elon Musk said the company made changes in its system so radar would play more of a role in detecting objects.
They got rid of radar, so I wonder if that's made it again more difficult to detect these scenarios.
There should be some sensor to distinguish hits on the chassis or on the wheels, at the very least it should create distinct noises on an accellerometer.
Should be able to tell from the suspension. If front extends before back, rocking the chassis backwards at all, then you hit something above the axle line.
This seems exactly the same kind of error as the Florida Tesla that underran a crossing semi trailer and killed the driver. Teslas seem bad at detecting obstacles that are at about windshield height but don't touch the ground. Possibly because they don't train the system with such obstacles. Or not enough, anyway.
Seems like the airplane owner would sue the owner of the car who should be covered by the insurance for some property damages and the car insurance might go back and sue tesla to recover funds given how unusual the situation is
It doesn't even stop after hitting the jet. It just keeps going right through it.
I don't think you can take this autopilot, and FSD (Fools Self Driving) contraption seriously or remotely ready for customers to use; even for a 'beta'. Since it's functionality doesn't even work as advertised, puts the lives of other drivers on the road at risk and it is completely useless at the most dangerous time to drive, which is at night.
For example, as a simple test against another autonomous vehicle with LIDAR at night. [0] The Tesla with FSD failed to stop and just ran over the object.
The best part is 100,000 crash dummies have this software installed and seem to be taking the risk in trying it out, even when it malfunctions on the public roads and what we can see here. This contraption needs to be thoroughly investigated.
> It doesn't even stop after hitting the jet. It just keeps going right through it.
I thought that was the worst part of it too, assuming theres nobody inside the car at the wheel. It keeps going, stops, then moves again and looks to go around whatever obstacle it thinks exists. What if this situation can be replicated on an empty road where the tesla runs over a human being? In the worst case, does the tesla keep trying to drive over the human being and turns a minor injury into a fatality?
If you wanted to do this safely, and you were ok with the massive data collection privacy hazard, what you'd do is collect real-time driving data from a bunch of cars:
a) the full sensor stream if possible, but if not, then at least the candidate software could process the sensor stream in real time and output a condensed state and save that
b) the human driver's actual control inputs
Collect these and analyze when the software's suggested inputs deviate significantly from the human's input. Some of the time, that'll be computer reacted to things faster, or human appears intoxicated, but based on real world reports, there's got to be a large space of human driver did the right thing and computer ignored important parts of the scene.
I also think there's some evidence that Tesla has the approach of continue driving unless computer can determine it's unsafe to continue, whereas Alphabet/Google/Waymo has the approach of only drive when computer can determine it's safe, which is way more frustrating to drive nearby, but leads to better outcomes. With Tesla, we usually have reports of collisions at the intended driving speed (which is slow for summon, so great), whereas Waymo collisions are usually at slow speeds often as a result of traffic hitting the Waymo vehicle which was stopped in an unexpected way.
Wow it's actually somehow even worse, the LIDAR equipped car passed the test with a moving dummy from behind an obstruction. The Tesla failed with an unobstructed stationary dummy.
This car reviewer tried it (with his mom holding the button) and it would've pancaked him too (they tried it twice, once with his mom's motherly instinct letting go of the button): https://youtu.be/kW2W2l2kXFM?t=410
I remember part of the report on the Uber that ran the person over was that the Uber’s classifier couldn’t figure out what a person with a bike in the road was, so it didn’t make it into the decision tree. I’m curious if that’s what happened here - Tesla never trained their cars to recognize airplanes, so it just didn’t “see” it.
The part that baffles me is that there isn't a "There's an object, not sure what it is, but there's something, I should stop/avoid it" instead it seems to be "I don't know what this object is. Oh well, it's probably fine to plow into it"
The Uber self-driving car was actually a Volvo -- its logs show that the Volvo emergency braking system had detected a pedestrian crossing almost 2 seconds before the collision. It would've initiated emergency braking, but Uber had disabled this feature, since, of course, they were testing their own shitty self-driving software.
The Uber software chose to take no action, the Uber employee at the wheel was apparently paying no attention, and Uber had disabled the Volvo software, the only agent involved that actually thought it would be a good idea to pay attention and try not to kill a woman with a bike on a dark night. Uber shouldn't be allowed to test their software on public roads in my opinion.
Insurance policies can exclude coverage off public roads. If the Tesla and aircraft owner are one and same, the owner may be holding the bag for the damage.
The aircraft owner and insurer could sue Tesla, but likely Tesla's terms of use require the car only be on public roads and private driveways.
I think that’s a prime example for the shortcomings of machine learning. It can only “learn” from what is known, anything unknown can be potentially a catastrophe. I believe with the current AI in its infancy, a combination of rule-based and machine learning would be a more dependable approach.
The problem with your statement is that AI drivers do rely on traditional logic already, but that cannot be applied to the perception part of the stack. It doesn’t matter if your rule is “if perceives truck, stop” if perception is telling you there is no truck stopped there. The rule also can’t be “if white pixel, stop” or “if Lidar bounces off steam and returns, stop”
if you're doing vision based obstacle avoidance and you're training your model on specific classes of objects and not just object you're doing it wrong
Can someone explain to me why Tesla didn't bootstrap itself with cameras/LIDAR, then remove the LIDAR? I mean teaching an AI depth perception and the ability to model objects by vision, without including training data from LIDAR just seems kind of silly.
Tesla very clearly states:
"Smart Summon is a BETA feature. You must continually monitor the vehicle and its surroundings and stay prepared to take immediate action at any time. It is the driver's responsibility to use Smart Summon safely, responsibly, and as intended."
"When using Smart Summon, you must maintain a clear line of sight between you and Model Y and stay prepared to stop the vehicle at any time by releasing the button on the mobile app."
I saw someone using it on the top of the parking garage at Disney land. It just circled around but I found it fascinating to watch and am looking forward to when it can go down the ramp :)
How thoughtful of Tesla ;-) One nearly backed into me a few months back. I banged on the car to alert the driver. It stopped. There was no one in it. Owner was summoning her car in a grocery store parking lot. People.
Somehow it makes sense, you wouldn't blame a traditional car manufacturer if you crashed and the airbag shot your phone into your face. If the summon warns that you must have line of sight because it has flaws, you damn well make sure you have line of sight, or don't use it.
It is the reference to the beta feature that stands out. If the airbag was a beta feature, it would not fly. Car manufacturers should not be able to hide behind such weasel words. If dangerous functionality is not fully tested, don't release it.
Sure, so why the beta label? What is it with the functionality that makes them put in this disclaimer? Could it be that it is actually not prepared to handle loss of network connection or loss of battery in the phone? Is the phone running on a real time OS as other life critical systems usually are? What are the fail safe modes and redundanices preventing the functionality from hurting someone? With a beta label, they are basically saying that it is not ready, and the driver is at fault for using it.
Thats all great but in the end it just didn't detect an obstacle that was placed above the road. Imho this has nothing to do with smart summon. Teslas have a history of not detecting higher placed objects.
“They we’re using a beta feature honey, it’s not Tesla’s fault.”
“Ok! I’ll go out and play.”
I’m so sick of people using BETA like some magic get-out-of-responsibility free card. Once you let people drive a 3000lb car remotely you can call it pre-alpha if you want. You’re still responsible.
Is Ford responsible if someone drives a Ford blindfolded or drunk?
While it would be wise for Tesla to introduce some form of verification (e.g. push and release the button according to the flashing pattern of the interior lights), I would consider the driver significantly more responsible than Tesla.
Smart summon requires the user to hold down the button in the app, the user is responsible. The only cases where I would blame Tesla is if the car suddenly rapidly accelerated (the user would not have time to stop the collision) or if the car moved after the user stopped holding the button (the car disobeyed the user's command).
I have a coworker with an early Model 3. He gleefully demonstrated summon when he got it.
Still, it’s not a person in physical control at the drivers seat. No one is even in the car. I can think of a LOT of ways it could go badly.
I agree it’s not as bad as it could be. But that’s not good enough for me. The feature can be dangerous, and not just to people.
Putting a ‘beta’ label on isn’t enough and doesn’t absolve them. They should be held at fault too, in my opinion. Obviously the ‘driver’(?) is at fault too.
Maybe the had line of sight but since they weren't standing right behind it simply misjudged the clearance? It does stop after a bit, like if someone was watching from a distance and got surprised by the plane moving.
To avoid liability the instructions should specify "monitor at such a distance that clearances can be accurately judged". The current guidance implies it's fine if you watch from a distance.
> To avoid liability the instructions should specify "monitor at such a distance that clearances can be accurately judged".
Smart summon intentionally doesn't work at far distances by design, and exactly for the reasons you've mentioned.
The distance limitation is 20 meters within a 6 meter radius of the phone's location. There is no need for a guidance on that, as it is automatically enforced. But there is no way to automatically enforce you looking at the car, hence why the warning about keeping line of sight is there.
Imagine flying with an airline that had a beta version of the autopilot installed in its airplane. The plane crashes and you get maimed. The airline is responsible but the airplane manufacturer is in the clear. It was a beta version, after all.
Question to the experts: Is this kind of a classification or a detection problem ... or both? I remember that visual algorithms for classificaion can get confused by objects or situations they were not trained on, so in the result there were unpredictable outcomes in situations that seemed safe before. Or is it more like a "shape and size of a plane" issue?
Not colliding with immobile objects at low speed is a solved problem. Tesla's just choosing to go without radar/lidar. This inability to model the 3d world makes me very skeptical that Tesla's self-driving cars are competitive with Google's.
Honestly it's embarrassing; if a human behaved this way behind the wheel of a car, I would assume they were high on something stronger than weed.
Why high? Someone else in the thread made a remark about the insinuation of a "phone in front of my face" but in fact this could very easily happen by a sober person just simply not paying attention, and that is fairly common.
82 comments
[ 2.0 ms ] story [ 147 ms ] threadNope. Just an actual Tesla colliding with an actual jet.
What’s surprising to me is that it keeps moving post collision… seems odd that it can meet with unexpected resistance and not trigger some sort of emergency measure. (eg. stopping)
I used to be bullish about Tesla’s “only cameras” approach, and I still think they’ll crack FSD in some form, but I am increasingly wondering if there needs to be additional sensors for an abundance of extra safety around collision avoidance.
This is a requirement unless you want it getting stuck on minor bumps and whatnot.
That airplane is small enough it probably gets dragged around by hand when it's not chocked. The chocks small airports tend to use are pretty small to fit under the wheel fairings many small planes have. I can see it pushing the aircraft around by the tail.
Needs additional forward facing sensors then? (For both long and short distance object detection?)
Earlier on in Autopilot/FSD beta testing it was easier to overlook edge cases like this but I think we’re several years into this now and I’m starting to feel that at this stage it should be almost impossible for these cars to be driving into stationary objects.
I’m a proponent of what Tesla are doing but they need to solve this ASAP now, whether by hardware or software, before it erodes too much more confidence (especially as it’s all too easy for people to extrapolate from these documented edge cases and believe these vehicles to be less safe than they actually are).
Vision Jet weighs 6,000 lb at MTOW. So, unlikely.
I've moved similar planes myself.
The owners here are alpha testing a complex and dangerous system and they should expect nothing less.
This is the part of Tesla's business that upsets me. Their marketing department is being allowed to do what marketing departments do, rather than biting the bullet and letting it be known that this is a complex, dangerous, young, and known to fail system.
There are simple decisions here that should be made but weren't, e.g. "keep the product code-name until it is out of beta"; nothing on a current model Tesla deserves being called "self-driving" in any way.
My understanding is that smart summon operates on a completely different set of rules than navigate on city streets, even though they might share some common functionality, like object detection and mapping.
The marketing text for Summon literally says "Your car comes to you while you deal with a fussy child".
Disclaimer: do not use while distracted (like by a fussy child, for example) and always pay attention.
It's understandable why it may not be designed to check that situation, as it is not one you'd normally find in a parking lot (unless maybe a parking lot full of semi trailers).
Tesla is going to do more harm to the prospects of self-driving cars than anyone. Their "test in production" attitude is going to not only cost (more) lives but also force the government to strictly regulate the tech delaying the other companies that are acting responsibly.
Sounds pretty similar to the accidents involving trailers, where the top of the cars were sheared off because it didn't detect anything there.
> Neither Brown nor the car braked for a tractor-trailer, which had turned left in front of the Tesla and was crossing its path. Brown’s Tesla also went beneath the trailer and its roof was sheared off. After that crash Tesla CEO Elon Musk said the company made changes in its system so radar would play more of a role in detecting objects.
They got rid of radar, so I wonder if that's made it again more difficult to detect these scenarios.
It seems like nothing irreplaceable (like human life or health) was lost or even at risk in this case.
Oh, whoops, our FSD car hit a deer. The driver should've been prepared to stop the car.
Well of course, but when a Volvo with last generation technology can safely stop the car, that's not an excuse.
I don't think you can take this autopilot, and FSD (Fools Self Driving) contraption seriously or remotely ready for customers to use; even for a 'beta'. Since it's functionality doesn't even work as advertised, puts the lives of other drivers on the road at risk and it is completely useless at the most dangerous time to drive, which is at night.
For example, as a simple test against another autonomous vehicle with LIDAR at night. [0] The Tesla with FSD failed to stop and just ran over the object.
The best part is 100,000 crash dummies have this software installed and seem to be taking the risk in trying it out, even when it malfunctions on the public roads and what we can see here. This contraption needs to be thoroughly investigated.
[0] https://twitter.com/TaylorOgan/status/1478802681141645322
I thought that was the worst part of it too, assuming theres nobody inside the car at the wheel. It keeps going, stops, then moves again and looks to go around whatever obstacle it thinks exists. What if this situation can be replicated on an empty road where the tesla runs over a human being? In the worst case, does the tesla keep trying to drive over the human being and turns a minor injury into a fatality?
a) the full sensor stream if possible, but if not, then at least the candidate software could process the sensor stream in real time and output a condensed state and save that
b) the human driver's actual control inputs
Collect these and analyze when the software's suggested inputs deviate significantly from the human's input. Some of the time, that'll be computer reacted to things faster, or human appears intoxicated, but based on real world reports, there's got to be a large space of human driver did the right thing and computer ignored important parts of the scene.
I also think there's some evidence that Tesla has the approach of continue driving unless computer can determine it's unsafe to continue, whereas Alphabet/Google/Waymo has the approach of only drive when computer can determine it's safe, which is way more frustrating to drive nearby, but leads to better outcomes. With Tesla, we usually have reports of collisions at the intended driving speed (which is slow for summon, so great), whereas Waymo collisions are usually at slow speeds often as a result of traffic hitting the Waymo vehicle which was stopped in an unexpected way.
Remember, we have to move fast and break t̶h̶i̶n̶g̶s̶ people, literally!
https://twitter.com/PatrickMoorhead/status/14785490274780651...
https://twitter.com/PatrickMoorhead/status/14787645152609116...
https://twitter.com/PatrickMoorhead/status/14787661657237790...
The Uber software chose to take no action, the Uber employee at the wheel was apparently paying no attention, and Uber had disabled the Volvo software, the only agent involved that actually thought it would be a good idea to pay attention and try not to kill a woman with a bike on a dark night. Uber shouldn't be allowed to test their software on public roads in my opinion.
The aircraft owner and insurer could sue Tesla, but likely Tesla's terms of use require the car only be on public roads and private driveways.
"When using Smart Summon, you must maintain a clear line of sight between you and Model Y and stay prepared to stop the vehicle at any time by releasing the button on the mobile app."
https://www.tesla.com/ownersmanual/modely/is_is/GUID-6B9A1AE...
Presumably the driver is insured, right? And insurance covers the his nonsense...
This makes reckless driving easier, but it should still be considered (primarily) the fault of the driver.
"The airbag is in beta! It may not work at night, so be sure to turn it off or it may kill you."
“They we’re using a beta feature honey, it’s not Tesla’s fault.”
“Ok! I’ll go out and play.”
I’m so sick of people using BETA like some magic get-out-of-responsibility free card. Once you let people drive a 3000lb car remotely you can call it pre-alpha if you want. You’re still responsible.
Might as well call not-it.
While it would be wise for Tesla to introduce some form of verification (e.g. push and release the button according to the flashing pattern of the interior lights), I would consider the driver significantly more responsible than Tesla.
Still, it’s not a person in physical control at the drivers seat. No one is even in the car. I can think of a LOT of ways it could go badly.
I agree it’s not as bad as it could be. But that’s not good enough for me. The feature can be dangerous, and not just to people.
Putting a ‘beta’ label on isn’t enough and doesn’t absolve them. They should be held at fault too, in my opinion. Obviously the ‘driver’(?) is at fault too.
To avoid liability the instructions should specify "monitor at such a distance that clearances can be accurately judged". The current guidance implies it's fine if you watch from a distance.
Smart summon intentionally doesn't work at far distances by design, and exactly for the reasons you've mentioned.
The distance limitation is 20 meters within a 6 meter radius of the phone's location. There is no need for a guidance on that, as it is automatically enforced. But there is no way to automatically enforce you looking at the car, hence why the warning about keeping line of sight is there.
Honestly it's embarrassing; if a human behaved this way behind the wheel of a car, I would assume they were high on something stronger than weed.