I'm curious why governments don't have a driving test for "self-driving" AI's just like they have a test for 16 yr old teenagers claiming they are ready to drive
They do. It's far stricter and requires far more hours on the road with an instructor and every time the machine fails it is reported unlike with a 16 yo.
Tesla's FSD beta should be thought of like you are a driving instructor/parent for a beginning driver on a driving permit but you have a full duplicate set of controls, like an airplane. Let the car drive but at any moment you have to be ready to take control or something bad might happen. We have millions of people training other people to drive every day and they can't even take over right away when there is a problem. People running FSD beta are training Tesla's driving AI. They should take the task very seriously, but I think its great they are allowed to do so. If it ever works well it will be great for safety and convenience.
Once FSD seems ready for level 4 or 5 automation, then I'm sure governments will have Tesla prove all sorts of things about how it functions and how good/safe it is before it is approved.
Driving instructors are licensed, paid, and liable; driving instructors do not pay ten thousand dollars for the opportunity to oversee driving learners.
If their customers are actually driving instructors for their cars then Tesla should be employing them, verifying they are competent at the task, and paying them the prevailing rate of ~$25 per hour. They should not be advertising it as a product that is fit for purpose when it requires constant supervision by trained professionals.
In the US we are a bit more liberal with driving training. Any parent with a drivers license can train their children to drive. Schools have driving classes but you are not always required to take them.
>We have millions of people training other people to drive every day and they can't even take over right away when there is a problem.
We don't. Over here driving school is several months of training by licensed drivers in cars with dual control so the teacher has full control over the car in case of an emergency. the idea that your Uncle Bob teaches you how to drive would probably be considered insane. The traffic related death rate here is also like 4x lower than in the US. This kind of stuff isn't normal.
Having computers that make video games from the 90s look bug free drive two tons of steel around at lethal velocities is bonkers. Public roads should not be playgrounds for 'beta tests'.
In the UK almost everyone uses trained instructors, although provided you have an L plate on your car anyone with over 5 years experience driving can take you out for practice. Most don't learn that way however, because the test is difficult enough that a good driving instructor is pretty much mandatory. The pass rate was 46.4% in 2019/20.
The difference? There is absolutely no improvement.
Expecting that the FSD (Fools Self Driving) beta software would get better over time as promised by Elon, this video shows that it doesn't look good at all with this latest failure which drivers on the road, and even pedestrians were no better off and are no safer with these updates.
It is not early days anymore and even with these updates, it is still an unsafe contraption which not only has been falsely and deceptively advertised by Tesla, it is not even near the claims of the robo-taxi level of FSD (Level 4 - 5) with the number of price increases over the years with a product which doesn't work looks more like a scam.
At this point, FSD is essentially Fools Self Driving.
FSD needs to be substantially better than the average human driver - not on par or worse (which this is).
Nobody is going to tolerate FSD vehicles bugging out and randomly driving into parked cars or pedestrians, killing people in the process.
You can tout millions of miles all day ever day, but it doesn't matter. When these vehicles are put into even semi-tricky situations, they fail far too often. Tesla's software doesn't even require tricky situations... it'll just randomly drive you into a wall or fire truck without warning.
Why? Human drivers do all of those things. Why should I care if I'm killed by a computer or a human? At least if I'm killed by a computer, the data will go into the training set and make future computers safer. But human drivers will learn nothing.
Despite the narrative being spun by FSD companies like Tesla and the rest - human drivers are actually really good at navigating complex, dynamic situations.
These systems however, are not.
I think most folks would rather not be killed because Elon's ego decided a decade ago that Tesla would never consider Lidar sensors, for example, despite the mounting evidence simple cameras are not sufficient.
Recall the Toyota stuck accelerator pedal issue years back? There are people who still refuse to own a Toyota.
What about the Boeing 737-MAX fiasco? An entirely software related bug caused a statistically trivial and insignificant amount of people to lose their lives - yet today there are still people who will never fly on a Boeing aircraft again, despite the issue being resolved.
People will not tolerate loss of life from these systems. They must be substantially better than the average human for the public to be interested.
> human drivers are actually really good at navigating complex situations. These systems however, are not.
This determination needs to be made with data. As far as I can tell there is no conclusive evidence that Teslas on autopilot are involved in more crashes per mile than human drivers, and some evidence that it is safer, though that comes from Tesla, so not giving that much weight. So either it's somewhere close to as safe as human drivers, or data collection in this area is just really bad. I don't know which.
The evidence provided by the likes of Tesla does not compare to real-world driving. They cannot produce that evidence yet, despite however many millions of miles on test circuits and well-known roads they want to tout - it's a false statistic designed to deliberately mislead the public into thinking the software is better than it really is.
The burden of proof is on the FSD companies, not average drivers. The likes of Tesla have to prove their software is far superior to human drivers - and so far they have failed to do so.
There are far too many FSD beta videos out recorded by real users that demonstrate obvious safety issues. The software is just not there yet, and may not be there for a long time, if ever.
Articles like this come out far too often to just be discarded as media "hit" pieces.
> FSD needs to be substantially better than the average human driver - not on par or worse (which this is).
The states of the average driver also include drunk and drugged drivers. Texting drivers. It need to be way better than the average driver until a driver that does not do those things gets a safety benefit.
I would expect Tesla FSD to behave worse than humans when there is a vehicle malfunction too.
Are all driving miles equivalent? Do you think accidents that occur with FSD enabled are the same as where they most often occur under normal human control? What proportion of miles are driven in sunny US states?
Is this an argument supported by data? What about a sound experimental design?
Perhaps we could look to medicine (instead of automotive engineering) for guidance regarding which reductions in stratified accident incidence rate are significant and warrant continued investment in sensor fusion and assistive AI?
Its not that it tries but fails, it doesnt even attempt to. Same goes for animals (deer running across etc) or debris on the road. FSD is only programmed to detect other cars and road markings. It doesnt even detect physical road boundaries and will happily drive into the ditch/concrete pillar if road markings tell it to.
that's an obvious lie. FSD beta that he tried detect people very good. It even detect a motion. When a human move a bit towards a car, even there is no direct intersection the car stops. I know that since I drive tesla through manhattan. It stops in every possible case, even the one I will never treat as a problem.
Not sure what kind of tricks this guy made.
> In several tests, a professional test driver found that the software – released in June – failed to detect the child-sized figure at an average speed of 25mph.
The car was 40mph when put into self drive mode, so the car was slowing itself down even though the mannequin, which looks like a cone to my eyes, wasn't moving.
Are these people legit but counterproductive to their claimed goals or entirely bogus?
Ah, prominent mentions of LIDAR suggests it's just an advert disguised as a consumer safety campaign.
> We Demand Software that Never Fails and Can’t Be Hacked
Yeah, pretty much. I'm sure they could find a way to cause similar systems from other manufacturers to fail. None of the deployed systems are actually perfect in the "never fails" sense.
I wish Tesla would have focused more on using the CV technology to strengthen its collision avoidance and safety systems which is lacking. There was a lot of emphasis on marketing the FSD but I think FSD would have been a byproduct of a robust and reliable collision avoidance system. I don't care about FSD -- I care about getting my family to where we are going safely! Tesla should warn me if I am about to run a stop light or stop sign. It would also be nice to have an infant protection system that detects children locked in vehicles and takes action to protect them from overheating -- which seems plausible based on the hardware inside my 2021 Model Y.
Cabin overheat protection exists, but it "only" keeps the vehicle below 105, and I don't think it automatically does anything based upon the detection of passengers.
Cabin overheat protection was apparently created to mitigate issues with the original LCD screens yellowing and failing because they were not rated for the heat that is experienced in a hot car on a summer day. Keeping the car at a temperature that won't kill a forgotten child was a convenient side effect. Also, the screens are properly specced now.
There is a "keep car cool for pets" mode that also works for children. Has a helpful message on the screen so passing do-gooders don't have to worry.
An automated system to prevent harm caused through parental negligence would be interesting, but how would you even detect that a baby was present? Maybe some kind of 'have' token attached to the carrier or car seat? OTOH that could cause more harm via accidental strandings. Would be interesting to search for an elegant solution.
My car seems to be able to tell there's a child seat in the back and the screen beeps and reminds me to check the rear seat for passengers when I turn off the car.
I haven't really fully thought it out. But you could maybe use the microphone to detect crying (if the baby is conscious). There is an internal camera that might help as well to detect movement but may be a bit more difficult with rear facing car seats. I wonder if the weight sensor that triggers the seat belt warnings is sensitive enough to detect a baseline weight of the car seat and the weight of a child.
I'm pretty sure Tesla had a patent or something that was some kind of sensor, not sure what exactly, to detect living beings in the car, but I don't remember the details.
Pure madness. Can we stop worshiping software? If these selfish assholes were using their own private infrastructure I would have nothing against it, but they are imposing themselves on the innocent public.
The reason it has taken the traditional car companies so long to catch up is because they are treating software development the same way as designing cars; whereas Tesla apparently treats FSD with the same attitude as those developing web pages.
I'm surprised they haven't coded in an easter egg where the screen displays an animation of Wile E. Coyote being squished into a pancake when it strikes an object.
Can't wait for Elon to tweet out "will be fixed in FSD Beta x.y!"
Read a comment here about someone who worked for a component manufacturer on firmware. Their test suites took around 30-36 hours to complete.
They sent some fixes to Tesla and advised them.
A few hours later, "Thanks so much for this, everything is working all great now."
"??"
"We flashed it onto a car and took it for a short drive, and couldn't see any more issues".
Same with (one of) the phantom braking issues. Yeeted a fix over the fence in a day or so, which "fixed" some of them, but introduced different ones. Yeeted another fix a day or two later that fixed some of the different ones but re-introduced some of the original ones.
Anyone who can tell you with a straight face that that is a safety-first culture is lying.
But actually it hasn't taken the traditional car companies long to catch up. Independent reviewers have generally rated GM Super Cruise as superior to Tesla Autopilot in most ways.
Generally they haven't rated super cruise higher at the actual task of driving. They've given it the nod for other things (eye tracking, hands free, etc).
I haven't seen anyone try a test. It seems to be limited to the kind of roads where they'd be visible well in advance. My guess is that it will shut itself off well in advance as a result of seeing an anomaly.
"It made it 8.8 miles before abruptly yanking the wheel left, toward the median, and canceling."
Nothing about this would keep me from using it. I'm sure its a great system, but like all of them, it requires attention. If you don't intervene (as it expects you to) accidents can happen.
It failed to detect a stationary mannequin in one specific scenario. Does this generalize? Does this mean there's a major safety problem? These would be assumptions that this test can not prove.
It always ultimately comes back to safety. How many times has AEB avoided a pedestrian that was not detected by the driver of the vehicle? Is that greater or smaller than the number that were hit by drivers with a false sense of confidence in the software?
The Dawn Project test cannot answer such questions.
I believe that NHTSA is working on this and related questions across the industry and I expect them to produce a more useful result on their own than what can be produced by single-minded reports like this.
No, it's not a purely single variable optimisation. If this system (and let's be honest, we're just talking about Tesla here) cannot avoid hitting an object that every driver should easily see, then it's not fit for purpose. The reason being that people take action based on certain assumptions about what drivers will do. If a pedestrian steps into a road to cross when an oncoming car had plenty of space to both see and slow, then it's not ever acceptable to hit the pedestrian. This is a normal urban interaction.
It couldn't detect a stationary child sized mannequin from over a 100 yards away on a wide open road completely clear of distractions in 3 out of 3 tries.
I'm struggling to see how any generalization would not be a much worse scenario.
Cars don't have a stopping distance of 100 yards until they are going highway speeds. So that doesn't seem like a safety issue as children standing still in the middle of the interstate isn't really something that happens.
Similar testing of Tesla on this channel https://youtu.be/p7lp5f0aqzU?t=74 shows it failing badly on simple hazard object scenarios like a chair, wooden pallet, trash can etc.
Also when the car does give an alert, it is too late for a human to react.
That guy is an obvious scam. He ordered ads after that research everywhere possible trying to push his tech. And that "research" is buzzing around artificially for several month. I can see tesla is rather prone to false positives.
My FSD Beta on intersection detected a 10 inch marble statue of a kid as a pedestrian.
There is a fundamental difference between Tesla's approach vs. Waymo and Cruise. Waymo and Cruise are starting at the top. They use expensive, bulky, unsightly hardware and only allow their self-driving cars to operate in limited geographic areas. This has a much higher probability of success, and once the tech works, they expand its capabilities. Eventually their tech will become mass-market. They don't kill anyone in the process, and they get very little bad press.
Tesla on the other hand is trying to make a mass-market version work right off the bat, even if it means debugging on the streets. Tesla will get a lot of bad press, will kill a few people, and the public will become sceptic about the tech. I think Elon Musk's approach is not just wrong, it is going to make it harder for Waymo and Cruise by making consumers distrust self-driving cars in general.
Here are things wrong with this video and the group (person) that made it. While I'm not a fan of Elon, I am fan of actually verifying information and placing criticism where criticism is due.
* This video was made by the The Dawn Project, whose goal is literally to make Tesla's self driving illegal. Their current goal isn't to make all unsafe self driving illegal, only Tesla's.
* The Dawn Project is founded and operated by Dan O'Dowd, who is campaigning for senate. His biggest platform issue is banning Tesla's self driving. He's definitely getting the press coverage he wants as a result of this publicly advertised test.
* Dan O'Dowd is the founder and owner of Green Hills Software, which makes self driving software for car manufacturers, and has more than a dozen partners with deep connections to the automotive industry. Dan does not disclose this conflict of interest.
* Green Hills Software has Ford and Toyota as direct customers. Dan does not disclose this conflict of interest.
* The Dawn Project explicitly outlined this test as "a small child walking across the road in a crosswalk" and it fails in both of these goals - the "child" isn't walking and the road isn't marked as a crosswalk.
* There is zero coverage of trials where Tesla did successfully brake. The test circumstances are clearly setup to make it fail. While noteworthy they were able to find the right conditions, not disclosing the work that went into making the test scenario only further fuels the bias of this test.
* It is unclear to me how they managed to get a Tesla to work in full self driving mode while plowing through clearly marked parking spaces in a parking lot. These are clearly not conditions that anyone would be using FSD. This is further marked by the fact that the Tesla braked hard from 40MPH to 25MPH as soon as FSD was enabled because it knew going that speed in a parking lot and driving over road markings it shouldn't was a stupid thing to be doing.
* FSD was enabled only seconds before being introduced to the stationary mannequin.
TLDR: There is a massive conflict of interest here that isn't disclosed anywhere. There is a massive incentive to use this as advertising for his senate campaign. The test doesn't actually put in any baseline effort to replicate what they claim to be testing. They put the Tesla in a situation that no one would ever actually be using FSD.
The article points this out, yes, but Dan and his organization do not. By "anywhere" I meant on his website, in the documentation around the test, or in any of his organizations communication in advertising this test. It's clearly not included in the million social media posts around this, and 99% of people seeing this video won't know this.
76 comments
[ 4.2 ms ] story [ 153 ms ] threadOnce FSD seems ready for level 4 or 5 automation, then I'm sure governments will have Tesla prove all sorts of things about how it functions and how good/safe it is before it is approved.
People learning to drive are not equivalent to a closed source driving AI.
If their customers are actually driving instructors for their cars then Tesla should be employing them, verifying they are competent at the task, and paying them the prevailing rate of ~$25 per hour. They should not be advertising it as a product that is fit for purpose when it requires constant supervision by trained professionals.
We don't. Over here driving school is several months of training by licensed drivers in cars with dual control so the teacher has full control over the car in case of an emergency. the idea that your Uncle Bob teaches you how to drive would probably be considered insane. The traffic related death rate here is also like 4x lower than in the US. This kind of stuff isn't normal.
Having computers that make video games from the 90s look bug free drive two tons of steel around at lethal velocities is bonkers. Public roads should not be playgrounds for 'beta tests'.
https://twitter.com/TaylorOgan/status/1478802681141645322 (This is January 5th 2022)
Same author, same test, months later:
https://twitter.com/TaylorOgan/status/1556991029814886404 (This is August 9th 2022)
The difference? There is absolutely no improvement.
Expecting that the FSD (Fools Self Driving) beta software would get better over time as promised by Elon, this video shows that it doesn't look good at all with this latest failure which drivers on the road, and even pedestrians were no better off and are no safer with these updates.
It is not early days anymore and even with these updates, it is still an unsafe contraption which not only has been falsely and deceptively advertised by Tesla, it is not even near the claims of the robo-taxi level of FSD (Level 4 - 5) with the number of price increases over the years with a product which doesn't work looks more like a scam.
At this point, FSD is essentially Fools Self Driving.
Which competing EVs do you like and why?
Nobody is going to tolerate FSD vehicles bugging out and randomly driving into parked cars or pedestrians, killing people in the process.
You can tout millions of miles all day ever day, but it doesn't matter. When these vehicles are put into even semi-tricky situations, they fail far too often. Tesla's software doesn't even require tricky situations... it'll just randomly drive you into a wall or fire truck without warning.
These systems however, are not.
I think most folks would rather not be killed because Elon's ego decided a decade ago that Tesla would never consider Lidar sensors, for example, despite the mounting evidence simple cameras are not sufficient.
Recall the Toyota stuck accelerator pedal issue years back? There are people who still refuse to own a Toyota.
What about the Boeing 737-MAX fiasco? An entirely software related bug caused a statistically trivial and insignificant amount of people to lose their lives - yet today there are still people who will never fly on a Boeing aircraft again, despite the issue being resolved.
People will not tolerate loss of life from these systems. They must be substantially better than the average human for the public to be interested.
This determination needs to be made with data. As far as I can tell there is no conclusive evidence that Teslas on autopilot are involved in more crashes per mile than human drivers, and some evidence that it is safer, though that comes from Tesla, so not giving that much weight. So either it's somewhere close to as safe as human drivers, or data collection in this area is just really bad. I don't know which.
The burden of proof is on the FSD companies, not average drivers. The likes of Tesla have to prove their software is far superior to human drivers - and so far they have failed to do so.
There are far too many FSD beta videos out recorded by real users that demonstrate obvious safety issues. The software is just not there yet, and may not be there for a long time, if ever.
Articles like this come out far too often to just be discarded as media "hit" pieces.
Boeing fixed their management/culture problems?
The states of the average driver also include drunk and drugged drivers. Texting drivers. It need to be way better than the average driver until a driver that does not do those things gets a safety benefit.
I would expect Tesla FSD to behave worse than humans when there is a vehicle malfunction too.
Are all driving miles equivalent? Do you think accidents that occur with FSD enabled are the same as where they most often occur under normal human control? What proportion of miles are driven in sunny US states?
Which metric is proposed?
Is this an argument supported by data? What about a sound experimental design?
Perhaps we could look to medicine (instead of automotive engineering) for guidance regarding which reductions in stratified accident incidence rate are significant and warrant continued investment in sensor fusion and assistive AI?
> In several tests, a professional test driver found that the software – released in June – failed to detect the child-sized figure at an average speed of 25mph.
The car was 40mph when put into self drive mode, so the car was slowing itself down even though the mannequin, which looks like a cone to my eyes, wasn't moving.
Are these people legit but counterproductive to their claimed goals or entirely bogus?
Ah, prominent mentions of LIDAR suggests it's just an advert disguised as a consumer safety campaign.
> We Demand Software that Never Fails and Can’t Be Hacked
Or possibly they're just weird?
Yeah, pretty much. I'm sure they could find a way to cause similar systems from other manufacturers to fail. None of the deployed systems are actually perfect in the "never fails" sense.
I thought Tesla announced something like that a while ago, was it just loose talk on Twitter by Elon?
An automated system to prevent harm caused through parental negligence would be interesting, but how would you even detect that a baby was present? Maybe some kind of 'have' token attached to the carrier or car seat? OTOH that could cause more harm via accidental strandings. Would be interesting to search for an elegant solution.
I'm surprised they haven't coded in an easter egg where the screen displays an animation of Wile E. Coyote being squished into a pancake when it strikes an object.
Can't wait for Elon to tweet out "will be fixed in FSD Beta x.y!"
sigh...
They sent some fixes to Tesla and advised them.
A few hours later, "Thanks so much for this, everything is working all great now."
"??"
"We flashed it onto a car and took it for a short drive, and couldn't see any more issues".
Same with (one of) the phantom braking issues. Yeeted a fix over the fence in a day or so, which "fixed" some of them, but introduced different ones. Yeeted another fix a day or two later that fixed some of the different ones but re-introduced some of the original ones.
Anyone who can tell you with a straight face that that is a safety-first culture is lying.
But its hard to say, since noone seems to have tested that publicly. Here's an article that gives a fairly balanced look at some of its other failure modes, though: https://www.caranddriver.com/features/columns/a36118292/supe...
"It made it 8.8 miles before abruptly yanking the wheel left, toward the median, and canceling."
Nothing about this would keep me from using it. I'm sure its a great system, but like all of them, it requires attention. If you don't intervene (as it expects you to) accidents can happen.
It failed to detect a stationary mannequin in one specific scenario. Does this generalize? Does this mean there's a major safety problem? These would be assumptions that this test can not prove.
A comprehensive safety analysis has to also include these scenarios: https://youtu.be/hx7BXih7zx8?t=205
I can't fugure out the "even slow down" claim when the car goes from 40 to 25mph by itself?
But it did not slow down anywhere near the mannequin, and even sped up a little bit when hitting the mannequin in 1 trial.
The Dawn Project test cannot answer such questions.
I believe that NHTSA is working on this and related questions across the industry and I expect them to produce a more useful result on their own than what can be produced by single-minded reports like this.
I'm struggling to see how any generalization would not be a much worse scenario.
No. But there could be protesters blocking the highway. As annoying as they are, running them over is not an option.
Tesla on the other hand is trying to make a mass-market version work right off the bat, even if it means debugging on the streets. Tesla will get a lot of bad press, will kill a few people, and the public will become sceptic about the tech. I think Elon Musk's approach is not just wrong, it is going to make it harder for Waymo and Cruise by making consumers distrust self-driving cars in general.
* This video was made by the The Dawn Project, whose goal is literally to make Tesla's self driving illegal. Their current goal isn't to make all unsafe self driving illegal, only Tesla's.
* The Dawn Project is founded and operated by Dan O'Dowd, who is campaigning for senate. His biggest platform issue is banning Tesla's self driving. He's definitely getting the press coverage he wants as a result of this publicly advertised test.
* Dan O'Dowd is the founder and owner of Green Hills Software, which makes self driving software for car manufacturers, and has more than a dozen partners with deep connections to the automotive industry. Dan does not disclose this conflict of interest.
* Green Hills Software has Ford and Toyota as direct customers. Dan does not disclose this conflict of interest.
* The Dawn Project explicitly outlined this test as "a small child walking across the road in a crosswalk" and it fails in both of these goals - the "child" isn't walking and the road isn't marked as a crosswalk.
* There is zero coverage of trials where Tesla did successfully brake. The test circumstances are clearly setup to make it fail. While noteworthy they were able to find the right conditions, not disclosing the work that went into making the test scenario only further fuels the bias of this test.
* It is unclear to me how they managed to get a Tesla to work in full self driving mode while plowing through clearly marked parking spaces in a parking lot. These are clearly not conditions that anyone would be using FSD. This is further marked by the fact that the Tesla braked hard from 40MPH to 25MPH as soon as FSD was enabled because it knew going that speed in a parking lot and driving over road markings it shouldn't was a stupid thing to be doing.
* FSD was enabled only seconds before being introduced to the stationary mannequin.
TLDR: There is a massive conflict of interest here that isn't disclosed anywhere. There is a massive incentive to use this as advertising for his senate campaign. The test doesn't actually put in any baseline effort to replicate what they claim to be testing. They put the Tesla in a situation that no one would ever actually be using FSD.
They said FSD failed but it wasn't even on.
He also used to go on about the "linux threat" of people using linux rather than his software https://www.eetimes.com/odowd-then-and-now-on-linux/