The driver told the car to maintain a speed and it did, there’s nothing particularly “autopilot-centric” about this case; any other cruise control (adaptive or not) would operate the same way.
I'm confused - what allows the victims to sue Tesla directly?
Legally the driver bears the responsibility, no? I can see driver suing tesla for a malfunction.
Can my neighbor below sue my washer manufacturer if they got flooded?
I thought the law is very specific in this case. The responsibility is driver's except in cases when it explicitly isn't - which definitely wasn't this case as no vehicle in EU had the necessary approval when it happened, and I'm not sure, but maybe there wasn't any law whatsoever about autonomous vehicles and transfer of responsibility back then - which makes the driver unquestionably responsible.
Edit: I thought this is in Germany, so I'm wrong. My bad.
The U.S. is a common law jurisdiction, which (among other things) means that when there is a novel factor involved, usually the courts are first to decide how much it matters. Autopilot was a very novel factor at this time, so the courts need to decide if it matters enough to reduce the criminal penalties. In the absence of an explicit law, that's just how it works. The driver doesn't become responsible -- responsibility becomes an open question.
In civil law jurisdictions, this works differently.
The families of the Sandy Hook school shooting victims sued Remington and settled for $73 million, even though Remington didn't directly sell the gun to the shooter, or violate any other laws regulating the sale of firearms:
The NYT describes the plaintiff's argument as focused on Remington's marketing strategy:
> The families contended that Remington violated state law by promoting the weapon with an approach that appealed to so-called couch commandos and troubled young men like the gunman who committed the Sandy Hook massacre.
> Nicole Hockley, whose 6-year-old son Dylan was killed, said the documents included in the settlement were crucial — and “paint a picture of a company that lost its way, choosing more aggressive marketing campaigns for profit.”
Not in Germany. In Germany, there needs to be direct damage by the suing party done to the suing party; and criminal cases are entirely public prosecutor-driven.
To sue someone, you need to have incurred damages of some sort and file a complaint alleging that who you're suing is responsible. At a high level, you can sue anyone for anything.
Legally the driver bears the responsibility, no?
This is a really vague question. Bears the responsibility for what, exactly? Are they solely responsible? These are things that exactly get sorted out in court...
Can my neighbor below sue my washer manufacturer if they got flooded?
The marketing of "autopilot" may end up being central to this case. It is indeed just adaptive cruise control, which is also present on many other cars, but the way it is advertised suggests otherwise. Additionally, Tesla has been selling people "Full Self-Driving Capability" when self driving beyond cruise control hasn't been figured out yet.
This is false advertising, and apparently it may be involved in turning Tesla drivers who believe it into negligent killers. The driver may be able to argue that Tesla told him the car would stop, and so there is no negligence on his part.
If I program my Bendix-King autopilot to fly heading 270º at 4K feet and there's a 5K foot mountain directly west of me, the autopilot will do exactly what I told it to do and I will make a smudge on the side of the mountain.
Should it be though? Obviously Joey Beercan and his soccermom wife with 4.5 kids are not ever going to do a pilot's level pre-flight walk around inspection of the s/aircraft/Chevy Tahoe/ to check that the tires are inflated and have proper tread. Nor would they ever spend the time to look at the indicator lights to ensure the equipment is operating correctly. No amount of tapping on the Check Engine light will ever make that indicator change value.
However, we've made getting licensed to operate a vehicle way too easy. We should also make drivers get training on equipment type so switching from a mid-sized sedan to a gianormous SUV requires more than just being handed new keys.
I don’t think vehicle licensing should be easy, and should fairly strict as I understand it to be in Germany.
The extent of training increases the investment a person has around the regulated activity. The person is more likely to abide by the rules because (a) they have a deeper knowledge, and (b) they don’t want to lose the privilege of being able to carry it out.
Increasing the strictness of license requirements could be a small nudge towards decarbonization to slow the growth of personal vehicles on the road.
>Increasing the strictness of license requirements...
Sadly, I'd have to disagree. I think all this would do would be to increase the number of drivers operating illegally. Increase the friction, and people will just find a way to avoid it all together. The only real way to stop it would to have mandatory checkpoints. The push back from the masses for any kind of change like this would be enormous that you'd be asking Roe v Who? in comparison.
Should everyone just have to take a high-end defensive driving course? Sure. Why not? I can afford it. You can't? What a pity.
But honestly I think demands to make a driver's license harder to get largely miss the point. From my own experience, time on the road makes far more difference than training or harder tests. As does, general maturity and attitude which isn't going to be taught by a class.
>Should everyone just have to take a high-end defensive driving course? Sure. Why not? I can afford it. You can't? What a pity.
What does this have to do with anything? Advocating that people that can't afford to take training shouldn't be restricted from getting licensed? With that logic, let's all become pilots for free too!
>time on the road
The rest of this I can almost agree with, but you're glossing over that bad habits become entreanched. Distracted driving from today's smartphone use to the classic putting on makeup while driving, constant changing of the radio/spotify/etc, or just shoving fries/chips down your gullet while driving is never going away no matter how many years you've drive. In fact, the longer one goes with out having a wreck with these bad habits, the more one is emboldened that they are safe.
It's not just teenagers with low hours behind the wheel that do any of the things listed above.
I'm not sure what training regime gets people to not do those things however. I assume US driver ed courses still include obligatory horrific videos of crashes resulting from people doing stupid things.
Like most training, I think the biggest thing is the learning before the official training that is avaialable. Mainly, I'm talking about a kid learning from watching their parent's behavior behind the wheel. If kiddo sees the Parental Training Unit(TM) doing stupid things behind the wheel, then kiddo will assume that's okay to do. Then when being told that's not the correct thing to be doing in official training courses, kiddo=>teenager now rolls their eyes to the back of their head as the teacher is just out of touch.
So, you must be the change you wish to see in the world. So in today's view, we're all screwed!!
Oh I agree with that as well. If kiddo sees parents driving drunk all the time, they'll dutifully say you shouldn't do that on the driver ed test then do it anyway. (And it's really broader than parents as there are broader social mores as well.)
I think we're in general agreement. My original point was that making getting a driver's license a leetcode interview-level requirement puts more barriers in place but probably doesn't (by itself) mean you have a lot better drivers on the roads.
To be clear, if you can afford it, a high-end driving course is probably both useful and a lot of fun. Though I wonder if it's more useful for someone who has some experience. (I've never take a course though I have done some driving on BMW's track in SC which was a great experience.)
I think we need some sort of Homer Simpson level whackiness that provides a gentle shock when your hands leave the wheel for a predetermined time, force pupil tracking to provide a gentle shock when no longer watching the road or checking mirrors for predetermined time. The shock would have to come from different places so you can't learn to avoid where it comes from when you want to deviate from task at hand.
There was a post here a while back where someone built a keyboard that would deliver shocks when it detected improper typing techniques. It wasn't enough to do anything other than make you really want to type well. It was funny that they went through with it, but that's the level of schtick I'm intending by gentle shock.
If you increase the burden of getting a license you don't simultaneously invalidate all existing licenses at once obviously you focus on new licenses and renewals over many years.
It is impossible to get insurance, plates, tabs, park your car in your apartments lot without proper licensing.
Everyone eventually messes up and gets pulled over or gets in an accident and then you are basically going to jail.
In theory your cousin can buy and register your car but if you kill someone he will be legally and monetarily responsible.
Logically it would have to be fantastically burdensome to convince many people to drive without licenses.
It is silly to both pretend we absolutely must impliment a law either without the slightest bit of sense or not at all and presuppose that everyone is going to ignore the law when clearly this is actually fairly hard to do.
> Logically it would have to be fantastically burdensome to convince many people to drive without licenses.
Based on experience (admittedly biased), plenty of people are driving without a valid license or insurance right now.
That’s not an argument not to increase proficiency standards, but I think people who naturally follow rules (like my spouse) have a hard time comprehending people who would see a rule and then evaluate whether or not to follow it.
I think you and I are experiencing different realities. I know first hand it is easy to drive without a valid license whether that means not having one at all, having one but currently it is suspended, or any other various ways. I also know direct* 2nd hand of how easy it is to have a car that does not have proper official paperwork/stickers/plates.
If you are operating a car in someone else's name does not mean they are ultimately on the hook. Yes, they will probably be main suspect, but they will/could be able to prove not involved. Easiest would be to say you were using the car without permission.
>Logically it would have to be fantastically burdensome to convince many people to drive without licenses.
WAT! this happens every. single. day.
*I know direct 2nd hand is counter-intuitive, but I mean I personally know people with cars like this.
You can't register a car without a drivers license and you cannot practically enter into a financial arrangement wherein the car is the collateral without such with most lenders. You cannot purchase car insurance.
You know that's not how they sell autopilot. They sell it as something that is more intelligent than dumb cruise control, and they sell it as the safest driver on the road. They don't sell it as "equivalent to airplane autopilot."
I believe there was a study that looked only at highway miles driven (where autopilot is mostly useful), and the advantage of Tesla's autopilot vanishes compared to a human.
It turns out it's a lot easier to not crash on a highway than to do it on surface roads with many stops and starts.
And note that this is the current marketing material.
Edit: It's not mentioned in the article, as far as I see, but it's likely the driver was on the cell phone. It mentions the data shows the driver had their hands on the steering wheel, but that can be easily thwarted, to my understanding. The fact that they didn't brake at all implies they weren't actually holding the steering wheel, at least in a way where they were paying attention. In my opinion, both the driver and Tesla bear responsibility.
I'll observe that that page contains the following text (inline and in the same font and size as the rest of the body text, not buried in a footnote somewhere):
Current Autopilot features require active driver supervision and do not make the vehicle autonomous.
If you look at the page from the time when the incident occurred[1], there is no such text anywhere on that page. It's almost as if they added the text in response to the threat of lawsuits. The only disclaimer is:
> Please note that Self-Driving functionality is dependent upon extensive software validation and regulatory approval, which may vary widely by jurisdiction.
And the rest is marketing copy explicitly written to make it sound more fully-featured than it actually is.
But that copy explicitly notes abouy the self driving features:
> It is not possible to know exactly when each element of the functionality described above will be available, as this is highly dependent on local regulatory approval.
So while you can argue that the marketing is deceptive, that can't really be read as saying that you could let the car drive itself at that time.
>I'll observe that that page contains the following text
Well I suppose it'll be up to the judge to determine if a relatively recently added disclaimer on the website trumps years worth of bold claims and tech demonstrations.
Please don't invent your own history. Tesla has never actively said you should take your hands off the wheel. In fact they warn the reverse, and always have. With even more warnings added recently.
Their video on the autopilot page has a person in the driver seat not touching the wheel the entire time. I know your response is going to be oh but they never "actively said" you should take your hands off the wheel but they sure as hell very very strongly imply you should.
That's false as evidenced by one of the other comments. That line is a new addition to that page. They also say in the video that the person is only there for legal reasons.
It sure it does. As I noted, this is their current material. Tesla has gotten in trouble with this before. However, it's buried amongst confusing and misleading diagrams and text discussing self-driving.
And you ignore all public statements from Elon Musk
Tesla can’t have it both ways. Tell everyone that they have autopilot/self driving to sell car and build hype, but then claim that the website says the opposite so there’s no problem.
And also on the website on the most visible feature on that page suggest the person is only needing to sit in the driver's seat for legal reasons. This is absurdly misleading
But Tesla clearly says “The person in the driver's seat is only there for legal reasons.” So it should be obvious to the driver that in court, they are on their own.
Your average Tesla owner definitely does know that because they had to go into the settings and enable one or both of those features (and pay a large sum of money for FSD), both of which are accompanied with a modal dialog on enabling indicating that they are in development and require your full attention.
The car doesn't ship with autopilot or FSD turned on.
That's their problem, then. Every reasonable step is taken to explain what each system does. If people can't understand something that simple, then they probably should not be driving anything, let alone a Tesla.
How many of the public know how a plane works and don't assume that auto pilot means you do nothing? Popular culture has people seeing pilots having their legs on the console and drinking, parting with the crew while the plane is in autopilot.
The general public is extremely stupid and believe everything they see in movies/TV. In fact a large part of the population believe that Fentanyl can kill you just by touching it which is the most ridiculous thing ever. Yet people are so paranoid about it that police officers have collapsed in hysteria after touching Fentanyl. [1]
Candy wrappers say "Do no eat!" on them in the Unites States so you can not assume people know how an actual Autopilot works. Now you may say this is because companies are getting sued but that proves the point that you can actually sue someone for being this stupid and win.
If I program my Bendix-King autopilot to fly heading 270º at 4K feet and there's a 5K foot mountain directly west of me
Bendix/King autopilot has a feature called Enhanced Ground Proximity Warning System which would be hard to ignore. Some models have "terrain avoidance maneuvering" which would actually prevent it.
I tried searching for those features in that airplane, as I'm not a pilot but assumed they existed.
On a related note, although they usually mean something's going wrong, which isn't cool, I've always been impressed by the way those warnings are delivered in airplanes. The tone and cadence of the warnings really communicate that this is something serious and you should respond to it and also communicates what exactly you should do.
Here's a video of the exact scenario being discussed in a Boeing 747-400:
It's not quite the same. If you are going to use aviation as an example, also consider that in Airbus planes the autopilot takes precedence and overrides pilot input to maintain a safe flight envelope.
Fighters with FBW like the raptor can't be over G'd because FBW takes precedence over pilot input.
Can you also defend the "Full Self Driving" naming is also super clear and it also implies that means something like "driving assist", not an english speaker so maybe "full" means incomplete in some american dialect , and "Self-driving" means "lane asist" in the same dialect.
It's a little weird to be defending Tesla's position here, because I think that Elon does overstate the capabilities of <whatever he's working on> and often to a needless and unhelpful degree. (It's like someone batting 0.409 and hitting 75 home runs per year claiming they're batting 0.600 and hitting 100 home runs. What he does is undeniably impressive and then he feels compelled to exaggerate it.)
That said, the selling of FSD has, to my knowledge, always been positioned as the selling of hardware that is expected to be capable in the future of FSD. Sometimes you have to read all of the words in a paragraph to get that meaning, but that is the clear reading I get from that same autopilot page:
Full Self-Driving
All new Tesla cars have the hardware needed in the future for full self-driving in almost all circumstances. The system is designed to be able to conduct short and long distance trips with no action required by the person in the driver’s seat.
The future use of these features without supervision is dependent on achieving reliability far in excess of human drivers as demonstrated by billions of miles of experience, as well as regulatory approval, which may take longer in some jurisdictions. As these self-driving capabilities are introduced, your car will be continuously upgraded through over-the-air software updates.
As others mention this disclaimers is not what is or was advertised, Elon fanboys always say "Ignore what Elon said, just read the latest Terms of Serive".
By the same logic an Hello World Python script has the capability for general AI, i just need to wait enough years then add later add the missing part that accidental is the essential thing.
Nobody disputes the terms of service language, but the advertising language, the intentional misleading names and stats greedy companies use. We are sure lawyers checked those terms of service very well and worked fine for US, seems German judges are not tricked by this trickery though, would be nice if more countries punish missleading advertising.
I imagining new Elon product "Clean Drinking Water" and ads showing people drinking it, BUT with small text that mentions that the "safe for drinking will happen in 10 years after we fix our shit" and fanboys defending the naming because you should already know that Elon is always doing this and you should read ignore the advertising and go to the legal documents about the product to figureut all the real details.
Did you pay for the full-self-flying version of the Bendix-King? More seriously though, collision avoidance system are now common in airplane autopilot system, so by that logic it should be reasonable for the Tesla to issue at least an overspeed warning.
Does the CEO of BendixKing regularly promote it on twitter as a full self-flying solution that will take care of intelligently avoiding obstacles like that mountain without your input?
I don't fly but have an autopilot on a boat, same thing. It'll happily hold a heading onto the rocks if I set it. But everyone knows that's what it does, the company doesn't make any claims or insinuations otherwise.
> Tesla cars come standard with advanced hardware capable of providing Autopilot features, and full self-driving capabilities—through software updates designed to improve functionality over time.
The marketing is weird. They mix up current capabilities and future aspirations deliberately.
For instance, here:
> The system is designed to be able to conduct short and long distance trips with no action required by the person in the driver’s seat.
They actually mean that the system is designed such that "in the future" it will be able to do this. The previous sentence is "All new Tesla cars have the hardware needed in the future for full self-driving in almost all circumstances", and you're supposed to carry that context forward into the following sentence.
Another aspiration, which if you weren't reading carefully you might assume was a present capability:
> All you will need to do is get in and tell your car where to go. If you don’t say anything, your car will look at your calendar and take you there as the assumed destination. Your Tesla will figure out the optimal route, navigating urban streets, complex intersections and freeways.
PS. Yes, there are caveats: "Current Autopilot features require active driver supervision and do not make the vehicle autonomous". That doesn't make the rest of the marketing copy clear.
While one would of course hope that people operating automobiles were aware of the limitations of safety and assistive systems, Tesla certainly pushes the boundaries of responsibly disclosing what its vehicles can and can't do--especially in a world where every product manual includes pages and pages of safety warnings that pretty much no one reads.
All Tesla vehicles produced in our factory, including Model 3, have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver."
Now, I can see how in a court of law you can make various arguments about what that promises, but I can see how someone would interpret that as it can drive better than they do.
But it doesn't matter how "someone" interprets it. It matters how lawyers and judges interpret it. "Someone" can interpret it to mean anything -- most people are idiots. I don't say this pejoratively, that's precisely why we need better consumer protection and advertising laws, but as they currently stand, this isn't likely to result in anything.
>that's precisely why we need better consumer protection and advertising laws
There are plenty of laws. Most products come with more pages of safety warnings than are in the rest of the user's manual. And, in California, you're informed that pretty much every building you enter or product you use causes cancer.
Could you source the material that states autopilot currently has "full self driving capability"? All the material I have seen are pretty clear it is not full driver automation. Or is this making assumptions based on name alone? In this case, Tesla wouldn't be the only one guilty.
It's not false advertising. I think there is a misconception among the public about what autopilot means. Autopilot is assistance, not replacement for human operators.
"An autopilot is a system used to control the path of an aircraft, marine craft or spacecraft without requiring constant manual control by a human operator. Autopilots do not replace human operators. Instead, the autopilot assists the operator's control of the vehicle, allowing the operator to focus on broader aspects of operations (for example, monitoring the trajectory, weather and on-board systems)."
If I said you were an awful human being would you take it as a compliment because awful's root etymology comes from the same place as awesome and it still has a definition in dictionaries of "inspiring awe" or would you for some reason insist on using the definition that the majority of people today use for the word?
I don’t know why tesla didn’t name autopilot copilot and full self driving autopilot. I guess in reality a copilot is much more sophisticated than an autopilot (because it’s a person) but I think it works much better with how those words are used colloquially. Look at GitHub copilot.
Tesla's value comes from a fantasy where they can say the cars drive themselves. Admitting that Autopilot is just a fancy Lane Keeping Assist + Adaptive Cruise Control is bad for that illusion.
I think the argument is that an aircraft autopilot is similarly limited.
I don’t like this argument, on the grounds that the average driver has no reason to know what actual autopilots do — I certainly don’t — and therefore naturally go by what it sounds like it does.
Also modern plane autopilots are significantly less limited than that argument claims.
Even a cheaper autopilot in a Cessna can take you from the airspace above your starting airport to the airspace of your destination, including lining up final approach maneuvers, with zero input from the pilot. Modern planes absolutely can fly and navigate themselves, but for safety, operational, and policy reasons pilots often don't use the full capability of these autopilots.
It seems pretty straightforward. He instructed the cruise control/autopilot to significantly speed, may have run a red light (question of fact for the jury), and is presumably guilty of something (matter of law for a judge). As others have noted, he can can try to shift blame to Tesla but, while I'm no fan of their autopilot marketing, I doubt he'll have much luck.
Correct me if I'm wrong, but doesn't Autopilot try to slow down and/or swerve to avoid a crash? I thought that was the main selling point, otherwise this is just a fancy name for cruise control.
Maybe now. But years ago, that's not how it was advertised. As shared by others in this thread:
From Tesla:
"Full Self-Driving Hardware on All Cars"
followed by:
"All Tesla vehicles produced in our factory, including Model 3, have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver."
Self-driving cars doesn't sound like fancy name for cruise control.
That's from Tesla. Who has already faced consequences for lying in marketing.
Right. I believe that they did market it like a proper safety feature in the past, absolutely capable of avoiding crashes. Also, the name is already misleading, as "Autopilot" suggests that the car actually "drives" itself to some extent.
Even if they weren't trying to mislead people in their marketing materials, these points seem enough to make a case.
What are you talking about? When was the last time you saw a Tesla advertisement?
> Self-driving cars doesn't sound like fancy name for cruise control
Having the hardware or the capacity to be a self-driving car is not equal to being a self-driving car. The hardware must be enabled, which in this case wasn't. The car only had Autopilot not FSD. Please learn the difference.
Depends on what's in front of the car. If it's a large, broad, stationary object like a firetruck or tractor trailer, then the sensors on the car aren't able to see those obstacles. Teslas have been known to frequently run into such things at top speed, completely blind to the fact they were there at all. It's so bad multiple people have been literally decapitated at this point, and Tesla has to date refused to fix the flaw (inadequate sensing) that caused the decapitations to occur.
People say Teslas are safer than humans, but humans who can't see large broad stationary objects would never be granted a license. If you are a robot driver, no problem though apparently. In fact, if you have proven to be recklessly homicidal while driving, your license is taken away and you are likely put on trial. Not if you're a robot! They will clone your flawed sensors and robot brain, and ship it to thousands of people to be deployed on roads across the world.
When your web service goes down you probably find the root cause, fix it, and issue a report about how it's not going to happen again due to your fix. Not Tesla! They are aware of the problem, know exactly how to fix it, and it seems like one man's ego is getting in the way of shipping actually safe autonomous cars. Tesla cars are not safe because Tesla as a company does not value safety as a priority. They value design, whizbang technologies, branding and image, while safety is a distant concern. Elon Musk put more effort into naming the line of cars SEXY than he did to actually producing reliably and predictably safe autonomous vehicles. Musk spent far more time lamenting over the fact he was forced to name his car the Model 3 than the fact it decapitated a human being due to his poor engineering choices.
Teslas on AP are death machines (read: they are machines who have killed people) and we in the public are the unwitting beta testers for Tesla Inc.
But, as it was expressed to me in a different thread, it's a sacrifice we all need to make to help Musk build the perfect software platform and advance our society. Just like we sacrificed people in plane crashes over the last 100 years to advance the aerospace industry (don't get me started there).
I'm with you on this. Nobody deserves this and this software should be in a test lab or closed road course until it's done. And others will argue that you can't perfect automated driving in those situations, and that's correct.
> But, as it was expressed to me in a different thread, it's a sacrifice we all need to make to help Musk build the perfect software platform and advance our society.
Yeah I can't help but cringe at this. Safety is hard, and you don't get safe systems without trying very hard and deliberately to make them. That's a fact of life and engineering. Trying hard to make safe systems does not look like deploying 2 ton automobiles that are critically blind in the very arena in which they are expected to operate. That's called reckless endangerment in my book.
This is a known limitation of radar based active cruise control systems, and is not specific to Tesla autopilot. Tesla was not the first company to deploy such a system, and the decision to allow it was made before Tesla. It makes sense because all cars are legally required to include another device that is capable of detecting such objects, a human driver. You can legitimately object to this feature being allowed, but the way you wrote it, as a screed against Tesla, makes it clear that your concern isn't really with the engineering limitations and safety implications of this product.
> This is a known limitation of radar based active cruise control systems, and is not specific to Tesla autopilot.
Which makes it all the more galling and reprehensible.
> makes it clear that your concern isn't really with the engineering limitations and safety implications of this product.
I was concerned about engineering in 2015 when the first decapitation happened. My present screed against Tesla is because they haven't fixed the problem in the intervening 7 years to the point where a second person was decapitated. That shows me the problem is not engineering but organizational. It makes no sense to talk about engineering issues when Tesla does not promote a culture of safety. Engineers are not in control there, hype men are.
The number of accidental decapitations your product gets to make is 1 and that's arguable. The second decapitation is criminal negligence.
And yes, I am incensed by this because as a roboticist, I think it's a travesty that Tesla has taken the industry in this direction.
When the product is a car, which has a rate of 36,000 accidental deaths (I don't really care whether or not they are decapitations) per year just in the US, then your standard of "more than one is criminal negligence" is ridiculous.
It's not a matter of statistics, it's a matter of demonstrated product defects that have predictably lead to deaths, that Tesla refuses to fix, even though they know it's an issue. I hardly consider that an accidental death. As you said, it's well known the sensors on Teslas can't accommodate the target use case (Level 5 autonomy). And yet Tesla won't admit this. That's what makes this negligent.
Imagine if it were your loved one who died in a Tesla using a technology that Tesla knew or should have known to be deficient and intentionally oversold its capabilities. Would you shrug and point to industry statistics? Or would you be mad at Tesla? Be honest.
Frankly, the attitude that Tesla should be allowed to test their beta-quality products on public roads (I certainly didn't agree to participate in the beta test) is what I find to be ridiculous.
I know the TOS for Autopilot require that the driver be fully engaged while using it — i.e. the driver completely accepts liability for everything that happens while AP is on.
But isn't one of AP's selling points is that it's sophisticated enough to mitigate if not outright prevent these kind of crashes? In previous fatal AP-involved accidents, Tesla drivers collided with stationary objects, which apparently are particularly difficult for AP to see. This case seems to involve one of the more common kind of serious head-on collisions: failing to yield to left-turning vehicle which has the right-of-way.
I know AP != FSD, and that AP doesn't necessarily promise it can handle traffic lights. But if it can't detect or react to vehicles slowly moving into the driver's lane...What are the dangerous day-to-day situations that it can confidently and reliably handle? Is it basically just lane control?
I don't think autopilot circa 2019 would detect and react to traffic lights?
The case seems pretty clear cut anyway, the guy had cruise control on and wasn't paying attention to traffic lights. We'll see if he turns around and sues Tesla, pleading ignorance about what "autopilot" could do.
Well ya, and so is Tesla. They/he only started making false claims around 2018 when the promises started to be about the actual FSD software that needs to make turns and handle the rules of the road: https://news.ycombinator.com/item?id=30856771
Earlier than that. It was October 2016 when Elon Musk promised that a Tesla vehicle would be able to drive from LA to Times Square without any driver input, including charging, by the end of 2017.
It would not. Traffic light detection arrived (for cars with the FSD package, it's not clear if this had it) in 2020. Autopilot on the car in question was limited to lanekeeping and collision avoidance with the vehicle in front.
In the past, you could order the cars with or without FSD hardware (on promises of future capability). These days (~2020+? I'm not sure the cutoff), you can order them without or without an FSD hardware enablement/cost, but they all have FSD hardware inside them, and the cost is just about turning it on or not, and you can always later opt to pay the cost or a subscription to enable it.
For cars that do have FSD hardware both installed + enabled (paid-for): if you're part of a smaller select group of opt-in, approved testers (currently ~100K drivers in the US plus a smaller number in Canada), you can enable "FSD Beta", which is a whole different animal than Autopilot and not really relevant to the discussion.
However, when you're using regular Autopilot, the capabilities of Autopilot are different for cars which have installed+enabled FSD hardware, and those which do not (either because you have the hardware but didn't pay the enabling cost, or because your car is old enough that some of them didn't come with the hardware at all if you didn't pay the cost up front or pay to have it retrofitted).
Non-FSD-capability cars, for example, can't detect stoplights on Autopilot at present, while FSD-enabled cars running Autopilot can. There are probably some other differentiators like that as well. Basically, Autopilot-on-FSD-capable-cars is slowly getting minor Autopilot feature upgrades that are only possible with that hardware while they're waiting for the "real" FSD to arrive. Autopilot-on-non-FSD-capable-cars doesn't get these upgrades and is stuck with a more-limited functionality.
Seriously. I thought those answers were great! He knows that even if Tesla does what is right, they're still going to get sued. So they're going to do what is right anyway. Super cool answer.
The statement that Tesla is "doing the right thing" has a lot of optimistic assumptions behind it.
To start with, Tesla assumes they will be able to accomplish self driving with the hardware they have. They also assume they need the system to drive on public roads (as opposed to simulations and private roads) to achieve this. They say that data collected from driving on public roads will get them there, even though this is a for-now-unsolved research problem in AI.
Furthermore, Musk seems to claim every year that Level 4 autonomous driving is within 6-18 months, going back to almost 10 years ago.
For now I feel there are too many people letting Tesla stick with their optimistic predictions and statements that they're "doing the right thing". Maybe the New York Times documentary that is about to come out will change this.
>> To start with, Tesla assumes they will be able to accomplish self driving with the hardware they have.
This is an absolute joke. There are videos of the autonomous "come pick me up" feature - whatever it's called - absolutely failing. One of them even runs into an airplane and pushes it around. Sometimes I think they're just using GPS with maps and PID loops and not actual AI of any sort. Nothing short of full general AI is going to be able to drive completely autonomously.
> We are excited to announce that, as of today, all Tesla vehicles produced in our factory – including Model 3 – will have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver.
October 19, 2016
Mid 2019 he envisioned a million robotaxis by the end of 2020
2014 - 90% capable of autopilot, for sure highway travel
2015 - autonomous driving for highways and relatively simple roads
2016 - S/X can drive autonomously with greater safety than a person - generally true, it still messes up for human-correctable situations but in general helps more than it hurts in terms of "crashes where autopilot was in control during impact or within 5 seconds of impact"[0]
2018 - incorrectly predicted - self driving will be 100-200% safer than a person by next year (this talking about their full self driving thing that will do turns and stop signs and whatnot)
the 2016 tweet about NY->SF was also incorrect by the time his prediction needed to be fulfilled, 2018.
Yeah, I'd suspect that if this lawsuit succeeds it's going to be on the grounds that Tesla's marketing gave users the impression that their self-driving tech was better than it really is, contributing to negligent driving even if they were technically warned. (And, of course, "you can mostly not pay attention, until it's incredibly urgent that you be paying attention" is quite the recipe for unfortunate accidents.)
"I think this is an extraordinary claim given some of the crashes and near-crashes that have happened"
Humans crash all the time.
It would be interesting to see a comparison of how often humans crash vs how often Teslas crash while the car is driving "autonomously".
I would not be at all surprised if humans fare worse, as there are plenty of horrible drivers, angry/distracted/inexperienced drivers, drunk drivers, sleepy drivers, etc...
> It would be interesting to see a comparison of how often humans crash vs how often Teslas crash while the car is driving "autonomously".
Again, Tesla cars cannot drive autonomously by their own admission (they always require human attention), so this comparison isn't possible.
Even if that weren't the case, there's also the issue that people only enable Autopilot in cherry-picked situations, which don't statistically represent general driving situations.
"Tesla cars cannot drive autonomously by their own admission (they always require human attention)"
Tesla instructs humans to be attentive while this feature is engaged, but obviously the car will keep driving even if the human's attention is elsewhere.
So Teslas do drive autonomously.. just not always as safely as (some) humans do under certain conditions.
Same with ordinary cruise control, which will keep the car going even when the human driver's attention is elsewhere, their foot is off the gas pedal and their hands are off the steering wheel.
While minor accidents are obviously much more common, in the US, there's approximately one fatality per 100 million miles driven. And that's across all drivers across all cars including old clunkers in poor mechanical condition and lacking modern safety systems.
I don't know anything about the Teslas driving themselves, but on the safety front you need to take into account how absolutely horrible most human drivers' skills are. 90% of them (sure, me included... no, wait, I don't drive... I do have the license, though, so I might as well be included) think they are "above average" and that other users of the road are the problem - how would they ever improve their skills with that mindset?
On the other hand, I don't think I will believe the problem to be solved before I see a self-driving bike. I mean, I would propel it, while it would steer without my input (other than setting destination). Once I do, I will have near total confidence in robo-taxis.
> Really? I think this is an extraordinary claim given some of the crashes and near-crashes that have happened (and again, the "autonomous" word).
The catch is that the situations where cars drive autonomously are easy, when the situation becomes complex, the autopilot bugs out and tells you to drive. Furthermore, the stereotypical easy case is on the highway, where you rack a lot of miles quickly, and statistics are typically "per mile driven".
Once you remove the bias, Tesla's self driving is not safer, I think I have seen a study that says that in the situations where Teslas can self-drive, they are on the same level as a human driver.
> I think I have seen a study that says that in the situations where Teslas can self-drive, they are on the same level as a human driver.
Maybe link to those studies. The one plus Tesla (and other ACC+lane keep systems that are obvious, as in 'loud beep when the system disengages'; my Honda notoriously gave the quietest beep whenever it lost its tracking of lane lines) has is that they don't look at their phone, don't get drowsy, and don't get distracted. So while many situations come up where a human could've prevented a crash, eg running into an overturned semi[0], it leads to about 3x less crashes compared to only passive safety features [1].
> one crash for every 4.31 million miles driven in which drivers were using Autopilot technology (Autosteer and active safety features). For drivers who were not using Autopilot technology (no Autosteer and active safety features), we recorded one crash for every 1.59 million miles driven.
> we count any crash in which Autopilot was deactivated within 5 seconds before impact, and we count all crashes in which the incident alert indicated an airbag or other active restraint deployed.
Meeting the bar of "helps more than it hurts" while requiring the human to be constantly on the lookout is absolutely nothing compared to "drive autonomously with greater safety than a person".
Tesla leads the industry in self-driving deaths. Out of all vehicle deaths caused by autonomous steering features, all but one involved a Tesla. Literally every other car manufacturer in the world, combined, has fewer accidents or fatalities than Tesla by itself.
(The one that didn't was the infamous Uber self-driving pedestrian fatality.)
In the context of this case - its also important to put it in perspective of what was being said about Autopilot in 2018-19 that this driver might have been reading about it as a feature in the marketing copy.
Sure in 2022 - we (including Tesla engineers) are all comfortably able to say that Autopilot is really just fancy cruise-control and the driver really needed to have been using FSD (cuz that's full self driving). But at the time - if my memory serves correct Autopilot was being marketed as the feature state of FSD is today.
It's not quite that simple though. "autopilot" is a blah term, but it's also not possible to say whether or not it has saved lives with auto-avoidance and braking. The hard truth is that for things to get better, they need billions of miles of testing, and you can't do that in a lab.
Tesla is an awesome company with so much potential. However, it's infatuation with Autopilot/self-driving/whatyouwannacallit is an almost impossible problem to solve with potentially fatal consequences as this news shows, and it diverts unnecessary resources from the company in creating a truly great electric vehicle. It's a strategic mistake from management imo.
> Tesla is an awesome company with so much potential.
I used to think the same, but after some years, I came to the realisation that Tesla is just the Silicon Valley's idea of a car company. They have been releasing unfinished products with glaring quality defects for a while now, and they have overpromised almost since the very beginning.
I honestly can't wait for more car makers to go fully electric, so consumers have better choices.
It's fair to say that Tesla turned the heat up on electric vehicles and the associated charging network.
So, yes, over the coming years I expect we'll see much more innovation in this area from established manufacturers (or even other new ones). But not sure how Tesla itself will play out.
I think the quality issues might have something to do with US auto manufacturing as well. I bought a new Ford in 2020 and the thing has had so many quality issues I am never buying another American car again. I just don't think the culture values doing quality work and building quality things because it costs too much and takes too much time and the current fiscal quarter is always almost over.
I think Ford, GM, VW, etc. have been on the end of enough accidents, recalls, and lawsuits to understand the dangers and consequences of what they’re doing and have their own safety standards even if all government standards went away overnight.
So I don’t think it will happen for them.
Tesla is not operating that way. I don’t see how they don’t end up at a reckoning.
This probably isn’t it. But it seems like it’s coming.
They are a Video game company. Unfinished products, forever in beta, DLC available before the finished product, QOL as DLC along with necessities (faster charging shouldn't be optional, now they are financially incentivized to slow charging down for forced upgrades), give us all your info so we can sell it, TOS 50 pages long and illegible except to the most hardened of contract lawyers, and not even then (this case is a perfect example- who is to blame, etc).
When some of us warned of this shit from the start we were called luddites, impeding progress, boomers afraid of the future.
I still want my promised hoverboard, instead we get an upgraded cruise control DLC laden car that kills people while claiming you can watch movies and sleep while it drives you.
For those confused by the headline here: the trial is of the driver of the car. It's a routine vehicular manslaughter case. The involvement of Tesla is only that "the car was on AP so it's not my fault" is being (presumably) attempted as a defense.
No one’s confused. From a technology perspective, the Autopilot was driving the car at the time of the crash. From a legal perspective, a human is always in charge of driving the car. It’s time Tesla claims were tested in court. No other manufacturer makes such grandiose claims, yet their systems are at par with what a Tesla can do.
FWIW, this case is considered to be notable because it is the first fatal car crash in which Tesla Autopilot was involved, and in which the driver faces felony charges. The charges were first announced in January [0]; today's news is that a judge is allowing it to go to trial.
In the submitted article, it says a Tesla engineer testified that Autopilot was activated "about 20 minutes before the crash". I assume that means it was active 20 minutes before, and into the moment of impact. If this case does actually reach trial, I guess it'll be interesting howhat the jury will be instructed in weighing the use of Autopilot against the allegation of "gross negligence"
>Riad had initially set his car’s speed to 77 mph; he then set it to 78 mph, and then finally changed the speed to 75 mph. At the time of the crash, the car was going approximately 74 mph.
For me, I’d convict. This isn’t a case of ‘autopilot’ as much as it’s a case where a person purposely (with intent) set his speed above legal limits and did not maintain the attention while doing so.. causing the death of two individuals on their first date.
He was on a freeway when it was set, but still above since the speed is 65. The freeway transitions into a surface street and this seemingly occurred at the first light after the transition.
Exactly. And from a systems standpoint, if we're blaming the autopilot we should also consider how much blame should be allocated to the traffic engineer that clubbed a 65MPH California freeway directly into a 45MPH local business street with no traffic-calming measures or other mechanism to signal to a sleepy human brain that the rules of the road had changed besides a lone speed limit sign and a "Good luck sport!" attitude. It'd be interesting to pull history on that first stoplight and see how many accidents have occurred there in the past five years.
(It turns out that's possible. California provides a GIS data digest of fatal accidents maintained by UC Berkeley. Data collection ends at 2021, but the 2016-2021 data set shows 22 crashes coming up on that intersection. https://tims.berkeley.edu/tools/gismap/).
Or we could limit the assignation of blame to the individual who is solely responsible for controlling his multi-ton high-speed vehicle at all times when operating in public thoroughfares.
Successfully executing on that design requires way more global data than the autopilot system alone requires.
... which is not to say they shouldn't do it. In fact, in the 21st century with ubiquitous communication we could even consider mandating that of all cruise controls. But if we were to codify that into law, we'd want a data source a touch more reliable than whatever Google, Apple, an Waze have cooked up as the speed limits on each road in the map (since those limits are currently at the safety threshold of "advisory to estimate travel times," not "someone could die if these numbers are wrong").
> all the maps have built in speedometers with an alarming red rectangle when you are speeding
By "all the maps," do you mean "Apple, Google, and Waze?" If so, see my previous statement; they're offering a driver-assist, but they're not ultimately responsible for your safety so their data doesn't need to be fully accurate. Tesla is pulling its data for the in-car map from Google. Nobody is going to try and sue Google if they're in a crash because "the little red box that tells me I'm speeding didn't show up, Your Honor." And Google has made clear that it indemnifies itself from such liability in provision of the map data (i.e. "You're holding it wrong" if you wire Google's idea of what speed limits are directly into the control plane a life-or-death machine).
So Tesla would have to take ownership of that problem directly... Put its own fleet of cars on the road to drive around and photograph speed limits. They could actually hypothetically do that given the cameras on the Teslas if they push data from the driver back to HQ. What's the data licensing agreement look like on owning a Tesla? Does it already grant Tesla that privilege by virtue of someone owning their car?
> the data for what the speed limit is is generally public, or very obviously derivable from principles-first, common sense engineering
Public, yes, but must be consolidated, validated, and kept up-to-date. And the principles-first "common sense engineering" does not describe how America's roads are actually labeled and structured, sadly.
> autopilot has more than enough data to know its not on a freeway
Does it? I don't think Autopilot has any concept of what road it's on; it's not consolidated with the Tesla's navigation map. It's viewing the road directly and looking for local lane markers and obstacles.
Such a consolidation with the global data to understand current speed limit would imply new design safety constraints for both the Autopilot and the onboard map systems.
Consumer gps doesn't always know which road one is on and could trivially decide to use an adjacent street with a wildly different speed.
It's also sometimes unclear about where transitions happen.
It also never accounts for actual speed of travel diverging substantially because citizens including Police have collectively decided that a different speed is more reasonable.
Basically such a system would probably be completely useless enough to ruin all cruise control.
I don't understand why limiting the upper bound of speed when a user is using autopilot would be difficult at all? These cars can already recognize speed limit signs and have settings for 'how much over the speed limit do you want to be'.
> These cars can already recognize speed limit signs
Do you mean they're actually reading the signs as they pass on the road, or they have a map associating the current position on the road to a speed limit for that road?
If the former, the cutting edge of that technology I'm aware of isn't good enough to trust it to a safety system. And that's before we factor in added risks like "speed limit sign was obscured by a passing truck while the car drove past it so the car maintained speed."
If the latter, that's possible but it's not where Tesla is right now. Tesla doesn't own its maps, it's trusting Google, which doesn't certify the speed limits are correct enough in its map data to wire a safety system to that output.
What would the reasoning be for this? Trucks are already limited in this regard and it doesn't cause catastrophe. Why should a company be allowed to knowingly break the law?
> Why should a company be allowed to knowingly break the law?
That's an extremely strong assertion. Do you believe all car makers are breaking the law by making cars that can drive 80+ mph? By having cruise control that can be set at 80+mph?
80ish dropping to 70 for tight curves and other "features" is pretty standard for everywhere in the Boston-RI-CT-NYC corridor for pre and post rush hour.
"Tesla should not allow setting autopilot above the speed limit"
This is likely where we're headed... especially with fully autonomous vehicles.
It'll probably still be possible to hack them to go any speed you want, though... and certain cars like emergency and military vehicles will have no limit.
I'm also extremely surprised that there's a 20 mph drop. While it's only a recommendation, the Uniform Vehicle Code and Model Traffic Ordinance suggests never changing the speed more than 10 mph at a time, and having no more than six such changes in a mile.
See page 28 of this PDF, as I was looking for this information specifically in relation to the Florida DOT, which does a pretty good job of following this rule:
No, precisely not. A stop-light as a traffic-calming measure is like "calming" traffic via an invisible wall-of-death that is sometimes present and sometimes not. It requires more higher-level thinking of the driver, not less.
The purpose of a traffic calming measure is to force slower driving via involuntary driver behavior or physics. Things like painting obstacle-like objects in the road to startle unattentive drivers, rumble strips, unnecessary curvature onto soft embankments so failure to make the turn doesn't risk injury, unnecessary narrowing of the road via cones or soft baffles (because people will subconsciously drive slower when they feel like their vehicle is inches from a barrier).
A common mis-design of highways is to assume traffic lights are sufficient for that purpose, and the result is creation of a high-crash location.
In a “simple” case of a road that causes crashes? Probably. Theoretically, they can held liable, but this is a pretty “small” issue to bring a suit.
In this sense the term “engineer” is someone that requires a license and training (like MDs or lawyers). States require engineers to “stamp” designs that they are taking responsibility for. Sometimes “city planners” LARP as (unlicensed) civil engineers and make designs that get built, which is somewhat problematic but happens.
Stamping and signing off on a design means legally accepting liability. How often they get prosecuted is rare. One famous example is Boston’s Big Dig fatal tunnel collapse. Of course politics and money had a big role in that case.
how much blame should be allocated to the traffic engineer that clubbed a 65MPH California freeway directly into a 45MPH local business street with no traffic-calming measures or other mechanism to signal to a sleepy human brain that the rules of the road had changed besides a lone speed limit sign and a "Good luck sport!" attitude.
None. The answer to how much culpability the traffic engineer should have is "none". Freeways end that's what they do and there's no expectation that they won't have a light, or a traffic jam or a deer or some other obstacle in front of you that you'll have to stop for. Lights crossing two-lane highways happen all the time outside of urban areas.
Drivers are, appropriately supposed to be alert enough to see a red light up ahead. Of the twenty-two accidents you cited how many do you think would have been prevented with more signs?
For that matter it's not the autopilot's fault either its the fault of the idiot driving (or not driving as it were).
I agree with your thinking; invoking the responsibility of the highway engineer was intended as a counter-example.
And when you dig into the five-year numbers, you find that while 22 is quite a bit, there have been more on the straightaway highways north-southbound east of the intersection and on the cloverleafs (If I were to hazard a wild-ass guess, based on California driving experience: driver going highway speed failing to realize traffic came to a standstill on the highway due to back-pressure of people exiting the cloverleaf at rush-hour).
Exactly. Most of the legal issues here were actually hashed out decades ago as courts addressed cruise control. Yes, the machine was running the throttle for you, but you are still responsible for instructions given to the machine. You are responsible for speeding and you are responsible if that speeding leads to accidents even though your foot was not actually on the accelerator.
> ...and did not maintain the attention while doing so
IANAL, but exceeding the speed limit wouldn't be enough for a prosecutor to want to prove felony vehicular manslaughter. Even when DUI is involved, juries can still decide that the driver tragically f-cked up, e.g. underestimated how drunk they were, but not in a "grossly negligent" way. Even though virtually everyone knows that drunk driving is illegal.
But how many potential jury members know or believe that violating the fine print of using Autopilot is illegal or wrong? Or that a Tesla owner should be convicted of a felony because they "obviously" should have known better? I'd assume far fewer in a jury pool have strong knowledge and opinions on Autopilot versus drinking-while-driving.
Of course, there's at least some question of what gets presented to a jury. The defense will presumably try to make this at least in part a question of "confusion" about the car's capabilities. However, I'd expect the prosecution to make a case for that all being irrelevant just as it would be for regular cruise control.
> However, I'd expect the prosecution to make a case for that all being irrelevant just as it would be for regular cruise control.
I can imagine this being a major issue in this case. Elon isn’t known for keeping his mouth shut, especially when it comes to bragging about Tesla. I can imagine that his statements and Tesla marketing being compared to the fine print on Autopilot. Or worse, he tweets counter points live.
Especially complex if Tesla gets involved further as part of data discovery.
Just replace the term "autopilot" with "cruise control" because that's all it really is with a bit of lane keeping added. Now does it seem reasonable for someone to set that to 75mph in a 45mph zone? was it 45 or less?
The article says the driver on a freeway, and had set Autopilot 20 minutes prior to the crash. Obviously there would've been a speed zone before the traffic light intersection, but presumably the speed limit was 65+ at the time the driver set Autopilot.
Is it understandable for you to set your traditional cruise control at 75mph and not expect things to change for 20+ minutes? I don't think so — I wouldn't trust CC to keep me on the road for more than 2 minutes without my direct guidance. Is it understandable that you might trust Autopilot to handle the road for 2o minutes, because you were confused or misled by Autopilot's capabilities? That feels like a substantively different question.
>Is it understandable that you might trust Autopilot to handle the road for 2o minutes, because you were confused or misled by Autopilot's capabilities? That feels like a substantively different question.
If you program a robot to commit an illegal act and someone dies, is it the robot manufacturer's fault?
Isn't 74 a pretty normal speed to drive on a highway? (Only drove there once)
It sounds to me like he was coming off the highway and didn't notice that he was leaving into an intersection with a red traffic light, where the accident occured. Not that I'm saying it is okay for him to not have noticed it.
> Riad was driving on the 91 freeway, heading west into Gardena. The freeway empties out into Artesia Boulevard at Vermont, but Riad, seemingly unaware of that transition, did not appear to react — and neither did his Tesla.
Speed limits in the United States are generally set such that, unless there is heavy traffic, nearly everyone is speeding. Using that as a pretext for charges is generally questionable.
(He was going 75 on a freeway; the freeway suddenly became a surface street, which is bad design, regardless of whether he should have been paying more attention.)
You'll only get ticketed unless you're going really slow for no good reason. But in general if you're poking along even in the right lane at 10mph under the speed limit and below the flow of traffic you're basically causing a whole lot of lane changes as everyone tries to flow around you.
One of the reasons I hardly ever use conventional cruise control is it makes you drive in a way that minimizes speed changes rather than doing what makes sense in the context of other traffic.
To add, this must _just_ be adaptive cruise and not autosteer since autosteer (referred to as 'autopilot') explicitly won't drive more than 5 mph over the limit except on highways.
I have a hard time seeing this case going to trial. From the article it seems no plea deals have been put on the table, likely because both sides wanted to see the results of this ruling first.
I’m guessing the criminal trial pleads out and the Autopilot issue will get more run in the civil complaints. I wouldn’t be surprised if Riad, the Tesla driver, files his own suit against Tesla to try to foist more culpability on to them.
I checked out this intersection on maps and street view.
1. The traffic lane the victims were turning into is in addition to the ending freeway's traffic lanes (i.e. it is one lane further to the right than the right-most through lane). So someone had to be crossing lanes one way or the other for the collision to occur in the first place. Maybe not, I am not a traffic expert or familiar with the area, but that is how it looks to me as an experienced driver.
2. If "Autopilot" was engaged why didn't the Tesla react to the obstruction? Perhaps the timing was too sudden and it didn't have time to reach before being smashed to pieces?
3. Was the victim driver tested for intoxication? Perhaps they were not following all traffic laws as well? Sometimes accidents are the culmination of multiple drivers' mistakes. Not trying to victim blame though. May they RIP.
I think it is clear the Tesla driver was being reckless here because of the speeding. It is unclear if he ran the red light or not. I'd say at the very least revoke the guy's license permanently.
Tesla is, to me, clearly reckless as well with their marketing tactics and their failure to tamp down unrealistic expectations and the hype-machine surrounding "FSD".
> So someone had to be crossing lanes one way or the other for the collision to occur in the first place. Maybe not, I am not a traffic expert or familiar with the area, but that is how it looks to me as an experienced driver.
The victims were making a left turn to get to the other side of the road to head east, the defendant was heading west.
> Perhaps they were not following all traffic laws as well?
They were the fourth car turning left. If the light was extremely short, maybe the light had turned red and they were late, in which case they were in the intersection illegally. NB that were that the case, they still would have right of way as they had entered the intersection before the Tesla. The defendant would still be fully culpable.
I don't see any situation in which the defendant was not driving recklessly. He was almost certainly driving at 74 mph directly at a red light, unless all four cars ran a red light across multiple lanes. When the light turned green he had at most a second or two to have stopped, if it had not changed. He did not even touch the brake.
Realistically, he was driving with his eyes closed because it was 1230 and he was tired. Emergency braking would not have saved the two people he killed. Evasive driving would have still caused a crash at that speed. The only thing that could have saved their lives was respecting the red light, and the car was not programmed to do that.
Even if the car was fully self-driving, the "hands on steering wheel" thing is a very clear "pay attention" signal. There's a valid argument that complacency is a natural result regardless, but this was not complacency. He had zero awareness of what was happening outside the vehicle, and that is reckless.
A compilation of Musk lies on autopilot through the years (starts 2014): Includes claims that you can "fall aleep" and "pay zero attention" , "watch movies"
But "our product will be able to do X" followed later by "and here's our great product!" can be understandably confusing for consumers, even if you don't technically say "our product can currently do X".
In my opinion Tesla has the largest part of the fault here. You can say "yeah we all know Elon Musk is lying and that is not a real auto-pilot", but I am having hard time to understand why the responsability of a blatantly lying company/CEO is not discussed.
The car was apparently going 74 MPH on a surface street, since the driver didn't reduce the speed on the cruise control after the freeway became a surface street. The driver may legitimately have been asleep at the wheel.
The best argument Tesla has to isolate itself from liability is that there is no way that the car could have reacted to the oncoming car and stopped at that speed, even if its AI system were perfect. They would have to dodge criticism about not using LIDAR to detect obstacles early and also the criticism that the AI system didn't notice the obstruction _at all_ and could have slowed down earlier.
All that said, this is not a good fact pattern for the people hoping to come after Tesla instead of the driver. However, the cracks are beginning to show.
>> ... may legitimately have been asleep at the wheel.
There is never a legitimate was to be asleep at the wheel. He was perhaps genuinely asleep, but short of being drugged against your will there is no non-negligent reason to be asleep at the wheel.
In this context, "legitimately" means "truthfully" not "rightfully."
Edit: to clarify, the adverb "legitimately" modifies the verb "have been" in this context. You can tell because it is immediately before "have been." If I had said "he was legitimately asleep" that would mean that there was legitimate reason for him to be asleep, because the adverb "legitimately" was attached to the word "asleep." The phrase "he legitimately is asleep" says nothing about the legitimacy of the sleep, but emphasizes the legitimacy of the "is" (as in, he was so asleep that he wouldn't notice his car beeping at him to wake him up).
Such distinction really matter. There are many cases addressing crimes committed by people while asleep, including an infamous case of a man who was asleep before he got into the car.
With peoples lives in the balance it is horrific to see Tesla be this reckless.
But I guess it is a top down culture issue perhaps?
Seeing how Elon Musk acts, and how he will chase the shiny thing when he wants attention, it’s all about ego right? Like what’s with the SpaceX always on demand massages for their executives?
Seems like they are losing sight of what really matters, and I can’t help but wonder if this build up of failure is going to cascade into the Musk companies faltering, which would be a big loss as space flight and green transportation are very important (imo)
I wouldn't say so. If anything his companies are doing better than ever. Tesla is doubling revenue / deliveries YoY with insane gross margins and just opened 2 new factories on 2 continents. SpaceX is sending a new batch of Starlink satellites is going up literally every week, and they've sent over 20 people to LEO and back (could you imagine if we had to rely on Russia for this today?!).
Not to mention that the autopilot program is full of a ton of really smart & hard working people who put their soul into their work, because the lives saved by bringing the product to fruition will be astronomical. They are making real practical progress on AI systems -- better than can be said of much of academia & industry.
I think they might be talking more specifically about the story about a SpaceX flight attendant being pressured to get her massage license, only for Musk to eventually expose himself to her and offer to buy her a horse in exchange for sex
Relevant detail at the bottom "According to one of the complaints, Riad had been cited at least six times for moving violations in the three years before the crash."
> Riad was driving on the 91 freeway, heading west into Gardena. The freeway empties out into Artesia Boulevard at Vermont, but Riad, seemingly unaware of that transition, did not appear to react — and neither did his Tesla.
While I don’t think the driver is without serious fault here, shouldn’t Tesla really be able to slow and stop a car coming off of a freeway and presented with a controlled intersection?
Don’t get me wrong. I own a Tesla and use Autopilot. I also think that the combination of map and sensor data should clearly prevent incidents such as this one.
My car (not as advanced as a Tesla) reads the speed limit using cameras to parse street signs.
Maps will always lag reality, but this is a commonly driven street, probably not the first Tesla to drive down it-- Waze was using info like this for years, hate to think Tesla is less capable.
In short, there is no excuse for Tesla's maps not knowing about the speed change on this stretch of road.
Teslas most certainly read speed limit signs. They have for years now. My stock Model Y purchased in January 2021 had the feature built-in.
With FSD beta, turning off a major highway onto my neighborhood street (very narrow and no lane markings), it defaults to 25 MPH. About 100 feet later, the car sees the 35 MPH sign and speeds up accordingly. In this case, 35 MPH is too fast given the curves, slight hills, and limited visibility. I slow down manually in these cases using the roller wheels.
Tesla has gradually improved its speed planning to account for different scenarios. But it's still a work in progress and often requires manual tweaking.
> Maps will always lag reality; I don’t think that it’s reasonable to expect Tesla maps to have perfect accuracy.
That's why you don't rely on maps as your only input to a feature such as this. Also, is there any information on how long this situation was in the way posted because I can't imagine that they built a city overnight at the end of that transition, it looks like a situation that had existed for many years prior to the accident.
The result of lawsuit will tell how much is the driver or company absolved. Also would be interesting if you'll need special licence and 1000 hr training in simulator before allowed to drive Tesla.
Assuming a "it's Tesla's fault" defense ever makes it before a jury. A judge could decide that the driver was in control of the car and trying to slough off that responsibility has no basis in law.
An aircraft autopilot controls speed and direction, within limits.
This is false, and I don't know why Tesla stans keep bringing this BS up like it's some sort of justification.
A modern aircraft autopilot is fully capable of taking a plane from takeoff to landing without any pilot intervention required. Pilots frequently use autopilot in flight, but rarely do so for takeoff or landing because it's more stressful to monitor the craft's autopilot system at those moments than it is to simply handle those parts of the flight themselves (and at least for autoland functions, the airport must be certified for autolanding).
It depends on exactly what iteration of Autopilot he was using. I'm not sure how old this case is. Early on it was little more than traffic-aware cruise control with lane holding. If that's what he was using then it has no knowledge of traffic lights or anything else and it was a 50/50 chance when you hit an exit if it would take the exit or stay in lane once it lost lane lines.
There's lots of things the media has called Autopilot.
> There's lots of things the media has called Autopilot.
The media, or Tesla? It's not the media's fault if there's confusion on the several marketing terms Tesla has come up with to imply it will drive for you.
I'm not sure how this is relevant. Either the autopilot is safe to use (regardless of "iteration") or the feature shouldn't exist and should be illegal to use. If some "iteration" of autopilot is incapable of _X_ where _X_ is some safety-critical thing, then you shouldn't be allowed to use autopilot.
There's signs for miles back. The speed limit progressively slows down to surface street speeds. If a Tesla can't prepare for a freeway ending then it shouldn't be on the road.
Yep - here's what you see today not the Auto-pilot feature landing page. They probably need to update this page - if only to upsell customers on full FSD (not to mention it being used against them legally)...
The video which is front and center and really the gist of the landing page for most casual viewers/buyer starts off with:
"The person in the driver's seat is only there for legal reasons. He is not doing anything."
I agree - though - as someone who holds Tesla stock indirectly through various indexes - I fully can understand the desire to want post facto look well past the webpage fold for fine print.
This was 2 years ago, with no indication of what software version the car was running. The fact that current versions of autopilot would now slow down when exiting a freeway to help prevent accidents caused by careless drivers is a testament to Tesla's approach to development and how it will save lives
> [Lawyer] argued that Riad’s actions — driving 74 mph on a surface street — did not rise to “gross negligence” and should not be considered a felony.
At that speed, the driver's momentum was around 67.000 kg*m/s.
That is an impact about 10 times bigger as getting thrown out of a plane and hitting the ground at terminal velocity.
I'd say it's gross negligence to drive that fast at all on any surface street with sidewalks, traffic lights, intersections, stores, whether the driver is asleep at the wheel or not, autopilot or not, drugged or not.
> I'd say it's gross negligence to drive that fast at all on any surface street
Agreed, but Riad set that speed when he was driving on a freeway, and Autopilot failed to slow down when the freeway became a surface street.
I think Riad was negligent for not maintaining control of the car (Tesla even warns drivers to "pay extra attention to the road" when using Autopilot or FSD), but I wonder if Tesla has any liability for marketing Autopilot and FSD capabilities that don't always work.
Mercedes has it, but most don’t. My CRV has the equivalent to autopilot and maybe would have applied emergency breaking just before impact. Not sure honestly. These advanced cruise control features are not self driving at all. Mine won’t even stay in the lane if I take my hand off the wheel.
Honda doesn't suggest the CRV is self driving. Tesla marketing encourages the idea that the car can drive itself, in fact it literally will with the Smart Summon feature. That's misleading, so it's not surprising if some people are misled.
Again, this is not what Tesla calls “full self driving.” No one driving a Tesla would ever be confused about this. The car will flip out if you take your hands off the wheel.
Every manufacture has some silly name for advanced cruise control. It’s called marketing.
Small nitpick: that's the impact if the car impacts an immobile structure square-on. Cars don't impact like that; static friction of wheels on road gives out way before the full kinetic energy is imparted to the passenger cabin of the victim vehicle, and then the car is thrown and can clear that energy by skidding instead of further deforming.
But the point stands that it's bad and not something we should be structuring our society to encourage.
>> At that speed, the driver's momentum was around 67.000 kg*m/s.
So? Any large truck moving below the speed limit and being perfectly safe can have vastly more energy. And a speeding motorcycle will have vastly less. In a highly-regulated behavior such as driving, negligence is not a function of energy. It is a matter of law rather than physics.
Ya, but that law isn't well respected. It hasn't been tested let alone affirmed by any court. It is very likely not constitutional. I would not cite it.
Then it doesn't make any sense to compare it to falling out of a plane. No matter what mass you're talking about, it cancels out when you compare it to another velocity, so the factor would still be 1.6, not 10.
This momentum argument doesn't follow. First you calculate the momentum of the vehicle, then you compare it to an impact without describing exactly what you're comparing. However, the momentum of the vehicle will absolutely not be completely transferred to the person in an collision, and so will be not much like hitting the ground at terminal velocity.
And why are we talking about momentum anyways? A fully loaded tractor trailer weighs about 80,000 lbs (~ 36,000 kg). Therefore, a tractor trailer traveling at 5 mph (~2 m/s) has a total momentum of 72,000 kg*m/s. By your argument, all tractor trailers driving through a city are grossly negligent.
I understand, that’s (part of) my point. The transfer of momentum is more important than how much exists. But more broadly, negligence isn’t measured in kg*m/s.
I think we're overextending the point a little, and apologies if my point did not get across well.
What I wanted to illustrate was that those kind of speeds, in heavy vehicles, in cities, on surface streets, are negligent - spiritually. The sheer amount of damage that can be inflicted, in the blink of an eye, by a moment of inattention, is beyond measure (literally, that 67k number looks goofy, unreal, but it's cold hard physics). Our traffic laws are wrong, our streets are wrong. We don't tolerate that anywhere else. On a factory floor with dangerous machinery there are higher standards of safety than our public realm. This news article is the perfect illustration. Good chance the driver fell asleep, or was distracted in some way (totally normal human traits, btw, and does not make somebody a bad person). Yet the lawyer claims, that at 75mph in a 5000 pound car, this is basically just worth a speeding ticket. That's an enormous disconnect. That delta between responsibility and accountability is totally totally out of wack for drivers. An enormous amount of responsivblity (67k kg*m/s worth of it), and very little accountability ($100 speeding ticket).
And yes, at slow speeds, there's just a lot more slack in the system. The driver of a 5mph tractor trailer has a lot of time to react, so does every one around them, stopping distances are exponentially lower etc... It doesn't change the fact that it will level a house, and so we should have extremely high standards.
It is if you assume the car is a vintage box car from before the 80's where you might or might not have airbags at all. New cars are designed to crumble and deploy airbags to absorb that impact so that the squishy humans inside the car don't experience anything nearly that high. Nowadays the biggest risk is speeding, as it negates the advances in car safety.
> a 5 mph increase in the maximum state speed limit was associated with an 8 percent increase in fatality rates on interstates and freeways and a 3 percent increase on other roads
Maybe it's just me, but I took the original concern to be for people and objects outside the car. It's a bit jarring for your only concern to be the passengers.
I was under the assumption that "the driver's momentum was around 67.000 kg*m/s" in the top parent comment meant we were talking about a human confined within a car. This is also supported by how @butwhywhyoh says " then you compare it to an impact without describing exactly what you're comparing" in which @trgn still doesn't say clarify what sort of comparison he's making.
It's an argument against using some irrelevant momentum number in a comment. Without showing this momentum number, their comment is just "I'd say it's gross negligence to drive that fast at all on any surface street with sidewalks, traffic lights, intersections, stores, whether the driver is asleep at the wheel or not, autopilot or not, drugged or not" which certainly wouldn't get as much engagement on HN.
> That is an impact about 10 times bigger as getting thrown out of a plane and hitting the ground at terminal velocity.
Even without computing any numbers, purely at the sanity-check level, this calculation is clearly wrong.
Think about it. While there have been some recorded incidents of people surviving a free-fall impact to the ground, that's mostly a certain death impact. Even if survived, you are in very bad shape (understatement). An impact 10x worse than that would completely destroy your body.
Then compare to race car drivers routinely having crashes at speeds higher than 74mph and most of the time hopping out of the car with no injury. Even at much much higher speeds.
The flaw in the argument is summarized in the common saying "It's not the speed that kills you it's the sudden stop". If you hit the ground from an airplane, the stop is very sudden, leading to extreme instantaneous g loads. That's what kills, not the absolute value of the speed itself.
In car crashes there is a lot of energy dissipation due to both structural crumple zones as well as motion of the car itself (it might spin, flip, skid more, etc). These factors lenghten the deceleration time and thus greatly reduce the g loads on the body.
A four lane "basically an interstate if it weren't for the traffic lights every couple miles" type boulevard or state highway typical of urban/suburban areas of sprawl type construction and a two lane residential street are both surface roads. And there's tons of both of those types of roads in the LA area (where this crash took place). There's a big difference between driving 74mph on a somewhat limited access boulevard vs driving 74mph on a residential street.
The outcome here will tell us a lot. Mercedes with their entirely different, seemingly more responsible, approach to this, may end up on top in all this.
This is interesting. Autopilot (without FSD Beta) will not go over 5mph of a road's known speed limit. Moreover, Autopilot will brake to a stop when it detects a potential obstacle and/or collision.
Riad hit and killed the couple after a left turn from exiting the freeway.
So either speed limit data was missing (in which case the limitation doesn't apply), he had his foot on the pedal and didn't know it (during which Autopilot will not brake; the driver is notified of this if the pedal is pressed down for more than a few seconds), or he turned Autopilot off before the crash (unlikely given the data that's in this article).
> Autopilot (without FSD Beta) will not go over 5mph of a road's known speed limit.
I don't remember such a limit on my M3 but will check tomorrow. You can simply accelerate to any speed below AP's global limit (90mph in this case) and then engage AP. It'll keep at that speed. Meanwhile, Model S'es manual doesn't seem to mention anything in this regard.
Well, over here in Oz, all roads have speed limits which Tesla knows about and displays. Will have to try myself and will report. Although I'm fairly sure you're confusing engaging AP when under the speed limit at which point it will set it to the speed limit, minus or plus the amount you configure in the settings. So if you configured it to be +5 mph of the speed limit, that's what it gets set to. But if you were already driving at 30mph over the speed limit and engage AP, it will keep your speed. And it makes sense, in case it has incorrectly determined the speed limit, it won't cause safety issue by driving too slowly.
surface street just means "not the interstate", in my experience it's a word colloquial to LA, where the interstate highway is almost entirely elevated, so exiting is like coming down to the surface.
I am not an expert in autopilot technologies, but my intuition is that its going to take 50 years, or more, for the technology to be ready for this, and the road conditions will need to remove the human-driven cars from the equation for this to work. Putting the driver's responsibility to be aware of what was going on around him aside (which I agree he did not seem to be), Telsa is being grossly negligent for testing this kind of tech in the public sphere.
IMHO, computers and software cannot, yet, and for a long time, will not, account for all the variables that are part of driving. People that drive these cars really need to understand that computers cannot think like people. Thats my two cents about this, and I hope that this trial puts the brakes on testing this tech in public.
"IMHO, computers and software cannot, yet, and for a long time, will not, account for all the variables that are part of driving."
Neither can every person who's allowed to drive.
The question is should we allow semiautonomous/autonomous vehicles that drive at least as good as the worst human drivers who are legally allowed to drive?
As far as the law and the courts are concerned, I think autopilot part of the story here is irrelevant. As drivers we are responsible for whatever happens when we are behind the wheel. Full stop. It’s pretty obvious they weren’t focused on the road, and failed to realize the highway was ending.
On a related note, I’m starting to think Tesla get a bad rap with all the negative press about their advanced cruise control system. And I didn’t call it ‘Autopilot’ here on purpose. Because there are many many versions of advanced cruise control on the market. Tesla does not have a monopoly on this type of technology. I saw an advertisement the other day for full size pickups that have it now.
The commercial showed a truck changing lanes and passing another vehicle, while towing a trailer, presumably all on its own. That is terrifying to me haha. Why do we all tend to focus on, and demonize Tesla’s version? Is it because it was one of the first, or maybe just the way Musk tend to be a magnet for criticism & scrutiny?
How many wrecks and deaths have been caused by other manufacturer’s systems? It’s got to be happening. This will continue to be a problem, not just for Tesla, as the tech becomes more widespread.
When it comes to trailers specifically, I could be pretty easily convinced that a computer will handle it better than a human. The physics are tricky to intuit and the feedback you get is not great.
I do not know the law in the US for that, but how the manufacturer represent the product and its capabilities can often make him liable.
Unlike most others, Tesla has been very brazen to describe its system as self piloting rather than driving assists (but see the tons of asterisk at the bottom who clearly state that we're lying in the big letters!). Naming it Autopilot being one of the most obvious cases, autopilot means something and that's not what theirs does.
I know that things like EULA, NDA, and other forms of agreement like that are more easily enforceable there so maybe legally they won't be in trouble, but morally my personal belief is that they absolutely are.
Some other brands have level 2 driver assistance technology as a standard feature. All of these can operate the steering wheel, brakes, and accelerator, and require the driver to intervene if they make mistakes.
The major difference is that Tesla pushes their system to operate in a wider variety of conditions, while still putting the same responsibility on the driver, and advertising the system using language that suggests that it is higher than level 2 functionality.
It should be illegal to advertise a SAE level 2 driver assistance feature as “self driving” (regardless of how much fine print you have); by definition, it never drives itself.
There's a conflation of issues here. I agree that "FSD 'beta'" is pushing it as advertising goes, and probably on the wrong side of the line. I don't accept that advertising copy makes operating a car more or less dangerous.
Tesla's system does more than the average level 2, so it's more useful. This may also mean operating it is more dangerous, but that isn't a given. Has it been demonstrated? If it has, I'd like to see that.
It seems to be mostly taken for granted in this sort of conversation, and it seems like if it were demonstrably the case, there would be a pithy blog on that subject online. I can only say I haven't seen that, but it could easily exist, this isn't something I focus on.
> I don't accept that advertising copy makes operating a car more or less dangerous.
Advertising copy doesn't alter the car's safety, but it can make the driver of a car more or less dangerous. Tesla sells their cars as being capable of fully driving themselves, hands people a contract with a bunch of fine print saying "actually you still need to pay attention", and then acts shocked when drivers respond to the flashy marketing rather than to the fine print.
It's absurd that people still defend Tesla after all that marketing and place full blame on the drivers. Yes the drivers were careless, but Tesla intentionally created an environment in which drivers could believe their cars would take care of themselves!
Mercedes is saying that about their L3 system, which is not yet available. Until that system is available and Mercedes has acted on their stated intent, then drivers are responsible as 40four said. And from your article: "It isn’t clear whether the company can legally take over liability from drivers."
I focus on Tesla's problems because Tesla deliberately markets their vehicles as driving themselves. They can have all the fine print they want, but when their marketing touts their "full self-driving capabilities", people will treat their car as though it can drive itself... fully.
Compare the marketing text for Tesla[0]:
> Tesla cars come standard with advanced hardware capable of providing Autopilot features, and full self-driving capabilities—through software updates designed to improve functionality over time.
to the text for Mercedes[1]:
> Driving assistance systems in S-Class: Intelligent Drive Next Level. With new and considerably extended driving assistance functions, the new S-Class will this autumn be taking a further, major step towards autonomous driving.
One of these is marketing responsibly in a way that makes it clear that the driver is still supposed to be in control, and it's not Tesla. In the marketing pages linked below, Mercedes uses the word "assist" 35 times and in 9 out of 14 headers. Read through Tesla's headers, in contrast, and you get "autopilot", "autosteer", "full self-driving". You come away with the impression that the car drives itself.
I think its also good to note that a lot of the other automakers were previously very conservative about their marketing claims and were getting more aggressive in response to Tesla's overtly aggressive claims. I'm sure a lot of execs/boards were a bit envious of the Tesla PE multiple that Tesla was getting based on the overly aggressive marketing claims.
> And I didn’t call it ‘Autopilot’ here on purpose .... Why do we all tend to focus on, and demonize Tesla’s version?
My guess is that it's because Tesla themselves are calling the advanced cruise control system "Autopilot".
This risks presenting drivers with a level of misconception, and trust that this system does not deserve.
You can tell somebody (including Tesla owners themselves) plainly that this is no different than the other "advanced cruise control systems", that they should not put too much stock in relying on the system to drive themselves, and that they must be even more alert; however, I think there's a cognitive view in place -- even if you are told the contrary -- that causes people to think subconsciously that Tesla's cruise control is smarter. Almost like the car has a built in "Autopilot."
Technically in the car and owner's manual "autopilot" refers to the suite of features[0] and "Autosteer (beta)" [1] refers to the lane keeping system, and enabling it pops up a warning about staying focused[2].
- If you go with the idea that Autopilot is dependable then it is Tesla's fault. The system is supposed to prevent that from happening and it didn't, killing two.
- If you go with the idea that Autopilot is just an assistance then it is the driver's fault, but it also means that the system doesn't bring much value. If you have to pay attention at all time, correcting mistakes, then you may as well just drive.
>If you go with the idea that Autopilot is dependable then it is Tesla's fault. The system is supposed to prevent that from happening and it didn't, killing two.
the system is very clear that you should be monitoring the car while driving, all the time.
>If you go with the idea that Autopilot is just an assistance then it is the driver's fault, but it also means that the system doesn't bring much value. If you have to pay attention at all time, correcting mistakes, then you may as well just drive.
Value is a subjective proposition and not inherent to anything but your own judgement.
How many wrecks and deaths have been caused by other manufacturer’s systems? It’s got to be happening.
We actually do have the numbers on this, because other manufacturers actually report this data.
Tesla has more self-driving accidents, and more fatalities, than the entire rest of the auto industry, worldwide, combined.
This will continue to be a problem, not just for Tesla, as the tech becomes more widespread.
This is uniquely a problem for Tesla, not the industry, because the rest of the industry took safety issues into account when designing their systems and testing them. Tesla deliberately did not for profit purposes.
At the low end of automation, there is the long precedent of cruise control being legal. (I assume this applies to TACC also; maybe some lawyer can weigh in.)
Likely almost everyone agrees a excellent-human-driver general-AI grade of full automation should be legal; with a to-be-determined degree of manufacturer liability when a collision occurs.
All of the dispute between those extremes boils down to this:
Should there be a range of automation that is illegal to make and/or use? What should that range be? What current cars fall in the range? Should the illegality be based only on the technical performance, or also on the branding? When such illegal use happens, who all should be liable / punished?
Here is my issue with all of this. It very clearly states on tesla's website that "Navigate on Autopilot is designed to get you to your destination more efficiently by actively guiding you from on-ramp to off-ramp, including suggesting lane changes, navigating motorway interchanges, and assist in taking exits."[0]
Its unclear which autopilot features the driver thought he had, but if believed he had purchased the "Enhanced autopilot" and that it was enabled wouldnt that mean the Tesla's system failed here? The intersection in question is at the end of a freeway[1] which would constitute an exit.
I do agree with most that the driver is partly to blame, but probably not as a felony manslaughter. Tesla on the other hand should be held criminally responsible since at best they are misleading customers into believing their car has features it doesn't and at worst they are claiming a level of system reliability in their product that just does not exist (something I would classify as gross negligence on their part).
My father has one of these cars with the enhanced self-driving enabled, and I am always gripped in mild terror whenever I ride with him, regardless of whether it is the car or my father who is driving...
Repeat after me: All AI's are guilty until proven innocent.
I've been saying this (and getting downvoted) for years: AI is a tool. It is no different from any other tool in that the person using it is responsible for whatever it does. If something happens when an AI is involved, it should be presumed to be at fault. And thus the user of the AI is also presumed to be at fault.
This comes into play specifically with the "child in the middle of the road" scenario. At some point our vehicles will become good enough to make actual life and death decisions. Does it swerve into a tree, into oncoming traffic, or take out the child? The only moral, ethical and practical answer is: An AI should choose to harm its user rather than anyone else, because there can never be 100% certainty that the AI did not make a mistake.
You choose to use Tesla autopilot and it kills someone? You are responsible. Period. Feel free to sue Tesla for selling you a faulty tool, but you're still culpable for it's use.
363 comments
[ 1.7 ms ] story [ 814 ms ] threadCan my neighbor below sue my washer manufacturer if they got flooded?
That’s the billion dollar question that will be relitigated for years as autonomous systems improve. No one knows.
Edit: I thought this is in Germany, so I'm wrong. My bad.
In civil law jurisdictions, this works differently.
https://www.nytimes.com/2022/02/15/nyregion/sandy-hook-famil...
The NYT describes the plaintiff's argument as focused on Remington's marketing strategy:
> The families contended that Remington violated state law by promoting the weapon with an approach that appealed to so-called couch commandos and troubled young men like the gunman who committed the Sandy Hook massacre.
> Nicole Hockley, whose 6-year-old son Dylan was killed, said the documents included in the settlement were crucial — and “paint a picture of a company that lost its way, choosing more aggressive marketing campaigns for profit.”
To sue someone, you need to have incurred damages of some sort and file a complaint alleging that who you're suing is responsible. At a high level, you can sue anyone for anything.
Legally the driver bears the responsibility, no?
This is a really vague question. Bears the responsibility for what, exactly? Are they solely responsible? These are things that exactly get sorted out in court...
Can my neighbor below sue my washer manufacturer if they got flooded?
Yes.
This is false advertising, and apparently it may be involved in turning Tesla drivers who believe it into negligent killers. The driver may be able to argue that Tesla told him the car would stop, and so there is no negligence on his part.
However, we've made getting licensed to operate a vehicle way too easy. We should also make drivers get training on equipment type so switching from a mid-sized sedan to a gianormous SUV requires more than just being handed new keys.
The extent of training increases the investment a person has around the regulated activity. The person is more likely to abide by the rules because (a) they have a deeper knowledge, and (b) they don’t want to lose the privilege of being able to carry it out.
Increasing the strictness of license requirements could be a small nudge towards decarbonization to slow the growth of personal vehicles on the road.
Sadly, I'd have to disagree. I think all this would do would be to increase the number of drivers operating illegally. Increase the friction, and people will just find a way to avoid it all together. The only real way to stop it would to have mandatory checkpoints. The push back from the masses for any kind of change like this would be enormous that you'd be asking Roe v Who? in comparison.
But honestly I think demands to make a driver's license harder to get largely miss the point. From my own experience, time on the road makes far more difference than training or harder tests. As does, general maturity and attitude which isn't going to be taught by a class.
What does this have to do with anything? Advocating that people that can't afford to take training shouldn't be restricted from getting licensed? With that logic, let's all become pilots for free too!
>time on the road
The rest of this I can almost agree with, but you're glossing over that bad habits become entreanched. Distracted driving from today's smartphone use to the classic putting on makeup while driving, constant changing of the radio/spotify/etc, or just shoving fries/chips down your gullet while driving is never going away no matter how many years you've drive. In fact, the longer one goes with out having a wreck with these bad habits, the more one is emboldened that they are safe.
It's not just teenagers with low hours behind the wheel that do any of the things listed above.
So, you must be the change you wish to see in the world. So in today's view, we're all screwed!!
I think we're in general agreement. My original point was that making getting a driver's license a leetcode interview-level requirement puts more barriers in place but probably doesn't (by itself) mean you have a lot better drivers on the roads.
To be clear, if you can afford it, a high-end driving course is probably both useful and a lot of fun. Though I wonder if it's more useful for someone who has some experience. (I've never take a course though I have done some driving on BMW's track in SC which was a great experience.)
There was a post here a while back where someone built a keyboard that would deliver shocks when it detected improper typing techniques. It wasn't enough to do anything other than make you really want to type well. It was funny that they went through with it, but that's the level of schtick I'm intending by gentle shock.
It is impossible to get insurance, plates, tabs, park your car in your apartments lot without proper licensing.
Everyone eventually messes up and gets pulled over or gets in an accident and then you are basically going to jail.
In theory your cousin can buy and register your car but if you kill someone he will be legally and monetarily responsible.
Logically it would have to be fantastically burdensome to convince many people to drive without licenses.
It is silly to both pretend we absolutely must impliment a law either without the slightest bit of sense or not at all and presuppose that everyone is going to ignore the law when clearly this is actually fairly hard to do.
Based on experience (admittedly biased), plenty of people are driving without a valid license or insurance right now.
That’s not an argument not to increase proficiency standards, but I think people who naturally follow rules (like my spouse) have a hard time comprehending people who would see a rule and then evaluate whether or not to follow it.
If you are operating a car in someone else's name does not mean they are ultimately on the hook. Yes, they will probably be main suspect, but they will/could be able to prove not involved. Easiest would be to say you were using the car without permission.
>Logically it would have to be fantastically burdensome to convince many people to drive without licenses.
WAT! this happens every. single. day.
*I know direct 2nd hand is counter-intuitive, but I mean I personally know people with cars like this.
In Germany, a court ruled that they misled customers on the capabilities of autopilot and banned advertising it: https://www.cnbc.com/2020/07/14/tesla-autopilot-self-driving...
And it is "statistically" which is how all safety measures are evaluated.
It turns out it's a lot easier to not crash on a highway than to do it on surface roads with many stops and starts.
https://www.tesla.com/autopilot
And note that this is the current marketing material.
Edit: It's not mentioned in the article, as far as I see, but it's likely the driver was on the cell phone. It mentions the data shows the driver had their hands on the steering wheel, but that can be easily thwarted, to my understanding. The fact that they didn't brake at all implies they weren't actually holding the steering wheel, at least in a way where they were paying attention. In my opinion, both the driver and Tesla bear responsibility.
> Please note that Self-Driving functionality is dependent upon extensive software validation and regulatory approval, which may vary widely by jurisdiction.
And the rest is marketing copy explicitly written to make it sound more fully-featured than it actually is.
[1]: https://web.archive.org/web/20180211121851/https://www.tesla...
> It is not possible to know exactly when each element of the functionality described above will be available, as this is highly dependent on local regulatory approval.
So while you can argue that the marketing is deceptive, that can't really be read as saying that you could let the car drive itself at that time.
Well I suppose it'll be up to the judge to determine if a relatively recently added disclaimer on the website trumps years worth of bold claims and tech demonstrations.
Tesla can’t have it both ways. Tell everyone that they have autopilot/self driving to sell car and build hype, but then claim that the website says the opposite so there’s no problem.
/s
Autopilot and FSD are two different things ... I'm sure the consumer is fully aware of that /s
The car doesn't ship with autopilot or FSD turned on.
The general public is extremely stupid and believe everything they see in movies/TV. In fact a large part of the population believe that Fentanyl can kill you just by touching it which is the most ridiculous thing ever. Yet people are so paranoid about it that police officers have collapsed in hysteria after touching Fentanyl. [1]
Candy wrappers say "Do no eat!" on them in the Unites States so you can not assume people know how an actual Autopilot works. Now you may say this is because companies are getting sued but that proves the point that you can actually sue someone for being this stupid and win.
[1] https://healthandjusticejournal.biomedcentral.com/articles/1...
Bendix/King autopilot has a feature called Enhanced Ground Proximity Warning System which would be hard to ignore. Some models have "terrain avoidance maneuvering" which would actually prevent it.
“Terrain, Terrain, PULL UP, PULL UP”
Hopefully you were not sleeping. ;)
On a related note, although they usually mean something's going wrong, which isn't cool, I've always been impressed by the way those warnings are delivered in airplanes. The tone and cadence of the warnings really communicate that this is something serious and you should respond to it and also communicates what exactly you should do.
Here's a video of the exact scenario being discussed in a Boeing 747-400:
https://www.youtube.com/watch?v=12jp5G1LWG0
Fighters with FBW like the raptor can't be over G'd because FBW takes precedence over pilot input.
That said, the selling of FSD has, to my knowledge, always been positioned as the selling of hardware that is expected to be capable in the future of FSD. Sometimes you have to read all of the words in a paragraph to get that meaning, but that is the clear reading I get from that same autopilot page:
By the same logic an Hello World Python script has the capability for general AI, i just need to wait enough years then add later add the missing part that accidental is the essential thing.
Nobody disputes the terms of service language, but the advertising language, the intentional misleading names and stats greedy companies use. We are sure lawyers checked those terms of service very well and worked fine for US, seems German judges are not tricked by this trickery though, would be nice if more countries punish missleading advertising.
I imagining new Elon product "Clean Drinking Water" and ads showing people drinking it, BUT with small text that mentions that the "safe for drinking will happen in 10 years after we fix our shit" and fanboys defending the naming because you should already know that Elon is always doing this and you should read ignore the advertising and go to the legal documents about the product to figureut all the real details.
I don't fly but have an autopilot on a boat, same thing. It'll happily hold a heading onto the rocks if I set it. But everyone knows that's what it does, the company doesn't make any claims or insinuations otherwise.
How is it advertised?
The marketing is weird. They mix up current capabilities and future aspirations deliberately.
For instance, here:
> The system is designed to be able to conduct short and long distance trips with no action required by the person in the driver’s seat.
They actually mean that the system is designed such that "in the future" it will be able to do this. The previous sentence is "All new Tesla cars have the hardware needed in the future for full self-driving in almost all circumstances", and you're supposed to carry that context forward into the following sentence.
Another aspiration, which if you weren't reading carefully you might assume was a present capability:
> All you will need to do is get in and tell your car where to go. If you don’t say anything, your car will look at your calendar and take you there as the assumed destination. Your Tesla will figure out the optimal route, navigating urban streets, complex intersections and freeways.
It's intentionally misleading.
https://www.tesla.com/en_GB/autopilot
---
PS. Yes, there are caveats: "Current Autopilot features require active driver supervision and do not make the vehicle autonomous". That doesn't make the rest of the marketing copy clear.
https://mobile.twitter.com/elonmusk/status/79991021385159065...
"Full Self-Driving Hardware on All Cars
All Tesla vehicles produced in our factory, including Model 3, have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver."
Now, I can see how in a court of law you can make various arguments about what that promises, but I can see how someone would interpret that as it can drive better than they do.
There are plenty of laws. Most products come with more pages of safety warnings than are in the rest of the user's manual. And, in California, you're informed that pretty much every building you enter or product you use causes cancer.
That it's far safer than the average human driver?
If anyone would read that as the car driving autonomy being far safer we would have fatal crashes every day.
This is in direct contradiction to Google/Waymo's research on the matter.
Does Tesla have research to back up this claim? Of course they do not.
Musk is in Brazil as we speak claiming the cars will be driverless within the year.
"An autopilot is a system used to control the path of an aircraft, marine craft or spacecraft without requiring constant manual control by a human operator. Autopilots do not replace human operators. Instead, the autopilot assists the operator's control of the vehicle, allowing the operator to focus on broader aspects of operations (for example, monitoring the trajectory, weather and on-board systems)."
https://en.wikipedia.org/wiki/Autopilot
I don’t like this argument, on the grounds that the average driver has no reason to know what actual autopilots do — I certainly don’t — and therefore naturally go by what it sounds like it does.
Even a cheaper autopilot in a Cessna can take you from the airspace above your starting airport to the airspace of your destination, including lining up final approach maneuvers, with zero input from the pilot. Modern planes absolutely can fly and navigate themselves, but for safety, operational, and policy reasons pilots often don't use the full capability of these autopilots.
From Tesla:
"Full Self-Driving Hardware on All Cars"
followed by:
"All Tesla vehicles produced in our factory, including Model 3, have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver."
Self-driving cars doesn't sound like fancy name for cruise control.
That's from Tesla. Who has already faced consequences for lying in marketing.
Even if they weren't trying to mislead people in their marketing materials, these points seem enough to make a case.
What are you talking about? When was the last time you saw a Tesla advertisement?
> Self-driving cars doesn't sound like fancy name for cruise control
Having the hardware or the capacity to be a self-driving car is not equal to being a self-driving car. The hardware must be enabled, which in this case wasn't. The car only had Autopilot not FSD. Please learn the difference.
People say Teslas are safer than humans, but humans who can't see large broad stationary objects would never be granted a license. If you are a robot driver, no problem though apparently. In fact, if you have proven to be recklessly homicidal while driving, your license is taken away and you are likely put on trial. Not if you're a robot! They will clone your flawed sensors and robot brain, and ship it to thousands of people to be deployed on roads across the world.
When your web service goes down you probably find the root cause, fix it, and issue a report about how it's not going to happen again due to your fix. Not Tesla! They are aware of the problem, know exactly how to fix it, and it seems like one man's ego is getting in the way of shipping actually safe autonomous cars. Tesla cars are not safe because Tesla as a company does not value safety as a priority. They value design, whizbang technologies, branding and image, while safety is a distant concern. Elon Musk put more effort into naming the line of cars SEXY than he did to actually producing reliably and predictably safe autonomous vehicles. Musk spent far more time lamenting over the fact he was forced to name his car the Model 3 than the fact it decapitated a human being due to his poor engineering choices.
Teslas on AP are death machines (read: they are machines who have killed people) and we in the public are the unwitting beta testers for Tesla Inc.
I'm with you on this. Nobody deserves this and this software should be in a test lab or closed road course until it's done. And others will argue that you can't perfect automated driving in those situations, and that's correct.
Yeah I can't help but cringe at this. Safety is hard, and you don't get safe systems without trying very hard and deliberately to make them. That's a fact of life and engineering. Trying hard to make safe systems does not look like deploying 2 ton automobiles that are critically blind in the very arena in which they are expected to operate. That's called reckless endangerment in my book.
Which makes it all the more galling and reprehensible.
> makes it clear that your concern isn't really with the engineering limitations and safety implications of this product.
I was concerned about engineering in 2015 when the first decapitation happened. My present screed against Tesla is because they haven't fixed the problem in the intervening 7 years to the point where a second person was decapitated. That shows me the problem is not engineering but organizational. It makes no sense to talk about engineering issues when Tesla does not promote a culture of safety. Engineers are not in control there, hype men are.
The number of accidental decapitations your product gets to make is 1 and that's arguable. The second decapitation is criminal negligence.
And yes, I am incensed by this because as a roboticist, I think it's a travesty that Tesla has taken the industry in this direction.
Imagine if it were your loved one who died in a Tesla using a technology that Tesla knew or should have known to be deficient and intentionally oversold its capabilities. Would you shrug and point to industry statistics? Or would you be mad at Tesla? Be honest.
Frankly, the attitude that Tesla should be allowed to test their beta-quality products on public roads (I certainly didn't agree to participate in the beta test) is what I find to be ridiculous.
But isn't one of AP's selling points is that it's sophisticated enough to mitigate if not outright prevent these kind of crashes? In previous fatal AP-involved accidents, Tesla drivers collided with stationary objects, which apparently are particularly difficult for AP to see. This case seems to involve one of the more common kind of serious head-on collisions: failing to yield to left-turning vehicle which has the right-of-way.
I know AP != FSD, and that AP doesn't necessarily promise it can handle traffic lights. But if it can't detect or react to vehicles slowly moving into the driver's lane...What are the dangerous day-to-day situations that it can confidently and reliably handle? Is it basically just lane control?
The case seems pretty clear cut anyway, the guy had cruise control on and wasn't paying attention to traffic lights. We'll see if he turns around and sues Tesla, pleading ignorance about what "autopilot" could do.
Just like we were in 2014: https://www.theverge.com/2014/10/2/6894875/elon-musk-says-ne...
> . A Tesla car next year will probably be 90 percent capable of autopilot. Like, so 90 percent of your miles can be on auto. For sure highway travel.
I don't see a reference to "full self-driving" and certainly not something that'd stop for stoplights and properly handle traffic signals.
In the past, you could order the cars with or without FSD hardware (on promises of future capability). These days (~2020+? I'm not sure the cutoff), you can order them without or without an FSD hardware enablement/cost, but they all have FSD hardware inside them, and the cost is just about turning it on or not, and you can always later opt to pay the cost or a subscription to enable it.
For cars that do have FSD hardware both installed + enabled (paid-for): if you're part of a smaller select group of opt-in, approved testers (currently ~100K drivers in the US plus a smaller number in Canada), you can enable "FSD Beta", which is a whole different animal than Autopilot and not really relevant to the discussion.
However, when you're using regular Autopilot, the capabilities of Autopilot are different for cars which have installed+enabled FSD hardware, and those which do not (either because you have the hardware but didn't pay the enabling cost, or because your car is old enough that some of them didn't come with the hardware at all if you didn't pay the cost up front or pay to have it retrofitted).
Non-FSD-capability cars, for example, can't detect stoplights on Autopilot at present, while FSD-enabled cars running Autopilot can. There are probably some other differentiators like that as well. Basically, Autopilot-on-FSD-capable-cars is slowly getting minor Autopilot feature upgrades that are only possible with that hardware while they're waiting for the "real" FSD to arrive. Autopilot-on-non-FSD-capable-cars doesn't get these upgrades and is stuck with a more-limited functionality.
I find the comment
"...we also know we are going to be sued...": https://youtu.be/VfyrQVhfGZc?t=3325
interesting grounds, for a lawyer request for internal Tesla emails via a legal discovery process.
To start with, Tesla assumes they will be able to accomplish self driving with the hardware they have. They also assume they need the system to drive on public roads (as opposed to simulations and private roads) to achieve this. They say that data collected from driving on public roads will get them there, even though this is a for-now-unsolved research problem in AI.
Furthermore, Musk seems to claim every year that Level 4 autonomous driving is within 6-18 months, going back to almost 10 years ago.
For now I feel there are too many people letting Tesla stick with their optimistic predictions and statements that they're "doing the right thing". Maybe the New York Times documentary that is about to come out will change this.
This is an absolute joke. There are videos of the autonomous "come pick me up" feature - whatever it's called - absolutely failing. One of them even runs into an airplane and pushes it around. Sometimes I think they're just using GPS with maps and PID loops and not actual AI of any sort. Nothing short of full general AI is going to be able to drive completely autonomously.
Yeah, I remember https://twitter.com/elonmusk/status/686279251293777920
> In ~2 years, summon should work anywhere connected by land & not blocked by borders, eg you're in LA and the car is in NY
Jan 10, 2016
They were full of this in 2016 https://www.tesla.com/blog/all-tesla-cars-being-produced-now....
> We are excited to announce that, as of today, all Tesla vehicles produced in our factory – including Model 3 – will have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver.
October 19, 2016
Mid 2019 he envisioned a million robotaxis by the end of 2020
https://twitter.com/elonmusk/status/1148070210412265473
Those did happen up until 2018
2014 - 90% capable of autopilot, for sure highway travel
2015 - autonomous driving for highways and relatively simple roads
2016 - S/X can drive autonomously with greater safety than a person - generally true, it still messes up for human-correctable situations but in general helps more than it hurts in terms of "crashes where autopilot was in control during impact or within 5 seconds of impact"[0]
2018 - incorrectly predicted - self driving will be 100-200% safer than a person by next year (this talking about their full self driving thing that will do turns and stop signs and whatnot)
the 2016 tweet about NY->SF was also incorrect by the time his prediction needed to be fulfilled, 2018.
0: https://www.tesla.com/VehicleSafetyReport
I don't think they achieved "autonomous" driving in any way, since they require the human driver to be paying attention.
> 2016 - S/X can drive autonomously with greater safety than a person - generally true
Really? I think this is an extraordinary claim given some of the crashes and near-crashes that have happened (and again, the "autonomous" word).
Humans crash all the time.
It would be interesting to see a comparison of how often humans crash vs how often Teslas crash while the car is driving "autonomously".
I would not be at all surprised if humans fare worse, as there are plenty of horrible drivers, angry/distracted/inexperienced drivers, drunk drivers, sleepy drivers, etc...
Again, Tesla cars cannot drive autonomously by their own admission (they always require human attention), so this comparison isn't possible.
Even if that weren't the case, there's also the issue that people only enable Autopilot in cherry-picked situations, which don't statistically represent general driving situations.
Tesla instructs humans to be attentive while this feature is engaged, but obviously the car will keep driving even if the human's attention is elsewhere.
So Teslas do drive autonomously.. just not always as safely as (some) humans do under certain conditions.
Same with ordinary cruise control, which will keep the car going even when the human driver's attention is elsewhere, their foot is off the gas pedal and their hands are off the steering wheel.
While minor accidents are obviously much more common, in the US, there's approximately one fatality per 100 million miles driven. And that's across all drivers across all cars including old clunkers in poor mechanical condition and lacking modern safety systems.
On the other hand, I don't think I will believe the problem to be solved before I see a self-driving bike. I mean, I would propel it, while it would steer without my input (other than setting destination). Once I do, I will have near total confidence in robo-taxis.
The catch is that the situations where cars drive autonomously are easy, when the situation becomes complex, the autopilot bugs out and tells you to drive. Furthermore, the stereotypical easy case is on the highway, where you rack a lot of miles quickly, and statistics are typically "per mile driven".
Once you remove the bias, Tesla's self driving is not safer, I think I have seen a study that says that in the situations where Teslas can self-drive, they are on the same level as a human driver.
Maybe link to those studies. The one plus Tesla (and other ACC+lane keep systems that are obvious, as in 'loud beep when the system disengages'; my Honda notoriously gave the quietest beep whenever it lost its tracking of lane lines) has is that they don't look at their phone, don't get drowsy, and don't get distracted. So while many situations come up where a human could've prevented a crash, eg running into an overturned semi[0], it leads to about 3x less crashes compared to only passive safety features [1].
0: https://electrek.co/2020/06/01/tesla-model-3-crashing-truck-...
1:
> one crash for every 4.31 million miles driven in which drivers were using Autopilot technology (Autosteer and active safety features). For drivers who were not using Autopilot technology (no Autosteer and active safety features), we recorded one crash for every 1.59 million miles driven.
> we count any crash in which Autopilot was deactivated within 5 seconds before impact, and we count all crashes in which the incident alert indicated an airbag or other active restraint deployed.
https://www.tesla.com/VehicleSafetyReport
Here: https://engrxiv.org/preprint/view/1973/3986
Discussed on HN here: https://news.ycombinator.com/item?id=30182141
(The one that didn't was the infamous Uber self-driving pedestrian fatality.)
Sure in 2022 - we (including Tesla engineers) are all comfortably able to say that Autopilot is really just fancy cruise-control and the driver really needed to have been using FSD (cuz that's full self driving). But at the time - if my memory serves correct Autopilot was being marketed as the feature state of FSD is today.
How it's going: People died.
I used to think the same, but after some years, I came to the realisation that Tesla is just the Silicon Valley's idea of a car company. They have been releasing unfinished products with glaring quality defects for a while now, and they have overpromised almost since the very beginning.
I honestly can't wait for more car makers to go fully electric, so consumers have better choices.
So, yes, over the coming years I expect we'll see much more innovation in this area from established manufacturers (or even other new ones). But not sure how Tesla itself will play out.
You assume they won't try and copy the Tesla product development cycle as well.
I think Ford, GM, VW, etc. have been on the end of enough accidents, recalls, and lawsuits to understand the dangers and consequences of what they’re doing and have their own safety standards even if all government standards went away overnight.
So I don’t think it will happen for them.
Tesla is not operating that way. I don’t see how they don’t end up at a reckoning.
This probably isn’t it. But it seems like it’s coming.
When some of us warned of this shit from the start we were called luddites, impeding progress, boomers afraid of the future.
I still want my promised hoverboard, instead we get an upgraded cruise control DLC laden car that kills people while claiming you can watch movies and sleep while it drives you.
It is about time that this feature is being under investigation.
In those clips you see the fsd computer avoiding accidents form the sides and the back, cases where we humans cannot see before it’s too late.
It’s not perfect yet, but it will be superhuman in term of safety in the near future.
That is exactly what this case (a routine felony manslaughter trial) is not doing though.
In the submitted article, it says a Tesla engineer testified that Autopilot was activated "about 20 minutes before the crash". I assume that means it was active 20 minutes before, and into the moment of impact. If this case does actually reach trial, I guess it'll be interesting howhat the jury will be instructed in weighing the use of Autopilot against the allegation of "gross negligence"
[0] https://apnews.com/article/tesla-autopilot-fatal-crash-charg...
Did you just invent a new word or have I been living under a rock?
>Riad had initially set his car’s speed to 77 mph; he then set it to 78 mph, and then finally changed the speed to 75 mph. At the time of the crash, the car was going approximately 74 mph.
For me, I’d convict. This isn’t a case of ‘autopilot’ as much as it’s a case where a person purposely (with intent) set his speed above legal limits and did not maintain the attention while doing so.. causing the death of two individuals on their first date.
Tragic.
(It turns out that's possible. California provides a GIS data digest of fatal accidents maintained by UC Berkeley. Data collection ends at 2021, but the 2016-2021 data set shows 22 crashes coming up on that intersection. https://tims.berkeley.edu/tools/gismap/).
Or we could limit the assignation of blame to the individual who is solely responsible for controlling his multi-ton high-speed vehicle at all times when operating in public thoroughfares.
... which is not to say they shouldn't do it. In fact, in the 21st century with ubiquitous communication we could even consider mandating that of all cruise controls. But if we were to codify that into law, we'd want a data source a touch more reliable than whatever Google, Apple, an Waze have cooked up as the speed limits on each road in the map (since those limits are currently at the safety threshold of "advisory to estimate travel times," not "someone could die if these numbers are wrong").
all the maps have built in speedometers with an alarming red rectangle when you are speeding.
the data for what the speed limit is is generally public, or very obviously derivable from principles-first, common sense engineering
autopilot has more than enough data to know its not on a freeway, and shouldn't be doing 60mph+ between nodes on a graph
By "all the maps," do you mean "Apple, Google, and Waze?" If so, see my previous statement; they're offering a driver-assist, but they're not ultimately responsible for your safety so their data doesn't need to be fully accurate. Tesla is pulling its data for the in-car map from Google. Nobody is going to try and sue Google if they're in a crash because "the little red box that tells me I'm speeding didn't show up, Your Honor." And Google has made clear that it indemnifies itself from such liability in provision of the map data (i.e. "You're holding it wrong" if you wire Google's idea of what speed limits are directly into the control plane a life-or-death machine).
So Tesla would have to take ownership of that problem directly... Put its own fleet of cars on the road to drive around and photograph speed limits. They could actually hypothetically do that given the cameras on the Teslas if they push data from the driver back to HQ. What's the data licensing agreement look like on owning a Tesla? Does it already grant Tesla that privilege by virtue of someone owning their car?
> the data for what the speed limit is is generally public, or very obviously derivable from principles-first, common sense engineering
Public, yes, but must be consolidated, validated, and kept up-to-date. And the principles-first "common sense engineering" does not describe how America's roads are actually labeled and structured, sadly.
> autopilot has more than enough data to know its not on a freeway
Does it? I don't think Autopilot has any concept of what road it's on; it's not consolidated with the Tesla's navigation map. It's viewing the road directly and looking for local lane markers and obstacles.
Such a consolidation with the global data to understand current speed limit would imply new design safety constraints for both the Autopilot and the onboard map systems.
It's also sometimes unclear about where transitions happen.
It also never accounts for actual speed of travel diverging substantially because citizens including Police have collectively decided that a different speed is more reasonable.
Basically such a system would probably be completely useless enough to ruin all cruise control.
do you actually drive in the US?
there's no shortage of stretches of perfectly straight road where the speed limit drops for no apparent reason.
Do you mean they're actually reading the signs as they pass on the road, or they have a map associating the current position on the road to a speed limit for that road?
If the former, the cutting edge of that technology I'm aware of isn't good enough to trust it to a safety system. And that's before we factor in added risks like "speed limit sign was obscured by a passing truck while the car drove past it so the car maintained speed."
If the latter, that's possible but it's not where Tesla is right now. Tesla doesn't own its maps, it's trusting Google, which doesn't certify the speed limits are correct enough in its map data to wire a safety system to that output.
I can't speak to what trucks do, but those are commercial vehicles, and have very different performance characteristics from cars.
That's an extremely strong assertion. Do you believe all car makers are breaking the law by making cars that can drive 80+ mph? By having cruise control that can be set at 80+mph?
In the US, that is an almost universally useless feature. Standard freeway speed is 10 over the limit.
It really depends. In my experience in the Northeast, when speed limits are 70-75 mph in more rural locations, I don't see most cars traveling at 85.
This is likely where we're headed... especially with fully autonomous vehicles.
It'll probably still be possible to hack them to go any speed you want, though... and certain cars like emergency and military vehicles will have no limit.
https://tims.berkeley.edu/
https://www.google.com/maps/@33.8729084,-118.2881048,3a,75y,...
I'm also extremely surprised that there's a 20 mph drop. While it's only a recommendation, the Uniform Vehicle Code and Model Traffic Ordinance suggests never changing the speed more than 10 mph at a time, and having no more than six such changes in a mile.
See page 28 of this PDF, as I was looking for this information specifically in relation to the Florida DOT, which does a pretty good job of following this rule:
https://safety.fhwa.dot.gov/speedmgt/ref_mats/fhwasa1304/res...
like a stop light?
The purpose of a traffic calming measure is to force slower driving via involuntary driver behavior or physics. Things like painting obstacle-like objects in the road to startle unattentive drivers, rumble strips, unnecessary curvature onto soft embankments so failure to make the turn doesn't risk injury, unnecessary narrowing of the road via cones or soft baffles (because people will subconsciously drive slower when they feel like their vehicle is inches from a barrier).
A common mis-design of highways is to assume traffic lights are sufficient for that purpose, and the result is creation of a high-crash location.
Has a traffic engineer ever been held legally responsible for accidents that result from their design?
In this sense the term “engineer” is someone that requires a license and training (like MDs or lawyers). States require engineers to “stamp” designs that they are taking responsibility for. Sometimes “city planners” LARP as (unlicensed) civil engineers and make designs that get built, which is somewhat problematic but happens.
Stamping and signing off on a design means legally accepting liability. How often they get prosecuted is rare. One famous example is Boston’s Big Dig fatal tunnel collapse. Of course politics and money had a big role in that case.
None. The answer to how much culpability the traffic engineer should have is "none". Freeways end that's what they do and there's no expectation that they won't have a light, or a traffic jam or a deer or some other obstacle in front of you that you'll have to stop for. Lights crossing two-lane highways happen all the time outside of urban areas.
Drivers are, appropriately supposed to be alert enough to see a red light up ahead. Of the twenty-two accidents you cited how many do you think would have been prevented with more signs?
For that matter it's not the autopilot's fault either its the fault of the idiot driving (or not driving as it were).
And when you dig into the five-year numbers, you find that while 22 is quite a bit, there have been more on the straightaway highways north-southbound east of the intersection and on the cloverleafs (If I were to hazard a wild-ass guess, based on California driving experience: driver going highway speed failing to realize traffic came to a standstill on the highway due to back-pressure of people exiting the cloverleaf at rush-hour).
IANAL, but exceeding the speed limit wouldn't be enough for a prosecutor to want to prove felony vehicular manslaughter. Even when DUI is involved, juries can still decide that the driver tragically f-cked up, e.g. underestimated how drunk they were, but not in a "grossly negligent" way. Even though virtually everyone knows that drunk driving is illegal.
But how many potential jury members know or believe that violating the fine print of using Autopilot is illegal or wrong? Or that a Tesla owner should be convicted of a felony because they "obviously" should have known better? I'd assume far fewer in a jury pool have strong knowledge and opinions on Autopilot versus drinking-while-driving.
I can imagine this being a major issue in this case. Elon isn’t known for keeping his mouth shut, especially when it comes to bragging about Tesla. I can imagine that his statements and Tesla marketing being compared to the fine print on Autopilot. Or worse, he tweets counter points live.
Especially complex if Tesla gets involved further as part of data discovery.
Is it understandable for you to set your traditional cruise control at 75mph and not expect things to change for 20+ minutes? I don't think so — I wouldn't trust CC to keep me on the road for more than 2 minutes without my direct guidance. Is it understandable that you might trust Autopilot to handle the road for 2o minutes, because you were confused or misled by Autopilot's capabilities? That feels like a substantively different question.
If you program a robot to commit an illegal act and someone dies, is it the robot manufacturer's fault?
It sounds to me like he was coming off the highway and didn't notice that he was leaving into an intersection with a red traffic light, where the accident occured. Not that I'm saying it is okay for him to not have noticed it.
> Riad was driving on the 91 freeway, heading west into Gardena. The freeway empties out into Artesia Boulevard at Vermont, but Riad, seemingly unaware of that transition, did not appear to react — and neither did his Tesla.
Surface street apparently. Maybe a 45/50/55mph zone.
(He was going 75 on a freeway; the freeway suddenly became a surface street, which is bad design, regardless of whether he should have been paying more attention.)
Not only that, but you can get a ticket for going too slow.
You also usually get tailgated if you're not driving over the speed limit -- just like everybody else.
One of the reasons I hardly ever use conventional cruise control is it makes you drive in a way that minimizes speed changes rather than doing what makes sense in the context of other traffic.
I’m guessing the criminal trial pleads out and the Autopilot issue will get more run in the civil complaints. I wouldn’t be surprised if Riad, the Tesla driver, files his own suit against Tesla to try to foist more culpability on to them.
1. The traffic lane the victims were turning into is in addition to the ending freeway's traffic lanes (i.e. it is one lane further to the right than the right-most through lane). So someone had to be crossing lanes one way or the other for the collision to occur in the first place. Maybe not, I am not a traffic expert or familiar with the area, but that is how it looks to me as an experienced driver.
2. If "Autopilot" was engaged why didn't the Tesla react to the obstruction? Perhaps the timing was too sudden and it didn't have time to reach before being smashed to pieces?
3. Was the victim driver tested for intoxication? Perhaps they were not following all traffic laws as well? Sometimes accidents are the culmination of multiple drivers' mistakes. Not trying to victim blame though. May they RIP.
I think it is clear the Tesla driver was being reckless here because of the speeding. It is unclear if he ran the red light or not. I'd say at the very least revoke the guy's license permanently.
Tesla is, to me, clearly reckless as well with their marketing tactics and their failure to tamp down unrealistic expectations and the hype-machine surrounding "FSD".
0: https://www.usatoday.com/videos/news/have-you-seen/2022/04/2...
The victims were making a left turn to get to the other side of the road to head east, the defendant was heading west.
> Perhaps they were not following all traffic laws as well?
They were the fourth car turning left. If the light was extremely short, maybe the light had turned red and they were late, in which case they were in the intersection illegally. NB that were that the case, they still would have right of way as they had entered the intersection before the Tesla. The defendant would still be fully culpable.
I don't see any situation in which the defendant was not driving recklessly. He was almost certainly driving at 74 mph directly at a red light, unless all four cars ran a red light across multiple lanes. When the light turned green he had at most a second or two to have stopped, if it had not changed. He did not even touch the brake.
Realistically, he was driving with his eyes closed because it was 1230 and he was tired. Emergency braking would not have saved the two people he killed. Evasive driving would have still caused a crash at that speed. The only thing that could have saved their lives was respecting the red light, and the car was not programmed to do that.
Even if the car was fully self-driving, the "hands on steering wheel" thing is a very clear "pay attention" signal. There's a valid argument that complacency is a natural result regardless, but this was not complacency. He had zero awareness of what was happening outside the vehicle, and that is reckless.
https://www.youtube.com/watch?v=-ILRTF_DRSc
That person being the worst driver in the world.
Watch the first video there and how it begins.
The best argument Tesla has to isolate itself from liability is that there is no way that the car could have reacted to the oncoming car and stopped at that speed, even if its AI system were perfect. They would have to dodge criticism about not using LIDAR to detect obstacles early and also the criticism that the AI system didn't notice the obstruction _at all_ and could have slowed down earlier.
All that said, this is not a good fact pattern for the people hoping to come after Tesla instead of the driver. However, the cracks are beginning to show.
There is never a legitimate was to be asleep at the wheel. He was perhaps genuinely asleep, but short of being drugged against your will there is no non-negligent reason to be asleep at the wheel.
Edit: to clarify, the adverb "legitimately" modifies the verb "have been" in this context. You can tell because it is immediately before "have been." If I had said "he was legitimately asleep" that would mean that there was legitimate reason for him to be asleep, because the adverb "legitimately" was attached to the word "asleep." The phrase "he legitimately is asleep" says nothing about the legitimacy of the sleep, but emphasizes the legitimacy of the "is" (as in, he was so asleep that he wouldn't notice his car beeping at him to wake him up).
https://en.wikipedia.org/wiki/R_v_Parks https://en.wikipedia.org/wiki/Sleepwalking#Homicidal_sleepwa...
Then they shouldn't have been driving at that speed.
But I guess it is a top down culture issue perhaps?
Seeing how Elon Musk acts, and how he will chase the shiny thing when he wants attention, it’s all about ego right? Like what’s with the SpaceX always on demand massages for their executives?
Seems like they are losing sight of what really matters, and I can’t help but wonder if this build up of failure is going to cascade into the Musk companies faltering, which would be a big loss as space flight and green transportation are very important (imo)
SpaceX has had onsite massages for a very long time as a comfort perk to all employees -- no different than FAANG perks https://www.protocol.com/bulletins/google-massages-return-to...
> losing sight of what really matters
I wouldn't say so. If anything his companies are doing better than ever. Tesla is doubling revenue / deliveries YoY with insane gross margins and just opened 2 new factories on 2 continents. SpaceX is sending a new batch of Starlink satellites is going up literally every week, and they've sent over 20 people to LEO and back (could you imagine if we had to rely on Russia for this today?!).
Not to mention that the autopilot program is full of a ton of really smart & hard working people who put their soul into their work, because the lives saved by bringing the product to fruition will be astronomical. They are making real practical progress on AI systems -- better than can be said of much of academia & industry.
While I don’t think the driver is without serious fault here, shouldn’t Tesla really be able to slow and stop a car coming off of a freeway and presented with a controlled intersection?
Don’t get me wrong. I own a Tesla and use Autopilot. I also think that the combination of map and sensor data should clearly prevent incidents such as this one.
Maps will always lag reality; I don’t think that it’s reasonable to expect Tesla maps to have perfect accuracy.
The bottom line is, the car is NOT self-driving. If you’re behind the wheel, you’re driving and you’re responsible, just like any other car.
Maps will always lag reality, but this is a commonly driven street, probably not the first Tesla to drive down it-- Waze was using info like this for years, hate to think Tesla is less capable.
In short, there is no excuse for Tesla's maps not knowing about the speed change on this stretch of road.
With FSD beta, turning off a major highway onto my neighborhood street (very narrow and no lane markings), it defaults to 25 MPH. About 100 feet later, the car sees the 35 MPH sign and speeds up accordingly. In this case, 35 MPH is too fast given the curves, slight hills, and limited visibility. I slow down manually in these cases using the roller wheels.
Tesla has gradually improved its speed planning to account for different scenarios. But it's still a work in progress and often requires manual tweaking.
That's why you don't rely on maps as your only input to a feature such as this. Also, is there any information on how long this situation was in the way posted because I can't imagine that they built a city overnight at the end of that transition, it looks like a situation that had existed for many years prior to the accident.
An autopilot in an airplane does not absolve the pilot of responsibility to maintain vigilance and avoid unsafe conditions. Same in the car.
This is false, and I don't know why Tesla stans keep bringing this BS up like it's some sort of justification.
A modern aircraft autopilot is fully capable of taking a plane from takeoff to landing without any pilot intervention required. Pilots frequently use autopilot in flight, but rarely do so for takeoff or landing because it's more stressful to monitor the craft's autopilot system at those moments than it is to simply handle those parts of the flight themselves (and at least for autoland functions, the airport must be certified for autolanding).
There are also tens of thousands of Cessna’s etc. that have the functionality I described and no more (sometimes less).
There's lots of things the media has called Autopilot.
The media, or Tesla? It's not the media's fault if there's confusion on the several marketing terms Tesla has come up with to imply it will drive for you.
Cars are inherently unsafe when the driver is drunk, would you say cars are either safe or shouldn't exist and be illegal?
There's signs for miles back. The speed limit progressively slows down to surface street speeds. If a Tesla can't prepare for a freeway ending then it shouldn't be on the road.
No.
But other car manufacturers are not claiming to “drive you from LA to NYC with zero interventions”
https://www.theverge.com/2021/5/7/22424592/tesla-elon-musk-a...
https://www.tesla.com/autopilot
Autopilot Full Self-Driving Hardware (Neighborhood Short) https://vimeo.com/192179726
i saw this:
“Current Autopilot features require active driver supervision and do not make the vehicle autonomous.”
"The person in the driver's seat is only there for legal reasons. He is not doing anything."
I agree - though - as someone who holds Tesla stock indirectly through various indexes - I fully can understand the desire to want post facto look well past the webpage fold for fine print.
not an insignificant detail. sounds like the driver was a menace.
> [Lawyer] argued that Riad’s actions — driving 74 mph on a surface street — did not rise to “gross negligence” and should not be considered a felony.
At that speed, the driver's momentum was around 67.000 kg*m/s.
That is an impact about 10 times bigger as getting thrown out of a plane and hitting the ground at terminal velocity.
I'd say it's gross negligence to drive that fast at all on any surface street with sidewalks, traffic lights, intersections, stores, whether the driver is asleep at the wheel or not, autopilot or not, drugged or not.
Agreed, but Riad set that speed when he was driving on a freeway, and Autopilot failed to slow down when the freeway became a surface street.
I think Riad was negligent for not maintaining control of the car (Tesla even warns drivers to "pay extra attention to the road" when using Autopilot or FSD), but I wonder if Tesla has any liability for marketing Autopilot and FSD capabilities that don't always work.
https://youtu.be/LEqM2FqUIa8?t=204
Surely the speed limit would change if there's a light-controlled intersection?
Every manufacture has some silly name for advanced cruise control. It’s called marketing.
Plainly false. Consider this guy, who was seen sleeping at the wheel three times, relying on the car to drive itself:
https://www.youtube.com/watch?v=pgnzK07cQkY
The thing that Tesla has named 'full self driving', but which it isn't is a completely separate product from Tesla.
But the point stands that it's bad and not something we should be structuring our society to encourage.
So? Any large truck moving below the speed limit and being perfectly safe can have vastly more energy. And a speeding motorcycle will have vastly less. In a highly-regulated behavior such as driving, negligence is not a function of energy. It is a matter of law rather than physics.
> That is an impact about 10 times bigger as getting thrown out of a plane and hitting the ground at terminal velocity.
That doesn't seem right.
Momentum is mass times velocity, p=mv. Average male weight is around 89kg, 74mph is about 33m/s. p_driver=33*89=2937 kg*m/s.
Terminal velocity of a skydiver is around 120mph, which is about 54m/s. p_skydiver=54*87=4698 kg*m/s.
p_skydiver/p_driver=4698/2937=1.6. So the momentum of the driver is smaller than that of falling at terminal velocity by a factor of 1.6.
Clearly GP's physics experiment makes little sense.
And why are we talking about momentum anyways? A fully loaded tractor trailer weighs about 80,000 lbs (~ 36,000 kg). Therefore, a tractor trailer traveling at 5 mph (~2 m/s) has a total momentum of 72,000 kg*m/s. By your argument, all tractor trailers driving through a city are grossly negligent.
What I wanted to illustrate was that those kind of speeds, in heavy vehicles, in cities, on surface streets, are negligent - spiritually. The sheer amount of damage that can be inflicted, in the blink of an eye, by a moment of inattention, is beyond measure (literally, that 67k number looks goofy, unreal, but it's cold hard physics). Our traffic laws are wrong, our streets are wrong. We don't tolerate that anywhere else. On a factory floor with dangerous machinery there are higher standards of safety than our public realm. This news article is the perfect illustration. Good chance the driver fell asleep, or was distracted in some way (totally normal human traits, btw, and does not make somebody a bad person). Yet the lawyer claims, that at 75mph in a 5000 pound car, this is basically just worth a speeding ticket. That's an enormous disconnect. That delta between responsibility and accountability is totally totally out of wack for drivers. An enormous amount of responsivblity (67k kg*m/s worth of it), and very little accountability ($100 speeding ticket).
And yes, at slow speeds, there's just a lot more slack in the system. The driver of a 5mph tractor trailer has a lot of time to react, so does every one around them, stopping distances are exponentially lower etc... It doesn't change the fact that it will level a house, and so we should have extremely high standards.
It is if you assume the car is a vintage box car from before the 80's where you might or might not have airbags at all. New cars are designed to crumble and deploy airbags to absorb that impact so that the squishy humans inside the car don't experience anything nearly that high. Nowadays the biggest risk is speeding, as it negates the advances in car safety.
> a 5 mph increase in the maximum state speed limit was associated with an 8 percent increase in fatality rates on interstates and freeways and a 3 percent increase on other roads
https://www.iihs.org/topics/speed#effects-of-speed-limits-on....
It's an argument against using some irrelevant momentum number in a comment. Without showing this momentum number, their comment is just "I'd say it's gross negligence to drive that fast at all on any surface street with sidewalks, traffic lights, intersections, stores, whether the driver is asleep at the wheel or not, autopilot or not, drugged or not" which certainly wouldn't get as much engagement on HN.
Even without computing any numbers, purely at the sanity-check level, this calculation is clearly wrong.
Think about it. While there have been some recorded incidents of people surviving a free-fall impact to the ground, that's mostly a certain death impact. Even if survived, you are in very bad shape (understatement). An impact 10x worse than that would completely destroy your body.
Then compare to race car drivers routinely having crashes at speeds higher than 74mph and most of the time hopping out of the car with no injury. Even at much much higher speeds.
The flaw in the argument is summarized in the common saying "It's not the speed that kills you it's the sudden stop". If you hit the ground from an airplane, the stop is very sudden, leading to extreme instantaneous g loads. That's what kills, not the absolute value of the speed itself.
In car crashes there is a lot of energy dissipation due to both structural crumple zones as well as motion of the car itself (it might spin, flip, skid more, etc). These factors lenghten the deceleration time and thus greatly reduce the g loads on the body.
I really hate these kinds of words.
A four lane "basically an interstate if it weren't for the traffic lights every couple miles" type boulevard or state highway typical of urban/suburban areas of sprawl type construction and a two lane residential street are both surface roads. And there's tons of both of those types of roads in the LA area (where this crash took place). There's a big difference between driving 74mph on a somewhat limited access boulevard vs driving 74mph on a residential street.
That's an insane speed to go somewhere that isn't decisively apparent to be a highway.
Riad hit and killed the couple after a left turn from exiting the freeway.
So either speed limit data was missing (in which case the limitation doesn't apply), he had his foot on the pedal and didn't know it (during which Autopilot will not brake; the driver is notified of this if the pedal is pressed down for more than a few seconds), or he turned Autopilot off before the crash (unlikely given the data that's in this article).
I don't remember such a limit on my M3 but will check tomorrow. You can simply accelerate to any speed below AP's global limit (90mph in this case) and then engage AP. It'll keep at that speed. Meanwhile, Model S'es manual doesn't seem to mention anything in this regard.
[1]: https://www.tesla.com/ownersmanual/models/en_us/GUID-5033143...
This is false.
Tl;dr: 45mph or less
IMHO, computers and software cannot, yet, and for a long time, will not, account for all the variables that are part of driving. People that drive these cars really need to understand that computers cannot think like people. Thats my two cents about this, and I hope that this trial puts the brakes on testing this tech in public.
Neither can every person who's allowed to drive.
The question is should we allow semiautonomous/autonomous vehicles that drive at least as good as the worst human drivers who are legally allowed to drive?
If not, why not?
On a related note, I’m starting to think Tesla get a bad rap with all the negative press about their advanced cruise control system. And I didn’t call it ‘Autopilot’ here on purpose. Because there are many many versions of advanced cruise control on the market. Tesla does not have a monopoly on this type of technology. I saw an advertisement the other day for full size pickups that have it now.
The commercial showed a truck changing lanes and passing another vehicle, while towing a trailer, presumably all on its own. That is terrifying to me haha. Why do we all tend to focus on, and demonize Tesla’s version? Is it because it was one of the first, or maybe just the way Musk tend to be a magnet for criticism & scrutiny?
How many wrecks and deaths have been caused by other manufacturer’s systems? It’s got to be happening. This will continue to be a problem, not just for Tesla, as the tech becomes more widespread.
Because Tesla has been claiming fully-autonomous L5 self-driving “will be released next year” for half a decade now.
Not to mention robo-taxis, and the continued lies covering up their insufficient camera/perception hardware. Tesla needs HW5 at least.
FYI: Andrew Karpathy left Tesla a few months ago and is on garden leave. For better marketing, Elon calls it a sabbatical.
Unlike most others, Tesla has been very brazen to describe its system as self piloting rather than driving assists (but see the tons of asterisk at the bottom who clearly state that we're lying in the big letters!). Naming it Autopilot being one of the most obvious cases, autopilot means something and that's not what theirs does.
I know that things like EULA, NDA, and other forms of agreement like that are more easily enforceable there so maybe legally they won't be in trouble, but morally my personal belief is that they absolutely are.
The major difference is that Tesla pushes their system to operate in a wider variety of conditions, while still putting the same responsibility on the driver, and advertising the system using language that suggests that it is higher than level 2 functionality.
It should be illegal to advertise a SAE level 2 driver assistance feature as “self driving” (regardless of how much fine print you have); by definition, it never drives itself.
Tesla's system does more than the average level 2, so it's more useful. This may also mean operating it is more dangerous, but that isn't a given. Has it been demonstrated? If it has, I'd like to see that.
It seems to be mostly taken for granted in this sort of conversation, and it seems like if it were demonstrably the case, there would be a pithy blog on that subject online. I can only say I haven't seen that, but it could easily exist, this isn't something I focus on.
FSD is an egregious misnomer. It doesn’t take over driving responsibilities under any circumstance. “Full automation” under SAE J3016 is level 5.
The dangerous part is the false impressions of capability that they give to some of their drivers.
Advertising copy doesn't alter the car's safety, but it can make the driver of a car more or less dangerous. Tesla sells their cars as being capable of fully driving themselves, hands people a contract with a bunch of fine print saying "actually you still need to pay attention", and then acts shocked when drivers respond to the flashy marketing rather than to the fine print.
It's absurd that people still defend Tesla after all that marketing and place full blame on the drivers. Yes the drivers were careless, but Tesla intentionally created an environment in which drivers could believe their cars would take care of themselves!
[0]: https://www.kbb.com/car-news/mercedes-well-be-liable-for-sel...
Compare the marketing text for Tesla[0]:
> Tesla cars come standard with advanced hardware capable of providing Autopilot features, and full self-driving capabilities—through software updates designed to improve functionality over time.
to the text for Mercedes[1]:
> Driving assistance systems in S-Class: Intelligent Drive Next Level. With new and considerably extended driving assistance functions, the new S-Class will this autumn be taking a further, major step towards autonomous driving.
One of these is marketing responsibly in a way that makes it clear that the driver is still supposed to be in control, and it's not Tesla. In the marketing pages linked below, Mercedes uses the word "assist" 35 times and in 9 out of 14 headers. Read through Tesla's headers, in contrast, and you get "autopilot", "autosteer", "full self-driving". You come away with the impression that the car drives itself.
[0] https://www.tesla.com/autopilot
[1] https://www.mercedes-benz.com/en/innovation/autonomous/the-n...
My guess is that it's because Tesla themselves are calling the advanced cruise control system "Autopilot".
This risks presenting drivers with a level of misconception, and trust that this system does not deserve.
You can tell somebody (including Tesla owners themselves) plainly that this is no different than the other "advanced cruise control systems", that they should not put too much stock in relying on the system to drive themselves, and that they must be even more alert; however, I think there's a cognitive view in place -- even if you are told the contrary -- that causes people to think subconsciously that Tesla's cruise control is smarter. Almost like the car has a built in "Autopilot."
0: https://www.tesla.com/ownersmanual/model3/en_us/GUID-EDA7728...
1: https://www.tesla.com/ownersmanual/model3/en_us/GUID-69AEB32...
2: https://electrek.co/wp-content/uploads/sites/3/2017/02/autos...
I'd like to see others duplicate such tests and a response from Tesla on whether this behavior is as designed.
- If you go with the idea that Autopilot is dependable then it is Tesla's fault. The system is supposed to prevent that from happening and it didn't, killing two.
- If you go with the idea that Autopilot is just an assistance then it is the driver's fault, but it also means that the system doesn't bring much value. If you have to pay attention at all time, correcting mistakes, then you may as well just drive.
the system is very clear that you should be monitoring the car while driving, all the time.
>If you go with the idea that Autopilot is just an assistance then it is the driver's fault, but it also means that the system doesn't bring much value. If you have to pay attention at all time, correcting mistakes, then you may as well just drive.
Value is a subjective proposition and not inherent to anything but your own judgement.
We actually do have the numbers on this, because other manufacturers actually report this data.
Tesla has more self-driving accidents, and more fatalities, than the entire rest of the auto industry, worldwide, combined.
This will continue to be a problem, not just for Tesla, as the tech becomes more widespread.
This is uniquely a problem for Tesla, not the industry, because the rest of the industry took safety issues into account when designing their systems and testing them. Tesla deliberately did not for profit purposes.
Likely almost everyone agrees a excellent-human-driver general-AI grade of full automation should be legal; with a to-be-determined degree of manufacturer liability when a collision occurs.
All of the dispute between those extremes boils down to this:
Should there be a range of automation that is illegal to make and/or use? What should that range be? What current cars fall in the range? Should the illegality be based only on the technical performance, or also on the branding? When such illegal use happens, who all should be liable / punished?
Its unclear which autopilot features the driver thought he had, but if believed he had purchased the "Enhanced autopilot" and that it was enabled wouldnt that mean the Tesla's system failed here? The intersection in question is at the end of a freeway[1] which would constitute an exit.
I do agree with most that the driver is partly to blame, but probably not as a felony manslaughter. Tesla on the other hand should be held criminally responsible since at best they are misleading customers into believing their car has features it doesn't and at worst they are claiming a level of system reliability in their product that just does not exist (something I would classify as gross negligence on their part).
My father has one of these cars with the enhanced self-driving enabled, and I am always gripped in mild terror whenever I ride with him, regardless of whether it is the car or my father who is driving...
[0] https://www.tesla.com/en_AE/support/autopilot-and-full-self-... [1]https://www.google.com/maps/@33.8715442,-118.2917755,17.35z
I've been saying this (and getting downvoted) for years: AI is a tool. It is no different from any other tool in that the person using it is responsible for whatever it does. If something happens when an AI is involved, it should be presumed to be at fault. And thus the user of the AI is also presumed to be at fault.
This comes into play specifically with the "child in the middle of the road" scenario. At some point our vehicles will become good enough to make actual life and death decisions. Does it swerve into a tree, into oncoming traffic, or take out the child? The only moral, ethical and practical answer is: An AI should choose to harm its user rather than anyone else, because there can never be 100% certainty that the AI did not make a mistake.
You choose to use Tesla autopilot and it kills someone? You are responsible. Period. Feel free to sue Tesla for selling you a faulty tool, but you're still culpable for it's use.