2021 Model Y Owner. Hard to tell, but it looks like that street is a one-way heading right which is legal. However, not yielding for pedestrians is no bueno though they were just barely off the sidewalk into the street when it turned.
It's a one-way to the right but there's a no right turn sign on the pillar to the left of the street as you approach it. Not signed great, but still not a legal turn.
I think that sign applies to the left lane only. It's to stop you from turning right from the left lane. I think turning right from the lane the car is in is legal.
There's also a sign on this half of the traffic wire that looks like it might be saying "yes turns" (low resolution, but I can't imagine what the sign says with an arrow pointing right other then that)
That sign says "One Way". Not "yes turns"... I don't think I've ever seen a sign saying that, actually, it's either "No Turns" or no sign at all, since by default turns are allowed.
I've definitely seen signs that say "you can turn here" and "you must turn here", admittedly not with language but with arrows (the "you can turn here" signs around here looking like a forwards arrow with a curved arrow bending off). They're not in general use, only for weird situations (like "two turning lanes" or "yes really this really strange intersection does work like this").
Ahhhhh I see the no right turn sign on the pillar. That is confusing as there are also one-way signs (with arrows pointing to the right) further along.
Moral of the story, do you think most humans would do a better job at this intersection than the Tesla did?
That sign in the pillar, if I was driving, I would assume applied to the other lane on the left side of the pillar. I think the intent is to make sure you don't turn left if you are in the right lane of the pillar side, and vice versa.
The entire talking point of FSD is that it's supposed to be better than human drivers. This ain't it.
>it didn't "veer" either, it performed a normal turn.
The definition of veer is "to change direction". This did that. Stop arguing semantics.
>The pedestrians had just started crossing
I fail to see how this matters at all. Even if the pedestrians hadn't started crossing yet, and were still on the sidewalk, it would have been wrong for the Tesla to make the turn without yielding to the peds.
"You must yield to pedestrians entering or using a crosswalk in your travel path."
At least one pedestrian was unambiguously "using a crosswalk" in the car's travel path, and the other one was entering the crosswalk as well. I wouldn't consider this a marginal case.
I think Teslas are otherwise great cars but I would never use FSD in a million years. I think it should be banned on public streets until it's gone through testing that is far more rigorous.
By the way, the "safer than the mean human driver" bit is irrelevant when you consider the distribution of human drivers. They need to be safer than (say) the 95th percentile of human drivers, which is a far higher bar to clear than the mean.
If you look at the blue track on the screen, the car is indicating it's going to go straight, not that it plans to turn right. I think the word veer is applicable because it is sudden and unexpected. If this was only a mistimed right turn, I might feel differently.
I mean it does look like it got lost, but the blue line in the "this is what I'm sensing and going to do in the next 30 seconds" view started indicating a full turn the instant it started turning.
I'm talking about on the right-hand-side, where it shows the route it's going to take. There's no indication on there it should turn right at this location. It looks like it should go straight through the intersection.
I see the display on the left side, which shows the car's trajectory, and it shows it's going to continue going straight and then suddenly switches to a turn. I would expect it to show that the turn is going to happen sooner, not suddenly switch from straight to turning.
what good is a blue line of what it’s planning, if the blue line changes the instant a new plan is made, such that the driver is surprised their car is making a right turn?
RCWs > Title 46 > Chapter 46.61 > Section 46.61.305
(1) No person shall turn a vehicle or move right or left upon a roadway unless and until such movement can be made with reasonable safety nor without giving an appropriate signal in the manner hereinafter provided.
(2) A signal of intention to turn or move right or left when required shall be given continuously during not less than the last one hundred feet traveled by the vehicle before turning.
Every other state has similar laws.
The Tesla was not signaling before or during that turn. That on its own makes the turn illegal.
RCWs > Title 46 > Chapter 46.61 > Section 46.61.305
(1) No person shall turn a vehicle or move right or left upon a roadway unless and until such movement can be made with reasonable safety nor without giving an appropriate signal in the manner hereinafter provided.
(2) A signal of intention to turn or move right or left when required shall be given continuously during not less than the last one hundred feet traveled by the vehicle before turning.
Do we think that every state in the US doesn't have a similar law?
What is the end-game here? What do you possibly want to argue? The car was perfectly aware it was making a turn, but decided to not use the turn signal because.. peer pressure?
Isn't that just "failing to signal a turn"? If you told me someone made an "illegal turn", I'd think they made right on a intersection with a non right turn sign.
You're right and I noted the section of the washington state driving code that applies. I'm pretty sure that every state in the US has a variant of this same law and picking the wrong one doesn't matter for the purposes of this thread.
Sure, but it's weird to bring up CA vehicle law, specifically, when it's completely irrelevant. You could have made the broader point without the irrelevant detail.
It looks like any entry of the car into the crosswalk would be illegal at that point. RCW 46.61.235, "Crosswalks", states:
> (1) The operator of an approaching vehicle shall stop and remain stopped to allow a pedestrian, bicycle, or personal delivery device to cross the roadway within an unmarked or marked crosswalk when the pedestrian, bicycle, or personal delivery device is upon or within one lane of the half of the roadway upon which the vehicle is traveling or onto which it is turning. For purposes of this section "half of the roadway" means all traffic lanes carrying traffic in one direction of travel, and includes the entire width of a one-way roadway.
The street it was turning and/or veering into is a one-way street abd so the entire width counts as "half of the roadway" and thus any pedestrian in the crosswalk counts as being "upon or within one lane of the half of the roadway upon which the vehicle is traveling or onto which it is turning".
You can only drive into the crosswalk if it is a two-way road, no pedestrians are in the part of the crosswalk on the side that you are driving, there are at least two lanes going the other way, and none of the pedestrians on the other side are in the lane on that side that is adjacent to your side.
Everything about FSD is truly mindblowing. They sold the product but are not letting most users get the version that tries to be autonomous (and thank god for that). It won't be available to most of these people on the car they purchased it for, and can't be transferred to other cars. That has to end in a refund or class action law suit right? And now they have a subscription available? That also doesn't include these features? Who are the suckers paying monthly for this, especially after years of failed promises about this product (that's giving the benefit of the doubt, I think "lies" is more appropriate at this point).
Companies seem to be able to get away with these kinds of shenanigans in some jurisdictions, but what blows my mind is that the Australian Competition and Consumer Commission (ACCC) hasn't yet come down on them like a ton of bricks.
Shitty sudden turn against the no-turn sign without using a turn signal aside, why was it turning right at all when the route showed it going straight for another several blocks and then turning left?
I've dealt with similar but less extreme behavior in lane keeping systems where it senses the right-side reference veering away so it "corrects" and follows, causing a veer to the right, so it might not have been trying to turn but instead drive next to the crosswalk at the same distance it was from the curb. Regardless of what happened, that's a really bad bug to have.
Interesting. This could explain why it doesn't signal. But honestly even veering like that should elicit a turn signal to notify other drivers of the move.
I'm not sure an extra alert would be helpful since it's either going to alert constantly on streets that twist and turn or it's going to think it's doing nothing wrong and not alert. Same for the turn signal.
I think it should. Having a car in front of you means that part of the view of the road is obstructed. IMO the veering car has a moral duty to signal to the next car especially if the veering is unexpected. Now the state law on signaling says "No person shall turn a vehicle..." but I can't find what the definition for "turn" is, so there may be no legal obligation, but I do see a social obligation.
You need to signal when you're leaving the lane or street you're currently on, not when it bends. Nobody ever signals on bends because that's sending bad information. Considering how erratic the car was with signaling and turning I really have no idea what it "thought" it was doing at all.
Nope. Watch the dotted line ahead of the car on the left side of the screen. At the last moment, it suddenly removes the "continue on this road" line, and replaces it with a "go down the road to the right" line. It literally decided to make a sudden turn.
You might also notice that while pedestrians and other vehicles are displayed on-screen, not a single one of the solid concrete pillars are displayed. That car is completely blind to them, and at any moment could have made a sudden LEFT turn into one of them.
Ah, good catch, I didn't see that before. Since it was signaling and displaying a lane change to the left just before that I wonder why it didn't signal a sharp right turn then. Maybe it thought that's how the road went?
If you have to anxiously hover your hands over the wheel constantly like the driver is in this video, is it really helpful even if it doesn't crash? I don't own a Tesla, curious how other drivers are when on Autopilot.
This is the city-driving beta that's only available to a small number of people. I have read that at this point, it is more stressful than driving yourself, but it is not yet at the point where anything more is expected.
Assuming it stays in limited beta until safer than humans (statistically causing 1/10th as many accidents or some acceptable level like that) and until people actually feel measurably more relaxed when using it before it’s released to the public.
Test driving a Tesla Model 3 with autopilot last year was one of the most stressful experiences I’ve ever had. It’s not at all intuitive, and I had no idea what autopilot would/wouldn’t handle.
I feel like if they could manage to get the self driving system to make a plan and stick to it it'd be a better driver already. In so many of these videos you can see the car dithering between two or more options, with the whole car doing these jittery unsure steering motions.
At least if it would pick a damn option it'd have a 50/50 chance of picking the right one.
The cognitive load of using FSD in a city honestly seems more than driving yourself. The car is no longer an extension of yourself, where you navigate the road carefully. Instead you are constantly trying to predict and determine if FSD is going to screw up, and how to quickly react to said screw up.
This is what I don't get at all about autopilot. It's fantastic that it can do X, but if I have to supervise and also take full responsibility, then what's my benefit? That I get to babysit the latest shiny tech? It either takes off the mental load of me paying attention to the road, or I'll drive myself.
Recently got a Tesla without Full Self Driving (FSD) for many reasons (it's $10k for a full install as one). Autopilot is now like a product name that has several features and FSD is a different beast. Within Autopilot, there's an Autosteer (Beta) feature.
The Autosteer seems to work decently well on highway without hover hands, but it does require no sharp turns and any traffic or stop and go isn't what it was built for. I can def. see it nice when doing longer road trips.
That said, I've turned Autosteer on several times and maybe 70% comfortable with it. It does like to hug the right side of a lane, and when I'm in the very right lane on the freeway with on-ramps and exits, there have been times I didn't want to take any chances and assumed full control of steering, or else it drifts into the exit lane and gets back into the right lane, depending on the road lines and how they're drawn. This is with me having knowledge of how computer vision works - I wonder how anyone w/o tech knowledge can put their full trust into these things or to manage the situations correctly.
So even w/o the full FSD feature, I think there's still some work to do on the Autopilot autosteer (it could be a subset of FSD, maybe different ML models all together underneath) - and city driving FSD probably needs even more work.
AGI implies things like "motivations" and "feelings" not just understand of the world (i.e. beyond understanding other things "motivations" and "feelings"), it's a fundamentally different problem. Even if you think you need extremely good world understanding to make a self driving car, AGI is irrelevant (and would actually make it much scary, because an AGI is perfectly capable of deciding to intentionally kill you).
Maybe we need a new catchphrase CAI comprehensive AI as an intermediate goal to AGI.
But then: maybe a car needs to understand the motivations of pedestrians and human drivers in order to predict their movements. And we are almost back to AGI requirements
But there have been plenty of failures, and remember that the only people who have the beta are employees who are presumably under an NDA, and devoted fans who have publicly said they don’t want to post the disengagements because critics “take them out of context”
Why is everyone freaking out over a beta test with a driver at the wheel ready to take over immediately? It's a beta test, finding bugs is kind of the point.
Or have we truly reached the point now where we expect "stable" software to be of the same calibre as beta software so nobody makes a distinction between the two?
I see people making similar mistakes pretty frequently. Not trying to excuse beta testing this software with live pedestrians (like, why on earth did it not signal? What was it's planning software doing?), but humans make this mistake too
Because a car is a life critical system and nobody should want an automated vehicle, even if controlled by a human, to "accidentally" try to run over some humans. I doubt the pilot's ability to stop all missteps before an injury.
The car should be much safer before it goes on the road, regardless of test status. This was a near miss and should be taken extremely seriously by highway safety regulators.
>Or have we truly reached the point now where we expect "stable" software to be of the same calibre as beta software so nobody makes a distinction between the two?
I expect every vehicle on the road to meet a safety standard. I do not accept "beta software" tests on public roads if this is the kind of mistake they can make.
I've been using Open Pilot for the last 3 months (Rav4 Hybrid). I've really enjoyed it a lot on the highways. It also works really well in stop and go traffic.
I'm starting to agree with George Hotz about making driving chill. Highway driving is chill. Stop and go traffic is chill. There's nothing chill about letting a car drive in these situations (pulling out intersections, right hand turn on red, congested streets with people everywhere) and I'm used to my car driving itself. There's nothing chill about your car having unlimited torque on the steering wheel where it can do something gnarly. Making your car do these maneuvers has the opposite effect, making you more stressed rather than taking the stress away.
I'll stick to open pilot for now. I'd rather just drive in these situations. With Tesla FSD you already have to have your hand on the wheel, might as well put your foot on the pedals.
So from Googling, Open Pilot is more a fancy HUD and camera system that assists the driver but leaves all control with him/her?
Did not follow Hotz for a while and the last thing I remember was that he was also just working on another full self driving startup.
This is in Seattle, on 5th ave. The concrete pillars are the monorail track. The street confused me driving as a human sometimes. There are more videos of the Tesla software being confused on this particular street floating around
I believe that it is not illegal to turn right there. Both streets are one way, and the "no right turn" sign is for the lanes to the left of the pillars.
For context, these pillars in the middle of the road are handled poorly by Tesla FSD and this guy is repeatedly stress testing the system by forcing it to drive there, for YouTube videos. The latest FSD build is the first one that actually recognizes the pillars as obstacles; earlier builds would simply plow into them. What I'm speculating happened here is the new obstacle detection failed and decided that the road ahead was blocked, so the path planner decided it had to turn right despite that not tracking the planned route.
As far as the pedestrians, it looks to me like there was plenty of space to turn into the near lane without hitting them, and the pedestrians shouldn't have been starting to cross while the red hand was flashing, but of course a law abiding driver should yield in this situation. I expect many human drivers would choose to turn anyway as long as the pedestrians were far enough away as they appeared to be. But FSD should have stopped.
It is illegal to turn right there without signaling though.
> What I'm speculating happened here is the new obstacle detection failed and decided that the road ahead was blocked
Know what would be able to tell if the road ahead is blocked? Literally any kind of three dimensional sensing that isn't just single frame image object classification. It's a real shame that no one knows how to do that...
Not signaling in advance for what the car likely considers to be an emergency maneuver is a pretty nitpicky criticism. The problem is the unnecessary emergency maneuver, not the lack of signaling.
As for lidar, well lidar isn't perfect either. You might get the same behavior from a lidar car if a trash bag happened to be blown into the road ahead by the wind. All types of sensors rely on machine learning to classify their observations. Lidar isn't a shortcut that eliminates that machine learning dependency.
Finally, calling what Tesla is doing "single frame image object classification" really betrays your bias here. That hasn't been an accurate description of the system for a long time.
I don't care about lidar. What I want is _any_ system that starts from a foundational basis of establishing a geometric model of the environment to determine "flat ground" vs "thing" without first trying to classify its pixels and ignoring the things it can't identify. Humans do that without lidar just fine, but Tesla's failures consistently look like it doesn't even try to establish geometry with its cameras.
That's exactly what they're doing now, predicting depth without relying on object classification first. That's why it doesn't run into the pillars anymore. Obviously it needs a lot of work, but to me it seems plausible that the general approach can work. My biggest concerns are that the cameras aren't good enough and/or that there isn't enough compute on the car.
Better than predicting would be enough cameras to get a stereo baseline. But I'm glad at least that 6 years after launch they're finally maybe sort of thinking about it poorly.
Yes, they really ought to dramatically upgrade their cameras. No argument there. Their stereo baseline is too small and the cameras have different lenses so it's unclear if they can even get any stereo information at all.
The problem for Tesla is that they already sold FSD on cars that have the existing camera setup. I think there's a good chance that they will get to a point where they realize their existing cameras aren't good enough and will have to either come up with retrofit kits or refund owners of existing cars with FSD.
> Finally, calling what Tesla is doing "single frame image object classification" really betrays your bias here. That hasn't been an accurate description of the system for a long time.
I might think differently if the moon hadn't turned into a traffic light a few months ago. But it did. That kind of failure only happens if you aren't sensing environment geometry.
Many of their error scenarios very strongly mimic what you'd get if you looked at a single frame, identified things in it, and then used the vehicle's own motion to hallucinate those things moving by zero-update dead reckoning.
And your analogy is imperfect. How many people died on the ground from airplane crashes from 1900-1950? Probably, not very many. (Why does the difference matter? The passengers got on the plane, knowing - at least approximately - what the risks were. The people on the ground didn't voluntarily accept any risk.)
I assume everybody understands there WILL be deaths due to AI making some kind of mistake.
It is just a consequence of billions of cars making trillions of trips.
But at the very least we should have a clear, reliable solution for all the easy cases. Let's summarize:
- clear, sunny day -- check,
- but not blinded by the sun -- check,
- slow speeds -- check,
- clearly marked intersection -- check,
- pedestrians not acting stupid -- check,
- not trying to avoid hitting another vehicle -- check.
Just make sure under no circumstances the car is allowed to turn into a path of a pedestrians like that.
And if it can't reliably detect pedestrians or their path in perfect conditions like these I can only conclude it is far from being ready.
> Just to clarify here. no pedestrians were hit, and no one died.
That because it drove slowly and the supervisor took IMMEDIATE control of the car. But it looked like it had every intention to drive into these people.
> Do you suggest that the government should have made commercial airline travel illegal until there were zero accidents?
No. I suggest that government should ground commercial airplanes when they make stupid software mistakes that kill people.
Maybe I’m missing something but what I saw is common with human drivers (myself included) as well.
I.e. pedestrian at corner sees car and stops, driver sees pedestrian and proceeds into turn because pedestrian is stopped at corner, pedestrian doesn’t wait and starts crossing while driver is still turning because they’re in a hurry, driver panics and slams brakes. Just seems like life at the corner of an intersection in a busy city to me?
I was thinking of this too when watching one of the videos testing whether their car would weave through concrete pillars. if you record yourself colliding with a concrete wall because you were testing beta software, does insurance pay out?
I wonder how well it does with stopping for school buses?
Here in Washington I see people getting that wrong all the time, usually in the direction of stopping when they do not have to.
The rule is that when a school bus has its flashers on and/or its stop sign out you stop unless:
1. You are going in the opposite direction, and
2. The road is either a divided road (separated by a median or a barrier) or has at least 3 lanes.
I see people going the opposite direction often stop on three or four lane roads.
(The reason you do not have to stop on a 3+ lane road or divided road is that the law requires that if school has students from both sides of such a road the school must have separate bus stops on each side, so that no child has to cross such a road. School bus stops can only serve both sides of a road if the road is 2 or fewer lanes).
(I've not been able to find a definitive answer on whether or not a road with one lane in each direction and a center turn lane counts as a 3 lane road).
It's low likelihood that it handles that. There are so many videos of it not handling even more mundane things, like this one, or the ones showing it trying to run in to a road closed sign.
The comments here are funny. Having lived in Seattle, owning 2 teslas and actually watching more than this single clip of a beta software: this is deliberate testing in a crazy corner case of a road. The driver is nervous because of the pillers. I use navigate on autopilot (FSD for the masses) literally every day and it navigates multiple highways, every day. It's awesome. Please continue skepticism and never partake, more room for me at the superchargers.
Attempts to prohibit repairing things is stupid and it is stupid on regulators part to not seeing it progresses.
But allowing live-testing auto-drive technology on public roads is beyound that. Police should just pull over anyone who turn this on and is on the move and revoke driver driving license. Simple as that.
"Oh, but it works for me..." is not how public security standards are created.
I think you're holding Tesla to a perhaps unreachable standard here.
Before I go on, I'd like to digress and come out as a Waymo fanboi. I've been that ever since I read about a demonstration involving the software developers walking in front of the car, blindfolded. I admire that attitude, I really do. They put their feet where it matters.
But back to Tesla. What other than public roads is similar to public roads, in the senses necessary for testing driving capability? I know Waymo tests its software on recordings of public roads, but those recordings were made on public roads too. Maybe Tesla does the same, I don't know one way or the other. It's not an answer, because this approach doesn't avoid public roads, it only extracts more value from each test drive.
It would be possible to set up a test area. Maybe rent one of those "towns" the film studios have, and use stunt pedestrians, also from the film studios. I worry that those stunt pedestrians wouldn't be really like pedestrians, though, so this too doesn't replace street tests, with the real variety of real people.
These things can supplant public tests. It would be possible to regulate this and say, "software may be used for tests on public tests only when the same version of the same software has been run on one million km of recorded test drives" or such. But as far as I can see, not replace.
I've been told that Waymo (and before that, Google) recorded those traces using employees in the driver's seat, but mostly with the computer making the decisions, not the human.
But suppose it was collected by a human. In that case, the first time the software is run completely (all sensors, all software, everything run together) is in the final production deployment. Is that better than testing the very same software earlier, in a beta setting which is identical to production except that sensor data is collected for the development process?
IMO each update should be considered a completely different new software, you would need to get again approval to put the shit on the roads. This is not 2020s video game industry where you ship buggy software as done and do monthly patches that fix some shit and break other shit.
129 comments
[ 89.9 ms ] story [ 3720 ms ] threadProbably regional though.
It depends on where you live. In downtown Vancouver there were arrows at each intersection telling you what directions of travel were legal.
Moral of the story, do you think most humans would do a better job at this intersection than the Tesla did?
The pedestrians had just started crossing, so it was in the wrong, but barely. It's the sort of manoeuvre that you see human drivers do all the time.
>it didn't "veer" either, it performed a normal turn.
The definition of veer is "to change direction". This did that. Stop arguing semantics.
>The pedestrians had just started crossing
I fail to see how this matters at all. Even if the pedestrians hadn't started crossing yet, and were still on the sidewalk, it would have been wrong for the Tesla to make the turn without yielding to the peds.
Contrast it to "running into the pedestrians"...
the entire point of a beta is that it's supposed to be production grade software?
"You must yield to pedestrians entering or using a crosswalk in your travel path."
At least one pedestrian was unambiguously "using a crosswalk" in the car's travel path, and the other one was entering the crosswalk as well. I wouldn't consider this a marginal case.
I think Teslas are otherwise great cars but I would never use FSD in a million years. I think it should be banned on public streets until it's gone through testing that is far more rigorous.
when it comes to hitting pedestrians, i dont think you can shade that in terms of "barely". either it is or isnt
It was on a course to illegally cut off some pedestrians.
I see the display on the left side, which shows the car's trajectory, and it shows it's going to continue going straight and then suddenly switches to a turn. I would expect it to show that the turn is going to happen sooner, not suddenly switch from straight to turning.
[edit] I linked to the law from the wrong state. Here's the right one:
https://app.leg.wa.gov/rcw/default.aspx?cite=46.61.305
RCWs > Title 46 > Chapter 46.61 > Section 46.61.305
(1) No person shall turn a vehicle or move right or left upon a roadway unless and until such movement can be made with reasonable safety nor without giving an appropriate signal in the manner hereinafter provided.
(2) A signal of intention to turn or move right or left when required shall be given continuously during not less than the last one hundred feet traveled by the vehicle before turning.
Every other state has similar laws.
The Tesla was not signaling before or during that turn. That on its own makes the turn illegal.
https://app.leg.wa.gov/rcw/default.aspx?cite=46.61.305
RCWs > Title 46 > Chapter 46.61 > Section 46.61.305
(1) No person shall turn a vehicle or move right or left upon a roadway unless and until such movement can be made with reasonable safety nor without giving an appropriate signal in the manner hereinafter provided.
(2) A signal of intention to turn or move right or left when required shall be given continuously during not less than the last one hundred feet traveled by the vehicle before turning.
Do we think that every state in the US doesn't have a similar law?
It's baffling.
You mean "making the turn illegally"? Potato potato.
This is Seattle, under the monorail (5th Ave).
> (1) The operator of an approaching vehicle shall stop and remain stopped to allow a pedestrian, bicycle, or personal delivery device to cross the roadway within an unmarked or marked crosswalk when the pedestrian, bicycle, or personal delivery device is upon or within one lane of the half of the roadway upon which the vehicle is traveling or onto which it is turning. For purposes of this section "half of the roadway" means all traffic lanes carrying traffic in one direction of travel, and includes the entire width of a one-way roadway.
The street it was turning and/or veering into is a one-way street abd so the entire width counts as "half of the roadway" and thus any pedestrian in the crosswalk counts as being "upon or within one lane of the half of the roadway upon which the vehicle is traveling or onto which it is turning".
You can only drive into the crosswalk if it is a two-way road, no pedestrians are in the part of the crosswalk on the side that you are driving, there are at least two lanes going the other way, and none of the pedestrians on the other side are in the lane on that side that is adjacent to your side.
The Tesla FSD would've caused a t-bone in the first two clips, and a head-on collision at decent speed in the third clip. Yikes.
You might also notice that while pedestrians and other vehicles are displayed on-screen, not a single one of the solid concrete pillars are displayed. That car is completely blind to them, and at any moment could have made a sudden LEFT turn into one of them.
Who then believes it will even be this decade?
Although implementing might be just as hard as self driving in the first place.
At least if it would pick a damn option it'd have a 50/50 chance of picking the right one.
https://twitter.com/28delayslater/status/1437866328950517768...
https://twitter.com/BS__Exposed/status/1437367617468129280?s...
https://twitter.com/TaylorOgan/status/1437385266046586890?s=...
https://twitter.com/bagguley/status/1437328015789367297?s=20
https://twitter.com/TaylorOgan/status/1437405124004089863?s=...
https://twitter.com/BS__Exposed/status/1437043592560791552?s...
https://twitter.com/bagguley/status/1437048381042475008?s=20
https://twitter.com/BS__Exposed/status/1436806907663159299?s...
The Autosteer seems to work decently well on highway without hover hands, but it does require no sharp turns and any traffic or stop and go isn't what it was built for. I can def. see it nice when doing longer road trips.
That said, I've turned Autosteer on several times and maybe 70% comfortable with it. It does like to hug the right side of a lane, and when I'm in the very right lane on the freeway with on-ramps and exits, there have been times I didn't want to take any chances and assumed full control of steering, or else it drifts into the exit lane and gets back into the right lane, depending on the road lines and how they're drawn. This is with me having knowledge of how computer vision works - I wonder how anyone w/o tech knowledge can put their full trust into these things or to manage the situations correctly.
So even w/o the full FSD feature, I think there's still some work to do on the Autopilot autosteer (it could be a subset of FSD, maybe different ML models all together underneath) - and city driving FSD probably needs even more work.
https://www.tesla.com/support/autopilot#capability-features
Maybe we need a new catchphrase CAI comprehensive AI as an intermediate goal to AGI.
But then: maybe a car needs to understand the motivations of pedestrians and human drivers in order to predict their movements. And we are almost back to AGI requirements
But there have been plenty of failures, and remember that the only people who have the beta are employees who are presumably under an NDA, and devoted fans who have publicly said they don’t want to post the disengagements because critics “take them out of context”
Other failures:
https://twitter.com/28delayslater/status/1437866328950517768...
https://twitter.com/BS__Exposed/status/1437367617468129280?s...
https://twitter.com/TaylorOgan/status/1437385266046586890?s=...
https://twitter.com/bagguley/status/1437328015789367297?s=20
https://twitter.com/TaylorOgan/status/1437405124004089863?s=...
https://twitter.com/BS__Exposed/status/1437043592560791552?s...
https://twitter.com/bagguley/status/1437048381042475008?s=20
https://twitter.com/BS__Exposed/status/1436806907663159299?s...
Or have we truly reached the point now where we expect "stable" software to be of the same calibre as beta software so nobody makes a distinction between the two?
The car should be much safer before it goes on the road, regardless of test status. This was a near miss and should be taken extremely seriously by highway safety regulators.
>Or have we truly reached the point now where we expect "stable" software to be of the same calibre as beta software so nobody makes a distinction between the two?
I expect every vehicle on the road to meet a safety standard. I do not accept "beta software" tests on public roads if this is the kind of mistake they can make.
I'm starting to agree with George Hotz about making driving chill. Highway driving is chill. Stop and go traffic is chill. There's nothing chill about letting a car drive in these situations (pulling out intersections, right hand turn on red, congested streets with people everywhere) and I'm used to my car driving itself. There's nothing chill about your car having unlimited torque on the steering wheel where it can do something gnarly. Making your car do these maneuvers has the opposite effect, making you more stressed rather than taking the stress away.
I'll stick to open pilot for now. I'd rather just drive in these situations. With Tesla FSD you already have to have your hand on the wheel, might as well put your foot on the pedals.
[0]https://twitter.com/giacaglia/status/1414605317841702914
[1]https://news.ycombinator.com/item?id=27811853
For context, these pillars in the middle of the road are handled poorly by Tesla FSD and this guy is repeatedly stress testing the system by forcing it to drive there, for YouTube videos. The latest FSD build is the first one that actually recognizes the pillars as obstacles; earlier builds would simply plow into them. What I'm speculating happened here is the new obstacle detection failed and decided that the road ahead was blocked, so the path planner decided it had to turn right despite that not tracking the planned route.
As far as the pedestrians, it looks to me like there was plenty of space to turn into the near lane without hitting them, and the pedestrians shouldn't have been starting to cross while the red hand was flashing, but of course a law abiding driver should yield in this situation. I expect many human drivers would choose to turn anyway as long as the pedestrians were far enough away as they appeared to be. But FSD should have stopped.
> What I'm speculating happened here is the new obstacle detection failed and decided that the road ahead was blocked
Know what would be able to tell if the road ahead is blocked? Literally any kind of three dimensional sensing that isn't just single frame image object classification. It's a real shame that no one knows how to do that...
As for lidar, well lidar isn't perfect either. You might get the same behavior from a lidar car if a trash bag happened to be blown into the road ahead by the wind. All types of sensors rely on machine learning to classify their observations. Lidar isn't a shortcut that eliminates that machine learning dependency.
Finally, calling what Tesla is doing "single frame image object classification" really betrays your bias here. That hasn't been an accurate description of the system for a long time.
Better than predicting would be enough cameras to get a stereo baseline. But I'm glad at least that 6 years after launch they're finally maybe sort of thinking about it poorly.
The problem for Tesla is that they already sold FSD on cars that have the existing camera setup. I think there's a good chance that they will get to a point where they realize their existing cameras aren't good enough and will have to either come up with retrofit kits or refund owners of existing cars with FSD.
I think it's less realize, and more being forced to acknowledge
I might think differently if the moon hadn't turned into a traffic light a few months ago. But it did. That kind of failure only happens if you aren't sensing environment geometry.
Many of their error scenarios very strongly mimic what you'd get if you looked at a single frame, identified things in it, and then used the vehicle's own motion to hallucinate those things moving by zero-update dead reckoning.
The truck carrying traffic lights is another fine example:
https://twitter.com/FSD_in_6m/status/1400207129479352323
Yes, killing pedestrians is not perfect.
I mean, seriously. You are too fucking optimistic guys. Go back to the drawing board. If your program can do that you are way off from the target.
You can't write software on which innocent lives depend like you would write a startup AI showcase.
I am curious how many people died on the first airplanes and commercial airline travel ( from 1900 - 1950's )
Do you suggest that the government should have made commercial airline travel illegal until there were zero accidents?
And your analogy is imperfect. How many people died on the ground from airplane crashes from 1900-1950? Probably, not very many. (Why does the difference matter? The passengers got on the plane, knowing - at least approximately - what the risks were. The people on the ground didn't voluntarily accept any risk.)
It is just a consequence of billions of cars making trillions of trips.
But at the very least we should have a clear, reliable solution for all the easy cases. Let's summarize:
- clear, sunny day -- check, - but not blinded by the sun -- check, - slow speeds -- check, - clearly marked intersection -- check, - pedestrians not acting stupid -- check, - not trying to avoid hitting another vehicle -- check.
Just make sure under no circumstances the car is allowed to turn into a path of a pedestrians like that.
And if it can't reliably detect pedestrians or their path in perfect conditions like these I can only conclude it is far from being ready.
They posted this:
> Yes, killing pedestrians is not perfect.
In a thread about a video in which an autonomous vehicle did not kill any pedestrians.
That because it drove slowly and the supervisor took IMMEDIATE control of the car. But it looked like it had every intention to drive into these people.
> Do you suggest that the government should have made commercial airline travel illegal until there were zero accidents?
No. I suggest that government should ground commercial airplanes when they make stupid software mistakes that kill people.
Oh, actually they already do: https://en.wikipedia.org/wiki/Boeing_737_MAX_groundings
Would you have the same opinion if a self driving car killed your loved one?
I.e. pedestrian at corner sees car and stops, driver sees pedestrian and proceeds into turn because pedestrian is stopped at corner, pedestrian doesn’t wait and starts crossing while driver is still turning because they’re in a hurry, driver panics and slams brakes. Just seems like life at the corner of an intersection in a busy city to me?
Here in Washington I see people getting that wrong all the time, usually in the direction of stopping when they do not have to.
The rule is that when a school bus has its flashers on and/or its stop sign out you stop unless:
1. You are going in the opposite direction, and
2. The road is either a divided road (separated by a median or a barrier) or has at least 3 lanes.
I see people going the opposite direction often stop on three or four lane roads.
(The reason you do not have to stop on a 3+ lane road or divided road is that the law requires that if school has students from both sides of such a road the school must have separate bus stops on each side, so that no child has to cross such a road. School bus stops can only serve both sides of a road if the road is 2 or fewer lanes).
(I've not been able to find a definitive answer on whether or not a road with one lane in each direction and a center turn lane counts as a 3 lane road).
But allowing live-testing auto-drive technology on public roads is beyound that. Police should just pull over anyone who turn this on and is on the move and revoke driver driving license. Simple as that.
"Oh, but it works for me..." is not how public security standards are created.
Before I go on, I'd like to digress and come out as a Waymo fanboi. I've been that ever since I read about a demonstration involving the software developers walking in front of the car, blindfolded. I admire that attitude, I really do. They put their feet where it matters.
But back to Tesla. What other than public roads is similar to public roads, in the senses necessary for testing driving capability? I know Waymo tests its software on recordings of public roads, but those recordings were made on public roads too. Maybe Tesla does the same, I don't know one way or the other. It's not an answer, because this approach doesn't avoid public roads, it only extracts more value from each test drive.
It would be possible to set up a test area. Maybe rent one of those "towns" the film studios have, and use stunt pedestrians, also from the film studios. I worry that those stunt pedestrians wouldn't be really like pedestrians, though, so this too doesn't replace street tests, with the real variety of real people.
These things can supplant public tests. It would be possible to regulate this and say, "software may be used for tests on public tests only when the same version of the same software has been run on one million km of recorded test drives" or such. But as far as I can see, not replace.
Recording a human driver on public roads is very different from testing a novel software product on public roads.
But suppose it was collected by a human. In that case, the first time the software is run completely (all sensors, all software, everything run together) is in the final production deployment. Is that better than testing the very same software earlier, in a beta setting which is identical to production except that sensor data is collected for the development process?