I thought i just read that they were just allowed to put a lot more vehicles on the road, maybe because of a promise that they would be better but now we find that they are not?
This is the standard to which the builders of autonomous vehicles wish to be held. However, this is unlikely in my opinion. I believe instead they will be held to the more rigorous standard of "is there evidence that this is dangerous?"
Note for example today's suit against Meta for alleged harms against children. No such suit against TV aimed at children or e.g. fashion magazines that are well-known to have deleterious effects on children. Autonomous vehicles are new tech, so the bar will be higher.
Specious and reductive. It's qualitative, not quantitative. Even though the latter is easier for people to understand. We have decades of data about human-caused accidents. We know the ceiling is relatively stable. We know an additional 20% of drivers won't wake up tomorrow incompetent or drunk. Anything above zero is bad, but we know and accept the status quo.
But every new model or software update will invalidate what we've observed and know about automated driving.
We're in an era of extreme obfuscation and enshittification by tech companies, even where safety is involved. I'd be more likely to trust self driving tech if I trusted the current form of the tech industry, and trusted that regulatory bodies had the power and ability to oversee them as effectively as when they were first regulating cars.
Of course even that wasn't perfect, but my trust that we can even do that well is nonexistent.
I do not believe "it's as dangerous as a human driver" is a good reason to look past all the problems with self-driving cars. There should be a higher standard than that.
Let's not forget that the companies that develop and lobby for self driving cars do it for profit. Whatever safety qualities they may or may not have are there to deliver and maximize profit. At this point in time.
Public transport is a safer alternative but not as profitable
I completely agree that self-driving and electric cars aren't the answer but investing in public transport is. You won't get an argument out of me about that.
I'm not who you were originally replying to so I definitely didn't think you were implying thing about me (I also didn't think you were implying anything about the person you were replying to). I was just qualifying my opinion on the safer-than-a-human standard as a non-industry one.
“A human driver” is not the standard, not should it be. Humans need training, experience, physical dexterity, and motivation (and more besides) to be good drivers. If you average human drivers you include barely-competent teenagers, barely-dextrous old people, drunk people, etc.
“A highly-skilled and attentive human driver” is a reasonable standard. So-called autonomous vehicles are nowhere near that.
There is evidence that when a self driving car is stalled in the middle of the street no one can actually figure out what they are going to do from moment to moment. No driver to yell at, doesn't respond to honking... should I go open the door and hit some big 'call mothership' button on the dash...Or will it try and run me over...
13 CCR §228.20 (b) (6) – Based upon the performance of the vehicles, the Department determines the manufacturer’s vehicles are not safe for the public’s operation.
13 CCR §228.20 (b) (3) – The manufacturer has misrepresented any information related to safety of the autonomous technology of its vehicles.
13 CCR §227.42 (b) (5) – Any act or omission of the manufacturer or one of its agents, employees, contractors, or designees which the department finds makes the conduct of autonomous vehicle testing on public roads by the manufacturer an unreasonable risk to the public.
13 CCR §227.42 (c) – The department shall immediately suspend or revoke the Manufacturer’s Testing Permit or a Manufacturer’s Testing Permit – Driverless Vehicles if a manufacturer is engaging in a practice in such a manner that immediate suspension is required for the safety of persons on a public road.
Edit: This Vice article [2] says Cruise tried to withhold pedestrian injury footage from the DMV.
> In the Order of Suspension, the California DMV said that the Cruise vehicle initially came to a hard stop and ran over the pedestrian. After coming to a complete stop, it then attempted to do a “pullover maneuver while the pedestrian was underneath the vehicle.” The car crawled along at 7 mph for about 20 feet, then came to a final stop. The pedestrian remained under the car the whole time.
> The day after the incident, DMV representatives met with Cruise to “discuss the incident.” During that meeting, Cruise only showed footage up to the first complete stop, according to the Order of Suspension. No one at Cruise told the officers or showed any footage of the subsequent pullover maneuver and dragging. The DMV only learned of that from “another government agency.” When DMV asked for footage of that part of the incident, Cruise provided it.
I understand why so many people hate Tesla, but they aren't allowed to drive fully automated without a human monitoring them, so not the same situation.
not sure if you're trolling or not but last i heard people simply strapping an exercise/weighted wrist band to the steering wheel which simulated torque being applied to the steering wheel which leads a Tesla to believe someone is paying attention. A quick search on Youtube reveals more than a few videos confirming this.
No I was being serious. Searching turns up defeating the system with someone in the seat but not without. The other commenter provided a video showing it is possible from two month ago.
It may have been true at the time, but now Tesla uses the interior camera to make sure you are paying attention to the road.
If you don’t want to use the FSD beta, you don’t have to activate the camera, or you can put tape over it. With either of these actions the user can opt out but will not have the ability to use the beta.
Be honest though, no one is doing this. At most some hacker messing around, but it's not at scale at all like the other companies. And even then, that's the fault of the driver, not Tesla.
My wife was telling me a new Prius she was driving freaks out and audio alerts if she looks away from the road. That sounded irritating but that car didn't claim to be self driving.
That would be Autopilot or Enhanched Autopilot. Those with the real FSD Beta have to deal with in-cabin camera tracking which includes gaze detection to make sure you are paying attention.
Yes, but the quality of the self-driving Tesla has is 10x less capable than Cruise - I don't know anyone who actually uses "self-driving" with any degree of confidence.
The people behind the wheel of the Tesla are taking full responsibility for the car's actions, whether they know it or not. No such driver exists in Cruise's case.
Just this morning on my bike ride to work, a Cruise car was sitting in the middle of a road after turning right at an intersection. It was almost blocking the road entirely. The cherry on top was the Waymo that was stuck on the other side of the intersection because of this. People were not happy and had to snake their way around the Waymo.
Having ridden in Cruise cars a couple of times, I am glad I was able to ride one while I had the chance. The novelty factor was very high, and while being hard to hail + very slow, it was a great thing to show to visitors to the city. Hopefully these safety issues can be resolved. As a biker, I feel much safer around a cruise than your average prius.
According to Cruise's X(Twitter) account [0], the suspension seems to be related to an incident in which a human driver hit-and-run propelled a pedestrian in front of a Cruise AV and the Cruise vehicle couldn't avoid hitting the person. When that incident was first reported [1], it seemed like a situation where if it was a human driver in the Cruise vehicle, they wouldn't be found to be at any fault. However given this suspension, I wonder if there's more to it than that.
The article you link to says there was more to it! The Cruise vehicle, after running over the woman, stopped on top of her with the rear axle/tires pinning her to the ground, trapping her there. And it just... sat there. I hesitate to believe that a human driver would do something like that.
You could, however, argue that staying in place was the right thing to do; continuing to drive to get the woman out from under the car may have injured her further. A follow-up article[0] suggests just that, and emergency crews ended up lifting the car off of her instead of trying to tow it.
If this is indeed why their permit was suspended, it seems like a questionable reason. Some of the other dumb things Cruise cars have done seem much more obviously egregiously bad. But in this case the crash may have been unavoidable (by human or robot), and avoiding injuring her further by not unpinning her may have been the right move.
This is such a wild edge case though. So we think the vehicle should do what, keep driving, potentially causing more damage?
It's highly situation specific, and I'd expect the emergency personal to make the judgement call on how to free the person (potentially by taking control of the vehicle, or other means).
Oh, I agree. You may have loaded the page before my edit where I expanded on my first paragraph.
(I have my comments appear on a several minute delay because I know I sometimes submit a comment before I've made sure my intent was clear, but I maybe I took too long this time. It's a bad habit, and I know I should slow down, but... yeah.)
Keeping autonomous vehicles on the road will result in thousands of edge cases, and without a clear safety benefit (as opposed to Level 1 systems) its better to stop everything and reconsider.
Honestly, I don't think this tech will be able to ever be fully automated without controlling the environment as well (dedicated lanes). There's just too many variables you can't account for, and too much risk.
Is there a way to measure the (potential) safety benefits without putting these vehicles on the road?
I'm not saying that means we should just put them on the road, consequences be damned, but I'm curious if there are alternative ways to prove them without risking bystanders' lives.
> Keeping autonomous vehicles on the road will result in thousands of edge cases, and without a clear safety benefit (as opposed to Level 1 systems) its better to stop everything and reconsider.
True, but I bet either a Cruise or a Waymo would have stopped rather than hit that woman crossing the street, and they definitely would not have fled the scene like the human driver did.
I mean it did hit her (again), ran her over, then stayed on top of her. That's not a better scenario at all. A Level 1 system can still avoid hitting people.
Circling back to add some info. From this[0] page:
"But the DMV also said it was not made aware that the Cruise vehicle then tried to pull over while the pedestrian was underneath it."
Kind of proves my point. Stopping completely would have been more ok. The issue the DMV took is that it then pulled over, and that cruise withheld that information.
While appalling, this seems to be because the law has been set up to incentivize that behavior, not because the drivers fail to realize they've hit someone and/or panic and freeze up after doing so. Quite the opposite.
And people getting away with it -- even with video evidence -- seems to be an indication of police corruption. Which creates further incentive to do the wrong thing here.
"Video from Cruise reviewed by the Chronicle showed the woman was in the middle of the crosswalk and that both the Cruise taxi and the other car had a green light. The Cruise applied its brakes and attempted to “minimize the impact,” a spokesperson said, but it was unable to avoid rolling over the victim."
So a pedestrian was crossing a crosswalk when they didn't have the green light. It doesn't really make sense why neither the human driver nor Cruise car slowed down in that case and kept driving normally. Both should have stopped despite having the green light.
The DMV press release specifically cites safety-related misrepresentation as part of the basis for the suspension.
It may not be the performance of the vehicle in the incident that led to the suspensions so much as Cruise's performance in the investigation of the incident (or information about Cruise's prior behavior on safety info that came to light through the investigation.)
Yeah I get the sense (and I am absolutely not surprised) that they have been heavily downplaying the safety aspects. Like publishing the accidents per miles driven but only reporting under ideal conditions. I vaguely remember coming across an HN post about this very thing, a paper saying driverless car accidents under similar conditions as humans are much worse that was previously reported or something like that, and that the companies have been massaging the data.
It's definitely unclear to me what to do if a person is trapped under a tire. Driving more may cause more injury or allow more bleeding from the existing injury. I guess a human driver would call 911 and the dispatcher may provide advice on what to do or suggest to wait for emergency services.
I think the bigger issue is the decision to move the vehicle after a collision involving a person not in a car. I've not had the misfortune to be in such a collision, but in the collisions I've been in, I would get out, and inspect the situation before taking any further action. Moving out of traffic is a good thing to do, but especially if there's a person, it's important to consider the situation before further movement. Even if it were just a vehicle to vehicle collision, I'd want to check if someone needed urgent medical attention, and acknowledge the collision with the other driver, make a plan for where to stop to discuss, and check for and clear any obviously sharp debris that may pierce tires of my vehicle or others' vehicle. A lot of that is outside the capability of an automated vehicle... but maybe human review before pulling to a safer place is a good choice especially if a person outside of a car is involved in the collision.
To be fair, it’s not like we hold human drivers this accountable when they cause crashes.
We should be suspending all bad driver’s licenses, AI and human alike. Today we give people passes for crashing while driving badly, and we should not.
While clearly the initial accident was caused by the hit-and-run driver, when the Cruise went to pull over because it detected an accident, it drop on top of the pedestrian AND STOPPED right on her - "hurled her into the path of a driverless taxi that then ran her over, stopping on top of her as she screamed in pain".
This poor woman was involved in two accidents really, and it is fair to shut down Cruise until they address this failure scenario of driving on top of a victim in road.
If you read the follow-up[0], it's not clear that the Cruise car did the wrong thing. It may have been unavoidable for the Cruise car to avoid hitting her after she was thrown in its path, and if it had then tried to continue driving to unpin her, she could have been injured much worse, or killed (say if the axle or another tire had then crushed her chest or head).
These autonomous cars have done some truly dumb things, but I don't think this incident is so obviously bad.
Meanwhile, the human driver who actually hit her first, and then fled the scene, still hasn't been found...
And that is what is the unacceptable failure scenario. It'll be cool in the future, I guess, to have autonomous taxis. But I'm sure glad they don't test these in public where I live.
From the article, it sounds like the Cruise vehicle didn't just strike the pedestrian but stopped on top of her and pinned her in place. A bystander needed to recognize the incident and reassure her until emergency services arrived and mechanically removed her from below the vehicle using the jaws of life.
While not impossible, it's hard to imagine a human-driver scenario that would look quite like that, regardless of how legal accountability might resolve. Would a human driver stop their vehicle on top of the person they just struck? Would they then ignore the person they'd trapped and wait for a bystander to intervene?
I'm not suggesting that this scenario necessarily warrants the suspension, or that most of the same problems might not occur in some human-driver scenario, but it points to some of the subtle differences of human vs automated drivers that come up in exceptional situations. There are a lot of exceptional situations when driving and it very much matters how automated processes navigate them.
You're right, a human driver might not have remained in place. And as a result they would have risked further injury or death.
"“When it comes to someone pinned beneath a vehicle, the most effective way to unpin them is to lift the vehicle,” Sgt. Kathryn Winters, a spokesperson for the department, said in an interview. Were a driver to move a vehicle with a person lying there, “you run the risk of causing more injury.”"[0]
Yes, but the Cruise dragged her for 20 ft first before stopping and pinning her. So it didn’t even remain in place initially, just after dragging her for 20 ft.
I was just trying to think that through a bit myself! Say this situation happens, the car is on someones legs, it realizes it, drives forward and forward is over the head or chest? It's hard to think how you would want to program it to react even if it was aware something was stuck under it. Interesting situation with AV's I'd not considered before.
Updating my comment to include this:
“When it comes to someone pinned beneath a vehicle, the most effective way to unpin them is to lift the vehicle,” Sgt. Kathryn Winters, a spokesperson for the department, said in an interview. Were a driver to move a vehicle with a person lying there, “you run the risk of causing more injury.”
How many surplus people will be killed by human drivers today, tomorrow, next year, and next decade because of this suspension I wonder? No one ever seems to consider them in these decisions[0].
[0]I can see myself not agreeing with the conclusions, but to not even see the question broached in these terms is maddening.
I think there is also a consideration that should be made for traffic flow. Every time I see a report on these cars it's because they are clogging up a street or stopped dead in traffic. Of course no one reports when they are behaving well but at least if some asshole door dasher double parks and fouls traffic flow you can through persuasion or violence resolve the situation.
We -- in the US -- could cut our traffic-related death rate in half (or more!) by just doing whatever it is that most European countries [UK, German, Sweden, etc] do different from us.
The thing is we aren't doing those things. Maybe its because we don't want to, maybe its because we can't. My point is let's not add advanced robotics to the things we can't or won't do: There is a future where 40k automotive deaths/year becomes closer to 40 on the back of self-driving cars, but we can't get to that future[1] with the current level of risk tolerance.
[1]And getting there slower has the opportunity cost of surplus deaths at the current rate minus the terminal rate - a year delay to that future costs 39,960 lives. (A year delay to ubiquitous public transport does as well ofc.)
I'm disappointed to hear this news. So many people are killed by cars every year. We just take that for granted. Self-driving cars have the potential to drastically improve this and save many lives. I worry that these regulators are weighing the costs heavily while ignoring the benefits of innovation.
That said, I don't have access to the data and I don't really know if Cruise has been acting inappropriately. I'm glad that Waymo seems to be generating far fewer negative press reports, which seems to be a good proxy for whether regulators are going to clamp down. So I hope Waymo continues to operate successfully (and expands their service to me in particular) and I hope that Cruise improves their system and gets back online.
[2] Not just fail to stop, but fail to reduce speed when turning off into a heavily congested side street with visual distractions, blind spots and a pedestrian crossing.
For some reason, I can't imagine literally any human driving quite like that?
As a pedestrian I'm not a fan of being an unwitting part of a private company's beta testing. Before this devolves into whatboutism vs the safety of human drivers, my opposition is to adding yet another unknown safety layer to walking down the street.
Usually when private companies are developing products that could kill, drugs for example, it is on the company to exhaustively prove the safety and effectiveness before launching it to the public. It seems odd that self driving cars have been skipping this step.
I haven't seen anything that indicates they don't work well. My anecdotal experience is feeling much safer inside and around them in comparison to vehicles with humans behind the wheel.
There are plenty of anecdotes of AVs behaving terribly, but anecdotes are beside the point. Normally when a company wants to introduce a dangerous new product on the market the burden of proof is on them to prove that it's safe enough. For various reasons tech companies have gotten to do this in reverse by testing on the public to find out how safe they are.
What does the article indicate then? Everything working as intended? I am wondering if the pedestrian that was dragged six meters by a autonomous machine felt much safer.
The AVs do not work well in the edge cases - exactly in those situations where humans can apply common sense and behave accordingly. Exactly the cases that sceptics seemed to be worried about, and it turns out that the sceptics were right.
I hope AVs will not be allowed on public roads where I live in the next decades.
No, the whole point of AVs, like any other product of a for-profit business, is that they make the manufacturers money.
A part of the sales pitch for AVs is that autonomy acts as a safety layer.
As with any sales pitch, the interest of the manufacturer is often more in getting people to believe the pitch than in it being true, validating that it is true is a concern of outside parties. Insofar as AV adoption requires loosening existing rules to allow them, that includes regulatory authorities.
Profitability will be needed for companies to succeed but it's not the point. Do you think Google's main motivation behind starting AV work 14 years ago and incinerating billions on it over the years was really rooted in speculative profit years down the road?
> Profitability will be needed for companies to succeed but it's not the point.
Yes, it is.
> Do you think Google's main motivation behind starting AV work 14 years ago and incinerating billions on it over the years was really rooted in speculative profit years down the road?
Yes, along with a pile of other moonshots. That's what moonshots are for.
There's a lot of profit potential in AVs, and even if you fail at it there is a lot of potential for profitable spinoffs of the research.
And if you fail at both, well, moonshots are high-risk, high-reward.
> Do you think Google's main motivation behind starting AV work 14 years ago and incinerating billions on it over the years was really rooted in speculative profit years down the road?
Honest question: why do you think Google funded this?
> Do you think Google's main motivation behind starting AV work 14 years ago and incinerating billions on it over the years was really rooted in speculative profit years down the road?
yes!!!
imagine if everyone that commutes could watch ads, sorry, use Google's valuable products and services for an extra 3 hours a day
A safety layer that kills people in novel ways... GPs whole point is that companies should have to prove that they are safe, not just assert that they probably will be and then test on public non-volunteers.
It's just that, in practice, it's almost certainly impossible to reach safe self-driving cars without a period of 'public beta testing', you'll just never get enough data and experience.
Not entirely unlike how there's a period for new drivers where they really aren't safe, but there's no way for them to become safe without going out onto the regular roads and gaining more experience.
I have doubts about this too. I remember seeing a Humvee autonomously driving around the CMU campus back in 1998. I get the feeling research institutions and, now, product companies have been working closely with local city governments about safely releasing these into the wild for some time.
"It seems odd that self driving cars have been skipping this step." is an opinion referencing well-known current events. You can't expect everyone commenting on this article to re-reference the saga of Cruise in San Francisco; especially when it's frequently covered on the front page of Hacker News.
Furthermore, you should know that a Hacker News discussion is not Wikipedia. These discussions are not the kind of things where every off-the-cuff statement needs to be backed up with references.
That's a question for regulators. What AV companies are doing is akin to human trials in pedestrian safety without consent. This wouldn't fly in another industry like pharmaceuticals.
Not really. Oharmaceuticals have many different tiers of trials, but even with that, there are still new discoveries and surprises once they reach the full population. Waymo has been testing their tech for over a decade in much smaller and controlled situations. Similarly, all of these companies also have done tests with humans behind the wheel for years. To claim that they've done no testing and are going straight to public testing is just plainly false.
You can argue about how much non-public testing is enough, but any roadmap for AV will eventually at some point have to include public testing.
Perhaps you know more than me about pharmaceutical testing, but people are aware that they're part of a trial, and more importantly, they consent to participating, correct?
I did not claim they're not testing, they clearly are. My argument is that people that didn't sign up to be test subjects are getting hurt in the process.
It's worth pointing out the discrepancy in human safety trials, is it not? Why should AV testing be less rigorous than pharmaceuticals?
The state ultimately is the one that is giving permission here on behalf of people, much the same way as the FDA decides when a medication is safe enough to be administered, or the FAA when a plane is ready, etc.,
It depends what you define as a "test". It's like how for Covid vaccines, antivaxxers were claiming that vaccines were still being tested on people despite it getting past phase 1, 2 and 3. At what point do you consider something to be "safe enough".
My point is that Waymo has done plenty of testing already. The question is, where do you draw the line? Are cars ever "done" being tested? Just like with drugs, you never know all the side effects for 100%, you can only make the error bar smaller and smaller, but eventually you have to distribute it to the public.
Same with the car, at some point they will eventually have to drive on public road. By your logic, they would never reach that threshold since they're still "testing" it.
They dont. This technology, aside from being infeasible at the level it would need to be to bring the claimed benefits, doesn't actually even have real benefits worth this dangerous development. If anything its a desperate attempt to keep private automobiles as the primary transportation system despite more communal, and public, systems being superior.
Its also a USian obsession. Some other places have been able to actually invest in superior public infrastructure but the US is so focused on techno-solutionism it, as usual, misses the extant and superior solutions. Self driving does nothing to mitigate the sprawl and atomization that wreaks havok on our society, but trains and busses, and density that cannot be achieved with car-dependence does.
The large opposition to AVs gives these companies zero-margin for error especially in a place like SF. Hopefully they'll be back on the road soon. My experience in Waymos have felt much safer than the average Uber.
Let’s agree that Waymo and Cruise are not in the same league. I don’t use AV in general anymore, I explicitly differentiate between Waymo and everyone else.
Something like 30,000 Americans alone are killed every year in car accidents. I think the primary idea is to reduce the constant death on the road by making machines that are safer at driving than humans. The toll that car accidents have on society is honestly unconscionable.
That is a good point. A secondary benefit, is a self driving car can literally give most workers that commute about an hour per day of their lives back. For some people it will be two to three hours.
If your favorite hobby is sitting by yourself, strapped to a chair, sure, you get your time back. But the reason I'm allergic to commuting is that I prefer to spend time with my family.
How is being stuck in a small box for an hour "getting that hour back"? Most people don't want to read a book during their commute, they want to have less commute, so they can do actual things they want, like play with their kids or cook dinner or enjoy a movie in the comfort of their own home.
"Fixing" commutes by making everyone sit alone in their car while it does the driving part as if that's the problem with an hour long commute, is the single dumbest way to make cities better.
If Americans were serious about reducing car deaths they'd ban gigantic pickups and SUVs with blind spots big enough to fit a Volkswagen. They'd ban cars from city centers and build bicycle highways. They'd invest in convenient rail infrastructure.
Instead we've got "it's a car but you can doomscroll while on your 45 minute commute and we harvest your data to sell ads".
It's just so stereotypically American to say "here's a problem society faces" and then in the same breath pretend the solution is to enrich the CEOs of a handful of tech companies peddling solutions that may or may not even ever succeed.
>But they want freedom more.
Sure, it's a very individualistic society where one's personal convenience is more important than societal good. But let's call a spade a spade and not pretend that self driving car companies are selfless benefactors out to solve pedestrian deaths.
I do not own a car. Love living in a transit forward city. But I understand the reality, that is not a real solution for most Americans who live in suburbs where good transit is just not possible due to the original layout. Instead of trying to shoehorn a solution that isn't practical we should focus on ones that potentially are.
Everybody says that transit isn't practical because suburbs exist. But nobody said that cars were impractical before suburbs existed (well probably some people did and it turns out they were right)
Better railroads would be a far better solution. But I think getting there will take a lot of convincing in America. So much money has gone into road infrastructure that I have to imagine there's a lot of sunk cost fallacy at play.
Oh certainly, sunk cost, the industries entirely dependent on cars, etc. But we regularly have to rebuild those roads anyways, and how much money is being thrown away trying to make cars less awful?
There's a rural cost/benefit that ensures we will never not need cars, probably. But I can't fathom why eight lane interstates exist. If we can justify that many lanes of road traffic, we can certainly replace six of them with a train.
Self-driving trains? Easy. And no matter how hard you try, even the most efficient electric car will never end up greener than a train on a per user basis.
The phenomenon of viewing public transit as somewhere that you'll be assaulted or robbed is an incredible indictment of the US. I don't know how true it is though.
People are swayed by anecdotes and dramatic headlines and politics and look for reasons to be afraid. People don't know how to measure risk in their day to day lives. It's an indictment of something, but not public transportation.
Assaulted and robbed are the least of my worries. Its a small chance but its there. What I hate about public transit is the public part.
I've witnessed tons of anti-social behavior over the years taking the DC metro and honestly, have never been comfortable ever since I was a child and got stuck between two drunk men arguing after a baseball game. I was feeling claustrophobic and could even feel the spit hit my face. The car was packed and no room to really get away from it. I think its gotten better but I wouldn't know as I haven't used it in years.
- Zero traffic fatalities would be nice
- Reduced congestion/traffic, people can get to places faster
- Never having to worry about parking again
- No more need for cars parked on streets, more space & safer for cyclists and pedestrians
- Reduced income requirements for those that need access to a car
There are countless benefits. The transition period to get there will be a little tumultuous, but we'll get there eventually as a society
They won't reduce congestion and traffic, because people won't have to drive they'll be OK with being on the road longer. You just hang out in your living room on wheels.
Parking... Where are the cars going to park exactly? How far from the restaurants and homes etc? Won't the extra trips from/to parking lots create more traffic?
Reduced income requirements, that's a new one to me. These cars will have more tech that's more expensive to build and maintain. How does that make them cheaper?
50%+ of car trips are less than three miles... as a society what we need is a way to make those trips without cars.
Traffic is a function of how many cars go through a road/intersection per unit time, which is reduced if cars keep shorter following distances, which self-driving cars can do because of faster reaction times.
In the extreme, imagine if a light turns green and all cars in line accelerate at the same time. Intersections would let through a lot more cars.
Or think of freeway speeds where the length of a car is near-negligible. A one-second following distance means almost 60 cars pass a given point in a minute, versus almost 30 cars per minute for a two-second following distance.
> They won't reduce congestion and traffic, because people won't have to drive they'll be OK with being on the road longer. You just hang out in your living room on wheels.
Traffic flows would become well coordinated and way more efficient, no more accordion effect and accidents to cause traffic jams.
> Parking... Where are the cars going to park exactly? How far from the restaurants and homes etc? Won't the extra trips from/to parking lots create more traffic?
It could be several miles from where the passengers are being dropped off. See above for why traffic won't be worse.
> Reduced income requirements, that's a new one to me. These cars will have more tech that's more expensive to build and maintain. How does that make them cheaper?
Of course new technology is expensive initially. Over time, it becomes cheaper and cheaper. Just like air travel, personal computers, broadband internet, etc.
> 50%+ of car trips are less than three miles... as a society what we need is a way to make those trips without cars.
More efficient use of the roadways from self driving cars would create opportunities for improvements in this area as well. Dedicated protected bike lanes, for example.
Besides what the other user pointed out, there are tens of millions of people driving an hour or more every day just to get to work, and that time sink might feel a bit more bearable if you could relax and do whatever you wanted during that time rather than focus on all the obstacles around you. I don't know that the state of self-driving vehicles will be quite there in my lifetime, but some time in the future they will be, if we keep making progress on them.
You might like driving. For many, driving is stressful.
This is besides the point someone else brought up about how notoriously bad at driving humans can be, and the capacity for how much safer an automated car could be when the tech is matured.
Public transit like efficient railroad infrastructure across the US would be even nicer, but I've about given up on that for the USA. They're doing self driving cars because we sunk too much money in roadway infrastructure (sunk cost fallacy.)
> that time sink might feel a bit more bearable if you could relax and do whatever you wanted during that time rather than focus on all the obstacles around you
We could use... trains & buses. Self-driving trains and buses!
But self-driving individual cars, because they will make the time more bearable, will induce even more driving, which is the opposite of what we want.
Oh, I completely agree. This is big tech solving a problem for humans, but ignoring a larger problem for humanity.
To give the benefit of the doubt, my assumption is that they're solving it this way because the road infrastructure and market for cars already exists. You'd see big tech working on automating public transit if states actually started improving their public transit infrastructure.
Meanwhile in reality:
- More traffic due to induced demand as sitting in traffic becomes comfortable
- Self-driving subscription in addition to all other car costs
- More car-centric infrastructure needed as we realize that self-driving cars can't be made fast and safe for mixed street use with pedestrians, bikes etc.
I highly doubt driverless cars will have an actually positive impact with regards to reducing traffic, nor that they can go uninsured, nor that they will be a cheap option for commuting.
They might be safer than human drivers, but there are already superior solutions to that in the form of transportation systems that are not overly focused on cars.
So much is discussed about self-driving car safety because the implications are really big but also uncomfortable. However, I always get stuck on how little enforcement there is for human drivers who habitually break driving laws and cause dangerous situations. DUIs are one thing sure but honestly most drivers who are bad and dangerous are sober and face no consequences for reckless behavior.
Is there something specific that caused this halt on Cruise's testing permits?
Not sure about the specifics here but to jump on your point about human drivers not being safe...
10000% we need safer designs for streets and roads, better education, automated enforcement of speed limits, dis-incentives for larger/heavier/more dangerous vehicles, ...
All of these things need to be done, and can be done with existing technology today. We don't need self-driving cars to address our problems; they're essentially a distraction and an excuse to postpone improvements.
Why the down votes? This seems like a reasonable discussion to be had, unless you're a binary thinker and think that anyone raising a point about humans must be secretly, implicitly arguing that cars should be able to do whatever they want.
Automatic enforcement of driving laws should absolutely be a thing. People often argue "but they can't prove who was driving the car!" Its simple answer to me. Whoever registered the car is by default responsible for the actions of the car.
If that car gets found doing something obviously illegal (speeding, running a red light, some kind of "reckless driving" camera review, etc) the registered owner is automatically assigned blame. They can potentially file paperwork to the court to prove someone else was driving (a lease arrangement, some valet ticket along with some proof from the restaurant or whatever at the time, a sworn statement from another individual confessing to it, etc). But ultimately by default the owner of the car is responsible for the safe handling of the vehicle. If you're worried someone is going to break the law with your car, don't give them your keys. If you don't like this, find another form of transportation.
Tag enforcement needs to be stronger. If an automated system sees a tag that doesn't make sense, it should be immediately flagged and enforcement dispatched. Lots of cops already have ALPRs, those should immediately trigger. I see so many cars out there with sketchy paper tags, this whole thing needs to be shut down.
Driving a car on public streets shouldn't be nearly as easy as it is in the US. Far too many people don't treat driving with the respect it needs.
From my limited understanding, web browsers were improved drastically by the creation of cross-vendor testing (e.g. Acid tests). I wonder if there is a way to do some sort of AV simulator testing that would allow us to test the different manufacturers on data collected from the others.
AVs seem to all have different "mental models" of the external world, so it seems like we've got a lot of work before we can pipe data collected from this Cruise incident into a simulator and test Waymo, Uber, Tesla, etc.
“The Nissan Sentra then tragically struck and propelled the pedestrian into the path of the AV. The AV biased rightward before braking aggressively, but still made contact with the pedestrian. The AV detected a collision, bringing the vehicle to a stop; then attempted to pull over to avoid causing further road safety issues, pulling the individual forward approximately 20 feet. The driver of the Nissan Sentra fled the scene after the collision.”
Really wish there was a way for tech (especially new AI tech) to wholesale abandon the city. Even just move south to the actual Silicon Valley. The city picking winners and losers like this is untenable
I don't know how to respond to this. If you screw up there needs to be consequences that bite. Suspending their operations is better than some meaningless fine and actually punishes them. If you can't see that, I can't help you.
Note this is suspending operations indefinitely with no pathway back. No "take these steps and we'll consider re-instatement". Basically a death knell. Moreover, the city had requested the DMV do this to Waymo as well (the state DMV just declined).
So yea, this is the city working to regulate driving, like it regulates housing and small business i.e. to the death while favoring incumbents that are far worse.
Cruise operates fine in Austin and Phoenix. They could just move their operations to South San Francisco or further south and be better off.
The city simply does not deserve the presence of tech, and for tech companies their presence in the city seems counter-effective
"The DMV has provided Cruise with the steps needed to apply to reinstate its suspended permits, which the DMV will not approve until the company has fulfilled the requirements to the department’s satisfaction."
Reading this, it rather seems like the AV wasn't at fault in the accident itself and handled itself well. However, after coming to a complete stop, while the pedestrian was still on the ground in front of it, it then started again and drove into the pedestrian and continued for another 20 feet pushing the pedestrian along (and presumably under) the car. They say it did this to leave the car in a safe place, but this differs drastically from a normal driver who'd get out and look round the vehicle before moving it anywhere.
I'd say the initial hit probably did a chunk of damage to the pedestrian, but quite likely not fatal unless their head hit the ground. But then being driven over and dragged 20 feet is going to be pretty scary and in all likelihood cause more serious injuries than the initial impact.
> but this differs drastically from a normal driver who'd get out and look round the vehicle before moving it anywhere
That is an optimistic view of the likely behavior of your hypothetical driver. Hit and runs are commonplace. Dragging struck pedestrians around isn't all that uncommon either. Panicked motorists frequently make very bad decisions.
> Reading this, it rather seems like the AV wasn't at fault in the accident itself and handled itself well.
Probably because you're reading their press release. Their previous press release was even more convincing - they somehow forgot to mention the part about draggin the pedestrian at the time.
Here they somehow forget to mention the car has already accellerated to 18mph while a pedestrian was still crossing the road.
I get your point, but on the other hand if a pedestrian was crossing on their when the light turns green and the pedestrian was already clear of the lane, I think most human drivers would also proceed. I'm in a different country but, both as a driver and as a pedestrian, I witness this exact situation on a daily basis.
I'm not trying to detract from the horrible outcome of this case, but the problem wasn't the car pulling away at a green light when there were no obstacles in front of it, but the way the car decided to move itself to a "safer place" immediately after an accident when it clearly didn't have enough sensors or the AI wasn't trained sufficiently on this kind of situation to be 100% sure that it was safe for it to make the maneuver.
"A dark colored Nissan Sentra was stopped in the adjacent lane to the left of the AV. When the light turned green, the Nissan Sentra and the AV entered the intersection. Against a red light, a pedestrian entered the crosswalk on the opposite side of Market Street across from the vehicles, passed completely through the AV’s lane of travel, then stopped mid-crosswalk in front of the Nissan Sentra. The Nissan Sentra then tragically struck and propelled the pedestrian into the path of the AV."
So there was this woman crossing a crosswalk in front of these two cars when she shouldn't have. I guess I don't understand why neither car stopped in front of the crosswalk when they saw this woman in the middle of the crosswalk (illegally).
IMO it was a bit aggressive for Cruise to have seen this woman jaywalking and still crossed the crosswalk after detecting that the woman had just passed the car but still in the middle of the crosswalk.
Have you driven in San Francisco or basically any large US city? If cars had to stop like that to give extra space for a jaywalker that had exited the path of travel, someone would rear end the AV, or at the very least the driving experience would be very jarring. If we wanted to legitimize this case then probably large areas of downtown in cities should be redefined to be free-for-all traffic with low speed limits.
I don't know if this was a homeless woman or not but there's plenty of times I've seen in SF of some mentally disturbed homeless person in the middle of the street.
In those situations I would 100% stop completely in front of them and/or drive very slowly around them. The difference between a regular jaywalker and someone who is mentally disturbed is the mentally disturbed jaywalker is unpredictable and you have no idea what they'll do next.
The right decision in this case was for both cars to at least slow down if not stop in front of the crosswalk. The Cruise car driving normally just because the walker happened to just pass to its left without slowing down doesn't seem like the best decision.
Slow down to a crawl, definitely, but if you stopped until they got off the street completely, you might never move again in some areas of SF (the accident happened near Market Street).
And it's not a given that doing so would make the situation safer, as other drivers would eventually drive around your car.
Normal drivers stop or proceed slowly through intersections when pedestrians are in the road regardless of whether the person is supposed to be there. With great power comes great responsibility.
Anyone who doesn’t slow down in situations like that is a waiting to be like the Nissan driver: a person a-okay with committing manslaughter.
And two wrongs don’t make a right. Just because some human drivers are negligent and horrible doesn’t mean we have to accept robots varieties that are also bad.
That wasn't what I was referring to. This is in reference to a jaywalker. My point is both cars should have stopped when they saw a jaywalker crossing the street.
It doesn't sound like the Cruise car stopped because it basically saw that the woman had just barely crossed to its left out of its path of travel.
A more cautious driver would have slowed down at the very least or stopped completely.
Incidentally, "jaywalking" will be legal in California in 2024, finally. Hopefully that will be the beginning of the end for those laws across the rest of the country.
(Of course, we're talking about crossing "when safe", not when you'd be walking into the path of a vehicle with the right of way.)
That's like the only time in Texas a pedestrian doesn't have the right of way in a marked or de-facto crosswalk; if there is a cross signal and it's red.
> IMO it was a bit aggressive for Cruise to have seen this woman jaywalking and still crossed the crosswalk after detecting that the woman had just passed the car but still in the middle of the crosswalk.
Wat? (Most) pedestrians aren't squirrels that hang out in the road running back and forth. You've put forth an unreasonable standard that basically nobody uses. Except for maybe those "special" types of drivers that will slam on their brakes and wave at you when your intended path was clearly behind their car after it had passed, as if you're supposed to be thankful for their putting themselves in your way to indulge in a simulation of altruism.
A good human driver may have slowed down from perceiving the emerging exceptional situation of the oncoming car not reacting to the pedestrian, but I'd say that's a much different framing than what you asserted.
Presumably this is why the permit is being suspended:
"The AV detected a collision, bringing the vehicle to a stop; then attempted to pull over to avoid causing further road safety issues, pulling the individual forward approximately 20 feet."
First of all, I find it ironic that the human driver is still at large. I'm sure the Cruise car recorded the plate and they passed this to the police.
Second, the corporate speak on the Cruise release is very good. Notice how it says "pulling the individual forward", in other words, the car dragged the person while it attempted to get out of the way. I guess nobody thought about that edge case. And they released a completely irrelevant simulation showing that if the other car had been an AV this wouldn't have happened. Yet, the real issue with the Cruise AV is that after a collision it just blindly tried to pull over and it dragged the person. The same thing would have happened if the person had simply collided with the car on their own.
> I find it ironic that the human driver is still at large
Ironic but completely expected. We have normalized and accepted the awfulness of human drivers to a degree that people just don't comprehend. Human would be terrible drivers even if you ignored the fact that many of them are chemically impaired, even if you ignored the fact that they are prone to rage and aggression.
As we consider the safety of self-driving vehicles, there is going to be a disorienting amount of cognitive dissonance as we are forced to confront the awfulness of what we already accepted and have been living with for a century. There were over 42,000 deaths due to motor vehicle accidents in 2022. That means if we created self-driving cars that were twice as safe as humans, they might save 21,000 lives per year and also kill 21,000 people per year. That sounds insane, but it would be a sane way of improving an insane situation.
Kibitzing about these individual incidents is a normal and inevitable human way to try to deal with the problem, but we need some way of measuring how deadly self-driving cars are in comparison to how deadly human drivers are. I never see that, so I wonder how we're going to know when we should loosen the reins on them? Are regulators actually doing their jobs, or are they just going to move inexorably forward while occasionally throwing bones to public outrage?
The stakes here are huge, tens of thousands of deaths and horrific injuries per year, and we will unnecessarily kill lots of people if we deploy self-driving technology too fast or too slowly.
The way this will probably work out in practice is regulators choosing the path of least resistance between uninformed public outrage and greed-driven industry pressure. What we should be asking for is a data-driven approach.
“We” have done nothing of this sort. SF is its own trashfire and they just gave up on enforcing any traffic violations except parking which brings revenue. You can look this up in their own stats for this year
If a human being had done that they would have absolutely had the book thrown at them. There was a case where a woman was drunk driving and hit a man, he came up over the hood and into her windshield. He was alive but bleeding. She drove home and parked her car in the garage. He bled out through the night. NO ONE was sympathetic towards that woman.
But if an automated vehicle does it, it just becomes a line item that someone somewhere has to pay. You watch, someone will start offering insurance for that liability.
What we should be doing is making it an existential crisis for these companies. That will never happen and thus self driving vehicles will become good enough and that's it.
You can punish the manufacturer of any item that creates an excessive danger to the public.
Lawn darts aren't intended to be thrown near people, and don't exist anymore because people used them negligently. Human-driven cars are used negligently and kill 40,000 Americans a year.
This is just a general problem. Companies very often kill people and get fines. The body count before a company starts to get close to "this might kill the company" is absurdly high.
Consider, the Dalkon shield which had a hospitalization rate of 5 in 1000 and a death toll easily in the 1000s (maybe 10s of thousands). [1] The company went bankrupt but not defunct. It is now a part of Pfizer. Nobody experienced any sort of jail time in association with the deaths (and, after it was banned, they started selling the dalkon shield in poorer regions).
A serial killer killing 10 people gets the death penalty, yet there exist mining companies that have easily killed 1000s of employees with silicosis (and are still operating today, looking at you Dow [2]).
yep, this is why I said it should be an existential crisis. mistakes happen, but if your mistakes can potentially cause bodily harm you better be damned well prepared to show how it was not negligence in any way or you should get a fine that scares the shit out of you.
But that's the idealist side of me talking, the realist understands that will never happen.
Yeah, the big issue we have is that we reward ignorance. It's in the legal best interest for companies producing physical products to be as oblivious as possible to issues of safety. Legally, the worst the worst outcome for them happens when there's documented evidence of safety problems. It's far better to feign ignorance. "Who could have predicted that opioids were addictive!". "How could we have known that this doctor was overprescribing, we are a big company without the resources to track everything".
It should really be the opposite in an ideal world. A company that has documented "this is what we did to make sure everything was safe" should less culpable than the company that moves fast and breaks things/people/the planet.
If a company killed someone, I'd want their defense to be a list of everything they did to prevent that person from dying, not a "well shucks, accidents happened, and this was a tragedy that couldn't be avoided!". That's the response that should end a company's existence.
> If a human being had done that they would have absolutely had the book thrown at them.
Really? If they had been in a car accident caused by another driver, and took a few seconds to pull over because they were unaware they were dragging a person initially hit by the other driver, they would have been charged?
It's very hard to hold people responsible, and it's telling that you had to cite such a grisly and outrageous example. Unless you're drunk or it's a hit-and-run, you're unlikely to face any charges beyond a traffic violation.
> You watch, someone will start offering insurance for that liability.
Of course? Exactly like we've mandated for human drivers since the 1970s?
All this hand-wringing about things that we've accepted our whole lives with human drivers.
I'll say it again: we need data-driven approach to figuring out when autonomous vehicles reach the threshold of becoming less deadly than human drivers. If we jump the gun and allow widespread adoption too soon, people will die unnecessarily. If we drag our feet and allow widespread adoption too late, people will die unnecessarily.
> Really? If they had been in a car accident caused by another driver, and took a few seconds to pull over because they were unaware they were dragging a person initially hit by the other driver, they would have been charged?
unless you believe this person stayed completely silent during this, the answer is yes. The measuring stick that the law tends to use for liability like this is the reasonable person test and no reasonable person drives 20 feet with someone screaming in pain after having HIT their car.
Are there circumstances in which someone would NOT be charged? sure, there have been cases of toddlers running out in front of cars and it was determined the driver couldn't possible have seen them due to their height.
Since you apparently misunderstood, the point about the insurance is that maybe we shouldn't allow companies to insure themselves against killing people. We as a society absolutely went after Ford for choosing not to do recalls because the estimated cost of the recall exceeded the estimated payout for the deaths and injuries. Under no circumstances should we allow companies to make decisions like this.
Sorry but this just comes off as completely out of touch to me. Yes, I agree that there would be a legal case to charge someone in such a scenario. But what actually happens, at least in NYC, is that the cops and prosecutors will bend over backwards to give a driver the benefit of the doubt.
Cruise showed the DMV a video that cut out just before their car dragged (ran over) the pedestration. Effectively they were trying to do a hit and run as well by not showing footage of their car's behaviour that caused the injuries.
My theory is that we’re just going to read about a never-ending (but possibly increasingly rare) series of edge cases. And in every one, it’ll be dismissed as “oh that was a really rare edge case” yet humans would have been able to make a better choice in the same circumstances.
The issue isn't even that Cruise was involved in an incident. They were not at fault for the collision and were only involved after a (human) driver launched the victim into the Cruise vehicle. Cruise had their permit suspended because they essentially tried to hide their involvement/responsibility and withheld video footage from the DMV.
This sort of nonsense where you take a serious statement and tell me the cutesy "name" of the autonomous vehicle really detracts from the seriousness of the issue. Also, the last paragraph where they try to talk about how their vehicle wouldn't have made the same mistake as the human driver, glosses over the actual issues the cruise car had which a human driver would have known not to do.
I had the same thought about the name. They also didn't refer to the name again in the rest of the article, so it wasn't even setting up a convenient name to refer to the vehicle and distinguish it from other AVs; it was entirely superfluous.
Ya, and since a Panini is a pressed sandwich, you'd think the PR people would immediately remove that since it's comparable to what the car did to the woman's leg.
"Cruise gives the cars cutesy names" is typical of fast turnaround journalism. The reporter doesn't have the time or resources to investigate the details to determine if this is indicative of a casual culture at Cruise. Instead, they can drop the easily verified vehicle name and let the readers fill-in-the-gaps. It doesn't exactly line up with CA DMV suspending the AV permit, but it adds so-called depth to the story.
I wouldn't expect this offhanded fact in long-form, slower journalism. But I expect this from Reuters.
In their original press release they didn't even mention dragging the pedestrian. Here they've only mentioned it because they were caught red handed by the DMV.
> The AV detected a collision, bringing the vehicle to a stop; then attempted to pull over to avoid causing further road safety issues, pulling the individual forward approximately 20 feet.
Jesus Christ! Good riddance, I'm glad their permits are revoked. Can you imagine witnessing this and watching an autonomous vehicle slowly crushing and dragging someone underneath? That 20 ft must have felt like forever to witness. Let alone experience.
It's always fun when the Silicon Valley "move fast and break things" ethos meets real world government bureaucracy and regulation.
Filling out forms with made-up bullshit just to get an approval works great when you are putting together a SaaS sales presentation, not so much when it's the state Department of Transport on the other end and you are working on matters of public safety.
the Cruise dragged her 20 feet and rolled over her leg though. Obviously this is mainly the other drivers fault, but self driving cars need to be able to react properly to unexpected scenarios. A reasonable human driver would not have continued driving with a person trapped under the car.
I think that a suspension pending investigation and possible remediation is appropriate.
I realize assuming this is the California DMV's line of reasoning is speculation, but this sentence from Cruise's own public statement on the event really does it for me (emphasis mine):
> The AV detected a collision, bringing the vehicle to a stop; then attempted to pull over to avoid causing further road safety issues, pulling the individual forward approximately 20 feet.
The tricky thing about these vehicles' control software is that it has absolutely no judgment or common sense. That's really on display here, where the vehicle immediately goes into a canned program instead of following a course of action that takes into account all the details of the situation in which it found itself, not just the ones that its creators had anticipated.
That doesn't necessarily mean that testing self-driving cars on the roads is inherently irresponsible, but it does mean that the ground rules for how we manage such programs need to be tailored to the situation. What we have here is a situation where a rather nasty software defect has been discovered. We can assume that this defect is present in all of the vehicles Cruise is operating on public roadways. It is therefore reasonable and prudent to temporarily suspend autonomous testing operations on public roadways pending analysis and remediation.
Because, let me emphasize again, TESTING. Pausing to analyze and iterate when you find problems is an essential part of testing. It's arguably the main thing that distinguishes testing from just YOLOing shit into production.
If an individual driver did what the cruise vehicle did, then I would expect a suspended license during investigation. It just happens the AI diver is the singular “driver” for an entire fleet.
Commenters like this understand that doing a hit-and-run does result in a human losing their license, right? We don't revoke all human licenses because humans are unique individuals, not a hive mind controlled by a shared common program. Similarly, the DMV is not revoking the license for all self-driving cars. It's only revoking Cruise.
This is just anecdote, but having taken trips in both Waymo and Cruise in SF, I feel very comfortable in Waymo: it drives carefully but confidently. It alerts me when cyclists are near when I'm getting out. It navigates coned-off work areas. Cruise required a lot of (remote) human intervention during my trips.
And snake oil salesmen exist everywhere, not sure what your point is.
Are you saying that they should all be equally safe? If so that would require a level of oversight and control (plus knowledge share?) not typically seen in US government, no?
I agree in principle, I’m not defending them at all. However this is exactly what much of the US wants, given how much de regulation and reduced “red tape” is often argued for.
Until then, we’re going to get some companies that put everyone’s lives at risk. Hell Tesla alone has over promised to insane degrees (the Full in FSD). I don’t see companies having less power any time soon.
No one was sold anything yet. These things don't go from 0 to 100 overnight. We've assumed that self-driving cars will be safer than humans in the future, though that's yet to be determined. And even if their safety record is better than humans, they will be more dangerous in different ways than human drivers, much like this recent incident where the woman was dragged and pinned.
Musk has repeatedly stated that Teslas on FSD/AP are safer than humans, now.
You can take issue with the veracity of his claims (and I certainly do, on several levels), but let's not pretend like "no-one is selling this, yet" - let's also not forget, "The driver is only in the seat for legal reasons. The car is driving itself" - that was six years ago now.
Tesla's Full Self Driving is driver assistance software and is not an autonomous vehicle[1].
There are rules governing autonomous vehicles in California. They must report all accidents involving their self driving vehicles[2], miles driven, disengagements,... Tesla's FSD does not follow these rules.
Tesla's marketing may imply that FSD is an autonomous vehicle. However, today, in California, these are driver assistance programs and not self driving cars.
Given the DMV’s wording I’m waiting to see if anything in the official statement doesn’t hold up, but from Cruise’s statement, this incident is absolutely not a good vindication of human drivers, either:
> The [human-driven] Nissan Sentra then tragically struck and propelled the pedestrian into the path of the AV. The AV biased rightward before braking aggressively, but still made contact with the pedestrian. The AV detected a collision, bringing the vehicle to a stop; then attempted to pull over to avoid causing further road safety issues, pulling the individual forward approximately 20 feet. The driver of the Nissan Sentra fled the scene after the collision.
They are. Not one has ever tried to harm me, and I encounter them daily. I can think offhand of a dozen human drivers, motorcyclists and cyclist who have.
I haven't taken a Cruise, but I've used Waymo several times and my experience mirrors yours. I've seen Cruise cars on the street get confused and back up traffic enough times not to trust them.
I have a friend who interviewed at Waymo to be a safety driver. He told me they have a human at command central monitoring the car and the safety driver, talking back and forth. I see no reason why they would remove the human monitor after no longer using safety drivers. For liability reasons at least. My point is I'm not sure Waymo cars can actually be considered self- driving since they have a remote human standing by to take over.
Seems like Cruise doesn't actually have this. If so, why aren't we hearing a report from the remote driver? Why did the remote driver let the car drag a woman? Why do remote drivers let Cruise cars screw up over and over again, block emergency vehicles, etc...
If I'm right, then Cruise will have to add remote drivers and then we can call them remotely operated cars instead of driver-less cars.
They have this, by their own admission and from my personal experiences in the car. A detail you might be mistaken on - they’re not really fully driving the car remotely. They’re basically sitting in a call center answering questions for the car or plotting a course, or giving it the all clear to proceed if things are uncertain. And generally this only happens if the situation is one where the car can ‘safely’ stop and wait for a response. In situations like the one discussed here, the car doesn’t have time to wait for a response from a human operator, so it just does what it thinks is best locally.
I don't think any operator has a 1:1 correspondence between vehicles and remote operators. Remote operation is intended to be used when a vehicle has realised it can't handle the current situation it's in and needs external assistance.
Does Cruise have a screen like Waymo that shows you what the car "sees"?
Context: When sitting in the backseat of a Waymo, there is a tablet-sized screen that shows you all the cars nearby. And humans! On one of my first trips (at night), it took me a while to figure out what the white circles were. And then I looked over and saw people in the shadows! It was honestly kinda spooky that the car was much more aware of nearby humans then I was...
They are testing their garbage in dense urban areas. Too many times their vehicles just stop in the middle of the road.
If it rains, vehicle stops.
If the light is out, vehicle panics and stops.
Somebody puts a cone on the hood, vehicle panics.
If a single variable not accounted for in their algorithms shifts beyond expected range, it fucking panics.
So much money dumped/wasted into “autonomous personal vehicles”. How about we invest in making our cities more pedestrian, bike, and public transportation friendly? Let’s move people more efficiently rather than further digging ourselves into this hole.
An advanced country is one where as many people as possible can go where they want, when they want, as cheaply as feasible. In the US, autonomous vehicles are the best bet for facilitating this in urban areas.
This is true - where I live only the relatively wealthy can afford close enough to schools to cycle to them. Anyone on a regular salary has to drive in.
Do you think the potential future where humans are freed from doing anything for their transportation (no driving, no pedaling, no steering, no danger, no waiting in traffic, etc) is worth some short term pain while we develop the tech? Was the pain and suffering during the industrial revolution days that have led to our current high tech world worth it? (I'm not just asking this rhetorically, I'm really interested in what you or somebody who agrees with you thinks about these things).
Alternatively if you reject the premise of those questions that this is a necessary phase to pass through to get to the other side, what do you think the impact on progress would be if no "real world" training opportunities were allowed for these companies? Do you think they'd be hampered and progress would take longer, and what would the impact of that be?
> Do you think the potential future where humans are freed from doing anything for their transportation (no driving, no pedaling, no steering, no danger, no waiting in traffic, etc) is worth some short term pain while we develop the tech? Was the pain and suffering during the industrial revolution days that have led to our current high tech world worth it?
Or more succinctly: "You can't make an omelette without breaking some eggs. Sorry, eggs [regular people].
Ends justify the means thinking leads to some dark places (e.g. what kind of medical-technological progress would we be able to make if it was possible to do testing on humans like we do on animals?).
> Or more succinctly: "You can't make an omelette without breaking some eggs. Sorry, eggs [regular people].
I'm not trying to make a statement or a point with the question. The original question is all that is part of the question. Please don't try to read more in than what is there.
> Ends justify the means thinking leads to some dark places.
For sure, but if we try to avoid any and all pain then we could still be nomadic people. Maybe life satisfaction then was higher, but certainly life expectancy wasn't. Just the death rate of babies and women alone was pretty dangerous.
> For sure, but if we try to avoid any and all pain then we could still be nomadic people. Maybe life satisfaction then was higher, but certainly life expectancy wasn't. Just the death rate of babies and women alone was pretty dangerous.
You're presenting a false dichotomy. Giving self-driving car companies free reign to cheaply experiment on the public is not the same as "avoid[ing] any and all pain" or avoiding all technological progress.
> You're presenting a false dichotomy. Giving self-driving car companies free reign to cheaply experiment on the public is not the same as "avoid[ing] any and all pain" or avoiding all technological progress.
I'm not presenting a dichotomy at all (unless you think the spectrum is binary and is only zero pain or non-zero pain, which I don't), let alone a false one. I'm not talking about just self-driving car companies. This is a broader philosophical discussion about technology in general, and how much pain is tolerable for progress. If you believe zero is tolerable in the case of self-driving, then do you believe that in general for all tech? If not, why is self-driving so different?
Do you disagree with the premise that on the other side of self-driving cars is a world where humans don't have to labor for their transportation?
Edit: to clarify because people seem incapable of non-binary thinking, I'm not making an argument that Cruise (or any other tech company or car company or whatever) should be able to have "free reign to cheaply experiment on the public." That's not something I said or implied. I don't think they should and I think this California decision is probably a good one. There, can we talk about the higher philosophical issue now?
> I'm not presenting a dichotomy at all (unless you think the spectrum is binary and is only zero pain or non-zero pain, which I don't)
Then why are you saying things like "if we try to avoid any and all pain then we could still be nomadic people"? That seems pretty binary to me.
> This is a broader philosophical discussion about technology in general, and how much pain is tolerable for progress.
No, it's about who bears that pain. Is it the public or the investors? Obviously the investors and their apologists would prefer to externalize as much as the cost ("pain") as possible onto the public, and shame the public into accepting it.
> Do you disagree with the premise that on the other side of self-driving cars is a world where humans don't have to labor for their transportation?
That's actually an irrelevant distraction, especially if for someone who rejects ends-justify-the-means logic. Do you disagree with the premise that on the other side of unrestricted and unethical experimentation on humans is a world where many more diseases are cured and life is greatly extended (for some, at least)?
> Then why are you saying things like "if we try to avoid any and all pain then we could still be nomadic people"? That seems pretty binary to me.
Good question and thank you for the opportunity to clarify. My position/current opinion is that practically nothing in life can be done without some sort of pain or "harm" to self or another person. For example if the standard was "no harm", then there's no way we could have cars or motorized transport, or even bicycles for that matter. That doesn't mean that any and all pain is acceptable (that would be the binary view), but that basically it's a question of balance, not one of binary "yes" or "no."
> No, it's about who bears that pain. Is it the public or the investors? Obviously the investors and their apologists would prefer to externalize as much as the cost ("pain") as possible onto the public, and shame the public into accepting it.
I don't agree that is what the broader discussion is about, but I would certainly agree that "who bears that pain" is highly relevant to any concrete example, and certainly humans are much more accepting/tolerant of damage/harm that they don't personally have to bear. I would agree that investors should not be unilaterally making decisions on what harm is acceptable.
> That's actually an irrelevant distraction, especially if for someone who rejects ends-justify-the-means logic. Do you disagree with the premise that on the other side of unrestricted and unethical experimentation on humans is a world where many more diseases are cured and life is greatly extended (for some, at least)?
This is a interesting point, and one I'll continue to think on. My initial reaction is that ends-justify-the-means isn't/shouldn't be binary either. Some level of ends-justify-the-means is necessary for nearly everything unpleasant we do in life. For example, I just cleaned the toilet (which I hate doing) because the ends justify the means: if I don't clean it, it will get disgusting, and having a clean toilet is great so I make the sacrifice.
That said, the broader understanding and common meaning of ends-justify-the-means is absolutely one I reject and I suspect we agree fully on: There are potentially great ends that don't justify unethical means. An example of ends-justify-the-means would be the Unabomber justifying his terrorism because on the other end would be a better world. I find that horrifying and very, very wrong.
> Do you disagree with the premise that on the other side of unrestricted and unethical experimentation on humans is a world where many more diseases are cured and life is greatly extended (for some, at least)?
I would not disagree with that, and I would agree that this is not on its surface a good reason to do the experiments. I think the non-binary part comes in on how we define "unethical" however. To some people unethical would include allowing dying cancer patients the option to try potentially life saving but still experimental drugs. To me, very little is unethical so long as the person is a capable adult, is fully informed on the process, has the ability to withdraw at any time, and provides full consent. There are quite a few people who would disagree with that however, so in practice I think we still end up at the spectrum rather than the binary.
I disagree on the endpoint. I think active transit is healthy for humans, and removing it causes people to need to excercise deliberately much more — and people don’t, causing obesity and a ton of health concerns. Walking 2-3km a day would be great for most people, and much better than sitting in an auto-pod.
We should talk about para transit for the few people who cannot walk (or push their wheelchair) 2-3km/day. However this is a minority of people. As you say most people would be better off if they did move 2-3km more/day.
Would it then also be better if people had to churn their own butter? Obviously not. Exercise is fantastic and should be encouraged. I think nothing would do more to facilitate human-powered transportation (walking and biking) than replacing distracted and dangerous human drivers with attentive and safe autonomous ones.
Do you think the patients who got incorrect blood test results from Theranos should suck it up and not complain because all progress is good progress, even if some people have to get sacrificed along the way?
Obviously not. That's an absurd question that's not even remotely based on what I stated or implied anywhere, let alone the question you responded to. This feels more to me like it's directed at some sort insane strawman who lives in an imaginary world of binary thought where everything is either fully good or fully evil, but if I'm wrong and this question is in good faith, I would encourage you to read the existing sibling thread: https://news.ycombinator.com/item?id=38004007
> Too many times their vehicles just stop in the middle of the road.
The problem in this case happens to be that the AV tried to pull over instead of stopping in the middle of the road, so not sure this is a great criticism to make.
>How about we invest in making our cities more pedestrian, bike, and public transportation friendly?
There are tons of companies that would love to help (get paid to) build out public transit infrastructure in cities. You think any major city would have trouble attracting bids to build tram systems, subways, or even buy more busses?
Look at your local leaders, how many of them are pro-public transit? How many of them have a long history of blocking projects because they want to "preserve neighborhood character" or some other lame excuse?
Cruise's valuation of $30 billion will get you 12 miles of subway in NYC [1], hardly the public transit revolution necessary to make autonomous driving redundant.
I don't hate public transit or anything, I cycle and subway in NYC all the time, but the level of investment the US needs in public transit is an order of magnitude above the amount that investors have spent on autonomous driving.
But not Waymo. Which confirms all my personal biases.
More seriously, the statement cites four specific clauses that caused the permit to be suspended. The second one seems really interesting as it implies that there was a lie of omission where the others are more or less what I’d expect regarding safety risks:
> Today’s suspensions are based on the following:
> 13 CCR §228.20 (b) (6) – Based upon the performance of the vehicles, the Department determines the manufacturer’s vehicles are not safe for the public’s operation.
> 13 CCR §228.20 (b) (3) – The manufacturer has misrepresented any information related to safety of the autonomous technology of its vehicles.
> 13 CCR §227.42 (b) (5) – Any act or omission of the manufacturer or one of its agents, employees, contractors, or designees which the department finds makes the conduct of autonomous vehicle testing on public roads by the manufacturer an unreasonable risk to the public.
> 13 CCR §227.42 (c) – The department shall immediately suspend or revoke the Manufacturer’s Testing Permit or a Manufacturer’s Testing Permit – Driverless Vehicles if a manufacturer is engaging in a practice in such a manner that immediate suspension is required for the safety of persons on a public road.
> "The AV braked aggressively before impact and because it detected a collision, it attempted to pull over to avoid further safety issues," Cruise added. "When the AV tried to pull over, it continued before coming to a final stop, pulling the pedestrian forward. Our thoughts continue to be with the victim as we hope for a rapid and complete recovery."
This looks like Cruise's fault, dragging a pedestrian while pulling over for "safety"?
Yes, and it sounds like a fairly fundamental flaw. Does the car have sensors that can determine pulling over is the wrong thing to do in this situation?
In general, getting out of the way is the right course of action, but this clearly shows that there are exceptions. I'm not sure that the vehicles are currently equipped for differentiating to even be possible.
It's not clear to me that this is purely a software problem; it might be a "the car doesn't even have the sensors necessary to detect this kind of situation" problem.
It could also be a "the software can't anticipate that this would be happening the way a human can because it doesn't have object permanence" problem. That's probably at least part of how a human would do it. But I have a suspicion that we're very, very far away from being able to get computers to do that in any sort of reliable way. So I'm guessing we're probably looking at specialized sensors if we want to handle this kind of situation appropriately.
Basically some sort of self-driving car equivalent of a human driver getting out of the car to check and make sure everything's OK.
I would be surprised if it turns out that those self-driving cars don't have object permanence. Making things move around without a physical model looks like a really nasty infinite bug-breeder.
For the longest time, at least the visual representation of trains at a level crossing strongly implied challenges around object permanence for Tesla.
I will say that I recognize that the visual representation may not be a 1:1 (in fact quite certainly isn't) mapping of the computer's "world view".
But it was quite ridiculous and definitely disconcerting seeing a level crossing with a train going through it and seeing the Tesla I was in acting like there was a traffic light that was erratically going red, while dozens of semi trucks flickered in and out of existence across the screen in front of me.
> strongly implied challenges around object permanence for Tesla
Anyone who has ever ridden in a Tesla knows they have no real perception of the world around them. Even just driving down the road they show icons of the things around them, which will randomly appear and disappear or change from busses to pedestrians to cars.
And yet a common thread in post-mortems on so many self-driving car collisions boils down to, "the car doesn't have object permanence."
It may have some ability to track specific things in specific ways in a manner that's hard-coded by engineers. But if that's the same thing as an actual mental model of the world then that implies that we've had a significant piece of AGI in the form of many video games for about half a century now and just not known it.
yeah, another poster mentioned they would have heard the screaming.
audio is difficult because a human can differentiate a woman screaming from the sounds of construction, etc. And also can tell that it's someone in shock vs someone in pain.
I'm unsure if audio processing is good enough for that yet, but imagine if someone were to try and you could get these vehicles to stop by screaming near them.
That's not hard for me to imagine. Now that I think about it, it should probably be standard for autonomous cars -- if there's an accident or something unexpected it should pull over, unless it can't reach the side of the road without hitting something or someone is trapped under the car.
> it should pull over, unless it can't reach the side of the road without hitting something or someone is trapped under the car.
Or…insert how many other edge cases? There are plenty of scenarios where it shouldn’t pull over, aren’t there? Training the myriad unique situations and proper responses just feels untenable, at least with current technology.
Or you can not code your autonomous vehicle to try to move after getting into a collision, because if someone is caught under the car, you shouldn't try to move the car anyway.
It's hilarious that the fucking things park themselves if you put a traffic cone on the hood, but if they get into a crash, especially one with a pedestrian? BEEP BOOP MUST MOVE!
That is the beauty of technology: once we identify a problem we can do something about it and apply the fix to everything. While this is the type of case you can argue nobody would have expected before, once it happens you know and can setup lab situations where similar things happen and then figure out how best to respond over months of effort.
This is absolutely not a case nobody would have expected before? People end up under cars in accidents regularly. Imo any company that "found out" about this scenario through the cruise incident is guilty of gross negligence.
I'm sure all the bigger ones had looked at the possibility before, but it's only natural to revisit it when something horrible like this happens to a competitor.
I am sure that you don't generally start with this scenario - but I have to say again that this is a common thing. People end up under cars all the time (in accidents). There is just no way anyone should excuse any company for "not considering" this scenario if their cars are on roads. I'm sure it's hard! Other industries recognize corner cases where they need to design to avoid really ugly long tail outcomes and do so with ethical rigor. Most planes do not crash into the ocean and yet you are educated on how to escape in a water landing at the start of every flight.
Note that it's not just a pedestrian ending up under a car, but the whole situation of "other car knocks pedestrian into your path, which then ends up under your car".
The problem is that the world is full of near infinite permutations of situations that will continue to prove difficult for these technologies. A group of pigeons, to most human drivers, is identifiable from any approachable angle, in various weather and lighting conditions. To a computer, it has to be trained on all of that and consider the countless permutations - is that two pigeons in the rain in the dark? Is it 50 in broad daylight? Is it 12 cardboard cutouts? Our brains are incredibly good at parsing the world around us, and I’m not sure we’re even remotely close to that level of accuracy with self-driving cars, and that should worry people in cities where these are being rolled out.
Code that identifies if anything is stuck under the car would be useful for a lot of cases but especially if an accident already has happended.
Besides that, hard edge cases will always exist and can only be solved by a human (or an AGI?) in the loop. But they are't too common, so just detecting that the situation is not solvable by the algorithm, doing nothing and waiting for a remote operator should scale well enough.
See the pinned comment at the top. Seems they showed an official only half the video of what happened, and neglected to show the other half which showed the stopped vehicle start moving again with the victim still under the car.
yeah, that's a pretty blatantly terrible cover up, definitely not the sort of behaviour you'd want from a company who's allowed to test experimental tech on public roads.
In previous discussions Waymo and Cruise were typically lumped together — “self driving bad” — usually by the original article even! However, my personal experience both as rider and as former Google employee says that Waymo is holding itself to a higher standard and therefore having higher order problems. That’s a lot of nuance for a flame-y topic but this permit revocation is rather unambiguous and I am having a smidge of schadenfreude as a result.
It's fair, I had the same sense (and am also former google) and wondered if I was the only one. It always seemed crazy to me to lump Waymo and Cruise together, but I never had a water tight argument for why.
I have the same sense but even in this discussion I see one angry comment about, “those companies” that doesn’t seem to distinguish among Waymo, Cruise, and (in context, I think) also Tesla.
> then attempted to pull over to avoid causing further road safety issues, pulling the individual forward approximately 20 feet.
That’s a pretty enormous screw up and not one I’d heard about when this was initially reported. Interesting the statement from the DMV implies they were lying…
The incident last month where that woman got dragged and pinned by the Cruise vehicle was pretty disconcerting. The initial collision wasn't the fault of the cruise vehicle (another vehicle hit her pushing her into the cruise vehicle and fled the scene), but the cruise vehicle then proceeded to try to "pull over" and just dragged this woman and rolled over her leg. Weird situation but definitely something made much much worse by not having a person in the loop once the incident occurred.
That's if they even saw the person was under them. My aunt got dragged by my grandfather as a child for a block and he at one point ran over my grandmother in a parking lot when she fell behind the car... this may say more about my grandfather than anything else, but average encompasses all kinds of drivers.
stop and get out of the car because someone just got flung onto/underneath your car? or perhaps have enough situational awareness to see that someone is about to walk into oncoming traffic?
This woman got dragged 20 feet and then the car rolled over her leg and stopped. The fire department had to come and remove the wheel of the car to get it off her leg. I don't know of a single person who would have done that.
Dragged 20 feet after she was pinned under the car, or after being hit by the (human) driver? This part is not clear to me, and DMV never makes it clear either.
I'm not too familiar with the details here, but probably hit the brakes and stop ASAP? If there could be someone under my car I wouldn't keep driving to "pull over safely".
Did Cruise release the video, its not clear to me a human driver would be better (other than some textual descriptions of the event)? The (human) person who actually caused the crash is still at large, and the bay area (SF + Oakland) still continues to one of the municipalities with negligble traffic enforcement [1] and massively dangerous (human) driving behavior [2].
So the city and DMV focussing their efforts on this corner case driving systems seems like dereliction of duty on the City attorney's and DoT's part.
This woman got dragged 20 feet by the car after the collision, and then the car rolled over her leg and left her pinned to the ground until the fire department came and removed the wheel to get her out. The cruise vehicle wasn't at fault for the initial accident, but the subsequent damage it caused was entirely the fault of the autonomous vehicle. A person would have stopped and helped the pedestrian.
The person who hit and run the woman in the first place shows this is obviously not true. You think they didn't notice?There was a muni bus who hit a pedestrian a few weeks ago and didn't even notice.
Except it didn't immediately do that. Cruise released a report today saying that the car actually drove 20 feet and dragged the woman. There is no simple answer as to what to do. It's simply difficult and acting like there are easy answers to this is unproductive and imo not in the spirit of hackernews.
It's difficult to program an autonomous vehicle to handle this correctly, sure. That doesn't mean an average human driver wouldn't have done better. Of course a human driver wouldn't just keep driving to pull over after hitting a pedestrian and not being able to see where they went.
The person who hit and run commuted a felony. Cruise not providing evidence as asked and required by their testing agreement did similar. A muni bus and a car are different.
I agree that hiding data is bad and the company should be punished. I was simply commenting on the fact that people frequently do not stop when running over pedestrians, which the parent comment claimed.
With respect this part is not clear. Only that the woman moved 20feet.
The DMV has not made clear whether she moved 20feet from the location of original Collision (ie with the human driver) or she was dragged 20feet as the cruise car kept driving. They’ve let the audience fill in the blanks and we’ve collectively come to the more exciting conclusion.
I'm pretty sure Cruise's statement worded this intentionally vaguely, and it's natural to assume they are trying to gloss over their responsibility in the most horrific part of the story.
It’s so painfully obvious to me that the solution here is to not having flying humans because of bad drivers that hit and run and not “actually let’s ban autonomous driving” (the city will inevitably come after Waymo next)
The first problem is much harder to solve, the second one only happens because of the first.
The city elected to do the easy useless thing first and is patting themselves in the back (people are still gonna be flying due to shitty human drivers tomorrow). This is a problem.
It's also worthwhile to point out that, although in this case the bad human driver was the immediate cause of the accident, at scale this will happen with some frequency, either from bad human drivers, bugs in the software or hardware, or just weird situations. The entire point of AVs is that these can also be corrected at scale: people who make excuses for the failure are removing the paramount incentive to make AVs as near to perfect as possible.
Pretty sure a human driver would have done more than what Cruise did, which was...nothing. They didn't even respond to anyone until firefighters covered the vehicle's sensors.
> Firefighters obstructed the sensors of the driverless car to alert the Cruise control center. He said representatives from Cruise responded to firefighters and “immediately disabled the car remotely.”
Seriously! This! This times a thousand! I get that technology is cool and we love seeing stuff like autonomous cars (and to be clear, where I live, I would say they should be the default because holy shit can people not drive here) but like, this is just not okay. Cruise's cars have a well documented history of disrupting traffic, and additionally, they have been found to:
- Not be forthcoming with authorities on releasing information/video
- Be misrepresenting the performance of their vehicles to the authorities
This is just not acceptable for a company to be doing. This is not how we disrupt industries. You cannot "fake it till you make it" with road safety when building vehicles that traverse roads safely is your business' entire raison d'etre! That's just unacceptable, full stop.
And, add to it, Cruise has been given a direct and clear path to resuming operations, which essentially boils down to doing the thing they should've been doing already. Like come on! How much more benevolent can we ask a regulator to be in this scenario? One could argue, frankly, that they're being too kid gloves about a company lying to an organization who's primary goal is ensuring safety about safety!
One problem is that the state taught them that it’s ok to lie, or at least that it’s ok to omit the full details.
During the certification process for these self-driving cars, incidents of the cars “misbehaving” (for lack of a better term) were supposed to be reported. However, if a human driver intervened, then it didn’t count as the car itself making a poor choice.
Example: self-driving car approaches an intersection clipping a reasonable speed and the light turns yellow. While a human driver would understand that it’s better to go through than to slam their brakes, the self-driving cat goes “yellow means red soon, stop now”. The human driver overrides and floors it through the intersection. Had the car stopped, that would’ve been a mark against the autonomous system. But since a human intervened, it doesn’t “count”.
This is a good watch on the state of self-driving cars and how policy makers have largely just said “eh, fuck it” in places like SF. https://youtu.be/pmGOjHi-7MM
Aside from the other comments, pulling over when you detect a collision is definitely not best human practice on an ordinary city street for a human driver. If you detect a collision, you stop the vehicle, access the situation and maybe pull over based on your assessment of the situation.
And this brings a question to me: How did they create, develop, this post-collision routine? Afaik, the overall program/system guiding a self-driving is some combination of trained neural-network routines and more standard robotic feedback loops. But "what to do after a collision" seems like one of those corner cases that neither programming approach would or could deal with so what development created this routine? It would have to be pure ad-hoc-ery and that suggests that in Cruise's approach to the last-mile/endless-corner-cases problem of self-driving cars is just that, a series of ad-hoc fixed. And this approach can neither be safe nor can it scale.
And further, I have to presume Cruise and the others have a bay of human pilots that are called-on to deal with difficult situations. Why didn't the "you've just had a collision" signal not pull a human? (maybe it did and not saying that is their problem but otherwise..) The main explanation I can think of is ... the car's routines are already calling pilots a lot and the team is working feverishly to reduce this. An ad-hoc pull-over routine is intended to reduce those calls back to the pilots. Admittedly this is speculation but if true it reinforces my conclusion above.
> And further, I have to presume Cruise and the others have a bay of human pilots that are called-on to deal with difficult situations.
And then there's a momentary thought of this, and then I think how bad/hard it would be. Sitting there in a "call center", "Ding!" and you're suddenly thrown in to a car mid-collision, needing to take instant surveying of the situation (like "there's someone partly trapped under the vehicle", "am I in the middle of an intersection/interstate", etc., et al.
Even getting past that I think such a job would play absolute hell on epi, norepi, dopamine, etc., response and those people would burn out, quickly (and I say that as someone who spends a lot of time waking up to alarm tones going off before jumping in an ambulance or fire engine...).
I heard that the way Waymo works is that the car actually presents solutions, multiple-choice style, to the call center operator. The operator assesses the situation and chooses one, and then the car executes it. I assume there’s also a “none of the above”’or some such which would flag someone to come on site.
The remote operator cannot manually operate the car under these circumstances and that is by design.
By the time you're paged to handle the situation, the pedestrian is already trapped under the vehicle and you cannot see her because the car does not have an under-the-vehicle camera. So the only way you can handle this case is to rewind and review what happened before the collision and respond only then. Considering that the speed with which you react can be a matter of life and death, I don't see a satisfactory solution to this problem.
> And this brings a question to me: How did they create, develop, this post-collision routine?
I’m just going to assume it’s a liability thing:
* Cruise gets in an accident and stays in the road, passenger gets out and gets hit.
* Cruise gets in an accident and pulls to the curb, passenger can get out on the curb (and if they don’t, and get hit, Cruise can claim it pulled over).
I could see this decision being made (or reviewed by) legal. Which sounds like a truly awful way to build a product.
I agree with you on the adhockery as well. Sounds like it.
1,287 comments
[ 3.1 ms ] story [ 432 ms ] threadNote for example today's suit against Meta for alleged harms against children. No such suit against TV aimed at children or e.g. fashion magazines that are well-known to have deleterious effects on children. Autonomous vehicles are new tech, so the bar will be higher.
But every new model or software update will invalidate what we've observed and know about automated driving.
We're in an era of extreme obfuscation and enshittification by tech companies, even where safety is involved. I'd be more likely to trust self driving tech if I trusted the current form of the tech industry, and trusted that regulatory bodies had the power and ability to oversee them as effectively as when they were first regulating cars.
Of course even that wasn't perfect, but my trust that we can even do that well is nonexistent.
Public transport is a safer alternative but not as profitable
is it accepted as a standard by any professionals? Why or why not?
One simple and obvious reason for this standard is that it optimizes for human safety.
“A highly-skilled and attentive human driver” is a reasonable standard. So-called autonomous vehicles are nowhere near that.
I think AVs need to be better than p95 or so to be socially accepted.
If we can achieve that, the best in 100 or best in 1000 may not be far off.
Today’s suspensions are based on the following:
13 CCR §228.20 (b) (6) – Based upon the performance of the vehicles, the Department determines the manufacturer’s vehicles are not safe for the public’s operation.
13 CCR §228.20 (b) (3) – The manufacturer has misrepresented any information related to safety of the autonomous technology of its vehicles.
13 CCR §227.42 (b) (5) – Any act or omission of the manufacturer or one of its agents, employees, contractors, or designees which the department finds makes the conduct of autonomous vehicle testing on public roads by the manufacturer an unreasonable risk to the public.
13 CCR §227.42 (c) – The department shall immediately suspend or revoke the Manufacturer’s Testing Permit or a Manufacturer’s Testing Permit – Driverless Vehicles if a manufacturer is engaging in a practice in such a manner that immediate suspension is required for the safety of persons on a public road.
Misrepresenting safety seems like a big deal.
[1] https://www.dmv.ca.gov/portal/news-and-media/dmv-statement-o...
Edit: This Vice article [2] says Cruise tried to withhold pedestrian injury footage from the DMV.
> In the Order of Suspension, the California DMV said that the Cruise vehicle initially came to a hard stop and ran over the pedestrian. After coming to a complete stop, it then attempted to do a “pullover maneuver while the pedestrian was underneath the vehicle.” The car crawled along at 7 mph for about 20 feet, then came to a final stop. The pedestrian remained under the car the whole time.
> The day after the incident, DMV representatives met with Cruise to “discuss the incident.” During that meeting, Cruise only showed footage up to the first complete stop, according to the Order of Suspension. No one at Cruise told the officers or showed any footage of the subsequent pullover maneuver and dragging. The DMV only learned of that from “another government agency.” When DMV asked for footage of that part of the incident, Cruise provided it.
[2] https://www.vice.com/en/article/4a3ba3/california-dmv-suspen...
Otherwise... not so much.
Which would make the driver responsible not the the company for incidents especially ones which explicitly circumvented controls.
The scenario with cruize is different because the company is responsible for everything
The FSD Beta uses the in-cabin camera's to do eye tracking and all the other attention stuff.
It may have been true at the time, but now Tesla uses the interior camera to make sure you are paying attention to the road.
If you don’t want to use the FSD beta, you don’t have to activate the camera, or you can put tape over it. With either of these actions the user can opt out but will not have the ability to use the beta.
Two points:
Try using Cruise outside its tiny comfort zones.
There is no such thing as the “self driving Tesla” you mention. There is a beta which requires human supervision at all times.
Having ridden in Cruise cars a couple of times, I am glad I was able to ride one while I had the chance. The novelty factor was very high, and while being hard to hail + very slow, it was a great thing to show to visitors to the city. Hopefully these safety issues can be resolved. As a biker, I feel much safer around a cruise than your average prius.
[0] https://twitter.com/Cruise/status/1716877217995894934
[1] https://www.sfchronicle.com/bayarea/article/woman-run-autono...
You could, however, argue that staying in place was the right thing to do; continuing to drive to get the woman out from under the car may have injured her further. A follow-up article[0] suggests just that, and emergency crews ended up lifting the car off of her instead of trying to tow it.
If this is indeed why their permit was suspended, it seems like a questionable reason. Some of the other dumb things Cruise cars have done seem much more obviously egregiously bad. But in this case the crash may have been unavoidable (by human or robot), and avoiding injuring her further by not unpinning her may have been the right move.
[0] https://www.sfchronicle.com/sf/article/cruise-driverless-tax...
It's highly situation specific, and I'd expect the emergency personal to make the judgement call on how to free the person (potentially by taking control of the vehicle, or other means).
(I have my comments appear on a several minute delay because I know I sometimes submit a comment before I've made sure my intent was clear, but I maybe I took too long this time. It's a bad habit, and I know I should slow down, but... yeah.)
Honestly, I don't think this tech will be able to ever be fully automated without controlling the environment as well (dedicated lanes). There's just too many variables you can't account for, and too much risk.
I'm not saying that means we should just put them on the road, consequences be damned, but I'm curious if there are alternative ways to prove them without risking bystanders' lives.
True, but I bet either a Cruise or a Waymo would have stopped rather than hit that woman crossing the street, and they definitely would not have fled the scene like the human driver did.
"But the DMV also said it was not made aware that the Cruise vehicle then tried to pull over while the pedestrian was underneath it."
Kind of proves my point. Stopping completely would have been more ok. The issue the DMV took is that it then pulled over, and that cruise withheld that information.
[0]https://spectrum.ieee.org/lta-airship-faa-clearance
They do! https://slate.com/news-and-politics/2015/09/why-drivers-in-c...
And people getting away with it -- even with video evidence -- seems to be an indication of police corruption. Which creates further incentive to do the wrong thing here.
So a pedestrian was crossing a crosswalk when they didn't have the green light. It doesn't really make sense why neither the human driver nor Cruise car slowed down in that case and kept driving normally. Both should have stopped despite having the green light.
It may not be the performance of the vehicle in the incident that led to the suspensions so much as Cruise's performance in the investigation of the incident (or information about Cruise's prior behavior on safety info that came to light through the investigation.)
I think the bigger issue is the decision to move the vehicle after a collision involving a person not in a car. I've not had the misfortune to be in such a collision, but in the collisions I've been in, I would get out, and inspect the situation before taking any further action. Moving out of traffic is a good thing to do, but especially if there's a person, it's important to consider the situation before further movement. Even if it were just a vehicle to vehicle collision, I'd want to check if someone needed urgent medical attention, and acknowledge the collision with the other driver, make a plan for where to stop to discuss, and check for and clear any obviously sharp debris that may pierce tires of my vehicle or others' vehicle. A lot of that is outside the capability of an automated vehicle... but maybe human review before pulling to a safer place is a good choice especially if a person outside of a car is involved in the collision.
We should be suspending all bad driver’s licenses, AI and human alike. Today we give people passes for crashing while driving badly, and we should not.
"When the AV tried to pull over, it continued before coming to a final stop, pulling the pedestrian forward," the company said.
This poor woman was involved in two accidents really, and it is fair to shut down Cruise until they address this failure scenario of driving on top of a victim in road.
The first picture in the SF Chronicle article is harrowing: https://s.hdnux.com/photos/01/34/54/62/24300697/3/1100x0.jpg
These autonomous cars have done some truly dumb things, but I don't think this incident is so obviously bad.
Meanwhile, the human driver who actually hit her first, and then fled the scene, still hasn't been found...
[0] https://www.sfchronicle.com/bayarea/article/woman-run-autono...
While not impossible, it's hard to imagine a human-driver scenario that would look quite like that, regardless of how legal accountability might resolve. Would a human driver stop their vehicle on top of the person they just struck? Would they then ignore the person they'd trapped and wait for a bystander to intervene?
I'm not suggesting that this scenario necessarily warrants the suspension, or that most of the same problems might not occur in some human-driver scenario, but it points to some of the subtle differences of human vs automated drivers that come up in exceptional situations. There are a lot of exceptional situations when driving and it very much matters how automated processes navigate them.
"“When it comes to someone pinned beneath a vehicle, the most effective way to unpin them is to lift the vehicle,” Sgt. Kathryn Winters, a spokesperson for the department, said in an interview. Were a driver to move a vehicle with a person lying there, “you run the risk of causing more injury.”"[0]
[0] https://www.sfchronicle.com/sf/article/cruise-driverless-tax...
It's not hard to imagine that it's simply unequipped to know something like that, and it does seem fair to require that it become so equipped.
Huge supporter of autonomous cars, still feel like this is completely reasonable to require a solution for.
Updating my comment to include this:
“When it comes to someone pinned beneath a vehicle, the most effective way to unpin them is to lift the vehicle,” Sgt. Kathryn Winters, a spokesperson for the department, said in an interview. Were a driver to move a vehicle with a person lying there, “you run the risk of causing more injury.”
https://www.sfchronicle.com/sf/article/cruise-driverless-tax...
[0]I can see myself not agreeing with the conclusions, but to not even see the question broached in these terms is maddening.
No advanced robotics necessary.
[1]And getting there slower has the opportunity cost of surplus deaths at the current rate minus the terminal rate - a year delay to that future costs 39,960 lives. (A year delay to ubiquitous public transport does as well ofc.)
That said, I don't have access to the data and I don't really know if Cruise has been acting inappropriately. I'm glad that Waymo seems to be generating far fewer negative press reports, which seems to be a good proxy for whether regulators are going to clamp down. So I hope Waymo continues to operate successfully (and expands their service to me in particular) and I hope that Cruise improves their system and gets back online.
Cruise was pushing too hard and fast for where their stack is at and they seem to have some severe safety related bugs.
Ex [1]: The car randomly starts swerving rapidly across traffic lanes towards an occupied sidewalk.
Ex [2] (@1:31): The car doesn't stop for someone already in the crosswalk and nearly hits them.
[1]: https://www.dailymail.co.uk/news/article-12640657/self-drivi...
[2]: https://abcnews.go.com/GMA/Living/video/driving-taxis-hittin...
For some reason, I can't imagine literally any human driving quite like that?
That's a common response from people who would prefer NOT to know the facts.
Usually when private companies are developing products that could kill, drugs for example, it is on the company to exhaustively prove the safety and effectiveness before launching it to the public. It seems odd that self driving cars have been skipping this step.
The AVs do not work well in the edge cases - exactly in those situations where humans can apply common sense and behave accordingly. Exactly the cases that sceptics seemed to be worried about, and it turns out that the sceptics were right.
I hope AVs will not be allowed on public roads where I live in the next decades.
A part of the sales pitch for AVs is that autonomy acts as a safety layer.
As with any sales pitch, the interest of the manufacturer is often more in getting people to believe the pitch than in it being true, validating that it is true is a concern of outside parties. Insofar as AV adoption requires loosening existing rules to allow them, that includes regulatory authorities.
Yes, it is.
> Do you think Google's main motivation behind starting AV work 14 years ago and incinerating billions on it over the years was really rooted in speculative profit years down the road?
Yes, along with a pile of other moonshots. That's what moonshots are for.
There's a lot of profit potential in AVs, and even if you fail at it there is a lot of potential for profitable spinoffs of the research.
And if you fail at both, well, moonshots are high-risk, high-reward.
To add to your comment, the billions "incinerated" in Waymo (~$5bn) is about what New York City pays for a single subway station qua shopping mall.
Honest question: why do you think Google funded this?
yes!!!
imagine if everyone that commutes could watch ads, sorry, use Google's valuable products and services for an extra 3 hours a day
It's just that, in practice, it's almost certainly impossible to reach safe self-driving cars without a period of 'public beta testing', you'll just never get enough data and experience.
Not entirely unlike how there's a period for new drivers where they really aren't safe, but there's no way for them to become safe without going out onto the regular roads and gaining more experience.
[Citation needed]
"It seems odd that self driving cars have been skipping this step." is an opinion referencing well-known current events. You can't expect everyone commenting on this article to re-reference the saga of Cruise in San Francisco; especially when it's frequently covered on the front page of Hacker News.
Furthermore, you should know that a Hacker News discussion is not Wikipedia. These discussions are not the kind of things where every off-the-cuff statement needs to be backed up with references.
You can argue about how much non-public testing is enough, but any roadmap for AV will eventually at some point have to include public testing.
I did not claim they're not testing, they clearly are. My argument is that people that didn't sign up to be test subjects are getting hurt in the process.
It's worth pointing out the discrepancy in human safety trials, is it not? Why should AV testing be less rigorous than pharmaceuticals?
My point is that Waymo has done plenty of testing already. The question is, where do you draw the line? Are cars ever "done" being tested? Just like with drugs, you never know all the side effects for 100%, you can only make the error bar smaller and smaller, but eventually you have to distribute it to the public.
Same with the car, at some point they will eventually have to drive on public road. By your logic, they would never reach that threshold since they're still "testing" it.
Its also a USian obsession. Some other places have been able to actually invest in superior public infrastructure but the US is so focused on techno-solutionism it, as usual, misses the extant and superior solutions. Self driving does nothing to mitigate the sprawl and atomization that wreaks havok on our society, but trains and busses, and density that cannot be achieved with car-dependence does.
"Fixing" commutes by making everyone sit alone in their car while it does the driving part as if that's the problem with an hour long commute, is the single dumbest way to make cities better.
Instead we've got "it's a car but you can doomscroll while on your 45 minute commute and we harvest your data to sell ads".
This has nothing to do with public safety.
>But they want freedom more.
Sure, it's a very individualistic society where one's personal convenience is more important than societal good. But let's call a spade a spade and not pretend that self driving car companies are selfless benefactors out to solve pedestrian deaths.
Every other country has seen a reduction in the number of deaths over time, without self driving cars.
Progress is always good IMO.
There's a rural cost/benefit that ensures we will never not need cars, probably. But I can't fathom why eight lane interstates exist. If we can justify that many lanes of road traffic, we can certainly replace six of them with a train.
Self-driving trains? Easy. And no matter how hard you try, even the most efficient electric car will never end up greener than a train on a per user basis.
If you mean it more generally, MattY would like to have a word. https://www.slowboring.com/p/the-techno-optimists-fallacy
Self-driving cars isn't progress. Larger and heavier cars isn't progress. Cars with insane eye-searing LED headlights isn't progress.
I think is the qualifier the other user put in to disqualify public transit. But yes, ideally.
I've witnessed tons of anti-social behavior over the years taking the DC metro and honestly, have never been comfortable ever since I was a child and got stuck between two drunk men arguing after a baseball game. I was feeling claustrophobic and could even feel the spit hit my face. The car was packed and no room to really get away from it. I think its gotten better but I wouldn't know as I haven't used it in years.
- Zero traffic fatalities would be nice - Reduced congestion/traffic, people can get to places faster - Never having to worry about parking again - No more need for cars parked on streets, more space & safer for cyclists and pedestrians - Reduced income requirements for those that need access to a car
There are countless benefits. The transition period to get there will be a little tumultuous, but we'll get there eventually as a society
They won't reduce congestion and traffic, because people won't have to drive they'll be OK with being on the road longer. You just hang out in your living room on wheels.
Parking... Where are the cars going to park exactly? How far from the restaurants and homes etc? Won't the extra trips from/to parking lots create more traffic?
Reduced income requirements, that's a new one to me. These cars will have more tech that's more expensive to build and maintain. How does that make them cheaper?
50%+ of car trips are less than three miles... as a society what we need is a way to make those trips without cars.
then you've only fixed 50% of car trips, and much less than 50% of vehicle miles and probably not done much at all for highway deaths.
all the urban bike infrastructure in the world isn't going to do much for rural and interstate problems.
In the extreme, imagine if a light turns green and all cars in line accelerate at the same time. Intersections would let through a lot more cars.
Or think of freeway speeds where the length of a car is near-negligible. A one-second following distance means almost 60 cars pass a given point in a minute, versus almost 30 cars per minute for a two-second following distance.
Traffic flows would become well coordinated and way more efficient, no more accordion effect and accidents to cause traffic jams.
> Parking... Where are the cars going to park exactly? How far from the restaurants and homes etc? Won't the extra trips from/to parking lots create more traffic?
It could be several miles from where the passengers are being dropped off. See above for why traffic won't be worse.
> Reduced income requirements, that's a new one to me. These cars will have more tech that's more expensive to build and maintain. How does that make them cheaper?
Of course new technology is expensive initially. Over time, it becomes cheaper and cheaper. Just like air travel, personal computers, broadband internet, etc.
> 50%+ of car trips are less than three miles... as a society what we need is a way to make those trips without cars.
More efficient use of the roadways from self driving cars would create opportunities for improvements in this area as well. Dedicated protected bike lanes, for example.
You might like driving. For many, driving is stressful.
This is besides the point someone else brought up about how notoriously bad at driving humans can be, and the capacity for how much safer an automated car could be when the tech is matured.
Public transit like efficient railroad infrastructure across the US would be even nicer, but I've about given up on that for the USA. They're doing self driving cars because we sunk too much money in roadway infrastructure (sunk cost fallacy.)
We could use... trains & buses. Self-driving trains and buses!
But self-driving individual cars, because they will make the time more bearable, will induce even more driving, which is the opposite of what we want.
To give the benefit of the doubt, my assumption is that they're solving it this way because the road infrastructure and market for cars already exists. You'd see big tech working on automating public transit if states actually started improving their public transit infrastructure.
- No more car insurance
- Cheap transportation to work/school
- Less deaths on the road
There is plenty to gain from having true driverless vehicles accessible to the public.
They might be safer than human drivers, but there are already superior solutions to that in the form of transportation systems that are not overly focused on cars.
Is there something specific that caused this halt on Cruise's testing permits?
10000% we need safer designs for streets and roads, better education, automated enforcement of speed limits, dis-incentives for larger/heavier/more dangerous vehicles, ...
All of these things need to be done, and can be done with existing technology today. We don't need self-driving cars to address our problems; they're essentially a distraction and an excuse to postpone improvements.
If that car gets found doing something obviously illegal (speeding, running a red light, some kind of "reckless driving" camera review, etc) the registered owner is automatically assigned blame. They can potentially file paperwork to the court to prove someone else was driving (a lease arrangement, some valet ticket along with some proof from the restaurant or whatever at the time, a sworn statement from another individual confessing to it, etc). But ultimately by default the owner of the car is responsible for the safe handling of the vehicle. If you're worried someone is going to break the law with your car, don't give them your keys. If you don't like this, find another form of transportation.
Tag enforcement needs to be stronger. If an automated system sees a tag that doesn't make sense, it should be immediately flagged and enforcement dispatched. Lots of cops already have ALPRs, those should immediately trigger. I see so many cars out there with sketchy paper tags, this whole thing needs to be shut down.
Driving a car on public streets shouldn't be nearly as easy as it is in the US. Far too many people don't treat driving with the respect it needs.
https://getcruise.com/news/blog/2023/a-detailed-review-of-th...
so pulling over after a collision, which resulted in dragging the human 20 feet is an interpretation of this regulation.
sounds like California and Federal AVs regulations are equally not safe for the public's operations
which is a good enough reason to revoke permits too, until they fix this together with the private industry
until they figure that out, this is a not-intolerable course of action. it doesn't need to be interpreted as punitive.
AVs seem to all have different "mental models" of the external world, so it seems like we've got a lot of work before we can pipe data collected from this Cruise incident into a simulator and test Waymo, Uber, Tesla, etc.
“The Nissan Sentra then tragically struck and propelled the pedestrian into the path of the AV. The AV biased rightward before braking aggressively, but still made contact with the pedestrian. The AV detected a collision, bringing the vehicle to a stop; then attempted to pull over to avoid causing further road safety issues, pulling the individual forward approximately 20 feet. The driver of the Nissan Sentra fled the scene after the collision.”
I welcome our robot overlords.
Really wish there was a way for tech (especially new AI tech) to wholesale abandon the city. Even just move south to the actual Silicon Valley. The city picking winners and losers like this is untenable
How else are they supposed to screw up?
If the city needs more information from the cars, they should sue for it not indefinitely suspend operations.
The city is looking to regulate all this like it regulates new housing (ie out of existence), there has to be consequences for it.
So yea, this is the city working to regulate driving, like it regulates housing and small business i.e. to the death while favoring incumbents that are far worse.
Cruise operates fine in Austin and Phoenix. They could just move their operations to South San Francisco or further south and be better off. The city simply does not deserve the presence of tech, and for tech companies their presence in the city seems counter-effective
"The DMV has provided Cruise with the steps needed to apply to reinstate its suspended permits, which the DMV will not approve until the company has fulfilled the requirements to the department’s satisfaction."
[1]: https://sfist.com/2023/08/17/sf-city-attorney-files-motion-t...
Some other factors are in play
I'd say the initial hit probably did a chunk of damage to the pedestrian, but quite likely not fatal unless their head hit the ground. But then being driven over and dragged 20 feet is going to be pretty scary and in all likelihood cause more serious injuries than the initial impact.
1 - https://www.sfchronicle.com/bayarea/article/woman-run-autono...
I like how you're parroting Cruise's spin on this: the pedestrian was "pushed along", not "run over and then dragged."
That is an optimistic view of the likely behavior of your hypothetical driver. Hit and runs are commonplace. Dragging struck pedestrians around isn't all that uncommon either. Panicked motorists frequently make very bad decisions.
Probably because you're reading their press release. Their previous press release was even more convincing - they somehow forgot to mention the part about draggin the pedestrian at the time.
Here they somehow forget to mention the car has already accellerated to 18mph while a pedestrian was still crossing the road.
I'm not trying to detract from the horrible outcome of this case, but the problem wasn't the car pulling away at a green light when there were no obstacles in front of it, but the way the car decided to move itself to a "safer place" immediately after an accident when it clearly didn't have enough sensors or the AI wasn't trained sufficiently on this kind of situation to be 100% sure that it was safe for it to make the maneuver.
So there was this woman crossing a crosswalk in front of these two cars when she shouldn't have. I guess I don't understand why neither car stopped in front of the crosswalk when they saw this woman in the middle of the crosswalk (illegally).
IMO it was a bit aggressive for Cruise to have seen this woman jaywalking and still crossed the crosswalk after detecting that the woman had just passed the car but still in the middle of the crosswalk.
In those situations I would 100% stop completely in front of them and/or drive very slowly around them. The difference between a regular jaywalker and someone who is mentally disturbed is the mentally disturbed jaywalker is unpredictable and you have no idea what they'll do next.
The right decision in this case was for both cars to at least slow down if not stop in front of the crosswalk. The Cruise car driving normally just because the walker happened to just pass to its left without slowing down doesn't seem like the best decision.
And it's not a given that doing so would make the situation safer, as other drivers would eventually drive around your car.
Anyone who doesn’t slow down in situations like that is a waiting to be like the Nissan driver: a person a-okay with committing manslaughter.
And two wrongs don’t make a right. Just because some human drivers are negligent and horrible doesn’t mean we have to accept robots varieties that are also bad.
It doesn't sound like the Cruise car stopped because it basically saw that the woman had just barely crossed to its left out of its path of travel.
A more cautious driver would have slowed down at the very least or stopped completely.
Ba dum tss
Jaywalking laws never cease to trigger culture shock for me
Crossing on a red signal (when safe) is even polite in my country, to not unnecessarily impede traffic
(Of course, we're talking about crossing "when safe", not when you'd be walking into the path of a vehicle with the right of way.)
Wat? (Most) pedestrians aren't squirrels that hang out in the road running back and forth. You've put forth an unreasonable standard that basically nobody uses. Except for maybe those "special" types of drivers that will slam on their brakes and wave at you when your intended path was clearly behind their car after it had passed, as if you're supposed to be thankful for their putting themselves in your way to indulge in a simulation of altruism.
A good human driver may have slowed down from perceiving the emerging exceptional situation of the oncoming car not reacting to the pedestrian, but I'd say that's a much different framing than what you asserted.
"The AV detected a collision, bringing the vehicle to a stop; then attempted to pull over to avoid causing further road safety issues, pulling the individual forward approximately 20 feet."
First of all, I find it ironic that the human driver is still at large. I'm sure the Cruise car recorded the plate and they passed this to the police.
Second, the corporate speak on the Cruise release is very good. Notice how it says "pulling the individual forward", in other words, the car dragged the person while it attempted to get out of the way. I guess nobody thought about that edge case. And they released a completely irrelevant simulation showing that if the other car had been an AV this wouldn't have happened. Yet, the real issue with the Cruise AV is that after a collision it just blindly tried to pull over and it dragged the person. The same thing would have happened if the person had simply collided with the car on their own.
Ironic but completely expected. We have normalized and accepted the awfulness of human drivers to a degree that people just don't comprehend. Human would be terrible drivers even if you ignored the fact that many of them are chemically impaired, even if you ignored the fact that they are prone to rage and aggression.
As we consider the safety of self-driving vehicles, there is going to be a disorienting amount of cognitive dissonance as we are forced to confront the awfulness of what we already accepted and have been living with for a century. There were over 42,000 deaths due to motor vehicle accidents in 2022. That means if we created self-driving cars that were twice as safe as humans, they might save 21,000 lives per year and also kill 21,000 people per year. That sounds insane, but it would be a sane way of improving an insane situation.
Kibitzing about these individual incidents is a normal and inevitable human way to try to deal with the problem, but we need some way of measuring how deadly self-driving cars are in comparison to how deadly human drivers are. I never see that, so I wonder how we're going to know when we should loosen the reins on them? Are regulators actually doing their jobs, or are they just going to move inexorably forward while occasionally throwing bones to public outrage?
The stakes here are huge, tens of thousands of deaths and horrific injuries per year, and we will unnecessarily kill lots of people if we deploy self-driving technology too fast or too slowly.
The way this will probably work out in practice is regulators choosing the path of least resistance between uninformed public outrage and greed-driven industry pressure. What we should be asking for is a data-driven approach.
If a human being had done that they would have absolutely had the book thrown at them. There was a case where a woman was drunk driving and hit a man, he came up over the hood and into her windshield. He was alive but bleeding. She drove home and parked her car in the garage. He bled out through the night. NO ONE was sympathetic towards that woman.
But if an automated vehicle does it, it just becomes a line item that someone somewhere has to pay. You watch, someone will start offering insurance for that liability.
What we should be doing is making it an existential crisis for these companies. That will never happen and thus self driving vehicles will become good enough and that's it.
You can punish the manufacturer of any item that creates an excessive danger to the public.
Lawn darts aren't intended to be thrown near people, and don't exist anymore because people used them negligently. Human-driven cars are used negligently and kill 40,000 Americans a year.
Lawn darts don't exist because the FDA banned them. Without that happening, they'd still be sold today.
> Kids can hurt themselves with bicycles and archery and rifles, too. Why aren’t they included? I’d rather be hit by a lawn dart than by a horseshoe
-- R. B. Jarts, inventor of lawn darts.
Consider, the Dalkon shield which had a hospitalization rate of 5 in 1000 and a death toll easily in the 1000s (maybe 10s of thousands). [1] The company went bankrupt but not defunct. It is now a part of Pfizer. Nobody experienced any sort of jail time in association with the deaths (and, after it was banned, they started selling the dalkon shield in poorer regions).
A serial killer killing 10 people gets the death penalty, yet there exist mining companies that have easily killed 1000s of employees with silicosis (and are still operating today, looking at you Dow [2]).
[1] https://en.wikipedia.org/wiki/Dalkon_Shield
[2] https://en.wikipedia.org/wiki/Hawks_Nest_Tunnel_disaster
But that's the idealist side of me talking, the realist understands that will never happen.
It should really be the opposite in an ideal world. A company that has documented "this is what we did to make sure everything was safe" should less culpable than the company that moves fast and breaks things/people/the planet.
If a company killed someone, I'd want their defense to be a list of everything they did to prevent that person from dying, not a "well shucks, accidents happened, and this was a tragedy that couldn't be avoided!". That's the response that should end a company's existence.
Really? If they had been in a car accident caused by another driver, and took a few seconds to pull over because they were unaware they were dragging a person initially hit by the other driver, they would have been charged?
It's very hard to hold people responsible, and it's telling that you had to cite such a grisly and outrageous example. Unless you're drunk or it's a hit-and-run, you're unlikely to face any charges beyond a traffic violation.
> You watch, someone will start offering insurance for that liability.
Of course? Exactly like we've mandated for human drivers since the 1970s?
All this hand-wringing about things that we've accepted our whole lives with human drivers.
I'll say it again: we need data-driven approach to figuring out when autonomous vehicles reach the threshold of becoming less deadly than human drivers. If we jump the gun and allow widespread adoption too soon, people will die unnecessarily. If we drag our feet and allow widespread adoption too late, people will die unnecessarily.
unless you believe this person stayed completely silent during this, the answer is yes. The measuring stick that the law tends to use for liability like this is the reasonable person test and no reasonable person drives 20 feet with someone screaming in pain after having HIT their car.
Are there circumstances in which someone would NOT be charged? sure, there have been cases of toddlers running out in front of cars and it was determined the driver couldn't possible have seen them due to their height.
Since you apparently misunderstood, the point about the insurance is that maybe we shouldn't allow companies to insure themselves against killing people. We as a society absolutely went after Ford for choosing not to do recalls because the estimated cost of the recall exceeded the estimated payout for the deaths and injuries. Under no circumstances should we allow companies to make decisions like this.
This sort of nonsense where you take a serious statement and tell me the cutesy "name" of the autonomous vehicle really detracts from the seriousness of the issue. Also, the last paragraph where they try to talk about how their vehicle wouldn't have made the same mistake as the human driver, glosses over the actual issues the cruise car had which a human driver would have known not to do.
I wouldn't expect this offhanded fact in long-form, slower journalism. But I expect this from Reuters.
So basically nothing you said is relevant.
Why is anyone paying attention to it?
In their original press release they didn't even mention dragging the pedestrian. Here they've only mentioned it because they were caught red handed by the DMV.
Jesus Christ! Good riddance, I'm glad their permits are revoked. Can you imagine witnessing this and watching an autonomous vehicle slowly crushing and dragging someone underneath? That 20 ft must have felt like forever to witness. Let alone experience.
Filling out forms with made-up bullshit just to get an approval works great when you are putting together a SaaS sales presentation, not so much when it's the state Department of Transport on the other end and you are working on matters of public safety.
Fixing all Cruise vehicles so that this doesn't happen might be hard, but it isn't impossible.
Sounds like a good argument to ban almost all human drivers.
I realize assuming this is the California DMV's line of reasoning is speculation, but this sentence from Cruise's own public statement on the event really does it for me (emphasis mine):
> The AV detected a collision, bringing the vehicle to a stop; then attempted to pull over to avoid causing further road safety issues, pulling the individual forward approximately 20 feet.
The tricky thing about these vehicles' control software is that it has absolutely no judgment or common sense. That's really on display here, where the vehicle immediately goes into a canned program instead of following a course of action that takes into account all the details of the situation in which it found itself, not just the ones that its creators had anticipated.
That doesn't necessarily mean that testing self-driving cars on the roads is inherently irresponsible, but it does mean that the ground rules for how we manage such programs need to be tailored to the situation. What we have here is a situation where a rather nasty software defect has been discovered. We can assume that this defect is present in all of the vehicles Cruise is operating on public roadways. It is therefore reasonable and prudent to temporarily suspend autonomous testing operations on public roadways pending analysis and remediation.
Because, let me emphasize again, TESTING. Pausing to analyze and iterate when you find problems is an essential part of testing. It's arguably the main thing that distinguishes testing from just YOLOing shit into production.
Are you saying that they should all be equally safe? If so that would require a level of oversight and control (plus knowledge share?) not typically seen in US government, no?
I agree in principle, I’m not defending them at all. However this is exactly what much of the US wants, given how much de regulation and reduced “red tape” is often argued for.
Until then, we’re going to get some companies that put everyone’s lives at risk. Hell Tesla alone has over promised to insane degrees (the Full in FSD). I don’t see companies having less power any time soon.
Musk has repeatedly stated that Teslas on FSD/AP are safer than humans, now.
You can take issue with the veracity of his claims (and I certainly do, on several levels), but let's not pretend like "no-one is selling this, yet" - let's also not forget, "The driver is only in the seat for legal reasons. The car is driving itself" - that was six years ago now.
There are rules governing autonomous vehicles in California. They must report all accidents involving their self driving vehicles[2], miles driven, disengagements,... Tesla's FSD does not follow these rules.
Tesla's marketing may imply that FSD is an autonomous vehicle. However, today, in California, these are driver assistance programs and not self driving cars.
[1]https://thelastdriverlicenseholder.com/2023/02/17/2022-disen...
[2]https://www.dmv.ca.gov/portal/vehicle-industry-services/auto...
> The [human-driven] Nissan Sentra then tragically struck and propelled the pedestrian into the path of the AV. The AV biased rightward before braking aggressively, but still made contact with the pedestrian. The AV detected a collision, bringing the vehicle to a stop; then attempted to pull over to avoid causing further road safety issues, pulling the individual forward approximately 20 feet. The driver of the Nissan Sentra fled the scene after the collision.
If I'm right, then Cruise will have to add remote drivers and then we can call them remotely operated cars instead of driver-less cars.
Context: When sitting in the backseat of a Waymo, there is a tablet-sized screen that shows you all the cars nearby. And humans! On one of my first trips (at night), it took me a while to figure out what the white circles were. And then I looked over and saw people in the shadows! It was honestly kinda spooky that the car was much more aware of nearby humans then I was...
Disclosure: Googler, and a big fan of Waymo
They are testing their garbage in dense urban areas. Too many times their vehicles just stop in the middle of the road.
If it rains, vehicle stops.
If the light is out, vehicle panics and stops.
Somebody puts a cone on the hood, vehicle panics.
If a single variable not accounted for in their algorithms shifts beyond expected range, it fucking panics.
So much money dumped/wasted into “autonomous personal vehicles”. How about we invest in making our cities more pedestrian, bike, and public transportation friendly? Let’s move people more efficiently rather than further digging ourselves into this hole.
A developed country is not a place where the poor have cars. It's where the rich use public transportation.
Cars also don't only exist in dense cities. They're even used, among other things, to get people into and out of those cities
Alternatively if you reject the premise of those questions that this is a necessary phase to pass through to get to the other side, what do you think the impact on progress would be if no "real world" training opportunities were allowed for these companies? Do you think they'd be hampered and progress would take longer, and what would the impact of that be?
Or more succinctly: "You can't make an omelette without breaking some eggs. Sorry, eggs [regular people].
Ends justify the means thinking leads to some dark places (e.g. what kind of medical-technological progress would we be able to make if it was possible to do testing on humans like we do on animals?).
I'm not trying to make a statement or a point with the question. The original question is all that is part of the question. Please don't try to read more in than what is there.
> Ends justify the means thinking leads to some dark places.
For sure, but if we try to avoid any and all pain then we could still be nomadic people. Maybe life satisfaction then was higher, but certainly life expectancy wasn't. Just the death rate of babies and women alone was pretty dangerous.
So, there must be a balance somewhere right?
You're presenting a false dichotomy. Giving self-driving car companies free reign to cheaply experiment on the public is not the same as "avoid[ing] any and all pain" or avoiding all technological progress.
I'm not presenting a dichotomy at all (unless you think the spectrum is binary and is only zero pain or non-zero pain, which I don't), let alone a false one. I'm not talking about just self-driving car companies. This is a broader philosophical discussion about technology in general, and how much pain is tolerable for progress. If you believe zero is tolerable in the case of self-driving, then do you believe that in general for all tech? If not, why is self-driving so different?
Do you disagree with the premise that on the other side of self-driving cars is a world where humans don't have to labor for their transportation?
Edit: to clarify because people seem incapable of non-binary thinking, I'm not making an argument that Cruise (or any other tech company or car company or whatever) should be able to have "free reign to cheaply experiment on the public." That's not something I said or implied. I don't think they should and I think this California decision is probably a good one. There, can we talk about the higher philosophical issue now?
Then why are you saying things like "if we try to avoid any and all pain then we could still be nomadic people"? That seems pretty binary to me.
> This is a broader philosophical discussion about technology in general, and how much pain is tolerable for progress.
No, it's about who bears that pain. Is it the public or the investors? Obviously the investors and their apologists would prefer to externalize as much as the cost ("pain") as possible onto the public, and shame the public into accepting it.
> Do you disagree with the premise that on the other side of self-driving cars is a world where humans don't have to labor for their transportation?
That's actually an irrelevant distraction, especially if for someone who rejects ends-justify-the-means logic. Do you disagree with the premise that on the other side of unrestricted and unethical experimentation on humans is a world where many more diseases are cured and life is greatly extended (for some, at least)?
> Then why are you saying things like "if we try to avoid any and all pain then we could still be nomadic people"? That seems pretty binary to me.
Good question and thank you for the opportunity to clarify. My position/current opinion is that practically nothing in life can be done without some sort of pain or "harm" to self or another person. For example if the standard was "no harm", then there's no way we could have cars or motorized transport, or even bicycles for that matter. That doesn't mean that any and all pain is acceptable (that would be the binary view), but that basically it's a question of balance, not one of binary "yes" or "no."
> No, it's about who bears that pain. Is it the public or the investors? Obviously the investors and their apologists would prefer to externalize as much as the cost ("pain") as possible onto the public, and shame the public into accepting it.
I don't agree that is what the broader discussion is about, but I would certainly agree that "who bears that pain" is highly relevant to any concrete example, and certainly humans are much more accepting/tolerant of damage/harm that they don't personally have to bear. I would agree that investors should not be unilaterally making decisions on what harm is acceptable.
> That's actually an irrelevant distraction, especially if for someone who rejects ends-justify-the-means logic. Do you disagree with the premise that on the other side of unrestricted and unethical experimentation on humans is a world where many more diseases are cured and life is greatly extended (for some, at least)?
This is a interesting point, and one I'll continue to think on. My initial reaction is that ends-justify-the-means isn't/shouldn't be binary either. Some level of ends-justify-the-means is necessary for nearly everything unpleasant we do in life. For example, I just cleaned the toilet (which I hate doing) because the ends justify the means: if I don't clean it, it will get disgusting, and having a clean toilet is great so I make the sacrifice.
That said, the broader understanding and common meaning of ends-justify-the-means is absolutely one I reject and I suspect we agree fully on: There are potentially great ends that don't justify unethical means. An example of ends-justify-the-means would be the Unabomber justifying his terrorism because on the other end would be a better world. I find that horrifying and very, very wrong.
> Do you disagree with the premise that on the other side of unrestricted and unethical experimentation on humans is a world where many more diseases are cured and life is greatly extended (for some, at least)?
I would not disagree with that, and I would agree that this is not on its surface a good reason to do the experiments. I think the non-binary part comes in on how we define "unethical" however. To some people unethical would include allowing dying cancer patients the option to try potentially life saving but still experimental drugs. To me, very little is unethical so long as the person is a capable adult, is fully informed on the process, has the ability to withdraw at any time, and provides full consent. There are quite a few people who would disagree with that however, so in practice I think we still end up at the spectrum rather than the binary.
The problem in this case happens to be that the AV tried to pull over instead of stopping in the middle of the road, so not sure this is a great criticism to make.
There are tons of companies that would love to help (get paid to) build out public transit infrastructure in cities. You think any major city would have trouble attracting bids to build tram systems, subways, or even buy more busses?
Look at your local leaders, how many of them are pro-public transit? How many of them have a long history of blocking projects because they want to "preserve neighborhood character" or some other lame excuse?
I don't hate public transit or anything, I cycle and subway in NYC all the time, but the level of investment the US needs in public transit is an order of magnitude above the amount that investors have spent on autonomous driving.
[1] https://www.bloomberg.com/news/articles/2023-02-23/in-nyc-su...
More seriously, the statement cites four specific clauses that caused the permit to be suspended. The second one seems really interesting as it implies that there was a lie of omission where the others are more or less what I’d expect regarding safety risks:
> Today’s suspensions are based on the following:
> 13 CCR §228.20 (b) (6) – Based upon the performance of the vehicles, the Department determines the manufacturer’s vehicles are not safe for the public’s operation.
> 13 CCR §228.20 (b) (3) – The manufacturer has misrepresented any information related to safety of the autonomous technology of its vehicles.
> 13 CCR §227.42 (b) (5) – Any act or omission of the manufacturer or one of its agents, employees, contractors, or designees which the department finds makes the conduct of autonomous vehicle testing on public roads by the manufacturer an unreasonable risk to the public.
> 13 CCR §227.42 (c) – The department shall immediately suspend or revoke the Manufacturer’s Testing Permit or a Manufacturer’s Testing Permit – Driverless Vehicles if a manufacturer is engaging in a practice in such a manner that immediate suspension is required for the safety of persons on a public road.
This looks like Cruise's fault, dragging a pedestrian while pulling over for "safety"?
In general, getting out of the way is the right course of action, but this clearly shows that there are exceptions. I'm not sure that the vehicles are currently equipped for differentiating to even be possible.
I'd expect the car to use cameras and sensors.
That might be the fix in this case, just hard stop for all cases of pedestrian impact.
Remain where you are until a trained professional gives you directions. Fire/police/ambulance should have some way to direct the vehicle.
Imagine having to write the code that "detects" if a pedestrain is caught underneath the car
It could also be a "the software can't anticipate that this would be happening the way a human can because it doesn't have object permanence" problem. That's probably at least part of how a human would do it. But I have a suspicion that we're very, very far away from being able to get computers to do that in any sort of reliable way. So I'm guessing we're probably looking at specialized sensors if we want to handle this kind of situation appropriately.
Basically some sort of self-driving car equivalent of a human driver getting out of the car to check and make sure everything's OK.
I will say that I recognize that the visual representation may not be a 1:1 (in fact quite certainly isn't) mapping of the computer's "world view".
But it was quite ridiculous and definitely disconcerting seeing a level crossing with a train going through it and seeing the Tesla I was in acting like there was a traffic light that was erratically going red, while dozens of semi trucks flickered in and out of existence across the screen in front of me.
But I really don't expect that from the cars that go around without a driver.
Anyone who has ever ridden in a Tesla knows they have no real perception of the world around them. Even just driving down the road they show icons of the things around them, which will randomly appear and disappear or change from busses to pedestrians to cars.
It may have some ability to track specific things in specific ways in a manner that's hard-coded by engineers. But if that's the same thing as an actual mental model of the world then that implies that we've had a significant piece of AGI in the form of many video games for about half a century now and just not known it.
audio is difficult because a human can differentiate a woman screaming from the sounds of construction, etc. And also can tell that it's someone in shock vs someone in pain.
I'm unsure if audio processing is good enough for that yet, but imagine if someone were to try and you could get these vehicles to stop by screaming near them.
Or…insert how many other edge cases? There are plenty of scenarios where it shouldn’t pull over, aren’t there? Training the myriad unique situations and proper responses just feels untenable, at least with current technology.
It's hilarious that the fucking things park themselves if you put a traffic cone on the hood, but if they get into a crash, especially one with a pedestrian? BEEP BOOP MUST MOVE!
Besides that, hard edge cases will always exist and can only be solved by a human (or an AGI?) in the loop. But they are't too common, so just detecting that the situation is not solvable by the algorithm, doing nothing and waiting for a remote operator should scale well enough.
yeah, that's a pretty blatantly terrible cover up, definitely not the sort of behaviour you'd want from a company who's allowed to test experimental tech on public roads.
Yes, very much so.
In previous discussions Waymo and Cruise were typically lumped together — “self driving bad” — usually by the original article even! However, my personal experience both as rider and as former Google employee says that Waymo is holding itself to a higher standard and therefore having higher order problems. That’s a lot of nuance for a flame-y topic but this permit revocation is rather unambiguous and I am having a smidge of schadenfreude as a result.
Edit: There are dozens of us https://news.ycombinator.com/item?id=38003860
The thing about trains is the rails are only made for trains.
I don’t think it is a good idea to mix human and machine driven cars, especially in city roads where there are cyclists, pedestrians, trucks.
I would love to see AV only lanes restricted to geofenced parts of the cities.
That’s a pretty enormous screw up and not one I’d heard about when this was initially reported. Interesting the statement from the DMV implies they were lying…
> After coming to a complete stop, it then attempted to do a “pullover maneuver while the pedestrian was underneath the vehicle.”
So the city and DMV focussing their efforts on this corner case driving systems seems like dereliction of duty on the City attorney's and DoT's part.
[1] https://sfstandard.com/2023/07/01/ask-the-standard-san-franc...
[2] https://www.youtube.com/watch?v=yUzc15BeAaE
And the hit and run driver did just drive off
But the big issue is hiding / eliding the data.
With respect this part is not clear. Only that the woman moved 20feet.
The DMV has not made clear whether she moved 20feet from the location of original Collision (ie with the human driver) or she was dragged 20feet as the cruise car kept driving. They’ve let the audience fill in the blanks and we’ve collectively come to the more exciting conclusion.
The city elected to do the easy useless thing first and is patting themselves in the back (people are still gonna be flying due to shitty human drivers tomorrow). This is a problem.
I dont think you can definitively say a person would have stopped and helped.
> Firefighters obstructed the sensors of the driverless car to alert the Cruise control center. He said representatives from Cruise responded to firefighters and “immediately disabled the car remotely.”
- Not be forthcoming with authorities on releasing information/video
- Be misrepresenting the performance of their vehicles to the authorities
This is just not acceptable for a company to be doing. This is not how we disrupt industries. You cannot "fake it till you make it" with road safety when building vehicles that traverse roads safely is your business' entire raison d'etre! That's just unacceptable, full stop.
And, add to it, Cruise has been given a direct and clear path to resuming operations, which essentially boils down to doing the thing they should've been doing already. Like come on! How much more benevolent can we ask a regulator to be in this scenario? One could argue, frankly, that they're being too kid gloves about a company lying to an organization who's primary goal is ensuring safety about safety!
The DMV was not aware of that until notified by another government agency that was investigating the incident.
During the certification process for these self-driving cars, incidents of the cars “misbehaving” (for lack of a better term) were supposed to be reported. However, if a human driver intervened, then it didn’t count as the car itself making a poor choice.
Example: self-driving car approaches an intersection clipping a reasonable speed and the light turns yellow. While a human driver would understand that it’s better to go through than to slam their brakes, the self-driving cat goes “yellow means red soon, stop now”. The human driver overrides and floors it through the intersection. Had the car stopped, that would’ve been a mark against the autonomous system. But since a human intervened, it doesn’t “count”.
This is a good watch on the state of self-driving cars and how policy makers have largely just said “eh, fuck it” in places like SF. https://youtu.be/pmGOjHi-7MM
And this brings a question to me: How did they create, develop, this post-collision routine? Afaik, the overall program/system guiding a self-driving is some combination of trained neural-network routines and more standard robotic feedback loops. But "what to do after a collision" seems like one of those corner cases that neither programming approach would or could deal with so what development created this routine? It would have to be pure ad-hoc-ery and that suggests that in Cruise's approach to the last-mile/endless-corner-cases problem of self-driving cars is just that, a series of ad-hoc fixed. And this approach can neither be safe nor can it scale.
And further, I have to presume Cruise and the others have a bay of human pilots that are called-on to deal with difficult situations. Why didn't the "you've just had a collision" signal not pull a human? (maybe it did and not saying that is their problem but otherwise..) The main explanation I can think of is ... the car's routines are already calling pilots a lot and the team is working feverishly to reduce this. An ad-hoc pull-over routine is intended to reduce those calls back to the pilots. Admittedly this is speculation but if true it reinforces my conclusion above.
And then there's a momentary thought of this, and then I think how bad/hard it would be. Sitting there in a "call center", "Ding!" and you're suddenly thrown in to a car mid-collision, needing to take instant surveying of the situation (like "there's someone partly trapped under the vehicle", "am I in the middle of an intersection/interstate", etc., et al.
Even getting past that I think such a job would play absolute hell on epi, norepi, dopamine, etc., response and those people would burn out, quickly (and I say that as someone who spends a lot of time waking up to alarm tones going off before jumping in an ambulance or fire engine...).
The remote operator cannot manually operate the car under these circumstances and that is by design.
I’m just going to assume it’s a liability thing:
* Cruise gets in an accident and stays in the road, passenger gets out and gets hit. * Cruise gets in an accident and pulls to the curb, passenger can get out on the curb (and if they don’t, and get hit, Cruise can claim it pulled over).
I could see this decision being made (or reviewed by) legal. Which sounds like a truly awful way to build a product.
I agree with you on the adhockery as well. Sounds like it.
Waymo handles this by notifying riders when vehicles or cyclists are incoming near passenger exits.