The guy seemed really impatient, talked over the operator while she was trying to give instructions, and then refused to follow instructions given by the phone operator.
Judging by how short the video is, I’m assuming that when he just did the 2-click thing the operator told him to do on his phone, the Waymo probably stopped and let him out.
I get vibes he did it on purpose for social media. For example, you’re supposed to be at the airport an hour before your flight, and here he is winging about a Waymo going in circles for a couple mins and threatening to send the bill for his flight to the operator? Eh?
Are you perfectly rational and calm when trapped in a malfunctioning vehicle that could harm you on a whim? I feel it's unfair to judge this person sitting relaxed in front of a computer.
I'd be impatient too if I was in a robot car that suddenly started driving erratically. There's no way to know whether it's going to continue harmlessly driving around a parking lot or suddenly do something dangerous.
I probably would have looked for a stop button in the app before calling a phone number because that's faster. It seems weird there isn't a physical stop button inside the car.
Well, erratic seems like a bit of an exaggeration. It’s continuously driving the same circular route around a parking lot, but it’s not driving on a kerb or anything like that.
You can see on the map that it goes into the drop off lane, for some reason thinks it can’t drop him off, so drives back into the car park, turns around, and heads back into the drop off lane. It’s not like it’s stuck in full lock doing donuts.
Sitting inside a malfunctioning robot taxi, helplessly, with no clarity why its doing that. And being afraid it could do some thing more erratic, like going fast and hit a wall, or do some other life threatening thing.
Bugs like these are not acceptable in machines where humans sit.
The headline is clickbait -- the car absolutely was not "spinning in circles".
It was driving around a parking lot multiple times.
I agree that clarity would be appreciated -- since this seems to be a routing issue, if the car explained it couldn't find a spot to stop so it was going around again. But of all the problems a self-driving car could have, safely repeating a loop around a parking lot is about as benign as you can get.
Today it is driving in circles, tomorrow it is spinning in circles, next month all your doors lock up in 100 degree heat and your phone dies while you wait for support to pick up. Undefined behavior is undefined behavior.
"hey, you were just stuck in an elevator going erratically up and down never opening the door for you to leave or stop. I don't know why you're freaking out about it"
With most software I am immediately concerned about what other unforeseen problems might happen as soon as it seems like the system has entered some kind of un- or semi-handled failure state.
It's not illegal, but definitely erratic — the kind of thing the cops might well pull you over for, no? I understand the urgency since having to make a flight is one of the most expensive scheduling issues you could face.
Why would cops pull you over for circling a parking lot a few times?
This is extremely normal driver behavior at airports when you're waiting for someone arriving to come out, and you're not allowed to stop at the curb for longer than a minute.
If it's literally "a few times", maybe not. If it went on for several minutes, and you weren't obviously looking for a space to park, it wouldn't surprise me. Any 'unusual' behaviour in a car is a potential cause for being pulled over.
>It seems weird there isn't a physical stop button inside the car.
Think it through once more.
For the down voters: It would be extremely dangerous if somebody pressed such a button when the car is going down the highway. The real world is not a video game where you can just abort or pause at any time. If you try, you will die.
An emergency stop button for a robot car shouldn't immediately disengage all other automation and apply maximum braking as the parent comment seems to be implying. That would, indeed be very dangerous.
An emergency stop routine should involve a code path designed to bring the car to a stop in a safe location as soon as practical. It should be thoroughly tested on degraded hardware with minimal branching relative to normal operation. It should also activate the hazard lights, contact the operator's emergency department, unlock all the doors, and lower the windows.
I bet it would. I read the comments before watching the video, then I watched it. All the comments saying it's a stressful and dangerous situation because it could potentially become a dangerous situation make me facepalm. It must be American "(over-)abundance of caution"...
I felt he was rather calm given that he was inside a malfunctioning autonomous vehicle. If a human taxi driver drove me in circles rather than taking me to my flight I would probably send them the bill!
In his defence, I am imagining this being a really stressful time for the passenger. Many people - most, even - would be panicking at this point, and people react differently and irrationally in these kind of not-in-control scenarios.
I mean, I get vibes you work for Waymo based on how speculative and dismissive you are in this comment... but that's probably not the case, eh? Perhaps the vibes are off?
“Please don’t post insinuations about astroturfing, shilling, bots, brigading, foreign agents and the like. It degrades discussion and is usually mistaken. If you’re worried about abuse, email hn@ycombinator.com and we’ll look at the data.”
https://news.ycombinator.com/newsguidelines.html
> Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.
I obviously don't think the person I'm responding to works at Waymo. The person I'm responding to is making a baseless claim that this is a manufactured issue. How do you degrade a discussion that starts at the lowest possible place when the claim is "the vibes are off"
> I’m assuming that when he just did the 2-click thing the operator told him to do on his phone, the Waymo probably stopped and let him out
That's not really good though. It is not his job to fix the car. His protestation that the operator should do it, whatever "it" is, is a reasonable one.
I was askance at him refusing the helpdesk request to do something in the app. One of my hats for decades was support and this feels like someone refusing to tell you what's on their screen, or whether a certain light is on, or to push a button.
Utopia? I never experienced it myself, but the footage of the American roads in the video looks as dystopian as it can possibly get already... Somehow, the elevated freeways and what's below them remind me of the worst cyberpunk sprawl nightmares. I guess being free from the constraints of dealing with any buildings or areas older than 200 years doesn't necessarily lead to better designs...
That might be true for some of the extreme performance EVs (Rimac Nevera: 2360 Nm, Lucid Air Sapphire: 1940 Nm), but I think the mechanical brakes can probably overpower the 700 Nm of Waymo's Jaguar I-Pace.
I read that Model S accelerates at more than 1g and I think it has 1000 nm of "instant torque" from zero. Roadster has 10000 nm. They say top notch mechanical disc braking can decelerate at less than 1.3g tops. Should be enough for 700 nm but seems like even many consumer grade EV literally can't stop with your foot if electronic brain doesn't cooperate.
The maximum deceleration is the traction limit of the tires; most braking systems on cars have the mechanical ability to exceed it, which is prevented by ABS on modern cars. If both the motors and the brakes can exceed the traction limit, as with the Nevera, then we don't know which is more powerful.
As an aside, spurious ABS activation is a form of runaway car automation that has been possible for decades. It is terrifying to experience.
But of course. Standard issue from Q Branch.
Anyways, Hey dudeinjapan, from also a dude in Japan (nearing 3 decades btw). Pardon the thread hijack because I don't see any way to DM here. I came across your contribution to an old thread (A mistake that killed Japan’s software industry); would like to pick you brains further.
Taking the whole software/communication/hardware complexity into account, imagine, how many bugs and vulnerabilities may be present in such non-open-source systems? Based on the recent history where we know even 0-day phone exploits exist, how dangerous would it be if a vehicle (or most of them simultaneously) can become being controlled remotely?
What a sensationalist headline. The car slowly drives around in circles on a parking lot and the passenger apparently does not feel in danger (being on the phone with CS and taking videos for social media).
These cars need an emergency stop button which brings the car to a halt as fast as could plausibly be safe and then opens the doors. This is standard equipment on all industrial machinery and used to be standard equipment on all trains as well; I got into a lot of trouble once for pulling the emergency stop cord on a train when I was young. The inconvenience and occasional danger of unwarranted emergency stops is almost certain to be outweighed by the lives saved in the rarer cases where the E-stop is necessary.
> I got into a lot of trouble once for pulling the emergency stop cord on a train when I was young.
You are the hero of every young child that has ever been on a train and wondered what would happen when they pulled that cord. I know that certainly was a dream of mine when I was four or so. I couldn't understand why my dad was so insistent I not try it.
"Well, I got in a wreck and had 3rd degree burns, but at least my account is in good standing!" (Yes I know it's a totally different scenario to the article's)
It wouldn't decide whether it's safe to stop. A separate E-stop processor would have a preprogrammed stop routine that stops the vehicle as fast as it can stop safely, which is to say, without skidding and without making rear-end collisions inevitable if you're on the freeway.
> and without making rear-end collisions inevitable if you're on the freeway.
That means it has to decide whether it's safe to stop.
Stopping safely is no less complicated than the self-driving software in general. Sure you can have backup driver software, but it's just going to be the same software.
It does not have to be the same software. Its task is much simpler, and many situations where the normal software has to look for an optimal resolution can be handled by activating the hazard lights, stopping the car, and telling the passengers that the emergency stop is complete.
People talk a bunch of stuff about malfunctioning, bugged software and so on. We can assume, the car was just trapped in some infinite loop so it'll never reach it's goal for some reason but also wasn't able to realize that and call the routine for that. Fine.
It's not a dangerous situation for the passenger but it definitely something that makes you feel very uncomfortable which could lead to panic because you're trapped. The doors are locked and even if they weren't you cannot simply exit the car. That's okay. Trains don't allow this either to open the doors on a track section. You need a supervisor to do that. In case of an emergency you can still unlock the doors mechanically. So after pulling the emergency brake you can still do that. It would be interesting how these cars behave in case of a fire inside or something which leads me to my final thought:
In case of issues on or with a ride in a theme park, there are usually operators. So in case of a technical malfunction there's an emergency stop which should make disarm systems in a safe manner and bring the ride into a safe position or at least make sure it stops safely. Of course it's not possible to trigger an e-stop as passenger but an operator can do that in case you can somehow make clear that it's needed right now.
The situation in the video is a very bad experience because a phone call from the passengers device is needed and he was asked to do something in the app. Probably to identify the car and enable the remote control. Elevators handle this much better. There's an emergency system in the cabin which transmits all important information to the service center, so it's immediately clear where the emergency call originated from. An app on a smartphone should never replace this installation!
When I pulled the emergency stop cord on the train, I also pulled the emergency door opening lever. The doors opened, on a track section. I've used an emergency door opening lever more recently when the train had already stopped and we were being instructed to exit because of a mechanical problem (possibly a fire), but that was at a station.
Elevators, in my experience, also universally have E-stop buttons or switches.
No, it doesn't. It just needs to brake slightly less hard than the antilock braking system is capable of. There is no complexity there. The only complexity is in ensuring that the E-stop processor can reliably override the rest of the car's control systems.
Yes, I agree that pushing the E-stop button is not free of risk. But it's a risk under the control of the person who will suffer most of the consequences.
No, I'm driving normally at 50 in the rain or whatever and some AI hype-wagon in front of me has slammed on the brakes and swerved across my lane.
The CEO of HypeAI releases a heartfelt video in which they say they are truly sorry for my family's loss and by way of compensation offers them unlimited credits for HypeRides in select US cities conditions apply.
If the hype-wagon is swerving across lanes because of control software bugs, you definitely want the person in it to be able to hit an E-stop button so that it stops swerving after a finite number of lanes.
It's only dangerous to the people behind you if they are following you and each other far too closely. Also, the vast majority of multi-car pileups in the US kill zero people. In the record-breaking 168-car pileup two years ago in Louisiana, only 7 people died, an average of 0.04 deaths per vehicle: https://edition.cnn.com/2023/10/23/us/louisiana-fog-i55-cras...
Also, as multiple people have pointed out, all non-self-driving cars already have this feature; it's called a brake pedal. Providing it should not be controversial.
> Also passenger might be inebriated and randomly push the button.
Modern societies are reasonably able to cope with potential misuse of publicly-accessible emergency equipment. Buildings have fire alarms. Airplanes have emergency exits with self-deploying slides. Trains have emergency brakes passengers can activate.
Misusing any of these can create a danger to others and is sure to be very disruptive. People do on occasion, and face punishment for their actions.
Embarrassing as it is to admit this, I was technically an adult. In my defense, I thought it was an emergency. I was wrong. They didn't even kick me off the train. We were back on our way a couple of minutes later.
There was no emergency! The car wasn't "spinning", it was just driving around a parking lot and failing to find a location to stop. This was a navigation failure, not a control failure. And the solution is to call the human operators who can control it remotely. Which is what happened, and they did. Pulling an "emergency stop" level would have made things worse and not better.
Is it embarassing for Waymo? Sure. But the BBC basically lied to you here, and you bought it. The circumstance you're imagining is not what happened. Just watch the video.
Presumably it pulls over and stops if the backend connection drops, no? This is a kind of goalpost motion: you're countering an argument about an incorrectly-identified failure mode by positing that the condition would still obtain if there was a dual mode failure instead (i.e. that the car would get stuck in a navigation loop and have a bug in its recovery from connectivity issues).
That's bad engineering, because you can always get to some kind of catastrophe if you keep piling on layers. Might as well just skip to "What if the motor circuit fails open and it accelerates forever?!" in that case.
Uh, that was sarcasm. Unintended acceleration failures are huge deals in the industry and absolutely already designed in with multiple (probably dozens) of layers of safety/redundancy at all levels up to and including human control. The point was that you can still imagine a N-mode failure that defeats all of that and use it to justifify whatever whizgig you're demanding in your local internet discussion. And that this was ridiculous, because that's not how you do engineering.
Yes, you can always imagine failures that disable your failsafes. But you seem to be using that as an excuse for advocating omitting even basic failsafes. That's also not how you do engineering.
i am not sure i would say it was "just driving around a parking lot" it was not spinning but it _was_ just going in circles, watch the steering wheel and the background.. right turn pause right turn pause etc you see the same buildings etc. my spouse would have probably barfed all over that back seat
I wasn't imagining a car spinning out of control. Being trapped inside a car indefinitely is dangerous: you could get carsick and have to throw up, urgently need to use a restroom, have a painful leg cramp that causes a lasting muscle injury, miss picking up your child from school, miss your appointment with your probation officer, be unable to obtain essential medication, suffer from heatstroke, or die of dehydration. An adult should never need to "call human operators" and ask their permission to get out of a car. Unless they're being arrested or something.
And if you hear that the car has run over a child that the operator didn't notice and is dragging it down the pavement under the car, it's essential for you to be able to hit the E-stop button rather than calling an operator to eventually stop the car.
> Being trapped inside a car indefinitely is dangerous
It was like two minutes, tops. Again, it's on video!
And I love that you started from a position of needing an emergency escape button and when challenged it's... because the occupant might have to pee really bad.
You literally argue in this thread that "These cars need an emergency stop button" because the occupants might "urgently need to use a restroom". I cherry picked that because it's hilarious, but it's not a lie and you know it. Those quotes are real: those are your words.
Accusing other people of lying when they point out your whoppers is... not good faith argument.
Your comment asserted that that was the only justification I gave. Even if that were true, it would still make you the kind of asshole who makes fun of elderly women with incontinence problems—in fact, who advocates omitting basic safety features from engineering designs because you think it's hilarious when elderly women with incontinence problems wet their pants. Or, interpreting your comment with more charity than the annual UNICEF budget, you think it would be hilarious if elderly women with incontinence problems had to pull down their pants and pee on the passenger seat.
But it's not true, which makes you a liar as well.
Comments like that one undermine my faith in the capacity of humanity for goodness. You managed to combine dishonesty, unprovoked rhetorical aggression, engineering irresponsibility, and fantasies of a breathtaking degree of sadism into a few short lines. It's honestly one of the most contemptible comments I've ever seen on HN.
> Your comment asserted that that was the only justification I gave.
It certainly did not. It's right up above, please re-read.
Just stop. You got yourself into an embarrassing situation with an untenable argument and are behaving badly, calling your fellow posters liars and assholes.
> the solution is to call the human operators who can control it remotely
Sounds like a recipe for failure, to be honest. In a potentially life or death situation, the last thing I want is to rely on a remote human being. Plus, if the device entered an erroneous state, I certainly don't trust it to correctly interpret a remote "emergency stop" signal.
If the machine goes crazy (and there is no world where driving around a parking lot until the end of time is the rational expected behavior), the only safe option is a big, red, cut-circuit emergency stop button.
Which, once more, was not happening here. The car was lost, not out of control. And the built-in solution was applied, and worked.
What do you really want here? You've never been in a Uber that took a wrong turn? Never argued with a cab driver about a route? Never been stuck in an airliner at the end of a runway waiting for clearance? Vehicles do things their occupants don't like all the time, and no one freaks out on the internet about it.
If a cab driver or Uber driver locks you into their car and doesn't allow you to get out, that's "false imprisonment", a felony in most jurisdictions, carrying a sentence of several years of prison. It is very unusual for them to even lock the doors such that you need their help to open them. The last time I had to open an Uber door in a moving car to force the driver to stop—because he'd lied about having air conditioning to get the fare—was two weeks ago.
Airliners are legally required to have emergency exit doors that can be activated by anyone inside the airliner as long as the cabin isn't pressurized. You also know, before you board the airliner, that you won't be able to get off it for several hours. So, if you have an appointment with your probation officer in half an hour or need to pick up your kids from school in half an hour, you won't board in the first place.
You're apparently arguing that passengers of self-driving cars should not have the same legal and mechanical protections that the passengers of these other types of vehicles have.
I'm not saying it was happening here, I'm just taking this anecdote as food for thought. I'm appalled to learn that those vehicles don't have an emergency stop system. A real emergency stop. Not some remote-controlled or AI-assisted stuff. A low-tech, last-chance emergency stop, like the one you have in trains or on any industrial machine.
> You've never been in a Uber that took a wrong turn? Never argued with a cab driver about a route?
If a cab driver keeps cricling on a parking lot, again and again and again, never finding the obvious exit, I'll get a bit concerned and eventually tell him "OK, never mind, I'll find another way, just drop me there". If he refuses to let me off, and locks the doors, and keeps circling, I'll be extremely anxious. This is horror movie material.
> I'll get a bit concerned and eventually tell him "OK, never mind, I'll find another way, just drop me there".
Isn't that exactly what happened here? Rider pushed the help button, got help, and was dropped off. Right there.
I think you get to the truth of the matter with your "horror movie material" quip. You don't think this situation was dangerous. You think it was scary because of the robot. Well, OK. But that's not a safety argument.
It's true that it's probably over in a short time, but that is easily a stressful enough situation to cause people to have panic attacks, heart attacks, and have a range of other issues.
I would almost argue that it's negligent of Waymo to not have a kill switch they can remotely fire, and then another layer of kill switch, and then another 5 layers of kill switches...
You can't fail to control a driverless car under any circumstance.
Waymo rider support seems to be overly limited in their abilities.
Requiring the user to press buttons on an app, and not being able to do it for them is just one example. Plenty of other videos show they can't control the car, and all they can do is talk to the user.
At a minimum, they should have the ability to stop the car and drive it, remote control, at 1 mph, to get it unstuck or out of the way.
I appreciate we have to defend the driving system and nobody was harmed. But for every such mishap the reminder should be that this has to be fixed only once.
The car should have an option to stop, let the passenger & baggage out, then resume its search for a parking spot. Or leave altogether since it no longer needs to be in the airport parking lot.
Were the doors locked, or could he have just opened one and gotten out at any time (obviously when it was going slowly) ? It just seems irresponsible to lock someone inside a car, but then if they weren't, then he wasn't exactly "trapped"
I mean... yeah. the car isn't exactly going fast, and if you get out of the other side then you're on the outside of the circle so you're not really likely to get run over, especially if you just do a little jump rather than rolling on the floor
I wish the video didn't end so quickly, because I'm actually curious what exactly had to happen next. I'm guessing having active involvement in both the passenger in the car and the support staff is a feature, since you wouldn't want rogue support operators to be able to just bring cars to abrupt halts or worse, so if the robot driver is unable to pull over safely, it sort of makes sense to require at least some sanity checking from the passengers themselves.
Of course, it's reasonable to be frustrated, especially if you're in a hurry, but anyone who has been in a vehicle accident in an Uber or taxi will know that there's just some inherent risk involved when you're getting into a car with another driver, just as there is risk involved when getting into a car and driving yourself, and I think the idea that a robot driver could fail and get stuck is something that you just have to be mindful of and account for, the same way you have to be mindful of the risk of getting stuck on an elevator. Without seeing the remainder of the altercation it's hard to judge if Waymo's design is lacking for fallbacks here. Another question would be what can be done if the passenger can't access the Waymo app; would they have to jump into the driver's seat?
While this did happen and obviously isn’t great, there’s a more skeptical side too.
The ‘trapped’ man is the CEO of an AI consulting firm. He posted the video to linked in, didn’t press the abort button, and had a relatively successful customer support experience (5 minutes delayed and not charged for the trip). Apparently he is using a PR firm to handle questions.
Make of that what you will. Waymo can clearly do better, although for what it’s worth — my worst uber experiences have been much worse than this (legitimately unsafe driving decisions).
What is the psychological term for people that defend defective heavy machinery that are not yet ready for production and yet being tested in production and in the general public? This will be the most highly unpopular opinion with this audience but if a human drove around in circles potentially blocking other drivers even in a parking lot a cop would pull them over and possibly have them take a sobriety test. If they passed they would get a warning and the incident would be logged with dispatch. Continued behavior would result in a ticket at best and drivers license suspension at worst. Please stop giving exciting tech a break and expect more. I am starting to lean towards requiring hardware and software developers and their respective companies that work on public heavy machinery to have federal certifications that can be revoked if they are moving fast and breaking things in public. Broken navigation in heavy machinery is a risk to the general public. This case was benign but the next one may not be.
I expect much more by now. I would expect that if anything cars would be actively protecting humans. If a toddler wanders into the road without a parent I would expect them by now to use a PA speaker to alert people and actively create a blockade around the child with only room for humans to pass, flash all their lights continuously and video stream to safety personnel on duty with the vehicle maker.
There's a belief in some circles that because technologies like self-driving have the potential to save lives, any failure to develop that technology as quickly as possible is akin to manslaughter.
I don't think it's a mental illness, just an extreme form of utilitarian ethics combined with an unreasonable faith in the outcome.
That makes sense but I think that if they wish to sacrifice humans in the name of innovation that in theory can save lives, it should be tested in locations where all the humans opted-in to this level of sacrifice. Surely there must be a tech community of people that think like this where such in-production testing can occur. Every port of entry into that zone can have checkpoints requiring vehicles and pedestrians to display an opt-in QR code and some incentive to get them to participate. I know we can't waive negligence but at least we could contain the blast radius to a known zone.
140 comments
[ 2.9 ms ] story [ 211 ms ] threadJudging by how short the video is, I’m assuming that when he just did the 2-click thing the operator told him to do on his phone, the Waymo probably stopped and let him out.
I get vibes he did it on purpose for social media. For example, you’re supposed to be at the airport an hour before your flight, and here he is winging about a Waymo going in circles for a couple mins and threatening to send the bill for his flight to the operator? Eh?
I probably would have looked for a stop button in the app before calling a phone number because that's faster. It seems weird there isn't a physical stop button inside the car.
You can see on the map that it goes into the drop off lane, for some reason thinks it can’t drop him off, so drives back into the car park, turns around, and heads back into the drop off lane. It’s not like it’s stuck in full lock doing donuts.
Sitting inside a malfunctioning robot taxi, helplessly, with no clarity why its doing that. And being afraid it could do some thing more erratic, like going fast and hit a wall, or do some other life threatening thing.
Bugs like these are not acceptable in machines where humans sit.
The headline is clickbait -- the car absolutely was not "spinning in circles".
It was driving around a parking lot multiple times.
I agree that clarity would be appreciated -- since this seems to be a routing issue, if the car explained it couldn't find a spot to stop so it was going around again. But of all the problems a self-driving car could have, safely repeating a loop around a parking lot is about as benign as you can get.
This is extremely normal driver behavior at airports when you're waiting for someone arriving to come out, and you're not allowed to stop at the curb for longer than a minute.
Think it through once more.
For the down voters: It would be extremely dangerous if somebody pressed such a button when the car is going down the highway. The real world is not a video game where you can just abort or pause at any time. If you try, you will die.
An emergency stop routine should involve a code path designed to bring the car to a stop in a safe location as soon as practical. It should be thoroughly tested on degraded hardware with minimal branching relative to normal operation. It should also activate the hazard lights, contact the operator's emergency department, unlock all the doors, and lower the windows.
Also, requiring someone to stop the car via an app is appalling. What if there's no phone reception?
I obviously don't think the person I'm responding to works at Waymo. The person I'm responding to is making a baseless claim that this is a manufactured issue. How do you degrade a discussion that starts at the lowest possible place when the claim is "the vibes are off"
That's not really good though. It is not his job to fix the car. His protestation that the operator should do it, whatever "it" is, is a reasonable one.
https://www.youtube.com/watch?v=040ejWnFkj0
It's very possible that it won't be the utopia it is portrayed to be.
I'm genuinely worried for the future if we keep going down this path. And all the hard work by urbanists will be undone.
NJB radicalised me, in the best way possible.
I read that Model S accelerates at more than 1g and I think it has 1000 nm of "instant torque" from zero. Roadster has 10000 nm. They say top notch mechanical disc braking can decelerate at less than 1.3g tops. Should be enough for 700 nm but seems like even many consumer grade EV literally can't stop with your foot if electronic brain doesn't cooperate.
As an aside, spurious ABS activation is a form of runaway car automation that has been possible for decades. It is terrifying to experience.
https://www.youtube.com/embed/tWPG8alh2Bs
You are the hero of every young child that has ever been on a train and wondered what would happen when they pulled that cord. I know that certainly was a dream of mine when I was four or so. I couldn't understand why my dad was so insistent I not try it.
Violating these rules will impact your account standing and may lead to permanent revocation of ride privileges..."
https://support.google.com/waymo/answer/15486190
That means it has to decide whether it's safe to stop.
Stopping safely is no less complicated than the self-driving software in general. Sure you can have backup driver software, but it's just going to be the same software.
It does not have to be the same software. Its task is much simpler, and many situations where the normal software has to look for an optimal resolution can be handled by activating the hazard lights, stopping the car, and telling the passengers that the emergency stop is complete.
It's not a dangerous situation for the passenger but it definitely something that makes you feel very uncomfortable which could lead to panic because you're trapped. The doors are locked and even if they weren't you cannot simply exit the car. That's okay. Trains don't allow this either to open the doors on a track section. You need a supervisor to do that. In case of an emergency you can still unlock the doors mechanically. So after pulling the emergency brake you can still do that. It would be interesting how these cars behave in case of a fire inside or something which leads me to my final thought:
In case of issues on or with a ride in a theme park, there are usually operators. So in case of a technical malfunction there's an emergency stop which should make disarm systems in a safe manner and bring the ride into a safe position or at least make sure it stops safely. Of course it's not possible to trigger an e-stop as passenger but an operator can do that in case you can somehow make clear that it's needed right now.
The situation in the video is a very bad experience because a phone call from the passengers device is needed and he was asked to do something in the app. Probably to identify the car and enable the remote control. Elevators handle this much better. There's an emergency system in the cabin which transmits all important information to the service center, so it's immediately clear where the emergency call originated from. An app on a smartphone should never replace this installation!
When I pulled the emergency stop cord on the train, I also pulled the emergency door opening lever. The doors opened, on a track section. I've used an emergency door opening lever more recently when the train had already stopped and we were being instructed to exit because of a mechanical problem (possibly a fire), but that was at a station.
Elevators, in my experience, also universally have E-stop buttons or switches.
Also passenger might be inebriated and randomly push the button.
While I agree with you on principle, it's a hard problem to solve.
You are on a beta, experimental, self driving vehicle. The time to update your life insurance has passed.
The CEO of HypeAI releases a heartfelt video in which they say they are truly sorry for my family's loss and by way of compensation offers them unlimited credits for HypeRides in select US cities conditions apply.
Anyone can slam the brakes in the middle of a highway.
I'm assuming the e-stop button would not be right next to the blinker.
Also, as multiple people have pointed out, all non-self-driving cars already have this feature; it's called a brake pedal. Providing it should not be controversial.
Modern societies are reasonably able to cope with potential misuse of publicly-accessible emergency equipment. Buildings have fire alarms. Airplanes have emergency exits with self-deploying slides. Trains have emergency brakes passengers can activate.
Misusing any of these can create a danger to others and is sure to be very disruptive. People do on occasion, and face punishment for their actions.
Is it embarassing for Waymo? Sure. But the BBC basically lied to you here, and you bought it. The circumstance you're imagining is not what happened. Just watch the video.
That's bad engineering, because you can always get to some kind of catastrophe if you keep piling on layers. Might as well just skip to "What if the motor circuit fails open and it accelerates forever?!" in that case.
And... you did.
And if you hear that the car has run over a child that the operator didn't notice and is dragging it down the pavement under the car, it's essential for you to be able to hit the E-stop button rather than calling an operator to eventually stop the car.
It was like two minutes, tops. Again, it's on video!
And I love that you started from a position of needing an emergency escape button and when challenged it's... because the occupant might have to pee really bad.
Accusing other people of lying when they point out your whoppers is... not good faith argument.
But it's not true, which makes you a liar as well.
Comments like that one undermine my faith in the capacity of humanity for goodness. You managed to combine dishonesty, unprovoked rhetorical aggression, engineering irresponsibility, and fantasies of a breathtaking degree of sadism into a few short lines. It's honestly one of the most contemptible comments I've ever seen on HN.
It certainly did not. It's right up above, please re-read.
Just stop. You got yourself into an embarrassing situation with an untenable argument and are behaving badly, calling your fellow posters liars and assholes.
Sounds like a recipe for failure, to be honest. In a potentially life or death situation, the last thing I want is to rely on a remote human being. Plus, if the device entered an erroneous state, I certainly don't trust it to correctly interpret a remote "emergency stop" signal.
If the machine goes crazy (and there is no world where driving around a parking lot until the end of time is the rational expected behavior), the only safe option is a big, red, cut-circuit emergency stop button.
Which, once more, was not happening here. The car was lost, not out of control. And the built-in solution was applied, and worked.
What do you really want here? You've never been in a Uber that took a wrong turn? Never argued with a cab driver about a route? Never been stuck in an airliner at the end of a runway waiting for clearance? Vehicles do things their occupants don't like all the time, and no one freaks out on the internet about it.
Airliners are legally required to have emergency exit doors that can be activated by anyone inside the airliner as long as the cabin isn't pressurized. You also know, before you board the airliner, that you won't be able to get off it for several hours. So, if you have an appointment with your probation officer in half an hour or need to pick up your kids from school in half an hour, you won't board in the first place.
You're apparently arguing that passengers of self-driving cars should not have the same legal and mechanical protections that the passengers of these other types of vehicles have.
> You've never been in a Uber that took a wrong turn? Never argued with a cab driver about a route?
If a cab driver keeps cricling on a parking lot, again and again and again, never finding the obvious exit, I'll get a bit concerned and eventually tell him "OK, never mind, I'll find another way, just drop me there". If he refuses to let me off, and locks the doors, and keeps circling, I'll be extremely anxious. This is horror movie material.
Isn't that exactly what happened here? Rider pushed the help button, got help, and was dropped off. Right there.
I think you get to the truth of the matter with your "horror movie material" quip. You don't think this situation was dangerous. You think it was scary because of the robot. Well, OK. But that's not a safety argument.
It's true that it's probably over in a short time, but that is easily a stressful enough situation to cause people to have panic attacks, heart attacks, and have a range of other issues.
I would almost argue that it's negligent of Waymo to not have a kill switch they can remotely fire, and then another layer of kill switch, and then another 5 layers of kill switches...
You can't fail to control a driverless car under any circumstance.
Requiring the user to press buttons on an app, and not being able to do it for them is just one example. Plenty of other videos show they can't control the car, and all they can do is talk to the user.
At a minimum, they should have the ability to stop the car and drive it, remote control, at 1 mph, to get it unstuck or out of the way.
Yep. Like virtually every phone support person ever. You would think that with driverless cars this would be different.
Maybe the solution is a car rental service where the car drives to you, you drive where you want to go, then just leave the car.
In Boston anyway navigating pickup/ drop off at the airport is tricky for people.
Not sure acting the ignorant consumer makes me trust the supposed dangerous situation you are in isn't manufactured in some way.
Still, typical of poorly designed software.
Of course, it's reasonable to be frustrated, especially if you're in a hurry, but anyone who has been in a vehicle accident in an Uber or taxi will know that there's just some inherent risk involved when you're getting into a car with another driver, just as there is risk involved when getting into a car and driving yourself, and I think the idea that a robot driver could fail and get stuck is something that you just have to be mindful of and account for, the same way you have to be mindful of the risk of getting stuck on an elevator. Without seeing the remainder of the altercation it's hard to judge if Waymo's design is lacking for fallbacks here. Another question would be what can be done if the passenger can't access the Waymo app; would they have to jump into the driver's seat?
"Mike Johns is an exceptional AI expert speaker specializing in AI", whatever you say man
The ‘trapped’ man is the CEO of an AI consulting firm. He posted the video to linked in, didn’t press the abort button, and had a relatively successful customer support experience (5 minutes delayed and not charged for the trip). Apparently he is using a PR firm to handle questions.
Make of that what you will. Waymo can clearly do better, although for what it’s worth — my worst uber experiences have been much worse than this (legitimately unsafe driving decisions).
I expect much more by now. I would expect that if anything cars would be actively protecting humans. If a toddler wanders into the road without a parent I would expect them by now to use a PA speaker to alert people and actively create a blockade around the child with only room for humans to pass, flash all their lights continuously and video stream to safety personnel on duty with the vehicle maker.
I don't think it's a mental illness, just an extreme form of utilitarian ethics combined with an unreasonable faith in the outcome.