It makes a lot of sense to me, we're way off an AI being able to cope with random construction sites. But then it does have to be able to cope with out comms, for example there was a recent self-driving truck case where the truck came to a stop because the remote office had just lost power - that would be a mess if there were thousands on the streets.
> we're way off an AI being able to cope with random construction sites.
And this is why level 5 autonomous cars are basically strong AI: current roads assume that a human controls every vehicle. When some guys want to unload a truck full of bricks, they park, turn on the flashers, and wave traffic through in alternating directions. Humans understand this, but good luck teaching a robot. The Nissan solution of having the robot freak out for 30 seconds before a remote operator moves it out of the way is better than nothing, but hardly great.
This just gave me a thought. What if the onboard self-driving system doesn't really need the teleoperator to take over completely, but to select from a set of decisions in difficult situations? The teleoperator doesn't need to watch the footage continuously, but only needs to respond quickly when the onboard system generates an alert.
Only a subset of decisions (can I proceed as normal) are easily dealt with as yes/no answers. "How do I proceed through this environment which is playing havoc with my sensors?" is less straightforward.
Of course, there's a big class of problems which can't be dealt with by remote takeover at all, such as "should I hit this unclassified small obstruction that's entered the roadway or brake/swerve in a manner almost certain to result in a collision?"
This is the obvious stopgap measure so you can get to market when your self-driving software is very very good but not quite perfect. You use the data from these situations to improve the software, requiring fewer and fewer manual interventions until you only need human assistance in truly exceptional situations like physical damage to the sensors.
I wouldn't be surprised to see an almost-entirely-self-driving car service in certain areas in the nearish future. As long as it's 100% safe and only requires assistance rarely, it makes sense to start building out the infrastructure sooner rather than later.
Level 4 self-driving seems the second-least desirable after Level 0, as it's entirely passive but requires full active attention. It's a ripe situation for distraction, something Tesla cars attempt to combat with the "wheel-squeeze".
From a cynical perspective, it's easy to see a future where we constantly approach, but never quite reach, full Level 5 autonomy - or at least not with existing technology. Maybe Level 5 will prove to be an NP problem. Marketing teams are likely going to promote terms like "Level 4.1 self-driving" or "Level 4.5 self-driving" or "Level 4.9 self-driving", as edge-cases are slowly resolved.
Or regulators step in after too many inattentive human drivers fail to prevent very avoidable Level 4 accidents, and we end up with Level 2 driving with Level 5 parking capability permitted only on specific private properties.
"Level 4 ("mind off"): As level 3, but no driver attention is ever required for safety, i.e. the driver may safely go to sleep or leave the driver's seat."
Level 4 is like a Level 5, but that doesn't let you enable it for every route (e.g. you might be manually driving until the highway, then enable self-driving and go to sleep, then wake up with it parked right after the exit you chosen).
Yup. Humans are allowed to make mistakes and fail fatally. If self-driving cars make a single fatal mistake, then watch out for negative press coverage for weeks. Why? Because every failure can be traced back to a root cause. Every root cause (other than something like another driver literally trying to kill you) looks like an incompetent screw-up after peeling back all the layers. In principle, self-driving cars are basically perfectly safe (as a system). So every failure causes righteous fingerpointing.
...which is also partly the motivation to improve.
I expect self-driving cars long-term to be a lot like air travel. It'll have a reputation for being super unsafe, with lots of people paranoid of it, for decades after it has far eclipsed the safety of regular human car driving. The paranoid hyper-focus and ridiculous press coverage is also what will enable it to become essentially perfect (like passenger air travel is today in the developed world).
> Yup. Humans are allowed to make mistakes and fail fatally.
It's because in almoost all cases, one fatal mistake results in one fatal accident. And in most cases, the failing party is part of the accident - giving us reason enough to believe people to what they can to avoid it.
With AI, one sloppy mistake can be the root cause of 100 fatalities, with no risk to the one who made the mistake. That's part of the reason we don't accept it.
I think you’re right that human drivers makes it a lot easier to blame the victim (even though it’s really the whole system that made it possible for the human to make such a mistake that is responsible... the system isn’t held to account because it’s just so easy to blame the human for making the proximate fatal error).
> I wouldn't be surprised to see an almost-entirely-self-driving car service in certain areas in the nearish future. As long as it's 100% safe and only requires assistance rarely, it makes sense to start building out the infrastructure sooner rather than later.
That's a huge if and it's not clear we're even close.
Seems to exacerbate skin in the game problems of the modern world. Why would a call center employee do a better job at protecting your life than you? And more importantly, why would anyone delegate their life to that call center employee?
I imagine the remote control will be some sort of situational guidance ("proceed slowly to the right of the object and then continue"). You'll still be trusting the soulless machine encasing you to mind your safety.
I'm totally blind so I'd trust the call center employee more then my self. I'd love to have a self driving car that could get me where I wanted to go 99% of the time, as long as the 1% failure mode aborted the trip and sent me back home with out killing me. Something like a call center employee turning the car around and sending me home if the weather was going to be to snowy to make the trip would be life changing for me. Uber and Lift are useful, but not cost effective for daily commuting.
It's a stop-gap if you believe automation is going handle 100% of the problems. It is how things are going to be, going forward, if you think the system is going to stop at 99%+something reliability.
Call-center employment would be fairly stressful and unpleasant but it might compensate for those made unemployed by self-driving trucks (there are 100's of thousands of truck drivers in the US).
Except it won’t work at scale. What happens when it snows and there aren’t enough call center workers to take over for all the drivers who need it? Probably a few times a year around here there is enough snow and it’s enough of a surprise where it takes a while to clean up even major roads so you can see the lines. If self driving cars need lines in the road or else a human to take over in that situation, you’re talking about cities grinding to a halt several times a year.
Having rented a 2017 Altima last summer, and finding it to have only non-adaptive cruise control, I'm somewhat surprised to see an article on Nissan cars having self-driving capabilities. From another article[0], Adaptive Cruise, Lane Assist, and Automated Braking (ACLAAB) are set to launch in the 2019 Leaf. But the article makes it clear that these in no way make the car autonomous, and the driver must constantly be ready to engage. Instead of using a "wheel-squeeze" like Tesla, Nissan's car uses steering wheel torque data to tell if you're holding the wheel.
Combining the two, it sounds like their system is ACLAAB that comes to a full stop and "phones home" when it gets confused. But how often will it get confused? Will it be better than the self-check bag sensor detecting your reusable bag correctly? And how long will it take someone in their call center to "pick up their phone call" and remotely drive the car?
This might be an acceptable situation for driving elderly people to the grocery store. But it's going to cause a great deal of frustration for other drivers - having to deal with cars that stop at random spots on the street or highway - and sit there for up to a minute as someone remotes in to drive.
>I think I find the prospect of a random call center employee being able to commandeer my vehicle more concerning than AI screwups.
Whether the commands are given by an AI or a call center employee, I would additionally highlight the "remote" part (please read as "over a possiby flaky GSM or satellite connection that can have an hiccup at any time).
The way the article (and pictures) describes it, the car will receive a full instruction package before starting to execute it. And all regular systems (collision avoidance, ...) will presumably still be active.
>The way the article (and pictures) describes it, the car will receive a full instruction package before starting to execute it. And all regular systems (collision avoidance, ...) will presumably still be active
Will receive the "full instruction package" IF the connection works at the needed time or however before it is needed.
The approach could be very good for whatever you have time for AND that is not "vital", just as an example to correct a navigation plan, but for something in an "emergency" the reliability of the connection at the right time needs to be taken into account.
You probably think the same about fixing your desktop computing problems, however, lots of people are happy for someone to dial in and fix the machine for them. This is effectively what Nissan will be doing, having tech support that can dial in and do things in superuser mode - remember we won't have steering wheels soon.
The thing is that here may be a small team that send the same fixes to cars all day whilst some complicated roadworks or an event happens.
> You probably think the same about fixing your desktop computing problems, however, lots of people are happy for someone to dial in and fix the machine for them.
That seems like a poor comparison, given how difficult it'd be to kill me via SSH access to my laptop.
This idea will never work. Putting aside all other reasons — the liability equation doesn’t make sense.
Imagine being a lowly “meat bag” in the call center. You’re presented with an endless stream of situations so difficult that the robot can’t solve. Eventually, you make a mistake, and people die. Checkmate.
Driving scenarios that are difficult for both humans and AI are rare. It's not that these are situations "so difficult that the robot can't solve"; it's that they're edge cases the AI isn't designed for. The vast majority of these situations will be absolutely trivial for a human operator, and only "difficult" for the AI because the designers haven't accounted for this 0.01% case yet.
Sounds like 9-1-1 operators. I'm sure there's a way to make it work, but it won't happen over night and unfortunately most processes are the result of learning from mistakes which in this context are potentially life threatening.
Seems different than 9-1-1 operators, who are responsible for passing information to first responders. These autonomous car operators would make decisions such as “go ahead and ignore the construction cones.” They won’t simply say, “okay, I’ll send a police crew out to the intersection to triage the confusing construction cones.”
I'm trying to articulate how pointless this sounds. First, you create some AI system that's supposed to help take the human out of the loop in order to improve safety (allegedly). You realise there are still situations your AI can't handle. You decide to give control to a human outside the car, in a call centre. So now the human brain is back in control, except it's not the human in the car (presumably, there is a human in the car?), but someone completely different.
So you have to spend the resources to develop self-driving technology and to maintain a stable of remote drivers. And all that- for what? Human brains are still in control of cars and car AI is still dumb as bricks often enough to make them less safe than humans. If you had problems with humans forced to jump in and take control at a moment's notice where the car failed, before, now you're making these problems even worse by having someone take control remotely so that they're even less aware of the unfolding situation than the human in the car.
It's just ...pointless. It makes even less sense than a car that drives itself safely only when you're paying attention (hey, I can kill myself when I'm not paying attention just fine on my own, I don't need an AI car to do that).
And what about the claim that the rest of the time the self-driving car is safer than a human? How do we know that self-driving cars are safer than cars driven by a human?
The current problem, as mentioned in the article, is that signage for exceptional situations is made good enough for human drivers, but not for self-driving cars.
When self-driving cars become more prevalent, couldn't we have standardized signage that allows the cars to deal with exceptional road conditions like the example in the article?
Agreeing on industry-wide standards (insert relevant xkcd link here) can make such situations solvable for the machine, by employing well-recognizable clues that signify, for example, 'ignore red light and drive on opposite side of the street'.
43 comments
[ 4.0 ms ] story [ 76.8 ms ] threadAnd this is why level 5 autonomous cars are basically strong AI: current roads assume that a human controls every vehicle. When some guys want to unload a truck full of bricks, they park, turn on the flashers, and wave traffic through in alternating directions. Humans understand this, but good luck teaching a robot. The Nissan solution of having the robot freak out for 30 seconds before a remote operator moves it out of the way is better than nothing, but hardly great.
Of course, there's a big class of problems which can't be dealt with by remote takeover at all, such as "should I hit this unclassified small obstruction that's entered the roadway or brake/swerve in a manner almost certain to result in a collision?"
I wouldn't be surprised to see an almost-entirely-self-driving car service in certain areas in the nearish future. As long as it's 100% safe and only requires assistance rarely, it makes sense to start building out the infrastructure sooner rather than later.
From a cynical perspective, it's easy to see a future where we constantly approach, but never quite reach, full Level 5 autonomy - or at least not with existing technology. Maybe Level 5 will prove to be an NP problem. Marketing teams are likely going to promote terms like "Level 4.1 self-driving" or "Level 4.5 self-driving" or "Level 4.9 self-driving", as edge-cases are slowly resolved.
"Level 4 ("mind off"): As level 3, but no driver attention is ever required for safety, i.e. the driver may safely go to sleep or leave the driver's seat."
Level 4 is like a Level 5, but that doesn't let you enable it for every route (e.g. you might be manually driving until the highway, then enable self-driving and go to sleep, then wake up with it parked right after the exit you chosen).
Humans are able to "Level 5" driving, so...
...which is also partly the motivation to improve.
I expect self-driving cars long-term to be a lot like air travel. It'll have a reputation for being super unsafe, with lots of people paranoid of it, for decades after it has far eclipsed the safety of regular human car driving. The paranoid hyper-focus and ridiculous press coverage is also what will enable it to become essentially perfect (like passenger air travel is today in the developed world).
It's because in almoost all cases, one fatal mistake results in one fatal accident. And in most cases, the failing party is part of the accident - giving us reason enough to believe people to what they can to avoid it.
With AI, one sloppy mistake can be the root cause of 100 fatalities, with no risk to the one who made the mistake. That's part of the reason we don't accept it.
That's a huge if and it's not clear we're even close.
https://blog.piekniewski.info/2017/05/11/a-car-safety-myths-...
https://blog.piekniewski.info/2018/02/09/a-v-safety-2018-upd...
How is a call center employee any different than management who ignored concerns over fatigue on Southwest plains?
But it’s a call center employee that you’re focused on
Not the more obvious “humans are largely garbage to each other if they stand to gain or retain some power over other humans”
People want to be the prison guard, not the prisoner, then prison guard manager
So the indifference doesn’t just stay in the call center.
It's a stop-gap if you believe automation is going handle 100% of the problems. It is how things are going to be, going forward, if you think the system is going to stop at 99%+something reliability.
Call-center employment would be fairly stressful and unpleasant but it might compensate for those made unemployed by self-driving trucks (there are 100's of thousands of truck drivers in the US).
Combining the two, it sounds like their system is ACLAAB that comes to a full stop and "phones home" when it gets confused. But how often will it get confused? Will it be better than the self-check bag sensor detecting your reusable bag correctly? And how long will it take someone in their call center to "pick up their phone call" and remotely drive the car?
This might be an acceptable situation for driving elderly people to the grocery store. But it's going to cause a great deal of frustration for other drivers - having to deal with cars that stop at random spots on the street or highway - and sit there for up to a minute as someone remotes in to drive.
[0] http://www.autoguide.com/auto-news/2017/07/nissan-propilot-a...
I can see a lot of rear-ended Nissan cars in my crystal ball.
> One method is to detect torque on the steering wheel, hinted Musk. “So we might issue a visual and auditory alert to make sure you’re okay.”
Whether the commands are given by an AI or a call center employee, I would additionally highlight the "remote" part (please read as "over a possiby flaky GSM or satellite connection that can have an hiccup at any time).
Will receive the "full instruction package" IF the connection works at the needed time or however before it is needed.
The approach could be very good for whatever you have time for AND that is not "vital", just as an example to correct a navigation plan, but for something in an "emergency" the reliability of the connection at the right time needs to be taken into account.
The thing is that here may be a small team that send the same fixes to cars all day whilst some complicated roadworks or an event happens.
That seems like a poor comparison, given how difficult it'd be to kill me via SSH access to my laptop.
Imagine being a lowly “meat bag” in the call center. You’re presented with an endless stream of situations so difficult that the robot can’t solve. Eventually, you make a mistake, and people die. Checkmate.
So you have to spend the resources to develop self-driving technology and to maintain a stable of remote drivers. And all that- for what? Human brains are still in control of cars and car AI is still dumb as bricks often enough to make them less safe than humans. If you had problems with humans forced to jump in and take control at a moment's notice where the car failed, before, now you're making these problems even worse by having someone take control remotely so that they're even less aware of the unfolding situation than the human in the car.
It's just ...pointless. It makes even less sense than a car that drives itself safely only when you're paying attention (hey, I can kill myself when I'm not paying attention just fine on my own, I don't need an AI car to do that).
And what about the claim that the rest of the time the self-driving car is safer than a human? How do we know that self-driving cars are safer than cars driven by a human?
When self-driving cars become more prevalent, couldn't we have standardized signage that allows the cars to deal with exceptional road conditions like the example in the article?
Agreeing on industry-wide standards (insert relevant xkcd link here) can make such situations solvable for the machine, by employing well-recognizable clues that signify, for example, 'ignore red light and drive on opposite side of the street'.