The thing that worries me about semi-autonomous cars is that we might end up in the same place we've ended up with aircraft. Aircraft are sufficiently autonomous in nominal conditions that pilot skills atrophy, and then when the plane encounters adverse conditions, these de-skilled pilots simply do not respond correctly to the problem, crashing the plane. Specifically, I'm thinking of something like Air France Flight 447, where it turned out that the pilot and co-pilot had lost familiarity with how the Airbus A330 behaved while at cruising altitude.
Similarly, I'm worried that auto-drive and smart cruise control systems will take care of highway driving in the normal case, but when things go wrong and the auto-driver disengages, the human at the wheel will have forgotten that a car traveling at 60mph behaves differently than one traveling at 25mph. With aircraft, we can mitigate this risk by forcing pilots to spend extra time in the simulator, keeing their feel for the aircraft from decaying too badly. I'm not sure what the solution is for drivers. Maybe we can force everyone to play a racing sim for a few hours every month ;)
This danger will be avoided in automobiles by making human control impossible as soon as that makes any sense. There will be no steering wheel, the front seat will face backwards to facilitate conversation, and accident rates will plummet. The thing that robocars can do, which roboplanes cannot, is slow down or stop completely when faced with vexing situations.
Pulling over to the side of the road and putting your blinkers on is (almost) always an acceptable solution to an automotive error. Much harder to pull that off at 35,000 feet.
It's apples and oranges. Overshooting a turn at by a few seconds at 35,000 feet isn't the end of the world. Missing a turn in a car by even a quarter second means blood on the pavement.
If you have a problem at cruising altitude, there's usually time to deal with it. Transport planes can glide 100-200 miles from that altitude if they have to. Most aircraft accidents happen during takeoff, approach, or landing. See NTSB reports.[1] High-altitude accidents are mostly in-flight fires, including two events involving lithium batteries.
There is, however, one commercial aircraft crash from problems at altitude (no passengers, ferry flight) worthy of a Darwin Award.[2]
That second report is fascinating. One might hope that even amateur pilots would know power curve characteristics at high altitude, and the flight team's "gosh we're saving a lot of fuel" comments on the CVR inspire a sort of grim amusement, but I wonder whether the incident might actually have saved lives in the long run.
If we're going to fly budget carriers like Pinnacle, those carriers are going to employ inexperienced flight crews and attempt to make up it up with training, simulation, and rules. This misadventure identified numerous shortfalls in those. I doubt anyone flying today's Endeavor Air ignores so many stick shakes, uses the same stall procedure at all altitudes, ignores airspeed, hasn't heard of "core lock", or lies so blatantly to ATC in potential emergency situations. I can understand why the report soft-pedaled the training gaps, because after all they didn't need to give the aircrew's survivors any more leverage in the inevitable lawsuit, but I'd expect training to improve anyway.
Is it common practice for pilots at budget airlines to read accident reports like this?
As long as you start having problems as high as 35,000 feet there are surprisingly many (although vomit-inducing) ways to deal with your problems by pure physics – i.e. as long as you can use some of your control surfaces, you have a reasonable chance of surviving.
The most important variables are velocity and lift vector. At 35,000 feet you have room to gain speed, and even just some of your control surfaces let you manipulate your lift vector in "unconventional" ways.
So how is the vexing situation resolved? Either the onboard computer must be able to manage 100% of all situations, or the human driver must have an option to take control to resolve it themselves.
You can't have a car that makes human control impossible, and then gets itself into vexing situations. You'll be stuck there forever.
Well the parent refers only to certain situations--those in which the robocar gets vexed and chooses to slow down or stop.
I would expect humans to be able to resolve those types of situations, since we rarely find humans sitting in perfectly working cars on the side of the road just because they are vexed.
Ah, I see that you are referring to a scenario in which the vexation and stopping are not both products of some external stimulus (e.g. [1]) or malfunction but something else altogether. For example, parents who pull their perfectly working car to the side of the road to discipline their vexatious children. It certainly seems possible that a self-driving car might occasionally get itself into a situation requiring a reboot, but that seems like an awfully low price to pay for the convenience of autonomous driving.
For nearly every mistake that human drivers make, a good way to prevent it is to slow down. Not every situation must be navigated at maximum speed. Robocars will be better drivers than humans, but slower will still be safer. I'm not talking about cars parked in the lane indefinitely because they're confused by some dirt obscuring the center line. If that had been a thing the Google cars would already have fixed it.
> For nearly every mistake that human drivers make, a good way to prevent it is to slow down.
This is an important point, one where the comparison to aircraft doesn't work. Aircraft and the Apollo missions couldn't have a failsafe that just says "I think something is wrong, come to a gentle stop".
Why assume that accident rates will drop in the all-autodrive scenario?
I haven't seen any such testing. I have seen many miles of google cars in amongst human-driven cars. I have yet to see any tests involving thousands of cars from a variety of manufacturers (and ages) all trying to drive together. To assume that the robots will cooperate perfectly, negating the needs for safety features ... wasn't there a movie about that? Big lizards in an autonomously-run park?
Silly suggestion: have the copilot pilot spend the flight playing crash-recovery scenarios that reset every `random.randint(30,90)` minutes. When there is an actual emergency, plug him in to the actual avionics. It would be kinda like [work of fiction you probably should have read by now].
Pilots might look like they are sitting on their asses up there, but they aren't. They are constantly evaluating situations. Think of them like a bodyguard. They might look like they are just standing around doing nothing, but in fact they are constantly scouting for that one-in-a-million scenario. You don't want them heads-down playing games all day.
I thought the pilots got confused because of https://en.wikipedia.org/wiki/Flight_control_modes#Alternate... not, particularly, because of de-skilling. I'm going to use a stupid metaphor, that's perhaps ton totally apt. In that case it's kind of like reversing the gas and break pedals. 99.9% of the time, practically all your time behind the wheel the car behaves one way, but in this one special circumstance, the 'normal' emergency reaction becomes exactly the wrong thing to do. Hit the brake not the gas, but the pedals are backwards.
Yeah, maybe drivers will deskill. cars will fully automate much faster than planes. google's little fleet has racked up a million miles. The 80% solutions will turn into 95% solutions in a few iterations, then 99.99% solutions in a few more, then a long tail of fixing edge cases as they are discovered.
While the confusion over normal mode and alternate mode did play a role in the crash, the BEA report investigating the crash found the following six proximate causes [1]:
- temporary inconsistency between the measured speeds, likely as a result of
the obstruction of the pitot tubes by ice crystals, causing autopilot
disconnection and reconfiguration to alternate law;
- the crew made inappropriate control inputs that destabilized the flight
path;
- the crew failed to follow appropriate procedure for loss of displayed
airspeed information;
- the crew were late in identifying and correcting the deviation from the
flight path;
- the crew lacked understanding of the approach to stall;
- the crew failed to recognize that the aircraft had stalled and
consequently did not make inputs that would have made it possible to
recover from the stall.
Of those six, three can be attributed to skills atrophy: failure to follow procedure for loss of displayed airspeed information, lateness in identifying deviation from the flight path, and failure to recognize stall. These are all skills that pilots are taught when they're certified to fly a given aircraft. The fact that they didn't have them means that they forgot, or, in other words, they didn't have enough ongoing training to keep their skills fresh for emergencies.
Yeah, but there's some other interesting stuff on that page, like "... difficulty in identifying and understanding the implications of the switch to alternate law, which does not protect the angle of attack."
99.9% of the time it works one way, you can't be steep enough to stall, then you change the rules on the guys. Yes, the pilots fucked up. But there's pretty broad criticism of the HCI. The pilots are professionals and should know better, but when you swap the gas and the brake, it's a rare driver that can cope with that. Sure, set a high bar for professionals, but leave out the arbitrary complexity.
Anywho, with regard to self driving cars, i think passive drivers will just give up and pull over when the autopilot can't handle the situation. Aggressive drivers will be driving manually all the time anyway. Of course there are exceptions, but i'd bet that's how it breaks out 95% of the time in practice.
edit
This is sorta like an anti-lock brake system that only works above 32 degrees fahrenheit. Just because there might be ice, we're going to make your brakes worse even when there is no ice.
I think i get your point about de-skilling. They'll need to be very careful of stuff that works sometimes
When a voice in the cockpit repeats "stall. stall. stall. stall" it should be almost instinct to deal with it (whether that is forward yoke and near-0 g flying or rolling the lift vector to the side).
...unless you haven't had to fly the aircraft manually in a long time, so it's no longer instinct how it reacts to your inputs.
I must be missing something, because it seems like Mr. Mindell is saying cars shouldn't be fully driverless because they can't be. He cites numerous examples of other promised automation technologies that seem to have become a pipe dream over the past several decades.
Except, Google's driverless cars have now logged over a million fully automated miles. It is a proven technology. These aren't exploratory machines going where no human has gone before, they're out there on the road doing what we've all been doing for a hundred years already.
I get the impression this article is just a shill for his book.
I would suggest that Google cars haven't logged any driverless miles, at least not on public roads. They have humans onboard with backup controls (parking brakes if nothing else). So they are not driverless just yet.
Hands off the wheel does not mean not in control if you can put those hands back on the wheel instantly. Take the hands off the wheel, get in the back seat, and take a nap. That's what I would call a driverless car.
The "hands not on wheel" argument has been made by many a defendant. Courts don't buy it anymore. If you are in the driver seat, inches from the controls, then you are responsible for the motion of the vehicle. Which is why you can be charged for drunk driving even though the car never moved. In the seat or at the controls = in control = not driverless.
This is an incredibly pedantic definition and has nothing to do with what Google is doing.
Google's car has logged driverless hours - despite where a human can be sitting. Are you seriously implying that a human sitting in the driver seat is controlling the car without touching it? If its not driverless, how is the car navigating? Through psychic powers?
They are "driverless" from an internal software or technology perspective. The car is making itself move. But they are not driverless from an external/legal view. There is still a person in there that is held responsible for the safe operation of the vehicle. When things go wrong, that person will still be answerable a the driver.
I seriously doubt Google is sending out "drivers" without proper licenses. Equally, I cannot imagine any court dismissing a traffic ticket on the basis that the car, the robot, was the one committing the crime.
I'll take that under advisement so long as everyone on HN promises not to use "driverless" in the context of legal, ethical, or moral debates re google cars. The term is ubiquitous from Google no matter the context. It is their stated goal. Other companies fear any suggestion that they will field driveless cars anytime soon. I smell PR.
I disagree; I think sandworm nailed an essential issue.
At the top of this subthread, Zikes claimed that Google's cars are a "proven technology." Not even Google claims that now, and legality is indeed part of that.
Driverless elevators are an example of an actual proven technology. There's no human operator standing by to jump on the manual brake in case the automation fails. And if a driverless elevator does fail and causes harm, the elevator's manufacturer can be sued for damages.
Now, the idea of a "driverless" elevator seems like a silly name for just a regular elevator, but I chose that for a reason. Elevators used to have drivers, and people are already drawing the comparisons to cars:
Is Google willing to assume the same type of liability for their self-driving cars? Certainly not right now, so it's hard to call their technology fully proven.
I agree with you here. Also, the example of "driverless elevator" is excellent; I'll be remembering it for consideration in further discussion about self-driving technologies. Thanks for bringing this up.
Also, wtf is with downvoting in both Tesla threads today? Especially 'sandworm101 is taking a heavy hit for bringing up reasonable counterpoints (with most I disagree, but it doesn't change they're reasonable).
His points really are not reasonable at all. He's completely ignoring that humans are generally pretty bad at driving cars and implying that AI driven cars need to be perfect before they should be replacing humans.
If we banned human driving today and replaced it with the current tech (which probably isn't near being ready for actual use), we would still be far better off than we are now.
Where's the evidence to support this sentiment? I see it all the time and it looks to me that people just take it on faith that computers will be better.
I mean, here on HN, discussions of computing products like phones and operating systems are full of complaints large and small about problems, shortcomings, etc. Are there any programmers here who think their product is bug-free and safely survives all edge cases?
There just seems to be a huge mismatch between how people think about computers they use today, and computers they expect to drive their car in the near future. But it all comes out of the same companies and software development processes.
I also don't think people know the stats on car safety. The injury rate is 75 per 100 million miles traveled. About 35,000 people died from vehicle accidents in 2013. That sounds like a lot, but more people died from the flu, or from falls, than died from cars.
I won't deny that vehicle travel is dangerous, but the question is why we think computers would be better at it than humans, who are already pretty darn good at it.
Humans aren't pretty good at it, and we know the problems we have are exactly the parts where machines are already stronger than us. We also have some good deployments of automated technology, notably in air travel, which is flying on decades old technology anyway.
> I mean, here on HN, discussions of computing products like phones and operating systems are full of complaints large and small about problems, shortcomings, etc. Are there any programmers here who think their product is bug-free and safely survives all edge cases?
The problems with phones and operating systems are often not because of engineers, but because of businessmen. Especially phones and webdev - you can often trace the problems back to decision made in order to increase sales or save money. Most phone vendors ship shitty UIs and buggy devices not because they have bad engineers.
When the current elevator models were being tested, they probably had someone ready to press the STOP button if they went awry. Yet they were still the exact same driverless elevators you're talking about, just in a test environment. And so are Google cars currently.
Google's driverless cars have now logged over a million fully automated miles. It is a proven technology.
Not even Google engineers go that far - see this Tech Review article [0] from last years that talks about how Google's driverless cars can't handle snow, heavy rain, or its sensors being blinded by the sun.
Driverless cars may be coming, but it's still along way off.
The Apollo program is hardly as convincing an example as TFA seems to think. That was early days for computing power. Humans have landed scores of other craft on various other bodies in the solar system, and that landing has been fully automated in every other case. If a manned moon mission were planned today it would probably be completely automated.
The first moon landings (Surveyor, Lunokhod) were robotic and were completely automated. The boost phase has been completely automated all the way back to the first manned space flights. After all, a monkey made the first flight. Other than lunar landing and some older docking operations, just about everything in space flight has been automated.
automated can mean 'pre-programmed' automated, instead of dynamic automation, where the actions required aren't known ahead of time, and the machine(s) have a split second to make a choice out of the hundreds of different options available.
He is comparing situations where human decision makers are highly trained professionals such as astronauts and passenger plane pilot to daily driving situations where the average human decision maker is more likely to be an idiot(people switch off while driving) distracted by his smartphone.
Which only strenghtens the argument for self-driving cars. Hell, we even make those highly-trained professional aviators and astronauts depend on auto-pilots because it's much safer this way.
So long as the self-driving car can cover every situation. What doesn't work is a system that expects the human to sit idle for hundreds of hours, then suddenly throws him/her the wheel and jumps ship. (Like the autodrive system in that simpsons episode where homer becomes a trucker.)
I agree. Self-driving system that expects humans to take over randomly every now and then when something unexpected happens is much more dangerous than 100% self-driving or no self-driving - it exposes only the worst aspects of human driving while removing the best.
Basically, the less often the worse. If it's like once every 5 seconds it's fine. Once every 5 minutes, it's dangerous and annoying. Once every hour is a lethal idea.
At this point you have to consider the thing 'Karunamon pointed out upthread. If, for some reason, you can't go full-self-driving and such system is safer on aggregate than having humans drive, then it's a good idea to implement it, even if suddenly handing over control once a month is an incredibly dangerous prospect (because with such timespans, the human will not really be expecting to drive and their skills will have atrophied).
That assumes the tech is crap though, doesn't it? I think there's a case to be made that once the tech has progressed enough, there will be fewer deaths/injuries overall even if some of the number has moved from the "human did something dumb" to "human couldn't take over from autopilot fast enough"
I think the danger is that semi-self-driving with occasional human intervention will be more dangerous than pure human driving. If the interventions are so rare that the average deaths/injuries fall below the current level, then of course we've made some progress. But hopefully by this time we'll be able to go full-auto. It's hard to imagine an extremely unexpected dangerous situation, in which giving control to human would be better than just making the algorithm try and recover.
full auto is not practical unless by full auto you mean the driver can opt-in at any point with a clear protocol for doing so.
I think the mistake people are making is thinking the vehicle is just going to be like 'welp, death imminent in 5 seconds, hairy pink skinned passenger, here ya go! do somefink abooot it!'.
In reality the vehicle is going to be assessing potential risk and let the passenger know when they need to be more attentive, even to the point of simply refusing to operate without the passenger signaling their attentiveness in some manner.
Yes, I think this is the key point. We've already seen [0] that even trained pilots have trouble when thrown into the middle of an emergency situation that they don't understand how they got into.
Capt. Warren VanderBurgh has held a bunch of lectures, some of which exist on youtube[0], where he keeps stressing this specific issue. Highly interesting stuff, even for someone like me who's basically never been in an airplane.
It's something that a lot of professions are specifically trained to avoid. Awareness is a hierarchy. No one can remain hyper vigilant all the time, but you can be trained to remain in a state of relaxed vigilance. In this state, you can react appropriately to situations and heighten your awareness state as the situation calls for it. If you are in a state of unawareness and are unprepared then your reaction to a situation that suddenly calls for heightened awareness will be very sub-optimal.
I think the issue with self driving cars that require a driver to be ready to step in at a moments notice, is that we, as a population are not trained to remain relaxed and aware at all times, nor is it reasonable to really expect most people to be able to do this.
Even a "switched off" human is benefiting from the supervision of a brain that is far more complex than any computer yet constructed, and has, on average, been trained in driving by decades of experience, and in recognizing patterns of movement by millions of years of evolution.
Meanwhile, computers in cars today cannot even reliably translate human throttle inputs into throttle actuations (see: Toyota).
A human brain is also more complex than a computer that can play chess, does that mean the average chess player can generally beat a good chess-playing software? More general complexity does not necessarily imply being better at a specific task - in fact, it can be detrimental.
That is not fair. The cars these days can and do it reliably, Toyota was an exception, not the rule. And the state of the art can only get better. A "switched off" human is useless and that creates suffering around it, why not avoid it if we can?
Let's be fair. Not much of our millions of years of evolution has fully prepared us to tackle the slew of random events and conditions the daily driver has to face. And while our hundreds of thousands of years of tool use has certainly managed to have some influence on our genome we're still barely equipped to operate tools that increase our speed, strength, and carrying capacity by orders of magnitude. This is demonstrated every day by the astonishingly huge number of terrible accidents caused by these tools. Automobiles are responsible for perhaps the most (no doubt due to the vastly varying levels of skill and practice across their operators). Human decision making is simply not fast enough to handle all of the situations one can encounter while operating a motor vehicle and is certainly many times slower in any given situation than a highly optimized specialized system designed to solve that type of problem. The area where human brains currently have the advantage is in reacting to novel edge cases, but at that point providing adequate autonomous conveyance at safer-than-human accident rates becomes a numbers game.
Erm... the main point he makes is like "oh, we could not do it before no matter how hard we tried, so let's give up and keep humans in charge" basically.
I'm sure no one believe we would send rockets into space if you look back long enough in time, yet we did it. Driverless cars are not a concept anymore, many prototypes are very close to being fully operational - it would be very disappointing to stop at this stage.
> “That’s just proven to be a loser of an approach in a lot of other domains,” Mindell says. “I’m not arguing this from first principles. There are 40 years’ worth of examples.”
Yeah, and I have several millennia worth of examples suggesting that organized society at large is a worthless endeavor...
Cars seem fundamentally different than his other examples: Spacecraft, underwater vehicles, and airplanes. The importance difference is cars have a clear, visual grid of where they can travel and where they cannot travel. Painted lines on the road and navigational signs provide a construct of the environment around them.
There are still some elements of uncertainty in car driving. For e.g. driverless cars are confused by cyclist doing track stand[1]. Hopefully with enough data driverless car could overcome these difficulties.
The hard part of self-driving cars is not staying within the painted lines of the road, the hard part is avoiding other valuable things like cars, trucks, bikes, people, wildlife, etc. There is no reliable visual grid for those things.
Subways handle this problem by being given dedicated underground tunnels which exclude all those things. And even subways hit people sometimes when they fall onto the tracks.
> difference is cars have a clear, visual grid of where they can travel and where they cannot travel.
Visual grid for people to look at. All those cues and signs were not designed to be read and processed by robots. And there are crazy pedestrians, crazy drivers, maybe cattle crossing the street, an accident, ice, fog, ice and fog and cattle, unpaved roads, etc.
I would think an airplane's environment when in the air seems more predictable and stable, than an environment for car on the streets. Same thing for underwater vehicles. There might be crazy pranksters playing games with the vehicle 100 feet under water, but probably not a high probability.
> And there are crazy pedestrians, crazy drivers, maybe cattle crossing the street, an accident, ice, fog, ice and fog and cattle, unpaved roads, etc.
All of which people often don't do that well on themselves.
> All those cues and signs were not designed to be read and processed by robots
No, but they were designed to be clear and quickly identifiable by humans with vague glances. We're not asking for them to identify a subtly sarcastic remark, signs are more of the "BIG RED CIRCLE WITH STRAIGHT WHITE LINE" variety.
> I would think an airplane's environment when in the air seems more predictable and stable, than an environment for car on the streets
Possibly, but the risks are significantly higher. If something unexpected happens you can't just come to a halt and put your warning lights on 30,000 feet in the air.
> Same thing for underwater vehicles
I disagree on this one, the environment is going to be largely unknown and more importantly for exploration you aren't just saying "go here". When I drive somewhere, many of the roads will have been driven over thousands of times the same day.
I think it may take decades before cars are 100% autonomous in all situations. But if the AI can handle almost everything, it could take over and let the humans read a book, and alert them in the rare case their input is needed. Fleets of autonomous taxis and trucks could have remote controlled operators monitoring them.
Mindel isn't saying full automation can't or shouldn't happen for cars. His point is empirical and cautionary: usually when we get excited about automating X, it turns out that X is better with some human supervision. It's not really a contentious claim. I think many proponents of self-driving cars would be ok with some human input – steering wheels, changing routes, feedback on driving performance, etc
I think many proponents of self-driving cars would be ok with some human input – steering wheels, changing routes, feedback on driving performance, etc
On the topic of boredom and attention lapses while driving, I wonder if it would actually be made safer by being made more difficult, in contrast to the never-ending search for ease of use. Something that's difficult, mentally stimulating, and requires actual skill like a video game or musical instrument. Would there be more or less mistakes (collisions) if people were forced to be engaged 100% of the time in order to operate the vehicle?
I'm not saying it's a good idea, because only a subset of people would be capable of it, but it's interesting to think about!
Including me, so I tried to find some statistics to see if a manual was enough to focus on. I didn't find much.
I found one Swedish study that looked at automatic and manual transmissions affect on young and old drivers' ability to turn left across traffic [1]. It found a manual had little impact on the young driver, and older drivers performed worse.
I'm not sure turning left across traffic is a large enough sample of driving for me to be satisfied.
> Would there be more or less mistakes (collisions) if people were forced to be engaged 100% of the time in order to operate the vehicle?
I think there would be less mistakes, but the problem is, you can't sustain the required level of engagement for long. 100% concentration for 15 minutes? Sure. 30? Yes. One hour? Maybe. Three hours? Probably not.
Though, come to think of it, one hour would probably be enough to cover like 95% cases of commuting.
Taken to some extreme, people realized that safer roads, no trees on the sides, aren't that safer, they just make people lazier and keep crashing even when trees have been removed, faster.
When I'm on a highway, I like driving a little faster than the cars around me precisely for this reason. If I drive just 1 km/h faster, I need to switch lanes and overtake cars occasionally, which brings some action into the driving. And the funny thing is, I usually do it while using cruise control. So I think it's not about a technical challenge, it's about changing what is happening around you and having the need to be aware of the changes.
In most of the given examples, humans have little control of the environment. We are, in fact, trying to use autonomous vehicles to explore.
In the case of cars, humans can change the physical environment to meet the needs of the car. Additionally, the car may be able to avoid a situation that it knows it doesn't handle well (e.g. avoid complicated intersections).
In the one case of an environment where we have some controls (air travel), it is more difficult to "pull over" as compared to a car (as other commenters have noted).
While the article doesn't actually use the word "never", neither does it hedge and say "anytime soon".
The central logic seems to be that humanity is starting from scratch building self driving cars, versus all the years of experience decades ago trying to automate spacecraft and submersibles.
Oddly, I reach entirely the opposite conclusion. The article reminds us that even decades ago, with rudimentary computers, engineers developed systems that landed men on the moon and returned them to earth.
This article has made it even easier for me to imagine a car taking a fully autonomous trip to a McDonald's drive through,
given the state of deep/machine learning, low cost computers a million times faster, and big advances in sensors,
Yes there will be a person supervising the car for a while, of course that's true. But it's only a matter of time before our children are surprised that humans were allowed to drive cars, causing millions of deaths.
As a website commenter I'm not sure I should predict anything. To explain a bit more my previous message, as complex as going to the moon was, it seems a problem scientists knew how to define. One vehicle, 2 planets, the system is describable.
Traffic with lots of agents potentially interacting with every other agent, hence my chaos analogy, lots and lots of variable, ... is blurry.
There's successful research to describe multi agents collision avoidance but I've never seen it used on the field yet.
ps: reminds me airplanes vs car accident rates. Even if flying is inherently complex in itself, there's almost no traffic and almost no obstacle in air. Nobody would crash a car alone in the desert.
By the way, I find that making a prediction causes me to net out the different aspects of an issue quickly, and snaps me out of the mode of enumerating pros and cons. That's why I asked.
> Even if flying is inherently complex in itself, there's almost no traffic and almost no obstacle in air.
Different aircraft are by and large keeping to the same standard routes, altitudes and holding patterns, which is why mid-air collisions are a thing. (The reason they keep to standards is to make air traffic control easier.)
And it's weird how people bring up the idea of an elevator to the moon more often than they bring up putting cars to run on rails. They could be 100% self-driving that way, and accidents would probably be nil.
His 40 years of experience obviously did not include the only recently available GPUs and ASIC deep neural networks which are at the heart of the current self-driving robotics research. There is a game changer, and any previous experience is irrelevant.
I think the difference between self-driving cars and Apollo program/subs examples is that the cars can be tested much more thoroughly. No matter how good your simulators are for space conditions, you're never going to really know how it's going to work in actual conditions. Self-driving cars can and are logging millions of miles in a variety of conditions, and every common and almost every remote case will be dealt with before launched to the general public.
40 years of history showing failure, of course for most of that time they have had between a millionth and half the computing power we have today, and most of those years had now where near the capability of image sensors and other sensors. So how much weight should we really give history on something like this?
I don't think they're arguing against automation, just that completely driverless vehicles won't be the optimum path. I think there will be some applications for which fully driverless vehicles will be fine, such as trucks transporting goods between loading bays and taxis collecting and dropping off at street addresses, and many others where some level of driver control will be useful.
Take their example of submarines. Fully autonomous submarines are completely possible, but what they point out is that some level of human control is more useful. They don't say why, but my guess is that this is because if the sub comes across something unusual that merits further investigation, humans are better at making that judgement call than a robot would. The same principle applies to the Mars rovers. But it also applies to plenty of routine uses of vehicles. Most of the time I'm driving from my house to a car park or street parking space and such journeys are probably completely automatable. But sometimes I arrive at e.g. a country fair and the parking is in a field and I know I want to park over there in a place which will soon be shaded by that tree, and not in a different place where it looks a bit muddy. At times like that I would want much more direct control over the vehicle and an automated system would not be able to make such decisions for me.
So while the article does a poor job saying so, I think the point is that 100% full driverless automation isn't the best answer in every situation. It might be a fine solution for many purposes, but not all. They're arguing against a (perhaps imagined and possibly straw man) maximalist automation position such as 'in 20 years time no cars will have manual controls'.
Here's the thing: we can do anything we want. And there is a huge demand for fully driverless cars that nobody is going to be able to stop, despite whatever sound and reasoned advice they may have. It's coming, like it or not.
> “That’s just proven to be a loser of an approach in a lot of other domains,” Mindell says. “I’m not arguing this from first principles. There are 40 years’ worth of examples.”
Well, that's the problem. Everything the author says is supported by the last N decades in engineering. That argument is so flawed it is astounding that it is coming out of someone from MIT. In looking at his bio it seems he has dedicated quite a bit of time to teaching and researching the history of engineering. It might just be the case that he isn't fully up to date on topics relevant to automation. Does he have an up-to-date CS/AI background? How about computing or chip design?
Saying that automation has failed to deliver in the past N decades is like the Wright Brothers saying "let's not bother because powered flight has failed to be developed in the past N decades". That is not how engineering works.
Today hundreds of millions of people have phones in their pockets that are orders of magnitude more powerful than the computers decades ago. We have better, smaller, cheaper sensors of all kinds. We have accumulated knowledge in relevant CS and AI fields. Today we have cars that can actually drive themselves to various degrees under their own computing power (without having to communicate with a supercomputer in a building). And this is just the beginning.
Fully auto-drive cars will happen. Of course, they will retain the ability to be human driven as well for some time, probably decades. There are many reason for which that makes sense. I am fairly confident we will get to the point where the average self-driver will be safer than the average human driver. All you have to do is point at a teenager behind the wheel. Commutes will be shorter (due to driving efficiently using data) and safer and, with electric cars, healthier (you won't have to breathe the crap coming out of all those exhaust pipes).
Besides, think of the wasted potential of millions of people staring at a road for hours rather than doing something more useful. If you commute one hour per day you'll get back 250 hours per year.
106 comments
[ 3.0 ms ] story [ 87.2 ms ] threadSimilarly, I'm worried that auto-drive and smart cruise control systems will take care of highway driving in the normal case, but when things go wrong and the auto-driver disengages, the human at the wheel will have forgotten that a car traveling at 60mph behaves differently than one traveling at 25mph. With aircraft, we can mitigate this risk by forcing pilots to spend extra time in the simulator, keeing their feel for the aircraft from decaying too badly. I'm not sure what the solution is for drivers. Maybe we can force everyone to play a racing sim for a few hours every month ;)
For a race driver perhaps. In traffic, you should keep a low enough speed and large enough distance, that 250 ms isn't a big deal.
There is, however, one commercial aircraft crash from problems at altitude (no passengers, ferry flight) worthy of a Darwin Award.[2]
[1] http://www.ntsb.gov/investigations/AccidentReports/Pages/avi... [2] http://www.ntsb.gov/investigations/AccidentReports/Reports/A...
If we're going to fly budget carriers like Pinnacle, those carriers are going to employ inexperienced flight crews and attempt to make up it up with training, simulation, and rules. This misadventure identified numerous shortfalls in those. I doubt anyone flying today's Endeavor Air ignores so many stick shakes, uses the same stall procedure at all altitudes, ignores airspeed, hasn't heard of "core lock", or lies so blatantly to ATC in potential emergency situations. I can understand why the report soft-pedaled the training gaps, because after all they didn't need to give the aircrew's survivors any more leverage in the inevitable lawsuit, but I'd expect training to improve anyway.
Is it common practice for pilots at budget airlines to read accident reports like this?
IANAPilot, but I'd like to be!
The most important variables are velocity and lift vector. At 35,000 feet you have room to gain speed, and even just some of your control surfaces let you manipulate your lift vector in "unconventional" ways.
You can't have a car that makes human control impossible, and then gets itself into vexing situations. You'll be stuck there forever.
*edit: put another way, is it possible for a human driver to extricate a car from any situation?
I would expect humans to be able to resolve those types of situations, since we rarely find humans sitting in perfectly working cars on the side of the road just because they are vexed.
[1] http://extras.mnginteractive.com/live/media/site36/2007/0811...
This is an important point, one where the comparison to aircraft doesn't work. Aircraft and the Apollo missions couldn't have a failsafe that just says "I think something is wrong, come to a gentle stop".
It could be a remote pilot (sort-of call center, except the people are trained to remotely drive cars, much like drones are piloted right now).
Then, the situation (and the remote pilot reaction) are sent to the engineering team, analysed and the software is improved.
Rinse and repeat until the need to solve situations is reduced to zero.
I haven't seen any such testing. I have seen many miles of google cars in amongst human-driven cars. I have yet to see any tests involving thousands of cars from a variety of manufacturers (and ages) all trying to drive together. To assume that the robots will cooperate perfectly, negating the needs for safety features ... wasn't there a movie about that? Big lizards in an autonomously-run park?
Yeah, maybe drivers will deskill. cars will fully automate much faster than planes. google's little fleet has racked up a million miles. The 80% solutions will turn into 95% solutions in a few iterations, then 99.99% solutions in a few more, then a long tail of fixing edge cases as they are discovered.
[1] https://en.wikipedia.org/wiki/Air_France_Flight_447#Final_re...
99.9% of the time it works one way, you can't be steep enough to stall, then you change the rules on the guys. Yes, the pilots fucked up. But there's pretty broad criticism of the HCI. The pilots are professionals and should know better, but when you swap the gas and the brake, it's a rare driver that can cope with that. Sure, set a high bar for professionals, but leave out the arbitrary complexity.
Anywho, with regard to self driving cars, i think passive drivers will just give up and pull over when the autopilot can't handle the situation. Aggressive drivers will be driving manually all the time anyway. Of course there are exceptions, but i'd bet that's how it breaks out 95% of the time in practice.
edit
This is sorta like an anti-lock brake system that only works above 32 degrees fahrenheit. Just because there might be ice, we're going to make your brakes worse even when there is no ice.
I think i get your point about de-skilling. They'll need to be very careful of stuff that works sometimes
...unless you haven't had to fly the aircraft manually in a long time, so it's no longer instinct how it reacts to your inputs.
Except, Google's driverless cars have now logged over a million fully automated miles. It is a proven technology. These aren't exploratory machines going where no human has gone before, they're out there on the road doing what we've all been doing for a hundred years already.
I get the impression this article is just a shill for his book.
The "hands not on wheel" argument has been made by many a defendant. Courts don't buy it anymore. If you are in the driver seat, inches from the controls, then you are responsible for the motion of the vehicle. Which is why you can be charged for drunk driving even though the car never moved. In the seat or at the controls = in control = not driverless.
Google's car has logged driverless hours - despite where a human can be sitting. Are you seriously implying that a human sitting in the driver seat is controlling the car without touching it? If its not driverless, how is the car navigating? Through psychic powers?
I seriously doubt Google is sending out "drivers" without proper licenses. Equally, I cannot imagine any court dismissing a traffic ticket on the basis that the car, the robot, was the one committing the crime.
At the top of this subthread, Zikes claimed that Google's cars are a "proven technology." Not even Google claims that now, and legality is indeed part of that.
Driverless elevators are an example of an actual proven technology. There's no human operator standing by to jump on the manual brake in case the automation fails. And if a driverless elevator does fail and causes harm, the elevator's manufacturer can be sued for damages.
Now, the idea of a "driverless" elevator seems like a silly name for just a regular elevator, but I chose that for a reason. Elevators used to have drivers, and people are already drawing the comparisons to cars:
http://www.npr.org/2015/07/31/427990392/remembering-when-dri...
Is Google willing to assume the same type of liability for their self-driving cars? Certainly not right now, so it's hard to call their technology fully proven.
Also, wtf is with downvoting in both Tesla threads today? Especially 'sandworm101 is taking a heavy hit for bringing up reasonable counterpoints (with most I disagree, but it doesn't change they're reasonable).
If we banned human driving today and replaced it with the current tech (which probably isn't near being ready for actual use), we would still be far better off than we are now.
I mean, here on HN, discussions of computing products like phones and operating systems are full of complaints large and small about problems, shortcomings, etc. Are there any programmers here who think their product is bug-free and safely survives all edge cases?
There just seems to be a huge mismatch between how people think about computers they use today, and computers they expect to drive their car in the near future. But it all comes out of the same companies and software development processes.
I also don't think people know the stats on car safety. The injury rate is 75 per 100 million miles traveled. About 35,000 people died from vehicle accidents in 2013. That sounds like a lot, but more people died from the flu, or from falls, than died from cars.
I won't deny that vehicle travel is dangerous, but the question is why we think computers would be better at it than humans, who are already pretty darn good at it.
> I mean, here on HN, discussions of computing products like phones and operating systems are full of complaints large and small about problems, shortcomings, etc. Are there any programmers here who think their product is bug-free and safely survives all edge cases?
The problems with phones and operating systems are often not because of engineers, but because of businessmen. Especially phones and webdev - you can often trace the problems back to decision made in order to increase sales or save money. Most phone vendors ship shitty UIs and buggy devices not because they have bad engineers.
Unless we've been driving in the rain. Or with the sun in our eyes. Or somewhere besides California.
Not even Google engineers go that far - see this Tech Review article [0] from last years that talks about how Google's driverless cars can't handle snow, heavy rain, or its sensors being blinded by the sun.
Driverless cars may be coming, but it's still along way off.
[0] http://www.technologyreview.com/news/530276/hidden-obstacles...
Car gets stuck in a cul-de-sac and can't get out? Sure.
Impending collision in 2 seconds? Bad idea.
I think the mistake people are making is thinking the vehicle is just going to be like 'welp, death imminent in 5 seconds, hairy pink skinned passenger, here ya go! do somefink abooot it!'.
In reality the vehicle is going to be assessing potential risk and let the passenger know when they need to be more attentive, even to the point of simply refusing to operate without the passenger signaling their attentiveness in some manner.
[0] https://en.wikipedia.org/wiki/Air_France_Flight_447
[0]: https://www.youtube.com/watch?v=pN41LvuSz10
I think the issue with self driving cars that require a driver to be ready to step in at a moments notice, is that we, as a population are not trained to remain relaxed and aware at all times, nor is it reasonable to really expect most people to be able to do this.
Meanwhile, computers in cars today cannot even reliably translate human throttle inputs into throttle actuations (see: Toyota).
I'm sure no one believe we would send rockets into space if you look back long enough in time, yet we did it. Driverless cars are not a concept anymore, many prototypes are very close to being fully operational - it would be very disappointing to stop at this stage.
Not even all that long ago, actually.
http://www.popsci.com/military-aviation-amp-space/article/20...
Yeah, and I have several millennia worth of examples suggesting that organized society at large is a worthless endeavor...
Maybe the closest comparison is subway / metro rails. There are already many fully automated subway systems: https://en.wikipedia.org/wiki/List_of_automated_urban_metro_...
[1] http://road.cc/content/news/162468-cyclist-doing-trackstand-...
[1]: http://www.roboticstrends.com/article/a_cyclists_encounter_w...
Subways handle this problem by being given dedicated underground tunnels which exclude all those things. And even subways hit people sometimes when they fall onto the tracks.
Visual grid for people to look at. All those cues and signs were not designed to be read and processed by robots. And there are crazy pedestrians, crazy drivers, maybe cattle crossing the street, an accident, ice, fog, ice and fog and cattle, unpaved roads, etc.
I would think an airplane's environment when in the air seems more predictable and stable, than an environment for car on the streets. Same thing for underwater vehicles. There might be crazy pranksters playing games with the vehicle 100 feet under water, but probably not a high probability.
All of which people often don't do that well on themselves.
> All those cues and signs were not designed to be read and processed by robots
No, but they were designed to be clear and quickly identifiable by humans with vague glances. We're not asking for them to identify a subtly sarcastic remark, signs are more of the "BIG RED CIRCLE WITH STRAIGHT WHITE LINE" variety.
> I would think an airplane's environment when in the air seems more predictable and stable, than an environment for car on the streets
Possibly, but the risks are significantly higher. If something unexpected happens you can't just come to a halt and put your warning lights on 30,000 feet in the air.
> Same thing for underwater vehicles
I disagree on this one, the environment is going to be largely unknown and more importantly for exploration you aren't just saying "go here". When I drive somewhere, many of the roads will have been driven over thousands of times the same day.
AKA "cruise control"...?
I'm not saying it's a good idea, because only a subset of people would be capable of it, but it's interesting to think about!
I found one Swedish study that looked at automatic and manual transmissions affect on young and old drivers' ability to turn left across traffic [1]. It found a manual had little impact on the young driver, and older drivers performed worse.
I'm not sure turning left across traffic is a large enough sample of driving for me to be satisfied.
[1] http://acrs.org.au/files/arsrpe/Why%20drive%20manual%20-%20a...
I think there would be less mistakes, but the problem is, you can't sustain the required level of engagement for long. 100% concentration for 15 minutes? Sure. 30? Yes. One hour? Maybe. Three hours? Probably not.
Though, come to think of it, one hour would probably be enough to cover like 95% cases of commuting.
In the case of cars, humans can change the physical environment to meet the needs of the car. Additionally, the car may be able to avoid a situation that it knows it doesn't handle well (e.g. avoid complicated intersections).
In the one case of an environment where we have some controls (air travel), it is more difficult to "pull over" as compared to a car (as other commenters have noted).
The central logic seems to be that humanity is starting from scratch building self driving cars, versus all the years of experience decades ago trying to automate spacecraft and submersibles.
Oddly, I reach entirely the opposite conclusion. The article reminds us that even decades ago, with rudimentary computers, engineers developed systems that landed men on the moon and returned them to earth.
This article has made it even easier for me to imagine a car taking a fully autonomous trip to a McDonald's drive through, given the state of deep/machine learning, low cost computers a million times faster, and big advances in sensors, Yes there will be a person supervising the car for a while, of course that's true. But it's only a matter of time before our children are surprised that humans were allowed to drive cars, causing millions of deaths.
Check out Tesla's live shipping code:
http://jalopnik.com/teslas-autopilot-system-is-awesome-and-c...
Are you predicting that cars will never be fully autonomous?
Traffic with lots of agents potentially interacting with every other agent, hence my chaos analogy, lots and lots of variable, ... is blurry.
There's successful research to describe multi agents collision avoidance but I've never seen it used on the field yet.
ps: reminds me airplanes vs car accident rates. Even if flying is inherently complex in itself, there's almost no traffic and almost no obstacle in air. Nobody would crash a car alone in the desert.
By the way, I find that making a prediction causes me to net out the different aspects of an issue quickly, and snaps me out of the mode of enumerating pros and cons. That's why I asked.
Different aircraft are by and large keeping to the same standard routes, altitudes and holding patterns, which is why mid-air collisions are a thing. (The reason they keep to standards is to make air traffic control easier.)
Take their example of submarines. Fully autonomous submarines are completely possible, but what they point out is that some level of human control is more useful. They don't say why, but my guess is that this is because if the sub comes across something unusual that merits further investigation, humans are better at making that judgement call than a robot would. The same principle applies to the Mars rovers. But it also applies to plenty of routine uses of vehicles. Most of the time I'm driving from my house to a car park or street parking space and such journeys are probably completely automatable. But sometimes I arrive at e.g. a country fair and the parking is in a field and I know I want to park over there in a place which will soon be shaded by that tree, and not in a different place where it looks a bit muddy. At times like that I would want much more direct control over the vehicle and an automated system would not be able to make such decisions for me.
So while the article does a poor job saying so, I think the point is that 100% full driverless automation isn't the best answer in every situation. It might be a fine solution for many purposes, but not all. They're arguing against a (perhaps imagined and possibly straw man) maximalist automation position such as 'in 20 years time no cars will have manual controls'.
Well, that's the problem. Everything the author says is supported by the last N decades in engineering. That argument is so flawed it is astounding that it is coming out of someone from MIT. In looking at his bio it seems he has dedicated quite a bit of time to teaching and researching the history of engineering. It might just be the case that he isn't fully up to date on topics relevant to automation. Does he have an up-to-date CS/AI background? How about computing or chip design?
Saying that automation has failed to deliver in the past N decades is like the Wright Brothers saying "let's not bother because powered flight has failed to be developed in the past N decades". That is not how engineering works.
Today hundreds of millions of people have phones in their pockets that are orders of magnitude more powerful than the computers decades ago. We have better, smaller, cheaper sensors of all kinds. We have accumulated knowledge in relevant CS and AI fields. Today we have cars that can actually drive themselves to various degrees under their own computing power (without having to communicate with a supercomputer in a building). And this is just the beginning.
Fully auto-drive cars will happen. Of course, they will retain the ability to be human driven as well for some time, probably decades. There are many reason for which that makes sense. I am fairly confident we will get to the point where the average self-driver will be safer than the average human driver. All you have to do is point at a teenager behind the wheel. Commutes will be shorter (due to driving efficiently using data) and safer and, with electric cars, healthier (you won't have to breathe the crap coming out of all those exhaust pipes).
Besides, think of the wasted potential of millions of people staring at a road for hours rather than doing something more useful. If you commute one hour per day you'll get back 250 hours per year.