I'm a little younger than Woz, and I am skeptical we'll have truly autonomous self-driving cars in my lifetime.
Three thoughts:
1. Any of us who work in software development know that "getting to 90% done is easy... it's the second 90% that gets you". And I believe that to the extent that we may seem to be 90% of the way to self-driving cars, it's definitely the "first 90%" not the second. The devil truly is in the details.
2. From what I've seen / heard, it seems that most of the testing done to date has been done in day-time, on limited access, smooth, flat, straight roads, OR urban surface streets where traffic speeds are relatively slow. I believe the leap from either of those environments to "any arbitrary twisty, curvy, back-country road, at night, when it's foggy and snowing", is quite substantial. And I don't consider a car truly autonomous unless it can handle those conditions at least as well as a human (to be fair, humans don't always handle those conditions well, so maybe this bar isn't as high as it seems).
3. I've developed software for 20 years... knowing how the sausage is made, seeing the compromises that are made in order to "hit the ship date", the bugs that are swept under the rug, the boundary conditions that are assumed away as "impossible", etc., I won't be trusting any self-driving car anytime soon.
> knowing how the sausage is made, seeing the compromises that are made in order to "hit the ship date", the bugs that are swept under the rug, the boundary conditions that are assumed away as "impossible", etc., I won't be trusting any self-driving car anytime soon.
As a general member of the public who is present on the roads, you have no choice but to trust any Alpha- or Beta-level self-driving technology that certain companies thrust out on the roads today.
The only thing I have ever wanted from phone assistants is to tell me whether a coffee shop was open or not, or what the closing time is. I still can't get this info, nor is this question understood. I still have to stop, launch a browser, navigate to the site, and dig around their website for opening hours, or in the worst case find a phone number and call to ask them.
I wonder if people assume Siri failed to understand the query when they hear its request for clarification. I think it should include some context, eg:
“Which x do you want opening hours for?”
The one thing that I can consistently get google assistant to do is to set timers: "Ok google set timer for 5 minutes". I want more of that. Could they please just implement and document simple recipes like this? Setting a timer hands-free is already incredibly convenient, and I can only imagine making something similar for a wide host of tasks isn't /that/ hard. I really don't mind if I have to formulate it in a rather specific way, as long as it works and I can find out what that way is.
In my various conversations on this topic around the bay area, people are generally overly optimistic, who believe it'll happen in next couple or so years, and people who are overly pessimistic.
Possibly a gross generalization but generally the people who are overly pessimistic are the ones who are or have till recently been engineers, while the people who are overly optimistic are the ones further away from engineering - with the farther they are the more optimistic they are.
Moreover, in addition to what you wrote there are a lot of other open questions, not necessarily in any order and definitely not a complete list:
1) Auto Insurance (cars below refers to fully autonomous cars):
1.1) What happens to auto insurance? Do I still need to buy it if I'm not driving it, or does it come bundled in the car price?
1.2) Does everyone pay the same auto insurance for the same car model/year.
1.3) Do cars with different autonomous models have different auto insurance?
1.4) What happens in the case of car accident between self-driving and non-self driving car?
1.5) What happens in the case of two self-driving cars' accident
1.6) What happens if a person is killed due to the Car's fault. Who is liable? The insurance company? The Car manufacturer? The self-driving module manufacturer?
2) Some Edge cases
2.1) What happens if on a 4-way stop 4 self-driving cars come to stop at the same time? Do they know how to handle that, as in which car is moving forward and which one is allowing other cars?
2.2) What happens in an emergency? Can I force my car to drive rash to the hospital or will it follow the speed limit to the T.
2.3) What happens if the Car runs into a situation which it is not trained for?
3) How are the cars tested?
3.1) As a consumer do we know that the cars are fully tested to handle ALL edge conditions?
3.1.1) What is the definition/definitive list of ALL edge conditions?
3.2) When will we know that the threshold has been passed, and the car is truly autonomous?
4) Security
4.1) Assuming the cars will need to be connected, how will the security be?
4.2) Can the cars be Pwned? I'm sure they can be.... so this is the rhetorical question
4.2.2) If the cars can be pwned, will the fully autonomous cars become the next WMD (Weapon of Mass Destruction) -- if you don't agree, close your eyes and imagine a fleet of hundreds of thousands or millions of fully autonomous cars under the control of a rogue entity hell bent on causing damage/killing people.
1) Car insurance can work exactly like it currently does. The car owner's insurance is liable by default, however if the car itself is in some way at fault (performing out-of-spec), then the insurance company goes after the car company. This can already happen if something like brakes fail or accelerator gets stuck.
2 & 3) Yes there are some edge cases that you can't train for, but same with humans. What do humans do in an edge case like a torrential downpour? Simply pull over.
Even if there are edge cases, consider that humans are pretty bad drivers on average, over 1.25 million deaths per year, and 50 million injuries in the US alone.
Suppose that in a couple years, self driving cars are not perfect due to the edge cases, and still cause 250k deaths / year. Would you allow them at that point? Even with those stats we should consider only allowing self-driving cars and begin to outlaw human drivers.
Car insurance: I had made a simplified argument, but to itemize, auto insurance may work the same for auto-theft and some liability coverage. But, it definitely cannot work the same for anything remotely related to driver (or in this case self-driving system's) fault.
Edge Cases: Humans use their brain and adapt. Self driving systems have no concept of adapting, hence the result is undefined.
> over 1.25 million deaths per year, and 50 million injuries in the US alone.
In US in 2016 there were 36,750 fatalities[0][1], so much much lower than the 1.25 million number you're using.
So, no, personally I will not prefer self-driving cars if they are not perfect due to the edge cases, and still cause 250k deaths / year
And the reason is simple, and why throwing statistics is now the right approach: I'd rather have my life in my hands and my driving where I can take a corrective defensive action than rely on an AI model where it is unpredictable when the lightning will strike and a bug/untrained edge case will kill me. I will prefer to be in control.
That all depends on the contract with whatever service your using. It would be totally legal for the cab driver to make you sign a contract making you liable for any damages. I'd be very unlikely to sign for a cab ride, but I might if I had my own self driving car depending on how reliable they were.
My topmost comment said these are open questions, as in undefined.
I, as in IMHO, highly doubt it would be like you're describing, but it is equally likely that it may be like how you're describing, it's just an open question.
In fact, a couple of years ago there was a small event at Tim Draper's Draper University in San Mateo about how auto insurance would look in the world of self-driving cars and it was very clear that at least till that point there was no consensus. I haven't seen anything to suggest if that has changed since then.
Self driving systems however do adapt. Most companies are using neural networks which can be retrained based on their mistakes. Unlike a human, they can also be trained on each and every crash for which sensor data is available.
They cannot "adapt". Your "retraining" is not a trivial process. Attempts to accommodate edge cases are not guaranteed to be able to cleanly accommodate all previous cases without discontinuities/regression/overfitting. The consequences of which could be especially catastrophic given that every other vehicle that safely operates may be relying on a particular quirk of a particular version of the neural network's particular function which it is imitating to safely operate in it's particular environment.
That's the problem with NN's. You can't say anything about them explicitly except that they meet the specification for their training datasets. Fitness for a task outside of that is not guaranteed. Humans, on the other hand, are constantly adapting, because they never stop "learning". This has the inverse effect in that you can say that a person can be trained for a task, but at the moment may not be.
When you apply an NN on an industrial scale, you can't just throw it out there without being willing to accept the liability for the destruction you cause as a result.
I don't expect level 4+ autonomous vehicles any time soon, but the "trolley problem" is an irrelevant distraction. It's not something that comes up in real world collisions. There's never an even choice between hitting one of two targets. Other factors are always present and dictate the safest course of action.
I don't agree. The problem of least 'harm' will come up continually. Moralistic decisions must be made by the car. And even if those incidents do not occur, the software will have to be programmed for these types of decisions.
Moral and ethical dilemmas, unfortunately (in my view as someone very sympathetic to philosophical investigation) seem only rarely to deter the inventors of technology and even more rarely deter the companies who employ these inventors and engineers.
And an autonomous vehicle should drive so that, whichever of those things happens, it will not drive into anything.
This may mean that the car drives rather slowly when the treeline is close to the road, or extremely slowly when the road is slick or foggy. But it can be done.
The Trolley Problem is only a problem in theory. In real life no one but the victim of a psychopath who has carefully stage managed the problem ever has to confront it.
When one in real life has such a problem one doesn't have time for reflection, adrenalin and social conditioning take over.
People jump in front of trains to rescue other people careless of the harm that might be done to their families and friend should they die in the attempt.
For my height 1.7 m I would have 10 weigh 115 kg to have a BMI of 40. Bloody hell, I count myself as overweight at the 74 kg I weigh now, BMI 26. Just can't imagine carrying an extra 41 kg!
Doesn't surprise me at all that it would shorten my life. Never mind that but the life one has must be worse as well.
I just did that same calculation. My god. Would I even want to go 10 more years weighing 300 lbs?
I know BMI isn't 'perfect'. I think once you reach a certain weight class it becomes more and more relevant. There's no way more than a few fringe cases of people are 6'+ and 350+lbs of primarily muscle. Even NFL linemen are on average 6'4" and around 300lbs.
Edit: That linked article is incredible. It shows the primary driver of the increased mortality is from heart disease and diabetes. 5x more deaths from diabetes, 3x from heart disease, and 2x from digestive neoplasms, 16x lower respiratory, 5x liver disease. I'm going to continue my good exercise and dieting...
>My god. Would I even want to go 10 more years weighing 300 lbs?
As someone who's dropped from 430 to 330 (6'3" 31). This is always a difficult question. The struggle is real, but even trying to hit the "healthy" weight for my height according to BMI is another 140lb away.
You're going the right way and that is something to be very proud of for yourself. It didn't happen overnight so it won't go away overnight.
Seriously, you should be very proud and I truly commend you. I was 133 pounds when I stopped running track and field. I went up to 200 pounds with some ease. It's been a struggle to return to 150 pounds.
This is why I’m perplexed by the concerns around AI. what is more likely to happen is that we’ll start evolving the infrastructure to support driverless cars in the same way paved roads enabled the automobile.
I think he literally means nothing more than "Apple isn't making a car OS." Their focus is your pocket and other carry-alls, and not at all interested in your car outside of how those pocket and carry-all devices can connect to it.
Hell, their focus is only barely on the desktop anymore. For the same reason. Tracking is way more interesting when not anchored to a desktop, and other companies already got the car thing going.
Clarke's first law comes to mind. Not an exact fit but pertinent nonetheless:
When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.
46 comments
[ 3.8 ms ] story [ 103 ms ] threadThree thoughts:
1. Any of us who work in software development know that "getting to 90% done is easy... it's the second 90% that gets you". And I believe that to the extent that we may seem to be 90% of the way to self-driving cars, it's definitely the "first 90%" not the second. The devil truly is in the details.
2. From what I've seen / heard, it seems that most of the testing done to date has been done in day-time, on limited access, smooth, flat, straight roads, OR urban surface streets where traffic speeds are relatively slow. I believe the leap from either of those environments to "any arbitrary twisty, curvy, back-country road, at night, when it's foggy and snowing", is quite substantial. And I don't consider a car truly autonomous unless it can handle those conditions at least as well as a human (to be fair, humans don't always handle those conditions well, so maybe this bar isn't as high as it seems).
3. I've developed software for 20 years... knowing how the sausage is made, seeing the compromises that are made in order to "hit the ship date", the bugs that are swept under the rug, the boundary conditions that are assumed away as "impossible", etc., I won't be trusting any self-driving car anytime soon.
As a general member of the public who is present on the roads, you have no choice but to trust any Alpha- or Beta-level self-driving technology that certain companies thrust out on the roads today.
Kinda like how speech recognition was just almost there for decades until modern ML made it actually as good as a human being.
How ML made it usable in some cases, would be more accurate. I still can't ask Alexa to play specific albums or tracks most of the time.
I don't want that level of accuracy driving cars on the road.
Me: “Hey Siri, how late is x open?”
Siri: One possibility is on x street, does that sound good?
Me: Yes
Siri:
“X is open from y-z.”
* https://www-cs.stanford.edu/~acoates/papers/coatesng_nntot20...
In my various conversations on this topic around the bay area, people are generally overly optimistic, who believe it'll happen in next couple or so years, and people who are overly pessimistic.
Possibly a gross generalization but generally the people who are overly pessimistic are the ones who are or have till recently been engineers, while the people who are overly optimistic are the ones further away from engineering - with the farther they are the more optimistic they are.
Moreover, in addition to what you wrote there are a lot of other open questions, not necessarily in any order and definitely not a complete list:
1) Auto Insurance (cars below refers to fully autonomous cars):
1.1) What happens to auto insurance? Do I still need to buy it if I'm not driving it, or does it come bundled in the car price?
1.2) Does everyone pay the same auto insurance for the same car model/year.
1.3) Do cars with different autonomous models have different auto insurance?
1.4) What happens in the case of car accident between self-driving and non-self driving car?
1.5) What happens in the case of two self-driving cars' accident
1.6) What happens if a person is killed due to the Car's fault. Who is liable? The insurance company? The Car manufacturer? The self-driving module manufacturer?
2) Some Edge cases
2.1) What happens if on a 4-way stop 4 self-driving cars come to stop at the same time? Do they know how to handle that, as in which car is moving forward and which one is allowing other cars?
2.2) What happens in an emergency? Can I force my car to drive rash to the hospital or will it follow the speed limit to the T.
2.3) What happens if the Car runs into a situation which it is not trained for?
3) How are the cars tested?
3.1) As a consumer do we know that the cars are fully tested to handle ALL edge conditions?
3.1.1) What is the definition/definitive list of ALL edge conditions?
3.2) When will we know that the threshold has been passed, and the car is truly autonomous?
4) Security
4.1) Assuming the cars will need to be connected, how will the security be?
4.2) Can the cars be Pwned? I'm sure they can be.... so this is the rhetorical question
4.2.2) If the cars can be pwned, will the fully autonomous cars become the next WMD (Weapon of Mass Destruction) -- if you don't agree, close your eyes and imagine a fleet of hundreds of thousands or millions of fully autonomous cars under the control of a rogue entity hell bent on causing damage/killing people.
4.2.3) How do we protect against that?
2 & 3) Yes there are some edge cases that you can't train for, but same with humans. What do humans do in an edge case like a torrential downpour? Simply pull over.
Even if there are edge cases, consider that humans are pretty bad drivers on average, over 1.25 million deaths per year, and 50 million injuries in the US alone.
Suppose that in a couple years, self driving cars are not perfect due to the edge cases, and still cause 250k deaths / year. Would you allow them at that point? Even with those stats we should consider only allowing self-driving cars and begin to outlaw human drivers.
Edge Cases: Humans use their brain and adapt. Self driving systems have no concept of adapting, hence the result is undefined.
> over 1.25 million deaths per year, and 50 million injuries in the US alone.
In US in 2016 there were 36,750 fatalities[0][1], so much much lower than the 1.25 million number you're using.
So, no, personally I will not prefer self-driving cars if they are not perfect due to the edge cases, and still cause 250k deaths / year
And the reason is simple, and why throwing statistics is now the right approach: I'd rather have my life in my hands and my driving where I can take a corrective defensive action than rely on an AI model where it is unpredictable when the lightning will strike and a bug/untrained edge case will kill me. I will prefer to be in control.
[0] https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in... [1] https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/...
Why not?
Think of it this way, do you take auto insurance for a cab ride, no you don't, because you're not driving it.
My topmost comment said these are open questions, as in undefined.
I, as in IMHO, highly doubt it would be like you're describing, but it is equally likely that it may be like how you're describing, it's just an open question.
In fact, a couple of years ago there was a small event at Tim Draper's Draper University in San Mateo about how auto insurance would look in the world of self-driving cars and it was very clear that at least till that point there was no consensus. I haven't seen anything to suggest if that has changed since then.
Self driving systems however do adapt. Most companies are using neural networks which can be retrained based on their mistakes. Unlike a human, they can also be trained on each and every crash for which sensor data is available.
That's the problem with NN's. You can't say anything about them explicitly except that they meet the specification for their training datasets. Fitness for a task outside of that is not guaranteed. Humans, on the other hand, are constantly adapting, because they never stop "learning". This has the inverse effect in that you can say that a person can be trained for a task, but at the moment may not be.
When you apply an NN on an industrial scale, you can't just throw it out there without being willing to accept the liability for the destruction you cause as a result.
This may mean that the car drives rather slowly when the treeline is close to the road, or extremely slowly when the road is slick or foggy. But it can be done.
When one in real life has such a problem one doesn't have time for reflection, adrenalin and social conditioning take over.
People jump in front of trains to rescue other people careless of the harm that might be done to their families and friend should they die in the attempt.
Here's a journal article:
https://journals.plos.org/plosmedicine/article?id=10.1371/jo...
but to summarize BMI vs years lost:
Doesn't surprise me at all that it would shorten my life. Never mind that but the life one has must be worse as well.
I know BMI isn't 'perfect'. I think once you reach a certain weight class it becomes more and more relevant. There's no way more than a few fringe cases of people are 6'+ and 350+lbs of primarily muscle. Even NFL linemen are on average 6'4" and around 300lbs.
Edit: That linked article is incredible. It shows the primary driver of the increased mortality is from heart disease and diabetes. 5x more deaths from diabetes, 3x from heart disease, and 2x from digestive neoplasms, 16x lower respiratory, 5x liver disease. I'm going to continue my good exercise and dieting...
As someone who's dropped from 430 to 330 (6'3" 31). This is always a difficult question. The struggle is real, but even trying to hit the "healthy" weight for my height according to BMI is another 140lb away.
Seriously, you should be very proud and I truly commend you. I was 133 pounds when I stopped running track and field. I went up to 200 pounds with some ease. It's been a struggle to return to 150 pounds.
It is truly amazing how much hype FSD has generated, even among smart technically-savvy people.
Hell, their focus is only barely on the desktop anymore. For the same reason. Tracking is way more interesting when not anchored to a desktop, and other companies already got the car thing going.
When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.