Yeah, I feel like your parent poster is ignoring the implications of the high false positive rate.
Like, let's say I had a grammar check. But the grammar check was bad. It flags every other sentence as needing a grammar change, anything other than a very simple "subject-verb-object" sentence structure looks wrong to it. For every correct grammar problem it identifies, it finds 20 incorrect grammar problems.
As a result, I turn off the UI for the grammar check.
Then I write a document that has a grammar problem.
It seems stupid to say that this is either a failure of my use of the software, or a UI failure of the software. The problem is that the fucking software doesn't work, specifically that it fails in a way that makes it too "noisy" to use.
And sure, if my job is to catch grammar errors, then it's also my fault. I'm supposed to be backing up the system. But we also know that I won't be perfect and will have failures of attention sometimes. But regardless of that, the grammar checker just didn't work.
In this case, as I understand it, they also disabled the emergency braking feature built into the Volvo. Which wouldn't be obviously crazy if they had better, adequately tested emergency braking integrated with the "autonomy" system. But to disable the existing safety feature in favor of their own system, and also disable their own system due to a high false positive rate, seems negligent on the face of it.
The danger of false positives does not justify having no mechanism to alert the driver. The system is relying on the driver to take over in exceptional circumstances, but gives no warning when an exception occurs.
If you focus on the crash itself, yes, this semi-autonomous murderbot failed to alert the human.
But the failure that killed Elaine Herzberg happened well before then, when the decision to disable the emergency brakes was made. It appears they disabled other safeties as well.
I don't disagree that playing a little loose with allowing new tech makes sense. We need to understand what we don't understand about them, accidents of course will happen, etc. But this looks a lot like foreseeable negligent homicide to me.
I'd argue it was a corporate culture failure. A company that can put a device like that on the road is failing on many levels, that there is a contributory issue with the AI or the UX has nothing to do with the root cause.
I guess this is both astonishing and completely unsurprising at the same time. How can one hope to gather decent statistics in exceptional situations without a simulation?
A big problem is that the software engineers and algorithms engineers are pressed to deliver demonstrations in very short timescales.
This precludes the necessary conversations about test interfaces, for example, making the task of the simulation team closer to one of reverse-engineering than of simulator integration.
And people getting run over is the result of those software decisions by management, software engineers, and alg. engineers. Don't just be a cog in the machine, doing what one is told to do. Write software in the ways that are known to be correct and responsible.
Sadly, sometimes this means saying goodbye to your colleagues and friends, and looking for new adventures in an organisation that is capable of acting in a more constructive and mature manner.
This is important. I hope engineers working autonomous cars think long and hard about whether they want to be on a team that, it looks to me like, negligently killed someone by choice.
The only difference between disabling the emergency braking mechanism and, say, intentionally disabling safeties on industrial machinery is the novelty. I think it was pretty obvious that something like this would happen, and that's why so many people have been making noise about it. Now that we know for a fact that Uber is willing to make this choice, there's (a) no excuse for not coming down on them for it, (b) no excuse for Uber not completely overhauling their safety protocols with the help of an external ethics and legal advisors, and (c) no excuse for engineers working on this for not at least seriously thinking about their own ethics and possible moral culpability.
Some people want to focus on the future. Self driving cars will be great! Gotta break some eggs to make omelets! Fine. But those who are killed before that bright future deserve some accountability, those who don't want to be killed deserve some respect, and the makers of said omelets can bloody well be expected to be held responsible for their actions.
It seems like a useful thing for someone (the "government"?) to develop would be a some sort of standard simulator interface that can also be used for regulatory purposes. It's not perfect (It would be something of a cat-and-mouse game to prevent companies from just reverse-engineering the tests and companies like VW would drive really safely in the simulator but then unsafely in real life so they can go faster or whatever), but at least it would be safer and cover more scenarios than real-life trials.
I don't understand why Google/Uber are live testing with lethal SUVs instead of polystyrene golf-carts that could only hurt someone if they tried to eat it. We could be getting acclimatised to near disposable pizza delivery carts earning the algorithm enough trust for 2 tons of metal and a passenger.
I can think of half a dozen likely reasons. I'm not sure which one it is, or if all of them could alone be fatal to your idea.
- Such vehicles would not be street legal.
- They would not be safe for passengers.
- They would be unable to carry the necessary payloads.
- Most of the weight of a car is in the chassis anyway, not the body.
Edit: at any rate, Google did do something like what you described at an earlier stage of testing. https://www.youtube.com/watch?v=HJnjuhBIhwc They seem to be past that point, though.
In a modern car, the chassis and body are one and the same (unibody construction). Some cars do use plastic panels for non-structural parts.
I would think another big reason though is that the handling and response of such a car would be very different from a real car. Stopping distances, acceleration, cornering, wind behavior, would all be different.
I can easily believe in code that adjusts to the underlying vehicle's differences.
It seems to me the real problem would be that A: You need the car to be on the real streets, so at a minimum, it must be street legal. B: You need it to be on all the streets, including highways and such, so we're at least up to a Smart car level of body. From there it really isn't that great a leap to a full sedan chassis. Would one of those little cars even have room for all the requisite sensors? Even if they did, sedans are just so much more useful.
So it seems like the minimum practical car to test with is a sedan, and that's enough to kill people. Even the little Smart cars are plenty enough to kill people.
On slow roads, I've often seen them driven around summer communities. It would be silly for the government to not allow this.
> They would not be safe for passengers.
The above commenter pointed out "enough trust for 2 tons of metal and a passenger," so I'm assuming the idea is that they don't have a passenger in them at this stage. Additionally, regardless of whether an SUV is safer for the individual behind the wheel, shouldn't it be that individual accepting the risk and not a random pedestrian who didn't consent to this experiment?
> They would be unable to carry the necessary payloads.
Do they really carry payloads at this stage? Isn't the focus on building a safe autonomous system?
> Most of the weight of a car is in the chassis anyway, not the body.
The golf carts are 1/8th to 1/4th the weight of a crossover, for example.
The substantial issue I'd guess would be the differences in all aspects of performance between a real car and a golf cart, but at this stage companies like Uber should clearly just be testing detection of pedestrians et al.
It seems reasonable that there should be some regulation on "x miles per incident" before autonomous vehicle companies are allowed to move forward from carts to full-sized cars.
Based on the stats that have been posted on HN, Google is very safe. Uber, on the other hand, is not.
I don’t understand why human drivers are live testing with lethal metal vehicles, causing 3000 deaths per day globally[1]. Human drivers should be banned from the roadways until vehicles include the safety systems needed to prevent these needless deaths.
> This makes motor vehicle collisions the leading cause of death among young adults of 15–29 years of age (360,000 die a year) and the ninth cause of death for all ages worldwide.
> It is estimated that motor vehicle collisions caused the deaths of around 60 million people during the 20th century,[6] around the same as the number of World War II casualties.
Approximately 1% of all deaths are in car incidents.
These may be facts, but extremely misleading facts. Why are you comparing WWII casualties (a five year period involving half the worlds population) with a century of the global population? Its also a fact the total deaths per year, at least in the US, is falling and the deaths per mile has dropped precipitously.
The problem with those statistics is that they include drunk driving accidents(particularly people who drunk drive and die) - it's like saying "10,000 people die on holidays every year!" while including people who do underwater cave diving. If you don't engage in a certain activity, it's impossible to die from it(obviously it's still possible to get killed by a drunk driver at no fault of your own, but to me that shouldn't count in the same category as drunk people dying when driving).
Because it makes it look like your risk of dying is much higher than it actually is. To pick another example - it's like saying "your risk of dying before 30 years old is 10%" and then calculating that risk including heroin users. If you don't use heroin that's not your risk.
That's a poor comparison. A heroin user is not going to accidentally kill anybody else when overdosing. Children that never even drove a car can die when a drunk driver smashes into a pedestrian crossing.
Nope, I also think humans are pretty good drivers given the complexity of the task. That's not to say we couldn't do better nor that machines cannot do better, but we aren't that bad.
Although these two statements appear to conflict, I believe both are true:
- the rate of traffic fatalities are unacceptably high
- the benefits of automobile usage far outweigh the costs, even with today's fatality rate.
The changes necessary to get fatalities completely down to zero would likely increase costs and reduce benefits too much. But reducing fatalities by an order of magnitude? Very feasible IMO.
This statement seems to be based on the assumption that Uber's efforts can only do better, never worse.
This is by no means proven, and evidence suggests that it is wrong.
Cardiovascular disease is the number one cause of death, but I would still like some testing to be done on possible preventative measures before they're given to patients.
Because in order to live a decent life, they have chosen to work and live in two different places which are not close enough to other people's destinations to warrant mass transit, and thus need to drive between them.
And better public transport, maybe. With no cars and no public transport, there would probably be no SF Bay Area at all, but a bunch of smaller communities, each with housing and jobs.
> but they need cars precisely because they use them.
Have you ever left the city for a destination which is not another city, or a suburb?
Sure, if space ships were impossible or too expensive to even consider, we wouldn't be planning to visit Mars; but it isn't equally valid to not visit Mars altogether as an alternative to visiting Mars, just because going to Mars is harder than going to the corner store.
I'm not criticizing, I'm pointing out some critical feedback loops that happen in our society.
> Have you ever left the city for a destination which is not another city, or a suburb?
Sure. My point is not about vacations, or Mars trips, but precisely about regular commute. As more people started having long-range vehicles, the landscape of work and living spaces changed so that now people are required to have such vehicles, and if you don't have one, you're significantly disadvantaged on the job and residential markets. The same thing happened with cellphones, bank accounts, plastic cards - cool tools at first, and then soon requirements for living in modern society. I'm just pointing out this phenomenon for consideration.
> My point is not about vacations, or Mars trips, but precisely about regular commute.
I'll set aside for a moment how you seem to think that the only reason to leave the city is for a vacation. My point is that there is a tremendous benefit to people being able to choose precisely where they want to live and work, and that the car is the means to that end with the fewest limitations. More people could probably take mass routes in larger vehicles, especially if bicycle racks became larger and common enough to rely on, but that won't diminish the value of the car, even for a commute. Cities also seem dead-set on installing trams and subways, when buses clearly offer the best bang for the buck (but nobody likes to say that they ride the bus, probably because buses are unloved).
Personally I think the more effective route is to encourage less commuting altogether, which would be an obvious natural effect if the roads were on the market, but in practice it would have to take the form of a "credit". Some companies (e.g. Microsoft) have company shuttles, those could be encouraged as well (since roads are not on the market, and thus do not encourage pooling).
1. Can polystyrene golfcarts move at real-traffic speeds? And be light enough to not hurt anyone even when they contain fuel, an engine, a human/backup driver and the recording equipment? I'm not an expert but it seems unlikely.
2. I understand that Google does the vast majority of its testing without any iron at all, but rather tests new software using recorded input.
Heck I don't understand why most people drive lethal SUVs for short trips that could be much more efficiently accomplished with a golf cart aka NEV. Especially in dense urban areas where your average speed ends up being only 10-20 mph anyway.
I don't see the problem. If you're going to own one vehicle, you need to size for peak loads. If you hail a self-driving vehicle, you can size for the trip. I see no reason to expect the composition of rental fleets to mirror purchased vehicle preferences.
Carpooling is not a meaningful comparison. Carpooling requires picking up (and tolerating) other people on a semi-rigid schedule. Hailing a self-driving car has all the benefits of driving solo, plus you don't have to drive.
GP wrote "most people", not "anyone" (though I did write my comment as a little dig ;)).
Though to be honest, me & my SO are driving a compact MPV which is larger than our basic needs (no children so far) - we were wondering whether to get something smaller, but it was the best used vehicle available within the budget at the time. And we frequently find use for the extra interior space.
I was rear ended and totaled and got an SUV, ended up that I would be moving every few weeks or months for a year. Having an SUV was amazing for this purpose. I'm finally settled and will probably get a car again for fuel efficiency (40 mile commute) but a truck/SUV is something that can be useful for anyone. I can fit a full 48u rack and have some spare storage left, and having it available if I see something cool is nice.
I grew up where it snowed a lot. I've never owned anything that sat particularly high nor with 4WD, and I've never missed a day of work due to snow.
My street floods.
If the water is above the air intake of a Honda Civic, your SUV isn't going to do you any good.
I frequently want to carry a large number of items.
I'm continually shocked at how few SUVs exceed the cargo volume of our 35mpg Scion xB. I've moved furniture for a friend because his Equinox wouldn't carry it. I rented a Nissan Xterra to carry a bicycle box that fit less snugly in the Scion than it did in the Nissan. SUVs might have a use, but carrying things is not one of them.
Until you threw in the ol' "I have a family" (because the children can not be seen in a Volvo wagon), I thought you were doing a parody of "Excuses SUV Owners Use to Justify Their Big-Ass Vehicle". I'm pretty sure I was wrong.
Edited to reflect the reasoning for snow. Ground clearance isn't your main issue in a flood. YouTube is filled with videos of SUVs getting washed away because the owner thought their ground clearance would save them (okay, maybe not "filled", but I've seen more than one video).
The difference in ground clearance between an accord and a cr-v is 2.4 inches. On unplowed side streets and my street that is 8 feet above sea level in a town that averages 14 feet above sea level that matters. Why you keep arguing when our situations are different is surprising.
>I'm continually shocked at how few SUVs exceed the cargo volume of our 35mpg Scion xB. I've moved furniture for a friend because his Equinox wouldn't carry it. I rented a Nissan Xterra to carry a bicycle box that fit less snugly in the Scion than it did in the Nissan. SUVs might have a use, but carrying things is not one of them.
That's like saying you don't need a fullsize truck because the methheads seem to do just find hauling scrap metal in their Dodge Caravan.
For a load with a constant amount of space usage (e.g. kids + sports stuff) having more space will make hauling that load less of a chore. If you do it regularly then the luxury of a bigger vehicle is often considered worth the cost.
I can and have hauled fridges, office desks, etc on the roof of my wagon. I always prefer to use my truck for those things unless the item can't stand getting dirty from the truck or I'm going so far that the extra fuel economy and better driving experience of the car makes it worth it. Strapping a desk to the roof of a car and not doing a crap job is a chore. Just tossing it in a truck and strapping it down takes like 30sec.
A vehicle with more space can haul more crap or haul the same crap more easily.
As an aside, I wouldn't let my kids be seen in a Volvo Wagon. I'm white trash, not some hipster. A rusted out Subaru is more my style.
That's like saying you don't need a fullsize truck because the methheads seem to do just find hauling scrap metal in their Dodge Caravan.
That's like saying that one kinda, umm, perhaps didn't completely comprehend my point. Unless you're arguing that a Dodge Caravan has more space than a Ford F-150 (and maybe it does, dunno).
having more space will make hauling that load less of a chore.
Yes, I completely agree, that's why I like the Scion over an SUV.
As an aside, I wouldn't let my kids be seen in a Volvo Wagon.
So you're worried about the image you project as well. Fair enough, I think that might apply to many vehicle owners including, to a degree larger than I care to admit, myself.
FWIW, crossovers (the bulk of the "SUV" market) are morphing into station wagons with slightly higher rooflines. Have you ever seen the subaru outback? That's considered a SUV.
The classic SUV, in other words a truck chassis, is an ever shrinking part of the market. FJ Cruiser, Xterra, 4Runner...
I'm super in favor of smaller transportation and in fact don't own a car, but there are a lot of practical reasons you're ignoring. The size of an SUV isn't such that it only helps in moving compared to a golf-car. There aren't golf-cart sized vehicles that are road-legal in the US and if there are, they're not well-known so who would think to buy one? Golf-carts would actually be less safe on roads where others are driving SUVs (there's a tragedy of the commons here but it's not impractical for each person making that choice). If you need a full-sized car 10% of the time, then you have to buy a golf-cart AND a full-sized car, so it'll always be more expensive to have a golf cart until there are viable alternative ways to access a full-sized car (ZipCar, I suppose). Current golf cart-sized vehicles are mostly open-air and do poorly in inclement weather.
Yeah, I didn't spell out practical benefits of having a SUV, though I feel that in most cases, those are just perceived benefits, not ones actually taken advantage of. Also, I don't think it's SUV vs. golf cart; it's SUV vs. regular cars, for city commute.
But my overall point is that I believe for most people (myself included), choice of a car is atypical in a way in which it includes emotions instead of pragmatism.
> For the off chance that once a year they might need to take some extra large load (e.g. help a friend with moving)
This sounds like something a caucasian single person would say.
Anyone with small kids would immediately have to think about the logistics of child car seats plus a stroller/wagon (at a minimum), plus whatever else would be appropriate to bring to the target destination (e.g. chairs and a cooler box for a picnic). Those are certainly not thing people do in an off-change.
Also don't forget cultures with tight-knit families (e.g. indians) where it's not uncommon for grandparents to live with you (and by extension travel with you) as well.
Anyone with small kids would immediately have to think
about the logistics of child car seats plus a
stroller/wagon [...]
Sure, but you can fit a 4-person family [1] into a Toyota Corolla, even with car seats and a stroller. I've seen it done plenty of times.
My theory is the increasing car sizes are driven by increasing waist sizes; a 220 lb couple could be cramped in a car their 140 lb grandparents might have found plenty large enough.
Oh sure, but "plenty of times for a subset of people for a subset of destinations" is a different thing than "every time for everyone for all destinations" (and a very real reason why people buy big cars in real life). When I was a kid, we could fit 4 of us (me, my 2 brothers and my grandma) in the back of a Civic since we were all scrawny and lol what's a backseat seatbelt in Brazil in the 80s, but now as an adult with 2 toddlers and their grandparents, a whole-family trip in a Civic to Costco to buy diapers and paper towels would be interesting to say the least.
Male caucasian singles often don't realize that there are cultures where your parents or in-laws come to live with you (often for several months) when your kid is born.
Doesn't this lead to an arms race where cars keep getting taller forever? Maybe we should solve this by making cars transparent at eye level and higher.
In theory, parking should not be allowed that close to an intersection, and bushes/trees should be trimmed or cut down as necessary to allow for adequate visibility. I know that reality is sometimes different, especially if you live in a tree-hugging community where even pulling weeds is a sin.
Right; drivers should have optimal visibility around corners so they can whip around turns without slowing down. That's the best design for a livable community.
Probably because people don't want to maintain an entire fleet of different vehicles such that they have the ideally suited one for every half mile increment in intended distance of travel, cross-producted with numerous increments in payload size and changes in weather.
Because you typically do not have a fleet of cars, one for each purpose.
It'd be really nice to have a golf-cart-ish car, a little SmartCar, a proper sportscar, an SUV, and a big pickup truck, and use the appropriate one in each situation.
But, even people that can afford the spare cash for that kind of stable of cars rarely do it -- it's just too inconvenient, too much maintenance (or too big a garage to keep them all in to minimize maintenance), depreciation, insurance, etc. and nevermind the constraints of city or near-city living where space is at a premium.
So, people just get the closest thing that fits most of their use-cases, and that's most often SUVs and crossovers.
I'm and example; while I'd be happiest in a small sportscar and it'd work for 85+% of my trips (and save fuel money), my occasional and unpredictable need to carry big packages for work keeps me using an SUV for my daily driver. Even if I wanted the extra expenses, I have winters and no spare garage. So...
The replies to your comment seem only to focus on ways that this type of safer vehicle differs from the eventual commercial traffic use case.
But who cares? It still seems like it would be valuable to collect many millions of simulation hours and eventual live testing hours with far safer vehicle types first, at minimum just as a proof of concept and a prove-out of the appropriate testing and evaluation infrastructure (e.g. we’ll probably be a lot better at extending the audit process that evaluates autonomous golf carts into an audit process for common cars than we would be just jumping straight to auditing cars without ever having audited anything else).
In fact, it seems irresponsible to not prove out the entire end to end ecosystem (from simulations to live testing to auditing to a commercially approved deployment in some limited scenario) using a much smaller scale application and vehicle type.
I can understand the arms race motivation for cutting corners and jumping straight to the widest use cases on the part of corporations. But I can’t see why society would want it handled that way.
> But I can’t see why society would want it handled that way.
Because these self driving cars aren't being developed to solve a hypothetical problem, they're being developed to solve the very real and current problem of cars killing 40,000 people in the US each year.
If extra safety measures result in killing ten fewer people in testing, but delay the introduction of self-driving cars by a year or two, that could mean thousands of preventable deaths still occurred. Are the lives of those people definitely getting killed right now worth 100x less than the lives of those who might get killed with these tests?
And this isn't baseless speculation. People are already obviously dying all the time, and improving these cars' software would be much harder without the kind of real-world data they're currently collecting.
It is baseless speculation. There is a high chance, that drivers most likeky to die in a car accident are less likely to use SDC.
Also, estimating how safe sekf-driving car is you should take into account all the cases where human took over.
Also there are many accidents in weather conditions where self driving car would not be able to drive at all.
> There is a high chance, that drivers most likeky to die in a car accident are less likely to use SDC.
Possible, but speculative.
> Also, estimating how safe sekf-driving car is you should take into account all the cases where human took over.
Sure, but on the other hand, the SDC's are continually improving as a group in a way that is not true for humans. This makes it harder to extrapolate into the future one way or the other.
> Also there are many accidents in weather conditions where self driving car would not be able to drive at all.
If you mean environments where humans also can't drive worth a damn, yes, probably true, although SDC's may yet prove better, since that's a technical problem rather than a social understanding one.
But 90%+ of accidents involve human error, so even if you can't completely eliminate deaths, you can still radically reduce them.
> "Because these self driving cars aren't being developed to solve a hypothetical problem, they're being developed to solve the very real and current problem of cars killing 40,000 people in the US each year."
Yes, exactly. This is the reason you would want the technology to be developed slowly and carefully, incubated in a safer scenario for a long time to ensure it can be effectively audited and deployed when it comes time to use it for the intended, safety-improving commercial traffic use cases.
What you sounds sounds like good reason to more strongly favor a lengthy test and prove-out cycle on safer and more limited vehicle types, long before testing on commercial traffic vehicles.
The rate of auto-related deaths happening today certainly would not be a reason to hastily rush the technology and jump directly to extensive testing on the final commercial traffic end use cases. Trying to "hurry up" and prevent those deaths could counter-intuitively lead to more deaths because the infrastructure and auditing processes were not proved out and slowly ramped up to real traffic scenarios.
> Trying to "hurry up" and prevent those deaths could counter-intuitively lead to more deaths because the infrastructure and auditing processes were not proved out and slowly ramped up to real traffic scenarios.
You assert this, but provide no reasoning. I mean, the status quo is "hurrying up", since there are no general standards companies are being held to, and yet only one company has killed anyone, and surprise surprise, it's the company with an extensive history of ethical problems, and they screwed up in a way that's really obvious and foreseeable.
The status quo is a free for all, and yet we don't have much in the way of deaths and injuries from the millions of miles being driven by computers. It seems like companies are already themselves being fairly safe, so how many deaths would you actually be avoiding by implementing more stringent standards right now?
Huh? The status quo is a free for all? You are not describing the world we live in at all.
In many industries, from food development to drug development to medical devices to clean energy tech to child car seats to film ratings we have nuanced and highly involved government oversight and regulations to try to ensure levels of safety, ethical and thorough clinical testing, ratings systems that map to outcomes, etc. etc. Yes, of course they are not always perfect, but that is beside the point entirely. The ubiquitous and widespread use of such regulations is a direct statement that companies cannot be trusted, have repeatedly proved through history to be untrustworthy regarding safety over and over across many industries, and that it's a civic need to have independent oversight. We can debate the amounts, regulatory capture, etc., but absolutely could not ever believe something like saying the status quo is just an industry free-for-all.
In fact, cases where start-ups try to skirt these issues in under-regulated industries are extremely rare and do not at all represent any type of status quo in any sense. There is absolutely no sense in which that maps to the way things work. It is rare for a industry to come up with its own testing and evaluation strategy where societal outcomes aren't brought up as first-principle constraints on how the innovation has to proceed.
> "It seems like companies are already themselves being fairly safe, so how many deaths would you actually be avoiding by implementing more stringent standards right now?"
This is just so egregiously unrelated to reality that I do not have the time resources to even start to reply. My mouth is agape that someone would think this.
> "You assert this, but provide no reasoning."
I agree with this for self-driving cars. I am not an expert of self-driving cars, and feel it's perfectly fair for me to say that in virtually every other industry, regulations are needed to ensure corporations don't actively harm people, falsify results, rush to market, etc., and that it logically and obviously extends to self-driving cars without any need for some onerous debate. The idea that companies are just trustworthy to rush this stuff to commercial markets does not need to be taken seriously. It's just obviously not so.
Beyond that, I completely agree I am not the expert on it, and would defer to e.g. the NTSB to decide how companies are behaving with regard to proper testing. The only part I would add is that as a citizen, my strong preference is for the testing to be gradual with a high emphasis on validation and a gradual proof of concept moving from simpler vehicles and situations to more complex ones. Regardless of what the raw machine learning capabilities are in terms of self-driving, this is needed to prove out the evaluation and auditing infrastructure as much as anything else -- essentially to prove that we have a framework for holding the companies accountable in the right ways.
> Huh? The status quo is a free for all? You are not describing the world we live in at all.
I'm specifically talking about the current development of self-driving cars. I'm not sure why you thought I was generalizing to all activity among all industries.
> This is just so egregiously unrelated to reality that I do not have the time resources to even start to reply. My mouth is agape that someone would think this.
What's weird about it? Again, we're not talking about solving some hypothetical future problem, in which case I would agree with you about being hyper-cautious and safe. Tens of thousands of people die every year right now. Where is your mouth-agape-level concern for those people?
When a disease like Ebola breaks out, sometimes an experimental drug is used, earlier than would normally happen. This is not because we are collectively giving in to greedy companies wanting to destroy all government regulation, it's because we recognize that the dire need for a solution sometimes outstrips the need for caution. Here is a recent example: https://www.nature.com/articles/d41586-018-05205-x
Do you disagree with this kind of response for serious disease breakouts, or does 40,000 deaths a year just not count as sufficiently dire? Would 100,000 breach that threshold? 500,000?
If the choice is between getting self-driving cars more quickly with minimal regulation -- which is what's happening right now -- versus much more regulation that will slow their development and introduction, it seems straightforwardly obvious that more people are likely to die in the status quo environment in the latter scenario than people die from testing in the former scenario. Companies would really have to go full Mad Max to kill even close to as many people as our current infrastructure and design kill.
> it logically and obviously extends to self-driving cars without any need for some onerous debate.
That's what I'm challenging, and I'm doing so on the basis that right now, things seem to be going fairly well. One person has died, which is very regrettable, but compared to the absolute mountain of corpses our usage of cars has created in general? It's obvious to me what's more important.
> "If the choice is between getting self-driving cars more quickly with minimal regulation -- which is what's happening right now -- versus much more regulation that will slow their development and introduction, it seems straightforwardly obvious that more people are likely to die in the status quo environment in the latter scenario than people die from testing in the former scenario. Companies would really have to go full Mad Max to kill even close to as many people as our current infrastructure and design kill."
I guess I am saying this seems to be straightforwardly false to me, to a huge huge degree, and it worries me that people would naively use the rate of auto deaths under current circumstances as a justification to allow self-driving technology development to proceed unregulated.
It seems we just believe very strongly in positions that are completely and diametrically opposite to each other, so we are unlikely to find productive communication on a message board about it.
> I guess I am saying this seems to be straightforwardly false to me, to a huge huge degree
Why?
> It seems we just believe very strongly in positions that are completely and diametrically opposite to each other, so we are unlikely to find productive communication on a message board about it.
I mean, if you're not willing to explaining your reasoning, then yes I suppose that's true. Although I think part of why you may feel that way is that, based on one of your earlier comments, you seem to think I'm some kind of libertarian who's just generally opposed to regulation. I'm not; if anything I'm more of a social democrat. I just think that in this particular case, things are different from normal for a new, potentially dangerous technology.
My reasoning on the above point is fairly simple and straightforward:
1. The status quo for SDC development has been relatively minimal regulation. Even California mostly just makes companies report some basic data.
2. Most companies seem to be taking safety very seriously as it is, such that injuries and deaths have been rare. Only one death, and from a company widely acknowledged as ethically-challenged.
3. Given 2, it seems like the number of deaths that you'd avoid with greater regulation is very low, simply because, well, very few people are dying as it is.
4. Conversely, letting humans drive cars has resulted in an enormous number of people dying each year.
5. Given 3 and 4, slowing down SDC development with greater regulation seems likely to indirectly kill many, many more people than the status quo of minimally-regulated development will kill.
Are you expecting Waymo to, any day now, start rampaging around cities and killing people by the dozens? What does the scenario look like where SDC development kills more people than human drivers currently kill? You didn't address my analogy with disease outbreaks; why not?
You've done a good job arguing for the importance of regulation in general. You haven't done a good job arguing for why you think greater regulation in this particular scenario will avoid killing more people than the current setup kills.
Tens of thousands of people die from specific medical problems without us deciding that non-consenting ordinary citizens should be human guinea pigs for risky speculative attempts to test the efficaac of possible prophylactics. I'm really not sure why you think autonomous cars would or should be treated any differently.
Especially when the evidence from the current development of self driving cars is that a self driving car killing someone results in an entire testing programme to be suspended for months, possibly permanently, whilst the company is excoriated for taking unnecessary risks and is considerably less likely to develop a commercial application of SDVs as a result. It also shows that SDVs kill at a higher rate than non-SDVs even with human backstops, which is a much better test of whether they will actually save lives than "does a handful of test vehicles kill as many people as a few hundred million humans"
Your statement is an indictment of SUVs, not self-driving vehicles. Every one of those reasons for it being a bad idea for Google/Uber applies to it being a bad idea for you, too.
Giant vehicles are a bad idea, period. But they're no less bad for robots.
As you become "good" at driving in harder conditions, you'll use your butt (which is really how you're physically connected to the vehicle) to figure out the traction state of the car, to know if the car is sliding, if it's close to losing the rear end or the front end, if it's pitching down or up, and so on. This is how you know which corrections to make to the vehicle to keep it going fast without losing control.
Pilots of small planes do the same thing. After a while you can just feel small accelerations and change of directions without needing to check some of the instruments.
Huh, fascinating. I remember reading a Star Wars novel ages ago in which they said that one of the things that made Luke Skywalker a good pilot was that he slightly detuned his craft's inertial compensators (or whatever the Star Wars version is) so he could feel how the craft responded. Didn't realize that was actually based on real life driving/flying!
The common complaint against "new" vehicles and their drive-by-wire systems is that they functionally act as inertial compensators, dampening road feel.
If the roads were perfect, you wouldn't necessarily want them (day to day), because it diminishes the minute vibrations that make their way up to the pedals, into the steering wheel, etc., and those things give you feedback into what the car is doing that are invaluable.
Of course, the roads aren't perfect, and most of us aren't doing high-performance driving on them, and we don't need anywhere near that degree of feel, so the benefits of driving by wire far outweigh the cons of analog driving.
Your last paragraph was what I came here to say: I fly a small Cessna 172 and when i first started to fly i was being cleared to solo and the instructor who cleared me gave feedback on my landings at the end. His main remark was "You'll start to feel it in your rear soon enough, it comes around the 150-200 hour mark, but in the meanwhile, trust the instruments and your visual indicators". Ironically you're simultaneously taught to always trust the instruments over your inner ear, and generally as much as I do get the buttock feeling now, I would say my primary focus remains on the visuals + instruments.
Though pilots have to be careful, because feelings can be deceptive especially if you can't see the horizon (clouds, night). In those conditions you must trust the instruments over what you are feeling.
Not a professional driver, but I've ripped up several racetracks and done plenty of drag racing as an older teen. Your butt tends to feel everything if you don't have ultra-cushy seating, and because of its position relative to the vehicles center of gravity, it will feel more of the effects of various vehicle movements versus the arms or legs. Tune in long enough and you can literally read the road you're travelling just by how it feels, roughness, amounts of liquid on the road, etc.
Ever notice how your butt tries to move more than your hands and feet when making a sharp turn at a moderate speed turn, and how you instinctively correct for it in your seat by leaning or shifting around? Know the phrase 'Fly by the seat of my pants?' Pretty much all of that rolled into one bundle of instinctive attunement.
It should be mandatory for all self-driving car (system/software) vendors to publish their test plans and all test logs for public to review before the car/software are allowed to be on the public road.
It likely allows everyone to learn from all the mistakes and improve the process for the whole industry - similar to the passenger airplane safety record improvements learned from every single accidents/crashes.
Perhaps we will end up with a standardized validation simulation model that autonomous vehicles must pass for licensing. The test suite could grow as more anomalous scenarios are analyzed (ex: acceleration into gore point; white-painted trailer truck turning across divided highway; debris on highway), and it would ensure that any new vehicle on the market or OTA update would meet a minimum of safety. Manufacturers could compete on test suite coverage and acceleration curve stats, and market using rendered video of simulation results showing superhuman skill.
In addition I hope to see independent 3rd party reviewers developing their own autonomous car test suites so we can see how off-the-lot cars operate under hostile test conditions.
Given time to market constraints and the potential value of having widely adopted autonomous vehicle software, it's not all surprising that vendors will take experimental shortcuts. And it's not at all surprising that sometimes, those shortcuts will result in fatalities.
That being said, I still want this tech on the road ASAP, not because it's perfect, but because human drivers are So. Fucking. Awful. They kill thousands of people a year due to negligence, incompetence, and occasional malice. Plus, the additional sensors available and the software response time mean autonomous drivers can outperform even sober, attentive, skilled human drivers in most situations.
A couple of weeks ago, I saw someone talking scary stuff about what someone here called "semi-autonomous murderbots" in a FB post that appeared on my feed immediately after several posts about a local accident, where a bicyclist was killed by a road-raging driver, who then hid/destroyed his car (the murder weapon still has not been found, even though his trial is now over). If that jackass had a self-driving vehicle, the bicyclist would still be alive today.
So are big corporations going to do kind of a careless job testing live vehicles in a rush to market? Yes. Will people get hurt? Yes. Is it worth it? Yes.
>That being said, I still want this tech on the road ASAP, not because it's perfect, but because human drivers are So. Fucking. Awful.
This narrative is so pervasive and I must admit a bit strange to me. Millions and millions of people strap themselves into thousands of pounds of metal and then drive 80 down the highway while texting and changing the radio and telling your kids to please stop fighting surrounded by other people doing the exact same thing. And we do that every single day and almost all of us are fine and will never have a 'major' accident. Humans are pretty fucking great at driving.
Besides to say that human are bad (or good) at driving would require some comparison, we're bad at it compared to what? As far as we know humans are the best drivers in universe. There's no evidence that self-driving is currently better than human drivers and there's no evidence that self-driving cars will soon be better than human drivers and there's no evidence that if we allow tons of self-driving cars on the roads then they will be better than humans. It's just blind hope, it really makes me want a word for whatever the opposite of a luddite is, instead of a belief that we can destroy machines to benefit people we believe we can destroy people to benefit machines. It's all twisted up.
If you want to be an Uber or Tesla crash test dummy and walk/bike across a test track while they see if their vehicle can avoid you then by all means go for it. But don't just foist that upon average people.
It's not that humans are bad, it's more about humans being unreliable by definition. That's why we have autopilots, industrial robots, elevator or traffic light control system, or, in general trying to automate stuff.
And we could install some cheap device in each cars to detect when a driver is tired and keep him awake, and detect a drunk driver and stop him drive or perform some other action.
Until most of people will afford a self driving car you would still have to face the drunk, or the ones that text so you will not remove those of the roads.
There are luxury cars that attempt to detect when a driver is falling asleep, and people are sometimes mandated to have an interlock on their cars that prevents starting if they have alcohol in their breath. So if these things are not universal, presumably it's because of cost-benefit analysis, not because they need to be invented.
I was involved back in 2003-2004 with an Australian company called SeeingMachines (I worked for a UK redistributor).
These guys made a remote video face-and-eye tracker that they sold to automotive companies wanting to research different dashboard designs -- they would look for blinks and head-nods to measure how fatigued a driver was, and to optimise the dashboard design to keep the driver awake.
I'm sure they are now selling their technology to automotive OEMs for placement in driver-facing camera ECUs.
Their technology worked OK back in 2003. I'm sure it is very robust and mature by now.
Which is why they aren't quite legal yet. And it's arguable anyway, given the noise in the signal (a tiny handful of fatalities to compare with the ~100k "normal" traffic deaths over the same period).
But like all new technology, it's improving. People aren't.
They are also better pilots than self-flying planes. And better writers then self-writing blogs.
And so on. There, of course, are many examples of things in which computers have outpaced human capabilities, and it's certainly plausible that some day this could apply to cars. But it hasn't happened yet and it's also more than plausible that none of this will happen in the lifetime of anyone now living.
For some values of 'human' e.g. alert, sober. Which is sadly, a minority?
Its arguable that self-driving cars are better already. What we need to do is, secure the roads so they won't get confused (fence them & remove pedestrians/bikes). I know, we can't do that everywhere but we could do it for long-haul etc. Like freeways already do.
>For some values of 'human' e.g. alert, sober. Which is sadly, a minority?
I think the vast majority of human drivers are alert and sober. It's the long tail of drunk / aggressive / distracted drivers that causes crashes.
>Its arguable that self-driving cars are better already. What we need to do is, secure the roads so they won't get confused (fence them & remove pedestrians/bikes). I know, we can't do that everywhere but we could do it for long-haul etc. Like freeways already do.
I mean as a human driver, I obviously love limited access roads (and human drivers perform very well on limited access roads). We won't have more of those though - the cultural paradigm for urban planning is one of tearing down freeways, adding bike-lanes, and ensuring pedestrian friendliness for maximum walkability.
Everything you've listed involves repetitive actions (or complete lack there of, if my limited understanding of aviation is accurate), which is known to cause humans to lose focus (and make mistakes).
Driving a car, on the other hand, is usually NOT a repetitive, dull process. You're comparing apples to oranges.
Driving a car is absolutely a repetitive process. It's improvisational to a degree, but the mechanics and decision-making in response to stimuli are consistent, learned behaviors. And it's hard to argue that humans are mechanically better at driving than machines can be - better sensors, better feedback, more direct connections, better reflexes. So we're only questioning whether self-driving cars can respond to unexpected stimuli better than humans.
Technically, you are correct, but the spirit of my argument is in disagreement. Driving has a repetitive list of tasks, but unlike an elevator, the "buttons" to accomplish the task are moved around every single time. Sometimes new buttons appear, and sometimes some go missing. It's a chaotic environment. Thus, it is not a repetitive process in my mind, because a chaotic environment is not repetitive. It's like saying "A chaotic environment is repetitive because it's always chaotic".
The problem is there is no proof AI is better then humans, probably it can't ever be better then a human with the current hardware in the current cars, but we have "hopeful" people that want to sacrifice lives to gather data for shitty startups that need the self driving cars ready this year or it will fail.
So we need this:
-real numbers of AI to have a meaningful discussion on real statistics
-we need good statistics, so not compare different car types and driver ages/categories
-IMO we have a low hanging fruit related to preventing drunk drivers to use a car, a system to detect a driver that is not looking ahead for more then n seconds, someone needs to handle this, it would improve the numbers a lot
-when we compare say a chess AI we compare it with professionals so it would be more honest if we want to say that self driving AI is better we should compare with a driver with experience that is not drunk
-we should not RUSH this because it will backfire,wait for hardware and software to be ready so if a concrete object is in the front of the car the radar can detect it's position and the cameras and image recognition can detect what it is and not ignore it like it is a road sign, hardware and software should be ready to detect signs, bottles, people, bikes, animals as good as humans, if is not as good as a human then we already lost since a human also many more "features" like intuition,
Why does everyone assume the vendors don't have data? They're recording full sensor logs in these tests, tracking everything they can. They wouldn't even be in this field if they didn't think they could outperform humans - and dozens of companies are doing this, from tiny startups to Google to General Motors. No one wants to take on the liability of crappy systems that perform worse than humans.
One interesting point of the development is that every failure is an opportunity to learn. The same cannot be said for human drivers. When the automation doesn't detect a hazard correctly, they can analyze it and tweak it so it detects better next time. Do humans do that?
Yes, that is the hope, but the problem is it "CAN" but this does not mean it actually learns, if you seen what this AI do is just ignoring most of the data since the hardware is not powerful enough, the reality should be that the radar and cameras would work together and identify all the detected objects and not ignore any input. Tesla hit a firetruck, then a police car, it killed 2 humans in similar conditions, did Tesla cars learned from it anything? if it did it was not enough
During some political protests a couple of years ago, in my part of town, a driver panicked and chose to drive through a crowd of high school kids crossing a street, with a walk light and police supervision. Luckily no one was killed, but one girl went under the car and had her leg crushed.
Do cars actually live stream everything?
If not did Tesla and Uber extracted the data and did any improvement? I did not read anything about that, just blaming the driver and obfuscating the truth.
I am not against the self driving car idea, I can't drive because of my eyes and my son also has bad eyes(not as bad as me ) and epilepsy so he probably will not be able to drive a car, I am disappointed because the startups fucked this up by pretending that they can solve the problem with the crap weak hardware and software they have, as long as you can't detect the exact object position and use image recognition to correctly tag all objects and compute all speed vectors this can't be solved and it is clear the current hardware Tesla and Uber use is not enough.
Self driving may the future but is not the present and it is not the near future either.
Imagine an AI doctor,would you feel good if you know is better then a drunk doctor?
Would you send your child with a self driving car that is better then a drunk driver but not then yourself? ,after 1000 kills the ANN would MAYBE adjust some weight somewhere and the changes will increase a bit
Oh, that wasn't a Tesla, just an ordinary car, driven by a person who was terrified of that many young black people in one place. Which rolls around to a point that I keep making, that humans are emotional, and their emotional states greatly affect their driving, to the point of deliberately doing things that will hurt or kill others. Road rage is a common problem.
I agree self driving is not the present, but I think it's the near future - 10-20 years.
And to be absolutely clear - I believe that full-on self-driving technology, with no human assist or failover in normal on-road cases, will be a better driver than me within the next decade. I've had my driver's license for 37 years, and I love driving. I've had two moving violations in my life (last one over a decade ago), haven't been in an accident in 17 years, and haven't been in an injury accident ever. I'm a really good driver. And I'm kind of neurotic about being in a car with other people driving - it makes me nervous, even when I know they're good too.
But I'm totally looking forward to the day I can trust my car to do the driving for me, knowing it's better than I am.
Then we agree in most things, except I am most pessimist on how much time it will take for the technology to be at the level of a good human driver. Maybe if we get cheap and good enough radar/lidar or similar sensors maybe we could at least get cars that would not hit static objects.
The AI needs/has advantages like you said but those are needed to compensate the big missing part , the human intelligence, I am afraid of stupid scenarios where say a bridge would collapse and all the self driving cars would continue jumping of the bridge because this exact bridge colapse was not programmed into the car.11
>Do humans do that?
do you have proof that with current software and hardware Tesla or Uber can make an AI better then a human?
Because it feels to me they are faking it to buy time until some new hardware is ready. They have the data but the hardware is to slow so they can't analyze it in real time so they ignore it, even if say they want to put a concrete related update it may be too intensive and it won't run in current hardware.
Well said. These SDV threads are getting pretty annoying that we keep cycling the same arguments at each other and there is no substantive change to how good SDVs are yet.
Interesting factoid... half of US traffic fatalities occur between the hours of 9pm and 3am on friday and saturday nights. Average drivers in average situations are, indeed, pretty safe. But drunk drivers? Angry drivers? Texting drivers?
If autonomous cars can drive at the skill level of average drivers - and not get drunk or angry or distracted by forwarding cat videos - then they will significantly outperform humans.
Additional evidence in the form of reason... autonomous drivers have better sensors than humans. Human drivers have limited vision, that only points in one direction at a time. Mirrors offer some slight augmentation, but. Ever backed out of a parking space in a crowded parking lot and gotten into an accident because a car you couldn't see before was suddenly right behind you? Common situation. Autonomous drivers won't have that problem. They can see out the back and to the sides, 360 degrees at all times. (I'll note that some of the first deployed autonomous tech was cars that can parallel-park themselves.)
Additionally, autonomous systems have better reflexes. Remember the two second rule? That's to deal with the delay from your eyes to your brain to your hands and feet. Machines can physically respond to changes in situation faster than humans.
Now, let's level up. Imagine a system where autonomous vehicles are transmitting short-range telemetry to nearby vehicles, letting them know their status, including velocity. They can relay back emergency status info - so the car that slams its brakes to avoid a pedestrian doesn't cause someone six cars back to rear-end the car in front of them. Merge-related accidents (related to parking-lot problems) can be reduced to nothingness with telemetry. Think about how Waze is doing this already with driving data for their mapping software. It'd work better with less human involvement.
I can go on and on, but really, I'm gonna throw a bullshit card here. Fear of autonomous vehicles is magical thinking trumping reason. Humans do magical thinking all the time. We think safe things are scary, and we think scary things are safe, for symbolism rather than fact. That's why people in this discussion used the phrase "murderbot", but not "murderer".
>Interesting factoid... half of US traffic fatalities occur between the hours of 9pm and 3am on friday and saturday nights. Average drivers in average situations are, indeed, pretty safe. But drunk drivers? Angry drivers? Texting drivers?
There is exactly zero proof that self-driving vehicles are better than drunk drivers. There is exactly zero proof that self-driving vehicles are better than angry drivers. There is exactly zero proof that self-driving vehicles are better than texting drivers. There is zero proof they will soon overtake any of those categories.
>Additional evidence in the form of reason... autonomous drivers have better sensors than humans.
Teslas have to ignore stationary objects because they can't accurately estimate the height of those objects. My 'sensors' know when a tire shred is on the ground and about 2 inches high and when a bicyclist is on the ground and about 5 feet high.
>Additionally, autonomous systems have better reflexes. Remember the two second rule? That's to deal with the delay from your eyes to your brain to your hands and feet. Machines can physically respond to changes in situation faster than humans.
I have driven while texting, I still sometimes drive angry and I have (unfortunately) driven drunk in the past. I have never made the very simple mistake made in that video. Where is the quick mechanical reflex you are leading me to believe exists in self-driving vehicles?
>Now, let's level up. Imagine a...
No. Your imagination is running wild imagining something that doesn't and won't in the near future exist. Your imaginary car not only is covered in sensors, but can p2p communicate with the cars around it, has no lag in processing complex input, can behave itself among human drivers, etc. None of this actually exists, none of it is actually out there and we really don't have good reason to expect it any time soon.
>I can go on and on,
I believe you could, but it's just your fantasizing, it's not a realistic portrait of what is actually happening.
>I'm gonna throw a bullshit card here.
Going to? What about the several you've already sent my way lol.
>Humans do magical thinking all the time. We think safe things are scary, and we think scary things are safe, for symbolism rather than fact.
Exactly! That's why people in this discussion called humans "So. Fucking. Awful." at driving and then wrote three paragraphs imagining a utopian, non-existent self-driving system which magically solves all problems and is really quite safe. Before diagnosing magical thinking in others you might attempt detecting it in yourself.
You keep insisting there's zero evidence of the performance of autonomous cars relative to human drivers (you're talking about drunk/texting, but I'm distilling the generalization from it). What evidence do you have that there is zero evidence? After all, we have off-road lab results from many manufacturers. We have on-road results. Some early-stage autonomous tech is already widely available and in use today, such as collision detection with auto-braking to prevent rear-end collisions. There is TONS of data. Stop pretending it doesn't exist.
Your video is an interesting example. That is not a fully autonomous vehicle, but rather an assisted one, one that expects human engagement. In the example case, it failed because of human error. Now, there's a good argument that semi-autonomous vehicles will lead to additional human error, but that's not saying what you're saying.
If you live in an icy climate (I do), and have been driving for a long time (I have), then at some point, you would remember the transition to anti-lock braking systems. Before them, we had to learn to pump brakes in certain ways on icy roads, to prevent locking. Now, just push down the pedal, and a computer does it for you - one that senses brake lock before it happens. It made winter driving much safer.
Your criticism of Tesla sensors is flawed. You're assuming that their sensors will not get better. That seems an unreasonable and unlikely conclusion. And you're ignoring my point about 360 degree sensor coverage. I'm an excellent driver, but I find myself in danger regularly because I can't see everything.
>What evidence do you have that there is zero evidence?
You're really contorting yourself here, just give me an apples to apples comparison of human driving safety to self-driving vehicle safety and I'll be happy. (Hint: none exists)
>That is not a fully autonomous vehicle, but rather an assisted one, one that expects human engagement. In the example case, it failed because of human error.
No no, no word games. That is a Tesla marketing/liability line. What you see in that video is the current state of self-driving. And a product marketed as self-driving which can't steer around a stopped vehicle in the lane isn't really self-driving at all. Besides if what you're talking about is Level 5 type self-driving systems, you have nothing supporting the idea that we get there soon or that current self-driving systems will get us there.
I mean besides if drunk drivers could navigate around that car then you're basically admitting that drunk drivers are still better than the current state of self-driving technology. (something I agree with)
>ABS
...
I think you're misunderstanding me. I'm not against technological progress which makes vehicles safer. I'm against the understanding that self-driving technology is making cars safer.
>Your criticism of Tesla sensors is flawed. You're assuming that their sensors will not get better. That seems an unreasonable and unlikely conclusion. And you're ignoring my point about 360 degree sensor coverage. I'm an excellent driver, but I find myself in danger regularly because I can't see everything.
And you're assuming that their sensors will leap and bound ahead magically. You're trying to make me argue that humans are better drivers than the perfect system that exists in your head. I'm saying the perfect system that exists in your head is irrelevant because it is not going to become reality.
I'm not saying autonomous vehicles will be perfect. I'm just saying they'll be better than humans. Which isn't exactly a high bar.
As for drunk drivers navigating around that car? There's an intersection a couple of miles from my house that has a stoplight. Southbound traffic is approaching from highway speeds, climbs over a hill, hits a 40mph zone, then the stoplight is a couple hundred meters. There have been multiple fatal collisions there by, well, drunk drivers, slamming into cars stopped at the red light at highway speeds (or faster). I'm terrified any time I'm stuck at that light after dark, especially on weekends.
Of course, an autonomous car would detect the speed zone change and the red light.
I'm saying that actually that bar is extremely high.
>Of course, an autonomous car would detect the speed zone change and the red light.
Would it? I think it's actually unclear that it would and besides this experience of yours is filling in for the place in your comment where you should be showing me the data that shows self-driving cars are safer.
If an autonomous car can't read speed limit signs and see stoplights (and doesn't have an up to date on-board map of them), then that system has no business being on the road. But I'm assuming that low-hanging fruit like reading traffic signs and signals is a solved problem.
Assuming that it isn't a solved problem is kind of irrational. If your argument is "Well, maybe the car won't see the traffic light it already knows is there", then perhaps your argument isn't very good.
>If an autonomous car can't read speed limit signs and see stoplights (and doesn't have an up to date on-board map of them), then that system has no business being on the road.
Then you're kind of going against your original comment where you said you wanted self-driving vehicles on the road asap. All you have to do is google: Tesla Speed Sign and you'll find tons of problems and examples of Teslas misreading signs. Here are some examples:
My AP1 car has interpreted a 35 mph sign as an 85 mph sign.
---
My understanding is AP1 visually identifies speed signs, but can screw up (reported that our highway 80 is misread as 80 mph)
---
Addendum: a 45 mph zone, permanently signed (white/black sign large - not orange/black sign small), which by the way AP2 completely ignores. See error-ridden, and in this case an example that creates a safety/financial hazard not present with AP1.
---
And on some highways in the area, especially where there has been recent construction, the speed limit database thinks the speeds are 45 or 50 MPH in sections - when the posted speed limit is supposed to be 60 or 65 MPH. If the car is running under AutoSteer when it hits one of these sections, the car immediately slows down, because AutoSteer won't exceed the speed limit by more than 5 MPH on non-highway roads.
>(and doesn't have an up to date on-board map of them)
This is also something that doesn't always happen in real life. Here's a quick example just from googling:
The "new" speed limit database distributed last summer (from TomTom?) has numerous areas with missing speed limit data or incorrect speeds (both high and low), which causes issues with AutoSteer. And if this happens while driving at highway speeds, AutoSteer will try to quickly slow down in the middle of high speed traffic. While you can disengage AutoSteer and retake control, the other option is just to temporarily leave AutoSteer/TACC running, and manage the speed yourself until the software sees it's back in an area with the correct speed limit.
>But I'm assuming that low-hanging fruit like reading traffic signs and signals is a solved problem.
You know what they say about assumptions lol.
>Assuming that it isn't a solved problem is kind of irrational.
One, it has nothing to do with rationality. Two, it's actually not a solved problem.
>If your argument is "Well, maybe the car won't see the traffic light it already knows is there", then perhaps your argument isn't very good.
Or maybe the state of self-driving technology isn't actually as advanced as you believe. I'm glad we can agree that these systems have no business being on the road though.
Waymo has been transporting around the 400 people that applied for 8 months without any safety driver.
It has 7 million miles under the belt without any casualties and orders of magnitude more of simulated miles.
>If autonomous cars can drive at the skill level of average drivers - and not get drunk or angry or distracted by forwarding cat videos - then they will significantly outperform humans.
As long as you use the word "if" you can say pretty much anything you want.
>Additional evidence in the form of reason... autonomous drivers have better sensors than humans. Human drivers have limited vision, that only points in one direction at a time.
That is not evidence. It is your own opinion. Nothing of the sort has been proven to work reliably.
>Autonomous drivers won't have that problem. They can see out the back and to the sides, 360 degrees at all times. (I'll note that some of the first deployed autonomous tech was cars that can parallel-park themselves.)
They can't "see". They can only sense what they have been programmed to sense. The envelope of what they can process is tiny. Sensors have flaws, humans programming those sensors have flaws. The human eye and visual cortex has been evolving for millions of years.
>I can go on and on, but really, I'm gonna throw a bullshit card here.
The biggest "bullshit" here is that you are claiming fantastical things that autonomous cars can do that have not been proven in the slightest. You implying that other people are stupid for being apprehensive about unproven tech is quite rich...
Just a side note, magical thinking doesn't quite mean what you're using it for. It's not synonymous with poor reasoning.
"A self driving car could kill me or others at any moment, and whether that happens is completely outside my control" is a reasonable fear. People may over-weight it compared to more familiar dangers that are in fact greater, but that's not magical thinking.
Magical thinking is more about believing in a causal relationship between unrelated events. Astrology is an example, but also things like "if I don't shave my beard my hockey team will win" or "I thought something bad about my kid and then they got hurt, I'm a terrible parent".
>So are big corporations going to do kind of a careless job testing live vehicles in a rush to market? Yes. Will people get hurt? Yes. Is it worth it? Yes.
Hey, another human driver on the freeway here. Sorry, but I don't want to be killed because you want to play with some toys. It is completely unethical and immoral to ship unproven tech that has the potential to kill people.
I'd suggest running an experiment with OPT-IN volunteers in a neighborhood or small city for a year or however long it takes and then coming back with data.
Becoming proven tech won't necessarily make it better. It will just mean that the failure cases are well known, such as with other tech like cruise control, or anti-lock breaks, or power steering.
For better or worse, our laws acknowledge that being alive has unnecessary risks, and accepting the social order means accepting the standard of risk caused by other people.
It's not about "play with some toys". It's about saving lives. Tens of thousands of people are killed by cars every year in the US alone. And the primary causes of those fatalities are the kinds of things most easily addressed by automation - lane drift. Running red lights and stop signs. Speeding. Driving too fast for conditions. Aggressive maneuvering leading to loss of control.
Drunken or distracted driving, by itself, doesn't kill. It leads to the behaviors that kill, like lane drift and speeding and running lights. These are all things that autonomous cars handle really well.
You’re begging the question. Given performance to date, real world deployment is no way certain to be as safe - much less safer - than human drivers at the current state of the tech.
I don’t doubt it will get there and, when it does, these profit-oriented corner-cutting behemoths had better have a shit ton of data to prove it. when they’ve proven the premise you’re taking for granted.
If it’s about saving lives like you say, your go-to solution should be public transportation. It’s already proven effective. There are already automomous trains in operation world-wide.
Until you show statistics that prove self-driving cars can save lives, you're just emoting. In the meantime, we have several proven solutions to save lives, like public transportation, or encouraging bicycling (as the Netherlands did in response to their accident crisis).
When a human-driven car kills someone, the driver is held responsible and faces some sort of punishment. When a self-driving car kills someone, it's not clear that there are consequences, meaningful or otherwise, for anyone besides the person killed. Until this changes, I am fine with having a hardline stance against autonomous vehicles on the same road I'm on.
Volvo has the right approach. If the fault is on their car when it is in autonomous mode they will take full responsibility. They have said so multiple times publicly. Then again they don’t trust their system enough to give it to actual customers yet.
Yes, if they are drunk or clearly driving recklessly. Otherwise, it's an accident. Their insurance will pay, and they will go on probably with some emotional scars but nothing really in the way of "punishment"
> And the primary causes of those fatalities are the kinds of things most easily addressed by automation - lane drift. Running red lights and stop signs. Speeding. Driving too fast for conditions. Aggressive maneuvering leading to loss of control.
That's the thing. Fair enough, autonomous cars cannot be drunk or sleepy, but they have _others_ issues that "are most easily addressed by human driving". A human might be distracted, but it won't see a bicyclist for 10 seconds while thinking "I shouldn't do anything in case it's a false positive". Human driver won't directly swerve in a concrete wall because it followed to wrong white line. Human driver will predict better the trajectory of other humans, especially infants.
All in all, I do agree that automation can bring improvements to road safety, but in the current situation, it is not clear at all if this can counteract the automation downsides. It will surely improve, but hastiness might not be the way to go here...
In the Uber case, the system that would have automatically braked upon seeing the bicyclist was disabled. According to their data, it would have stopped on its own, had it been engage. Additionally, there was no system to automatically alert the driver of the hazard the car had been ordered to ignore. And the driver wasn't paying attention.
So it's not nearly as bad as it's made out to be. the fatality was a consequence of multiple human errors, both consciously and neglectfully. The automation was not in error.
Of course the automation was in error: it was so much in error they have to disable it!
All the arguments about the driver that should have been alerted or reacted are simply not autonomous driving so unrelated.
Again, I'm not saying that automation will never be a good replacement for human drivers, but as of right now, it is not clear at all if its positive aspects counterbalance its shortcomings.
Right. So the question is, at what point do we start turning auto-autos (I should use that word) loose on the road for live testing? Do we have to wait until they are demonstrably better than human drivers? How do we prove that?
After reading the Atlantic's Waymo article, I'm more convinced that Uber is doomed in this race. They're not investing properly in simulation, and Waymo is getting an enormous amount of programming done based on controlled simulations that Uber isn't - billions of miles, and repeatable scenarios.
Tesla is really in the game, I think, because they're able to collect so much data from the AutoPilot systems currently on the road, even if they're not in use. They still get all the telemetry for complex real-world situations.
But at some point, you have to let experimental software out on the road, to learn from the real world. Simulation is insufficient. Or, we simply refuse to allow auto-autos at all, permanently establishing that humans will always be more capable, without evidence. So it's a question of when, not if.
There's a couple of currents going in the arguments here. Some say "not yet", and some say "never". I think the "never" people are nuts. Auto-autos that broadly outperform human drivers safety-wise seems achievable to me in the not-too-distant future.
My neighborhood is practically littered with white bicycles, chained like ghosts at the sites where bicyclists were killed by cars. So clearly, human drivers aren't perfect at avoiding cyclists either.
Singling out the "previous high profile fatality" is magical thinking. Drawing statistical "evidence" from it is pseudoscience. Meanwhile, a quick googling shows over 800 cyclists killed per year by cars, and nearly 50,000 injured, in the US alone.
Yes, and how many more miles are driven by human drivers in the US compared to autonomous vehicles?
Obviously there is going to be a disparity in the number of deaths. What is important is the number of deaths per mile.
Further complicating matters is the fact that most of the miles being driven by these autonomous vehicles are being done with human operators trained to be alert and the unique quirks of an autonomous vehicle. They will likely be much prepared to stop an accident from occurring when the car makes a mistake than an average consumer.
Autonomous driving may in fact be revolutionary and reduce the number of traffic deaths dramatically, but it is still early enough in development that there isn't any evidence to bank on anything at this point. There certainly isn't enough to rush the tech out the door either on the public either.
Right, and machines could prevent other forms of murder, or help the elderly, or reduce fatal injuries in construction by using robots, etc, etc. It would be cool to get to that stage. Potential != Reality. The reality is that we're far far away from autonomous vehicles operating safely on roads as they are currently designed.
If you want to simply reduce vehicle related deaths, you could address this through various ways like improving road safety/visibility, adding more driver assistance aids. Or heck, design a protocol to disallow cars from starting with a drunk driver or without a seat-belt, etc etc. Maybe people who get into even minor accidents need extra training, etc. etc. There's so many low hanging fruit here..
Or rebuilding America's crippled mass transit infrastructure. Billions are going in subsidies to self-driving tech. We could at least get the largest cities back into mass transit, train, express bus form with late night operations. You allow the blue collar worker to get home safely after drinking for a minimal amount of money and you'll see an improvement in road safety as well as economic growth.
I don't get it - that's so short sighted. It really saddens me. Especially in cities like Nashville, who are not very large or dense, but have the opportunity to grow by a lot in the coming decades. At least having an approved and implemented project in plans would be immensely beneficial to that community.
Disclaimer, I have never been to Nashville, but I've heard about it from people who have been there and did a bit of reading 5 minutes ago.
You'd get more than you think. Seattle's $58 billion ST3 project gets a train from Seattle to Redmond, Edmonds, a west loop and a street car expansion:
Second, yes, they can't come soon enough because traffic is a meat grinder: it's human sacrifice to the gods of speed and convenience.
However, you do not test dangerous industrial equipment in the street with kids and dogs and bikes crossing the freeway, etc...
You just don't do that. It's insane.
A big robot with a couch on it should never travel faster than it can reliably stop quickly enough to prevent any conceivable collision. That's the hard limit. Until we can do that, any machine heavy enough and traveling fast enough to kill is a killbot.
But that said, aren't you holding automation to a different standard than humans? Is it okay to have the big robot with a couch travel quicker than it can stop to prevent any conceivable collision, just because a human is in charge of the brakes? Because that's what cars do now! I can conceive of situations where it's not possible to hit the brakes on time in an ordinary car - hell, I once hit a bicyclist when I was only going about 15mph, because she basically appeared out of nowhere before I (or she) could stop.
> I like the auto-auto term. Trying to use it now.
Cheers!
> But that said, aren't you holding automation to a different standard than humans?
Yes and no.
Humans drive preternaturally well en mass. So much so that when I realized it I began to suspect that "Guardian Angels" might be real. Then I learned more about human unconscious motor neurology. Driving performance is one of the few places where the unconscious mind is blatant. Nevertheless we have set up a system that amounts to a "mayhem lottery". When I say "they can't come soon enough" I really mean it.
Now eventually auto-autos will be super-human in capabilities and performance. They don't have blind-spots, they can use radar and lidar and mass sensors, etc. They will also have the benefit of perfect hindsight across the whole fleet (if we set it up sensibly!) Further, traffic infrastructure will evolve to facilitate automated traffic, with sensors and comms embedded in signals and even the road. They'll be able to share knowledge in real-time of local road conditions and keep track of people and (non-self-driving) vehicles around them cooperatively. Sooner or later, it won't be possible for someone to surprise one of these things.
And all this will happen at an ever-accelerating rate. (Which means that if you just wait, making them gets exponentially easier!)
In the meantime, it's not okay to use public space to test crude early prototypes. People have already died.
I was truly shaken and upset when I read that the Uber car that killed that woman crossing the road with her bike had determined 1.3 seconds before it struck and killed her that it needed to initiate emergency braking, yet the emergency braking system had been disabled because it didn't work well enough. The machine knew it had to stop, it had the capability to stop, but it was prevented from stopping by bad decision-making driven by the desire to put unsafe machinery on public roads.
That's tragic and truly pathetic, and I would say the makers committed murder. Certainly, if you or I built a robot and sent it out into the world and it killed a person we would have to defend our actions with something better than, "Well it's not perfect. Give it some time."
We could make a e.g. self-driving golf cart that could safely take my mother to her doctor's appointments and back. She's got dementia so we don't want her to take the bus even though she's physically able. It would have a top speed of 3~5 mph, and obviously it would have to travel restricted routes where it wouldn't interfere with traffic, and a there are a host of other concerns. But it would work, we could build it today, there's a market for it, and it would have a really hard time killing anyone in a collision. It would be lightweight as well as slow-moving and covered in airbags inside and out with pre-impact detection and triggering.
Once you start to think outside the box (sorry) there's all kinds of scope for incrementally automating traffic.
Trying to go straight to imitating full-on automobiles with human drivers on public roads is insane, it's hubris. And again, people have already died. So this is not a theoretical concern. There's been a funeral man.
> Is it okay to have the big robot with a couch travel quicker than it can stop to prevent any conceivable collision, just because a human is in charge of the brakes? Because that's what cars do now!
It's not okay for robots to kill people just because people do, no.
(I think a sane transportation network would have at least four interlocking but separate systems: pedestrian, bike, car/truck, & rail. Cf. "Pattern Language" C. Alexander. But I'm getting off on a tangent.)
> I can conceive of situations where it's not possible to hit the brakes on time in an ordinary car - hell, I once hit a b...
The first automated driving solutions are, and will continue to be, used in situations where humans are at their best - the easiest driving conditions. So it's misguided to compare their performance to humans in non-perfect conditions. If an AI driver is driving in other than perfect conditions, and there is a crash, apologists can and will say "well, the idiot human should have switched off the AI". The AI is inherently considered better because of mental gymnastics that deny responsibility. This is an example of how humans are exploiting defects in other humans' reasoning, not how AI is superior.
Your "think of the children" argument is pretty weak, really. Humans are pretty good drivers, and if we had more strict licensing requirements/testing and actual severe penalties for DUI we'd probably cut the death rate by 3/4 and not have to use random people for guinea pigs.
> It is completely unethical and immoral to ship unproven tech that has the potential to kill people.
Everything "has the potential to kill people" if you put enough of it in a box and drop it from high enough. The question is how much potential, and compared to what. If we want to use strong moralizing language like this, we could just as easily go the other way: "It is completely unethical and immoral to delay technology that has the potential to save lives." That would be true in exactly the same ways, which is to say, it's complicated and we need to avoid moralizing.
>we could just as easily go the other way: "It is completely unethical and immoral to delay technology that has the potential to save lives."
No, you couldn't "just as easily" go the other way. Those are not identical positions. My position promotes caution over over-confidence. Further more, the "potential to save lives" is not the exclusive domain of autonomous vehicles. So nobody is "delaying" saving peoples lives. Various methods that can be utilized to reduce causalities that do not rely on technology that has not been shown to work in a conclusive fashion. Planes were unproven tech at some point, and people rightfully refused to fly in them, and they aren't as unsafe now. Technology gets better, and we can hope that autonomous vehicles would get better too.
I'm a motorcyclist. I've had the misfortune of flying through the air because someone didn't bother looking before making a left hand turn. The literal mantra of riding a motorcycle in highway traffic is "assume you're completely invisible and that everyone is trying to kill you."
The baseline here really isn't that high. How many miles have you driven, and how how many accidents have you been in?
You are actually the one doing bad math. In statistics, you start with a null hypothesis, which is represented by the status quo (in this case, autonomous vehicles aren't any better than humans in terms of crashing), and use the data to try and prove it wrong.
But in this case, the outcome of interest is quite rare. Despite the high numbers of people dying in car related accidents each year, the number of accidents that occur for the total number of miles driven by US drivers is very small. So in order to draw inference about whether autonomous vehicles improve on that rate, you need a huge amount of miles driven by autonomous vehicles. None of the companies have really approached that level of testing, to say nothing of the fact that most of the miles driven so far aren't on similar, real world conditions that US drivers face, making any statistical conclusions even more difficult.
I'm not doing math at all, except to say that all the people saying "fatalities per mile" and pointing to the Uber accident are misusing math.
I don't believe the Uber accident applies, because it was caused by human error, not the driving automation. The emergency braking was deliberately disabled (it would have prevented the accident), the detection system was not programmed to inform the driver, and the driver was not paying attention. If the Uber accident is disregarded, then we're at 0 fatalities, or infinitely perfect automated driving. Which is, of course, BS. But it's as rational as the argument others are making.
Given the inadequate data set, I'm relying on inference, not experience.
Only if we're separating empirical evidence from inferential evidence.
We can't demonstrate empirically, because there is far too little data at this point (and refusing to do the thing to collect the data because we don't have enough data is not a sound moral argument).
But we can infer a great deal. And I've already established grounds for inference...
- Better sensors. Sensors that can detect 360 degrees at all times, and see into infrared, not to mention LIDAR. The sensors can easily outperform human eyes attached to human heads.
- Better mechanics. Automation can be directly connected to steering and brakes, as opposed to modulating through human hands and feet and into other mechanical linkages.
- Better reflexes. Less lag from observation to action, compared to the eyes/brain/hands system.
- No distraction. Human minds wander.
- No intoxication.
- No sleeping! Falling asleep at the wheel causes many fatal accidents.
- No emotions, no anger, no aggression.
So emotionally and mechanically, autonomous systems are clearly superior to humans. The only real argument left is interpretation of the environment. Can they detect hazards more effectively than humans can? Let's look at some hazard categories...
Lane drift - easily solved.
Running red lights and stop signs - easily solved.
Speeding - easily solved.
Maintaining braking distance in traffic - easily solved.
I could go through more, but you get the idea. We're focusing on "What if a pedestrian steps out in front of it and it doesn't recognize them as a hazard?", and ignoring much more common categories of accident-causing situations, categories where machines easily outperform humans.
So "empirical base"? Sure, no basis. But inferentially, the argument for the superiority of autonomous driving systems is very strong.
We do have data on accidents and human interventions preventing accidents. It just points towards current generation self driving technology being much less safe than the average human driver. Selectively listing things which technology (often fairly simple technology like cruise control which doesn't pretend to have the intelligence to drive your car) is better at than humans doesn't change that, particularly not when it is easy to draw a similar arbitrary list of things which technology is inferior to humans and far from evident that it might catch up in the foreseeable future.
Even if "current generation self driving technology [is] much less safe than the average human driver" - which I will grant is a possibility, although I'm not convinced - that does not at all imply that it is "far from evident that it might catch up in the foreseeable future".
I believe the evidence strongly suggests that autonomous driving will not only catch up, but greatly exceed human drivers in the foreseeable future. Mechanically, automation is superior to humans for simply operating the car. And the large categories of causes for fatal accidents are low-hanging fruit for automation - lane drift, running red lights, driving too fast for conditions, etc. And sensors can observe the environment as well as or better than eyeballs.
The only real point of contention is in interpreting unexpected hazards, like the infamous bicyclist. But interpreting sensor input is subject to continuous improvement, due to programmer experience, log data, and Moore's Law. So even if that point is weak now, there's no reason to believe it will not improve substantially - in fact, it is unreasonable to believe that it will not improve.
It is unreasonable to believe the current technology will not improve in any way. It is also unreasonable to expect people to accept an increased chance of death resulting from relaxation of test safety guidelines because some futurists have essentially blind faith that exponential curves will prevail over diminishing returns in the very near future.
Large categories of potential causes for fatal accidents are things which humans very seldom have issues with but computer programs often struggle with like inferring intent and identifying whether something is animate, which can easily cancel out reductions in other types of accident due to (e.g.) better average lane discipline. Especially when you consider
the benchmark for fatalities per billion km driven is around 3.5[1] including fatalities caused by humans considered unfit to drive, and humans who are considerably less experienced and safety-conscious than the likely early adopters of SDVs. SDVs have already had one fatality and several incidents where human intervention prevented accidents in a few million miles of low speed, good condition driving. I think we're a very long way from seeing any indication computers alone will ever beat that benchmark, still less a plausible case that the additional deaths resulting from riskier testing programmes will be cancelled out by lives saved if the first few SDV models go on sale a few months earlier.
[1]a few European countries; it's double that in the US, which also suggests that if you want to reduce road deaths there's a lot of other steps that can be taken which don't involve risky technology experiments.
So what we're debating is when, rather than if. You seem to accept that, someday, this technology will be superior to human drivers in the general case. But how will we know, without testing in real-world conditions? At what point do we start saying a given program is safe enough to test on actual roads, not just simulations or closed courses?
The alternative is to simply never allow it at all, to assume that humans will always be better drivers than computers. Which opens up an ethical dilemma. If auto-autos could save tens of thousands of lives a year - and it seems at least plausible that it could happen - then to reject it out of hand is to cause tens of thousands of innocent deaths a year.
You're severely armchair architecting here. What happens if sensors fail? How often do sensors need to be replaced? Do sensors behave well under inclement weather conditions (snow is a common source of glare). What if an emergency vehicle flashes their sirens and floods the street with light? There's a lot we don't know, and while there may be satisfying solutions to these problems, we're not there yet. In the meantime, subjecting humans to this insanity due to some "inferential evidence" that doesn't hold water is highly reckless.
Your argument is "Sensors are better than humans!", and that's a non-argument, inferential or otherwise.
Do you seriously think that the much qualified software engineers and other specialists working on these projects are not thinking through all these points?
There's a lot of problems we don't know the solutions to, but that certainly does not mean that these teams don't know the solutions either.
> Do you seriously think that the much qualified software engineers and other specialists working on these projects are not thinking through all these points?
You are commenting in a thread about how a team of those "much qualified software engineers and other specialists" failed to do simulation testing. So yes, there is reason to believe that these companies are failing to address many of these potential problems.
I know a lot of people working in the space who know very well what (some of) the problems are -- but are unable to do anything about it because of organisational politics. :-(
I once had a retinal vein occlusion (basically, a stroke in my retina) while driving. I recognized that my vision had deteriorated and got off the road. Humans have heart attacks, strokes, seizures, and other systemic failures while driving, too.
Detecting sensor failure is child's play. We have over a half-century of experience at detecting and handling sensor failures in computer systems. Hell, I left the gas cap off my car once, and every idiot light in the car flipped out on me. Cruise control stopped working. Etc.
Detecting glare, flashing lights, and other visual interference is likewise well-established technology. And how does the autonomous car respond? By slowing down, staying within safe parameters. Meanwhile, one of the most common causes of fatal accidents is humans driving too fast for conditions, often due to poor visibility. I trust the self-driving car more than the human driver for that.
Right now, per mile, self-driving cars are much less safe than humans. As has been noted before though, we don't have that many miles of data on self-driving cars, and the tech is still quite young and there's (probably) a lot of room for improvement.
Not all self driving car programs were created equal. Tesla and Uber have been acting like cowboys from the start. Most of the advanced lane keeping type setups GM supercruise and Nissan Propilot are exposed to similar risk as Autopilot, but excepting the gong show that is Uber, all the other companies developing L4 have a pretty stellar track record.
And, as you mentioned, we need to accumulate billions of test miles before we'll have statistically comprehensive data.
You can't compare statistics unless the underlying miles are comparable, though. The only miles that humans drive which should be compared are those that a given AI is capable of driving.
The fatality rate for human drivers is 1 death per 141 million km driven (US only, many other industrialized countries are significantly better)[1]
There are currently at least 4 confirmed deaths from autonomous vehicles[2]. While beat is correct in stating that those are not enough deaths to draw reliable conclusions from (and Model S owners are not like the average driver), there would have to be more than 600 million km driven autonomously to be comparable to the death rate of normal cars. Probably significantly more, as the cohort of human drivers that is equivalent to Model S owners is way less often involved in accidents in the first place.
And I say: bring on the utilitarianists. I'm tired of endless studies and safety precautions that different bureaucracies have put into place. For example: I want experimental life-saving drugs to be available to willing patients faster than what the FDA will bother to allow.
Just read about the hoops one has to jump through to run a simple medical study if you still think we don't need a dose of utilitarianism.
The means are the ends. An organization that disregards safety during product development is not going to have the culture required to actually produce a product that improves safety.
> That being said, I still want this tech on the road ASAP, not because it's perfect, but because human drivers are So. Fucking. Awful.
And yet, so far, Uber's self-driving vehicles have a higher fatality rate than the status quo.
Sure, it could be improved over time, but the whole point of TFA is that Uber is currently not doing the requisite diligence and testing with their vehicles to ensure safety.
I wonder how Uber's fatality rate compares to other high risk groups. For example, I suspect teenagers or new drivers in general have a higher fatality rate. And yet obviously new/young drivers must be allowed to operate vehicles on the open road. Would your opinion change if, for example, Uber's fatality rate was better than $worst_category_of_current_drivers?
Do you get to decide this? No. Does Uber get to decide this? No. Is this attitude going to produce a backlash against the idea of the technology, similar to the GMO backlash? Probably.
But he's not deciding it - he's merely stating his opinion. Uber apparently did decide this earlier by taking the shortcuts they did.
I also agree with GP. We need to take more _calculated_ risks instead of trying for strictly risk-free science. Personally I'll gladly accept more dead astronauts (and hail them as heroes) if they push spaceflight further.
Me too, and the astronauts make that choice themselves knowing their butts are on the line... we don't have some "move fast" CEO grab random people off the street and shove them into poorly built and untested rockets.
The driver had actually been road-raging with another driver when he hit the bicyclist. He wasn't directing his anger at the cyclist. (edit: He had also left a bar, and may have been drunk, but he was not connected to the hit-and-run for days, so there was no evidence to charge him with drunk driving.)
This doesn't seem like a huge leap to me at all. If he'd been driving responsibly, he probably would not have hit the cyclist. And autonomous drivers don't get angry or careless. Your follow-on about imaginary scenarios kind of underscores the weakness of that argument.
And I do agree that a raging driver taking control from the automation continues to be a problem. But that's no worse than the original situation, now is it?
Well, yes, it's easy to attack my "argument" when you didn't originally give the full details.
Bugs are not imaginary scenarios. Also, did the US outlaw gun ownership without the rest of the world noticing?
> And I do agree that a raging driver taking control from the automation continues to be a problem. But that's no worse than the original situation, now is it?
I only replied to one point: that you cannot state with 100% conviction that the cyclist wouldn't have died had there been a self-driving vehicle.
You could back down and just concede you really meant to add at least one condition: "if that jackass had a completely autonomous vehicle, with no manual input possible, it is highly unlikely the cyclist would have been killed"
I'll concede to the point of "If the car had a good quality collision detection system, and the jackass did not deliberately disable it". You don't need "no manual input possible" for this. You just need a system smart enough to apply the brakes when the driver doesn't.
I was out car shopping recently, and even affordable midrange cars (Subaru Crossfire and Honda HRV) offer technology that could have done this, as a standard feature.
> So are big corporations going to do kind of a careless job testing live vehicles in a rush to market? Yes. Will people get hurt? Yes. Is it worth it? Yes.
There is a big difference between humans killing others as one off incidents vs one bug, defect or security flaw making a crash possible with the whole fleet. It can be far more catastrophic.
I'm actually far more afraid of a security flaw than a defect. Security flaws could be deliberately exploited by terrorists or other bad actors. That's a really serious concern.
On the other hand, that's the kind of thing that gets addressed by brute-force improvement. Security is a well-understood field, and the main problems we've seen with car security so far have been caused by insufficient effort, not insufficient technology.
> Plus, the additional sensors available and the software response time mean autonomous drivers can outperform even sober, attentive, skilled human drivers in most situations.
[citation needed]
All the current evidence is that humans grossly outperform computers at the task of "driving vehicles to arbitrary locations in all conditions", and that that isn't changing any time soon. Like, it isn't close.
You can talk about your speculative perfect-performance car if you like, but there's no evidence that it will ever exist. Why not give it hyperdrive too?
I'm not talking about perfect performance. I'm talking about "superior to humans in the general case* performance. And there is plenty of evidence that it will exist, starting with the huge number of things that humans used to do that are now done by computers, faster and more accurately.
Driving vehicles to all locations in all conditions? Yeah, humans will have advantages there for a long time to come. Driving on highways and city streets within the bounds of traffic laws and expected velocities? I think computers will be better than humans very soon, if they aren't already so.
> Plus, the additional sensors available and the software response time mean autonomous drivers can outperform even sober, attentive, skilled human drivers in most situations.
Self driving cars are a hard problem, a very hard problem. So is CAPTCHA and machines are still not that great at solving them.
The problem space of autonomus vehicles is a large one, and even with how many mistakes human drivers make, I still don't think we're anywhere near the tech to make safe autonomous driving vehicles. We're at a minimum, 15 years off.
I want you to think about all the technology we use every day. Our phones, desktops, laptops, are full of bugs. Hell I found bugs in the ticketing system I buy metro cards from the other day. We see display kiosk crash all the time.
A Tesla car with lane assistance (please stop calling it auto-pilot) recently drove a car into barrier. Another kept driving for hours without hands on the wheel (fixed in an update. Was that driver passed out? No way to know.)
Now you might say, this is different. The engineers working on self-driving tech are at the level of the ones that build bridges, or make pacemaker software (which by the way, some were found to have a bluetooth vulnerability recently) or plane avionics.
Maybe, but Uber certainly isn't. They're a culture of scum shit shop. This isn't a small thing here. It's a huge, huge problem space that no one completely understands yet. Why are there not any self-driving cars on the streets of NYC or Chicago? Because it's difficult, and dangerous and if there were, people would be dead right now.
For all the money we're investing in self driving cars, we could build tech that already works. Singapore and London both have fully autonomous trains; some that arrive at 30 second intervals and carry millions of people a day. America needs to get its normal infrastructure back before we go down this fantasy world of self driving cars.
> I want you to think about all the technology we use every day. Our phones, desktops, laptops, are full of bugs. Hell I found bugs in the ticketing system I buy metro cards from the other day. We see display kiosk crash all the time.
These arguments don't make a lot of sense to me:
- Do you think display kiosks would still crash all the time if the industry spent a few hundred millions at improving them?
- Do you think the metro ticketing system would have as many bugs if the transit agency spent a billion on it?
If not, why are you comparing very different situations?
Maybe we need to use average number of bugs per amount spent on dev instead of average number of bugs per project or per # lines of code
We as a society just need to focus on getting out of this local minima that we are stuck in. It is clear to any of us here on HN that autonomous cars will result in fewer deaths, I have no doubts about that whatsoever, humans are statistically terrible drivers. It will only be when the data becomes clear that autonomous vehicles (Uber at the bottom, Waymo at the top currently) are better than humans, that naysayers will not be able to fight this. There is not enough data at this point to fight naysayers. The argument will necessarily proceed to the point where "who is responsible" becomes less important than "reducing vehicular deaths". Since big money is driving all of this, the conversation is still about financial liability, the conversation needs to move past that. In 5-10 years time, we should be able to judge just based on statistics:
For the USA:
37,000 vehicle deaths / year
2.35 million injuries / year
1,600 child deaths / year
8,000 killed in crashes involving drivers ages 16-20
$230.6 billion per year in crash associated costs
If any company can show that these statistics can be reduced (halved/thirded/decimated) in any meaningful way with their autonomous tech, that's when we'll finally reach a new era in transportation. How can anyone advocate for human-driven cars once the implications become clear? Let's say that the Uber autonomous car reduces vehicular injuries and deaths by only 5%, well shouldn't that be enough evidence to say, put that on our roads? I think that the current level of tech is already at a point where it will save lives on average more than a human driver, what more do you need from a regulatory point of view?
The numbers can be dramatically reduced by better safety standards for roads, vehicles, and better training (stricter driving tests, better control of distracted and drunken driving).
Take the UK for example. It has many of these safety improvements, and gets one quarter of the risk of dying in a car accident (one half the risk per mile driven, since Americans drive further).
"We as a society just need to focus on getting out of this local minima that we are stuck in. It is clear to any of us here on HN that autonomous cars will result in fewer deaths, I have no doubts about that whatsoever, humans are statistically terrible drivers."
Getting out of a "local minimum" by definition requires things to get worse before they get better - that's what makes it a local minimum. So it requires faith, which is belief in the absence of clear evidence. Faith is always uncertain, which is why people get strident and appeal to emotion when it is challenged.
> It is clear to any of us here on HN that autonomous cars will result in fewer deaths, I have no doubts about that whatsoever, humans are statistically terrible drivers.
Not really. We have a little bit of data on how mostly automated cars handle driving on controlled access roads in good conditions, and tons of data on what humans end up doing on all types of roads in all types of conditions.
Wait - this presumably happened in the USA, and a car which is primary evidence in a murder investigation could not be located?
This is despite automatic numberplate recognition on most freeways, many cars having trackable cell/wifi/bluetooth connections, automated RFID identifiers for toll roads, every car having a unique VIN which is printed on nearly all components and will be registered by scrapyards, and tracking on all mobile phones (presumably the driver had a phone!).
It's probably in a lake somewhere. I live in Minnesota, which is called the "Land of 10,000 Lakes". Actually, it's closer to 20,000, but we're modest. :)
What's known is that the driver, after hitting the cyclist, drove to his mother's workplace, and she left with him. The car he was driving has not been seen since. If it was driven into a remote lake somewhere, it will probably never be found.
Ground scanning radar can look into lakes down to about 60 feet deep and find metal objects rather easily.
One would think a murder investigation would (a) know this, and (b) have the ability to get data from a military or private satellite, both before and after the suspected incident.
Failing that, tracking data from the pairs phones would likely indicate which lake, and probably which part of the lake even.
Maybe. But that sort of thing is expensive for a court case, especially when there is sufficient evidence for conviction already.
The driver was convicted of vehicular homicide (max five years), and leaving the scene of an accident. He can't be forced to testify about the location of the car (Fifth Amendment), and I don't know if he was charged with obstruction of justice. His mother was charged with obstruction-type charges, and her case hasn't been tried yet.
It's very generous of you to sacrifice other people's family members in service of your vision of a new, more perfect society. But if you were all that serious about your "killing people saves lives" vision you'd be out there walking back and forth in front of Uber cars on their test tracks.
But since you aren't, I'm guessing even you don't really believe what you're saying. Which is good, because a key assumption of your theory is proven wrong by the article you're commenting on.
Yes, these vehicles will eventually save lives once their error rate drops below that of a human driver. But there's no reason to think that running over a bunch of real people is necessary for progress until these cars work well in simulations. As the article explains, they get orders of magnitude more testing done in simulation than on the road.
Uber's rush to test things in the real world was not about advancing the technology or saving lives. It was about getting a short-term commercial advantage on the cheap.
And likewise? Your spirited defense of the status quo also sacrifices innocent lives. Over 30,000 Americans die in traffic accidents each year. Just how much better do autonomous cars need to be in order to be acceptable to test in the real world?
The data and experience gathered from real-world testing should (I'm not saying "will") accelerate the progress to the point you mention - "once their error rate drops below that of a human driver".
Let's look at the Uber case, since everyone infers so much from it. One fatality. One. And the program was immediately shut down. And how did it happen? Engineers had deliberately disabled the automated emergency braking system to reduce "erratic behavior", but had no automated notification to the driver. And the driver was distracted, doing data entry for the test as he was instructed to do. If the emergency braking system had been on, it would have stopped the car on time - they have the data for this.
This sure looks like human error to me.
And that single accident put Uber's program drastically behind schedule, relative to more cautious/thorough competitors. Uber paid a market price for its eager carelessness.
This one incident is not the cause of Uber being behind schedule. They shut down their program in California because they didn't want to follow the rules. So Uber moves to a state with lax rules and a woman paid for it with her life.
If Uber was more focused on doing things the right way their program would be much further ahead.
To test in the real world, they need to be at least as good unless we have strong reason to believe that surplus deaths are the only way to move forward.
We don't have reason to believe that. Even Uber now agrees that more simulation should have been done, and they're trying to catch up. All the necessary data for this could have been gleaned from instrumenting human-driven cars, and then testing the object recognition, object avoidance, and other driving algorithms against the resulting data.
Uber's Arizona death wasn't "human error". It was negligence. They also weren't "behind schedule" in any meaningful sense; some exec picked a too-aggressive goal, and people cut corners in a vain attempt to meet it. They shouldn't just be paying a "market price". They should be facing charges for criminal negligence leading to involuntary manslaughter.
>Yes, these vehicles will eventually save lives once their error rate drops below that of a human driver.
Aren't self driving error rates already below human drivers? The fatality rate is extremely skewed by one incident in which the system detected an obstacle but wasn't allowed to stop the car because it was disabled (so... not a failure of the self driving aspect of that system) and another in which a partial self driving system warned a driver repeatedly to take the wheel, but they did not.
In both instances the problem was a failure to force action by the human drivers (or failed reliance on them). In the former case, the system would have braked, in the second, the system should brake instead of asking for help from the driver.
Meanwhile the overall crash rate for self-driving cars is lower, correct?
> Meanwhile the overall crash rate for self-driving cars is lower, correct?
I don't think that's correct in general, and especially not so if you compare the conditions that each are driven under. Said a bit differently, the human error rate is in all conditions (rain / snow / nighttime / unmarked roads / gravel) where most self-driving miles are on well-marked city and highway roads in near ideal conditions.
The Uber system would not have braked (and didn't) the system that would have braked was the Volvo system built into the car which Uber purposefully shut off. If Uber cant drive on a clear dry night without killing people they shouldn't be allowed on public roads endagering people who didn't sign up to be around these cars.
I'd be interested to see your stats comparing the two crash rates. Note that you have to compare equal conditions; most self-driving tests are done in unrealistically good ones. (Which is fine for pilot programs, but makes the stats hard to compare.)
But you're already no-true-scotsmanning your way to giving the self-driving cars unrealistically good scores. Yes, a cause behind the Uber crash is indeed Uber disabling a safety system they thought unnecessary. But we have no reason to believe some other companies aren't also either turning off or just never adding safety systems.
On the other hand, if we're worried about the performance of these systems, shouldn't we be testing them in ideal conditions before submitting them to suboptimal conditions?
Yes, which is why I explicitly said it was fine for pilot programs.
My objection is not to better testing. It's to bad comparisons. Including, as here, with somebody handwaving at data that, at best, couldn't answer the question.
Or, you could actually make it harder to pass the driving license test which is a joke compared to many other countries, pass laws making it mandatory to insert speed control software into all vehicles and alcohol test machines (and also dramatically reduce the limit). And of course, create safer roads. But hey, why not kill a few people instead of these unpopular but proven options (compare with Sweden for example).
The right question is not "why did Uber do this?", but rather, "what was it you thought would stop them from doing this?" Current answer: nothing. But hopefully, the attitude of governments towards letting a company like Uber test fast-moving heavy machinery on their citizenry, is changing. Different industries have different attitudes towards innovation and risk, and there are good reasons for that difference.
It’s not about learning, it’s about allowing. Some companies like Uber and Tesla will do whatever they can get away with, for advantage in the market and ultimately money. The aerospace industry is safe because it is massively regulated.
A lot of the carriers and manufacturers already have a very strong safety culture. The regulator spends most of its time working with smaller companies.
That safety culture wasn’t born de novo, it was legislated into being after years of fighting with them. Remember Ralph Nader? The companies that now have a “strong safety culture” were the same that fought against seatbelts for Christ’s sake. Airlines had to experience a decade+ of terror attacks from rank amateurs, Pan Am had to die before basic security screening was accepted.
Most of the automotive industry already knows how to do things safely. It's really just companies like Uber and Tesla that either don't know or don't care.
Aerospace and defence seem to have a much stronger notion of systems engineering than I experienced in Automotive -- but it is entirely possible that what I experienced was an outlier, not representative of the whole.
There is also likely difficulty in testing an ML black-box. When I took the ML class (Stanford's, online), they actually recommended setting aside some percentage of the training data as test data, but this is (IMHO) sub-par because it doesn't eliminate the possibility of things like overfitting
Testing in Production. It still boggles my mind that that's the norm in software engineering these days. Interesting how in this case it seems "inappropriate" in real life, and the article fails to make the connection how normal this is in the software world nowadays unfortunately.
Not sure if this is possible, but couldn't they capture all the LIDAR inputs from a car being driven by a person, then feed that into the software to see if it actually recognizing objects correctly?
Or even do it on a closed track. That way you could throw all sorts of things at it like kids, bikes, garbage cans, whatever.
Should be able to see what works and what doesn't work pretty quickly.
EDIT: Never mind! That's what Waymo does in the article another person posted.
When it comes to "closed track" and other "simulation" style things, you need to be very careful when ML is involved.
I don't know if Uber/Waymo/Others have "solved" this problem, but ML systems have a tendency to "learn" the wrong thing. You could run millions of simulations (both virtual or on a closed course) with "test obstacles", but the second the car encounters one a child with red balloons in the early morning on a foggy day it might not know what to do with it because that exact scenario wasn't in any simulations.
Exactly. Throw something at it the system should stop for (e.g. a child). If it doesn't, go back to the drawing board and figure out why.
Agreed the test track is not ideal, that's why I was wondering if they could throw real recorded data at it. No risk to the public, but puts the system through real-world trials.
100% agreed. Somehow large corporations get away with a diffusion of responsibility. Enough people are involved that it's hard to blame one person. But certainly for issues resulting in death, I don't think responsibility should be diluted.
If 10 different executives knew or should have known that their decision had life-critical implications, then I think all of them should face the appropriate penalty. (Here, I presume it's for involuntary manslaughter due to criminal negligence.) And I think the same penalty should apply to developers who knew or should have known that their actions put lives at risk.
I never understood why the starting point for these projects was not simply to drive a car in GTA V (or some other game) around the city. It would test pedestrian detection, street signs, traffic lights, merging, emergency vehicles, etc... While certainly not perfect, it would seems like a good starting point in a controlled environment that is already modeled on the real world.
Qasar here (x-YC founder and partner). So im building a simulation company. Not commenting on Uber specifically but the reason lots of companies don't have great efforts is simulation is actually really hard. It’s not building a web application (tested methodologies, established best practices, lots of experienced engineers in the space...etc).
You can get a lot done (and get wonderful demos) through research vehicles. The issue with sim is similar to the issues with AVs as a whole - getting to 90% is much easier than getting to 99.9999%.
Also, there is this weird thing in AV (and specifically in-house sim) where pointy-haired managers think "if we put 30 SWEs on the problem - we'll make progress!". While raw eng horsepower matters in some software problems, when you are on the line between research and engineering, more SWEs might actually make getting a robust product up harder.
Another thing we've seen is in-house efforts overfit solutions. They don't really have a wide viewpoint (nature of how secretive the space is), and therefore go down paths which are just plain wrong but no one is there to challenge them.
I always assumed people would integrate an external simulator like CarMaker rather than build their own.
I'd be exceedingly interested in hearing about your product when it comes available (although my interest is in simulation for maritime autonomy rather than automotive).
We're super familiar with CarMaker -- great for dynamics and some level 2 stuff but there are some big difference between autonomy levels (from simulation perspective at least). At higher levels, autonomy algo development is more like modern software development. CarMarker is a fantastic product (along with CarSim, dSpace ASM...etc) but they were built for a different use cases. A very crude analogy would be tools for J2MEE vs modern tools for iOS/Android. Our product is available, just not very public about our more detailed approaches because that is really where our IP is. Feel free to reach out if you want to chat. Obv, i'm super interested in the topic.
Nobody (well, theoretically someone, I'd like to be proven wrong) has a self-driving algorithm which is formally proven. If we can't get, to use your example, web engineers to formally prove their web applications, when they have all these structural advantages, then how can we expect something unproven (har, har) to be formally proven? So there needs to be an extensive QA test suite.
And for all of these other relatively-mature engineering sectors like web engineering, how many of these companies are really investing in QA? In strong and comprehensive test suites, in security reviews, in performance testing and UX scoring before release? Almost nobody, right? I mean, the only reason why society can get civil and biomedical engineers to submit to strict quality control is through regulation, but no regulatory body has any real idea of how to apply that to software. And autonomous driving systems are supposed to be different?
Mythical Man-Month came out 38 years ago. The Pareto Principle, not so much as being explicitly a 80/20 split but about how the vast majority of the effort is needed to achieve actual completion rather than just a proof of concept, was originally coined in 1896. Suddenly it doesn't apply to Uber?
Is it really so surprising that it's so difficult to find senior practioners in these areas when few companies are committed to that level of quality, and the companies which are tend to be so successful that they can afford to pay compensation at rates so far above the rest of the market so as to ensure that few of their senior engineers ever leave?
The reason why there isn't perfect QA for things like webapps is a cost reason, not a knowledge reason.
Society usually doesn't need a formally proven cat pics sharing app, and most web apps, including facebook, are that level of application. They would rather pay a cheaper price than to have a perfect expensive thing.
You'll also notice that as organizations get bigger, they do invest more in testing, QA and regression detection.
A counter example is in the aerospace industry. Most software has to be d-179b certified, which requires not only testing with complete code coverage, but also complete branch coverage. Every possible outcome in every “if” statement must be tested.
Needless to say, this is expensive. I’ve heard estimates that it costs roughly $1000/line of code.
The point is that there are methods to increase reliability, but there isn’t the financial incentive to do so.
Separate point, I’d argue that small organizations also have quite a bit to gain from a solid testing program.
If there is only a single developer working on a project, that developer can manage much much more code if there are solid tests behind it than if she was just winging it with every commit.
At some point the developer starts to forget the code they write and starts relying on the tests they built long ago.
Agreed. AV regulation is early and we're involved in that conversation. We (and I think many in this business) deeply care about safety and believe better technical understanding among lawmakers around software safety will result in better laws. Re: mythical man-month and pareto - 100% true for all AV makers but theory and practice is different. I imagine both their investors and their perception of market forces drive decisions to some degree. This is where you see some gaps (not always) between traditional software lineage companies and traditional OEMs.
297 comments
[ 4.8 ms ] story [ 262 ms ] threadThe car saw the obstacle and detected a collision, but there was no warning system to alert the driver about it.
Meanwhile, the driver was looking at a laptop and taking notes on the system logs (which they were told to do), not watching the road.
Like, let's say I had a grammar check. But the grammar check was bad. It flags every other sentence as needing a grammar change, anything other than a very simple "subject-verb-object" sentence structure looks wrong to it. For every correct grammar problem it identifies, it finds 20 incorrect grammar problems.
As a result, I turn off the UI for the grammar check.
Then I write a document that has a grammar problem.
It seems stupid to say that this is either a failure of my use of the software, or a UI failure of the software. The problem is that the fucking software doesn't work, specifically that it fails in a way that makes it too "noisy" to use.
And sure, if my job is to catch grammar errors, then it's also my fault. I'm supposed to be backing up the system. But we also know that I won't be perfect and will have failures of attention sometimes. But regardless of that, the grammar checker just didn't work.
And, in this analogy, bad grammar kills people.
But the failure that killed Elaine Herzberg happened well before then, when the decision to disable the emergency brakes was made. It appears they disabled other safeties as well.
I don't disagree that playing a little loose with allowing new tech makes sense. We need to understand what we don't understand about them, accidents of course will happen, etc. But this looks a lot like foreseeable negligent homicide to me.
This precludes the necessary conversations about test interfaces, for example, making the task of the simulation team closer to one of reverse-engineering than of simulator integration.
Sadly, sometimes this means saying goodbye to your colleagues and friends, and looking for new adventures in an organisation that is capable of acting in a more constructive and mature manner.
The only difference between disabling the emergency braking mechanism and, say, intentionally disabling safeties on industrial machinery is the novelty. I think it was pretty obvious that something like this would happen, and that's why so many people have been making noise about it. Now that we know for a fact that Uber is willing to make this choice, there's (a) no excuse for not coming down on them for it, (b) no excuse for Uber not completely overhauling their safety protocols with the help of an external ethics and legal advisors, and (c) no excuse for engineers working on this for not at least seriously thinking about their own ethics and possible moral culpability.
Some people want to focus on the future. Self driving cars will be great! Gotta break some eggs to make omelets! Fine. But those who are killed before that bright future deserve some accountability, those who don't want to be killed deserve some respect, and the makers of said omelets can bloody well be expected to be held responsible for their actions.
I would think another big reason though is that the handling and response of such a car would be very different from a real car. Stopping distances, acceleration, cornering, wind behavior, would all be different.
It seems to me the real problem would be that A: You need the car to be on the real streets, so at a minimum, it must be street legal. B: You need it to be on all the streets, including highways and such, so we're at least up to a Smart car level of body. From there it really isn't that great a leap to a full sedan chassis. Would one of those little cars even have room for all the requisite sensors? Even if they did, sedans are just so much more useful.
So it seems like the minimum practical car to test with is a sedan, and that's enough to kill people. Even the little Smart cars are plenty enough to kill people.
On slow roads, I've often seen them driven around summer communities. It would be silly for the government to not allow this.
> They would not be safe for passengers.
The above commenter pointed out "enough trust for 2 tons of metal and a passenger," so I'm assuming the idea is that they don't have a passenger in them at this stage. Additionally, regardless of whether an SUV is safer for the individual behind the wheel, shouldn't it be that individual accepting the risk and not a random pedestrian who didn't consent to this experiment?
> They would be unable to carry the necessary payloads.
Do they really carry payloads at this stage? Isn't the focus on building a safe autonomous system?
> Most of the weight of a car is in the chassis anyway, not the body.
The golf carts are 1/8th to 1/4th the weight of a crossover, for example.
The substantial issue I'd guess would be the differences in all aspects of performance between a real car and a golf cart, but at this stage companies like Uber should clearly just be testing detection of pedestrians et al.
https://www.youtube.com/watch?v=HJnjuhBIhwc
That stage of testing is past, though.
It seems reasonable that there should be some regulation on "x miles per incident" before autonomous vehicle companies are allowed to move forward from carts to full-sized cars.
Based on the stats that have been posted on HN, Google is very safe. Uber, on the other hand, is not.
[1] 3000 deaths per day is 1.25 million annually, from https://en.m.wikipedia.org/wiki/List_of_countries_by_traffic...
Edit: Uber’s program is a fiasco wrapped around a joke, but Waymo has taken an extremely careful approach and it’s a mistake to conflate the two.
https://en.wikipedia.org/wiki/Epidemiology_of_motor_vehicle_...
> This makes motor vehicle collisions the leading cause of death among young adults of 15–29 years of age (360,000 die a year) and the ninth cause of death for all ages worldwide.
> It is estimated that motor vehicle collisions caused the deaths of around 60 million people during the 20th century,[6] around the same as the number of World War II casualties.
Approximately 1% of all deaths are in car incidents.
Why? Isn't the entire premise of a fully autonomous vehicle that you can get wasted and still hop in your own car and get home safely?
- the rate of traffic fatalities are unacceptably high
- the benefits of automobile usage far outweigh the costs, even with today's fatality rate.
The changes necessary to get fatalities completely down to zero would likely increase costs and reduce benefits too much. But reducing fatalities by an order of magnitude? Very feasible IMO.
This is by no means proven, and evidence suggests that it is wrong.
Cardiovascular disease is the number one cause of death, but I would still like some testing to be done on possible preventative measures before they're given to patients.
Don't forget the feedback loops. People use cars because they need cars, but they need cars precisely because they use them.
Have you ever left the city for a destination which is not another city, or a suburb?
Sure, if space ships were impossible or too expensive to even consider, we wouldn't be planning to visit Mars; but it isn't equally valid to not visit Mars altogether as an alternative to visiting Mars, just because going to Mars is harder than going to the corner store.
> Have you ever left the city for a destination which is not another city, or a suburb?
Sure. My point is not about vacations, or Mars trips, but precisely about regular commute. As more people started having long-range vehicles, the landscape of work and living spaces changed so that now people are required to have such vehicles, and if you don't have one, you're significantly disadvantaged on the job and residential markets. The same thing happened with cellphones, bank accounts, plastic cards - cool tools at first, and then soon requirements for living in modern society. I'm just pointing out this phenomenon for consideration.
I'll set aside for a moment how you seem to think that the only reason to leave the city is for a vacation. My point is that there is a tremendous benefit to people being able to choose precisely where they want to live and work, and that the car is the means to that end with the fewest limitations. More people could probably take mass routes in larger vehicles, especially if bicycle racks became larger and common enough to rely on, but that won't diminish the value of the car, even for a commute. Cities also seem dead-set on installing trams and subways, when buses clearly offer the best bang for the buck (but nobody likes to say that they ride the bus, probably because buses are unloved).
Personally I think the more effective route is to encourage less commuting altogether, which would be an obvious natural effect if the roads were on the market, but in practice it would have to take the form of a "credit". Some companies (e.g. Microsoft) have company shuttles, those could be encouraged as well (since roads are not on the market, and thus do not encourage pooling).
2. I understand that Google does the vast majority of its testing without any iron at all, but rather tests new software using recorded input.
https://www.google.com/search?q=google+self+driving+bubble+c...
- For the perceived feeling of safety in larger cars.
- For the off chance that once a year they might need to take some extra large load (e.g. help a friend with moving).
AKA. emotional needs, not practical ones.
Carpooling is not a meaningful comparison. Carpooling requires picking up (and tolerating) other people on a semi-rigid schedule. Hailing a self-driving car has all the benefits of driving solo, plus you don't have to drive.
I get that SUV's aren't for everyone. Acting like they are for no one is silly though.
Though to be honest, me & my SO are driving a compact MPV which is larger than our basic needs (no children so far) - we were wondering whether to get something smaller, but it was the best used vehicle available within the budget at the time. And we frequently find use for the extra interior space.
I grew up where it snowed a lot. I've never owned anything that sat particularly high nor with 4WD, and I've never missed a day of work due to snow.
My street floods.
If the water is above the air intake of a Honda Civic, your SUV isn't going to do you any good.
I frequently want to carry a large number of items.
I'm continually shocked at how few SUVs exceed the cargo volume of our 35mpg Scion xB. I've moved furniture for a friend because his Equinox wouldn't carry it. I rented a Nissan Xterra to carry a bicycle box that fit less snugly in the Scion than it did in the Nissan. SUVs might have a use, but carrying things is not one of them.
Until you threw in the ol' "I have a family" (because the children can not be seen in a Volvo wagon), I thought you were doing a parody of "Excuses SUV Owners Use to Justify Their Big-Ass Vehicle". I'm pretty sure I was wrong.
That's like saying you don't need a fullsize truck because the methheads seem to do just find hauling scrap metal in their Dodge Caravan.
For a load with a constant amount of space usage (e.g. kids + sports stuff) having more space will make hauling that load less of a chore. If you do it regularly then the luxury of a bigger vehicle is often considered worth the cost.
I can and have hauled fridges, office desks, etc on the roof of my wagon. I always prefer to use my truck for those things unless the item can't stand getting dirty from the truck or I'm going so far that the extra fuel economy and better driving experience of the car makes it worth it. Strapping a desk to the roof of a car and not doing a crap job is a chore. Just tossing it in a truck and strapping it down takes like 30sec.
A vehicle with more space can haul more crap or haul the same crap more easily.
As an aside, I wouldn't let my kids be seen in a Volvo Wagon. I'm white trash, not some hipster. A rusted out Subaru is more my style.
That's like saying that one kinda, umm, perhaps didn't completely comprehend my point. Unless you're arguing that a Dodge Caravan has more space than a Ford F-150 (and maybe it does, dunno).
having more space will make hauling that load less of a chore.
Yes, I completely agree, that's why I like the Scion over an SUV.
As an aside, I wouldn't let my kids be seen in a Volvo Wagon.
So you're worried about the image you project as well. Fair enough, I think that might apply to many vehicle owners including, to a degree larger than I care to admit, myself.
The classic SUV, in other words a truck chassis, is an ever shrinking part of the market. FJ Cruiser, Xterra, 4Runner...
Also, both the FJ Cruiser and Xterra have been discontinued, which further emphasizes that "traditional" SUVs hardly exist these days.
But my overall point is that I believe for most people (myself included), choice of a car is atypical in a way in which it includes emotions instead of pragmatism.
This sounds like something a caucasian single person would say.
Anyone with small kids would immediately have to think about the logistics of child car seats plus a stroller/wagon (at a minimum), plus whatever else would be appropriate to bring to the target destination (e.g. chairs and a cooler box for a picnic). Those are certainly not thing people do in an off-change.
Also don't forget cultures with tight-knit families (e.g. indians) where it's not uncommon for grandparents to live with you (and by extension travel with you) as well.
My theory is the increasing car sizes are driven by increasing waist sizes; a 220 lb couple could be cramped in a car their 140 lb grandparents might have found plenty large enough.
[1] The average family is 3.14 persons in the US in 2017 according to https://www.statista.com/statistics/183657/average-size-of-a...
WTF? White people don't have trucks (or SUVs) and families?
Of course caucasians buy SUVs too...
Source: I drive a small car in Oakland.
I suppose this could be solvable with proper road infrastructure design and actually punishing people for bad parking.
It'd be really nice to have a golf-cart-ish car, a little SmartCar, a proper sportscar, an SUV, and a big pickup truck, and use the appropriate one in each situation.
But, even people that can afford the spare cash for that kind of stable of cars rarely do it -- it's just too inconvenient, too much maintenance (or too big a garage to keep them all in to minimize maintenance), depreciation, insurance, etc. and nevermind the constraints of city or near-city living where space is at a premium.
So, people just get the closest thing that fits most of their use-cases, and that's most often SUVs and crossovers.
I'm and example; while I'd be happiest in a small sportscar and it'd work for 85+% of my trips (and save fuel money), my occasional and unpredictable need to carry big packages for work keeps me using an SUV for my daily driver. Even if I wanted the extra expenses, I have winters and no spare garage. So...
But who cares? It still seems like it would be valuable to collect many millions of simulation hours and eventual live testing hours with far safer vehicle types first, at minimum just as a proof of concept and a prove-out of the appropriate testing and evaluation infrastructure (e.g. we’ll probably be a lot better at extending the audit process that evaluates autonomous golf carts into an audit process for common cars than we would be just jumping straight to auditing cars without ever having audited anything else).
In fact, it seems irresponsible to not prove out the entire end to end ecosystem (from simulations to live testing to auditing to a commercially approved deployment in some limited scenario) using a much smaller scale application and vehicle type.
I can understand the arms race motivation for cutting corners and jumping straight to the widest use cases on the part of corporations. But I can’t see why society would want it handled that way.
Because these self driving cars aren't being developed to solve a hypothetical problem, they're being developed to solve the very real and current problem of cars killing 40,000 people in the US each year.
If extra safety measures result in killing ten fewer people in testing, but delay the introduction of self-driving cars by a year or two, that could mean thousands of preventable deaths still occurred. Are the lives of those people definitely getting killed right now worth 100x less than the lives of those who might get killed with these tests?
And this isn't baseless speculation. People are already obviously dying all the time, and improving these cars' software would be much harder without the kind of real-world data they're currently collecting.
Possible, but speculative.
> Also, estimating how safe sekf-driving car is you should take into account all the cases where human took over.
Sure, but on the other hand, the SDC's are continually improving as a group in a way that is not true for humans. This makes it harder to extrapolate into the future one way or the other.
> Also there are many accidents in weather conditions where self driving car would not be able to drive at all.
If you mean environments where humans also can't drive worth a damn, yes, probably true, although SDC's may yet prove better, since that's a technical problem rather than a social understanding one.
But 90%+ of accidents involve human error, so even if you can't completely eliminate deaths, you can still radically reduce them.
Yes, exactly. This is the reason you would want the technology to be developed slowly and carefully, incubated in a safer scenario for a long time to ensure it can be effectively audited and deployed when it comes time to use it for the intended, safety-improving commercial traffic use cases.
What you sounds sounds like good reason to more strongly favor a lengthy test and prove-out cycle on safer and more limited vehicle types, long before testing on commercial traffic vehicles.
The rate of auto-related deaths happening today certainly would not be a reason to hastily rush the technology and jump directly to extensive testing on the final commercial traffic end use cases. Trying to "hurry up" and prevent those deaths could counter-intuitively lead to more deaths because the infrastructure and auditing processes were not proved out and slowly ramped up to real traffic scenarios.
You assert this, but provide no reasoning. I mean, the status quo is "hurrying up", since there are no general standards companies are being held to, and yet only one company has killed anyone, and surprise surprise, it's the company with an extensive history of ethical problems, and they screwed up in a way that's really obvious and foreseeable.
The status quo is a free for all, and yet we don't have much in the way of deaths and injuries from the millions of miles being driven by computers. It seems like companies are already themselves being fairly safe, so how many deaths would you actually be avoiding by implementing more stringent standards right now?
In many industries, from food development to drug development to medical devices to clean energy tech to child car seats to film ratings we have nuanced and highly involved government oversight and regulations to try to ensure levels of safety, ethical and thorough clinical testing, ratings systems that map to outcomes, etc. etc. Yes, of course they are not always perfect, but that is beside the point entirely. The ubiquitous and widespread use of such regulations is a direct statement that companies cannot be trusted, have repeatedly proved through history to be untrustworthy regarding safety over and over across many industries, and that it's a civic need to have independent oversight. We can debate the amounts, regulatory capture, etc., but absolutely could not ever believe something like saying the status quo is just an industry free-for-all.
In fact, cases where start-ups try to skirt these issues in under-regulated industries are extremely rare and do not at all represent any type of status quo in any sense. There is absolutely no sense in which that maps to the way things work. It is rare for a industry to come up with its own testing and evaluation strategy where societal outcomes aren't brought up as first-principle constraints on how the innovation has to proceed.
> "It seems like companies are already themselves being fairly safe, so how many deaths would you actually be avoiding by implementing more stringent standards right now?"
This is just so egregiously unrelated to reality that I do not have the time resources to even start to reply. My mouth is agape that someone would think this.
> "You assert this, but provide no reasoning."
I agree with this for self-driving cars. I am not an expert of self-driving cars, and feel it's perfectly fair for me to say that in virtually every other industry, regulations are needed to ensure corporations don't actively harm people, falsify results, rush to market, etc., and that it logically and obviously extends to self-driving cars without any need for some onerous debate. The idea that companies are just trustworthy to rush this stuff to commercial markets does not need to be taken seriously. It's just obviously not so.
Beyond that, I completely agree I am not the expert on it, and would defer to e.g. the NTSB to decide how companies are behaving with regard to proper testing. The only part I would add is that as a citizen, my strong preference is for the testing to be gradual with a high emphasis on validation and a gradual proof of concept moving from simpler vehicles and situations to more complex ones. Regardless of what the raw machine learning capabilities are in terms of self-driving, this is needed to prove out the evaluation and auditing infrastructure as much as anything else -- essentially to prove that we have a framework for holding the companies accountable in the right ways.
I'm specifically talking about the current development of self-driving cars. I'm not sure why you thought I was generalizing to all activity among all industries.
> This is just so egregiously unrelated to reality that I do not have the time resources to even start to reply. My mouth is agape that someone would think this.
What's weird about it? Again, we're not talking about solving some hypothetical future problem, in which case I would agree with you about being hyper-cautious and safe. Tens of thousands of people die every year right now. Where is your mouth-agape-level concern for those people?
When a disease like Ebola breaks out, sometimes an experimental drug is used, earlier than would normally happen. This is not because we are collectively giving in to greedy companies wanting to destroy all government regulation, it's because we recognize that the dire need for a solution sometimes outstrips the need for caution. Here is a recent example: https://www.nature.com/articles/d41586-018-05205-x
Do you disagree with this kind of response for serious disease breakouts, or does 40,000 deaths a year just not count as sufficiently dire? Would 100,000 breach that threshold? 500,000?
If the choice is between getting self-driving cars more quickly with minimal regulation -- which is what's happening right now -- versus much more regulation that will slow their development and introduction, it seems straightforwardly obvious that more people are likely to die in the status quo environment in the latter scenario than people die from testing in the former scenario. Companies would really have to go full Mad Max to kill even close to as many people as our current infrastructure and design kill.
> it logically and obviously extends to self-driving cars without any need for some onerous debate.
That's what I'm challenging, and I'm doing so on the basis that right now, things seem to be going fairly well. One person has died, which is very regrettable, but compared to the absolute mountain of corpses our usage of cars has created in general? It's obvious to me what's more important.
I guess I am saying this seems to be straightforwardly false to me, to a huge huge degree, and it worries me that people would naively use the rate of auto deaths under current circumstances as a justification to allow self-driving technology development to proceed unregulated.
It seems we just believe very strongly in positions that are completely and diametrically opposite to each other, so we are unlikely to find productive communication on a message board about it.
Why?
> It seems we just believe very strongly in positions that are completely and diametrically opposite to each other, so we are unlikely to find productive communication on a message board about it.
I mean, if you're not willing to explaining your reasoning, then yes I suppose that's true. Although I think part of why you may feel that way is that, based on one of your earlier comments, you seem to think I'm some kind of libertarian who's just generally opposed to regulation. I'm not; if anything I'm more of a social democrat. I just think that in this particular case, things are different from normal for a new, potentially dangerous technology.
My reasoning on the above point is fairly simple and straightforward:
1. The status quo for SDC development has been relatively minimal regulation. Even California mostly just makes companies report some basic data.
2. Most companies seem to be taking safety very seriously as it is, such that injuries and deaths have been rare. Only one death, and from a company widely acknowledged as ethically-challenged.
3. Given 2, it seems like the number of deaths that you'd avoid with greater regulation is very low, simply because, well, very few people are dying as it is.
4. Conversely, letting humans drive cars has resulted in an enormous number of people dying each year.
5. Given 3 and 4, slowing down SDC development with greater regulation seems likely to indirectly kill many, many more people than the status quo of minimally-regulated development will kill.
Are you expecting Waymo to, any day now, start rampaging around cities and killing people by the dozens? What does the scenario look like where SDC development kills more people than human drivers currently kill? You didn't address my analogy with disease outbreaks; why not?
You've done a good job arguing for the importance of regulation in general. You haven't done a good job arguing for why you think greater regulation in this particular scenario will avoid killing more people than the current setup kills.
Especially when the evidence from the current development of self driving cars is that a self driving car killing someone results in an entire testing programme to be suspended for months, possibly permanently, whilst the company is excoriated for taking unnecessary risks and is considerably less likely to develop a commercial application of SDVs as a result. It also shows that SDVs kill at a higher rate than non-SDVs even with human backstops, which is a much better test of whether they will actually save lives than "does a handful of test vehicles kill as many people as a few hundred million humans"
Giant vehicles are a bad idea, period. But they're no less bad for robots.
https://www.roadandtrack.com/new-cars/car-technology/a100168...
He's describing, incidentally, how professional drivers drive.
Pilots of small planes do the same thing. After a while you can just feel small accelerations and change of directions without needing to check some of the instruments.
If the roads were perfect, you wouldn't necessarily want them (day to day), because it diminishes the minute vibrations that make their way up to the pedals, into the steering wheel, etc., and those things give you feedback into what the car is doing that are invaluable.
Of course, the roads aren't perfect, and most of us aren't doing high-performance driving on them, and we don't need anywhere near that degree of feel, so the benefits of driving by wire far outweigh the cons of analog driving.
Ever notice how your butt tries to move more than your hands and feet when making a sharp turn at a moderate speed turn, and how you instinctively correct for it in your seat by leaning or shifting around? Know the phrase 'Fly by the seat of my pants?' Pretty much all of that rolled into one bundle of instinctive attunement.
It likely allows everyone to learn from all the mistakes and improve the process for the whole industry - similar to the passenger airplane safety record improvements learned from every single accidents/crashes.
At the very least we should have a standard set of crash or near-miss scenarios so that we can collectively learn lessons when things go wrong.
Particularly when people have already paid the price with their lives.
That being said, I still want this tech on the road ASAP, not because it's perfect, but because human drivers are So. Fucking. Awful. They kill thousands of people a year due to negligence, incompetence, and occasional malice. Plus, the additional sensors available and the software response time mean autonomous drivers can outperform even sober, attentive, skilled human drivers in most situations.
A couple of weeks ago, I saw someone talking scary stuff about what someone here called "semi-autonomous murderbots" in a FB post that appeared on my feed immediately after several posts about a local accident, where a bicyclist was killed by a road-raging driver, who then hid/destroyed his car (the murder weapon still has not been found, even though his trial is now over). If that jackass had a self-driving vehicle, the bicyclist would still be alive today.
So are big corporations going to do kind of a careless job testing live vehicles in a rush to market? Yes. Will people get hurt? Yes. Is it worth it? Yes.
This narrative is so pervasive and I must admit a bit strange to me. Millions and millions of people strap themselves into thousands of pounds of metal and then drive 80 down the highway while texting and changing the radio and telling your kids to please stop fighting surrounded by other people doing the exact same thing. And we do that every single day and almost all of us are fine and will never have a 'major' accident. Humans are pretty fucking great at driving.
Besides to say that human are bad (or good) at driving would require some comparison, we're bad at it compared to what? As far as we know humans are the best drivers in universe. There's no evidence that self-driving is currently better than human drivers and there's no evidence that self-driving cars will soon be better than human drivers and there's no evidence that if we allow tons of self-driving cars on the roads then they will be better than humans. It's just blind hope, it really makes me want a word for whatever the opposite of a luddite is, instead of a belief that we can destroy machines to benefit people we believe we can destroy people to benefit machines. It's all twisted up.
If you want to be an Uber or Tesla crash test dummy and walk/bike across a test track while they see if their vehicle can avoid you then by all means go for it. But don't just foist that upon average people.
Until most of people will afford a self driving car you would still have to face the drunk, or the ones that text so you will not remove those of the roads.
These guys made a remote video face-and-eye tracker that they sold to automotive companies wanting to research different dashboard designs -- they would look for blinks and head-nods to measure how fatigued a driver was, and to optimise the dashboard design to keep the driver awake.
I'm sure they are now selling their technology to automotive OEMs for placement in driver-facing camera ECUs.
Their technology worked OK back in 2003. I'm sure it is very robust and mature by now.
But like all new technology, it's improving. People aren't.
And so on. There, of course, are many examples of things in which computers have outpaced human capabilities, and it's certainly plausible that some day this could apply to cars. But it hasn't happened yet and it's also more than plausible that none of this will happen in the lifetime of anyone now living.
Its arguable that self-driving cars are better already. What we need to do is, secure the roads so they won't get confused (fence them & remove pedestrians/bikes). I know, we can't do that everywhere but we could do it for long-haul etc. Like freeways already do.
I think the vast majority of human drivers are alert and sober. It's the long tail of drunk / aggressive / distracted drivers that causes crashes.
>Its arguable that self-driving cars are better already. What we need to do is, secure the roads so they won't get confused (fence them & remove pedestrians/bikes). I know, we can't do that everywhere but we could do it for long-haul etc. Like freeways already do.
I mean as a human driver, I obviously love limited access roads (and human drivers perform very well on limited access roads). We won't have more of those though - the cultural paradigm for urban planning is one of tearing down freeways, adding bike-lanes, and ensuring pedestrian friendliness for maximum walkability.
Driving a car, on the other hand, is usually NOT a repetitive, dull process. You're comparing apples to oranges.
So we need this:
-real numbers of AI to have a meaningful discussion on real statistics
-we need good statistics, so not compare different car types and driver ages/categories
-IMO we have a low hanging fruit related to preventing drunk drivers to use a car, a system to detect a driver that is not looking ahead for more then n seconds, someone needs to handle this, it would improve the numbers a lot
-when we compare say a chess AI we compare it with professionals so it would be more honest if we want to say that self driving AI is better we should compare with a driver with experience that is not drunk
-we should not RUSH this because it will backfire,wait for hardware and software to be ready so if a concrete object is in the front of the car the radar can detect it's position and the cameras and image recognition can detect what it is and not ignore it like it is a road sign, hardware and software should be ready to detect signs, bottles, people, bikes, animals as good as humans, if is not as good as a human then we already lost since a human also many more "features" like intuition,
One interesting point of the development is that every failure is an opportunity to learn. The same cannot be said for human drivers. When the automation doesn't detect a hazard correctly, they can analyze it and tweak it so it detects better next time. Do humans do that?
A "auto" can learn from a failure, and every car using that AI system learns. 1 car results in thousands of systems all learning together.
That's the win of cooperative cybernetics, and something we humans haven't grasped very well at all.
I wonder how many drivers learned from that one?
I am not against the self driving car idea, I can't drive because of my eyes and my son also has bad eyes(not as bad as me ) and epilepsy so he probably will not be able to drive a car, I am disappointed because the startups fucked this up by pretending that they can solve the problem with the crap weak hardware and software they have, as long as you can't detect the exact object position and use image recognition to correctly tag all objects and compute all speed vectors this can't be solved and it is clear the current hardware Tesla and Uber use is not enough.
Self driving may the future but is not the present and it is not the near future either.
Imagine an AI doctor,would you feel good if you know is better then a drunk doctor?
Would you send your child with a self driving car that is better then a drunk driver but not then yourself? ,after 1000 kills the ANN would MAYBE adjust some weight somewhere and the changes will increase a bit
I agree self driving is not the present, but I think it's the near future - 10-20 years.
And to be absolutely clear - I believe that full-on self-driving technology, with no human assist or failover in normal on-road cases, will be a better driver than me within the next decade. I've had my driver's license for 37 years, and I love driving. I've had two moving violations in my life (last one over a decade ago), haven't been in an accident in 17 years, and haven't been in an injury accident ever. I'm a really good driver. And I'm kind of neurotic about being in a car with other people driving - it makes me nervous, even when I know they're good too.
But I'm totally looking forward to the day I can trust my car to do the driving for me, knowing it's better than I am.
The AI needs/has advantages like you said but those are needed to compensate the big missing part , the human intelligence, I am afraid of stupid scenarios where say a bridge would collapse and all the self driving cars would continue jumping of the bridge because this exact bridge colapse was not programmed into the car.11
Because it feels to me they are faking it to buy time until some new hardware is ready. They have the data but the hardware is to slow so they can't analyze it in real time so they ignore it, even if say they want to put a concrete related update it may be too intensive and it won't run in current hardware.
If autonomous cars can drive at the skill level of average drivers - and not get drunk or angry or distracted by forwarding cat videos - then they will significantly outperform humans.
Additional evidence in the form of reason... autonomous drivers have better sensors than humans. Human drivers have limited vision, that only points in one direction at a time. Mirrors offer some slight augmentation, but. Ever backed out of a parking space in a crowded parking lot and gotten into an accident because a car you couldn't see before was suddenly right behind you? Common situation. Autonomous drivers won't have that problem. They can see out the back and to the sides, 360 degrees at all times. (I'll note that some of the first deployed autonomous tech was cars that can parallel-park themselves.)
Additionally, autonomous systems have better reflexes. Remember the two second rule? That's to deal with the delay from your eyes to your brain to your hands and feet. Machines can physically respond to changes in situation faster than humans.
Now, let's level up. Imagine a system where autonomous vehicles are transmitting short-range telemetry to nearby vehicles, letting them know their status, including velocity. They can relay back emergency status info - so the car that slams its brakes to avoid a pedestrian doesn't cause someone six cars back to rear-end the car in front of them. Merge-related accidents (related to parking-lot problems) can be reduced to nothingness with telemetry. Think about how Waze is doing this already with driving data for their mapping software. It'd work better with less human involvement.
I can go on and on, but really, I'm gonna throw a bullshit card here. Fear of autonomous vehicles is magical thinking trumping reason. Humans do magical thinking all the time. We think safe things are scary, and we think scary things are safe, for symbolism rather than fact. That's why people in this discussion used the phrase "murderbot", but not "murderer".
There is exactly zero proof that self-driving vehicles are better than drunk drivers. There is exactly zero proof that self-driving vehicles are better than angry drivers. There is exactly zero proof that self-driving vehicles are better than texting drivers. There is zero proof they will soon overtake any of those categories.
>Additional evidence in the form of reason... autonomous drivers have better sensors than humans.
Teslas have to ignore stationary objects because they can't accurately estimate the height of those objects. My 'sensors' know when a tire shred is on the ground and about 2 inches high and when a bicyclist is on the ground and about 5 feet high.
>Additionally, autonomous systems have better reflexes. Remember the two second rule? That's to deal with the delay from your eyes to your brain to your hands and feet. Machines can physically respond to changes in situation faster than humans.
Watch the video here: https://www.bbc.com/news/technology-44439523
I have driven while texting, I still sometimes drive angry and I have (unfortunately) driven drunk in the past. I have never made the very simple mistake made in that video. Where is the quick mechanical reflex you are leading me to believe exists in self-driving vehicles?
>Now, let's level up. Imagine a...
No. Your imagination is running wild imagining something that doesn't and won't in the near future exist. Your imaginary car not only is covered in sensors, but can p2p communicate with the cars around it, has no lag in processing complex input, can behave itself among human drivers, etc. None of this actually exists, none of it is actually out there and we really don't have good reason to expect it any time soon.
>I can go on and on,
I believe you could, but it's just your fantasizing, it's not a realistic portrait of what is actually happening.
>I'm gonna throw a bullshit card here.
Going to? What about the several you've already sent my way lol.
>Humans do magical thinking all the time. We think safe things are scary, and we think scary things are safe, for symbolism rather than fact.
Exactly! That's why people in this discussion called humans "So. Fucking. Awful." at driving and then wrote three paragraphs imagining a utopian, non-existent self-driving system which magically solves all problems and is really quite safe. Before diagnosing magical thinking in others you might attempt detecting it in yourself.
Your video is an interesting example. That is not a fully autonomous vehicle, but rather an assisted one, one that expects human engagement. In the example case, it failed because of human error. Now, there's a good argument that semi-autonomous vehicles will lead to additional human error, but that's not saying what you're saying.
If you live in an icy climate (I do), and have been driving for a long time (I have), then at some point, you would remember the transition to anti-lock braking systems. Before them, we had to learn to pump brakes in certain ways on icy roads, to prevent locking. Now, just push down the pedal, and a computer does it for you - one that senses brake lock before it happens. It made winter driving much safer.
Your criticism of Tesla sensors is flawed. You're assuming that their sensors will not get better. That seems an unreasonable and unlikely conclusion. And you're ignoring my point about 360 degree sensor coverage. I'm an excellent driver, but I find myself in danger regularly because I can't see everything.
Magical thinking, oh yeah.
You're really contorting yourself here, just give me an apples to apples comparison of human driving safety to self-driving vehicle safety and I'll be happy. (Hint: none exists)
>That is not a fully autonomous vehicle, but rather an assisted one, one that expects human engagement. In the example case, it failed because of human error.
No no, no word games. That is a Tesla marketing/liability line. What you see in that video is the current state of self-driving. And a product marketed as self-driving which can't steer around a stopped vehicle in the lane isn't really self-driving at all. Besides if what you're talking about is Level 5 type self-driving systems, you have nothing supporting the idea that we get there soon or that current self-driving systems will get us there.
I mean besides if drunk drivers could navigate around that car then you're basically admitting that drunk drivers are still better than the current state of self-driving technology. (something I agree with)
>ABS
...
I think you're misunderstanding me. I'm not against technological progress which makes vehicles safer. I'm against the understanding that self-driving technology is making cars safer.
>Your criticism of Tesla sensors is flawed. You're assuming that their sensors will not get better. That seems an unreasonable and unlikely conclusion. And you're ignoring my point about 360 degree sensor coverage. I'm an excellent driver, but I find myself in danger regularly because I can't see everything.
And you're assuming that their sensors will leap and bound ahead magically. You're trying to make me argue that humans are better drivers than the perfect system that exists in your head. I'm saying the perfect system that exists in your head is irrelevant because it is not going to become reality.
As for drunk drivers navigating around that car? There's an intersection a couple of miles from my house that has a stoplight. Southbound traffic is approaching from highway speeds, climbs over a hill, hits a 40mph zone, then the stoplight is a couple hundred meters. There have been multiple fatal collisions there by, well, drunk drivers, slamming into cars stopped at the red light at highway speeds (or faster). I'm terrified any time I'm stuck at that light after dark, especially on weekends.
Of course, an autonomous car would detect the speed zone change and the red light.
>Of course, an autonomous car would detect the speed zone change and the red light.
Would it? I think it's actually unclear that it would and besides this experience of yours is filling in for the place in your comment where you should be showing me the data that shows self-driving cars are safer.
Assuming that it isn't a solved problem is kind of irrational. If your argument is "Well, maybe the car won't see the traffic light it already knows is there", then perhaps your argument isn't very good.
Then you're kind of going against your original comment where you said you wanted self-driving vehicles on the road asap. All you have to do is google: Tesla Speed Sign and you'll find tons of problems and examples of Teslas misreading signs. Here are some examples:
My AP1 car has interpreted a 35 mph sign as an 85 mph sign.
---
My understanding is AP1 visually identifies speed signs, but can screw up (reported that our highway 80 is misread as 80 mph)
---
Addendum: a 45 mph zone, permanently signed (white/black sign large - not orange/black sign small), which by the way AP2 completely ignores. See error-ridden, and in this case an example that creates a safety/financial hazard not present with AP1.
---
And on some highways in the area, especially where there has been recent construction, the speed limit database thinks the speeds are 45 or 50 MPH in sections - when the posted speed limit is supposed to be 60 or 65 MPH. If the car is running under AutoSteer when it hits one of these sections, the car immediately slows down, because AutoSteer won't exceed the speed limit by more than 5 MPH on non-highway roads.
https://forums.tesla.com/forum/forums/new-car-has-trouble-re...
>(and doesn't have an up to date on-board map of them)
This is also something that doesn't always happen in real life. Here's a quick example just from googling:
The "new" speed limit database distributed last summer (from TomTom?) has numerous areas with missing speed limit data or incorrect speeds (both high and low), which causes issues with AutoSteer. And if this happens while driving at highway speeds, AutoSteer will try to quickly slow down in the middle of high speed traffic. While you can disengage AutoSteer and retake control, the other option is just to temporarily leave AutoSteer/TACC running, and manage the speed yourself until the software sees it's back in an area with the correct speed limit.
https://forums.tesla.com/forum/forums/fooling-speed-limit-de...
>But I'm assuming that low-hanging fruit like reading traffic signs and signals is a solved problem.
You know what they say about assumptions lol.
>Assuming that it isn't a solved problem is kind of irrational.
One, it has nothing to do with rationality. Two, it's actually not a solved problem.
>If your argument is "Well, maybe the car won't see the traffic light it already knows is there", then perhaps your argument isn't very good.
Or maybe the state of self-driving technology isn't actually as advanced as you believe. I'm glad we can agree that these systems have no business being on the road though.
As long as you use the word "if" you can say pretty much anything you want.
>Additional evidence in the form of reason... autonomous drivers have better sensors than humans. Human drivers have limited vision, that only points in one direction at a time.
That is not evidence. It is your own opinion. Nothing of the sort has been proven to work reliably.
>Autonomous drivers won't have that problem. They can see out the back and to the sides, 360 degrees at all times. (I'll note that some of the first deployed autonomous tech was cars that can parallel-park themselves.)
They can't "see". They can only sense what they have been programmed to sense. The envelope of what they can process is tiny. Sensors have flaws, humans programming those sensors have flaws. The human eye and visual cortex has been evolving for millions of years.
>I can go on and on, but really, I'm gonna throw a bullshit card here.
The biggest "bullshit" here is that you are claiming fantastical things that autonomous cars can do that have not been proven in the slightest. You implying that other people are stupid for being apprehensive about unproven tech is quite rich...
"A self driving car could kill me or others at any moment, and whether that happens is completely outside my control" is a reasonable fear. People may over-weight it compared to more familiar dangers that are in fact greater, but that's not magical thinking.
Magical thinking is more about believing in a causal relationship between unrelated events. Astrology is an example, but also things like "if I don't shave my beard my hockey team will win" or "I thought something bad about my kid and then they got hurt, I'm a terrible parent".
Hey, another human driver on the freeway here. Sorry, but I don't want to be killed because you want to play with some toys. It is completely unethical and immoral to ship unproven tech that has the potential to kill people.
I'd suggest running an experiment with OPT-IN volunteers in a neighborhood or small city for a year or however long it takes and then coming back with data.
For better or worse, our laws acknowledge that being alive has unnecessary risks, and accepting the social order means accepting the standard of risk caused by other people.
Drunken or distracted driving, by itself, doesn't kill. It leads to the behaviors that kill, like lane drift and speeding and running lights. These are all things that autonomous cars handle really well.
I don’t doubt it will get there and, when it does, these profit-oriented corner-cutting behemoths had better have a shit ton of data to prove it. when they’ve proven the premise you’re taking for granted.
Then I’ll happily see this tech widely deployed.
Yes, if they are drunk or clearly driving recklessly. Otherwise, it's an accident. Their insurance will pay, and they will go on probably with some emotional scars but nothing really in the way of "punishment"
That's the thing. Fair enough, autonomous cars cannot be drunk or sleepy, but they have _others_ issues that "are most easily addressed by human driving". A human might be distracted, but it won't see a bicyclist for 10 seconds while thinking "I shouldn't do anything in case it's a false positive". Human driver won't directly swerve in a concrete wall because it followed to wrong white line. Human driver will predict better the trajectory of other humans, especially infants.
All in all, I do agree that automation can bring improvements to road safety, but in the current situation, it is not clear at all if this can counteract the automation downsides. It will surely improve, but hastiness might not be the way to go here...
So it's not nearly as bad as it's made out to be. the fatality was a consequence of multiple human errors, both consciously and neglectfully. The automation was not in error.
All the arguments about the driver that should have been alerted or reacted are simply not autonomous driving so unrelated.
Again, I'm not saying that automation will never be a good replacement for human drivers, but as of right now, it is not clear at all if its positive aspects counterbalance its shortcomings.
After reading the Atlantic's Waymo article, I'm more convinced that Uber is doomed in this race. They're not investing properly in simulation, and Waymo is getting an enormous amount of programming done based on controlled simulations that Uber isn't - billions of miles, and repeatable scenarios.
Tesla is really in the game, I think, because they're able to collect so much data from the AutoPilot systems currently on the road, even if they're not in use. They still get all the telemetry for complex real-world situations.
But at some point, you have to let experimental software out on the road, to learn from the real world. Simulation is insufficient. Or, we simply refuse to allow auto-autos at all, permanently establishing that humans will always be more capable, without evidence. So it's a question of when, not if.
There's a couple of currents going in the arguments here. Some say "not yet", and some say "never". I think the "never" people are nuts. Auto-autos that broadly outperform human drivers safety-wise seems achievable to me in the not-too-distant future.
Singling out the "previous high profile fatality" is magical thinking. Drawing statistical "evidence" from it is pseudoscience. Meanwhile, a quick googling shows over 800 cyclists killed per year by cars, and nearly 50,000 injured, in the US alone.
Obviously there is going to be a disparity in the number of deaths. What is important is the number of deaths per mile.
Further complicating matters is the fact that most of the miles being driven by these autonomous vehicles are being done with human operators trained to be alert and the unique quirks of an autonomous vehicle. They will likely be much prepared to stop an accident from occurring when the car makes a mistake than an average consumer.
Autonomous driving may in fact be revolutionary and reduce the number of traffic deaths dramatically, but it is still early enough in development that there isn't any evidence to bank on anything at this point. There certainly isn't enough to rush the tech out the door either on the public either.
If you want to simply reduce vehicle related deaths, you could address this through various ways like improving road safety/visibility, adding more driver assistance aids. Or heck, design a protocol to disallow cars from starting with a drunk driver or without a seat-belt, etc etc. Maybe people who get into even minor accidents need extra training, etc. etc. There's so many low hanging fruit here..
Disclaimer, I have never been to Nashville, but I've heard about it from people who have been there and did a bit of reading 5 minutes ago.
http://soundtransit3.org/calculator
It's a sizable chunk of transit. It's all about the priorities of a city.
Second, yes, they can't come soon enough because traffic is a meat grinder: it's human sacrifice to the gods of speed and convenience.
However, you do not test dangerous industrial equipment in the street with kids and dogs and bikes crossing the freeway, etc...
You just don't do that. It's insane.
A big robot with a couch on it should never travel faster than it can reliably stop quickly enough to prevent any conceivable collision. That's the hard limit. Until we can do that, any machine heavy enough and traveling fast enough to kill is a killbot.
First Law of Robotics?
But that said, aren't you holding automation to a different standard than humans? Is it okay to have the big robot with a couch travel quicker than it can stop to prevent any conceivable collision, just because a human is in charge of the brakes? Because that's what cars do now! I can conceive of situations where it's not possible to hit the brakes on time in an ordinary car - hell, I once hit a bicyclist when I was only going about 15mph, because she basically appeared out of nowhere before I (or she) could stop.
Cheers!
> But that said, aren't you holding automation to a different standard than humans?
Yes and no.
Humans drive preternaturally well en mass. So much so that when I realized it I began to suspect that "Guardian Angels" might be real. Then I learned more about human unconscious motor neurology. Driving performance is one of the few places where the unconscious mind is blatant. Nevertheless we have set up a system that amounts to a "mayhem lottery". When I say "they can't come soon enough" I really mean it.
Now eventually auto-autos will be super-human in capabilities and performance. They don't have blind-spots, they can use radar and lidar and mass sensors, etc. They will also have the benefit of perfect hindsight across the whole fleet (if we set it up sensibly!) Further, traffic infrastructure will evolve to facilitate automated traffic, with sensors and comms embedded in signals and even the road. They'll be able to share knowledge in real-time of local road conditions and keep track of people and (non-self-driving) vehicles around them cooperatively. Sooner or later, it won't be possible for someone to surprise one of these things.
And all this will happen at an ever-accelerating rate. (Which means that if you just wait, making them gets exponentially easier!)
In the meantime, it's not okay to use public space to test crude early prototypes. People have already died.
I was truly shaken and upset when I read that the Uber car that killed that woman crossing the road with her bike had determined 1.3 seconds before it struck and killed her that it needed to initiate emergency braking, yet the emergency braking system had been disabled because it didn't work well enough. The machine knew it had to stop, it had the capability to stop, but it was prevented from stopping by bad decision-making driven by the desire to put unsafe machinery on public roads.
That's tragic and truly pathetic, and I would say the makers committed murder. Certainly, if you or I built a robot and sent it out into the world and it killed a person we would have to defend our actions with something better than, "Well it's not perfect. Give it some time."
We could make a e.g. self-driving golf cart that could safely take my mother to her doctor's appointments and back. She's got dementia so we don't want her to take the bus even though she's physically able. It would have a top speed of 3~5 mph, and obviously it would have to travel restricted routes where it wouldn't interfere with traffic, and a there are a host of other concerns. But it would work, we could build it today, there's a market for it, and it would have a really hard time killing anyone in a collision. It would be lightweight as well as slow-moving and covered in airbags inside and out with pre-impact detection and triggering.
Once you start to think outside the box (sorry) there's all kinds of scope for incrementally automating traffic.
Trying to go straight to imitating full-on automobiles with human drivers on public roads is insane, it's hubris. And again, people have already died. So this is not a theoretical concern. There's been a funeral man.
> Is it okay to have the big robot with a couch travel quicker than it can stop to prevent any conceivable collision, just because a human is in charge of the brakes? Because that's what cars do now!
It's not okay for robots to kill people just because people do, no.
(I think a sane transportation network would have at least four interlocking but separate systems: pedestrian, bike, car/truck, & rail. Cf. "Pattern Language" C. Alexander. But I'm getting off on a tangent.)
> I can conceive of situations where it's not possible to hit the brakes on time in an ordinary car - hell, I once hit a b...
Everything "has the potential to kill people" if you put enough of it in a box and drop it from high enough. The question is how much potential, and compared to what. If we want to use strong moralizing language like this, we could just as easily go the other way: "It is completely unethical and immoral to delay technology that has the potential to save lives." That would be true in exactly the same ways, which is to say, it's complicated and we need to avoid moralizing.
No, you couldn't "just as easily" go the other way. Those are not identical positions. My position promotes caution over over-confidence. Further more, the "potential to save lives" is not the exclusive domain of autonomous vehicles. So nobody is "delaying" saving peoples lives. Various methods that can be utilized to reduce causalities that do not rely on technology that has not been shown to work in a conclusive fashion. Planes were unproven tech at some point, and people rightfully refused to fly in them, and they aren't as unsafe now. Technology gets better, and we can hope that autonomous vehicles would get better too.
The baseline here really isn't that high. How many miles have you driven, and how how many accidents have you been in?
But in this case, the outcome of interest is quite rare. Despite the high numbers of people dying in car related accidents each year, the number of accidents that occur for the total number of miles driven by US drivers is very small. So in order to draw inference about whether autonomous vehicles improve on that rate, you need a huge amount of miles driven by autonomous vehicles. None of the companies have really approached that level of testing, to say nothing of the fact that most of the miles driven so far aren't on similar, real world conditions that US drivers face, making any statistical conclusions even more difficult.
I don't believe the Uber accident applies, because it was caused by human error, not the driving automation. The emergency braking was deliberately disabled (it would have prevented the accident), the detection system was not programmed to inform the driver, and the driver was not paying attention. If the Uber accident is disregarded, then we're at 0 fatalities, or infinitely perfect automated driving. Which is, of course, BS. But it's as rational as the argument others are making.
Given the inadequate data set, I'm relying on inference, not experience.
If we're seeing these fatalities with these companies having only tens of cars on the road, what can we expect if they deployed millions?
2) right now self driving cars are more deadly than human drivers by several orders of magnitude. Thats the reality.
2. You're misusing statistics. There isn't enough data on autonomous driving performance to make your claim, not enough road miles.
We can't demonstrate empirically, because there is far too little data at this point (and refusing to do the thing to collect the data because we don't have enough data is not a sound moral argument).
But we can infer a great deal. And I've already established grounds for inference...
- Better sensors. Sensors that can detect 360 degrees at all times, and see into infrared, not to mention LIDAR. The sensors can easily outperform human eyes attached to human heads. - Better mechanics. Automation can be directly connected to steering and brakes, as opposed to modulating through human hands and feet and into other mechanical linkages. - Better reflexes. Less lag from observation to action, compared to the eyes/brain/hands system. - No distraction. Human minds wander. - No intoxication. - No sleeping! Falling asleep at the wheel causes many fatal accidents. - No emotions, no anger, no aggression.
So emotionally and mechanically, autonomous systems are clearly superior to humans. The only real argument left is interpretation of the environment. Can they detect hazards more effectively than humans can? Let's look at some hazard categories...
Lane drift - easily solved. Running red lights and stop signs - easily solved. Speeding - easily solved. Maintaining braking distance in traffic - easily solved.
I could go through more, but you get the idea. We're focusing on "What if a pedestrian steps out in front of it and it doesn't recognize them as a hazard?", and ignoring much more common categories of accident-causing situations, categories where machines easily outperform humans.
So "empirical base"? Sure, no basis. But inferentially, the argument for the superiority of autonomous driving systems is very strong.
I believe the evidence strongly suggests that autonomous driving will not only catch up, but greatly exceed human drivers in the foreseeable future. Mechanically, automation is superior to humans for simply operating the car. And the large categories of causes for fatal accidents are low-hanging fruit for automation - lane drift, running red lights, driving too fast for conditions, etc. And sensors can observe the environment as well as or better than eyeballs.
The only real point of contention is in interpreting unexpected hazards, like the infamous bicyclist. But interpreting sensor input is subject to continuous improvement, due to programmer experience, log data, and Moore's Law. So even if that point is weak now, there's no reason to believe it will not improve substantially - in fact, it is unreasonable to believe that it will not improve.
Large categories of potential causes for fatal accidents are things which humans very seldom have issues with but computer programs often struggle with like inferring intent and identifying whether something is animate, which can easily cancel out reductions in other types of accident due to (e.g.) better average lane discipline. Especially when you consider the benchmark for fatalities per billion km driven is around 3.5[1] including fatalities caused by humans considered unfit to drive, and humans who are considerably less experienced and safety-conscious than the likely early adopters of SDVs. SDVs have already had one fatality and several incidents where human intervention prevented accidents in a few million miles of low speed, good condition driving. I think we're a very long way from seeing any indication computers alone will ever beat that benchmark, still less a plausible case that the additional deaths resulting from riskier testing programmes will be cancelled out by lives saved if the first few SDV models go on sale a few months earlier.
[1]a few European countries; it's double that in the US, which also suggests that if you want to reduce road deaths there's a lot of other steps that can be taken which don't involve risky technology experiments.
The alternative is to simply never allow it at all, to assume that humans will always be better drivers than computers. Which opens up an ethical dilemma. If auto-autos could save tens of thousands of lives a year - and it seems at least plausible that it could happen - then to reject it out of hand is to cause tens of thousands of innocent deaths a year.
Your argument is "Sensors are better than humans!", and that's a non-argument, inferential or otherwise.
There's a lot of problems we don't know the solutions to, but that certainly does not mean that these teams don't know the solutions either.
You are commenting in a thread about how a team of those "much qualified software engineers and other specialists" failed to do simulation testing. So yes, there is reason to believe that these companies are failing to address many of these potential problems.
Detecting sensor failure is child's play. We have over a half-century of experience at detecting and handling sensor failures in computer systems. Hell, I left the gas cap off my car once, and every idiot light in the car flipped out on me. Cruise control stopped working. Etc.
Detecting glare, flashing lights, and other visual interference is likewise well-established technology. And how does the autonomous car respond? By slowing down, staying within safe parameters. Meanwhile, one of the most common causes of fatal accidents is humans driving too fast for conditions, often due to poor visibility. I trust the self-driving car more than the human driver for that.
And, as you mentioned, we need to accumulate billions of test miles before we'll have statistically comprehensive data.
There are currently at least 4 confirmed deaths from autonomous vehicles[2]. While beat is correct in stating that those are not enough deaths to draw reliable conclusions from (and Model S owners are not like the average driver), there would have to be more than 600 million km driven autonomously to be comparable to the death rate of normal cars. Probably significantly more, as the cohort of human drivers that is equivalent to Model S owners is way less often involved in accidents in the first place.
[1] = https://en.wikipedia.org/wiki/List_of_countries_by_traffic-r...
[2] = https://en.wikipedia.org/wiki/List_of_autonomous_car_fatalit...
Just read about the hoops one has to jump through to run a simple medical study if you still think we don't need a dose of utilitarianism.
The means are the ends. An organization that disregards safety during product development is not going to have the culture required to actually produce a product that improves safety.
And yet, so far, Uber's self-driving vehicles have a higher fatality rate than the status quo.
Sure, it could be improved over time, but the whole point of TFA is that Uber is currently not doing the requisite diligence and testing with their vehicles to ensure safety.
Do you get to decide this? No. Does Uber get to decide this? No. Is this attitude going to produce a backlash against the idea of the technology, similar to the GMO backlash? Probably.
I also agree with GP. We need to take more _calculated_ risks instead of trying for strictly risk-free science. Personally I'll gladly accept more dead astronauts (and hail them as heroes) if they push spaceflight further.
How noble of you.
That seems like a huge leap?
What if the driver had a gun and just expressed his rage with bullets?
What if the self-driving vehicle, prior to the rage part, hit a bug and hit the cyclist?
And by self-driving vehicle you mean e.g. Level 5? The driver could not take control to express his rage?
This doesn't seem like a huge leap to me at all. If he'd been driving responsibly, he probably would not have hit the cyclist. And autonomous drivers don't get angry or careless. Your follow-on about imaginary scenarios kind of underscores the weakness of that argument.
And I do agree that a raging driver taking control from the automation continues to be a problem. But that's no worse than the original situation, now is it?
Bugs are not imaginary scenarios. Also, did the US outlaw gun ownership without the rest of the world noticing?
> And I do agree that a raging driver taking control from the automation continues to be a problem. But that's no worse than the original situation, now is it?
I only replied to one point: that you cannot state with 100% conviction that the cyclist wouldn't have died had there been a self-driving vehicle.
You could back down and just concede you really meant to add at least one condition: "if that jackass had a completely autonomous vehicle, with no manual input possible, it is highly unlikely the cyclist would have been killed"
I was out car shopping recently, and even affordable midrange cars (Subaru Crossfire and Honda HRV) offer technology that could have done this, as a standard feature.
There is a big difference between humans killing others as one off incidents vs one bug, defect or security flaw making a crash possible with the whole fleet. It can be far more catastrophic.
On the other hand, that's the kind of thing that gets addressed by brute-force improvement. Security is a well-understood field, and the main problems we've seen with car security so far have been caused by insufficient effort, not insufficient technology.
[citation needed]
All the current evidence is that humans grossly outperform computers at the task of "driving vehicles to arbitrary locations in all conditions", and that that isn't changing any time soon. Like, it isn't close.
HUMANS: =========================================================
MACHINES: ==
You can talk about your speculative perfect-performance car if you like, but there's no evidence that it will ever exist. Why not give it hyperdrive too?
Driving vehicles to all locations in all conditions? Yeah, humans will have advantages there for a long time to come. Driving on highways and city streets within the bounds of traffic laws and expected velocities? I think computers will be better than humans very soon, if they aren't already so.
Self driving cars are a hard problem, a very hard problem. So is CAPTCHA and machines are still not that great at solving them.
The problem space of autonomus vehicles is a large one, and even with how many mistakes human drivers make, I still don't think we're anywhere near the tech to make safe autonomous driving vehicles. We're at a minimum, 15 years off.
I want you to think about all the technology we use every day. Our phones, desktops, laptops, are full of bugs. Hell I found bugs in the ticketing system I buy metro cards from the other day. We see display kiosk crash all the time.
A Tesla car with lane assistance (please stop calling it auto-pilot) recently drove a car into barrier. Another kept driving for hours without hands on the wheel (fixed in an update. Was that driver passed out? No way to know.)
Now you might say, this is different. The engineers working on self-driving tech are at the level of the ones that build bridges, or make pacemaker software (which by the way, some were found to have a bluetooth vulnerability recently) or plane avionics.
Maybe, but Uber certainly isn't. They're a culture of scum shit shop. This isn't a small thing here. It's a huge, huge problem space that no one completely understands yet. Why are there not any self-driving cars on the streets of NYC or Chicago? Because it's difficult, and dangerous and if there were, people would be dead right now.
For all the money we're investing in self driving cars, we could build tech that already works. Singapore and London both have fully autonomous trains; some that arrive at 30 second intervals and carry millions of people a day. America needs to get its normal infrastructure back before we go down this fantasy world of self driving cars.
I wrote more about this a while back:
https://penguindreams.org/blog/self-driving-cars-will-not-so...
These arguments don't make a lot of sense to me:
- Do you think display kiosks would still crash all the time if the industry spent a few hundred millions at improving them? - Do you think the metro ticketing system would have as many bugs if the transit agency spent a billion on it?
If not, why are you comparing very different situations? Maybe we need to use average number of bugs per amount spent on dev instead of average number of bugs per project or per # lines of code
For the USA:
37,000 vehicle deaths / year
2.35 million injuries / year
1,600 child deaths / year
8,000 killed in crashes involving drivers ages 16-20
$230.6 billion per year in crash associated costs
If any company can show that these statistics can be reduced (halved/thirded/decimated) in any meaningful way with their autonomous tech, that's when we'll finally reach a new era in transportation. How can anyone advocate for human-driven cars once the implications become clear? Let's say that the Uber autonomous car reduces vehicular injuries and deaths by only 5%, well shouldn't that be enough evidence to say, put that on our roads? I think that the current level of tech is already at a point where it will save lives on average more than a human driver, what more do you need from a regulatory point of view?
Take the UK for example. It has many of these safety improvements, and gets one quarter of the risk of dying in a car accident (one half the risk per mile driven, since Americans drive further).
https://en.wikipedia.org/wiki/List_of_countries_by_traffic-r...
Getting out of a "local minimum" by definition requires things to get worse before they get better - that's what makes it a local minimum. So it requires faith, which is belief in the absence of clear evidence. Faith is always uncertain, which is why people get strident and appeal to emotion when it is challenged.
It's anything but clear
This is despite automatic numberplate recognition on most freeways, many cars having trackable cell/wifi/bluetooth connections, automated RFID identifiers for toll roads, every car having a unique VIN which is printed on nearly all components and will be registered by scrapyards, and tracking on all mobile phones (presumably the driver had a phone!).
What's known is that the driver, after hitting the cyclist, drove to his mother's workplace, and she left with him. The car he was driving has not been seen since. If it was driven into a remote lake somewhere, it will probably never be found.
One would think a murder investigation would (a) know this, and (b) have the ability to get data from a military or private satellite, both before and after the suspected incident.
Failing that, tracking data from the pairs phones would likely indicate which lake, and probably which part of the lake even.
The driver was convicted of vehicular homicide (max five years), and leaving the scene of an accident. He can't be forced to testify about the location of the car (Fifth Amendment), and I don't know if he was charged with obstruction of justice. His mother was charged with obstruction-type charges, and her case hasn't been tried yet.
But since you aren't, I'm guessing even you don't really believe what you're saying. Which is good, because a key assumption of your theory is proven wrong by the article you're commenting on.
Yes, these vehicles will eventually save lives once their error rate drops below that of a human driver. But there's no reason to think that running over a bunch of real people is necessary for progress until these cars work well in simulations. As the article explains, they get orders of magnitude more testing done in simulation than on the road.
Uber's rush to test things in the real world was not about advancing the technology or saving lives. It was about getting a short-term commercial advantage on the cheap.
The data and experience gathered from real-world testing should (I'm not saying "will") accelerate the progress to the point you mention - "once their error rate drops below that of a human driver".
Let's look at the Uber case, since everyone infers so much from it. One fatality. One. And the program was immediately shut down. And how did it happen? Engineers had deliberately disabled the automated emergency braking system to reduce "erratic behavior", but had no automated notification to the driver. And the driver was distracted, doing data entry for the test as he was instructed to do. If the emergency braking system had been on, it would have stopped the car on time - they have the data for this.
This sure looks like human error to me.
And that single accident put Uber's program drastically behind schedule, relative to more cautious/thorough competitors. Uber paid a market price for its eager carelessness.
If Uber was more focused on doing things the right way their program would be much further ahead.
We don't have reason to believe that. Even Uber now agrees that more simulation should have been done, and they're trying to catch up. All the necessary data for this could have been gleaned from instrumenting human-driven cars, and then testing the object recognition, object avoidance, and other driving algorithms against the resulting data.
Uber's Arizona death wasn't "human error". It was negligence. They also weren't "behind schedule" in any meaningful sense; some exec picked a too-aggressive goal, and people cut corners in a vain attempt to meet it. They shouldn't just be paying a "market price". They should be facing charges for criminal negligence leading to involuntary manslaughter.
Aren't self driving error rates already below human drivers? The fatality rate is extremely skewed by one incident in which the system detected an obstacle but wasn't allowed to stop the car because it was disabled (so... not a failure of the self driving aspect of that system) and another in which a partial self driving system warned a driver repeatedly to take the wheel, but they did not.
In both instances the problem was a failure to force action by the human drivers (or failed reliance on them). In the former case, the system would have braked, in the second, the system should brake instead of asking for help from the driver.
Meanwhile the overall crash rate for self-driving cars is lower, correct?
I don't think that's correct in general, and especially not so if you compare the conditions that each are driven under. Said a bit differently, the human error rate is in all conditions (rain / snow / nighttime / unmarked roads / gravel) where most self-driving miles are on well-marked city and highway roads in near ideal conditions.
But you're already no-true-scotsmanning your way to giving the self-driving cars unrealistically good scores. Yes, a cause behind the Uber crash is indeed Uber disabling a safety system they thought unnecessary. But we have no reason to believe some other companies aren't also either turning off or just never adding safety systems.
My objection is not to better testing. It's to bad comparisons. Including, as here, with somebody handwaving at data that, at best, couldn't answer the question.
A lot of aerospace money has been spent learning how to do things safely.
It would be a shame to spend the same money all over again, re-learning the exact same lessons in automotive.
Or even do it on a closed track. That way you could throw all sorts of things at it like kids, bikes, garbage cans, whatever.
Should be able to see what works and what doesn't work pretty quickly.
EDIT: Never mind! That's what Waymo does in the article another person posted.
https://www.theatlantic.com/technology/archive/2017/08/insid...
I don't know if Uber/Waymo/Others have "solved" this problem, but ML systems have a tendency to "learn" the wrong thing. You could run millions of simulations (both virtual or on a closed course) with "test obstacles", but the second the car encounters one a child with red balloons in the early morning on a foggy day it might not know what to do with it because that exact scenario wasn't in any simulations.
Agreed the test track is not ideal, that's why I was wondering if they could throw real recorded data at it. No risk to the public, but puts the system through real-world trials.
If 10 different executives knew or should have known that their decision had life-critical implications, then I think all of them should face the appropriate penalty. (Here, I presume it's for involuntary manslaughter due to criminal negligence.) And I think the same penalty should apply to developers who knew or should have known that their actions put lives at risk.
I was not considering the different sensor inputs that would not work on an existing game.
You can get a lot done (and get wonderful demos) through research vehicles. The issue with sim is similar to the issues with AVs as a whole - getting to 90% is much easier than getting to 99.9999%.
Also, there is this weird thing in AV (and specifically in-house sim) where pointy-haired managers think "if we put 30 SWEs on the problem - we'll make progress!". While raw eng horsepower matters in some software problems, when you are on the line between research and engineering, more SWEs might actually make getting a robust product up harder.
Another thing we've seen is in-house efforts overfit solutions. They don't really have a wide viewpoint (nature of how secretive the space is), and therefore go down paths which are just plain wrong but no one is there to challenge them.
I always assumed people would integrate an external simulator like CarMaker rather than build their own.
I'd be exceedingly interested in hearing about your product when it comes available (although my interest is in simulation for maritime autonomy rather than automotive).
Nobody (well, theoretically someone, I'd like to be proven wrong) has a self-driving algorithm which is formally proven. If we can't get, to use your example, web engineers to formally prove their web applications, when they have all these structural advantages, then how can we expect something unproven (har, har) to be formally proven? So there needs to be an extensive QA test suite.
And for all of these other relatively-mature engineering sectors like web engineering, how many of these companies are really investing in QA? In strong and comprehensive test suites, in security reviews, in performance testing and UX scoring before release? Almost nobody, right? I mean, the only reason why society can get civil and biomedical engineers to submit to strict quality control is through regulation, but no regulatory body has any real idea of how to apply that to software. And autonomous driving systems are supposed to be different?
Mythical Man-Month came out 38 years ago. The Pareto Principle, not so much as being explicitly a 80/20 split but about how the vast majority of the effort is needed to achieve actual completion rather than just a proof of concept, was originally coined in 1896. Suddenly it doesn't apply to Uber?
Is it really so surprising that it's so difficult to find senior practioners in these areas when few companies are committed to that level of quality, and the companies which are tend to be so successful that they can afford to pay compensation at rates so far above the rest of the market so as to ensure that few of their senior engineers ever leave?
Society usually doesn't need a formally proven cat pics sharing app, and most web apps, including facebook, are that level of application. They would rather pay a cheaper price than to have a perfect expensive thing.
You'll also notice that as organizations get bigger, they do invest more in testing, QA and regression detection.
Needless to say, this is expensive. I’ve heard estimates that it costs roughly $1000/line of code.
The point is that there are methods to increase reliability, but there isn’t the financial incentive to do so.
If there is only a single developer working on a project, that developer can manage much much more code if there are solid tests behind it than if she was just winging it with every commit.
At some point the developer starts to forget the code they write and starts relying on the tests they built long ago.