I feel that this sentiment could be applied to almost anything and isn't really meaningful. Almost every accident ever could be attributed to negligence somewhere.
I think the point here is a firearm in particular is designed to project deadly force and should be treated as such. If someone "accidentally" shoots someone, it means they were negligent in their treatment of the instrument in their hand.
Autonomous vehicles are expected to be able to safely navigate a large variety of streets and highways, and possibly trails and countryside in the future, without killing anybody else around or killing the people inside, nor destroying the fixtures and buildings that make up your typical city street. The only problem is cars are inherently deadly, go look up the number of people that have died in automobile "accidents" since the automobile was invented, total it up for just one country. They need to be treated with and afforded the same respect as any other instrument intended to project deadly force, because they do, as do whatever algorithms are controlling the instrument.
In NYC the urbanists have a trend of saying "crash" instead of "accident" when referring to traffic collisions. They feel that "accident" gives the driver too much credit in situations where they were completely at fault for not looking, going too fast, running a light, etc.
> In NYC the urbanists have a trend of saying "crash" instead of "accident" when referring to traffic collisions.
Everyone I've known in law enforcement, especially CHP officers, does this by default (though they tend to prefer “collision”) when referring to an event where a car hit something. As one of them explained, “collision” or “crash” is a readily discernable physical fact; “accident” is a conclusion about the mental state of the participants.
Which by the way is idiotic. Last century, people switched from "crash" to "accident" for exactly the same reason -- to shift liability from the machine to the operator. IT's a nonsensical euphemism treadmill.
The idea that an "accident" isn't the drivers false is an absurd premise. Drivers are obligated to be diligent. "Crash" is a neutral term that doesn't assign liability to anyone.
Agreed; more generally, there's a movement to avoid calling these incidents "crashes" instead of "accidents", since it's arguable that crashes are avoidable: https://www.crashnotaccident.com/
This is a dumb saying that ignores the fact that humans are fallible beings who make mistakes.
Sure, you might shoot yourself because you're just careless with your gun. Or you might be also thinking of an argument you had with your spouse, or you might be tired when you're handling your everyday carry on your way to work, or you might get startled, or a million and one other things that might prevent you from singly focusing your attention the gun you're handling.
Machines exist to serve us, not the other way around. If a product doesn't take human habits and flaws into account, it's a bad product, because over a large enough sample size someone will make a mistake.
> This is a dumb saying that ignores the fact that humans are fallible beings who make mistakes.
It's not, because the basic rules of gun handling are designed in such a way that failing to obey one of them will not result in injury. They are:
* Never point a gun at anything you don't wish to destroy
* Keep your finger off the trigger until you're ready to fire
* Always assume the gun is loaded
* Know your target and what's beyond it.
> Machines exist to serve us, not the other way around. If a product doesn't take human habits and flaws into account, it's a bad product, because over a large enough sample size someone will make a mistake.
Modern firearms are absolutely designed to this criteria - it takes a willful act to discharge them. If someone "mistakenly" fires a gun and hits someone, it requires multiple failures and clearly rises to the point of negligence.
As long as you follow the basic rules of car safety, you won't crash. Stay within the speed limit, be aware of your surroundings, keep a safe distance from the car in front of you, signal 100 feet before a turn. Unless someone else doesn't follow the basic rules of car safety!
You know what system we've come up with to improve car safety that has only one rule to remember? Always wear your seatbelt.
You know what we've come up with that requires zero effort from the driver? Laminated glass. Crumple zones. Airbags. Automatic braking.
This isn't about whether people use a product negligently — put it in enough hands and it will happen. The point is to design products such that the consequence of negligence is not injury or death.
They're basically asking if the car avoided doing something that would have been more destructive (e.g not hitting 2 people but only hitting one person instead). Or if there was no conflict decision making that happened and the car accidentally ran into the person and it resulted in a fatality.
- AI decides between one life lost as a better potential outcome
- AI made no decision and just happened to hurt someone
I think it's a reference to the imaginary trolley problem, which is often raised as an objection to self-driving capability.
The idea is that making a decision between killing one person vs killing something more valuable than one person (eg. multiple people) is something that drivers do on a regular basis. Computers may not be capable of correctly evaluating the ethics of that decision.
One of the most oft-discussed questions around self-driving cars is what should happen if your car can choose between two options: 1) An action that will have a better chance of protecting the people inside the car, 2) An action that will protect people outside of the car, with a whole bunch of variations.
For example - What if someone steps out into the road and you can either hit them, or run into a wall? The former makes it far more likely the occupants of the car will be safe, the latter saves the life of a pedestrian at the expense of possible injury to people in the car.
It is kind of an interesting question, because people can't generally react quickly and rationally enough in a situation like that for it to matter, but in theory, a computer can. But it also dominates the discussion in a way that likely far overstates how frequently this will actually be a decision a computer has to make.
I think that is what the person you're replying to is asking. Given that this occurred at night and on (very likely) uncrowded streets, I feel like its far more likely the car (and backup driver) just somehow missed the pedestrian.
I suspect they mean the trolley car problem: "you are either going to crash into a group of school kids, or into a group of nuns. Who do you kill?"
I.e. did the uber car kill this 1 pedestrian, because it avoided killing 2 other people? Did the uber car opt for the "least killing" in this siutation?
I think this whole trolley car question - while a nice philosophical question - is silly though. I dont think humans would do any better in an emergency situation. In that split second when you suddenly find yourself in the "oh sh-<IMPACT>" situation, do you have time to a) make a deliberate, reasoned & fully-informed decision, and then b) control the vehicle effectively to follow through on that decision? I doubt it. If you had a second or two to think it through and then deliberately make a choice and steer towards that crowd of nuns, you'd probably be able to entirely avoid the accident anyway.
More likely is you'd probably do what I am sure most people do which is gasp, stamp on the brakes, shut their eyes and hope for the best ... assuming you even had time to realise there was about to be a crash before it happened. Some people might swerve, but I suspect they do that instinctively to avoid something in their way, not as a decision to hit something else.
I just dont think self-driving cars will get themselves into situations where they have to chose who to kill in the first place. And even if they did get into those situations, I really, really doubt it would be due their actions, and I'd certainly trust it to avoid the crash in the first place a whole lot more than the average human driver in the same scenario.
Some could argue "Ah but yes computers are so fast that they CAN make that informed decision about who to kill in milliseconds! So they have to make a choice! The question stands!". I'd argue back that in those milliseconds before the crash was inevitable, they'd act to prevent the accident before a human would even know what the hell was going on anyway. After that it just starts getting into a game of who can conjure up the most ludicrous hypothetical situation that rarely - if ever - happens in real life.
The most obvious example I've seen is when the car (or indeed driver) has the choice to endanger the passenger or a pedestrian.
For example, if a kid runs out into the road in front of the car, it can drive into the kid, likely with no injuries to the passengers. Or it can swerve and drive off of the road, or into another lane, likely saving the kid but putting the passengers at much greater risk.
As you say, in this or any similar situation the human reaction is probably going to be to instinctively brake or swerve without a chance to consider options or consequences. I feel like a computer will have a few cycles to spare to make decisions like that.
Yeah, it will be interesting to see if this was an intentional (algorithmic) response or a failure of sensors somewhere.
I go back and forth with thinking the algorithms in things like self-driving cars should be public. Anything which must make a choice such as smash into bus or smash into bicycle — the public should be aware of the choices the company chooses to implement. Obviously this example is simplistic but it illustrates my point.
Today I’m definitely leaning towards it should be public and scrutinized. Next month I may be worried this would hamper progress.
Hm, there have been other crashes where the 'driver' in the car died (I recall one where a car in autonomous mode rear-ended a truck, something about how the sensor was blinded by the setting sun? And it had alerted the driver to take control several times before the crash) but this is the first incident I can recall where a pedestrian was struck and killed by an autonomous vehicle.
IIRC in that case the driver T-boned a truck crossing the road, because the side of the truck was so shiny that it read as the horizon to the car's sensors.
Obviously more information is needed, but I thought the entire point of having a driver behind the wheel is to manually intervene to prevent this very situation?
I'm very curious to see how they'll investigate this and who will be determined to be at fault (person behind the wheel or Uber). It will likely set a precedent.
Human beings simply cannot switch between "not focussed" and "in charge of a car, taken out of autonomous mode, and actively avoiding collision" fast enough to avoid most accidents, unfortunately. Neither can humans maintain the focus required to be ready to do that when 99.9% of the time they're not required to do anything.
Exactly like what has been done successfully in aviation for decades now. For example Auto-GCAS. I don't understand why car companies are trying to go against a proven model.
Modern aviation includes fully autonomous modes though. An airplanes autopilot is simpler than a car's autonomy, but they accomplish essentially the same goal: get you from point A to point B with no human interaction. AFAIK, even takeoff and landing can be done mostly autonomously on modern aircraft.
If anything, airplanes prove that machines doing most of the work and humans stepping in only when necessary is a proven model.
That's not at all the same thing. When commercial pilots engage the autopilot (or autoland) they're still actively flying the airplane, just operating at a higher level of abstraction to decrease the workload and fatigue. They're not sitting back and playing Candy Crush on their smartphones.
Automation for airplanes is really so much simpler, a lot of that thanks to a lot of hidden human effort in keeping planes well separated so pretty much all it has to do to fly is keep course and control. If that was all we had to do for cars it'd be pretty simple.
> If anything, airplanes prove that machines doing most of the work and humans stepping in only when necessary is a proven model.
Airplanes have far less traffic to deal with. And the points at which they deal with traffic (e.g. takeoff and landings) is completely controlled by humans, including many humans outside of the plane.
Apparently pilots generally prefer to not use them. Not because they don't work but because pilots still need to be on alert and so its easier to just land manually.
They're used in low visibility conditions with relatively calm weather, but don't work well in bad weather.
A significant number of air accidents have resulted from unintended interaction of the autopilot and the pilot. Usually through some level of confusion about whether the autopilot is engaged or not (and 'how engaged', aircraft autopilots can have a complicated array of modes). There's a lot of learning in autonomous system to human interface design embodied in modern aircraft autopilots.
When an unusual situation causing the autopilot to disengage happens in the air, you often have minutes of time to deal with and correct the issue, almost never less than 5-10 seconds. In a car, you're lucky if you have even a single second, and that's just not enough time to take over.
For a couple years now we've had auto braking and then more recently lane keep assist which is pretty much exactly the 'automated backup to human drivers' option. It's great but people still want more automated driving systems.
The military is actively working to remove pilots from the equation, at least for certain missions. Eventually some of that technology will be spun off to civil air transport.
Your car's software already has the capability to kill you. Things like automatic braking are much more likely to save your life than to endanger it. To say otherwise is just paranoid scaremongering.
This breaks the site guideline against calling names in comments. Could you please not do that? Your comment would be much better without the last sentence.
> Human beings simply cannot switch between "not focussed" and "in charge of a car
This is not how many people drive in non-autonomous cars? Commuting is on auto-pilot for many of us; you don't really remember how you got to work and I see people tying their shoelaces, calling, eating sandwiches, reading the paper, chatting on whatsapp all the time. Sure it's all illegal (in a lot of countries), but people are bored and drove that road 1000x.
> This is not how many people drive in non-autonomous cars?
No, that is not how people drive in non-autonomous cars. The phenomenon you're referring to is related to how sparsely
information is stored to memory when doing a routine task (even when you were fully concentrating). Just because you can't recall an activity in detail does not mean you were not paying full attention.
I agree with what you are saying, and that it would be very dangerous to sell cars where drivers may be lulled into complacency of thinking they don't need to pay attention to the road, until self driving cars get good enough that human attention really isn't needed.
However, these are test vehicles. The driver's full-time job is to be focused on the road and what the car is doing. They shouldn't have any misconceptions about the need for them to stay focused on the road. Now granted, even then attention will lapse occasionally, just like it does when you are actually driving. But I don't think that being physically in control of the wheel is necessary for maintaining focus at the same level as a good driver. Driver's Ed instructors need this skill. Also some back seat drivers I know are quite good at maintaining strong focus on the road regardless of who is driving :)
It's not about misconceptions, humans just aren't capable of it. Maintaining concentration when you're actively taking part in a task is doable, but maintaining concentration when you haven't had to do _anything_ for the past two hours is not. It isn't about whether the driver believes they should be paying attention or not, it's just not how human attention works.
How about autonomous on the highway, turnpikes, etc. And human-operated on streets where a car might ever need to yield to pedestrians or cyclists outside of red-lights.
Certainly, humans will never be perfect but there is the wicked problem of how we interact with systems that behave correctly 99% of the time but require human attention the other 1%. There was an excellent Econ Talk podcast about this topic a while ago:
However, in my opinion, the driver and by extension his employer Uber must be held responsible in this case. Uber have a license to test their self-driving cars only if there is a human driver in control at all times. Regardless of the crazy things their software decides to do, the human is there as a person of ultimate legal liability.
The only way this is a reasonable failure of both the computer and the human driver is if they both physically had no time to react. Maybe that was the case, maybe it wasn't.
> I'm very curious to see how they'll investigate this and who will be determined to be at fault (person behind the wheel or Uber). It will likely set a precedent.
It's also plausible that the pedestrian was at fault. Under some circumstances it would be impossible for a human driver to avoid collision with a pedestrian, this may be no different.
Motor vehicle occupants are much more likely to survive than pedestrians and cyclists are. But blaming victims (especially dead ones) has a long and storied history as far as law enforcement and the road.
Hopefully with self driving vehicles we can start to move beyond this attitude, especially when there should be ample camera footage available to help dispel the usual claims that the person struck "came out of nowhere" or whatever.
" Obviously more information is needed, but I thought the entire point of having a driver behind the wheel is to manually intervene to prevent this very situation?"
I think it's time to point out the obvious, and require that autonomous cars apply the brakes first, and THEN require driver intervention.
And that they be a whole lot quicker to err on the side of braking.
Cameras getting fuzzy ?
Slow down.
Your ML algorithms are showing lower confidence measures for how they classify nearby objects and trajectories ?
Slow down.
nearby vehicles slowing down and you don't know why?
Nothing I love more than HackerNews armchair engineering.
It's not that simple, you're assuming the car even had some indication that something was wrong. For all we know the car's vision was showing high confidence it saw an open road.
If humans can't prevent themselves from having accidents 100% of the time when they're in total control of the vehicle, why should we expect they could prevent an accident 100% of the time when they're in partial control of the vehicle?
Uber's self-driving program seems to be one of the most unethical, most poorly-run in the entire industry. There were plenty of reports before warning us of the questionable quality of their research: cars changing lanes suddenly, blowing red lights, and now one finally killed someone. Every prior incident should have been treated as a serious matter.
Waymo et al has had none of these issues. Time to revoke their licence to test?
Nobody is suggesting they're doing that. They're saying Uber cuts corners to save a dime among other unethical practices (like stealing research from competitors).
Uber is embodiment of its employees. It is a flame that the unethical moths are drawn to, so they can realize their vision by the means that delight them. Uber is the Phoenix of Enron.
Not intentionally programming them to run red lights, just intentionally or unintentionally not taking necessary precautions to prevent it from happening. They can get it right, or have it right now, and it's pretty clear which side they've chosen.
Not intentionally doing that, but, intentionally ignoring bugs.
"Sir, we have several bugs, one of them would cause the car to run a red in the following conditions... [fictional made-up condition: in case it sees a clown car with Brad Pitt wearing a pink dress in it]".
"What are the chances of that ever happening? Ship it!".
I somewhat doubt they even know or care that there might be bugs before they do something awful. It's like they're using the public highway as their simulator.
I don't know if this job ad for a self-driving simulation software engineer is still current, but it is somewhat worrying given the broad scope of the role they appear to be advertising for.
They did manage to get some local governments to do that, yes. I’m not sure how to interpret your comment other than as an apology/excuse for their illegal and unethical behavior.
If there’s more to your thought than “the ends justify the means,” I’m curious what it is.
Uber has been rewarded for employing schticks that GP called "unethical and poorly-run." Is it unethical? Yes, and I strongly believe that "Move fast and break things" is wholly inappropriate when people's lives are involved. Is it poorly-run? I think Uber is actually well-run if you accept that "ends justify the means" is in their DNA.
What would compel me to accept that an ethical standard that can be used to justify any amount of immoral and unethical behavior is “in a company’s DNA?” Why would I grade them on that curve?
This is like saying the Duterte is a very effective anti-drug crusader if you accept that murdering innocents and drug dealers is in his DNA.
We do not yet know whether another autonomous vehicle or human for would have done better in this situation. THAT SAID, asshole company fields asshole autonomous driver. Did we expect differently? Uber cuts every corner.
It means the cars are sufficiently dangerous in general that the human drivers have to constantly take over manual control to avoid accidents. So they are dangerous to someone. Apparently the human didn't make it in time this time, and turns out that someone was 'pedestrians/bicyclists'.
a) what % of disengagements are 'to avoid accidents' vs navigating an entirely safe but complex situations. For ex: a parked UPS truck on a narrow 2 lane rd.
b) the degree of real-world complexity the tests the cars are engaging in (compared to other vendors). For ex: testing cars on simple suburban routes = fewer disengagements.
Not trying to defend Uber here, I'm just trying to understand the distinction between a high-level general statistic and it's real world implications.
A complex situation == a dangerous one for a prototype system. If it's complex, the car doesn't know what to do or has any reliability, and it makes the situation dangerous even if it were 'complex but safe' for some hypothetical human driver. The uncertainty is in the map, not the territory. Waymo checks disengagements in simulators to see how many are truly dangerous, and most aren't; Uber, for some reason, declines to discuss whether they do so and what the results are. Much like they declined to tell the truth about incidents like running a red light at high speed.
There is no reason whatsoever to believe that Uber's horrific disengagement record, which is orders of magnitude worse than competitors often in the same cities like SF, is because they are tackling orders of magnitude harder situations per mile or are orders of magnitude more conservative, and every reason, even before this fatality (the only one so far despite many self-driving programs running concurrently over years), to believe they were just plain worse.
> Not trying to defend Uber here, I'm just trying to understand the distinction between a high-level general statistic and it's real world implications.
Again, it has to be dangerous for someone. It can't be more dangerous yet not dangerous to anyone in particular. That just doesn't make any sense. Uber runs on the same roads and the same traffic with the same basic approach as everyone else, there's no confounder which could produce a reversal.
Yup - I've personally witnessed one of these almost zoom through a red light at an intersection and was appalled. I'm honestly not surprised...in their effort to race to the bottom Uber's autonomous vehicle program has been extremely aggressive - to the point of total disregard for safety. Their business model may be built on pipe dreams of autonomous travel (without drivers to pay), and hopefully this shutters that. In fact, the whole company should probably be shut down IMO - but I digress.
It had nothing to do with the $150, but part of accepting the permit is taking legal responsibility to report disengagement from autonomous testing. Uber didn't want to share this information since it would make them look bad.
Agreed. I've been in the cars in Pittsburg and the operators laughed about how poorly it performed on certain roads. Slanted telephone polls caused the vehicle to literally slalom down the road.
We should wait for the results of investigation first. It could easily happen the killed person did some bonehead move that resulted in unavoidable crash. I was once told how a friend of mine fatally hit an older man that decided to cross the highway quickly, giving him no chance to react and a life-long trauma.
Calls to wait for facts never stopped the internet outrage machine in the early moments of news release. I doubt it will do much here, sadly. But it's worth pushing back against, even if it won't work IMO.
People will always relish in their biases being confirmed... even after a rare one-off event with a sample size of 1 in a burgeoning industry.
Not sure how that regulation would have made their cars safer in a way that's relevant to this accident?
Local governments have stopped other self-driving companies before with cease-and-desist letters with threats of serious punitive fines. I highly doubt by not buying a $150 license Uber would suddenly have carte-blanche to do what they want, that's not how it works.
If the state gov was really interested in stopping them they easily could have regardless.
> Waymo et al has had none of these issues. Time to revoke their licence to test?
I think Alphabet is to blame here too in a sense (or at least the industry as a whole). Why didn't they push for more strict testing rules? It's in their interest too.
This is how cars came to be on the road in the first place. Despite the fact that they were erratic and deadly, and spewed noxious gas everywhere, the manufacturers and automobiling clubs succeeded in convincing everybody that roads were for cars. Uber may well be the worst actor in the self-driving industry, but the industry as a whole is going to need to invent new crimes akin to jaywalking. This will allow it to shift blame to those killed by autonomous vehicles, just as jaywalking implies pedestrians are to blame for being hit by manned vehicles.
Striking a pedestrian in a crosswalk is horrible and terrible news. The reality though is how many pedestrians are hit by manual driver's vs automous drivers? On a percentage basis I gotta believe autonomous cars are orders of magnitude safer. Self driving cars aren't going to be perfect. How many people lost their lives in early factories to machines during the industral revolution? Imagine if they had pulled the plugs back then.
Fine/sue the entity that made the algorithm, just like every other situation like that. Or, if you can prove negligence on the part of someone inside that company, prosecute them.
>What do you do with an alogrithm to provide justice?
Assuming Arizona hasn't produced law specifically for fatalities in their public road self driving test program... The algorithms are just one detail of the total system design, and the design of the system isn't really the issue. People don't get killed by designs or code.
The verification of it's safety, and the decision to deploy that system on uncontrolled public streets will be the issue. People made these decisions, not an algorithm.
It appears the woman was not using the crosswalk, but really that doesn't make much difference in the reaction to the report. Uber's self driving tech seems a bit under-baked at the moment.
I always take the "outside the crosswalk" reports with a grain of salt. Hereabouts, if the pedestrian lands outside the crosswalk, they're assumed to have been crossing outside of it, even if later eyewitness reports and video footage show they were in the crosswalk and the car knocked them out of it.
There is also the problem that in some areas (e.g. California) all intersections are crosswalks even if unlabeled. It's likely that most pedestrians cross in a crosswalk whether they know it or not.
Arizona has something similar:
> By legal definition, there are three or more crosswalks at every intersection whether marked or unmarked.
Source: ADOT Traffic Engineering Guidelines and Processes, Section 910.1
If that's something the software can't handle well then it's going to be a huge problem in many parts of the world. In many places in Europe, Africa and Asia for instance pedestrians will cross anywhere and everywhere at any time.
Does it really need to be said that an autonomous vehicle still needs to manage to not kill people regardless of their position relative to any nearby crosswalks?
Today you can cross in the middle of the street and take a calculated risk that the people approaching down the street will stop or slow down enough to not kill you. If they go ahead and kill you anyway, they'll still be prosecuted because their attention should be on the road, and the most basic assumption behind their driver's license is that this person is competent enough and fit enough to drive what amounts to a deadly weapon without killing the people around them. When you violate that assumption, the fault is probably on you. Obviously there are exceptions, such as when somebody intentionally jumps in front of your vehicle, but crossing outside of the crosswalk to get across the street is not even close to the same level as trying to commit suicide and fault will be found accordingly.
> Does it really need to be said that an autonomous vehicle still needs to manage to not kill people regardless of their position relative to any nearby crosswalks?
If you require 100% impossibility of killing anybody regardless of what the vehicle and the person is doing - it is achievable only by making those vehicles nearly useless - such as lowering their max speed to something 10 mph (maybe even lower since it's still possible to push a person who will slip, hit their head on the pavement and die). If the vehicle is moving fast and somebody jumps onto a street, there are physical limitation of what can be done. So, if this technology is to exist there will always be a space where accidents can - and eventually will - happen.
I don’t know the number of hours their system has been driving but I highly doubt it is enough to draw comparisons to the average Arizona driver who hits 0 pedestrians. I also don’t think this works like this in the court of public opinion anyways.
This system can never make a mistake of this magnitude and be publically accepted.
>The reality though is how many pedestrians are hit by manual driver's vs automous drivers? On a percentage basis I gotta believe autonomous cars are orders of magnitude safer.
That might be a fair question, except that one death at this early stage of limited realistic trials makes me doubt the correctness of your belief. On the statistics that we have right now, I'm not seeing how autonomous cars are safer.
>Self driving cars aren't going to be perfect.
Shouldn't self-driving cars be held to a better standard rather than compared generously to objectively terrible existing standards? That we think of pedestrians regularly killed in "accidents" as acceptable collateral is already pretty horrific.
Our society has evolved quite a bit from the times of the Industrial Revolution. It’s possible for progress to be made while having life-saving regulations.
> On a percentage basis I gotta believe autonomous cars are orders of magnitude safer.
Why?
NHTSA reports 1.15 fatalities per 100 million vehicle miles travelled in 2015.
Waymo advertises >5 million road miles travelled. Let's say Waymo + Uber have driven 10 million vehicle miles and killed 1 person. That makes them 10 times more dangerous than human driven cars.
what about tesla though? by november 2016 they've accumulated 1.3 billion self droven miles [1]. autopilot was released in september 2014, so that's 2 years, or 600 million miles per year. and that was ramping up during those years, so i think it's safe to assume at least 1 billion miles / year these days.
2 deaths in over 2 billion miles is roughly 10 times safer than human drivers.
Absolutely true, but keep in mind as well that each of these companies is running a completely different tech stack, for different use cases. They can't be lumped together for comparison.
Autopilot is not self-driving. The autopilot mode disengages if your hands are not on the wheel. Furthermore, users are advised to only use it in relatively safe situations.
I don't think that is a fair comparison/calculation, autopilot is not "autonomous" (or not as autonomous as the Uber experimental car involved in this crash), and the demography of Tesla drivers is most probably much off "generic" car drivers, and driven miles, which include any kind of car (in the sense of including older cars in bad maintenance state or simply with inferior braking and steering systems when compared to brand new cars) and any kind of drivers (in the sense of drivers at a higher risk of accident).
This is the important statistic. Evidence so far points to autonomous drivers being worse than human drivers. In which they definitely have no place on the streets without a lot more R&D.
"The Uber vehicle was reportedly driving early Monday when a woman walking outside of the crosswalk was struck.
...
Tempe Police says the vehicle was in autonomous mode at the time of the crash and a vehicle operator was also behind the wheel."
That's a very bad look; the whole point of self-driving cars is that they can react to unexpected circumstances much more quickly than a human operator, such as when someone walks out into the road. Sounds like Uber's platform may not be up to that standard yet, which makes me wonder why they're on public roads.
On the other hand, it sounds like it happened very recently; I guess we'll have to wait and see what happened.
They will have video for sure if since they are testing, then we will see. Your statement about reaction time is assuming so many things, may as well declare the AI guilty now right?
It's programmed by humans. Get over it. It's not intelligent, it's not "learning" it's following instructions, saving data, and iterating. Your comment is obvious to the point of meaninglessness.
This tendency to buzzword out of obvious truths (NN's are programs for example) derails rational discussion.
Doing these things derails discussion much faster than an obvious comment, so would you please (re-)read the rules and abide by the spirit of this site when posting here?
nope, its trained as a neural network. Nobody ever sits down and programs in 'how to handle an intersection with a bike lane' other than putting that sort of driving experience into its training data. It may have modules designed for general scenarios (i.e. one module is great at freeway driving, another is great at intersections etc) but still trained by experience.
It is programmed by humans. Machine learning algorithms are designed and implemented by humans. If machine learning algorithms do not perform well in a way that puts human lives at risk, the humans who designed and implemented them are responsible.
I imagine these vehicles (especially test vehicles) are also recording regular video and therefore getting a clear picture of what happened should be straight forward.
That kind of thing might fly in some arenas, but judges have literally no sense of humor when evidence is “lost” in a criminal case, or civil wrongful death suit. That’s more or less a quick way to lose in the worst possible way.
Judge - you don't understand - the office that's responsible for this remote deletio^w securing of data is in another jurisdiction (Canada!). So Uber USA is absolved[1].
As I said, context matters. In the context of a death, this kind of thing is doom. When someone is killed, there is also a lot of cooperation across jurisdictions.
I can guarantee you (at least one) other major tech companies have the same system in place. Companies that operate in multiple jurisdictions don’t want all of their assets compromised because of a raid in one city/state/country.
E.g. Brazil was at odds with WhatsApp over their message encryption — Brazil wanted WhatsApp to stop encrypting messages so that it would be possible to subpoena chat logs. If Brazilian police raided a hypothetical WhatsApp Brazil office, would it not be prudent for Facebook to cut off data access from that office? Especially given that the nature of technology means that once police have an employee’s laptop and can compel them to enter their password, they have access to essentially all of the company’s data worldwide.
I agree completely. One thing that is important to me is that the whole self-driving field will learn from every mistake. In other words, every self-driving car should only make the same mistake once, whereas with humans, each human has to learn only from their own mistakes.
That isn't correct. Culture is learning from other people's mistakes. The only question is whether the human can and is willing to accept the lessons (and whether the lessons are correct).
Culture is comprised of many things, and some similar things, but not that. By definition, culture is merely a collective persistence, and therefor inherently superficial. Culture cannot capacitate “learning” or even lessons. Culture is limited to memetic abstractions, with a lot of alleged noise. Insofar as culture supports persistence of worthwhile norms, it would be limited to evolutionary adaptation (survival of the fittest). It’s important to distinguish to avoid the fallacy of argumentum ad populum.
Making sharing of all sensor data of an accident and the preceding couple of minutes mandatory could be a simple regulation that would be very helpful for all companies trying to improve their algorithms.
I disagree. I would be astounded if Waymo, Tesla, and every other player in the field isn't already trying to craft 1000 new test cases about this incident. It's now the second(?) known death involving this technology. The first question their bosses will be asking them in their next meeting would be "would our cars have done this, too?"
If there's one company who hasn't demonstrated it's learned from its mistakes it's Uber.
But let's extrapolate. Say one day there are 20 self driving car companies. Should they be required to share what they learn so the same mistakes aren't repeated by each company or does the competitive advantage outweigh the public benefit from this type of information sharing?
"recently" is a pretty big caveat to throw in there. Historically Uber hasn't changed any of their behaviour until they've been caught doing something. Let's wait and see how they react to something like this.
Good point, but I added that in because I think recent is the most important for trying to know if something learns from its mistakes.
They seem to be adjusting quite a bit in the last 1-2 years so that is the definition of learning. Otherwise, just showing how companies were once dumb is usually not very useful in knowing how adaptable they are.
I agree that we’ll know better looking back how they react to this event.
Airlines are the same way. Every time there’s been a crash they’ve learned something and changed procedures to prevent it again. It’s made flying pretty safe, unless you’re an animal flying on United...
I have seen no evidence that any autonomous vehicle currently deployed can react faster than an alert and aware human. The commentariat tends to imagine that they can, and it's certainly plausible that they may eventually be. But I've never seen anyone official claim it, and the cars certainly don't drive as though they can quickly understand and respond to situations.
> I have seen no evidence that any autonomous vehicle currently deployed can react faster than an alert and aware human.
The argument I've always heard is that an autonomous systems will outperform humans mostly by being more vigilant (not getting distracted, sleepy, etc.) rather than using detectors with superhuman reaction times. Obviously, whether or not this outweighs the frequency of situations where the autonomous system gets confused when a human would not is an empirical question that will change as the tech improves.
Lots of people imagine both: that the system will never be distracted and also that it will have superhuman reaction time.
And, like, once the systems are mature and hardware has evolved and so forth, I think that's right. It will. It might today under certain circumstances, like if it gets a really unambiguous sensor input (or it might not).
But I've never heard anyone actually associated with a driverless car program assert that their vehicles have superhuman reaction times today, and the vehicles drive extremely cautiously. I think it's likely that due to their difficulty in understanding their sensor readings, if you look at total time necessary to make a course correction from the point when an obstruction first could be noticed by an alert human driver, driverless cars are not winning and may be substantially losing in at least some cases.
"Alert and aware human" is already a high standard, given how most humans drive in routine mode, which is well understood to be much worse than "alert and aware".
From what I've seen I wouldn't trust autonomous cars to "understand" all situations. I would trust Waymo cars to understand enough to avoid hitting anything (at a level better than a human), at the risk of being rear-ended more often. Everything I've seen from Tesla and Uber has given me significantly less confidence than that.
I live and commute in Waymo country, and see evidence of quick reactions, though I can't say for sure whether it's an alert human taking over. Mostly, the Waymo vehicles still drive conservatively.
Not a truly autonomous vehicle example but this is a case where most likely the car reacted before the driver was even aware of a problem: https://www.youtube.com/watch?v=APnN2mClkmk
I agree with the sentiment though. This has been a major selling point for this technology, but it has not been sufficiently demonstrated yet.
From the video, it looks like the car noticed the crash at the same time a human would, but the car decelerated immediately, whereas the human might have slower reaction time.
(It certainly didn't predict the crash "seconds before", as the car accelerated into the imminent colission, until <1sec before collision.
Humans correctly gauge stopping distance and "activate appropriately" like tens or hundreds of billions of times per day. Try again.
Something that is endemic in HN discussions of driverless cars is commenters who dramatically overestimate the dangers of humans driving. Like an order of magnitude or more. You see it all over this thread, in which people imagine that killing one pedestrian in like let's say 10 to 20 million miles driven (with those miles overwhelmingly done in unchallenging conditions) constitutes "vastly safer than human drivers" rather than "vastly less safe than human drivers."
Driverless cars simply don't have the mileage yet to prove they are safer than driverful cars. But there's also no indication they are less safe, based on casualty rate so far.
Waymo said that they'd hit 4 million miles back in Nov 2017, they seem like they've done the most miles, there are several other contenders, so I took a wild guess, trying to err on the side of overestimating # of miles.
There is an indication that they're less safe! They have (very conservatively) 5x the number fatalities per mile driven! Now, look, the error bars on that estimate are of course massive. It is plausible that they are much safer and they just rolled the dice and got unlucky. But this is data, as long as you include the error bars.
> Humans correctly gauge stopping distance and "activate appropriately"
Toyota has enough UX data that they added "brake assist", turns out that a lot of accidents happen when the user stomps the brake, but then releases, just never presses it hard enough to come to a complete stop, or was trained before anti lock brakes.
Another scary thing about braking is that most cars probably don't have their brake fluid changed often enough; most owners and even dealers treat it as a lifetime interval, when it may be closer to an annual thing interval.
> That's a very bad look; the whole point of self-driving cars is that they can react to unexpected circumstances much more quickly than a human operator, such as someone walking out into the road. Sounds like Uber's platform may not be up to that standard yet, which makes me wonder why they're on public roads.
That's an invalid conclusion to draw from this accident.
There were 34,439 fatal motor vehicle crashes in the United States in 2016 in which 37,461 deaths occurred. This resulted in 11.6 deaths per 100,000 people and 1.16 deaths per 100 million miles traveled.[1] As far as I know, the instance this article is about is the first death in an autonomous vehicle accident, meaning that all the 2016 accidents were humans driving.
Why is it that you see one death from an autonomous car and conclude that autonomous cars aren't ready to be driving, but you see 37,461 deaths from human drivers and don't conclude that humans aren't ready to be driving?
I admit that there just aren't enough autonomous cars on the road to prove conclusively that autonomous cars are safer than human-operated cars at this point. But there's absolutely no statistical evidence I know of that indicates the opposite.
EDIT: Let's be clear here: I'm not saying autonomous cars are safer than human drivers. I'm saying that one can't simply look at one death caused by an autonomous car and conclude that autonomous cars are less safe.
> don't conclude that humans aren't ready to be driving?
I'm not sure I see anyone here making that conclusion. I think you're the only one who's brought it up.
For one, I personally sure as fuck do not think humans are ready to be driving.
> That's an invalid conclusion to draw from this accident.
The conclusion was not invalid at all. Other self-driving car companies have driven more miles than Uber has and they have done so safely. Uber has even taken its cars off the road, so even Uber agrees that their self-driving cars are not ready for the roads yet.
It is also important to take into account what Uber is like when it comes to safety and responsibility. They have knowingly hired convicted rapists for drivers, they have ridiculed and mocked and attempted to discredit their paying customers who have been raped by their employees/drivers, they have spied on journalists, they have completely disregarded human safety on numerous accounts. A company with a track record like Uber's probably not should be granted a license to experiment with technology like these self-driving cars on public roads.
> I'm not sure I see anyone here making that conclusion. I think you're the only one who's brought it up.
Look at the post I'm responding to, and the section I quoted.
> Uber has even taken its cars off the road, so even Uber agrees that their self-driving cars are not ready for the roads yet.
Uber avoiding a PR nightmare should not be taken as Uber thinking their self-driving cars aren't ready.
> It is also important to take into account what Uber is like when it comes to safety and responsibility. They have knowingly hired convicted rapists for drivers, they have ridiculed and mocked and attempted to discredit their paying customers who have been raped by their employees/drivers, they have spied on journalists, they have completely disregarded human safety on numerous accounts. A company with a track record like Uber's probably not should be granted a license to experiment with technology like these self-driving cars on public roads.
I'm not defending Uber as a whole. I think in general they're a fairly typical rent-seeking middle man whose only real innovation has been figuring out a way to break the law ambiguously enough to get away with it (admittedly, a law I don't agree with). I don't even necessarily think autonomous vehicles are a good thing. I just think that if you're going to criticize autonomous vehicles, safety isn't the criticism I'd level against them, because it's not backed up by the evidence when compared to how unsafe human drivers are.
He's arguing from foundations. The hypothesis is that autonomous drivers should perform better at task X than the null hypothesis (aka human drivers). So any instances where autonomous drivers do not seem to perform better are all potential counter-arguments to that hypothesis.
The fact that human drivers aren't particularly good isn't really relevant, beyond setting a correspondingly low bar within the null hypothesis.
This all ties to regulation allowing these vehicles to drive on public roads, because that regulation was permitted due to the above hypothesis and hopeful expectations that it would be true.
Obviously, I haven't seen the entirety of the data set to know fatalities per car-mile. Which would be the relevant statistic here. I also didn't see such a number in your post, which I'm assuming means you are probably not aware either. But simply providing the numbers for the null hypothesis doesn't do anything.
> He's arguing from foundations. The hypothesis is that autonomous drivers should perform better at task X than the null hypothesis (aka human drivers). So any instances where autonomous drivers do not seem to perform better are all potential counter-arguments to that hypothesis.
> The fact that human drivers aren't particularly good isn't really relevant, beyond setting a correspondingly low bar within the null hypothesis.
These two paragraphs are so contradictory to each other that I'm just not even sure how to attempt to correct your confusion.
You'll have to make an attempt, because I don't see anything contradictory. If human drivers are your null hypothesis, than you cannot use the fact that "humans are bad" as a blanket acceptance of autonomous vehicles. They are already built into the equation, by virtue of being the null hypothesis. So you can argue that hypothesis, but that's not what your comment did. Your comment stated some numbers for the null hypothesis, but had no numbers for the hypothesis under test, and so therefore didn't really mean much at all.
I'd like to note that I am not the person you replied to, and I personally am not arguing that any program should be shut down based on this one incident. But it's certainly not encouraging that the autonomous vehicle seems to have failed a test at which everyone would have expected it to perform well at.
Also, to quote from your previous post:
> Why is it that you see one death from an autonomous car and conclude that autonomous cars aren't ready to be driving, but you see 37,461 deaths from human drivers and don't conclude that humans aren't ready to be driving?
I think we conclude, quite often in fact, that individual humans aren't fit to be driving. Death is one of those scenarios that will quickly lead to such a conclusion.
One huge difference between individual humans and autonomous vehicles is that we can reasonably argue that any Uber vehicle would have performed the same in this scenario. So this is perhaps more akin to saying that this particular driver is not fit for the task, except that this particular driver happens to be driving dozens or more vehicles all at once.
> You'll have to make an attempt, because I don't see anything contradictory. If human drivers are your null hypothesis, than you cannot use the fact that "humans are bad" as a blanket acceptance of autonomous vehicles.
I said, "I admit that there just aren't enough autonomous cars on the road to prove conclusively that autonomous cars are safer than human-operated cars at this point." and I've vocally criticized autonomous cars elsewhere, so I'm not sure where you get the idea that I favor a blanket acceptance of autonomous vehicles.
I'm not saying that autonomous vehicles are safer, I'm saying this isn't evidence that autonomous vehicles are less safe.
> One huge difference between individual humans and autonomous vehicles is that we can reasonably argue that any Uber vehicle would have performed the same in this scenario.
I disagree: you can argue that the autonomous vehicles will behave the same given the same inputs, but they will never have exactly the same inputs even if the situation were identical to a human observer, so that's a fairly moot point. If you step back to a larger description of the situation (car crossing bike lane to get into a turn lane, car doesn't identify and avoid bicyclist in bike lane) then you are going to be looking at a percentage of the time where autonomous car will make a mistake. There's also a percentage of the time where a human driver will make the same mistake. The only way you can compare the safety autonomous cars to human drivers in this situation is to compare those percentages. And that's ignoring the fact that there are thousands of other situations in driving--even if autonomous cars fail 100% of the time in this situation, there may be enough other situations where they perform better enough than human drivers that they're safer. Simply saying that a car made a mistake in this situation doesn't give us any information at all.
> I'm not saying that autonomous vehicles are safer, I'm saying this isn't evidence that autonomous vehicles are less safe.
> And that's ignoring the fact that there are thousands of other situations in driving--even if autonomous cars fail 100% of the time in this situation, there may be enough other situations where they perform better enough than human drivers that they're safer. Simply saying that a car made a mistake in this situation doesn't give us any information at all.
But it does give us information. This incident, along with all the other incidents and non-incidents that do or do not occur, as measured by incidents per car-mile. One cannot simply wish away this one incident and make it disappear. It is now forever part of the statistics which will either prove or disprove the hypothesis of autonomous cars being safer.
> I disagree: you can argue that the autonomous vehicles will behave the same given the same inputs, but they will never have exactly the same inputs even if the situation were identical to a human observer, so that's a fairly moot point.
It's not a moot point. Yes, this exact scenario with these exact parameters only occurred once. However, we can still reasonably argue that if we reversed time and replaced that exact Uber autonomous driver with another instance of the autonomous Uber driver and turned time back on, it would have reacted exactly the same. The same way that I expect the same version of Notepad to open my text file exactly the same on this computer as on another computer. The alternative being that there is some undeterministic behavior in the driver that is not tied to input... In which case, good luck with that in court.
However, we cannot make that same argument by replacing human drivers. Because each human is, in fact, different.
This is only important in the context that a fanciful revocation of Uber's "autonomous driver's license" would apply to all instances of the autonomous driver, since they would all have been reasonably expected to perform the same.
> But it does give us information. This incident, along with all the other incidents and non-incidents that do or do not occur, as measured by incidents per car-mile. One cannot simply wish away this one incident and make it disappear. It is now forever part of the statistics which will either prove or disprove the hypothesis of autonomous cars being safer.
Can you point out the part of the article or the post that I was responding to which mentions how many car-miles were traveled?
This is exactly what I'm pointing out.
> The same way that I expect the same version of Notepad to open my text file exactly the same on this computer as on another computer.
Notepad doesn't have to read your text file through a lens with slightly different focus, viewing area, and patterns of dust on it each time.
> The alternative being that there is some undeterministic behavior in the driver that is not tied to input...
The undeterministic behavior is that the hardware which collects the input will never be the same. I don't know whether the software is non-deterministic (it wouldn't surprise me) but I know the hardware is never going to be identical--hardware is always made to tolerances and always has some degree of variability.
Your claim is tantamount to saying that if we put the same person in the same situation but with two different sets of eyes, the eyes would have no effect on the results.
> However, we cannot make that same argument by replacing human drivers. Because each human is, in fact, different.
Autonomous cars are, in fact, different. Just because they're running the same software doesn't mean they're the same; even if the software is completely deterministic, software is only a component of the autonomous driver.
> Can you point out the part of the article or the post that I was responding to which mentions how many car-miles were traveled?
> This is exactly what I'm pointing out.
I think, if your intent is to show that this is incomplete information, that...
1) No one is arguing that.
2) You have not done a great job of attempting to relay that, given phrases like, "Simply saying that a car made a mistake in this situation doesn't give us any information at all."
3) Sometimes that doesn't matter. For instance, Florida law is a mandatory 6 months to one year license revocation on DUI, regardless of the circumstances or information.
> Notepad doesn't have to read your text file through a lens with slightly different focus, viewing area, and patterns of dust on it each time.
Difficulty of the task is unrelated to expected outcomes of the task given the same inputs. And we already tread the topic of duplicating the exact situation... Not sure what you're trying to gain through this line of argument.
> Your claim is tantamount to saying that if we put the same person in the same situation but with two different sets of eyes, the eyes would have no effect on the results.
I am making no such claim, and I cannot believe that you are so adamant about not understanding my actual claim. This is a hypothetical situation. There is no mention in this scenario about changing the car, including any of the sensor hardware. I am interested in replacing only the driver (or driver software) into the exact same circumstance.
(EDIT: OK, reading back, I did say "any Uber vehicle". While your point stands, I think it's a very uncharitable reading. If hardware sensor tolerances and specs of dust on the camera are going to determine whether a life is lost or not, either those tolerances need to be driven down or this entire idea needs to be rethought. After all, we don't allow those who are legally blind to drive unless they have corrective lenses...)
Assuming the software is deterministic [1], by definition given the same inputs it will result in the same output. Therefore, "replacing" the autonomous driver with another would have resulted in the same incident. You cannot say that with any measure of confidence for any two pairs of human drivers.
[1] Which seems like it would be a good thing to assume, since I don't think one would get much traction by arguing that we should be putting vehicles with non-deterministic behavior on the road...
No human driver will ever receive the same input twice, but we still suspend people's licenses sometimes after a single incident. Are you arguing that we need to let Uber kill a few more pedestrians so we can more accurately determine the safety of their platform vis a vis human drivers? Why can't they fabricate some tests that demonstrate their safety in a controlled environment first?
> Are you arguing that we need to let Uber kill a few more pedestrians so we can more accurately determine the safety of their platform vis a vis human drivers?
Are you making accusations in question form so you don't have to back them up? I certainly didn't say that.
> Why can't they fabricate some tests that demonstrate their safety in a controlled environment first?
I'll admit that I'm of the GP's mind in wanting to know what you consider to be "evidence". Because you keep reiterating this sentiment:
> I'm not saying that autonomous vehicles are safer, I'm saying this isn't evidence that autonomous vehicles are less safe.
It's true that we need to wait for more information about this particular incident -- which is exactly the caveat that the commenter you initially responded to had said [0]. But assuming the facts aren't abnormally different than what they seem to be -- an Uber AV hit and killed a jaywalker, how is that not evidence toward the argument that AVs are less safe? It obviously isn't conclusive evidence. But if Uber goes on to kill a pedestrian for every million miles driven, this first data point would surely be part of the empirical evidence, no?
> Are you making accusations in question form so you don't have to back them up? I certainly didn't say that.
Nope, I'm just having trouble grokking your argument and I thought that might be it. It's true I added a rhetorical edge to the language that was probably unnecessary. I apologize for that--I'm not trying to put words in your mouth. What is your argument?
you have to take into account miles driven. Yes, we have 37K fatalities, but trillions of miles driven. So, it comes out something like 1.2 fatalities per 100 million miles driven for human drivers. Which means, so far, self driving cars have a worse record per 100 million miles driven. So far.
I haven't been able to find any reliable source of miles driven by self-driving cars to make that claim, but if one exists it wouldn't surprise me.
But that's not really my point. My point isn't that self-driving cars are safer, it's that merely looking at one car accident doesn't prove that self-driving cars are less safe.
as of today, yes, the data indicates that self driving cars are less safe. The fact is, as of today, self driving cars are generating more fatalities per mile driven than human drivers. That may change in the future as technology matures.
If solution does not solve anything, what's the point of the „solution“?
Most of the deaths on the road I hear in my country are the result of someone doing something really reckless and stupid. „Normal“ drivers do not kill themselves or others. So if self-driving car is only as good as a dumb driver — I do not want these cars on the road.
Also, interetingly enough there is little talk about human driver assisted by technology, not replaced by it. For some reason it is binary: either humand driver or self-driving. How about some human drivers plus collsion avoidance system, infrared sensing system (way too many people die there simply becaue they walk on the road in the dark without any reflectors/lights), etc.
> If solution does not solve anything, what's the point of the „solution“? Most of the deaths on the road I hear in my country are the result of someone doing something really reckless and stupid. „Normal“ drivers do not kill themselves or others. So if self-driving car is only as good as a dumb driver — I do not want these cars on the road.
If self-driving cars are between dumb drivers and normal drivers, and self-driving cars take dumb drivers off the road, it may be a net positive.
That depends how many dumb drivers it takes off the road compared to normal drivers, and the relative percentages of accidents caused by each.
Let's say we have 9 normal drivers and 1 dumb driver. The 9 normal drivers cause 0 accidents per year each, and the 1 dumb driver causes 15 accidents per year. Let's say autonomous cars cause 1 accident per year. If autonomous cars replace all the drivers, there's 10 accidents per year instead of 15.
Obviously these are hypothetical numbers. All I'm saying is that if you only have one of these numbers, you don't know which kind of driver is safer.
I guess I'm just working on anecdotes and guesswork here, but I always assumed the vast majority of drivers are pretty reasonable most of the time. I imagine if this were not the case the clusterfuck on the roads would be considerably worse than what I see in practice. It is certainly true that the dumb drivers heavily outweigh the sane ones in my memories of traffic, but I also know the details of more plane crashes than successful flights.
As of late 2017, Waymo/Google reported having driven a total of 4 million miles on roads. Given that Waymo is one of the biggest players, it's hard to see how all of autonomous cars have driven 100 million miles at this point.
Nevermind that the initial rollouts almost always involve locales in which driving conditions are relatively safe (e.g. nice weather and infrastructure). Nevermind that the vast majority of autonomous testing so far has involved the presence of a human driver, and that there have been hundreds of disengagements:
Humans may be terrible at driving, but the stats are far from being in favor of autonomous vehicles. Which is to be expected given the early stage of the tech.
My point is that "the stats" don't exist if you're only looking at one autonomous car accident without comparing it to the relevant statistics from human accidents.
If the stats don't exist then what was your whole basis for making an argument? On what grounds do you have to argue that we shouldn't be pessimistic about the state of autonomous driving? That you pulled out the IIHS stats without apparently bothering to look up the number of reported self-driving miles driven seems to suggest that you assumed that the fatality rate was obviously skewed against human driving.
That's fine, we don't have to consider just fatal accidents:
> One result of the sharp increase in GM’s number of miles driven is a plethora of accidents. The auto giant’s autonomous cars were involved in 22 fender benders over the course of the reporting period (and two more in 2018). That’s one crash for every 5,985 miles of testing.
> On what grounds do you have to argue that we shouldn't be pessimistic about the state of autonomous driving?
I didn't say that.
I'm not saying autonomous cars are safer, I'm saying that this accident doesn't prove autonomous cars are less safe.
> That you pulled out the IIHS stats without apparently bothering to look up the number of reported self-driving miles driven seems to suggest that you assumed that the fatality rate was obviously skewed against human driving.
I brought up the IIHS stats to show that merely reporting an accident in isolation from a full statistical model doesn't prove anything.
OK. And the person you responded to with the fatality stats said nothing about proving which way either:
> That's a very bad look; the whole point of self-driving cars is that they can react to unexpected circumstances much more quickly than a human operator, such as when someone walks out into the road. Sounds like Uber's platform may not be up to that standard yet, which makes me wonder why they're on public roads.
Actually there was a metric of accident per distance drived. It was often pull out by Tesla before their first fatal accident last year (IA didn’t saw rear of truck because of sun or smthg like that).
This metric was often decried because it had poor statistical significance. It would however be nice to update this metric according to this death.
Maybe now this metric would indicate that IA are more dangerous than human, will be interesting if the perception of this flawed metric will evolve in reaction...
> Actually there was a metric of accident per distance drived. It was often pull out by Tesla before their first fatal accident last year (IA didn’t saw rear of truck because of sun or smthg like that).
I'm curious to see that, do you have a link? That would actually be a valid comparison.
According to that "update", Tesla Autopilot can never be at fault right? Because the driver is supposed to be in charge. So, even if Tesla rams full speed into a semi, the driver is at fault!
That's a really good point that many (myself included) aren't mentioning. Stopping distance is a universal, reaction times be damned. Would be curious to see if that played a part.
Perhaps (self driving) cars can be redesigned to mitigate the outcome of pedestrian collisions, eg, like cowcatchers on trains [1], except something more appropriate for the situation. External airbags?
Sometimes the best reaction in this situation is not to stop, but to swerve around the obstacle, perhaps even accelerating slightly to do so quickly. Deer can be pretty unpredictable here, but most on foot humans will not put themselves into the new path of your vehicle.
> That's a very bad look; the whole point of self-driving cars is that they can react to unexpected circumstances much more quickly than a human operator, such as when someone walks out into the road.
Some of these accidents are unpreventable by the (autonomous) driver. If a pedestrian suddenly rushes out into the street or a cyclist swerves into your path, the deciding factor is often simply the coefficient of friction between your tyres and the road.
The autonomous vehicle and the human attendant might have made a glaring error, or they might have done everything correctly and still failed to prevent a fatality. It's far too early to say. It's undoubtedly a dent to the public image of autonomous vehicles, but hopefully the car's telemetry data will reveal whether this was a case of error, negligence or unavoidable tragedy.
Not obviously. Let's wait to judge until we have the facts. The pedestrian was outside of the crosswalk. There's not a whole lot of information about the events that lead up to the accident. It is possible that a pedestrian is at fault if they stepped in the way of moving traffic outside of a crosswalk. It's also possible Uber's cars are not up to the task of driving on public roadways.
Arizona requires drivers to "exercise due care to avoid colliding with a pedestrian". If a fatality results, but due care was taken by the driver to avoid it, then the tragic accident is just that, an accident.
Like my comment says, let's wait until we have facts before passing judgement.
But from this distance, things don't look good for the technology or the future of autonomous trials on public roads.
The evidence that due care was not taken will pretty much be the existence of the fatality.
There surely will be video, from the car itself and also perhaps third party security or traffic cameras. The level of carelessness we are going to have to see in order for a jury to blame the victim will be pretty high.
In law you split that into criminal and civil. Very unlikely there will be any criminal charge, hence no criminal liability but it’s possible.
Civil liability is more interesting, but if the deceased’s estate brings suit it opens up a can of worms in the discovery process. Every internal communication, prior incident, etc...
At the end of the day it’s not much different than any other vehicular fatality case, but there will be a defective product component (again nothing unheard of in the case of vehicular fatalities). At the end of the day the car will be insured for accidents, so we will probably learn more about the insurance coverage for self-driving cars.
Exactly, you can never reduce accident rates to zero, even if you slow to a crawl or stop (now you're endanger traffic behind you). Debris could fall on your car and make the steering non-responsive. A flat tire could burst at any speed and cause a bicyclist to veer into traffic/off a cliff/whatever.
When it comes to human drivers we deal with probabilities, but for some reason people want absolutes with autonomous ones.
If pedestrians or especially children are present, one should be driving very slowly. 15mph sounds about right. At that speed one is unlikely to kill, since even if a pedestrian "jumps out" (an event that is vanishingly less common than drivers not paying attention), one has enough time to stop.
Most drivers drive too fast. (Me too!) Most drivers have not killed anyone. All drivers who have killed someone, were driving too fast at the time. Many drivers will disagree with me, but they are simply wrong about appropriate speeds, and we will all be safer when robocars are driving for them.
Even when the minimum braking distance is not small enough to avoid hitting a pedestrian, a lower speed will impart a lower kinetic energy vastly reducing the odds of fatally injuring the victim.
This statement is either very poorly thought out, or needlessly accusatory and inflammatory.
I would say that if you claim that a massively complex system like public traffic, with thousands of participants, many of them badly or completely untrained, can be organized in such a fashion that any and all accidents can be prevented, the burden falls on you to show how.
This subthread is not about the massively complex system. Rather we are discussing the good judgment of individual drivers. Even if my statement were not literally true (the best kind, and it literally is), it would still be good practice for all drivers to commit themselves to driving slowly enough to prevent collisions with pedestrians.
Sure, but in the above user's hypothetical, that would mean that in such an area a concerned human driver with a greater ability to predict general human behavior would have a statistical safety improvement on the autonomous vehicle, which might not understand, for instance, that since it's a Friday night and the big game just ended and I'm in the city center I should be more careful than usual because there will be more intoxicated people.
Which is a lot of high level reasoning and inference with information from a variety of sources which aren't on the face of it strictly related to the driving task.
This is the reason robocar firms will fight to keep all their data private. If it were public, researchers could show and personal injury attorneys could argue persuasively that there are some speeds that would never cause fatal collisions. Since those speeds will be slower than most passengers wish to travel, this mode of travel will be more vulnerable to lawsuits.
I’m kind of in the Elon Musk camp here where you gotta break some eggs to make an omelette? Human-driven cars kill a lot of pedestrians today, but we can actually do something to improve the human-recognition algorithms in a self-driving car.
As long as self driving cars represent an improvement over human drivers, I’m ok with them having a non-zero accident date while we work out the kinks.
Omelet metaphor aside,that doesn't really seem fair. We're talking about accepting more than 0 deaths caused by AI because the death rate should still be much lower than with humans driving. IOW, the perfect is the enemy if the good.
The problem is the blame game. Human behind a wheel hits a pedestrian, if they're found at fault then that "horrible inattentive/drunk/whatever driver" goes to jail for the death of another human being. It never makes anything "right", and I won't even start on how a jail sentence regardless of length can ruin your life in the US, but the public as a whole gets a feel good that "justice has been served".
How do we handle this for autonomous vehicles? Do we just fine/sue the company that made the vehicle/developed the software? Do we send imperfect human developers to jail because they made a mistake, even if in the grand scheme of things they have saved lives compared to humans being behind every action made by a vehicle?
A big part of the public image for autonomous cars is increased safety, any deaths at their hands starts raising where and how to place the blame - a subject I think very few are prepared for right now, which is likely part of why Tesla explicitly states autopilot needs a human driver present right now, and why Google has been extremely cautious with operator-supervised tests up until recently.
But is it any surprise that Uber had the first self driving car to kill a human being? You can cook in an orderly, careful way, or you can turn the kitchen into a disaster zone and expect other people to clean up for you.
I know which I prefer when it comes to human lives.
While I agree that the AV safety record is better than the human one, how is "you gotta kill random bystanders to make an omelette" okay just for this one industry? (As opposed to e.g. medical or military testing)
Military testing is typically done by putting a few pieces of tested equipment in the field. Russia is doing exactly that in Syria with their new Su-57. Pretty sure it's where a few other of the planes got a first taste of live combat, that's part of testing. I'm sure it'll either cause some unintended fatalities.
And, medicine takes tens of thousands of lives unintentionally, if I recall correctly.
Not at all. You had mentioned military testing, which is safe in early stages, and I addressed that first: the testing process does follow through into live environments. Where things can go wrong.
But no, medical testing doesn't kill tens of thousands, practiced medicine does. I may have read your post wrong an figured you meant medicine in practice separately from military testing. There's research to suggest mistakes are the third leading cause of death in the US [1]. My point wasn't so much that killing random bystanders is okay, but that there's a level of unintended death in both, and not just in testing. Society decides what's acceptable, and if for miles driven the number of deaths that self driving vehicles cause is lower than the number of deaths caused by human drivers per miles driven, well... I'd say that's a fair way to look at it.
There's no way for autonomous driving to become a reality without people dying?
I think we could achieve autonomous driving without needless deaths along the way. It might take longer, but I'd say it's worth it to, I can't believe I have to make this argument, avoid killing people.
you seem to be giving uber a big benefit of the doubt. these autonomous cars generally go slow. tempe has flat roads with great line of sight and clear weather. coefficient of friction? highly doubt it. the sensors should be looking more than just straight a head
It's not unreasonable for there to an expectation of basically zero accidents of this nature during testing in cities. The public puts a huge amount of trust in private companies when they do this. And, pragmatically, Google, Uber, etc, all know that it would be horrible publicity for something like this to happen. One would think they'd be overly cautious and conservative to avoid even the possibility of this.
Lastly, the whole point of the human operator is to be the final safety check.
You're right that we have no idea of the cause until the data is analyzed (and the human operator interviewed). Yet, my first thought was, "Of course it'd be Uber."
>If a pedestrian suddenly rushes out into the street or a cyclist swerves into your path, the deciding factor is often simply the coefficient of friction between your tyres and the road.
This only true for the uninitiated, never let it get to that. I drove for Uber/Lyft/Via in New York city so I can experience and study these situation. These sort of accidents are preventable. The following is the basic:
1.) Drive much slower in areas where a pedestrian and cyclist can suddenly cross your past.
2.) Know the danger zone. Before people "jump into traffic" or a cyclist swerve in front of you, they have to get into position, this position is the danger zone.
3.) Extra diligence in surveying the area/danger zone to predict a potential accident.
4.) Make up for the reduce speed by using highways and parkways as much as possible.
It helps that Manhattan street traffic tends to be very slow to begin with. Ideally I will like use my knowledge to offer a service to help train autonomous vehicles to deal with these situation. It has to be simulated numerous times in a closed circuit for the machine to learn what I've learn intuitively driving professionally in NYC.
You're making an observation about what it's like in an urban area. Sadly, in a suburban area like Tempe the presence of any pedestrian anywhere is unexpected.
The ASU campus area around Tempe has diverse traffic conditions. There are spots near the campus where you would expect to stop at every crosswalk, while in some roadways, a pedestrian would be completely unexpected.
Around the bars, people stumble into traffic all the time. Any driver, automated or human, would need to anticipate that.
> 1.) Drive much slower in areas where a pedestrian and cyclist can suddenly cross your past.
So essentially everywhere? I have seen pedestrians walk unexpectedly into traffic on high-speed 4-lane divided roads with no crosswalks.
> 2.) Know the danger zone. Before people "jump into traffic" or a cyclist swerve in front of you, they have to get into position, this position is the danger zone.
What "position" are you referring to? There are places where you can account for or predict pedestrians. There are also places where you cannot, such as when someone walks into traffic from behind a tall parked vehicle, where you have no chance to see them in advance.
Simply counting traffic fatalities suggests that crazy pedestrians causing unavoidable accidents cannot be common, even if every single pedestrian accident were both unavoidable and the pedestrians "fault" (although I'd argue that ethically it must be primarily the vehicles fault, but that's another story).
And that'ss with humans behind the wheel: lazy, distracted, slow to react fleshbags that we are.
I'm not sure that a truly unavoidable accident would occur even once a year in a fictional world in which all drivers were perfect and had millisecond reaction speeds.
What is obvious however, is that these situations are so rare as to be irrelevant. In practice accidents are avoidable by the driver of the vehicle - or at least avoidable to such an extent that it's not worth considering the other cases.
Also: although I personally don't object to some rational victim blaming I think it's a little distasteful that we're already speculating about how this must be the victims's fault, when there's simply not enough evidence to make that kind of determination yet. Let's not forget that part of the privilege of being allowed to participate in traffic implies a responsibility not to kill people even when they behave unexpectedly.
For some statistical perspective: if human drivers had as many fatal accidents per mile as uber has, then the average male driver would kill 1 person in his lifetime (men drive more). Clearly: that's absurd; people may cause too many accidents, but not nearly that many - and that's being rather charitable to uber's self-driving vehicles, since they have back-up drivers and thus don't count complicated traffic situations, and safety drivers, and thus may well have caused more accidents by themselves. So going purely by the unusual-ness of such an accident with so few miles, I'd say the initial assumption must be that this is likely a bug in uber's car, even if I'm sure there were contributing factors.
Edit: I guess it's not surprising wikipedia has stats on the influence of alcohol on fatalities, but it tops out at 4 times the legal limit - at which point human drivers are still safer than this (sample size of one...) uber record so far. :-/
> Simply counting traffic fatalities suggests that crazy pedestrians causing unavoidable accidents cannot be common
Of course not. I'm not saying they're common, merely that they exist. I do not agree with the idea that accidents would go away if drivers were just trying harder, though. There are legitimate unavoidable accidents, and there are also limits to practical human driving.
> Also: although I personally don't object to some rational victim blaming I think it's a little distasteful that we're already speculating about how this must be the victims's fault
To be really clear, I am not blaming the victim here. I have no idea what happened. I'm actually very inclined to blame Uber, though I recognize that's just my personal bias against them.
Why can't you slow down when driving past tall parked vehicles that you can't see through? If you are going slower you will have a chance to see them in advance, select a speed where your stopping distance is less than the length of the obstruction and be prepared to brake. You will have no chance of striking all but the most willfully suicidal of pedestrians.
Near my house there is an arterial I often cross where a large bush on the corner of the intersection obscures the view to the left from the stop sign roughly 10 feet back. I don't just stop at the sign then YOLO through the intersection, I stop, then creep forward, first looking for pedestrians or bicyclist that might step out from the bushes, then at the edge of bushes I stop again and look again both left and right for cars and bikes or pedestrians in the far lane before proceeding across. I often have to stop another time between the sign and the bushes for the pedestrian or bicyclist that just emerged, if I was focusing on getting across the arterial and beating cross traffic I would have killed every one of them.
At what speed do tall vehicles suddenly become transparent?
The idea that cars should drive past tall vehicles at 5 mph is slightly ridiculous. I have never ridden in a car with someone who constantly adjusted their driving speed based on the cars parked on the road. Choosing a reasonable speed? Of course. Extra care at obvious obstructions? Absolutely. Slowing traffic to 5mph because a van happens to be parked? No.
It is not ridiculous if you are driving in a narrow lane close to the obstacles. When I am driving on a residential (one lane, parked cars on both sides) street and I am approaching a tall opaque van I start by noticing if anyone is around it as I approach, when I have better sight lines, then as I get near it I absolutely slow down from an average speed of 15-20mph to well under 10mph and yes, depending on the situation sometimes as low as 5mph or slower. When I am on a faster road with two or more lanes there is often room to move laterally in these situations and so my speed reduction is less extreme, but I absolutely do still slow down as I pass these vans/trucks, cover the brakes and also check my blind spot to see if there is room for evasive maneuvers should they be required. If I see people in the area as I approach and have reason to believe they might try to cross, enter or exit a car, or otherwise be near the lane of traffic I will often change lanes to the left if possible.
Agreed that it's too early to really say one way or another. In maybe 100k miles of urban driving I've had one cyclist run into my car and a girl on her phone walk directly into the corner of the front, I was at a complete stop watching them both times.
Until there's a detailed report it's really hard to say if it was preventable or not - but I think regardless the optics are bad and this is going to chill a lot of people's feelings on self driving whether or not that is an emotion backed by data.
> If a pedestrian suddenly rushes out into the street or a cyclist swerves into your path, the deciding factor is often simply the coefficient of friction between your tyres and the road.
This holds iff the control input you apply is only braking. Changing the steering angle is generally far more effective for the "pedestrian darts out from hidden place onto road" situation. It's far better to sharply swerve away to ensure that there's no way the pedestrian can get into your path before your car arrive there than it is to stand on the brakes and hope for the best.
Indeed, the faster you're moving, the more you should consider swerving away over braking -- take advantage of that lethal speed to clear the pedestrian's potential paths before he can get into yours.
Yes, this intentionally violates the letter of the traffic laws (and might involve colliding into a a parked or moving automobile on the other side of the road) and also involves potentially unusual manoeuvring on a very short deadline; but it's far better to avoid a vehicle-pedestrian collision even if it's at the cost of possibly busting into the opposing lane / driving off the road / hitting a parked car. Decently experienced drivers can do this, I can do this (and have successfully avoided a collision with a pedestrian who ran out between parked cars on a dark and raining night), there's no fundamental reason that computer-controlled cars can't do this.
>Yes, this intentionally violates the letter of the traffic laws...
...so you probably won't find any support for it in this crowd.
> there's no fundamental reason that computer-controlled cars can't do this.
Sure, there's no fundamental reason you can't swerve but you need to identify the thing in the road you're trying not to hit before you can decide whether to swerve or brake. Self driving cars can't yet reliably ID things well enough to take the only evasive action they know (stopping).
I think you touched on the fundamental reason self driving cars cannot currently do this. It would mean programming self driving cars to, in some circumstances, break the law.
I agree with the thrust of your comment though, and think that a change in the law may be required to allign incentives for machines as well as people to drive humanely.
They already do that for speed limits - iirc Waymo cars will 'go with the flow' to a certain extent if everyone around them is speeding. Swerving out of your lane (assuming it is safe to do so) to avoid a collision seems pretty straightforward.
This is a good point. Robocars will have more evidence for their defense in such a case than human drivers would have. "Oh there was a kid who jumped out? And that's why you plowed into this parked car? Likely story!"
Obviously if there's room and time to swerve and avoid the pedestrian, you should, but swerving into an oncoming car? That sounds extreme. There's no way I'd risk killing myself and anyone in the oncoming vehicle to save one person who made a poor decision. In any situation where there's other traffic and off-street foot traffic, swerving might just cause more harm than good. Everything is situational, and I'm sure there are bound to be cases where there is no sensible option for avoiding a fatality because of exceedingly bad timing on the pedestrian's part.
That’s technically true but the number of truly unavoidable cases is orders of magnitude lower. With human drivers, “jumped out in front of me” really means inattention 99.9% of the time and police departments historically tend to be unlikely to question such claims. (For example, here in DC there have been a number of cases where that was used to declare a cyclist at fault only to have private video footage show the opposite - which mattered a lot for medical insurance claims)
With self-driving cars this really seems to call for mandatory investigations by a third-party with access to the raw telemetry data. There’s just too much incentive for a company to say they weren’t at fault otherwise.
The hope is that AV can see 360 and observe things outside of blind spots further away. So the kid running from a yard into the road after a ball should be safer, but a person walking out from behind a parked truck wouldn't.
If a pedestrian suddenly rushes out into the street or a cyclist swerves into your path, the deciding factor is often simply the coefficient of friction between your tyres and the road.
I mentioned in another comment that something I use to try to improve my own driving is watching videos from /r/roadcam on reddit, and trying to guess where the unexpected vehicle or pedestrian is going to come from.
Here's an example of a pedestrian suddenly appearing from between stopped cars (and coming from a traffic lane, not from a sidewalk), and a human driver spotting it and safely stopping:
Could a human have reacted fast enough to stop for someone jumping out in front of them? If the person jumped out so fast that nobody could have possibly reacted in time, then it's not a stain on the technology- even with an instantaneous application of brakes, a car still takes a while to come to a stop. If the human jumped out ten seconds earlier and was waving her hands for help, then it's an issue.
"the whole point of self-driving cars is that they can react to unexpected circumstances much more quickly than a human"
This statement isn't really true. Do you think Uber is investing in this because it makes their passengers safer? No, they are pretty much immune to any car crashes with a human driver, the risk and insurance hit is assumed by the driver. They are doing this to save money. They don't have to pay a driver for each trip. The safety aspect is just a perk, but Uber will be pushing this full force as long as the self-driving cars are good enough.
I'm more concerned that Uber will "lose" the video evidence showing what kind of situation it was, and we'll never be able to know if a human would have had ample time to react.
Video says bicyclist, written article says pedestrian. Video says they don't know if a person was behind the wheel, written article says there was a person behind the wheel.
There's enough discrepancy here that I'm not sure what happened until we see the onboard data (unlikely to be released to the public) or a statement from a trusted public official like the Tempe police chief.
I've been saying this from the beginning: it's not enough for self-driving cars to be "better than the average driver". They need to be at least 10x better than the best drivers.
I find it crazy that so many people think it is. First off, by definition like 49% of the drivers are technically better than the "average driver".
Second, just like with "human-like translation" from machines, errors made by machines tend to be very different than errors made by humans. Perhaps self-driving cars can never cause a "drunken accident", but they could cause accidents that would almost never happen with most drivers.
Third, and perhaps the most important to hear by fans of this "better than average driver" idea, is that self-driving cars are not going to take off if they "only" kill almost as many people as humans do. If you want self-driving cars to be successful then you should be demanding from the carmakers to make these systems flawless.
The quote is "by definition like 49% of the drivers are technically better than the "average driver"." It's obvious to anyone who's not pedantic that they're referring to the median.
As for an example of a mode average: "The average participant's favorite polygon was a triangle."
That's not the original use. That's a response to the (unattributed) idea that self-driving cars would need to be "better than the average driver". In the original context of people giving thresholds for self-driving cars, I disagree that median is what is meant.
> As for an example of a mode average: "The average participant's favorite polygon was a triangle."
But that mode might be 5 out of 40, and I'd just call that a lie. In a lot of distributions the mode just gives you the biggest or smallest number. And it's affected a lot by bucket size; if I measure "how long did you sleep last night" with a resolution of seconds, the mode is probably going to be 0, which is hilariously misleading. Mode, much like a broken clock, might pick a suitable number sometimes, but all the factors you need to check to see if it's suitable basically render the idea of 'mode' redundant. Just use those factors to pick your number. Mean and median are pretty reliable in being useful. Mode isn't.
Sure, and that’s what average may refer to before you drill in with a specific definition. If I say by definition, rectangles are technically polygons with 4 edges of equal length, you’re going to say “technically” I meant a square.
> First off, by definition like 49% of the drivers are technically better than the "average driver".
No, that’s not the definition of mean, that’s (something like) the definition of median.
In a skewed distribution (which driving skill may very well be), the mean can be very far from the middle. If a relatively small number of people are extraordinarily bad at driving, most people are above average. If a relatively small number of people are extraordinarily good at driving, most people are below average.
Why this nitpick matters: if the leading point of your thesis depends on is quantifying how many people are above average, you should really know if the average is at the 10th, 50th or 90th percentiles (or somewhere else entirely).
If 1% fewer people died with self driving cars that would be 400 fewer deaths a year. That's absolutely a good change.
I agree that the _goal_ should be 10x or even 100x better than the best human drivers, but better than average would result in net good and it's hard for me to see an argument in which that's not true because it's less net suffering/death.
>If you want self-driving cars to be successful then you should be demanding from the carmakers to make these systems flawless.
We're still talking about tons of metal moving at very high speeds around other vehicles. That's just not possible. Should my self driving car handle a catastrophic tire failure at 90mph better than me? Absolutely. Is that still a situation that's likely to result in a crash regardless of the best inputs on a compromised system? Yes. There will _always_ be situations in which fatalities can occur with cars or any other kind of vehicle no matter how well they're engineered.
Even if, worldwide, self driving cars only caused there to be 1 less death per year, why on Earth would you not want that? What's the advantage to you in having more people die while waiting for self driving cars to be perfect?
Your first point depends on what question we're asking. Are we asking which is morally justifiable? Or are we asking what the public will accept?
Having seat belts is obviously statistically safer, but when they were first introduced, many people loudly protested that they'd rather be thrown clear in an accident.
The main problem with self-driving cars is that they can't "read" humans' body language. A human driver can see pedestrians and cyclists (and other cars) and have a rough idea of what they're likely to do in the next few seconds, i.e. the pedestrian leaning out on the crosswalk curb is likely to step into the road soon. Even a reaction time of milliseconds can't make up for the (currently exclusive) human ability to read other humans and prepare accordingly.
With sufficient data, I'd expect self-driving cars to be better at predicting what such leans mean. Moreover, for every one human driver who notices such a lean, there may be another human driver that doesn't even notice a pedestrian who has already started walking.
This comment seems unnecessarily hostile. But, still, I'll change my previous comment from "there is" to "there may be" so that it's clear that I'm not claiming any data. Just guessing (reasonably, based on experience).
I'm speaking more of the current state of the self-driving cars I've been in - future improvements will likely narrow the gap or eventually surpass human abilities in most scenarios. How or when is the main question.
They also fail to "write" human body language. Nobody else can predict what the autonomous vehicle will do.
It gets worse when a person is sitting in what appears to be the driver's seat. If the car is stopped and that person is looking toward another passenger or down at a phone, nobody will expect the vehicle to begin moving. Making eye contact with the person in that seat is meaningless, but people will infer meaning.
that is an interesting point. cycling, I often am forced to rely on reading the driver's intentions-something I don't really want to have to do; signals obtained from person sitting in the driver seat but not operating the vehicle could be totally irrelevant to predicting the behavior of the vehicle itself (and cars are not equipped to really signal those things very well).
I have never driven a car. I walk, bike, skateboard everywhere. If there isn't a light I require human feedback before walking in front of a car. Normally I wave and they wave back. They know I am there.. I can walk.
Humans also can't read humans' body language. A pedestrian waiting at a corner isn't waiting for the weather to change. They are waiting for passing cars to stop, as required by law. But passing cars speed by instead of stopping, unless the pedestrian does a lunge into the street -- preferable a bluff lunge, since most drivers still won't stop, preferring to race the pedestrian to the middle of the lane.
OT, but I would love to see how self-driving AIs handle something like Vietnam moped traffic and pedestrian crossings. The standard behavior for pedestrians is to walk slowly and keep walking -- stopping, even if it seems necessary to avoid a speeding driver, can be very dangerous, as generally, all of the street traffic is expecting you to go on your continuous path. It's less about reading body language than expectation of the status quo:
> Sounds like Uber's platform may not be up to that standard yet, which makes me wonder why they're on public roads.
Who exactly do you think is granting Google, Uber, etc. approval for trials like this on public roads? It's going to be some bureocrat with zero ability to gauge what sort of safety standard these companies' projects have reached.
There are no standards here... what were you expecting would happen?
It's an interesting dynamic. We want this tech to be much better than us terrible humans before we deploy it and anything like us terrible humans is not acceptable.
I'd also add the goal of self-driving cars is to decrease costs for ride-sharing, home-delivery companies, etc, and also to decrease congestion via coordination amongst autonomous vehicles.
>> Sounds like Uber's platform may not be up to that standard yet, which makes me wonder why they're on public roads.
They're on public roads because they've decided that deploying their tech is more important than safety.
I don't see any other explanation. We know that it's pretty much impossible to prove the safety of autonomous vehicles by driving them, so Uber (and almost everyone else) have decided that, well, they don't care. They'll deploy them anyway.
How do we know that? The report by RAND corporation:
Key Findings
* Autonomous vehicles would have to be driven hundreds
of millions of miles and sometimes hundreds of billions
of miles to demonstrate their reliability in terms of
fatalities and injuries.
* Under even aggressive testing assumptions, existing
fleets would take tens and sometimes hundreds of years
to drive these miles — an impossible proposition if the
aim is to demonstrate their performance prior to
releasing them on the roads for consumer use.
* Therefore, at least for fatalities and injuries,
test- driving alone cannot provide sufficient evidence
for demonstrating autonomous vehicle safety.
That report ends by saying essentially, "it may not be possible to prove the safety of self-driving cars". [1] So the value here is questionable and the same logic could apply to anything with a low frequency of occurrence. The value of air bags by this measure was not proven until they were already mandated.
[1] "even with these methods, it may not be possible
to establish the safety of autonomous vehicles prior to making them available for public use"
The report also says that these hypothetical new methods may not be able to prove safety. It’s not a straightforward problem. How do you prove that you’ve reduced (or at least not increased) a problem that occurrs so infrequently?
Realistically no one will trust “new methods” and establishing their relevance is really difficult. I would imagine that most of these companies are running lots of simulations, because why wouldn’t you? But how many people will see that and trust it more than data gathered on the road?
> That's a very bad look; the whole point of self-driving cars is that they can react to unexpected circumstances much more quickly than a human operator, such as when someone walks out into the road.
No, the whole point of self-driving vehicles is that firms operating them can pay the cheaper capital and maintenance costs of the self-driving system rather than the labor cost of a driver.
Driver assistance that takes care of everything (i.e. steering, braking, navigating, parking), that at the time the accident occurred didn't even require hands on the steering wheel?
Seems like an attempt to shift the blame off the autopilot system and onto the driver.
What counts as autonomous seems often to be inflated when considering positive statistics such as miles driven, and minimised when considering negative statistics such as fatalities.
At least, there needs to be agreed consistency before we can claim anything statistical.
Interesting, apparently there's no way to know if autopilot was enabled or not.
> A Tesla spokesperson said the car in the Chinese wreck was badly damaged and unable to transmit data. “We therefore have no way of knowing whether or not Autopilot was engaged at the time of the crash,” the spokesperson said in a statement.
Why not the same for human-driven cars? Human-driven cars kill thousands, if 2 victims of auto-driven cars make black boxes necessary, sure thousands of victims of human-driven ones do?
Again... 'Uber has temporarily suspended the testing of its self-driving cars following the crash of one of them in Tempe, Ariz. The ride-hailing company had also been conducting experiments in Pittsburgh and San Francisco.'
It would explain the "outside a crosswalk" remark better if the person were riding the bike. This is one of those cases where video evidence should be subpoenaed, though. It's far too common for the police to accept the statement from the only party still alive, and come up with incorrect conclusions about what happened on the street.
I've heard that police reports often refer to all non-drivers as "pedestrians", even if they were riding a bike (which I learned after a neighbor was killed while riding a bike). I don't know if this is legalese or police jargon, but either way you would hope a journalist who covers these things could translate.
A police officer in Nashville shouted at my wife for riding her bike in the road in accordance with Nashville law. Basically, he didn't know the law he was enforcing.
Unfortunately, many police forces don't train the police on how to handle bicycles. For example, in many places riding on the sidewalk is illegal, yet police will suggest a cyclist get off the road and ride on the sidewalk.
From studying many bicycle-related police reports, I was dismayed at how often I read the phrase "a pedestrian riding a bicycle."
The article also mentions the pedestrian as "a woman walking", so it's not clear whether they are referring to a cyclist as a pedestrian, or whether they are getting different information and actually think it was a pedestrian. Very confusing
The video was released. Turns out it was a woman walking her bike.
Sad event, but it's likely a human driver would have killed her as well. She was on the road at night without lights, jaywalking without attention to the on-coming traffic.
One could argue the car was driving faster than it's headlight distance safely allowed. If so, humans are going to be quite frustrated with how slow autonomous cars drive at night.
Riding on sidewalk is often safer for everyone, though. And, in some places I've lived, it's only illegal in specific marked areas of high foot-traffic. The police maybe shouldn't be giving advice that is technically illegal (if it was), but it may have been pragmatic advice, at least.
Personally, as a biker, I get slightly annoyed by pedestrians who seem to think I have little control or awareness. I ride on the road where practical, and, if on sidewalk, I go much more slowly and sometimes just get off and walk if the area calls for it. Generally, bikers are much more aware of their surroundings than drivers (both out of self-preservation and an unobstructed, elevated view).
Of course, the best solution is fully separated paths for walkers, bikers, and cars. And, in this instance, it looks like there is ample room for building that, with no street parking to complicate things (and the sidewalk also looks wide and sparsely used).
Given that so many of autonomous car accidents logged by the California DMV seem to be the fault of the human driver trying to take control, I’m surprised that the accident here reportedly happened in autonomous mode, which for all its flaws seems to have been good at not hitting things in front of it. In fact, seems like most of the CA accidents were the autonomous car being rear ended or side hit from other cars, usually for going too slow or stopping unexpectedly.
It's a sad reality, but regardless of how many times per day a pedestrian is hit by a human-driven car, such incidents involving self-driving cars will be headline news for years to come.
Having a hard time sympathizing here. It's one thing to fear air travel based on the very few but very publicized plane incidents (considering all the data we have on the safety of air travel). It's another thing to hold these self-driving car companies accountable, considering a) the lack of data and history of such programs, and b) their touted benefit as a safer alternative to human-driven cars.
If the miles are not enough, they kill less until they first kill someone. At which point they kill several orders of magnitude more. # of events per something are not always a statistically valid way to measure things.
Only if you aggregate them across companies and ignore the fact that the distribution those miles are not comparable to to distribution of human driven miles.
The entire potential of self-driving cars shouldn't be dismissed based on accidents like this - that would be an unfortunate side effect of these headlines. But I also reckon this coverage actually incentivizes better and safer programs. It's such early days, we simply need more data to form sound opinions. In the meantime, journalism is serving an important role here.
What if self driving cars kill less people, but the type of people they kill are different from the type of people who die in human driving accidents?
For example, what if instead of 100 people per day dying from human-driven car accidents(where 95 of them are car drivers/passengers, and the other 5 are pedestrians/bicyclists), self-driving cars only kill 30 people per day, but 28 out of 30 are pedestrians/bicyclists?
As they should: humans fail independently, but autonomous cars can fail systemically, including hacking. Bicycles are known to be tricky for autonomous cars.
I had been assuming that self driving cars would be a 'reset' for our decision to value cars more than the countless lives they end. Self driving cars could take the irrationality and emotion out of driving. It would be America's "stop the murder of children" moment (The Netherlands rallying cry to reform auto traffic in 1970's).
But now I see that it can, and probably will go the other way. These giant companies could push to further restrict the rights of cyclists/pedestrians around roadways, vilify the 'stupid' people who made a reckless decision that resulted in an accident, and push our country even further into car culture.
Even if its provable that self driving cars are 10x safer, every accident of this unfortunate kind will be used against autonomous cars. I don't see how to overcome this -- I don't expect reason will overcome the perceived danger.
You're goddamn blind. Yes, of course it matters more if I'm killed by an AI machine than it does if I'm killed by another human being. This is a new development, humans killing humans are not. What if it was autonomous warfare? It is surely coming about in a very near shipment, SOON. The world and history are not static, and techno-utopia is big fat lie, and the idea that things just get better as time advances frontwards is plain infantile. Reality is way more fucked up than you can imagine.
The video mentions she was on a bike and shows a mangled bicycle. Also the line markings on the street indicate there is a bike lane to the left of a right turn lane. Definitely very bad.
Not to be insensitive to the death. But, that bike lane design is just horrible. Are human drivers really known for respecting the "do not cross solid white line" rules? 9/10 drivers that realize at the last moment they need to turn right are going to quickly cross over this bike lane probably without much time to check their blind spot. If I was cycling on this path, I'd want to use the sidewalk and be extra cautious of peds, I don't trust drivers human or not.
There aren't a lot of options when you need to have cars turning on roads with bike lanes. At some point the car lane and the bike lane have to cross paths unless you're going to build a fully grade-separated bike lane.
I suppose you could have a wall that separates the bike lane from the road except for a short area where cars can cross over, so that at least there's only a small space where the bikes & cars can interact, but that introduces a bunch of new problems (cars that can't see bikes behind the wall when they go to cross over, cars that run into the wall, probably even more swerving to get into the lane because your choices are do it RIGHT NOW or you're stuck/hit a wall).
I think it'd be safer to keep the bikes in the far right even if they go straight. Yes, it means they have to pay attention to cars turning right. They might even need to stop until it's safe. This is a place where "bikes/peds having right of way" doesn't really make sense.
Users to the right have the right of way over users further towards the center of the street. Thus, pedestrians crossing in the crosswalk have priority over bicycles and cars turning right, and bicycles going straight have priority over cars turning right.
I guess in a perfect world having right of way means you can cross the street without looking left or right. I’m more willing to yield as a bike/ped because my risk is higher and I know people are distracted idiots.
Absolutely worth accepting the reality of drivers on phones' as a threat to safety. I'm just trying to explain the rules of the road from which intersection design can be derived.
There are ways to design an intersection so that those rules are followed naturally, and there are ways to miss it. C.f. right turn lanes.
> There aren't a lot of options when you need to have cars turning on roads with bike lanes. At some point the car lane and the bike lane have to cross paths unless you're going to build a fully grade-separated bike lane.
The problem is that the lane markings encourage vehicles to make turns from the non-rightmost or non-leftmost lane. One should not be crossing another lane when making a turn at an intersection. One should merge into the rightmost or left most lane in order to make the corresponding turn.
I would greatly appreciate a barrier of some kind between me and car drivers looking at the cells.
Or just do what Germany has sensibly done and split sidewalks between pedestrians and cyclists (though, cycling enthusiasts are unlikely to enjoy the experience as much)
However, I do need to mention that when you choose to bike on the sidewalk, you're creating the same type of problem that cars that ignore traffic rules create for bikes. There's a reason why biking on the sidewalk is illegal in most places, and why bike lanes/roads exist.
> you're creating the same type of problem that cars that ignore traffic rules create for bikes
Agree. I'd do it anyways because sidewalks are 99.999% empty where I live. I could ride for 20 miles and not see a person on the sidewalk. So biking on sidewalks is safe as long as you pay attention to what's ahead. No idea if that's the case in this AZ location.
Cyclists here (in Seattle) have a healthy habit of calling out "to your left" when approaching from behind. Things like this need to become more widespread.
I do that regularly on my commute on a mixed-use trail. Half the time the pedestrian looks to their left and their feet follow. The other half have headphones on so they don't hear me anyway.
You still have to cross at intersections and driveways. I've been hit multiple times (at low speed) by motor vehicles crossing the sidewalk at a driveway. I don't ride on sidewalks anymore.
This sounds like a speed issue. If you’re doing 20 mph on the sidewalk then yes you weren’t in view when the driver started backing out of their driveway.
I don’t fully think I’m a bike I get right of way. If I see a car reversing ahead I move into the street then prepare to full stop but proceed with caution and watch the cars acceleration to see if they see me. If I get the hint they don’t. I stop. It’s scratch on the bumper for them and life for me I am more than glad to yield.
>you're creating the same type of problem that cars that ignore traffic rules create for bikes
Which are what? Such a broad statement doesn't seem accurate and the sidewalk shown in the map looks low traffic with excellent sight lines, as long as you're not cycling head down at high speed the danger to pedestrians doesn't seem comparable to danger posed by car driver's ignoring traffic rules.
The problem is that something is somewhere and/or does something unexpected. The bicyclist doesn't expect the car to drive in the bike lane. The car driver turning across the sidewalk doesn't expect the bicyclist to be riding across the driveway on a bicycle. If you're not expecting it you're not going to look for it.
Riding a bicycle on the sidewalk isn't illegal most places because it's dangerous for the pedestrians, it's illegal because it's dangerous for the bicyclist.
I understand and personally am very cautious around driveways especially if they don't have perfectly clear sight lines.
I just find it odd when cyclists out of principal decry riding on the sidewalk when the danger of riding with traffic is well known and America has an abundance of long empty sidewalks in suburbia and around office parks.
Riding on the sidewalk is dangerous despite the sidewalks being usually empty. There's another factor: when sidewalks become intersections. Very few drivers check for cyclists on the sidewalk before going through an intersection. From what I understand this is shown quite clearly in the crash statistics.
Many dedicated cycletracks have similar problems. Here in Austin there are several that I refuse to use because drivers far too frequently turn into my path without looking. I ride in the lane so that I can be seen. It's counterintuitive to most drivers, but visibility is often the deciding factor in bike safety. This would be more obvious to drivers if they spent more time cycling. If a cyclist is stuck using a path with poor visibility, I find paying attention to the cars and using light touches of an air horn when approaching an intersection to help. But I still prefer being visible to that.
More generally, intersections are relatively more dangerous (crashes per person mile or something like that) than straight segments of road for any mode of transportation using the road.
The problem is that pedestrians on a sidewalk do not follow traffic rules. They don't signal turns or stopping in advance. They choose random sides of the sidewalk to walk on. They don't have mirrors to see approaching cyclists behind them.
Vehicles, on the other hand, do follow traffic rules with regards to lane usage, signaling, and checking what's behind them. Even if they don't, they're still more predictable compared to pedestrians.
Personally as a I cyclist, I don't intend just yet to give up cycling on roads (which are for multi-transport users). After getting munged on a separator, I also don't think they are the be and end all.
Yes, but the reason why biking on the sidewalk is illegal in most places is mostly unrelated to safety.
The only bicycle riders willing to get organized and lobby the legislators are the very most serious ones. These are the people who want to race along at 50 MPH.
The resulting laws force pitifully slow riders to be out in the street. Even if you can only manage 5 MPH, you have to be in the street.
Lots of us are too arthritic, too fat, too weak, too inflexible, too limited by lung capacity, too limited by heart capacity, and too limited by cheap steel mountain bikes from Walmart.
I don't think that's insensitive to the death. It's properly placing (partial) blame on the cause of the accident.
A better design for that intersection would be to keep the bike lane against the sidewalk and the turn lane inside of that, so that they don't cross, and have a "no turn on red" sign, so that the paths of bicyclists and people turning in cars never cross as long as everyone obeys the stoplight and sign.
From a the perspective of policies such as designing an intersection, placing blame (even if that blame is justified) is utterly pointless. Systems should be designed for the users you have, not for the users you want, especially since in this case it's literally a matter of life and death.
Sure, drivers should check before crossing a lane, but some percentage don't, so it makes sense to minimize lane crosses.
Do you have a statistically large enough sample to conclude that they're worse at driving that you are? Most cases of driver inattention don't escalate to a collision, so trying to infer inattention rates from collision rates is pretty noisy.
And even assuming this was simply a case of a bad driver, what policy approach would you suggest to protect the general public against such drivers? Accepting that drivers are fallible and designing our road systems to be robust against that seems a more effective approach than berating those drivers who are particularly unlucky in the consequences of their failures of attentiveness.
That kind of design only makes right hook collisions more likely. Pushing cyclists closer to the curb, to the periphery of a driver's view, makes them less visible and more likely to be unseen before a turn is made.
...which isn't relevant because bikes shouldn't be in the intersection at all while the turn is being made. The bicyclist going straight has a red light when the turn lane has a green arrow, the turn lane has a red light when the bicyclist going straight has a green.
Perhaps I should have said "wait for turn indicator" instead of "no turn on red", that's a bit clearer.
> (It's hard to see a bike at speed going straight when a car is turning right.)
This is addressed by having a "no turn on red" sign as I suggested: you don't have to see the bicyclist at all because the bicyclist shouldn't be in the intersection at all when you're turning with a green turn arrow.
The bike isn't going at speed when the light is red (at least, if they're following the law). This is when the green turn arrow should be lit: when bikes aren't moving.
AZ driving rules language on solid white lines is the same as FL (where I live). They use the words "should not cross", so it's not technically an infraction for a driver to cross a solid white line. I was in an accident once due to someone crossing a solid white while at a light and cutting me off so I'm familiar with the RAW on those lines.
I'm much more a fan of the design that places a buffer (either sidewalk or shrubs) between the CARS and any other type of vehicles. Sidewalks are tolerable as they are often raised above street grade and a vehicle usually has to be actively controlled to overtake such a raised divider.
Encouraging bikes and other personal mobility devices to use the sidewalk pushes the danger to pedastrians. I'd argue there really shouldn't be right turn lanes except in really select circumstances. Cities that have a lot of bikes like Copenhagen don't really have that many turns lanes. And when they do the mixing area from protected lane to unprotected bike lane is pretty clear.
Those sort of bike lanes teach cyclists that they should under take slower moving traffic - the problem is the majority of deaths in London are due to vehicles turning into undertaking cyclists.
If it's a bike lane, it is not undertaking. It is a lane. You wouldn't blindly turn across another lane of traffic, why do you do it when it's a bike lane?
And the majority of deaths in London is due to turning transporters/trucks. They are purposely built such that the operator sits very high above and can't even see a bike or pedestrian unless it's many meters away, only by looking into one of 10 mirrors (but of course they don't). On a work site nobody moves these things unless there is an outside instructor, but on public streets we've decided to just blame whoever died. Or, if that fails, blame the infrastructure, even though we oppose any other kind of infrastructure.
What the problem is that those sort of cycle lanes conditions cyclists that its always right to undertake in any circumstances - its got to the point now in London that cyclists feel entitled to undertake at speed in stopped traffic.
That's logically impossible. Under/Over taking doesn't mean moving faster than neighboring traffic, it means "merging into slower moving traffic". If the cyclist isn't moving into the car lane, they aren't undertaking.
Anec-dataly, I agree. We live at the edge of a residential area where there is a long stretch of road with bike lanes and turns into the residential area every block. Not only do people regularly drive in the bike lanes, the people who do so are typically the most aggressive and inattentive drivers who are trying to angrily speed around the lines of cars who are driving the speed limit and respecting the stop signs. I'm not aware of any recent accidents, but only because cyclists seem to just completely avoid the entire stretch of road.
That looks completely identical to all bike lanes I've seen in suburban areas. I think bike lanes are just inherently dangerous. Would be interesting to see what a well-designed suburban bike lane looks like.
This is both terrifying and normal. The bike lanes in my area are frequently used as turn lanes... by vehicles that don't fit in them - despite being clearly marked as bike lanes.
That actually makes sense since it encourages all vehicles to use the right most lane to turn right, rather than crossing in front of a straight through lane when making a right turn.
In fact, it's all bike lanes in California, unless they are marked otherwise. Drivers are required to move into the rightmost lane in the last 200 feet before turning right, including merging into a bike lane.
There are exceptions, but only if there is signage or road markings. For example, if there are two or more lanes marked for right turns, then obviously you can turn from any such lane.
(A little-known fact about that: most California drivers are aware that you're allowed to turn right on a red light after making a full stop, unless there's a "no right turn on red" sign - but few seem to know that this rule applies to all the lanes marked for right turns, not just the rightmost lane.)
Most bike lane markings change from a solid white line to a dotted line about 200 feet from a corner, to give drivers a hint to merge into the bike lane before turning. But even if the line doesn't turn dotted, drivers are still required to merge into the bike lane unless there is a specific indication otherwise.
However, the majority of drivers seem to be unaware of this rule and turn right from the auto traffic lane, creating the risk of a "right hook" collision, which the law is intended to avoid. I've actually had other drivers honk at me when they were waiting in a line of cars to make an illegal right turn from the auto lane while I made a legal and proper turn from the bike lane.
The San Francisco Bicycle Coalition has an excellent info page about this:
Laws on this do vary in other states. But at least in California, if you see a driver merging into the bike lane before turning right, it's not because they are a terrible driver, it's because they are following the law and (hopefully) increasing safety for bicyclists.
I think you've missed the point. I'm talking about merging into the bike line prior to the turn.
If you still don't see the difference, here's one way you can tell. If you are stopped, waiting at the intersection, if you've merged, then you'll be occupying the bike line while you're waiting.
Where I live the law explicitly requires cars to take the bike lane before turning in this situation. It's probably better to separate the blind spot for bicycles check from the look for pedestrians and turn check.
The car turn-lane is to the right of it, so at least it's maintained as part of the road. Bike lanes that are truly a shoulder with bike symbols painted on are often unmaintained, and full of pot-holes, debris, and such.
Yeah, they do the similar stupid "bike lane" crap here in Indiana as well.
Yeah, like a line is going to protect me from run amok car drivers who are too buy on their cell phones. Or it's going to stop the bus drivers from using them as pickup/dropoff areas, cutting bicyclists off.
Long story short, I bike on the sidewalks. I can be a responsible biker on sidewalks while heeding to pedestrians. My life is more important than to be biking on a vehicle road.
I mean, I've seen the pictures... Just not in the wild. One of those things that seems like a nice idea, but good luck ever getting anyone to pay for such en mass. Have you seen any of these around Tampa Bay? I've never seen anything except the unprotected margins, excepting trails which are not sharing infrastructure with roads.
When someone actually takes the little money to build one, there is an army of people essentially saying they'd rather have 5 more parking spots than safe infrastructure. Then when somebody is hit, like here, the same people argue you can't blame the driver because look how bad the infrastructure is. You can see it in this very thread with people arguing bikes should fuck off to the sidewalk.
How about a "sharrow", where cyclists drive right-of-center, in hopes of avoiding the "parked car door lane", and encouraging drivers to cross the double-yellow to overtake.
This design seems pretty common in the US. Some bike lanes on major streets in LA are like this, except there they also have to contend with buses pulling into stops. Biking on those streets would be terrifying, but I saw people doing it.
Yes, I bike almost daily on Venice Blvd in LA, and every intersection I encounter is setup like the one discussed here, along with buses weaving in and out. A vicious battle flared up last year when the city replaced a lane of traffic on Venice with protected bike lanes (some discussion here https://la.curbed.com/2017/10/25/16528864/road-diets-los-ang...). There seems to be a marked increase in the hostility between cyclists and drivers now.
Unfortunately, this is a design that is pretty common in many of Tempe's major intersections. I pass through a similiar intersection biking home from ASU and have had multiple close calls with human drivers.
Anecdata: The times I have interacted with one of Uber's autonomous vehicles at this intersection, they tend to hit the brakes pretty hard as they are about to enter the turn lane and I am around 10ft behind them. All of my interactions with them, however, have been during the day, and it was unclear if it was the human driver in control.
The video says bicyclist and shows a bicycle, while the written article says pedestrian and said the car hit "a woman walking"... I'm confused as to why there is a contradiction, but considering the video shot of the bicycle I assume you're correct
It seems like a bunch of what-ifs that normally come up with self-driving cars are about to get answered and precedents are about to be set.
I assume this case will also be one of the most well-recorded cases of a fatal car accident in history as well, given the amount of sensors and equiment on-board a self-driving car, along with eye witness testimony from the operator on board.
Can't tell if Uber has just been incredibly unlucky as of late or if just enough of their employee-base is incompetent as to prevent them from just having a quiet year with no large failures.
Perhaps time as opposed to distance? After all, we drive much fewer miles in a city, but there are much greater opportunities for accidents, particularly pedestrian accidents.
Maybe deaths per speed per time (or distance, since speed over time is the same as distance at speed). More deaths at higher speeds and fewer at lower would seem more likely. Purely using time or distance may skew the interpretation, if there are more deaths at some speed ranges than others.
This is an interesting metric. Although self driving test cars are rare, the whole point of their existence is to drive so they probably clock more hours than a normal car.
About 5400 pedestrians are killed each year in the US.
US drivers go 3.1 trillion miles a year. So they kill a pedestrian about every 5.7 billion miles. Last November, Waymo said they had 4 million self driven miles, so well short of statistically expecting to hit a pedestrian. In September of some unspecified year Axios claims Uber had self driven over 1 million miles.
My estimate needs help, the distribution of pedestrian bearing roads and pedestrian free roads likely does not match from my total miles per year number and what carbot testers cover. Also, this may have been a cyclist death, which adds another 800 or so deaths per year.
But, in any event, in rough numbers, Uber appears to have beat their expected time to pedestrian fatality by two or three orders of magnitude.
It occurs to me that because the deaths caused by autonomous vehicles may not follow the same distribution across types of deaths, it might make more sense to compare total deaths per million miles between human and autonomous drivers.
If deaths are broken out by pedestrian vs passenger, I wonder how people will respond if safety skews heavily towards one of those groups with self driving cars.
Distance is the metric used so far but also isn't appropriate for comparing self-driving cars to human driven cars. The human driven bucket contains all miles driven, highway cruise control, light snow, heavy rain, tricky merges, etc. The self-driving metric is only the easiest possible miles. Overtime those miles will expand and harder scenarios will be incorporated but to really know if self-driving cars are safer we need apples to apples comparisons which is going to require matching humans vs robots on miles driven and a categorization of those miles, maybe a count of tricky unexpected scenarios as well.
Why am I not surprised that, among all the teams competing in this space, many of which have clocked in far more miles than them, Uber is the first one to kill someone?
This may well just be bad luck. But I cannot shake the feeling that if Uber started an ice cream venture, they would store their molasses on a hill in a Boston. The only way to get “humanity” associated with Uber would involve an Uber zeppelin.
Or it’s a conspiracy, because nothing is as threatening to Uber than autonomous cars. This is certain to invite more regulatory scrutiny. Just kidding... I think..
You mentioned bad luck and that could very well be it. Unless Uber starts having more incidents, it isn't fair to say that Uber's technology is inferior.
Not to be the guy who says "I TOLD YOU SO" but it had to happen at some point, sadly & unfortunately - if this is the first (known) case of A.I. controlled cars killing humans, the number of casualties can only grow from now onwards
Every year between 30000 and 40000 Americans are killed in car accidents. With more self-driving cars on the road it comes as no surprise that self-driving cars will cause some of them.
To ensure that all automotive software incorporates lessons learned from such fatalities, it would be beneficial to develop a common data set of (mostly synthetic) data replicating accident and 'near miss' scenarios.
As we understand more about the risks associated with autonomous driving, we should expand and enrich this data-set, and to ensure public safety, testing against such a dataset should be part of NHTSA / Euro NCAP testing.
I.e. NHTSA and Euro NCAP should start getting into the business of software testing.
Dr. Mary Cummings has been working on talking to NHTSA about implementing V and V for autopilots/AI in unmanned vehicles for a few years now. She's also been compiling a dataset exactly like what you are talking about.
I think the idea is to build a "Traincar of Reasonability" to test future autonomous vehicles with.
You ultimately have to at some stage, since any test track is a biased test by its nature.
It is more an issue of how sophisticated these vehicles should be before they're let loose on public roads. At some stage they have to be allowed onto public roads or they'd literally never make it into production.
They were unwilling to legally obtain a self-driving license in California because they did not want to report "disengagements" (situations in which a human driver has to intervene).
Uber would just set their self-driving cars free and eat whatever fine/punishment comes with it.
This is a strange question to ask. The regulation is not there to benefit Uber, it is to benefit public good. Very few companies would follow regulation if it was a choice. The setup of such regulation would be for it to be criminal to not comply. And if Uber could not operate in California (or the USA) if they did not comply, it would in their interest to provide the requested information.
Uber has shown very often that they are willing to break the law. It seems within their modus-operandus to just ignore these rules.
Essentially, Uber engages in regulatory arbitrage but taking into account the cost-benefits of breaking the law. I.e. if it breaks the law but is profitable for them, they seem to do it.
Sure, so make the regulation expensive. For example, if a company is not in compliance then the executive team can be charged for any crime their self-driving toy committed under their guidance.
I don't believe this will be effective. Thinking back to the VW scandal, did any executive get punished for this? Same question for the Equifax breach and the insider trading issue.
My 'money' is on people with money figuring out loopholes, like plausible deniability.
Yes, it means that we need to write new regulations with real teeth, and vote out the politicians on all sides of the aisle that continue to punt on this issue.
One of my biggest complaints about the self-driving car space is that real lives are at stake; light-touch "voluntary" rules suitable for companies that publish solitaire clones aren't going to cut it here.
Very cynical but - if your self-driving is way behind of your competitors- wouldnt it help to have your lousy car in a accident - so that your competitors get hit with over-regulation and you thus kill a market- on which you cant compete?
I‘m quite sure this would backfire A LOT in terms of brand damage. Uber in a sense made history today and now has actual blood on their hands. And if such a strategy should EVER leak (Dieselgate anyone?), people are going to prison.
GM killed 124 people with faulty ignition switches[1], yet the brand still survived. It's a cost calculation: will the brand damage outweigh the benefit to the company? Sadly, human lives don't factor into that equation.
Sadly, that’s a common occurrence with big automakers.
I can’t say anything about GM‘s rep in the US, but here in Europe they are not doing so well. Chevrolet was killed in 2015, and Vauxhall/Opel are doing only ok-ish. Chevy had SO many recalls in the years before they killed it.
It's a cynical approach, but they could be playing both angles. Take enough risks that maybe you do succeed and you can cut your costs enormously by actually having SDCs. But if you fail, you also protect yourself by taking the competition down with you.
No, because what you risk is associating your brand with death, rather than AD.
Uber already has a terrible reputation with everyone in the tech industry for the sexism, bullying, law breaking, and IP theft. Do they really want to be the self-driving car company with a reputation for killing people?
It doesn't take a lot for people to think "Maybe I'll take a lyft" or "Maybe I'll ban uber from London because of their safety record"[0].
They aren't going to kill the market for this - the other players not only have big incentives to make sure they look safe, but you've got a really unique problem when your biggest competitor is a company that controls access to news and which adverts your customers will see.
Oh, contrair- if you have a decade sitting unemployed in a room- and you get a chance, by taking all the responsibilities and liabilitys, to live your golden years in the sun- what would you have to loose?
Maybe they start paying compensation to the family, making sure that they're set even if you fail - firmly stepping into mafia territory.
Still, corporations, while being essentially a different kind of life, are not entirely separate entities - they are composed of people. Fear of jail might just be enough to get some high-level executives to exert proper influence on the direction of the corporation.
Possibly there's enough business risk that if Uber doesn't, someone else will, and then they will have SDCs but Uber won't, and then Uber will go bankrupt just about instantly.
There may eventually be standard test suites that can be applied to any of the self-driving systems in simulation. This would give us a basis of comparison for safety, but also for speed and efficiency.
As well as some core set of tests that define minimum competence, these tests could include sensor failure, equipment failure (tire blowout, the gas pedal gets stuck, the brakes stop working) and unexpected environmental changes (ice on the road, a swerving bus).
Manufacturers could even let the public develop and run their own test cases.
- philosophical dilemma
- physics-constrained reaction times
- actions that violate the rules of the system
Solutions:
- protected car lanes
- protected bike lanes
- protected pedestrian lanes
Reasons why these solutions are not put in place:
- cost
Until humans determine that the cost of human life is higher than the cost of upgrading infrastructure, we should accept human death as a regular part of autonomous driving, just the same as we do for non-autonomous driving. 37k dead people every year in the US due to human drivers.
Top reasons for auto accidents today include inclement weather, reckless driving, speeding, driving under the influence, and distracted driving. In theory, most of those could be solved by autonomous driving. But then the list of reasons for accidents would change to whatever new reasons cause autonomous car accidents, such as damaged sensors, programming errors, equipment failure, road hazards, etc.
Even with autonomous cars, we will still need protected lanes, and we will still never implement them, because we don't really care when people we don't know die.
If by "protected passenger lane" you mean putting a Jersey barrier between the sidewalk and the road, no thanks. Maybe I'm a spoiled suburbanite, but I can't help but recall that, scant a century ago, pedestrians could walk along or on the streets wherever they pleased.
In order to allow transportation to co-exist with pedestrians without collisions, you need some kind of separation between the two. With subways, the protected lane is literally underground, but it does definitely have a protected lane. If you don't go under ground, you can go above ground, like several subways and metros do around the world.
If you don't do either of these, you have to make concessions on the ground level. My personal preference would be tall fences around the roadway, and pedestrian bridges that go over or under the roadway (but both have problems). Another would be to still have the fences, but automate some sliding barricades that would activate when traffic halted, which is somewhat like how train crossings work. We could also implement hybrid methods, like that at Shibuya crossing, for very congested intersections.
This has little to nothing to do with the specific company (although is a convenient deflection ATM). The very idea that pre-programmed computers can deal with driving is the fundamental problem. In time, people will realize that, but at the moment the "solution" is to "patch" and make more laws to hide the truth: AI has nothing to do with intelligence.
Ultimately they will blame humans, humans are the bugs in their code.
I'm not the original poster, but maybe they just mean that as it stands right now, artificial intelligence refers to nothing that is actually intelligent in a way that an average person probably thinks about intelligence, and it's a little hard to see where the leap to actual intelligence is going to come. Maybe?
As far as I know all self-driving car programs use some form of machine learning, so they aren't exactly "pre-programmed" in the sense of good old-fashioned AI based on strict, pre-determined rules.
What exactly do you mean by "deal with driving"? Drive without a single accident, ever? That's obviously impossible in practice. Drive better than humans, who in 2016 killed 37,461 people in the US alone? I don't see how that would be impossible - human drivers have a limited field of view, slow response times (average time to break is ~2.3 seconds), and are frequently distracted, sleepy, drunk, etc.
Let me know when self driving cars can get sleepy, distracted or drunk. Until then, I'm much more concerned with "stop before hitting things 101". This "pre programmed cars must be better than X" is getting old fast.
This is one of those self correcting problems, and it's going to be fixed way faster than the startups pushing this "smart machines" propaganda are going to like.
The people leading the development should demonstrate that they can stop for pedestrians by personally jumping out in front of them on a closed test road. If they're not able to demonstrate this, they shouldn't be putting them on public roads.
Why? It’s not a problem if Uber kills pedestrians, even in situations where it’s completely avoidable. It’s only (legally) a problem if they’re violating the rules of the road while doing so.
I think their (idiotically stated) point was that there are plenty of situation where you can kill someone on the road in an accident and not have any legal liability for it.
In general motorists are not held strongly accountable for killing pedestrians or cyclists. Juries in the U.S. are very reluctant to convict. I'm not sure Uber can rely on that, however. While juries are reluctant to convict drivers, driverless vehicles are another matter. As I said in another comment, they're going to have to invent a crime that puts cyclists and pedestrians positively at fault in these situations.
People jump in front of the NYC subways multiple times per week in an attempt to commit suicide, how often do you read about the train engineers going to prison for that? Literally never. If someone tries to commit suicide by jumping in front of your car then you're generally not legally liable, as long as you're not intoxicated and you pull over immediately and phone it in.
First, I assume the rules for trains are different than for cars. Second, I expect that someone jumping in front of your car is different than running over a cyclist in a bike lane.
The ancient Romans would have the civil engineer stand under the bridge they'd just built while a Legion marched over it. That's why Roman structures are still around today!
>The ancient Romans would have the civil engineer stand under the bridge they'd just built while a Legion marched over it. That's why Roman structures are still around today!
Can't speak for the romans, but his seems legit enough, and predates the romans by quite a bit:
Hammurabi's code:
> If a builder builds a house for a man and does not make its construction firm, and the house which he has built collapses and causes the death of the owner of the house, that builder shall be put to death.
> If it causes the death of the son of the owner of the house, they shall put to death a son of that builder.
A more verifiable story in a similar vein would be Frank Lloyd Wright standing under an exemplar of his "dendriform" columns, developed for the Johnson Wax Headquarters in Racine, laughing and whacking it with a cane as it was undergoing a loading test for the building code enforcement officers.
Heard a similar myth about a Danish king. We was tired of cannons blowing up so he ordered the manufacturer to sit on top of the cannon when it was fired the first time.
A similar story comes from World War II where an alarmingly high number of parachutes were failing to open when deployed. They started picking random chutes and their packer and sent them up for a test drop. The malfunction rate dropped to near zero.
The comparison is odd to me. Somehow building bridges seems more of an exact science to me than making cars drive themselves. I sure wouldn't step on a bridge if its engineer doesn't dare going under it. Shit is supposed to stand up.
Said that few times: how does any AI recognize whether object in front of me is just a white plastic bag rolling in the street, or a baby that rolled out of crib. AI cannot know and will never know. And we cannot have cars drive smoth in traffic if every self driven car will stop before hitting an empty plastic bag.
I'm surprisingly okay with self-driving cars stopping for plastic bags in the near term.
Self driving tech is up against the marketing arm of the auto-industry, they've got to be better than perfect to avoid public backlash. If they're a bit slower but far safer then I think they'll do well.
How do we recognize whether an object in front of a car is just a plastic bag in the wind or a baby?
o At speed we're pretty ok with cars hitting things people drop on the road, examples of cars hitting wagons and babies are already plentiful
o Visual recognition & rapidly updated multi-mode sensor data, backed by hard-core neural networks and multi-'brained' ML solutions, have every reason to be way better at this job than we are given sufficient time... those models will be working with aggregate crash data of every accident real and simulated experienced by their systems and highly accurate mathematical models custom made to chose the least-poor solution to dilemmas human drivers fail routinely
o AIs have multiple vectors of possible improvement over human drivers in stressed situations. Assuming they will bluntly stop ignores how vital relative motion is to safe traffic, not to mention the car-to-car communication options available given a significant number of automated cars on the road -- arguably human drivers can never be as smooth as a fleet of automated cars, and "slamming the brakes" is how people handle uncertainty already
Presently the tech is immature, but the potential for automated transport systems lies well beyond the realms of human capabilities.
Self driving cars are still subject to the laws of physics... unless you're going to dictate that self-driving cars never go above 15mph, I wouldn't advocate jumping in front of even a "perfect" self-driving car.
Braking distance (without including any decision time) for a 15mph car is 11 ft, for a 30mph is 45 ft. Self driving cars won't change these limits. (well, they may be a little better than humans at maximizing braking power through threshold braking on all 4 wheels, but it won't be dramatically different)
So even with perfect reaction times, it will still be possible for a self-driving car to hit a human who enters its path unexpectedly.
Then we should at least learn their capabilities by throwing jumping dummies at them. Call it the new dummy testing. It's the least we can do. Did Travis bro do this when he set the plan in motion?
Because it's assumed human drivers will at least have a desire to stop. If a machine isn't properly programmed, it will plow right through a crowd and never look back.
Maybe not in the U.S., but in Norway, as part of getting the driver's license, you'll have to take an ice driving course, where, amongst other things, a dummy is swung at your car while driving on an oiled lane in 30 mph. (It is practically impossible to steer away from the dummy, which is the meaning of the exercise; to realize the folly of driving too fast on icy roads)
Good point -- there's a 4% chance of fatality when struck by a 15mph car versus 20% at 30mph.
There's a 30mph city street near me where cars routinely go 45mph -- the fatality rate jumps up to 60% at that speed. So just having cars follow the speed limit would go a long way toward reducing fatalities.
I suppose you're suggesting there's a distinction between someone losing x hours of life due to travel time vs. someone being killed and losing y hours of life. I'm sure you can see why someone attempting to create a reasonable policy might avoid making that distinction.
I thought you were asking about "the cost of driving slower"? I'm perfectly serious in my answer. Any transportation goal (with the possible exception of ambulance service?) that may be accomplished at high speed, also can be accomplished at lower speed, with sufficient planning.
jerf addressed this above. What you're saying isn't accurate. Doing things more quickly avoids wasting hours of life. That's a benefit. The cost is measured in hours of life of people killed and injured as a result of doing things at a chosen speed.
What activity are you talking about, that is feasible at 40mph in an urban environment but not at e.g. 25mph? Can you not imagine a different way of conducting that activity?
There is a fundamental inequity between the operator of dangerous equipment comparing hours and years of life, and the pedestrian who suffers the consequence of that comparison. The USA auto industry is built on this inequity, which is why no one ever talks about it.
Many people meet friends and make deliveries e.g. via bicycle. Those motorized vehicles that are set aside specifically for deliveries often travel more slowly than other motorized vehicles.
But I shouldn't pick nits; you were speaking in generalities! So when I said "any transportation goal" and you disagreed, you hadn't actually thought of a particular exception to my universal statement. And you still haven't thought of one. Are your contributions to this discussion offered in good faith?
1. I want to visit a friend. He lives an hour away by car. New speed limit changes it to two hours. I lose an hour of life in the car.
2. Auto plant needs radiators. Truck delivering them takes longer to get them there. Costs $x more. Car costs more. Car buyer has to work longer to buy car. Car buyer wastes y hours of life.
I suppose this subthread is complete, because you have now completely agreed with my original statement: you'll have to leave earlier for your trip to visit your friend, and delivery trucks will have to allocate more time (routes, trucks, drivers, etc.) for their deliveries. Alternatively, freight trucks might make fewer mostly-empty trips. Note that these two examples clearly match the characterization I provided: both may be accomplished at lower speed, with sufficient planning. This will be the "cost" of safe driving.
Or maybe you don't visit your friend so often. Or you decide that 90 minutes on the train is better than driving, or maybe your friend gets tired of making a 2 hour trip to the city, so he moves closer.
There are lots of alternatives that don't involve you spending more time driving.
While it's true that goods will cost more to transport if it takes longer, that higher charge is amortized across many products in the truck, so is a very small portion of the finished product.
So if a radiator fits in a box 16x24x6" or 1.3ft^3 and a 40ft truck holds 2400 ft^3 (subtract 20% since it won't be a perfect fix, so 2000 ft^3, so you can fit 1500 of them in the truck.
If a truck+driver costs $100/hour, that means each radiator will cost 13 cents more.
Or, another way at looking at it -- all of the parts that make up a car aren't going to be bigger than a car (sure, some space is lost to packaging, but there's a lot of empty space in a car), and 6 - 10 cars can fit on a car carrier truck, so each car will end up costing around $25 more.
Though since we're talking about urban speed limits, and there aren't many urban car manufacturers, slow urban speed limits won't affect the price of cars.
But what about an even lower speed? If 15 mph is good, then 5 mph is better. And if 5 mph is better, 1 mph is superior once more.
I think we can agree there is a point where slow becomes too slow and the 'sufficient planning' becomes an unreasonable burden. So given we aren't operating off the notion that slower is inherently better, then there is some equation giving us our optimal point. What if that point is 45 mph instead of 15 mph?
In short, how do we argue that 15 mph is better than 45 mph that can't also be applied to speeds lower than 15 mph?
It's easy - you have diminishing improvements in pedestrian survival rates. 40MPH+ is associated with a fatality rate of over 50%, whereas you get to 30MPH and you have 7%, and 20MPH is essentially zero: https://nacto.org/docs/usdg/relationship_between_speed_risk_...
Even if it is close to zero, how do we decide if the lives saved from going from a .1% fatality rate to a .09% fatality rate is worth the speed reduction or not?
I'd like to think that the economic cost of regularly killing people on the streets is higher than getting to a place ten minutes faster. American traffic fatalities per year are basically equivalent to killing off a large town.
No one is saying reduce speeds everywhere. But in an urban context with lots of pedestrians, these speeds matter, and urban traffic is generally so stop-and-go and congested that drivers rarely sustain the top speed, and reducing it doesn't actually affect travel time by all that much.
"I'd like to think that the economic cost of regularly killing people on the streets is higher than getting to a place ten minutes faster."
However comforting that logic may be, it is unusable in the real world. If you value lives infinitely, then you will never ever do anything that risks your or somebody else's life in order to gain on any other need. You routinely engage in things that are not the safest possible option in order to fulfill other needs, ranging from food or water acquisition through mere entertainment. Therefore you place a finite value on your life. Don't feel bad, so does everybody else. It is possible to determine the value placed on life, I believe there are studies that show the value is more stable than you might think, and balance things appropriately.
It may be uncomfortable thinking, but, again, unless you literally never take even the smallest risk in the pursuit of other goals, you are already thinking this way. You just haven't lifted it up to the conscious level yet.
No one is asking for infinite value. A cursory glance at causes of death rates in USA reveals that far too many people are dying in automobile collisions. There is nothing about our world that requires that level of carnage. Future generations will find our customs ghastly.
As in the famous Churchhill quote, once you agree it's not infinite, now we're just dickering about price.
I'd say you're almost right about nobody asking for infinite value, but I'd say it's more like nobody who has pulled this up to the conscious level is asking for infinite value. People who have not examined the belief are quite prone to speaking as if life's value is infinite... but their own actions inevitably belie that claim. Once examined, it becomes rationally obvious that life is not infinitely valuable (including your own), but, well, if humans automatically accepted and believed all rational things they examine without emotional consequence the world would be a very different place.
Agreed, your parent post was strawmanning. Is there a name for this second-order sort of meta-strawmanning, in which we imagine people's unconscious inclinations, rather than merely imagining their arguments?
The United States government led by the Obama administration came up with values from 6-9 million dollars when weighing marginal costs of safety regulations.
>It is possible to determine the value placed on life, I believe there are studies that show the value is more stable than you might think, and balance things appropriately.
For evaluating safety regulations relative to the cost of inplementation, NHTSA values the risk of loss of human life in relation to the market value of risk reducing products, and the safety they provide.
They extrapolate from the take rate of airbags and their cost and effectivity, to how much value the average American places on their own life. IIRC on the order of $5 million.
FYI, juries do not look kindly on companies that implement this in liability suits. The data is that if a corporation writes down a $-figure for a (statistical) human life, it anchors punitive fines at a higher level.
By that logic, we should mandate all cars go 1 mph everywhere. Doesn't matter if it takes forever to get anywhere as long as no idiot jumping out in front of cars dies, right?
Well, you do waste minutes of peoples lives.
Let's see: 100 mio people driving 250 days/year, losing 10 min each way (so 20/day). That's about 12k lives of 80 years. It's actually worse, because you are wasting "awake time", so add 30%. It seems America isn't /that/ far of from the optimum. Maybe better driver education or better roads would be more effective?
The real issue with driving slower is enforcement. Enough people don't take into account the speed limit that changing the speed limit without changing the roads leads to unsafe mixed speeds.
Universally adhered to lower speed limits in urban environments would be great.
Correct. I often walk down a road with a 20mph 'limit' where most cars are doing around 40 - and some appear to be doing more like 50. There's simply no economically viable way to enforce it, so it will continue like this until there have been a couple of fatalities.
Not a chance. And that is a good thing. This knee jerk towards "lets just monitor everyone, everywhere and automate the law" is antithetical to a free society. Most people know that, which is why speed camera votes always send that company (RedFlex or whoever) packing.
I don't know about the US, but where I live speed camera are relatively large bright colored boxes with reflective stripes on the side of the road, with mandatory "speed camera ahead" warnings.
Most of them are empty but people unfamliar with the place will usually slow down.
The laws are state and local. In AZ it's a city or township, then the people put it on the ballot and it gets shut down. Tucson voted 65% no cameras, and later we got a state wide ban on highways, so it's a still a work in progress. The tickets are civil law, so you can throw it out, frame it, or make a coffee table book if you get enough:)
You can lower speeds quite drastically with better street design, no enforcement needed. Make the road narrower and curvier instead of wide open and straight. You can even add bumps.
Certainty of enforcement. People are careful to obey a rule punished 100% of the time with a $1 fine. They brush off a 10^-5 probability of a $100,000 loss by thinking "It won't happen to me."
>Indeed. This is why many cities are reducing speed limits.
In fact, self-driving cars may actually improve the situation if cars actually start complying with speed limits en masse.
The vast majority of people just go however fast they feel comfortable (considering conditions, etc) regardless of the speed limit.
Mixed traffic speeds decrease safety.
Raising speed limits so that you don't have the % of people who comply with the letter of the law traveling slower than the people who go however fast they're comfortable usually improves safety.
Unless your goal is to increase ticket revenue or appease the "think of the children crowd" there's no point to lowering speed limits. It doesn't do much to affect traffic speed. To do that you have to modify the road or do something to change the traffic flow.
Self driving cars will improve safety because they'll result in political pressure to raise speed limits to match reality and they'll make dynamic speed limits more practical.
"Mixed traffic speeds" is a concept for 4-lane highways, not 1 or 2 lane city streets. It would not take many law abiding vehicles to bring a one or two lane road down to the speed limit.
Sure, but city streets are already mixed traffic. There are pedestrians, bikes, vehicles parking or turning, etc. It's not reasonable to raise the limit to what people want to drive and just ignore all the other users of the street.
Also, the optical narrowing mentioned in a sibling comment is quite effective. They've done that on a few streets near me via things like sidewalk bulb-outs at intersections, and swapping the parking lane & bike lane (so it goes curb-bike-parking-drive, rather than curb-parking-bike-drive). Everyone drives more slowly on those streets now - myself included.
It does work when the roads are designed properly. No, posting a random speed sign isn't going to slow down traffic. But speed bumps, chicanes, turns, smaller lanes, etc. will naturally slow down traffic, making it impossible to drive dangerously fast.
Exactly. That is why we should ban all cars anywhere there is people, and they should be limited to highways. Because safety is most important, right? Right???
> Unless your goal is to increase ticket revenue or appease the "think of the children crowd" there's no point to lowering speed limits. It doesn't do much to affect traffic speed. To do that you have to modify the road or do something to change the traffic flow.
It’s very simple, put speed cameras on every corner, and fine in terms of day wages. E.g., one week of your income as fine for going x% above.
Several countries are doing parts of this already, or moving towards it.
I'm greatly in favor of a slower-moving but more efficient automotive network. How many human inconveniences are caused by what boils down to impatience? See: gridlock, people entering intersections they cannot leave, and jamitons.
(IMHO) jamitons would dissolve if people would leave a flexible buffer between them and the car in front of them and focusing on minimizing braking, rather than driving up to their bumper, brake, wait for moving, accelerate, brake, repeat. The lag due to reaction time and ac/deceleration exacerbates the "viscosity" of traffic flow. If most drivers focused on "staying fluid" rather than hurry-up-and-wait, traffic ought to improve. Like fluidized beds.
Unlike humans who have limited vision, self-driving cars are generally able to observe all obstacles in all directions and compute, in real-time, the probability of a collision.
If a car can't observe any potential hazards that might impact it using different threat models it should drive more slowly. Blowing down a narrow street with parked cars on both sides at precisely the speed limit is not a good plan.
You could do that, but how far do you take it? Do you program a car to slow to 10mph every time it passes a parked delivery van just in case? Would people find that acceptable?
Yes? If self driving is being hailed as being safer, it, you know, should be. If that means doing all the boring stuff I would not bother to, so be it. How else will they be safer unless by ignoring our driving biases?
Once upon a time when I was learning to drive, one of the exercises my instructor used was to put me in the passenger seat while he drove, and have me try to point out every person or vehicle capable of entering the lane he was driving in, as soon as I became aware of them. Every parked vehicle along the side of a road. Every vehicle approaching or waiting to enter an intersection. Every pedestrian standing by a crosswalk or even walking along the sidewalk adjacent to the traffic lane. Every bicycle. Every vehicle traveling the opposite direction on streets without a hard median. And every time I missed one, he would point and say "what about that car over there?" or "what about that person on the sidewalk?" He made me do this until I didn't miss any.
And then he started me on watching for suspicious gaps in the parked cards along the side that could indicate a loading bay or a driveway or an alley or a hidden intersection. And so on though multiple categories of collision hazards, and then verbally indicating them to him while I was driving.
And the reason for that exercise was to drive home the point that if there's a vehicle or a person that could get into my lane, it's my job as a defensive driver to be aware of that and be ready to react. Which includes making sure I could stop or avoid in time if I needed to.
I don't know how driving is taught now, but I would hope a self-driving system could at the very least match what my human driving instructor was capable of.
Sounds like you had a great driving instructor. Although I never had an experience like that when learning to drive, we did have to complete something in the UK called a "hazard perception test"[1] in order to get a drivers license. Basically a video version of what your instructor did for you. Until reading your comment today, I'd never really put much thought into how useful this is and how ingrained in my everyday driving it is.
A thing I like to do to stay in practice is to browse /r/roadcam over on reddit. I open a video at random and watch it, and try to guess where the collision (or near-collision) is going to come from.
That sounds like a great exercise, I wish it was standard. But I'm guessing you must have taken this course in a rural area, or else be able to talk faster than an auction caller. I don't think I could list off all the hazards in an urban area fast enough. :)
Phoenix transportation infrastructure is notoriously bad at handling pedestrians and bicycles because so few people are going to be out biking in 120F summers. There's a knock off effect of exclusively designing a city around middle class car and house types of folks. It's why I left after 20 years in Phoenix and Tempe.
So, there's a performance envelope expected. Sure, someone can bungee jump off an overpass and not be avoidable. :)
But they should be willing to walk in front of it in an in-spec performance regime. There's some really good Volvo commercials along that line, with engineers standing in front of a truck.
Apologies for going off topic here, but I'm curious about this. I've tested every car I've ever owned and all of the recent cars with all-round disc brakes have outperformed this statistic, but I've never been able to get agreement from other people (unless I demonstrate it to them in person).
I'm talking about optimal conditions here, wet roads would change things obviously but each of these cars was able to stop within it's own car length (around 15 feet) from 30mph, simply by stamping on the brake pedal with maximum force, triggering the ABS until the car stops:
2001 Nissan Primera SE
2003 BMW 325i Touring (E46)
2007 Peugeot 307 1.6 S
2011 Ford S-Max
I can't work out how any modern car, even in the wet, could need 45 feet to stop. In case it's not obvious, this is only considering mechanical stopping distance, human reaction time (or indeed computer reaction time which is the main topic here) would extend this distance, but the usual 45 feet from 30mph statistic doesn't include reaction time either.
In addition, some cars such as the Nissan Leaf have a feature that locks the brakes at full power when there is a sudden control shift from accelerator to brakes, meaning that if you stomp on the brakes and then reduce pressure, the car will continue to brake at maximum power. This was done for two reasons: one, people hesitating in emergency situations, and two, people being taught to pump brakes, which increases braking distance in cars with ABS.
I learned about this feature when reading car forums for my car and finding threads from people who were rear-ended when they accidentally triggered this feature by slamming on the brakes when they didn't intend to come to a complete stop.
> In addition, some cars such as the Nissan Leaf have a feature that locks the brakes at full power when there is a sudden control shift from accelerator to brakes, meaning that if you stomp on the brakes and then reduce pressure, the car will continue to brake at maximum power.
Which is utterly stupid. Braking is the natural reflex, but not always the right one. I've been in more than one close call (think left turn on incoming traffic) where the correct response was not to floor the brake, but floor the gas.
> people being taught to pump brakes, which increases braking distance in cars with ABS.
Which is a mechanical turk version of what the ABS is doing under the hood.
I knew I'd get called out for not including sources. Those figures are from published sources, and do not include decision time. I'd imagine that these sources are for "average" roads and "average" cars
Car and Driver did a test with sports cars and profesional drivers and came up with 142 - 155 ft from 70mph, while my first reference quotes 245 ft (around 40% less, so extrapolating, their stopping distance from 35mph would be around 20 feet).
The average car on the road is not a sports car with performance tires and is not stopping on a clean, dry track. So I don't think it's a stretch to assume that an average car on average roads with tires optimized for tread life would be 40% worse than a $100K sports car with $400 tires that are optimized for grip rather than lifetime.
> I'm talking about optimal conditions here, wet roads would change things obviously but each of these cars was able to stop within it's own car length (around 15 feet) from 30mph, simply by stamping on the brake pedal with maximum force, triggering the ABS until the car stops
How did you measure that? Because plugging these figures into a uniform acceleration calculator, 50km/h to 0 in 4.47m requires a deceleration of 2.2g but "Analysis of emergency braking of a vehicle"[0] experimentally measured very best case deceleration as barely scraping 1g (with ABS at 80km/h, significantly lower at lower speeds or without ABS).
If self driving cars are limited to speeds that allow them to stop within their lidar max range is that too slow? Humans don't have the pinpoint accuracy of lidar but our visual algorithms are very flexible and robust and also have very strong confidence signals e.g driving more carefully in dark rain.
Cameras are not accurate enough though, their dynamic range being terrible. Wonder how humans would fare if forced to wear goggles that approximated a lidar sensors information.
The advantage self-drivers have is in
1. minimizing distraction / optimizing perception
2. minimizing reaction time.
Theoretically self-drivers will always see everything that is relevant, unlike a human driver. And theoretically a robot-driver will always react more quickly than even a hyper-attentive human driver, who has to move meat in order to apply the brake.
But is that the actual situation we're talking about? Or are we actually talking about a situation where the person may have been jaywalking but would have had a reasonable expectation that a human driver would stop? I walk a decent distance to work every day and I don't think anyone totally adheres to the lights and crosswalks (not least because if you do you will be running into the road just as everyone races to make a right turn into the same crosswalk you're in).
My opinion is that we will come to a point where self-driving cars are demonstrably, but only marginally, safer road users than humans.
From an ethical standpoint the interesting phase will only start then. It‘s one thing to bring a small fleet of high tech (e.g. having LIDAR) vehicles to the road. It‘s another to bring that technology to a saturated mass market which is primarily cost driven. Yes, I assume self-driving cars will eventually compete with human driven ones.
Will we, as a society, accept some increase in traffic fatalities in return for considerable savings that self-driving cars will bring?
Will you or me as an individual accept a slightly higher risk in exchange for tangible time savings?
Indeed, history gives us ample examples, but I would‘t call humanity stupid because of that. Each and every such example is a ethical question that we have to solve one way or the other. Burying our heads in the sand will not make technologically and expensive self-driving vehicles go away. Neither will it prevent cheap self-driving clunker.
> demonstrably, but only marginally, safer road users than humans.
I believe the claim is 38 multitudes better than humans, significantly better than marginally.
> accept some increase in traffic fatalities
No. And the question is more about "some" than "some increase"
> accept a slightly higher risk in exchange for tangible time savings?
Texting while driving and even hands free talking were becoming laws in many states before smart phones -- and my experience is that many people readily accept this risk and the legal risk just to communicate faster. The same can be said for the risk of drunk driving -- it's a risk that thousands of Americans take all of the time.
IIRC the people who programmed the first auto pilot for a major airliner were required to be on board the first test flight, so I have to think their testing methodology was pretty meticulous.
It wasn't but it increasingly is getting to that point. You take off (with fly by wire adjustments from the computer) from Airport A and tell it to fly to Airport B and it can even automatically land there.
Some airlines do not allow pilots to manually fly above 3k feet, nor allow first mates to land manually[0].
Which then gets you pilots frobbing the autoland selector, not getting the mode right - and now what, if you have zero practice, having scripted yourself out of it? Now you fly the plane into the runway: https://en.wikipedia.org/wiki/Asiana_Airlines_Flight_214
There is a (prevalent) happy path where autopilot and autoland can get you the whole illusion "but the plane flies itself" - but the whole system is built around humans catching and handling all and any exceptions.
> The people leading the development of these horse-drawn carriages should demonstrate that they can stop for pedestrians by jumping in front of them on a closed test dirt-path. If they're not able to demonstrate this, they shouldn't be putting them on public carriageways
Sounds silly when compared against old tech.
Accidents happen, best we can do is try to prevent them.
very good point, I would not be surprised if Uber put those cars out with a barely working model to collect enough training data, the human operator to correct such errors was tired and didn't intervene. Some basic driving test for self driving cars or other mitigating factors need to be added IMMEDIATELY otherwise tons more will probably die or be injured. Train operators need to prove they are awake and with attention using some button presses, similar things need to be required for those research vehicle if you want to allow them at all.
"The people leading the development should demonstrate that they can stop for pedestrians by personally jumping out in front of them on a closed test road. If they're not able to demonstrate this, they shouldn't be putting them on public roads."
Although the actual logistics of your proposal might be challenging (child comments point out that some speeds/distances might be impossible to solve) your instinct is a correct one: the people designing and deploying these solutions need to have skin in the game.
I don't think truly autonomous cars are possible to deploy safely with our current level of technology but if I did ... I would want to see their family and their children driving in, and walking around, these cars before we consider wide adoption.
goes to show, sdv's have to be perfect in the eyes of the public. you wouldn't seriously recommend adding a test like that to a regular driving licence.
also that testing does happen in the case of every av program i know of. closed obstacle courses with crap that pops out forcing the cars to react. look up gomentum station.
This isn't a good argument because it implies if these AVs have successfully not killed their CEOs during a closed (i.e. controlled) test, that they are safe on public roads. But it seems like the majority of AV accidents so far involve unpredictable and uncontrolled conditions.
IOW, setting this up as some kind of quality standard gives unjustified cover ("Hey, our own CEO risked his life to prove the car was safe!") if AVs fail on the open road, because the requirements of open and closed tests are so different.
human driven cars are currently, as we speak running people over on the streets and they have human drivers who don't particularly want to run over other humans. It was inevitable that this happened, and no matter how many people self driving cars run over will be worth it, since it will still be less than what cars are currently doing in terms of death toll.
This reminds me of a story I heard in college... an owner of a company that builds table saws demonstrated his saw's safety feature - a killswitch that pulls the blade down into the machine as soon as it detects flesh - by putting his hand on it while it was spinning.
The reason self-driving car companies have flocked to Arizona is that Arizona chose to pretty much completely deregulate them: They don't even have to report statistics to the state like they do in California.
It's unfortunate, but unsurprising, the first pedestrian killed by an experimental self-driving car allowed on public roads was in Arizona.
What will the consequences of this person’s killing be? Will someone lose out on a promotion, or miss their performance bonus?
We need to discuss how the developers self-driving cars will be held accountable for the crimes they commit. There is no reason the person who programs or the person who makes money from a self-driving car should be held less accountable for a crime that if committed directly by a person would almost certainly result in jail time. You can’t replace yourself with a computer program and then choose to take only the benefits and not the responsibilities.
This is crazy. The developers likely have no say in where and when the cars go out on public roads. That's obviously a decision for someone higher up in the company.
The executives should be held accountable, not the developers.
I partially agree. “There is no reason the person who programs or the person who makes money from a self-driving car...”
The person who profits the most should be held the most responsible. But the separation of roles between the executives and the developers is likely to mean that no one gets punished at all.
Why should the person who profits most be held the most responsible? Suppose the person in charge of safety clearance makes less money than the original developer. How does that shift responsibility to the developer, as opposed to the situation where the developer makes less money?
I disagree, you're just passing the buck. Accountability needs to be had at all levels. If an engineer writes a bug into code like this (deliberate or not) and such a bug results in somebody's death, the engineer should be held accountable just as much as the person who approved its release. The executive could just as easily say "my engineers promised me it was fully tested", etc. Engineers could say "yep it was, but that was an edge case we missed" or something like that. In any case, there needs to be shared accountability. Maybe execs take the brunt, but engineers should not be allowed to write code that kills people (inadvertently or otherwise) and face zero consequences.
What software developer would ever sign on to a project where they could be held criminally liable for a single bug?
Do you want software development to turn into healthcare, where every developer needs millions of dollars of malpractice insurance? Because shit like this will turn it into a healthcare like system real quick.
I'm sure plenty of people would but that isn't the point. If you're writing code that potentially costs people their lives, you need to be able to be held accountable otherwise it will lead to negligence. This isn't a new problem... maybe for the software space, but not for industry as a whole.
Humans don't suddenly become perfect actors just because incentives align. The stress of that risk and efforts taken to mitigate it seems like it would actually make the software worse.
It's up to the product (the collective of individuals that deliver the product) to address and mitigate the risk it creates, that's not solely on the shoulders of individual software contributors.
If A writes a generic computer vision algorithm and open sources it, B integrates that into a "is this a bomb or not" product with a white paper outlining its failure rate in a specific situation, then C sells that product to D who uses it in an entirely different situation and E gets blown up... who gets sued? It definitely should be somebody, there should certainly be a liability and incentive to avoid such a liability but I it probably lies somewhere in C-D space, not A-B space.
If the only options you’re presenting are “move fast and break things” where those things are human lives, or introducing burdensome bureaucracy, I’ll take the bureaucracy. Time and again society chose that latter option, and it will again. Unaccountablility is worse than regulation, and history has shown that repeatedly.
What software developer would ever sign on to a project where they could be held criminally liable for a single bug?
Do you want software development to turn into healthcare, where every developer needs millions of dollars of malpractice insurance? Because shit like this will turn it into a healthcare like system real quick.
How else to interpreted that? When a single bug can cause loss of life, and given that this in a thread about Uber, it’s hard to draw other conclusions. By all means though, offer another perspective on how regulating industries with significant number of lives on the line can’t manage regulation. While you’re doing that, I’d point to the aerospace sector which seems capable of both innovation and regulation.
There's a difference between holding someone criminally responsible for a bug in code that they wrote, and some sort of regulation. They are not the same.
Although the NTSB investigated the accident, it was unable to conclusively identify the cause of the crash. The rudder PCU from Flight 585 was severely damaged, which prevented operational testing of the PCU.[3]:47 A review of the flight crew's history determined that Flight 585's captain strictly adhered to operating procedures and had a conservative approach to flying.[3]:47 A first officer who had previously flown with Flight 585's captain reported that the captain had indicated to him while landing in turbulent weather that the captain had no problem with declaring a go-around if the landing appeared unsafe.[3]:48 The first officer was considered to be "very competent" by the captain on previous trips they had flown together.[3]:48 The weather data available to the NTSB indicated that Flight 585 might have encountered a horizontal axis wind vortex that could have caused the aircraft to roll over, but this could not be shown conclusively to have happened or to have caused the rollover.[3]:48–49
On December 8, 1992, the NTSB published a report which identified what the NTSB believed at the time to be the two most likely causes of the accident. The first possibility was that the airplane's directional control system had malfunctioned and caused the rudder to move in a manner which caused the accident. The second possibility was a weather disturbance that caused a sudden rudder movement or loss of control. The Board determined that it lacked sufficient evidence to conclude either theory as the probable cause of the accident.[2]:ix[3]:49 This was only the fourth time in the NTSB's history that it had closed an investigation and published a final aircraft accident report where the probable cause was undetermined.[4]
Second:
In 2004, following an independent investigation of the recovered PCU/dual-servo unit, a Los Angeles jury, which was not allowed to hear or consider the NTSB's conclusions about the accident, ruled that the 737's rudder was the cause of the crash, and ordered Parker Hannifin, a rudder component manufacturer, to pay US$44 million to the plaintiff families.[16] Parker Hannifin subsequently appealed the verdict, which resulted in an out-of-court settlement for an undisclosed amount.
You interpret it as written, which is that holding developers routinely criminally liable for bugs is going to have very negative effects. One of them is that the only developers you'll get are precisely those too unwise to realize what an incredibly stupid deal that is, no matter what the pay rate is. I don't think I'd like to see all my critical software written by such "unwise developers".
I have no problem "piercing the veil" for egregious issues. I'd have no problem holding a developer liable for failing to secure a project but just continuing on rather than quit. But "Let's just hold all the engineers criminally liable all the time!" is a bad idea and it is not already done for a reason.
It’s not done because software development is an unregulated shitshow full of wildly unethical companies scrambling for the bottom. It’s not unlike early aerospace, or early medicine, or any frontier which develops rapidly before legal frameworks inevitably close in.
This is not true at all. First of all, there's no such thing as being able to mathematically prove a design is sound in any engineering discipline, software or non-software. After all, it is infeasible if not impossible to encapsulate all the details of the implementation of _any_ system in mathematics or any other system of reasoning (down to every last atom, if you stretch your imagination).
All we have in engineering (non-software) is something like safety factors and confidence, and this is done with (usually) rigorous mathematical models as well as loads and loads of testing to fill in the gaps of mathematics (think unknown constant/parameters, assumptions, etc).
None of this is impossible to do for software. There are systems that enable one to do easy/entry level verification (such as something like TLA+), to much more complicated reasoning (something like COQ). This will allow the system designers to gain confidence in if the system will work and gain understanding about under what scenario they will fail. Contrast this with the existing software landscape, which is mostly, at least from my perspective, just let me write some stuff until things do approximately what I want. Even at the top of the ladder, I feel the tests conducted are "adhoc" at best and with none of the rigours that you associate with traditional engineering fields.
Healthcare costs are not unreasonable in much of the developed and developing world. Most countries have better outcomes and lower costs than here in the US. As another commentor says, healthcare seems to be doing fine; you seem to be assuming the US is the norm when it isn’t.
> Healthcare costs are not unreasonable in much of the developed and developing world. Most countries have better outcomes and lower costs than here in the US. As another commentor says, healthcare seems to be doing fine; you seem to be assuming the US is the norm when it isn’t.
I'm going to cauterize the off-topic debate about the US healthcare system by pointing out that OP was talking about the expense to doctors of malpractice insurance, not about costs to the patients or medical outcomes.
Malpratice liability varies widely by country, but it's a non-trivial expense for doctors everywhere, and significantly higher in states with strong tort liability for doctors.
It's hard to imagine a world with criminal liability (or tort liability) for software engineers that doesn't ultimately end up with a system of insurance for engineers, roughly analogous to the medical malpractice insurance system for physicians.
I am afraid that you are introducing the off-topic debate. The end goal of healthcare is better outcomes for lower prices. Likewise, the end goal of engineering should be better technology for lower costs.
That healthcare in other countries is able to achieve this in spite of the medical malpractice insurance system points to the fact that such a system is not certain have to have the deleterious effects you confidently assume.
Whether it is a burden for engineers is another question. But the article and the discussion aren’t about the inconveniences faced by the engineers who programmed this system.
Which, as someone else noted, exists and is probably a good idea if you're an independent consultant or possibly a professional (i.e. licensed) engineer who signs off on drawings or other documents for clients or regulators.
Criminal liability is a different situation as there are very few industries with specific criminal liabilities (finance maybe).
But there are many industries where civil liabilities are required. In fact, any software independent consultant is civilly liable for their work, but it’s not specific to software.
The costs aren’t that high or at least they weren’t 15 years ago when I purchased it for less than $1k/year for $1M in coverage. Most people need this even if they think they are safe. If you’re the one who wrote the deployment script that erased $1M in data, it won’t be entirely mitigated that the script made it through qa.
Also interesting is that the engineer who wrote the Uber software is currently liable for criminal negligence, like pretty much everyone else. But you would have to prove culpability. I can’t find any examples of software engineers convicted so it’s hard to tell who goes to jail-developer, qa, or executive.
Nobody in their right mind would work with such liability without insurance, which is all well and good for civil liability, but insurance won't help if you're going to jail.
Almost all employees have the possibility of criminal negligence based on their work. For programmers, this could mean that if you fuck up the code for a pacemaker and someone dies, you could go to jail. That’s a big risk and I can’t find any programmer who has been found culpable for someone’s death. This is the current law in the US.
If Uber was negligent in its code, then the programmers could go to jail. They have programmers and they work and assume this extremely low risk.
Now maybe you’re arguing that some special law should or should not exist for Uber drivers.
This happens all the time in aerospace. You need to sign off the software personally and you need to be an accredited engineer to be allowed to do that.
They do have say in who they work for and what they work on. It's not as if a developer capable of doing self-driving work isn't in high demand. Maybe they should stop working for unethical companies doing unethical things. It's not as if the executives could do this work themselves.
We already have plenty of case law and policy for cases where people are killed by mechanical equipment operated by businesses, and the type of penalties and compensation are appropriately different if the cause was negligence, malice, or impossible-to-eliminate fluke events. (Generally in the latter case, compensation is due but there are no criminal charges.) Obviously there will be policy adjustments and clarifications for the case of self-driving cars, but I don't think there's reason to think we can't apply normal and existing legal principles here.
There is a massive difference in terms of scale and choice (FWIW). Industrial automation is most likely to kill you if you work in the plant. The person who died here was a random pedestrian. If these cars were restricted to special areas the analogy might make more sense, but I don’t expect to be dealing with a self-driving car or an industrial robot when I step outside my front door.
Moreover it is not clear to me that not holding the companies that create industrial robots that kill people criminally responsible is what most people would consider just. Again, I think it’s just that there is a massive difference in the scale of exposure; there were not enough interested people to have a debate.
> If these cars were restricted to special areas the analogy might make more sense, but I don’t expect to be dealing with a self-driving car or an industrial robot when I step outside my front door.
As a pedestrian you already run a significant risk of being killed by a car. To the extent that we hold autonomous car makers responsible for these deaths (and I'm not saying we shouldn't), we should hold non-autonomous car makers responsible for the deaths their vehicles cause as well.
We do hold non-self-driving car makers responsible for bad manufacturing. But in accidents not due to manufacturing we primarily hold the human drivers responsible. I agree with you overall, but the problem is that people seem overeager to hold no one responsible at all, sometimes based solely on a blind faith that self-driving cars will be safer than humans soon, and that the deaths along the way are just the price we will have to pay—as if there is no other option between no self driving cars at all, and the “move fast and break things” attitude that here resulted in a person’s death.
The thing to remember is that limiting self-driving cars is not safe either. Human-driven cars kill thousands of people every day; a policy that saved this person's life but set back self-driving car development by even (say) a month might well do more harm than good.
cars are already machines built by companies which sometimes malfunction and kill people (both drivers of the vehicles and people around them), this is just a new way in which they can malfunction, I don't think it's as dramatically different as you're saying
You’re right that there are already ways in which non-self-driving cars can malfunction. But previously we held human drivers responsible for certain kinds of accidents. For these same kinds of accidents we now propose holding no one responsible. That seems to be the dramatic change to me.
We have held humans responsible because assuming a correctly functioning car they are performing the most complex and risky task, and are most able to cause problems. Likewise self-driving car software performs a complex and risky task in which failure can have serious consequences.
There's already such a thing as a no-fault collision. There's also already such a thing as a collision where the manufacturer is at fault. I feel like this stuff is all covered in driver's ed.
And there is such a thing as an at-fault collision. Is what you are saying supposed to be a contradiction? Also, I have a license and drive regularly; I don’t see how your strange assertion I must not is productive.
Because those vehicles have licensed human operators. The malfunctions may be to blame on the manufacturer, but are also licensed and regulated. The cars have to pass certain crash test standards for example.
In this case, the operator was an AI that was negligent and it was unlicensed/unregulated. That's a new scenario. In the human case a person might go to jail for negligent vehicular manslaughter. What does 2 years of jail time look like to an AI? What does a suspended license look like to an unlicensed entity?
Right now we have something of a two tier level of liability which would for the most part work fine with automated vehicles. The primary liability falls on the owner/operator, who usually carries insurance. The owner/operator has some level of self interest in maintenance of the vehicle - otherwise an automated vehicle might have a perfect design, but the maintainer never changes the brake pads or operates with the tires worn, etc. If the insurance company finds that some model vehicle has reason to doubt it's design integrity, then that liability may be passed on in a separate case to the manufacturer. An individual owner is actually in a poor position to know systematically if there is reason to bring suit over a subtle design or manufacturing defect, but an auto insurance company has both data and the resources to see and react to defects.
For choice: manufacturer failures happen with normal cars, and you risk that every time you step outside your door. Likewise with building failures, construction accidents, etc.
For scale: the risk of death from a self driving car will probably be less than the current risk of death from normal cars, and will definitely be less than the risks incurred in the 20th century from cars, buildings, etc.
Self-driving cars are definitely a new and large legal development, but there's no reason to think existing legal principles can't handle them.
No, this is not equivalent to the risk of existing manufacturing defects in cars. Car bodies undergo safety tests by the government; the software for these self-driving cars is being tested on public streets. Same with buildings, which must be inspected.
As the GP states, the entire reason Uber is testing in Arizona is because their state government completely got rid of reporting regulations which were present in CA; the status quo is decidedly not the same as it is for established technologies.
As for scale, look at the other comments where people analyze the risk posed by self driving cars. Your assumption that the risk of death from self-driving cars is less is not backed up by the evidence.
It’s fine to say that self-driving cars might eventually be better drivers than humans, just like robots might eventually be better at conversing than humans.
There is no reason self-driving cars can’t be be tested in private. Uber can hire pedestrians to interact with them—I don’t volunteer to be their test subject by deciding to take a walk.
First you started by claiming the difference was due to scale and choice. You're now retreating to a third distinction: the difference between established technology and experimental technology. Well, all established technology was experimental technology at one point, and it was not uniformly regulated. We could play this game all day.
Self-driving cars are a new and important industrial development that will require adjustments to policy. They don't require revolutionary new legal principles.
If you hold developers responsible, you can kiss self driving cars goodbye.
What should be passed (but I can't see how) is a percentage of allowed deaths, at least in the early years, and set it to something like 5-10% of the current rate, reducing downwards to 1% after 20 years.
People will die from self driving cars, and undoubtably their will eventually be a case that is 100% the self driving car's fault. The benefit of self driving cars comes from the mistake being permanently fixed, while with human drivers it can be committed over and over again.
There needs to be some kind of protection on the companies (and obviously the developers, I've never heard someone try to say they should be held responsible before) from lawsuits. Otherwise all it'll take a is a small handful beefore companies will just let it die.
An act doesn’t become okay because two people (the executive and developer) and a robot are now responsible instead of one. What is your justification for the sort of utilitarian calculation you’ve made here? Why do you assume self-driving cars will be safer without any evidence?
If we are going to be arguing from a utilitarian standpoint, suppose we hold the executives of self-driving car companies as responsible as if they were themselves drivers. Then if self-driving cars truly are safer as you optimistically claim, both fewer people will die from accidents involving them and fewer people will go to jail for those same accidents. Seems like a win to me.
It's quite obvious that self-driving cars will at the very least eventually be safer than human beings. There's plenty of things humans do that a machine simply can't do for starters (drink, lose focus, drive aggressively). Add in that a machine can react instantaneously in a defined manner (versus panicking, or just not reacting fast enough), this becomes such a stupid argument it's really just not worth anyones time to entertain.
Why is it at all quite obvious? How is arguing for being careful “such a stupid argument it’s really just not worth anyone’s time to entertain”?
Someone in this discussion has an insane amount of blind faith in technology which here literally killed a pedestrian, and it’s not the people who are arguing for just consequences.
Are you arguing that a machine does not have better reaction times than a human being? Are you arguing that a machine can fall asleep, drink and drive, panic in a high stress situation?
Aren't you the same person who called for holding the developer liable for writing software with a bug? Are you accusing the developer of promising something that is impossible (not hitting a pedestrian in a crosswalk?) or simply implementing it wrong?
It's worth pointing out that we have no idea yet who is at fault in this accident. It could easily be someone who simply walked out in front of traffic when they weren't paying attention.
The answer to your first “question” of course is that it depends on the machine, how it’s built, programmed, and the context of operation. Machines can have much faster reflexes, or they can freeze.
"Are you saying X" is a pretty aggressive way to frame your argument.
The above poster seems pretty clear that it is NOT obvious that cars will necessarily drive safer than humans on average, in the same way it is NOT obvious that we will ever have General Artificial Intelligence.
These are very complicated problems, and the machines are currently (significantly) worse than human drivers, so I think it's fair to question the argument that "everything will work out eventually"
Theoretically it could be otherwise, perhaps, though the human brain has an extremely parallel pattern-matching engine honed by about half a billion years of evolution.
Realistically, the self-driving system will be made of layered distinct components that all add latency. This is how we build both hardware and software. An image is sensed, it gets compressed, it gets passed along the CAN bus, it gets queued up, it gets decompressed, it gets queued up again, object detection runs, the result of that gets queued up for the next stage... and before long you're lucky if you haven't burned a whole second of time.
Machines can drive aggressively.
There was a university that had self-driving cars do parallel parking... by drifting. Driving along, the car would find a parking spot on the other side of the road. It would steer hard to that side, break traction, swing the rear of the vehicle around sideways through a 180-degree turn, and finally skid sideways into the spot. The car did this perfectly.
That kind of ability is something that I personally don't have. I would consider a self-driving car that could do this. If I'm paying, and that kind of driving is my preference, I expect to get it.
I really don't want you to get your wish. We have no need to invest in flashy self-driving car stuntmen, building a car that can get you from A to B safely and in a reasonable time frame is all that we should be aiming for.
That sort of drifting parallel park might work most or nearly all of the time, but if the road conditions are poor and the car loses handling then it will be a lot more risky.
The camera feed going straight to the neural network will not have a lot of latency. The neural net will not take very long to process the image and make a decision. Humans need at best half a second and at worst several seconds to recognize, process, and act. These systems are designed to be fast to responds. They do not have a second of latency.
> Civil engineering and medical device manufacturing seems to be doing fine, despite having similar principles of engineers' liability.
The idea that software engineers should be held responsible for something that (as far as we can tell so far) was an accident and not the result of negligence or malice is several orders of magnitude beyond the level of liability that civil engineers and medical device manufacturers have.
> that software engineers should be held responsible for something that (as far as we can tell so far) was an accident and not the result of negligence or malice
Nobody said that. The original comment said developers should be held "accountable for a crime that if committed directly by a person would almost certainly result in jail time" [1].
The standards from medical devices and/or civil engineering, with the associated licensing requirements and verification processes, make sense. Even in the case of a careless mistake or strategic oversight, individuals who could have known but nevertheless signed off should be identified, if not explicitly punished.
> > that software engineers should be held responsible for something that (as far as we can tell so far) was an accident and not the result of negligence or malice
> Nobody said that.
Well, they quite literally did, because the original comment in this thread was:
> We need to discuss how the developers self-driving cars will be held accountable for the crimes they commit. There is no reason the person who programs or the person who makes money from a self-driving car should be held less accountable for a crime that if committed directly by a person would almost certainly result in jail time. You can’t replace yourself with a computer program and then choose to take only the benefits and not the responsibilities.
I guess you can quibble about the difference between "accountable" and "liable", but that's not a discussion that's particularly interesting to have here, especially given OP's other comments in this thread which make it quite clear that this is what they had in mind.
The quibble in this case would be the meaning of "developers." In the case of a medical device, the developer is considered to be the Manufacturer, not the specific software developers on the team. Considering how often teams change, etc., using the latter definition would be meaningless.
If you don't understand the difference between proving software and proving civil engineering projects, you aren't qualified to be participating in a discussion on software liability.
> What should be passed (but I can't see how) is a percentage of allowed deaths, at least in the early years, and set it to something like 5-10% of the current rate, reducing downwards to 1% after 20 years.
Whith so few self driving cars that number sold be zero. If you can't assure safety with a few cars whith a human as backup, you should not be in the streets. And it's not the first dangerous accident of an Uber self driving car where Uber was at fault.
I'm guessing people are downvoting because of the implication that the software team should be held responsible?
Something does feel wrong about punishing them when the decision to put the car on the road in the first place was almost certainly not their own.
Though I agree Uber should be held accountable for it and it shouldn't be a token fine since the whole point of punishing an accident like this is to discourage them from occurring in the first place.
This sort of accident orchestrated by a group of people probably won't be gracefully handled by our legal system.
> You can’t replace yourself with a computer program and then choose to take only the benefits and not the responsibilities.
Look, I'm all for developers and (software companies in general) to be considering the ethical implications of the work they do, and the moral obligations that they take on as a result of it. However:
> We need to discuss how the developers self-driving cars will be held accountable for the crimes they commit. There is no reason the person who programs or the person who makes money from a self-driving car should be held less accountable for a crime that if committed directly by a person would almost certainly result in jail time. You can’t replace yourself with a computer program and then choose to take only the benefits and not the responsibilities.
This is a bad mentality to take with postmortems for software failures in general, at least from the outset. You need to look at the underlying factors that contributed to the issue, not simply looking for a person to assign blame to. It's possible that negligence is the underlying cause, but not necessarily - and even if negligence is a cause, what were the other cultural factors that led to the negligence happening, without being caught somewhere else in the pipeline? It's tempting to look to assign blame, but if you do that, you'll actually miss out on the systemic improvements that would be necessary to prevent similar incidents in the future.
But moreover, this is a bad outlook to take here, because this wouldn't be criminal behavior if committed by a human. From the best we can tell, given the details available so far, it's an accident, and it's very rare for criminal charges to even be considered in accidents like these, unless it's a hit-and-run.
> this wouldn't be criminal behavior if committed by a human
I can assure you that a human who is driving carelessly would be held criminal liable. Why do you assume an accident that was severe enough to have resulted in a persons death—the car didn’t just scrape them because they ran across the street—is not due to a reckless programming?
> I can assure you that a human who is driving carelessly would be held criminal liable.
Do you see evidence that the car was "driving carelessly"? That's an honest question - from the reporting so far, it doesn't seem clear what the underlying cause was.
Secondly, this is demonstrably false: most pedestrian fatalities by vehicles do not result in criminal charges. If you don't believe me, look up the stats. Or talk to the countless bikers' advocacy groups that have been lodging this exact complaint for decades: drivers are not generally held criminally responsible, unless there are mitigating circumstances (the driver is drunk, the accident was a hit-and-run, etc.).
> Why do you assume an accident that was severe enough to have resulted in a persons death—the car didn’t just scrape them because they ran across the street—is not due to a reckless programming?
When a pedestrian dies, just because they died, that doesn't mean the driver is automatically responsible. It could have been the pedestrian's fault, or it could have been the driver's fault. Or it could be both. Or it could even be neither (a true accident, with no assignment of blame).
The same thing holds here. You can't assume that this is the result of "reckless programming", and to be entirely blunt, by jumping to that conclusion on the basis of literally no evidence whatsoever (and misinterpreting existing case law on vehicular accidents in the process), you're actually undermining the success of any future efforts to prevent these sorts of accidents in the future, whether or not it ultimately turns out to be the fault of someone at Uber.
You have good points, thanks for discussing this. I think for me the fundamental problem is that with a human we can characterize reckless driving as driving that a normal, competent human would not do. But there is no “normal, competent” self-driving car-so by what standard do we determine the program’s behavior to be reckless as opposed to just acceptable?
I accept your point that this accident might not have led to criminal charges if a human had been responsible. But I don’t waver on my argument that if a human driver would have been held criminally responsible for this accident, then we should we hold the executives (or in extreme cases programmers) of Uber responsible in exactly the same way, whether that be criminal or not.
Finally, with humans and pedestrian fatalities many cases involve drunk driving or sleepy driving. Self-driving cars can’t get drunk or sleepy; they can just have bad programming or bad hardware, both installed by their manufacturer.
There are soooo many instances of negligent drivers killing cyclists with basically no follow-up from the police. Police all over the US seem to consider cyclists as second-class road users, and trust the driver when they say a cyclist "came out of nowhere". Since these sorts of collisions are more often fatal for the cyclist than the driver, there often isn't anyone to tell the other side of the story. There are rarely criminal charges, and even more rarely convictions (juries are mostly drivers, not cyclists).
The developers have no control over the sensors, tires, weight of car, testing budget, human backup operator, or a million other things that went into this happening. Hell developers probably told who ever they could to not release this. Management at Uber and who ever approved this thing should be held accountable.
> The developers have no control over the sensors, tires, weight of car, testing budget, human backup operator, or a million other things that went into this happening
If one of those things caused the accident, the developer isn't to blame. Civil engineering has experience tracing liability from mistakes (and incentivizing prevention).
The realistic alternative isn't California-esque regulations, it's a country with poorer (more disposable) people. Arizona is a nice compromise, at least we can see what's happening.
"ha ha only serious" it's an observation of how the world is (see: factories and recycling operations in Asia, mining in Africa), not how it ought to be.
Would you please stop posting uncivil and/or unsubstantive comments to Hacker News? We eventually ban accounts that do this, and have already asked you once.
The other reason is that AZ has very simple grid-based roads without a lot of complexity in determining right-of-way. As a lifelong Arizonian who recently moved to Oregon - I'm consistently surprised at the amount of computing I have to do to figure out how I'm supposed to handle some intersections/on+off ramps around here.
1,862 comments
[ 3.1 ms ] story [ 444 ms ] threadI think the same thing applies to autonomous things.
So yeah.
Autonomous vehicles are expected to be able to safely navigate a large variety of streets and highways, and possibly trails and countryside in the future, without killing anybody else around or killing the people inside, nor destroying the fixtures and buildings that make up your typical city street. The only problem is cars are inherently deadly, go look up the number of people that have died in automobile "accidents" since the automobile was invented, total it up for just one country. They need to be treated with and afforded the same respect as any other instrument intended to project deadly force, because they do, as do whatever algorithms are controlling the instrument.
Everyone I've known in law enforcement, especially CHP officers, does this by default (though they tend to prefer “collision”) when referring to an event where a car hit something. As one of them explained, “collision” or “crash” is a readily discernable physical fact; “accident” is a conclusion about the mental state of the participants.
The idea that an "accident" isn't the drivers false is an absurd premise. Drivers are obligated to be diligent. "Crash" is a neutral term that doesn't assign liability to anyone.
Sure, you might shoot yourself because you're just careless with your gun. Or you might be also thinking of an argument you had with your spouse, or you might be tired when you're handling your everyday carry on your way to work, or you might get startled, or a million and one other things that might prevent you from singly focusing your attention the gun you're handling.
Machines exist to serve us, not the other way around. If a product doesn't take human habits and flaws into account, it's a bad product, because over a large enough sample size someone will make a mistake.
It's not, because the basic rules of gun handling are designed in such a way that failing to obey one of them will not result in injury. They are:
* Never point a gun at anything you don't wish to destroy * Keep your finger off the trigger until you're ready to fire * Always assume the gun is loaded * Know your target and what's beyond it.
> Machines exist to serve us, not the other way around. If a product doesn't take human habits and flaws into account, it's a bad product, because over a large enough sample size someone will make a mistake.
Modern firearms are absolutely designed to this criteria - it takes a willful act to discharge them. If someone "mistakenly" fires a gun and hits someone, it requires multiple failures and clearly rises to the point of negligence.
You know what system we've come up with to improve car safety that has only one rule to remember? Always wear your seatbelt.
You know what we've come up with that requires zero effort from the driver? Laminated glass. Crumple zones. Airbags. Automatic braking.
This isn't about whether people use a product negligently — put it in enough hands and it will happen. The point is to design products such that the consequence of negligence is not injury or death.
- AI decides between one life lost as a better potential outcome
- AI made no decision and just happened to hurt someone
The idea is that making a decision between killing one person vs killing something more valuable than one person (eg. multiple people) is something that drivers do on a regular basis. Computers may not be capable of correctly evaluating the ethics of that decision.
For example - What if someone steps out into the road and you can either hit them, or run into a wall? The former makes it far more likely the occupants of the car will be safe, the latter saves the life of a pedestrian at the expense of possible injury to people in the car.
It is kind of an interesting question, because people can't generally react quickly and rationally enough in a situation like that for it to matter, but in theory, a computer can. But it also dominates the discussion in a way that likely far overstates how frequently this will actually be a decision a computer has to make.
I think that is what the person you're replying to is asking. Given that this occurred at night and on (very likely) uncrowded streets, I feel like its far more likely the car (and backup driver) just somehow missed the pedestrian.
I.e. did the uber car kill this 1 pedestrian, because it avoided killing 2 other people? Did the uber car opt for the "least killing" in this siutation?
I think this whole trolley car question - while a nice philosophical question - is silly though. I dont think humans would do any better in an emergency situation. In that split second when you suddenly find yourself in the "oh sh-<IMPACT>" situation, do you have time to a) make a deliberate, reasoned & fully-informed decision, and then b) control the vehicle effectively to follow through on that decision? I doubt it. If you had a second or two to think it through and then deliberately make a choice and steer towards that crowd of nuns, you'd probably be able to entirely avoid the accident anyway.
More likely is you'd probably do what I am sure most people do which is gasp, stamp on the brakes, shut their eyes and hope for the best ... assuming you even had time to realise there was about to be a crash before it happened. Some people might swerve, but I suspect they do that instinctively to avoid something in their way, not as a decision to hit something else.
I just dont think self-driving cars will get themselves into situations where they have to chose who to kill in the first place. And even if they did get into those situations, I really, really doubt it would be due their actions, and I'd certainly trust it to avoid the crash in the first place a whole lot more than the average human driver in the same scenario.
Some could argue "Ah but yes computers are so fast that they CAN make that informed decision about who to kill in milliseconds! So they have to make a choice! The question stands!". I'd argue back that in those milliseconds before the crash was inevitable, they'd act to prevent the accident before a human would even know what the hell was going on anyway. After that it just starts getting into a game of who can conjure up the most ludicrous hypothetical situation that rarely - if ever - happens in real life.
For example, if a kid runs out into the road in front of the car, it can drive into the kid, likely with no injuries to the passengers. Or it can swerve and drive off of the road, or into another lane, likely saving the kid but putting the passengers at much greater risk.
As you say, in this or any similar situation the human reaction is probably going to be to instinctively brake or swerve without a chance to consider options or consequences. I feel like a computer will have a few cycles to spare to make decisions like that.
I go back and forth with thinking the algorithms in things like self-driving cars should be public. Anything which must make a choice such as smash into bus or smash into bicycle — the public should be aware of the choices the company chooses to implement. Obviously this example is simplistic but it illustrates my point.
Today I’m definitely leaning towards it should be public and scrutinized. Next month I may be worried this would hamper progress.
(Do we count Tesla's autopilot as self-driving?)
I'm very curious to see how they'll investigate this and who will be determined to be at fault (person behind the wheel or Uber). It will likely set a precedent.
Semi-autonomous cars have drawbacks.
If anything, airplanes prove that machines doing most of the work and humans stepping in only when necessary is a proven model.
https://www.usatoday.com/story/travel/columnist/cox/2014/08/...
Airplanes have far less traffic to deal with. And the points at which they deal with traffic (e.g. takeoff and landings) is completely controlled by humans, including many humans outside of the plane.
They're used in low visibility conditions with relatively calm weather, but don't work well in bad weather.
This is preferable, because a child, blind person etc needs to be able to use the autonomous vehicle alone.
https://news.ycombinator.com/newsguidelines.html
This is not how many people drive in non-autonomous cars? Commuting is on auto-pilot for many of us; you don't really remember how you got to work and I see people tying their shoelaces, calling, eating sandwiches, reading the paper, chatting on whatsapp all the time. Sure it's all illegal (in a lot of countries), but people are bored and drove that road 1000x.
No, that is not how people drive in non-autonomous cars. The phenomenon you're referring to is related to how sparsely information is stored to memory when doing a routine task (even when you were fully concentrating). Just because you can't recall an activity in detail does not mean you were not paying full attention.
However, these are test vehicles. The driver's full-time job is to be focused on the road and what the car is doing. They shouldn't have any misconceptions about the need for them to stay focused on the road. Now granted, even then attention will lapse occasionally, just like it does when you are actually driving. But I don't think that being physically in control of the wheel is necessary for maintaining focus at the same level as a good driver. Driver's Ed instructors need this skill. Also some back seat drivers I know are quite good at maintaining strong focus on the road regardless of who is driving :)
http://www.econtalk.org/archives/2015/11/david_mindell_o.htm...
However, in my opinion, the driver and by extension his employer Uber must be held responsible in this case. Uber have a license to test their self-driving cars only if there is a human driver in control at all times. Regardless of the crazy things their software decides to do, the human is there as a person of ultimate legal liability.
It's also plausible that the pedestrian was at fault. Under some circumstances it would be impossible for a human driver to avoid collision with a pedestrian, this may be no different.
Hopefully with self driving vehicles we can start to move beyond this attitude, especially when there should be ample camera footage available to help dispel the usual claims that the person struck "came out of nowhere" or whatever.
I think it's time to point out the obvious, and require that autonomous cars apply the brakes first, and THEN require driver intervention.
And that they be a whole lot quicker to err on the side of braking.
Cameras getting fuzzy ?
Slow down.
Your ML algorithms are showing lower confidence measures for how they classify nearby objects and trajectories ?
Slow down.
nearby vehicles slowing down and you don't know why?
Slow down.
This is inexcusable.
It's not that simple, you're assuming the car even had some indication that something was wrong. For all we know the car's vision was showing high confidence it saw an open road.
Slow down?
Waymo et al has had none of these issues. Time to revoke their licence to test?
I mean, it's Uber. I'm not sure exactly what people were expecting. Their entire schtick is being unethical and poorly-run.
Criminal negligence, not direct malicious criminality.
[sorry, dang. you're right.]
https://news.ycombinator.com/newsguidelines.html
"Sir, we have several bugs, one of them would cause the car to run a red in the following conditions... [fictional made-up condition: in case it sees a clown car with Brad Pitt wearing a pink dress in it]".
"What are the chances of that ever happening? Ship it!".
I don't know if this job ad for a self-driving simulation software engineer is still current, but it is somewhat worrying given the broad scope of the role they appear to be advertising for.
[0] https://www.uber.com/en-US/careers/list/27029/
If there’s more to your thought than “the ends justify the means,” I’m curious what it is.
This is like saying the Duterte is a very effective anti-drug crusader if you accept that murdering innocents and drug dealers is in his DNA.
a) what % of disengagements are 'to avoid accidents' vs navigating an entirely safe but complex situations. For ex: a parked UPS truck on a narrow 2 lane rd.
b) the degree of real-world complexity the tests the cars are engaging in (compared to other vendors). For ex: testing cars on simple suburban routes = fewer disengagements.
Not trying to defend Uber here, I'm just trying to understand the distinction between a high-level general statistic and it's real world implications.
There is no reason whatsoever to believe that Uber's horrific disengagement record, which is orders of magnitude worse than competitors often in the same cities like SF, is because they are tackling orders of magnitude harder situations per mile or are orders of magnitude more conservative, and every reason, even before this fatality (the only one so far despite many self-driving programs running concurrently over years), to believe they were just plain worse.
> Not trying to defend Uber here, I'm just trying to understand the distinction between a high-level general statistic and it's real world implications.
Again, it has to be dangerous for someone. It can't be more dangerous yet not dangerous to anyone in particular. That just doesn't make any sense. Uber runs on the same roads and the same traffic with the same basic approach as everyone else, there's no confounder which could produce a reversal.
Exactly. Fool me once, shame on you; fool me twice, shame on me.
It tells a lot about Uber that they weren't willing to fork over $150 for a permit.
People will always relish in their biases being confirmed... even after a rare one-off event with a sample size of 1 in a burgeoning industry.
https://www.theverge.com/2017/2/27/14698902/uber-self-drivin...
Nothing to revoke if they never had a license to begin with!
Local governments have stopped other self-driving companies before with cease-and-desist letters with threats of serious punitive fines. I highly doubt by not buying a $150 license Uber would suddenly have carte-blanche to do what they want, that's not how it works.
If the state gov was really interested in stopping them they easily could have regardless.
> Waymo et al has had none of these issues.
... absence of negative media coverage doesn't prove anything.
I think Alphabet is to blame here too in a sense (or at least the industry as a whole). Why didn't they push for more strict testing rules? It's in their interest too.
[1] - https://www.theguardian.com/technology/2016/dec/19/uber-self...
Assuming Arizona hasn't produced law specifically for fatalities in their public road self driving test program... The algorithms are just one detail of the total system design, and the design of the system isn't really the issue. People don't get killed by designs or code.
The verification of it's safety, and the decision to deploy that system on uncontrolled public streets will be the issue. People made these decisions, not an algorithm.
Arizona has something similar:
> By legal definition, there are three or more crosswalks at every intersection whether marked or unmarked.
Source: ADOT Traffic Engineering Guidelines and Processes, Section 910.1
Today you can cross in the middle of the street and take a calculated risk that the people approaching down the street will stop or slow down enough to not kill you. If they go ahead and kill you anyway, they'll still be prosecuted because their attention should be on the road, and the most basic assumption behind their driver's license is that this person is competent enough and fit enough to drive what amounts to a deadly weapon without killing the people around them. When you violate that assumption, the fault is probably on you. Obviously there are exceptions, such as when somebody intentionally jumps in front of your vehicle, but crossing outside of the crosswalk to get across the street is not even close to the same level as trying to commit suicide and fault will be found accordingly.
But yeah. We all agree it's not cool to run down pedestrians or bikers or pedestrians walking bikes etc. no matter where they may be.
If you require 100% impossibility of killing anybody regardless of what the vehicle and the person is doing - it is achievable only by making those vehicles nearly useless - such as lowering their max speed to something 10 mph (maybe even lower since it's still possible to push a person who will slip, hit their head on the pavement and die). If the vehicle is moving fast and somebody jumps onto a street, there are physical limitation of what can be done. So, if this technology is to exist there will always be a space where accidents can - and eventually will - happen.
That might be a fair question, except that one death at this early stage of limited realistic trials makes me doubt the correctness of your belief. On the statistics that we have right now, I'm not seeing how autonomous cars are safer.
>Self driving cars aren't going to be perfect.
Shouldn't self-driving cars be held to a better standard rather than compared generously to objectively terrible existing standards? That we think of pedestrians regularly killed in "accidents" as acceptable collateral is already pretty horrific.
Why?
NHTSA reports 1.15 fatalities per 100 million vehicle miles travelled in 2015.
Waymo advertises >5 million road miles travelled. Let's say Waymo + Uber have driven 10 million vehicle miles and killed 1 person. That makes them 10 times more dangerous than human driven cars.
https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in...
2 deaths in over 2 billion miles is roughly 10 times safer than human drivers.
[1] https://electrek.co/2016/11/13/tesla-autopilot-billion-miles...
How about we test it properly before allowing it on public roads?
"The Uber vehicle was reportedly driving early Monday when a woman walking outside of the crosswalk was struck.
...
Tempe Police says the vehicle was in autonomous mode at the time of the crash and a vehicle operator was also behind the wheel."
That's a very bad look; the whole point of self-driving cars is that they can react to unexpected circumstances much more quickly than a human operator, such as when someone walks out into the road. Sounds like Uber's platform may not be up to that standard yet, which makes me wonder why they're on public roads.
On the other hand, it sounds like it happened very recently; I guess we'll have to wait and see what happened.
This tendency to buzzword out of obvious truths (NN's are programs for example) derails rational discussion.
https://news.ycombinator.com/newsguidelines.html
Doing these things derails discussion much faster than an obvious comment, so would you please (re-)read the rules and abide by the spirit of this site when posting here?
AI (short of AGI) is never guilty; it's creators and operators, OTOH...
Perhaps there will be unexpected 'issues' meaning that such data will have been lost in this case.
1. Uber really had a tool that did this IRL: https://www.theverge.com/2018/1/11/16878284/uber-secret-tool...
E.g. Brazil was at odds with WhatsApp over their message encryption — Brazil wanted WhatsApp to stop encrypting messages so that it would be possible to subpoena chat logs. If Brazilian police raided a hypothetical WhatsApp Brazil office, would it not be prudent for Facebook to cut off data access from that office? Especially given that the nature of technology means that once police have an employee’s laptop and can compel them to enter their password, they have access to essentially all of the company’s data worldwide.
But let's extrapolate. Say one day there are 20 self driving car companies. Should they be required to share what they learn so the same mistakes aren't repeated by each company or does the competitive advantage outweigh the public benefit from this type of information sharing?
They seem to be adjusting quite a bit in the last 1-2 years so that is the definition of learning. Otherwise, just showing how companies were once dumb is usually not very useful in knowing how adaptable they are.
I agree that we’ll know better looking back how they react to this event.
Should "not killing pedestrians" be a competitive advantage? No. It should be mandatory for every vehicle.
Therefore,we should make sure that everything works towards this goal, including sharing data!
The argument I've always heard is that an autonomous systems will outperform humans mostly by being more vigilant (not getting distracted, sleepy, etc.) rather than using detectors with superhuman reaction times. Obviously, whether or not this outweighs the frequency of situations where the autonomous system gets confused when a human would not is an empirical question that will change as the tech improves.
And, like, once the systems are mature and hardware has evolved and so forth, I think that's right. It will. It might today under certain circumstances, like if it gets a really unambiguous sensor input (or it might not).
But I've never heard anyone actually associated with a driverless car program assert that their vehicles have superhuman reaction times today, and the vehicles drive extremely cautiously. I think it's likely that due to their difficulty in understanding their sensor readings, if you look at total time necessary to make a course correction from the point when an obstruction first could be noticed by an alert human driver, driverless cars are not winning and may be substantially losing in at least some cases.
From what I've seen I wouldn't trust autonomous cars to "understand" all situations. I would trust Waymo cars to understand enough to avoid hitting anything (at a level better than a human), at the risk of being rear-ended more often. Everything I've seen from Tesla and Uber has given me significantly less confidence than that.
I agree with the sentiment though. This has been a major selling point for this technology, but it has not been sufficiently demonstrated yet.
(It certainly didn't predict the crash "seconds before", as the car accelerated into the imminent colission, until <1sec before collision.
Something that is endemic in HN discussions of driverless cars is commenters who dramatically overestimate the dangers of humans driving. Like an order of magnitude or more. You see it all over this thread, in which people imagine that killing one pedestrian in like let's say 10 to 20 million miles driven (with those miles overwhelmingly done in unchallenging conditions) constitutes "vastly safer than human drivers" rather than "vastly less safe than human drivers."
Driverless cars simply don't have the mileage yet to prove they are safer than driverful cars. But there's also no indication they are less safe, based on casualty rate so far.
There is an indication that they're less safe! They have (very conservatively) 5x the number fatalities per mile driven! Now, look, the error bars on that estimate are of course massive. It is plausible that they are much safer and they just rolled the dice and got unlucky. But this is data, as long as you include the error bars.
Toyota has enough UX data that they added "brake assist", turns out that a lot of accidents happen when the user stomps the brake, but then releases, just never presses it hard enough to come to a complete stop, or was trained before anti lock brakes.
https://www.youtube.com/watch?v=grcuorbrYxA
Another scary thing about braking is that most cars probably don't have their brake fluid changed often enough; most owners and even dealers treat it as a lifetime interval, when it may be closer to an annual thing interval.
That's an invalid conclusion to draw from this accident.
There were 34,439 fatal motor vehicle crashes in the United States in 2016 in which 37,461 deaths occurred. This resulted in 11.6 deaths per 100,000 people and 1.16 deaths per 100 million miles traveled.[1] As far as I know, the instance this article is about is the first death in an autonomous vehicle accident, meaning that all the 2016 accidents were humans driving.
Why is it that you see one death from an autonomous car and conclude that autonomous cars aren't ready to be driving, but you see 37,461 deaths from human drivers and don't conclude that humans aren't ready to be driving?
I admit that there just aren't enough autonomous cars on the road to prove conclusively that autonomous cars are safer than human-operated cars at this point. But there's absolutely no statistical evidence I know of that indicates the opposite.
[1] http://www.iihs.org/iihs/topics/t/general-statistics/fatalit...
EDIT: Let's be clear here: I'm not saying autonomous cars are safer than human drivers. I'm saying that one can't simply look at one death caused by an autonomous car and conclude that autonomous cars are less safe.
I'm not sure I see anyone here making that conclusion. I think you're the only one who's brought it up.
For one, I personally sure as fuck do not think humans are ready to be driving.
> That's an invalid conclusion to draw from this accident.
The conclusion was not invalid at all. Other self-driving car companies have driven more miles than Uber has and they have done so safely. Uber has even taken its cars off the road, so even Uber agrees that their self-driving cars are not ready for the roads yet.
It is also important to take into account what Uber is like when it comes to safety and responsibility. They have knowingly hired convicted rapists for drivers, they have ridiculed and mocked and attempted to discredit their paying customers who have been raped by their employees/drivers, they have spied on journalists, they have completely disregarded human safety on numerous accounts. A company with a track record like Uber's probably not should be granted a license to experiment with technology like these self-driving cars on public roads.
They just aren't responsible enough.
Look at the post I'm responding to, and the section I quoted.
> Uber has even taken its cars off the road, so even Uber agrees that their self-driving cars are not ready for the roads yet.
Uber avoiding a PR nightmare should not be taken as Uber thinking their self-driving cars aren't ready.
> It is also important to take into account what Uber is like when it comes to safety and responsibility. They have knowingly hired convicted rapists for drivers, they have ridiculed and mocked and attempted to discredit their paying customers who have been raped by their employees/drivers, they have spied on journalists, they have completely disregarded human safety on numerous accounts. A company with a track record like Uber's probably not should be granted a license to experiment with technology like these self-driving cars on public roads.
I'm not defending Uber as a whole. I think in general they're a fairly typical rent-seeking middle man whose only real innovation has been figuring out a way to break the law ambiguously enough to get away with it (admittedly, a law I don't agree with). I don't even necessarily think autonomous vehicles are a good thing. I just think that if you're going to criticize autonomous vehicles, safety isn't the criticism I'd level against them, because it's not backed up by the evidence when compared to how unsafe human drivers are.
https://www.newscientist.com/article/2095740-tesla-driver-di...
The fact that human drivers aren't particularly good isn't really relevant, beyond setting a correspondingly low bar within the null hypothesis.
This all ties to regulation allowing these vehicles to drive on public roads, because that regulation was permitted due to the above hypothesis and hopeful expectations that it would be true.
Obviously, I haven't seen the entirety of the data set to know fatalities per car-mile. Which would be the relevant statistic here. I also didn't see such a number in your post, which I'm assuming means you are probably not aware either. But simply providing the numbers for the null hypothesis doesn't do anything.
> The fact that human drivers aren't particularly good isn't really relevant, beyond setting a correspondingly low bar within the null hypothesis.
These two paragraphs are so contradictory to each other that I'm just not even sure how to attempt to correct your confusion.
I'd like to note that I am not the person you replied to, and I personally am not arguing that any program should be shut down based on this one incident. But it's certainly not encouraging that the autonomous vehicle seems to have failed a test at which everyone would have expected it to perform well at.
Also, to quote from your previous post:
> Why is it that you see one death from an autonomous car and conclude that autonomous cars aren't ready to be driving, but you see 37,461 deaths from human drivers and don't conclude that humans aren't ready to be driving?
I think we conclude, quite often in fact, that individual humans aren't fit to be driving. Death is one of those scenarios that will quickly lead to such a conclusion.
One huge difference between individual humans and autonomous vehicles is that we can reasonably argue that any Uber vehicle would have performed the same in this scenario. So this is perhaps more akin to saying that this particular driver is not fit for the task, except that this particular driver happens to be driving dozens or more vehicles all at once.
I said, "I admit that there just aren't enough autonomous cars on the road to prove conclusively that autonomous cars are safer than human-operated cars at this point." and I've vocally criticized autonomous cars elsewhere, so I'm not sure where you get the idea that I favor a blanket acceptance of autonomous vehicles.
I'm not saying that autonomous vehicles are safer, I'm saying this isn't evidence that autonomous vehicles are less safe.
> One huge difference between individual humans and autonomous vehicles is that we can reasonably argue that any Uber vehicle would have performed the same in this scenario.
I disagree: you can argue that the autonomous vehicles will behave the same given the same inputs, but they will never have exactly the same inputs even if the situation were identical to a human observer, so that's a fairly moot point. If you step back to a larger description of the situation (car crossing bike lane to get into a turn lane, car doesn't identify and avoid bicyclist in bike lane) then you are going to be looking at a percentage of the time where autonomous car will make a mistake. There's also a percentage of the time where a human driver will make the same mistake. The only way you can compare the safety autonomous cars to human drivers in this situation is to compare those percentages. And that's ignoring the fact that there are thousands of other situations in driving--even if autonomous cars fail 100% of the time in this situation, there may be enough other situations where they perform better enough than human drivers that they're safer. Simply saying that a car made a mistake in this situation doesn't give us any information at all.
> And that's ignoring the fact that there are thousands of other situations in driving--even if autonomous cars fail 100% of the time in this situation, there may be enough other situations where they perform better enough than human drivers that they're safer. Simply saying that a car made a mistake in this situation doesn't give us any information at all.
But it does give us information. This incident, along with all the other incidents and non-incidents that do or do not occur, as measured by incidents per car-mile. One cannot simply wish away this one incident and make it disappear. It is now forever part of the statistics which will either prove or disprove the hypothesis of autonomous cars being safer.
> I disagree: you can argue that the autonomous vehicles will behave the same given the same inputs, but they will never have exactly the same inputs even if the situation were identical to a human observer, so that's a fairly moot point.
It's not a moot point. Yes, this exact scenario with these exact parameters only occurred once. However, we can still reasonably argue that if we reversed time and replaced that exact Uber autonomous driver with another instance of the autonomous Uber driver and turned time back on, it would have reacted exactly the same. The same way that I expect the same version of Notepad to open my text file exactly the same on this computer as on another computer. The alternative being that there is some undeterministic behavior in the driver that is not tied to input... In which case, good luck with that in court.
However, we cannot make that same argument by replacing human drivers. Because each human is, in fact, different.
This is only important in the context that a fanciful revocation of Uber's "autonomous driver's license" would apply to all instances of the autonomous driver, since they would all have been reasonably expected to perform the same.
Can you point out the part of the article or the post that I was responding to which mentions how many car-miles were traveled?
This is exactly what I'm pointing out.
> The same way that I expect the same version of Notepad to open my text file exactly the same on this computer as on another computer.
Notepad doesn't have to read your text file through a lens with slightly different focus, viewing area, and patterns of dust on it each time.
> The alternative being that there is some undeterministic behavior in the driver that is not tied to input...
The undeterministic behavior is that the hardware which collects the input will never be the same. I don't know whether the software is non-deterministic (it wouldn't surprise me) but I know the hardware is never going to be identical--hardware is always made to tolerances and always has some degree of variability.
Your claim is tantamount to saying that if we put the same person in the same situation but with two different sets of eyes, the eyes would have no effect on the results.
> However, we cannot make that same argument by replacing human drivers. Because each human is, in fact, different.
Autonomous cars are, in fact, different. Just because they're running the same software doesn't mean they're the same; even if the software is completely deterministic, software is only a component of the autonomous driver.
> This is exactly what I'm pointing out.
I think, if your intent is to show that this is incomplete information, that...
1) No one is arguing that.
2) You have not done a great job of attempting to relay that, given phrases like, "Simply saying that a car made a mistake in this situation doesn't give us any information at all."
3) Sometimes that doesn't matter. For instance, Florida law is a mandatory 6 months to one year license revocation on DUI, regardless of the circumstances or information.
> Notepad doesn't have to read your text file through a lens with slightly different focus, viewing area, and patterns of dust on it each time.
Difficulty of the task is unrelated to expected outcomes of the task given the same inputs. And we already tread the topic of duplicating the exact situation... Not sure what you're trying to gain through this line of argument.
> Your claim is tantamount to saying that if we put the same person in the same situation but with two different sets of eyes, the eyes would have no effect on the results.
I am making no such claim, and I cannot believe that you are so adamant about not understanding my actual claim. This is a hypothetical situation. There is no mention in this scenario about changing the car, including any of the sensor hardware. I am interested in replacing only the driver (or driver software) into the exact same circumstance.
(EDIT: OK, reading back, I did say "any Uber vehicle". While your point stands, I think it's a very uncharitable reading. If hardware sensor tolerances and specs of dust on the camera are going to determine whether a life is lost or not, either those tolerances need to be driven down or this entire idea needs to be rethought. After all, we don't allow those who are legally blind to drive unless they have corrective lenses...)
Assuming the software is deterministic [1], by definition given the same inputs it will result in the same output. Therefore, "replacing" the autonomous driver with another would have resulted in the same incident. You cannot say that with any measure of confidence for any two pairs of human drivers.
[1] Which seems like it would be a good thing to assume, since I don't think one would get much traction by arguing that we should be putting vehicles with non-deterministic behavior on the road...
Are you making accusations in question form so you don't have to back them up? I certainly didn't say that.
> Why can't they fabricate some tests that demonstrate their safety in a controlled environment first?
Are you assuming they haven't done this?
> I'm not saying that autonomous vehicles are safer, I'm saying this isn't evidence that autonomous vehicles are less safe.
It's true that we need to wait for more information about this particular incident -- which is exactly the caveat that the commenter you initially responded to had said [0]. But assuming the facts aren't abnormally different than what they seem to be -- an Uber AV hit and killed a jaywalker, how is that not evidence toward the argument that AVs are less safe? It obviously isn't conclusive evidence. But if Uber goes on to kill a pedestrian for every million miles driven, this first data point would surely be part of the empirical evidence, no?
[0] https://news.ycombinator.com/item?id=16620042
Nope, I'm just having trouble grokking your argument and I thought that might be it. It's true I added a rhetorical edge to the language that was probably unnecessary. I apologize for that--I'm not trying to put words in your mouth. What is your argument?
But that's not really my point. My point isn't that self-driving cars are safer, it's that merely looking at one car accident doesn't prove that self-driving cars are less safe.
https://www.forbes.com/sites/bizcarson/2017/12/22/ubers-self...
so, 6 million miles driven and one fatality is significantly worse record than human drivers. So far.
If self-driving cars are between dumb drivers and normal drivers, and self-driving cars take dumb drivers off the road, it may be a net positive.
Let's say we have 9 normal drivers and 1 dumb driver. The 9 normal drivers cause 0 accidents per year each, and the 1 dumb driver causes 15 accidents per year. Let's say autonomous cars cause 1 accident per year. If autonomous cars replace all the drivers, there's 10 accidents per year instead of 15.
Obviously these are hypothetical numbers. All I'm saying is that if you only have one of these numbers, you don't know which kind of driver is safer.
https://techcrunch.com/2017/11/27/waymo-racks-up-4-million-s...
Nevermind that the initial rollouts almost always involve locales in which driving conditions are relatively safe (e.g. nice weather and infrastructure). Nevermind that the vast majority of autonomous testing so far has involved the presence of a human driver, and that there have been hundreds of disengagements:
https://www.theverge.com/2018/1/31/16956902/california-dmv-s...
Humans may be terrible at driving, but the stats are far from being in favor of autonomous vehicles. Which is to be expected given the early stage of the tech.
That's fine, we don't have to consider just fatal accidents:
https://www.theverge.com/2018/1/31/16956902/california-dmv-s...
> One result of the sharp increase in GM’s number of miles driven is a plethora of accidents. The auto giant’s autonomous cars were involved in 22 fender benders over the course of the reporting period (and two more in 2018). That’s one crash for every 5,985 miles of testing.
I didn't say that.
I'm not saying autonomous cars are safer, I'm saying that this accident doesn't prove autonomous cars are less safe.
> That you pulled out the IIHS stats without apparently bothering to look up the number of reported self-driving miles driven seems to suggest that you assumed that the fatality rate was obviously skewed against human driving.
I brought up the IIHS stats to show that merely reporting an accident in isolation from a full statistical model doesn't prove anything.
> That's a very bad look; the whole point of self-driving cars is that they can react to unexpected circumstances much more quickly than a human operator, such as when someone walks out into the road. Sounds like Uber's platform may not be up to that standard yet, which makes me wonder why they're on public roads.
> On the other hand, it sounds like it happened very recently; I guess we'll have to wait and see what happened.
How in the world do you see that as an assertion of proof.
This metric was often decried because it had poor statistical significance. It would however be nice to update this metric according to this death.
Maybe now this metric would indicate that IA are more dangerous than human, will be interesting if the perception of this flawed metric will evolve in reaction...
I'm curious to see that, do you have a link? That would actually be a valid comparison.
This is Elon musk bragging before the first autopilot accident : https://www.telegraph.co.uk/technology/2016/04/25/elon-musk-...
This is a counterpoint with some fact: http://safer-america.com/safe-self-driving-cars/
And concerning the crash, last update seemed to point driver’s fault... with autopilot... don’t know if final ruling is already out:
https://www.washingtonpost.com/news/the-switch/wp/2017/06/20...
People still need to look both ways when coming out from behind cars.
I want more details.
[1] https://en.wikipedia.org/wiki/Pilot_(locomotive)
Some of these accidents are unpreventable by the (autonomous) driver. If a pedestrian suddenly rushes out into the street or a cyclist swerves into your path, the deciding factor is often simply the coefficient of friction between your tyres and the road.
The autonomous vehicle and the human attendant might have made a glaring error, or they might have done everything correctly and still failed to prevent a fatality. It's far too early to say. It's undoubtedly a dent to the public image of autonomous vehicles, but hopefully the car's telemetry data will reveal whether this was a case of error, negligence or unavoidable tragedy.
Like my comment says, let's wait until we have facts before passing judgement.
But from this distance, things don't look good for the technology or the future of autonomous trials on public roads.
The evidence that due care was not taken will pretty much be the existence of the fatality.
There surely will be video, from the car itself and also perhaps third party security or traffic cameras. The level of carelessness we are going to have to see in order for a jury to blame the victim will be pretty high.
As for a fatality proving that due care was not taken, I agree. It doesn't tell you who failed to take due care though.
In law you split that into criminal and civil. Very unlikely there will be any criminal charge, hence no criminal liability but it’s possible.
Civil liability is more interesting, but if the deceased’s estate brings suit it opens up a can of worms in the discovery process. Every internal communication, prior incident, etc...
At the end of the day it’s not much different than any other vehicular fatality case, but there will be a defective product component (again nothing unheard of in the case of vehicular fatalities). At the end of the day the car will be insured for accidents, so we will probably learn more about the insurance coverage for self-driving cars.
When it comes to human drivers we deal with probabilities, but for some reason people want absolutes with autonomous ones.
Most drivers drive too fast. (Me too!) Most drivers have not killed anyone. All drivers who have killed someone, were driving too fast at the time. Many drivers will disagree with me, but they are simply wrong about appropriate speeds, and we will all be safer when robocars are driving for them.
I would say that if you claim that a massively complex system like public traffic, with thousands of participants, many of them badly or completely untrained, can be organized in such a fashion that any and all accidents can be prevented, the burden falls on you to show how.
Which is a lot of high level reasoning and inference with information from a variety of sources which aren't on the face of it strictly related to the driving task.
As long as self driving cars represent an improvement over human drivers, I’m ok with them having a non-zero accident date while we work out the kinks.
How do we handle this for autonomous vehicles? Do we just fine/sue the company that made the vehicle/developed the software? Do we send imperfect human developers to jail because they made a mistake, even if in the grand scheme of things they have saved lives compared to humans being behind every action made by a vehicle?
A big part of the public image for autonomous cars is increased safety, any deaths at their hands starts raising where and how to place the blame - a subject I think very few are prepared for right now, which is likely part of why Tesla explicitly states autopilot needs a human driver present right now, and why Google has been extremely cautious with operator-supervised tests up until recently.
But is it any surprise that Uber had the first self driving car to kill a human being? You can cook in an orderly, careful way, or you can turn the kitchen into a disaster zone and expect other people to clean up for you.
I know which I prefer when it comes to human lives.
And, medicine takes tens of thousands of lives unintentionally, if I recall correctly.
Medicinal testing kills tens of thousands?
But no, medical testing doesn't kill tens of thousands, practiced medicine does. I may have read your post wrong an figured you meant medicine in practice separately from military testing. There's research to suggest mistakes are the third leading cause of death in the US [1]. My point wasn't so much that killing random bystanders is okay, but that there's a level of unintended death in both, and not just in testing. Society decides what's acceptable, and if for miles driven the number of deaths that self driving vehicles cause is lower than the number of deaths caused by human drivers per miles driven, well... I'd say that's a fair way to look at it.
1. http://www.bmj.com/content/353/bmj.i2139
There's no way for autonomous driving to become a reality without people dying?
I think we could achieve autonomous driving without needless deaths along the way. It might take longer, but I'd say it's worth it to, I can't believe I have to make this argument, avoid killing people.
Lastly, the whole point of the human operator is to be the final safety check.
You're right that we have no idea of the cause until the data is analyzed (and the human operator interviewed). Yet, my first thought was, "Of course it'd be Uber."
This only true for the uninitiated, never let it get to that. I drove for Uber/Lyft/Via in New York city so I can experience and study these situation. These sort of accidents are preventable. The following is the basic:
1.) Drive much slower in areas where a pedestrian and cyclist can suddenly cross your past.
2.) Know the danger zone. Before people "jump into traffic" or a cyclist swerve in front of you, they have to get into position, this position is the danger zone.
3.) Extra diligence in surveying the area/danger zone to predict a potential accident.
4.) Make up for the reduce speed by using highways and parkways as much as possible.
It helps that Manhattan street traffic tends to be very slow to begin with. Ideally I will like use my knowledge to offer a service to help train autonomous vehicles to deal with these situation. It has to be simulated numerous times in a closed circuit for the machine to learn what I've learn intuitively driving professionally in NYC.
Around the bars, people stumble into traffic all the time. Any driver, automated or human, would need to anticipate that.
So essentially everywhere? I have seen pedestrians walk unexpectedly into traffic on high-speed 4-lane divided roads with no crosswalks.
> 2.) Know the danger zone. Before people "jump into traffic" or a cyclist swerve in front of you, they have to get into position, this position is the danger zone.
What "position" are you referring to? There are places where you can account for or predict pedestrians. There are also places where you cannot, such as when someone walks into traffic from behind a tall parked vehicle, where you have no chance to see them in advance.
And that'ss with humans behind the wheel: lazy, distracted, slow to react fleshbags that we are.
I'm not sure that a truly unavoidable accident would occur even once a year in a fictional world in which all drivers were perfect and had millisecond reaction speeds.
What is obvious however, is that these situations are so rare as to be irrelevant. In practice accidents are avoidable by the driver of the vehicle - or at least avoidable to such an extent that it's not worth considering the other cases.
Also: although I personally don't object to some rational victim blaming I think it's a little distasteful that we're already speculating about how this must be the victims's fault, when there's simply not enough evidence to make that kind of determination yet. Let's not forget that part of the privilege of being allowed to participate in traffic implies a responsibility not to kill people even when they behave unexpectedly.
For some statistical perspective: if human drivers had as many fatal accidents per mile as uber has, then the average male driver would kill 1 person in his lifetime (men drive more). Clearly: that's absurd; people may cause too many accidents, but not nearly that many - and that's being rather charitable to uber's self-driving vehicles, since they have back-up drivers and thus don't count complicated traffic situations, and safety drivers, and thus may well have caused more accidents by themselves. So going purely by the unusual-ness of such an accident with so few miles, I'd say the initial assumption must be that this is likely a bug in uber's car, even if I'm sure there were contributing factors.
Edit: I guess it's not surprising wikipedia has stats on the influence of alcohol on fatalities, but it tops out at 4 times the legal limit - at which point human drivers are still safer than this (sample size of one...) uber record so far. :-/
Of course not. I'm not saying they're common, merely that they exist. I do not agree with the idea that accidents would go away if drivers were just trying harder, though. There are legitimate unavoidable accidents, and there are also limits to practical human driving.
> Also: although I personally don't object to some rational victim blaming I think it's a little distasteful that we're already speculating about how this must be the victims's fault
To be really clear, I am not blaming the victim here. I have no idea what happened. I'm actually very inclined to blame Uber, though I recognize that's just my personal bias against them.
Near my house there is an arterial I often cross where a large bush on the corner of the intersection obscures the view to the left from the stop sign roughly 10 feet back. I don't just stop at the sign then YOLO through the intersection, I stop, then creep forward, first looking for pedestrians or bicyclist that might step out from the bushes, then at the edge of bushes I stop again and look again both left and right for cars and bikes or pedestrians in the far lane before proceeding across. I often have to stop another time between the sign and the bushes for the pedestrian or bicyclist that just emerged, if I was focusing on getting across the arterial and beating cross traffic I would have killed every one of them.
The idea that cars should drive past tall vehicles at 5 mph is slightly ridiculous. I have never ridden in a car with someone who constantly adjusted their driving speed based on the cars parked on the road. Choosing a reasonable speed? Of course. Extra care at obvious obstructions? Absolutely. Slowing traffic to 5mph because a van happens to be parked? No.
Until there's a detailed report it's really hard to say if it was preventable or not - but I think regardless the optics are bad and this is going to chill a lot of people's feelings on self driving whether or not that is an emotion backed by data.
This holds iff the control input you apply is only braking. Changing the steering angle is generally far more effective for the "pedestrian darts out from hidden place onto road" situation. It's far better to sharply swerve away to ensure that there's no way the pedestrian can get into your path before your car arrive there than it is to stand on the brakes and hope for the best.
Indeed, the faster you're moving, the more you should consider swerving away over braking -- take advantage of that lethal speed to clear the pedestrian's potential paths before he can get into yours.
Yes, this intentionally violates the letter of the traffic laws (and might involve colliding into a a parked or moving automobile on the other side of the road) and also involves potentially unusual manoeuvring on a very short deadline; but it's far better to avoid a vehicle-pedestrian collision even if it's at the cost of possibly busting into the opposing lane / driving off the road / hitting a parked car. Decently experienced drivers can do this, I can do this (and have successfully avoided a collision with a pedestrian who ran out between parked cars on a dark and raining night), there's no fundamental reason that computer-controlled cars can't do this.
...so you probably won't find any support for it in this crowd.
> there's no fundamental reason that computer-controlled cars can't do this.
Sure, there's no fundamental reason you can't swerve but you need to identify the thing in the road you're trying not to hit before you can decide whether to swerve or brake. Self driving cars can't yet reliably ID things well enough to take the only evasive action they know (stopping).
I agree with the thrust of your comment though, and think that a change in the law may be required to allign incentives for machines as well as people to drive humanely.
With self-driving cars this really seems to call for mandatory investigations by a third-party with access to the raw telemetry data. There’s just too much incentive for a company to say they weren’t at fault otherwise.
I mentioned in another comment that something I use to try to improve my own driving is watching videos from /r/roadcam on reddit, and trying to guess where the unexpected vehicle or pedestrian is going to come from.
Here's an example of a pedestrian suddenly appearing from between stopped cars (and coming from a traffic lane, not from a sidewalk), and a human driver spotting it and safely stopping:
https://www.youtube.com/watch?v=wYvKPMaz9rI
Why can't a self-driving car do this?
"the whole point of self-driving cars is that they can react to unexpected circumstances much more quickly than a human"
This statement isn't really true. Do you think Uber is investing in this because it makes their passengers safer? No, they are pretty much immune to any car crashes with a human driver, the risk and insurance hit is assumed by the driver. They are doing this to save money. They don't have to pay a driver for each trip. The safety aspect is just a perk, but Uber will be pushing this full force as long as the self-driving cars are good enough.
I'm more concerned that Uber will "lose" the video evidence showing what kind of situation it was, and we'll never be able to know if a human would have had ample time to react.
There's enough discrepancy here that I'm not sure what happened until we see the onboard data (unlikely to be released to the public) or a statement from a trusted public official like the Tempe police chief.
I find it crazy that so many people think it is. First off, by definition like 49% of the drivers are technically better than the "average driver".
Second, just like with "human-like translation" from machines, errors made by machines tend to be very different than errors made by humans. Perhaps self-driving cars can never cause a "drunken accident", but they could cause accidents that would almost never happen with most drivers.
Third, and perhaps the most important to hear by fans of this "better than average driver" idea, is that self-driving cars are not going to take off if they "only" kill almost as many people as humans do. If you want self-driving cars to be successful then you should be demanding from the carmakers to make these systems flawless.
Average means something else, technically.
And I strongly doubt "average [X]er" would be interpreted as a median by most people. (Except when median and mean overlap)
The quote is "by definition like 49% of the drivers are technically better than the "average driver"." It's obvious to anyone who's not pedantic that they're referring to the median.
As for an example of a mode average: "The average participant's favorite polygon was a triangle."
That's not the original use. That's a response to the (unattributed) idea that self-driving cars would need to be "better than the average driver". In the original context of people giving thresholds for self-driving cars, I disagree that median is what is meant.
> As for an example of a mode average: "The average participant's favorite polygon was a triangle."
But that mode might be 5 out of 40, and I'd just call that a lie. In a lot of distributions the mode just gives you the biggest or smallest number. And it's affected a lot by bucket size; if I measure "how long did you sleep last night" with a resolution of seconds, the mode is probably going to be 0, which is hilariously misleading. Mode, much like a broken clock, might pick a suitable number sometimes, but all the factors you need to check to see if it's suitable basically render the idea of 'mode' redundant. Just use those factors to pick your number. Mean and median are pretty reliable in being useful. Mode isn't.
Of course, to impute a categorical variable in a data set with empty values, for example.
No, that’s not the definition of mean, that’s (something like) the definition of median.
In a skewed distribution (which driving skill may very well be), the mean can be very far from the middle. If a relatively small number of people are extraordinarily bad at driving, most people are above average. If a relatively small number of people are extraordinarily good at driving, most people are below average.
Why this nitpick matters: if the leading point of your thesis depends on is quantifying how many people are above average, you should really know if the average is at the 10th, 50th or 90th percentiles (or somewhere else entirely).
I agree that the _goal_ should be 10x or even 100x better than the best human drivers, but better than average would result in net good and it's hard for me to see an argument in which that's not true because it's less net suffering/death.
>If you want self-driving cars to be successful then you should be demanding from the carmakers to make these systems flawless.
We're still talking about tons of metal moving at very high speeds around other vehicles. That's just not possible. Should my self driving car handle a catastrophic tire failure at 90mph better than me? Absolutely. Is that still a situation that's likely to result in a crash regardless of the best inputs on a compromised system? Yes. There will _always_ be situations in which fatalities can occur with cars or any other kind of vehicle no matter how well they're engineered.
Having seat belts is obviously statistically safer, but when they were first introduced, many people loudly protested that they'd rather be thrown clear in an accident.
It gets worse when a person is sitting in what appears to be the driver's seat. If the car is stopped and that person is looking toward another passenger or down at a phone, nobody will expect the vehicle to begin moving. Making eye contact with the person in that seat is meaningless, but people will infer meaning.
https://www.youtube.com/watch?v=nKPbl3tRf_U
Even on crosswalks. If I just strolled out on a crosswalk without looking, and waiting for drivers who were paying no attention, I’d be long dead.
http://www.nytimes.com/1998/04/05/automobiles/behind-the-whe...
Who exactly do you think is granting Google, Uber, etc. approval for trials like this on public roads? It's going to be some bureocrat with zero ability to gauge what sort of safety standard these companies' projects have reached.
There are no standards here... what were you expecting would happen?
Uber cutting corners and playing fast and loose with legislation? Unheard of!
Here's hoping they get hit with a massive, massive wrongful death lawsuit.
They're on public roads because they've decided that deploying their tech is more important than safety.
I don't see any other explanation. We know that it's pretty much impossible to prove the safety of autonomous vehicles by driving them, so Uber (and almost everyone else) have decided that, well, they don't care. They'll deploy them anyway.
How do we know that? The report by RAND corporation:
https://www.rand.org/pubs/research_reports/RR1478.html
[1] "even with these methods, it may not be possible to establish the safety of autonomous vehicles prior to making them available for public use"
More to the point, the report notes that new methods to determine the safety of self-driving cars are required.
Which the industry is not exactly falling head over heels trying to develop.
Realistically no one will trust “new methods” and establishing their relevance is really difficult. I would imagine that most of these companies are running lots of simulations, because why wouldn’t you? But how many people will see that and trust it more than data gathered on the road?
No, the whole point of self-driving vehicles is that firms operating them can pay the cheaper capital and maintenance costs of the self-driving system rather than the labor cost of a driver.
[1] https://www.theguardian.com/technology/2016/jul/01/tesla-dri...
This case is in fact the first autonomous test vehicle caused fatality.
There were at least several Tesla autopilot related fatalities and injuries, but I would seriously not put those into self driving bag.
"Tesla driver killed while using autopilot"
"The Tesla driver killed in the first known fatal crash involving a self-driving car"
"It's been nearly a year and a half since Joshua Brown became the first person to die in a car driving itself."
"Tesla bears some blame for self-driving crash death, feds say"
This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html.
Seems like an attempt to shift the blame off the autopilot system and onto the driver.
At least, there needs to be agreed consistency before we can claim anything statistical.
> A Tesla spokesperson said the car in the Chinese wreck was badly damaged and unable to transmit data. “We therefore have no way of knowing whether or not Autopilot was engaged at the time of the crash,” the spokesperson said in a statement.
"[Uber] said it had suspended testing of its self-driving cars in Tempe, Pittsburgh, San Francisco and Toronto"[1]
1: https://www.nytimes.com/2018/03/19/technology/uber-driverles...
I was resumed shortly after.
27 Mar 2017 (1 year ago)
https://spectrum.ieee.org/cars-that-think/transportation/sel...
From studying many bicycle-related police reports, I was dismayed at how often I read the phrase "a pedestrian riding a bicycle."
Sad event, but it's likely a human driver would have killed her as well. She was on the road at night without lights, jaywalking without attention to the on-coming traffic.
One could argue the car was driving faster than it's headlight distance safely allowed. If so, humans are going to be quite frustrated with how slow autonomous cars drive at night.
Personally, as a biker, I get slightly annoyed by pedestrians who seem to think I have little control or awareness. I ride on the road where practical, and, if on sidewalk, I go much more slowly and sometimes just get off and walk if the area calls for it. Generally, bikers are much more aware of their surroundings than drivers (both out of self-preservation and an unobstructed, elevated view).
Of course, the best solution is fully separated paths for walkers, bikers, and cars. And, in this instance, it looks like there is ample room for building that, with no street parking to complicate things (and the sidewalk also looks wide and sparsely used).
If they kill fewer people, they can be run by a joint venture of Satan and the Mafia for all I care.
For example, what if instead of 100 people per day dying from human-driven car accidents(where 95 of them are car drivers/passengers, and the other 5 are pedestrians/bicyclists), self-driving cars only kill 30 people per day, but 28 out of 30 are pedestrians/bicyclists?
I can see banning autonomous vehicles in city centers.
I had been assuming that self driving cars would be a 'reset' for our decision to value cars more than the countless lives they end. Self driving cars could take the irrationality and emotion out of driving. It would be America's "stop the murder of children" moment (The Netherlands rallying cry to reform auto traffic in 1970's).
But now I see that it can, and probably will go the other way. These giant companies could push to further restrict the rights of cyclists/pedestrians around roadways, vilify the 'stupid' people who made a reckless decision that resulted in an accident, and push our country even further into car culture.
We ban accounts that do this. Please read https://news.ycombinator.com/newsguidelines.html and follow the rules when posting if you want to keep commenting here.
https://www.google.com/maps/@33.4369934,-111.9429875,3a,75y,...
I suppose you could have a wall that separates the bike lane from the road except for a short area where cars can cross over, so that at least there's only a small space where the bikes & cars can interact, but that introduces a bunch of new problems (cars that can't see bikes behind the wall when they go to cross over, cars that run into the wall, probably even more swerving to get into the lane because your choices are do it RIGHT NOW or you're stuck/hit a wall).
There are ways to design an intersection so that those rules are followed naturally, and there are ways to miss it. C.f. right turn lanes.
The problem is that the lane markings encourage vehicles to make turns from the non-rightmost or non-leftmost lane. One should not be crossing another lane when making a turn at an intersection. One should merge into the rightmost or left most lane in order to make the corresponding turn.
Or just do what Germany has sensibly done and split sidewalks between pedestrians and cyclists (though, cycling enthusiasts are unlikely to enjoy the experience as much)
However, I do need to mention that when you choose to bike on the sidewalk, you're creating the same type of problem that cars that ignore traffic rules create for bikes. There's a reason why biking on the sidewalk is illegal in most places, and why bike lanes/roads exist.
Agree. I'd do it anyways because sidewalks are 99.999% empty where I live. I could ride for 20 miles and not see a person on the sidewalk. So biking on sidewalks is safe as long as you pay attention to what's ahead. No idea if that's the case in this AZ location.
I might be exaggerating a bit.
I don’t fully think I’m a bike I get right of way. If I see a car reversing ahead I move into the street then prepare to full stop but proceed with caution and watch the cars acceleration to see if they see me. If I get the hint they don’t. I stop. It’s scratch on the bumper for them and life for me I am more than glad to yield.
Which are what? Such a broad statement doesn't seem accurate and the sidewalk shown in the map looks low traffic with excellent sight lines, as long as you're not cycling head down at high speed the danger to pedestrians doesn't seem comparable to danger posed by car driver's ignoring traffic rules.
Riding a bicycle on the sidewalk isn't illegal most places because it's dangerous for the pedestrians, it's illegal because it's dangerous for the bicyclist.
I just find it odd when cyclists out of principal decry riding on the sidewalk when the danger of riding with traffic is well known and America has an abundance of long empty sidewalks in suburbia and around office parks.
Many dedicated cycletracks have similar problems. Here in Austin there are several that I refuse to use because drivers far too frequently turn into my path without looking. I ride in the lane so that I can be seen. It's counterintuitive to most drivers, but visibility is often the deciding factor in bike safety. This would be more obvious to drivers if they spent more time cycling. If a cyclist is stuck using a path with poor visibility, I find paying attention to the cars and using light touches of an air horn when approaching an intersection to help. But I still prefer being visible to that.
More generally, intersections are relatively more dangerous (crashes per person mile or something like that) than straight segments of road for any mode of transportation using the road.
Vehicles, on the other hand, do follow traffic rules with regards to lane usage, signaling, and checking what's behind them. Even if they don't, they're still more predictable compared to pedestrians.
The only bicycle riders willing to get organized and lobby the legislators are the very most serious ones. These are the people who want to race along at 50 MPH.
The resulting laws force pitifully slow riders to be out in the street. Even if you can only manage 5 MPH, you have to be in the street.
Lots of us are too arthritic, too fat, too weak, too inflexible, too limited by lung capacity, too limited by heart capacity, and too limited by cheap steel mountain bikes from Walmart.
So our choices are:
a. break the law
b. annoy drivers and risk death
c. don't ride a bicycle
A better design for that intersection would be to keep the bike lane against the sidewalk and the turn lane inside of that, so that they don't cross, and have a "no turn on red" sign, so that the paths of bicyclists and people turning in cars never cross as long as everyone obeys the stoplight and sign.
I can do it, why can't they?
Sure, drivers should check before crossing a lane, but some percentage don't, so it makes sense to minimize lane crosses.
And even assuming this was simply a case of a bad driver, what policy approach would you suggest to protect the general public against such drivers? Accepting that drivers are fallible and designing our road systems to be robust against that seems a more effective approach than berating those drivers who are particularly unlucky in the consequences of their failures of attentiveness.
Perhaps I should have said "wait for turn indicator" instead of "no turn on red", that's a bit clearer.
(It's hard to see a bike at speed going straight when a car is turning right.)
Here's an image that illustrates the danger: http://www.sfbike.org/wp-content/uploads/2013/02/Right-Turn....
This is addressed by having a "no turn on red" sign as I suggested: you don't have to see the bicyclist at all because the bicyclist shouldn't be in the intersection at all when you're turning with a green turn arrow.
And the majority of deaths in London is due to turning transporters/trucks. They are purposely built such that the operator sits very high above and can't even see a bike or pedestrian unless it's many meters away, only by looking into one of 10 mirrors (but of course they don't). On a work site nobody moves these things unless there is an outside instructor, but on public streets we've decided to just blame whoever died. Or, if that fails, blame the infrastructure, even though we oppose any other kind of infrastructure.
Human drivers are terrible.
There are exceptions, but only if there is signage or road markings. For example, if there are two or more lanes marked for right turns, then obviously you can turn from any such lane.
(A little-known fact about that: most California drivers are aware that you're allowed to turn right on a red light after making a full stop, unless there's a "no right turn on red" sign - but few seem to know that this rule applies to all the lanes marked for right turns, not just the rightmost lane.)
Most bike lane markings change from a solid white line to a dotted line about 200 feet from a corner, to give drivers a hint to merge into the bike lane before turning. But even if the line doesn't turn dotted, drivers are still required to merge into the bike lane unless there is a specific indication otherwise.
However, the majority of drivers seem to be unaware of this rule and turn right from the auto traffic lane, creating the risk of a "right hook" collision, which the law is intended to avoid. I've actually had other drivers honk at me when they were waiting in a line of cars to make an illegal right turn from the auto lane while I made a legal and proper turn from the bike lane.
The San Francisco Bicycle Coalition has an excellent info page about this:
http://www.sfbike.org/news/bike-lanes-and-right-turns/
Laws on this do vary in other states. But at least in California, if you see a driver merging into the bike lane before turning right, it's not because they are a terrible driver, it's because they are following the law and (hopefully) increasing safety for bicyclists.
What's more concerning is that my state (Texas) seems to have a lot of vagueness around the laws.
Well, that's confusing to say the least.
If you still don't see the difference, here's one way you can tell. If you are stopped, waiting at the intersection, if you've merged, then you'll be occupying the bike line while you're waiting.
Yeah, like a line is going to protect me from run amok car drivers who are too buy on their cell phones. Or it's going to stop the bus drivers from using them as pickup/dropoff areas, cutting bicyclists off.
Long story short, I bike on the sidewalks. I can be a responsible biker on sidewalks while heeding to pedestrians. My life is more important than to be biking on a vehicle road.
Please support the cause as a local citizen!
http://amp.abcactionnews.com/2064993596/tampa-will-be-home-t...
http://www.bikewalktampabay.org/featured/new-protected-cycle...
https://en.wikipedia.org/wiki/Shared_lane_marking
Anecdata: The times I have interacted with one of Uber's autonomous vehicles at this intersection, they tend to hit the brakes pretty hard as they are about to enter the turn lane and I am around 10ft behind them. All of my interactions with them, however, have been during the day, and it was unclear if it was the human driver in control.
I assume this case will also be one of the most well-recorded cases of a fatal car accident in history as well, given the amount of sensors and equiment on-board a self-driving car, along with eye witness testimony from the operator on board.
Can't tell if Uber has just been incredibly unlucky as of late or if just enough of their employee-base is incompetent as to prevent them from just having a quiet year with no large failures.
I know the scale is different but since this is the first death I’m curious if the percents fall in line.
[1] is distance the right metric?
About 5400 pedestrians are killed each year in the US. US drivers go 3.1 trillion miles a year. So they kill a pedestrian about every 5.7 billion miles. Last November, Waymo said they had 4 million self driven miles, so well short of statistically expecting to hit a pedestrian. In September of some unspecified year Axios claims Uber had self driven over 1 million miles.
My estimate needs help, the distribution of pedestrian bearing roads and pedestrian free roads likely does not match from my total miles per year number and what carbot testers cover. Also, this may have been a cyclist death, which adds another 800 or so deaths per year.
But, in any event, in rough numbers, Uber appears to have beat their expected time to pedestrian fatality by two or three orders of magnitude.
Which seems unbelievably low. I'm getting these figures from this Wikipedia graph[1]
[0] https://en.wikipedia.org/wiki/Transportation_safety_in_the_U... [1] https://en.wikipedia.org/wiki/File:US_traffic_deaths_per_VMT...
If deaths are broken out by pedestrian vs passenger, I wonder how people will respond if safety skews heavily towards one of those groups with self driving cars.
This may well just be bad luck. But I cannot shake the feeling that if Uber started an ice cream venture, they would store their molasses on a hill in a Boston. The only way to get “humanity” associated with Uber would involve an Uber zeppelin.
Or it’s a conspiracy, because nothing is as threatening to Uber than autonomous cars. This is certain to invite more regulatory scrutiny. Just kidding... I think..
This is how counting works.
As we understand more about the risks associated with autonomous driving, we should expand and enrich this data-set, and to ensure public safety, testing against such a dataset should be part of NHTSA / Euro NCAP testing.
I.e. NHTSA and Euro NCAP should start getting into the business of software testing.
I think the idea is to build a "Traincar of Reasonability" to test future autonomous vehicles with.
You might want to check out her research https://hal.pratt.duke.edu/research
I'm sure Dr. Cummings would be more than happy to talk about issues facing validation and verification in the context of NHTSA/FAA.
It is more an issue of how sophisticated these vehicles should be before they're let loose on public roads. At some stage they have to be allowed onto public roads or they'd literally never make it into production.
If they're not willing to dogfood their own potential murder machines then why should the public trust them on the public roads?
They were unwilling to legally obtain a self-driving license in California because they did not want to report "disengagements" (situations in which a human driver has to intervene).
Uber would just set their self-driving cars free and eat whatever fine/punishment comes with it.
https://www.reuters.com/article/us-enron-skilling/former-enr...
Enron's founder, Kenneth Lay, was also convicted and faced as much as life in prison. However, he died before sentencing: http://www.nytimes.com/2006/07/05/business/05cnd-lay.html
This is a strange question to ask. The regulation is not there to benefit Uber, it is to benefit public good. Very few companies would follow regulation if it was a choice. The setup of such regulation would be for it to be criminal to not comply. And if Uber could not operate in California (or the USA) if they did not comply, it would in their interest to provide the requested information.
Essentially, Uber engages in regulatory arbitrage but taking into account the cost-benefits of breaking the law. I.e. if it breaks the law but is profitable for them, they seem to do it.
My 'money' is on people with money figuring out loopholes, like plausible deniability.
One of my biggest complaints about the self-driving car space is that real lives are at stake; light-touch "voluntary" rules suitable for companies that publish solitaire clones aren't going to cut it here.
[1] http://money.cnn.com/2015/12/10/news/companies/gm-recall-ign...
I can’t say anything about GM‘s rep in the US, but here in Europe they are not doing so well. Chevrolet was killed in 2015, and Vauxhall/Opel are doing only ok-ish. Chevy had SO many recalls in the years before they killed it.
Uber already has a terrible reputation with everyone in the tech industry for the sexism, bullying, law breaking, and IP theft. Do they really want to be the self-driving car company with a reputation for killing people?
It doesn't take a lot for people to think "Maybe I'll take a lyft" or "Maybe I'll ban uber from London because of their safety record"[0].
They aren't going to kill the market for this - the other players not only have big incentives to make sure they look safe, but you've got a really unique problem when your biggest competitor is a company that controls access to news and which adverts your customers will see.
[0] https://www.theguardian.com/politics/2017/oct/01/sadiq-khan-...
Uber doesn't get to pick and choose what regulations they wish to follow.
> Uber would just set their self-driving cars free and eat whatever fine/punishment comes with it.
That sounds quite negligent and a cause for heightened repercussions if anything happens.
The strange attitude you display is the _reason_ there are regulations.
It’s the only real solution to corporations misbehaving.
The only working regulation is one that is a existential threat to the company. Which means- huge financial punnishments.
Still, corporations, while being essentially a different kind of life, are not entirely separate entities - they are composed of people. Fear of jail might just be enough to get some high-level executives to exert proper influence on the direction of the corporation.
As well as some core set of tests that define minimum competence, these tests could include sensor failure, equipment failure (tire blowout, the gas pedal gets stuck, the brakes stop working) and unexpected environmental changes (ice on the road, a swerving bus).
Manufacturers could even let the public develop and run their own test cases.
Top reasons for auto accidents today include inclement weather, reckless driving, speeding, driving under the influence, and distracted driving. In theory, most of those could be solved by autonomous driving. But then the list of reasons for accidents would change to whatever new reasons cause autonomous car accidents, such as damaged sensors, programming errors, equipment failure, road hazards, etc.
Even with autonomous cars, we will still need protected lanes, and we will still never implement them, because we don't really care when people we don't know die.
In order to allow transportation to co-exist with pedestrians without collisions, you need some kind of separation between the two. With subways, the protected lane is literally underground, but it does definitely have a protected lane. If you don't go under ground, you can go above ground, like several subways and metros do around the world.
If you don't do either of these, you have to make concessions on the ground level. My personal preference would be tall fences around the roadway, and pedestrian bridges that go over or under the roadway (but both have problems). Another would be to still have the fences, but automate some sliding barricades that would activate when traffic halted, which is somewhat like how train crossings work. We could also implement hybrid methods, like that at Shibuya crossing, for very congested intersections.
Ultimately they will blame humans, humans are the bugs in their code.
I don't even know what to make of that
What exactly do you mean by "deal with driving"? Drive without a single accident, ever? That's obviously impossible in practice. Drive better than humans, who in 2016 killed 37,461 people in the US alone? I don't see how that would be impossible - human drivers have a limited field of view, slow response times (average time to break is ~2.3 seconds), and are frequently distracted, sleepy, drunk, etc.
This is one of those self correcting problems, and it's going to be fixed way faster than the startups pushing this "smart machines" propaganda are going to like.
I doubt that's true, but even if it were, I believe the rules of the road are "always yield to pedestrians, even if you have the right-of-way".
It is not possible to extract more punishment on an individual than their death, technically, unless you believe in an afterlife.
It is not punitive and meant to 'correct' the builder; it is meant to 'prevent' the builder from cutting corners.
Not true at all. Not that I'd recommend it, but there's torture, and punishing family. They were far less outrageous in Ancient times.
JFYI, it is most probably apocryphal: https://skeptics.stackexchange.com/questions/18558/were-roma...
still it's a nice one, I have heard the same about engineers/architects in ancient Babylon and Egypt.
Hammurabi's code:
> If a builder builds a house for a man and does not make its construction firm, and the house which he has built collapses and causes the death of the owner of the house, that builder shall be put to death.
> If it causes the death of the son of the owner of the house, they shall put to death a son of that builder.
https://www.fs.blog/2017/11/hammurabis-code/
Heard about it on Econtalk's recent episode with Nassim Taleb.
http://journaltimes.com/news/local/confronting-a-legacy-of-a...
"In order to gain the trust of the employees he asked his wife to enter the building while the movement was taking place."
https://99percentinvisible.org/episode/managed-retreat/2/
Self driving tech is up against the marketing arm of the auto-industry, they've got to be better than perfect to avoid public backlash. If they're a bit slower but far safer then I think they'll do well.
o At speed we're pretty ok with cars hitting things people drop on the road, examples of cars hitting wagons and babies are already plentiful
o Visual recognition & rapidly updated multi-mode sensor data, backed by hard-core neural networks and multi-'brained' ML solutions, have every reason to be way better at this job than we are given sufficient time... those models will be working with aggregate crash data of every accident real and simulated experienced by their systems and highly accurate mathematical models custom made to chose the least-poor solution to dilemmas human drivers fail routinely
o AIs have multiple vectors of possible improvement over human drivers in stressed situations. Assuming they will bluntly stop ignores how vital relative motion is to safe traffic, not to mention the car-to-car communication options available given a significant number of automated cars on the road -- arguably human drivers can never be as smooth as a fleet of automated cars, and "slamming the brakes" is how people handle uncertainty already
Presently the tech is immature, but the potential for automated transport systems lies well beyond the realms of human capabilities.
Braking distance (without including any decision time) for a 15mph car is 11 ft, for a 30mph is 45 ft. Self driving cars won't change these limits. (well, they may be a little better than humans at maximizing braking power through threshold braking on all 4 wheels, but it won't be dramatically different)
So even with perfect reaction times, it will still be possible for a self-driving car to hit a human who enters its path unexpectedly.
In fact, self-driving cars may actually improve the situation if cars actually start complying with speed limits en masse.
There's a 30mph city street near me where cars routinely go 45mph -- the fatality rate jumps up to 60% at that speed. So just having cars follow the speed limit would go a long way toward reducing fatalities.
https://www.propublica.org/article/unsafe-at-many-speeds
There is a fundamental inequity between the operator of dangerous equipment comparing hours and years of life, and the pedestrian who suffers the consequence of that comparison. The USA auto industry is built on this inequity, which is why no one ever talks about it.
Lives, dollars, wasted time are all fungible.
Your new, inequity point adds externalities to the discussion. That's fine, but those are also measured in lives, dollars, wasted time.
But I shouldn't pick nits; you were speaking in generalities! So when I said "any transportation goal" and you disagreed, you hadn't actually thought of a particular exception to my universal statement. And you still haven't thought of one. Are your contributions to this discussion offered in good faith?
2. Auto plant needs radiators. Truck delivering them takes longer to get them there. Costs $x more. Car costs more. Car buyer has to work longer to buy car. Car buyer wastes y hours of life.
There are lots of alternatives that don't involve you spending more time driving.
While it's true that goods will cost more to transport if it takes longer, that higher charge is amortized across many products in the truck, so is a very small portion of the finished product.
So if a radiator fits in a box 16x24x6" or 1.3ft^3 and a 40ft truck holds 2400 ft^3 (subtract 20% since it won't be a perfect fix, so 2000 ft^3, so you can fit 1500 of them in the truck.
If a truck+driver costs $100/hour, that means each radiator will cost 13 cents more.
Or, another way at looking at it -- all of the parts that make up a car aren't going to be bigger than a car (sure, some space is lost to packaging, but there's a lot of empty space in a car), and 6 - 10 cars can fit on a car carrier truck, so each car will end up costing around $25 more.
Though since we're talking about urban speed limits, and there aren't many urban car manufacturers, slow urban speed limits won't affect the price of cars.
But what about an even lower speed? If 15 mph is good, then 5 mph is better. And if 5 mph is better, 1 mph is superior once more.
I think we can agree there is a point where slow becomes too slow and the 'sufficient planning' becomes an unreasonable burden. So given we aren't operating off the notion that slower is inherently better, then there is some equation giving us our optimal point. What if that point is 45 mph instead of 15 mph?
In short, how do we argue that 15 mph is better than 45 mph that can't also be applied to speeds lower than 15 mph?
Often speeding just gets you to the next intersection, or red lighter, quicker, where you then have to wait longer for traffic, or a green light.
(edit: just to be clear, I'm not in support of speeding, however I do value good logic.)
No one is saying reduce speeds everywhere. But in an urban context with lots of pedestrians, these speeds matter, and urban traffic is generally so stop-and-go and congested that drivers rarely sustain the top speed, and reducing it doesn't actually affect travel time by all that much.
However comforting that logic may be, it is unusable in the real world. If you value lives infinitely, then you will never ever do anything that risks your or somebody else's life in order to gain on any other need. You routinely engage in things that are not the safest possible option in order to fulfill other needs, ranging from food or water acquisition through mere entertainment. Therefore you place a finite value on your life. Don't feel bad, so does everybody else. It is possible to determine the value placed on life, I believe there are studies that show the value is more stable than you might think, and balance things appropriately.
It may be uncomfortable thinking, but, again, unless you literally never take even the smallest risk in the pursuit of other goals, you are already thinking this way. You just haven't lifted it up to the conscious level yet.
I'd say you're almost right about nobody asking for infinite value, but I'd say it's more like nobody who has pulled this up to the conscious level is asking for infinite value. People who have not examined the belief are quite prone to speaking as if life's value is infinite... but their own actions inevitably belie that claim. Once examined, it becomes rationally obvious that life is not infinitely valuable (including your own), but, well, if humans automatically accepted and believed all rational things they examine without emotional consequence the world would be a very different place.
http://www.nytimes.com/2011/02/17/business/economy/17regulat...
For evaluating safety regulations relative to the cost of inplementation, NHTSA values the risk of loss of human life in relation to the market value of risk reducing products, and the safety they provide.
They extrapolate from the take rate of airbags and their cost and effectivity, to how much value the average American places on their own life. IIRC on the order of $5 million.
FYI, juries do not look kindly on companies that implement this in liability suits. The data is that if a corporation writes down a $-figure for a (statistical) human life, it anchors punitive fines at a higher level.
Universally adhered to lower speed limits in urban environments would be great.
Conduent offers a turnkey solution for this. They provide and manage speed cameras: https://www.conduent.com/solution/transportation-solutions/r...
Most of them are empty but people unfamliar with the place will usually slow down.
The vast majority of people just go however fast they feel comfortable (considering conditions, etc) regardless of the speed limit.
Mixed traffic speeds decrease safety.
Raising speed limits so that you don't have the % of people who comply with the letter of the law traveling slower than the people who go however fast they're comfortable usually improves safety.
Unless your goal is to increase ticket revenue or appease the "think of the children crowd" there's no point to lowering speed limits. It doesn't do much to affect traffic speed. To do that you have to modify the road or do something to change the traffic flow.
Self driving cars will improve safety because they'll result in political pressure to raise speed limits to match reality and they'll make dynamic speed limits more practical.
The trick is to make people feel like driving fast is unsafe, without actually making driving any unsafer.
Sure, but city streets are already mixed traffic. There are pedestrians, bikes, vehicles parking or turning, etc. It's not reasonable to raise the limit to what people want to drive and just ignore all the other users of the street.
Also, the optical narrowing mentioned in a sibling comment is quite effective. They've done that on a few streets near me via things like sidewalk bulb-outs at intersections, and swapping the parking lane & bike lane (so it goes curb-bike-parking-drive, rather than curb-parking-bike-drive). Everyone drives more slowly on those streets now - myself included.
For more, see https://www.vox.com/the-big-idea/2016/11/30/13784520/roads-d...
Exactly. That is why we should ban all cars anywhere there is people, and they should be limited to highways. Because safety is most important, right? Right???
It’s very simple, put speed cameras on every corner, and fine in terms of day wages. E.g., one week of your income as fine for going x% above.
Several countries are doing parts of this already, or moving towards it.
(IMHO) jamitons would dissolve if people would leave a flexible buffer between them and the car in front of them and focusing on minimizing braking, rather than driving up to their bumper, brake, wait for moving, accelerate, brake, repeat. The lag due to reaction time and ac/deceleration exacerbates the "viscosity" of traffic flow. If most drivers focused on "staying fluid" rather than hurry-up-and-wait, traffic ought to improve. Like fluidized beds.
https://math.mit.edu/projects/traffic/
If a car can't observe any potential hazards that might impact it using different threat models it should drive more slowly. Blowing down a narrow street with parked cars on both sides at precisely the speed limit is not a good plan.
And then he started me on watching for suspicious gaps in the parked cards along the side that could indicate a loading bay or a driveway or an alley or a hidden intersection. And so on though multiple categories of collision hazards, and then verbally indicating them to him while I was driving.
And the reason for that exercise was to drive home the point that if there's a vehicle or a person that could get into my lane, it's my job as a defensive driver to be aware of that and be ready to react. Which includes making sure I could stop or avoid in time if I needed to.
I don't know how driving is taught now, but I would hope a self-driving system could at the very least match what my human driving instructor was capable of.
[1] https://www.youtube.com/watch?v=SdQRkmdhwJs
But they should be willing to walk in front of it in an in-spec performance regime. There's some really good Volvo commercials along that line, with engineers standing in front of a truck.
Apologies for going off topic here, but I'm curious about this. I've tested every car I've ever owned and all of the recent cars with all-round disc brakes have outperformed this statistic, but I've never been able to get agreement from other people (unless I demonstrate it to them in person).
I'm talking about optimal conditions here, wet roads would change things obviously but each of these cars was able to stop within it's own car length (around 15 feet) from 30mph, simply by stamping on the brake pedal with maximum force, triggering the ABS until the car stops:
2001 Nissan Primera SE
2003 BMW 325i Touring (E46)
2007 Peugeot 307 1.6 S
2011 Ford S-Max
I can't work out how any modern car, even in the wet, could need 45 feet to stop. In case it's not obvious, this is only considering mechanical stopping distance, human reaction time (or indeed computer reaction time which is the main topic here) would extend this distance, but the usual 45 feet from 30mph statistic doesn't include reaction time either.
I learned about this feature when reading car forums for my car and finding threads from people who were rear-ended when they accidentally triggered this feature by slamming on the brakes when they didn't intend to come to a complete stop.
Which is utterly stupid. Braking is the natural reflex, but not always the right one. I've been in more than one close call (think left turn on incoming traffic) where the correct response was not to floor the brake, but floor the gas.
> people being taught to pump brakes, which increases braking distance in cars with ABS.
Which is a mechanical turk version of what the ABS is doing under the hood.
http://www.government-fleet.com/content/driver-care-know-you...
http://www.brake.org.uk/facts-resources/15-facts/1255-speed
Car and Driver did a test with sports cars and profesional drivers and came up with 142 - 155 ft from 70mph, while my first reference quotes 245 ft (around 40% less, so extrapolating, their stopping distance from 35mph would be around 20 feet).
https://www.caranddriver.com/features/rocket-sleds-the-best-...
The average car on the road is not a sports car with performance tires and is not stopping on a clean, dry track. So I don't think it's a stretch to assume that an average car on average roads with tires optimized for tread life would be 40% worse than a $100K sports car with $400 tires that are optimized for grip rather than lifetime.
The 45 feet from 30mph is a common figure (the UK government's Highway Code uses it as well).
The cars I tested are normal cars, but I concede I had good tyres and I always choose smooth roads to test on. Once you include:
1. Driver ability
2. Vehicle quality
3. Road quality
4. Prevailing conditions
5. 4 passengers plus luggage
I guess you can explain the difference.
How did you measure that? Because plugging these figures into a uniform acceleration calculator, 50km/h to 0 in 4.47m requires a deceleration of 2.2g but "Analysis of emergency braking of a vehicle"[0] experimentally measured very best case deceleration as barely scraping 1g (with ABS at 80km/h, significantly lower at lower speeds or without ABS).
[0] https://www.tandfonline.com/doi/pdf/10.1080/16484142.2007.96...
Cameras are not accurate enough though, their dynamic range being terrible. Wonder how humans would fare if forced to wear goggles that approximated a lidar sensors information.
Theoretically self-drivers will always see everything that is relevant, unlike a human driver. And theoretically a robot-driver will always react more quickly than even a hyper-attentive human driver, who has to move meat in order to apply the brake.
From an ethical standpoint the interesting phase will only start then. It‘s one thing to bring a small fleet of high tech (e.g. having LIDAR) vehicles to the road. It‘s another to bring that technology to a saturated mass market which is primarily cost driven. Yes, I assume self-driving cars will eventually compete with human driven ones.
Will we, as a society, accept some increase in traffic fatalities in return for considerable savings that self-driving cars will bring?
Will you or me as an individual accept a slightly higher risk in exchange for tangible time savings?
I believe the claim is 38 multitudes better than humans, significantly better than marginally.
> accept some increase in traffic fatalities
No. And the question is more about "some" than "some increase"
> accept a slightly higher risk in exchange for tangible time savings?
Texting while driving and even hands free talking were becoming laws in many states before smart phones -- and my experience is that many people readily accept this risk and the legal risk just to communicate faster. The same can be said for the risk of drunk driving -- it's a risk that thousands of Americans take all of the time.
Some airlines do not allow pilots to manually fly above 3k feet, nor allow first mates to land manually[0].
0: https://www.wired.com/story/boeing-autonomous-plane-autopilo...
There is a (prevalent) happy path where autopilot and autoland can get you the whole illusion "but the plane flies itself" - but the whole system is built around humans catching and handling all and any exceptions.
A USAF C-54 made a complete transatlantic flight (takeoff and landing included) on autopilot back in 1947: https://www.flightglobal.com/pdfarchive/view/1947/1947%20-%2...
Modern Cat IIIc (zero-zero) autopilots handle everything except taxiing and takeoff. And I think the takeoff thing is more political than technical.
Sounds silly when compared against old tech.
Accidents happen, best we can do is try to prevent them.
It's impossible to fully remove the issue. Vehicles still need to react to and reduce accidents, but they will absolutely never eliminate them.
Although the actual logistics of your proposal might be challenging (child comments point out that some speeds/distances might be impossible to solve) your instinct is a correct one: the people designing and deploying these solutions need to have skin in the game.
I don't think truly autonomous cars are possible to deploy safely with our current level of technology but if I did ... I would want to see their family and their children driving in, and walking around, these cars before we consider wide adoption.
also that testing does happen in the case of every av program i know of. closed obstacle courses with crap that pops out forcing the cars to react. look up gomentum station.
† i did it for you. http://gomentumstation.net/
IOW, setting this up as some kind of quality standard gives unjustified cover ("Hey, our own CEO risked his life to prove the car was safe!") if AVs fail on the open road, because the requirements of open and closed tests are so different.
https://www.youtube.com/watch?v=_47utWAoupo
Just like they take the repair mechanics on the first test flight after a major repair of a big airplane.
It's unfortunate, but unsurprising, the first pedestrian killed by an experimental self-driving car allowed on public roads was in Arizona.
We need to discuss how the developers self-driving cars will be held accountable for the crimes they commit. There is no reason the person who programs or the person who makes money from a self-driving car should be held less accountable for a crime that if committed directly by a person would almost certainly result in jail time. You can’t replace yourself with a computer program and then choose to take only the benefits and not the responsibilities.
The executives should be held accountable, not the developers.
The person who profits the most should be held the most responsible. But the separation of roles between the executives and the developers is likely to mean that no one gets punished at all.
Do you want software development to turn into healthcare, where every developer needs millions of dollars of malpractice insurance? Because shit like this will turn it into a healthcare like system real quick.
It's up to the product (the collective of individuals that deliver the product) to address and mitigate the risk it creates, that's not solely on the shoulders of individual software contributors.
If A writes a generic computer vision algorithm and open sources it, B integrates that into a "is this a bomb or not" product with a white paper outlining its failure rate in a specific situation, then C sells that product to D who uses it in an entirely different situation and E gets blown up... who gets sued? It definitely should be somebody, there should certainly be a liability and incentive to avoid such a liability but I it probably lies somewhere in C-D space, not A-B space.
How else to interpreted that? When a single bug can cause loss of life, and given that this in a thread about Uber, it’s hard to draw other conclusions. By all means though, offer another perspective on how regulating industries with significant number of lives on the line can’t manage regulation. While you’re doing that, I’d point to the aerospace sector which seems capable of both innovation and regulation.
For example: https://en.wikipedia.org/wiki/Boeing_737_rudder_issues
Although the NTSB investigated the accident, it was unable to conclusively identify the cause of the crash. The rudder PCU from Flight 585 was severely damaged, which prevented operational testing of the PCU.[3]:47 A review of the flight crew's history determined that Flight 585's captain strictly adhered to operating procedures and had a conservative approach to flying.[3]:47 A first officer who had previously flown with Flight 585's captain reported that the captain had indicated to him while landing in turbulent weather that the captain had no problem with declaring a go-around if the landing appeared unsafe.[3]:48 The first officer was considered to be "very competent" by the captain on previous trips they had flown together.[3]:48 The weather data available to the NTSB indicated that Flight 585 might have encountered a horizontal axis wind vortex that could have caused the aircraft to roll over, but this could not be shown conclusively to have happened or to have caused the rollover.[3]:48–49
On December 8, 1992, the NTSB published a report which identified what the NTSB believed at the time to be the two most likely causes of the accident. The first possibility was that the airplane's directional control system had malfunctioned and caused the rudder to move in a manner which caused the accident. The second possibility was a weather disturbance that caused a sudden rudder movement or loss of control. The Board determined that it lacked sufficient evidence to conclude either theory as the probable cause of the accident.[2]:ix[3]:49 This was only the fourth time in the NTSB's history that it had closed an investigation and published a final aircraft accident report where the probable cause was undetermined.[4]
Second:
In 2004, following an independent investigation of the recovered PCU/dual-servo unit, a Los Angeles jury, which was not allowed to hear or consider the NTSB's conclusions about the accident, ruled that the 737's rudder was the cause of the crash, and ordered Parker Hannifin, a rudder component manufacturer, to pay US$44 million to the plaintiff families.[16] Parker Hannifin subsequently appealed the verdict, which resulted in an out-of-court settlement for an undisclosed amount.
I have no problem "piercing the veil" for egregious issues. I'd have no problem holding a developer liable for failing to secure a project but just continuing on rather than quit. But "Let's just hold all the engineers criminally liable all the time!" is a bad idea and it is not already done for a reason.
Software isn't like civil engineering where you can mathematically prove that a design is sound.
All we have in engineering (non-software) is something like safety factors and confidence, and this is done with (usually) rigorous mathematical models as well as loads and loads of testing to fill in the gaps of mathematics (think unknown constant/parameters, assumptions, etc).
None of this is impossible to do for software. There are systems that enable one to do easy/entry level verification (such as something like TLA+), to much more complicated reasoning (something like COQ). This will allow the system designers to gain confidence in if the system will work and gain understanding about under what scenario they will fail. Contrast this with the existing software landscape, which is mostly, at least from my perspective, just let me write some stuff until things do approximately what I want. Even at the top of the ladder, I feel the tests conducted are "adhoc" at best and with none of the rigours that you associate with traditional engineering fields.
I'm going to cauterize the off-topic debate about the US healthcare system by pointing out that OP was talking about the expense to doctors of malpractice insurance, not about costs to the patients or medical outcomes.
Malpratice liability varies widely by country, but it's a non-trivial expense for doctors everywhere, and significantly higher in states with strong tort liability for doctors.
It's hard to imagine a world with criminal liability (or tort liability) for software engineers that doesn't ultimately end up with a system of insurance for engineers, roughly analogous to the medical malpractice insurance system for physicians.
That healthcare in other countries is able to achieve this in spite of the medical malpractice insurance system points to the fact that such a system is not certain have to have the deleterious effects you confidently assume.
Whether it is a burden for engineers is another question. But the article and the discussion aren’t about the inconveniences faced by the engineers who programmed this system.
Which, as someone else noted, exists and is probably a good idea if you're an independent consultant or possibly a professional (i.e. licensed) engineer who signs off on drawings or other documents for clients or regulators.
But there are many industries where civil liabilities are required. In fact, any software independent consultant is civilly liable for their work, but it’s not specific to software.
IEEE has a section in their member toolkit that goes into why professional liability insurance is needed, https://m.ieee.org/membership_services/membership/discounts/...
The costs aren’t that high or at least they weren’t 15 years ago when I purchased it for less than $1k/year for $1M in coverage. Most people need this even if they think they are safe. If you’re the one who wrote the deployment script that erased $1M in data, it won’t be entirely mitigated that the script made it through qa.
Also interesting is that the engineer who wrote the Uber software is currently liable for criminal negligence, like pretty much everyone else. But you would have to prove culpability. I can’t find any examples of software engineers convicted so it’s hard to tell who goes to jail-developer, qa, or executive.
More info on criminal/civil negligence- https://www.theblanchlawfirm.com/?practice-areas=criminal-ne...
Almost all employees have the possibility of criminal negligence based on their work. For programmers, this could mean that if you fuck up the code for a pacemaker and someone dies, you could go to jail. That’s a big risk and I can’t find any programmer who has been found culpable for someone’s death. This is the current law in the US.
If Uber was negligent in its code, then the programmers could go to jail. They have programmers and they work and assume this extremely low risk.
Now maybe you’re arguing that some special law should or should not exist for Uber drivers.
Moreover it is not clear to me that not holding the companies that create industrial robots that kill people criminally responsible is what most people would consider just. Again, I think it’s just that there is a massive difference in the scale of exposure; there were not enough interested people to have a debate.
As a pedestrian you already run a significant risk of being killed by a car. To the extent that we hold autonomous car makers responsible for these deaths (and I'm not saying we shouldn't), we should hold non-autonomous car makers responsible for the deaths their vehicles cause as well.
Slow your roll. Nobody know why this person died yet.
We have held humans responsible because assuming a correctly functioning car they are performing the most complex and risky task, and are most able to cause problems. Likewise self-driving car software performs a complex and risky task in which failure can have serious consequences.
In this case, the operator was an AI that was negligent and it was unlicensed/unregulated. That's a new scenario. In the human case a person might go to jail for negligent vehicular manslaughter. What does 2 years of jail time look like to an AI? What does a suspended license look like to an unlicensed entity?
For scale: the risk of death from a self driving car will probably be less than the current risk of death from normal cars, and will definitely be less than the risks incurred in the 20th century from cars, buildings, etc.
Self-driving cars are definitely a new and large legal development, but there's no reason to think existing legal principles can't handle them.
As the GP states, the entire reason Uber is testing in Arizona is because their state government completely got rid of reporting regulations which were present in CA; the status quo is decidedly not the same as it is for established technologies.
As for scale, look at the other comments where people analyze the risk posed by self driving cars. Your assumption that the risk of death from self-driving cars is less is not backed up by the evidence.
It’s fine to say that self-driving cars might eventually be better drivers than humans, just like robots might eventually be better at conversing than humans.
There is no reason self-driving cars can’t be be tested in private. Uber can hire pedestrians to interact with them—I don’t volunteer to be their test subject by deciding to take a walk.
Self-driving cars are a new and important industrial development that will require adjustments to policy. They don't require revolutionary new legal principles.
What should be passed (but I can't see how) is a percentage of allowed deaths, at least in the early years, and set it to something like 5-10% of the current rate, reducing downwards to 1% after 20 years.
People will die from self driving cars, and undoubtably their will eventually be a case that is 100% the self driving car's fault. The benefit of self driving cars comes from the mistake being permanently fixed, while with human drivers it can be committed over and over again.
There needs to be some kind of protection on the companies (and obviously the developers, I've never heard someone try to say they should be held responsible before) from lawsuits. Otherwise all it'll take a is a small handful beefore companies will just let it die.
If we are going to be arguing from a utilitarian standpoint, suppose we hold the executives of self-driving car companies as responsible as if they were themselves drivers. Then if self-driving cars truly are safer as you optimistically claim, both fewer people will die from accidents involving them and fewer people will go to jail for those same accidents. Seems like a win to me.
Someone in this discussion has an insane amount of blind faith in technology which here literally killed a pedestrian, and it’s not the people who are arguing for just consequences.
Aren't you the same person who called for holding the developer liable for writing software with a bug? Are you accusing the developer of promising something that is impossible (not hitting a pedestrian in a crosswalk?) or simply implementing it wrong?
It's worth pointing out that we have no idea yet who is at fault in this accident. It could easily be someone who simply walked out in front of traffic when they weren't paying attention.
The above poster seems pretty clear that it is NOT obvious that cars will necessarily drive safer than humans on average, in the same way it is NOT obvious that we will ever have General Artificial Intelligence.
These are very complicated problems, and the machines are currently (significantly) worse than human drivers, so I think it's fair to question the argument that "everything will work out eventually"
I think that is why the very assertive "Are you saying..." is appropriate.
But humans always overestimate the rate of progress, and think we will be living in some amazing futurescape in the next 10 years.
Theoretically it could be otherwise, perhaps, though the human brain has an extremely parallel pattern-matching engine honed by about half a billion years of evolution.
Realistically, the self-driving system will be made of layered distinct components that all add latency. This is how we build both hardware and software. An image is sensed, it gets compressed, it gets passed along the CAN bus, it gets queued up, it gets decompressed, it gets queued up again, object detection runs, the result of that gets queued up for the next stage... and before long you're lucky if you haven't burned a whole second of time.
Machines can drive aggressively.
There was a university that had self-driving cars do parallel parking... by drifting. Driving along, the car would find a parking spot on the other side of the road. It would steer hard to that side, break traction, swing the rear of the vehicle around sideways through a 180-degree turn, and finally skid sideways into the spot. The car did this perfectly.
That kind of ability is something that I personally don't have. I would consider a self-driving car that could do this. If I'm paying, and that kind of driving is my preference, I expect to get it.
That sort of drifting parallel park might work most or nearly all of the time, but if the road conditions are poor and the car loses handling then it will be a lot more risky.
Civil engineering and medical device manufacturing seems to be doing fine, despite having similar principles of engineers' liability.
The idea that software engineers should be held responsible for something that (as far as we can tell so far) was an accident and not the result of negligence or malice is several orders of magnitude beyond the level of liability that civil engineers and medical device manufacturers have.
Nobody said that. The original comment said developers should be held "accountable for a crime that if committed directly by a person would almost certainly result in jail time" [1].
The standards from medical devices and/or civil engineering, with the associated licensing requirements and verification processes, make sense. Even in the case of a careless mistake or strategic oversight, individuals who could have known but nevertheless signed off should be identified, if not explicitly punished.
[1] https://news.ycombinator.com/user?id=jonathanyc
> Nobody said that.
Well, they quite literally did, because the original comment in this thread was:
> We need to discuss how the developers self-driving cars will be held accountable for the crimes they commit. There is no reason the person who programs or the person who makes money from a self-driving car should be held less accountable for a crime that if committed directly by a person would almost certainly result in jail time. You can’t replace yourself with a computer program and then choose to take only the benefits and not the responsibilities.
I guess you can quibble about the difference between "accountable" and "liable", but that's not a discussion that's particularly interesting to have here, especially given OP's other comments in this thread which make it quite clear that this is what they had in mind.
If the developer was negligent or malicious in their duties, why not prosecute them?
Whith so few self driving cars that number sold be zero. If you can't assure safety with a few cars whith a human as backup, you should not be in the streets. And it's not the first dangerous accident of an Uber self driving car where Uber was at fault.
Something does feel wrong about punishing them when the decision to put the car on the road in the first place was almost certainly not their own.
Though I agree Uber should be held accountable for it and it shouldn't be a token fine since the whole point of punishing an accident like this is to discourage them from occurring in the first place.
This sort of accident orchestrated by a group of people probably won't be gracefully handled by our legal system.
Look, I'm all for developers and (software companies in general) to be considering the ethical implications of the work they do, and the moral obligations that they take on as a result of it. However:
> We need to discuss how the developers self-driving cars will be held accountable for the crimes they commit. There is no reason the person who programs or the person who makes money from a self-driving car should be held less accountable for a crime that if committed directly by a person would almost certainly result in jail time. You can’t replace yourself with a computer program and then choose to take only the benefits and not the responsibilities.
This is a bad mentality to take with postmortems for software failures in general, at least from the outset. You need to look at the underlying factors that contributed to the issue, not simply looking for a person to assign blame to. It's possible that negligence is the underlying cause, but not necessarily - and even if negligence is a cause, what were the other cultural factors that led to the negligence happening, without being caught somewhere else in the pipeline? It's tempting to look to assign blame, but if you do that, you'll actually miss out on the systemic improvements that would be necessary to prevent similar incidents in the future.
But moreover, this is a bad outlook to take here, because this wouldn't be criminal behavior if committed by a human. From the best we can tell, given the details available so far, it's an accident, and it's very rare for criminal charges to even be considered in accidents like these, unless it's a hit-and-run.
I can assure you that a human who is driving carelessly would be held criminal liable. Why do you assume an accident that was severe enough to have resulted in a persons death—the car didn’t just scrape them because they ran across the street—is not due to a reckless programming?
Do you see evidence that the car was "driving carelessly"? That's an honest question - from the reporting so far, it doesn't seem clear what the underlying cause was.
Secondly, this is demonstrably false: most pedestrian fatalities by vehicles do not result in criminal charges. If you don't believe me, look up the stats. Or talk to the countless bikers' advocacy groups that have been lodging this exact complaint for decades: drivers are not generally held criminally responsible, unless there are mitigating circumstances (the driver is drunk, the accident was a hit-and-run, etc.).
> Why do you assume an accident that was severe enough to have resulted in a persons death—the car didn’t just scrape them because they ran across the street—is not due to a reckless programming?
When a pedestrian dies, just because they died, that doesn't mean the driver is automatically responsible. It could have been the pedestrian's fault, or it could have been the driver's fault. Or it could be both. Or it could even be neither (a true accident, with no assignment of blame).
The same thing holds here. You can't assume that this is the result of "reckless programming", and to be entirely blunt, by jumping to that conclusion on the basis of literally no evidence whatsoever (and misinterpreting existing case law on vehicular accidents in the process), you're actually undermining the success of any future efforts to prevent these sorts of accidents in the future, whether or not it ultimately turns out to be the fault of someone at Uber.
I accept your point that this accident might not have led to criminal charges if a human had been responsible. But I don’t waver on my argument that if a human driver would have been held criminally responsible for this accident, then we should we hold the executives (or in extreme cases programmers) of Uber responsible in exactly the same way, whether that be criminal or not.
Finally, with humans and pedestrian fatalities many cases involve drunk driving or sleepy driving. Self-driving cars can’t get drunk or sleepy; they can just have bad programming or bad hardware, both installed by their manufacturer.
If one of those things caused the accident, the developer isn't to blame. Civil engineering has experience tracing liability from mistakes (and incentivizing prevention).
There's a reason sweat shops and slave labor happen more in some places than in others.
https://news.ycombinator.com/newsguidelines.html
There's a reason why high pollution manufacturing moved to China. Because China was more willing to let their people die in exchange for jobs.