Everyone who saw the video has said it's the fault of camera for seeming so dark (since even the street lights have almost no effect anywhere), and regardless of that AVs work with lidar and such. Where is the telemetry and analysis? If it didn't see the pedestrian, then why?
Exactly, especially considering that this is car by Volvo, manufacturer that quite often demoes it's new features in the are of detecting pedestrians or animals on the the road.
And it begs the question: did Uber alter the video?
Will it alter the telemetry, too?
If the Tempe police doesn't get forensics experts on this, they're doing their job wrong. Uber is already known as a company that tends to hide and destroy this sort of evidence.
It would be really hard to believe that Uber would do this. There is only downside, the probability of being caught is high (given they have to hand over a ton of data streams to the NTSB).
Not true. There is a lot of upside if Uber is declared blameless and innocent in this accident. It could be the difference between Uber deploying self-driving taxis within a year or four. Uber hasn't shied away from this sort of "risky gambles" before. I mean, their entire business model was based on avoiding or breaking laws in multiple countries.
So don't tell me "it's very unlikely Uber did this". With Uber's track record, I'd rather believe the opposite until proven otherwise. You know the saying: "Fool me once, shame on you, fool me 7 times, and I'm the idiot". I don't like being an idiot.
Uber, like many companies, is filled with people who may put their own interests or even the very-near-term corporate interests over the long term interests of the company.
Given other missteps by Uber's management (which led to a business impact), I think we should not count on them being purely rational actors.
Uber's history defends the position that they're willing to do things regardless of the chance of getting caught and being willing to deal with the fallout later. That's kind of their whole thing - drive in a city no matter the regulations of taxis and the like, and deal with the bureaucracy after. Why wouldn't they alter things if given the chance?
Why is this being down-voted? This was the first thought I had. Their history as a company does suggest that they would do exactly this and try to get away with it.
The lidar should have enough time to detect the pedestrian (who is crossing the road at a leisurely pace) and reduce speed, yet the car doesn't seem to slow down at all. The whole "dark road" narrative doesn't hold a candle, something is seriously wrong with the sensors and/or algorithm.
I think the chances of that data being released without a court order is fairly low. Ideally both the raw data, plus the product (actions based on the raw input, not the model/program) should be released, so that it can be included in everyone's dataset.
However, being that the sensors used are supposedly secret and innovative, along with the programs/models, I don't see the motivation for any AV company for releasing the data
Isn't NTSB usually releasing this kind of material after investigation? I do think it is reasonable that they do not release it before. The idea of NTSB looking at things is to analyse and find what to improve, not really to assign guilt (which appears to be the primary motivation just now).
I'm not in the US either, and my knowledge of the matter is largely from Mayday or Air Crash Investigations which runs on the National Geographic terrestrial TV channel here. But I would expect that this Uber case will get a somewhat similar treatment as airline crash investigations.
(Assuming it's the same as aviation, with which I'm very familiar), NTSB factual reports are the only reports admissible into court. NTSB preliminary and final reports, which are the ones which contain findings or probable or contributing causes are statutorily barred from being admitted into evidence of other trials. (This is to encourage open participation from potential parties to future lawsuits.)
Hugely sensible policy, I think, that contributes to the amazing safety record of aviation. I wonder whether it's the same for other modes of transportation.
The first two responses (sorted by "best") respectively blame the safety driver and provide their own anecdote of hitting something at night, you need response #3 (500 comments below if you don't fold subthreads) to see LIDAR mentioned and #4 and #5 to note that the dashcam footage is not representative of real visibility.
"When the pedestrians shoes first become visible in the video there is approximately 59' between the car and the pedestrian, in 1 second the car will have already covered 52.5' of that gap leaving 6.5' left to stop the car.
"In order for a human driver, or the driver in this car to have avoided this collision by merely hitting the brakes and traveling in a straight line, "as is the reaction when startled by something on the road" there would have needed to be at least another 127.5' of distance between the car and the pedestrian."
Wow, that comment is...bad.
(As an aside, the stopping distance of a car, including reaction time, traveling at 35MPH is about 100 feet.)
I'm not trying to be inflammatory, but should we expect the Tempe AZ City administration to be a neutral source of information in this case? Might they be biased by their decision to host this experiment in their jurisdiction (in the sense of anticipating criticism)?
> “Arizona welcomes Uber self-driving cars with open arms and wide open roads. While California puts the brakes on innovation and change with more bureaucracy and more regulation, Arizona is paving the way for new technology and new businesses. In 2015, I signed an executive order supporting the testing and operation of self-driving cars in Arizona with an emphasis on innovation, economic growth, and most importantly, public safety. This is about economic development, but it’s also about changing the way we live and work. Arizona is proud to be open for business. California may not want you, but we do.”
California's self-driving car rules are essentially what Google asked for. Google/Waymo is fine with that; they have had lots of self driving cars running around Mountain View for years. The main California requirements are that the manufacturer takes total financial responsibility, has to file accident and disconnect reports, and can't use the technology commercially under a test license. Uber violated that last condition in San Francisco, and CA DMV revoked their vehicle licenses.
So Uber moved to Arizona, where they could kill more freely.
I don't know about that, but I imagine the police chief received a lot of advice, whether solicited or not, from Uber's PR, about how to interpret the video.
As an occasional commuter cyclist, I'm well aware of the position cars and drivers hold in our social pecking order. It's a well known "joke" that if you want to get away with murder, you do it with a car.
While we should not absolve Uber, including the driver who was too busy looking at his phone to intervene, I do wonder if a standard driver would have also been deemed not at fault by the police. The answer is likely, "yes".
> I do wonder if a standard driver would have also been deemed not at fault by the police. The answer is likely, "yes".
But it seems, we do not for sure, that the sensors completely failed and on top of that the human driver also failed, so the question is :are this tests safe? next time the failure could happen on a crosswalk , IMO the question of who is to blame is not as important then the question if is safe to do this tests on public roads with such poor hardware and software.
> including the driver who was too busy looking at his phone to intervene
I've yet to get any information on one thing though, are safety drivers operating under the assumption that the car works at SAE2 or at SAE3? Because if it's the latter, the driver has no cause to keep looking at their phone. If it's the former, the car should have a deadman's switch to ensure the driver stay alert.
An other thing that is not clear is whether they were looking at their phone or at instrumentation (e.g. telemetry or the like).
The safety drivers should theoretically be operating under the assumption that the self-driving system can fail randomly at any moment. Of course, humans are not wired as reliable backups to handle random split-second problems when the system, in fact, works correctly most of the time.
> The safety drivers should theoretically be operating under the assumption that the self-driving system can fail randomly at any moment.
Then the car should be equipped with a dead man's switch / vigilance devices to ensure the driver pays attention, trains have been equipped with these equipments for decades.
> Of course, humans are not wired as reliable backups to handle random split-second problems when the system, in fact, works correctly most of the time.
Indeed, but again that is a long-known issue and we've had solutions for a long time.
>While we should not absolve Uber, including the driver who was too busy looking at his phone to intervene, I do wonder if a standard driver would have also been deemed not at fault by the police. The answer is likely, "yes".
Add in the characterization of the victim that has been going on. If the car had hit an ASU student it would have been assumed they were drunk/on the phone. If it had been a Mormon missionary (not uncommon in AZ) there would be a lot more focus on the car/driver. Instead, they hit a homeless/low value person. The discourse reflects that.
Certainly not. As long as the cars still require someone to sit behind the wheel for taking the fall when something bad happens, putting your citizens up for inverse target practice creates a handsome number of VC funded low qualification jobs: all you need is a drivers license and, apparently, something to read against the boredom.
It's pretty hilarious, at least with the ones who aren't blaming the victim, watching autonomous car fans use the "see, it was too dark out!" excuse for this incident. This excuse is either maliciously dishonest, or and indicator that the apologists don't actually understand anything about the technology they're rushing to defend.
The videos that show everything super bright look like they have their high beams on, whereas the Uber video looks like the standard headline mode. Although I can not be completely sure. If that is the case it is apples-to-oranges.
Impact of light sensitivity of camera, and light resolution, is probably more relevant here than even low/high beams. The car must be able to drive with low beams as well, but it's really relevant for human observers, not what the car actually sees.
Which might form an excuse if it was a human driver. However the human wasn't looking at the road. (I can't blame them, its pretty difficult to concentrate when you are not _actually_ driving) So headlights or not, it's irrelevant.
However the article goes on to explain that dip'd beam headlights don't dramatically increase your vision.
However you are correct its apples to oranges. The Dashcam sensor is had not adjusted correctly to the light conditions, or it isn't sensitive enough. However thats not entirely relevant, as the dashcam doesn't (or at least I hope it doesn't) form part of the sensor array.
Sadly Inevitable Meta: Howdy folks, look, I appreciate the upvotes on a comment I made quickly (because I've been following this topic with some interest) but it's not nearly as content rich as some of the other top level comments around here.
That said, I appreciate a site like Arstechnica doing a great job of dissecting just how badly Uber's tech failed here, legal responsibility or not.
I think the best explanation for the very poor quality of the video released by the police is that it's a video of a video, as if they recorded a video of the original video being played back on an LCD display.
To be fair, cell phone cameras as I recall have a tendency to push the ISO up, so it potentially could look different IRL (I'm purposefully being vague).
Ideally /u/ghdana would have posted the camera settings which filmed this, so you could get a better understanding of the situation.
Not that I'm defending UBER on this, as others have said LIDAR and IR in addition to the cameras should have picked the woman up.
>To be fair, cell phone cameras as I recall have a tendency to push the ISO up, so it potentially could look different IRL (I'm purposefully being vague).
That is true, but raising the ISO for proper (well 18% grey anyway.) exposure is something pretty much every retail camera made in the past decade does. I doubt UBER is using some experimental esoteric camera here.
The Uber victim did come from the shadows. Arizona gets dark at night, especially during a new moon. The question isn't why the optical cameras didn't see her, it's why did the LIDAR failed to detect her?
I did, including the comments to the article where they clarify that the follow up videos were taken with high-beams on, i.e., in dissimilar circumstances. And the picture in the comments showing the location of the crash, where there were no streetlights in the immediate vicinity, and a large bush shielding the median, from which the victim is alleged to have come from.
I think there are serious questions about why the Uber car didn't detect the victim, given all the sensors it was supposed to be using, but "came from the shadows" is an accurate description based on the Google Street View and overhead views of the crash site.
It helps to read the entire article before commenting.
The first thing I noticed in watching the video is that the Uber car's lights seem to be ridiculously focused on a very small patch in front. The dark spot the cyclist appears in remains so until little more than a second before she is tragically hit.
The question is indeed why the electronics failed to detect her, but there's something really wrong with car lights like that.
AZ gets dark at night. The workaround is for cars to have headlights.
Not knowing the law as it applies for (semi-)autonomous vehicle testing in Arizona, I am very curious how this one turns out.
Based on my experience driving at night (In Norway, where it also gets pretty dark), even in rural areas with no street lights, you tend to discover pedestrians -even the ones not wearing safety reflectors- significantly more than a second and a half before you cross paths with them.
So - while the vehicle is driving autonomously, there's still a driver (or whatever we should call it, seeing as they do not drive as such) behind the wheel.
If the driver had paid attention to the road, rather than her cell phone, this accident likely wouldn't have happened.
So - who is to blame, legally speaking? The 'driver' for not paying attention? The coder implementing the control algorithms? The HW engineering team deciding which sensors to use? Someone else?
This will be a lot simpler when vehicles are 'properly' autonomous - but right now, it seems AVs simply give the 'driver' all sorts of incentive for not paying attention, while still not absolving them of responsibility. That's the worst of both worlds.
Uber is a corporation, so it as a legal entity is liable, any kind of individual liability is possible to be litigated but very difficult to win. One problem is that corporate charters once granted are never revoked so corporations just keep doing terrible things and diffusing the costs of their actions onto society with minuscule repercussions. You can bet if I was looking down at my phone and not at the road and then hit a woman in the dark (all captured on camera) I would be going to jail... But, Uber has done some really despicable things (stealing the medical records of a rape victim to use for intimidation), and now they've likely killed someone and no one will ever see cell time.
Notwithstanding the fact that a typical visible light imager in a driver assistance system alternates exposure times between frames, especially at night: one over-exposed image to discern dark texture, and one under-exposed image to find the color of point light sources, such as traffic or brake lights.
The released video looks like the latter, though it could also just be standalone dashcam footage. It is also pretty mangled by compression, especially the dark parts are just completely blocky, so it is probably not a good indicator of what the vehicle saw.
OTOH, bikes can be notoriously hard on image recognition, for example it is not unheard of that reflective rims are interpreted as continuation of a curb or lane marking. LIDAR and RADAR should see them, though, and they probably did, so it is indeed unclear where it all went wrong.
I think a big problem with this whole debacle is that we're discussing cameras and lighting, before discussing the actual matter: a person died and someone is responsible.
It doesn't matter if I were to run someone over when it's dark. I would still be accountable.
The discussion about footage removes focus from the actual issue which should be a legal issue, not a technical debugging session.
Edit: Sorry, am not native English speaker. What I meant when I wrote accountable was that I would have to own up to what happened. Not that I would get a sentence if it was an accident.
> It doesn't matter if I were to run someone over when it's dark. I would still be accountable.
As long as you weren't drinking or otherwise found to be obviously negligent this is typically not the case.
The interesting part of this to me is that the police are treating this as if it were a human driver and holding the car to that level of responsibility. A human driver would not be charged with anything for a typical accident like this one (hitting a pedestrian in low light conditions outside of a crosswalk on a high speed road).
The court case will happen since this is so high profile, but if an average Joe driving the exact same route home from work had hit her, it likely would not even go that far.
>A human driver would not be charged with anything for a typical accident like this one (hitting a pedestrian in low light conditions outside of a crosswalk on a high speed road).
It wasn't low light conditions and it isn't a high speed road. It most definitely could be pushed to court in the case of a human driver.
It would go to court if the victim were a young white woman, or an older white woman with money, and the perpetrator were not one of those. A white male victim might have a chance at "justice" if the driver were a minority. A drunk driver would be more likely, but by no means certain, to be charged with anything more than drunk driving. The victim in this case was 49, has been described as homeless (I automatically discount this detail in reporting on pedestrian fatalities), and she had a bicycle. Not the sort of person who inspires prosecutors to inconvenience upstanding taxpayers.
People are also saying it might be OK if a human wouldn’t have been able to do better. Setting aside whether a human could have done better by at least braking, is “as good as the average human” really a good or reasonable standard?
I thought the whole point was that the average human sucked at driving. If a human driver had LIDAR they would not have crashed into this woman; more precisely, a human driver with LIDAR would have had to have been criminally negligent to have crashed into and then killed this woman.
Sure it is, for the moment. We can't require a high bar from the get go. To use an analogy, where would we be if we required the reliability of modern airplane from the advent of the Wright Flyer?
Not really. The equivalent would be criticizing the Wright brothers for (hypothetically) having built and deployed a plane over a city that they knew had a strong risk of crashing and killing someone, when that risk could have been averted by basic engineering available and known to them. We are talking about negligence, not sure why you are trying to change the subject.
No one is saying that self-driving cars need to be perfect. People are asking whether it is responsible to deploy unsafe ones on public roads.
This is just one case, we need to know the relative incidence rate before we can state anything. Even if we could somehow prove that a human driver wouldn't have hit the women that still doesn't mean that self driving cars are not massively better than humans: if self driving cars prevented hundreds of deaths they would still be better even though they miss this one situation.
Of course in the end self driving cars need to fix the bug that caused this to happen.
A computer can have access to sensory information beyond anything that a human can have.
A computer can process that information hundreds or thousands of times faster than a human can.
Therefore the computer should be held to a much higher standard than a human driver in a scenario like this.
> It doesn't matter if I were to run someone over when it's dark. I would still be accountable.
I don't know where you live, but in most legal systems I am aware of there are definitely situations where you wouldn't be responsible. If you were not impaired, driving legally and someone suddenly walks in front of your car, why would you be accountable for that?
Someone didn't "suddenly walk in front of the car". The time they stepped into the road, you were still half a mile away.
Time to learn some new favourite german words. The first would be Sichtfahrgebot; you may only drive as fast as you can see. If your stopping distance exceeds your vision, you are driving too fast for conditions. This is doubly obvious with autonomous vehicles.
The other is Betriebsgefahr. It was your decision to drive a multi-ton vehicle with enough power for the electricity needs of a city block. She was walking, you were driving - you introduced the vast vast majority of the risk and it was your choice that is responsible for the lethal injuries. For this mere fact, you bear a significant percentage of the fault, always.
In a lot of no-fault states you may not be criminally liable for a car accident but in a civil suit you won't get off scot-free. In every case of car vs human that I have come across the driver pays something, usually medical bills. Reason being a driver is required to have insurance while a pedestrian does not. And as far as I know, auto insurance companies are more willing to pay a settlement.
Do remember that on a civil suit culpability is a sliding scale. It's not guilt or not but what percentage of this accident is attributable to you. A driver looking at their phone, with an enhanced sensor safety system (lidar, sonar, nightvision) will get far more blame than a typical driver.
I expect the civil case to be settled quickly and quietly. T
Uber would like to settle that way, but they don't make that decision. Why wouldn't Uber competitors or detractors set up some kind of Gawker situation?
If there had been no interior video, and a regular car, the driver wouldn't be charged with anything. The prosecutor would keep the investigation open for a few months to make the woman's family happy, and then decline to file charges.
With the interior video, and the guy looking at his phone, it gets a little trickier. I think most prosecutors would let the guy twist in the wind for a few more months and then let him plead to a misdemeanor. A lot depends on the victim's family.
The problem is that the case is political and prosecutors tend to behave badly when media and publicity is involved. Hopefully the Tempe prosecutor isn't a camera hound.
This problem is too complex to call for a single 'actual' issue. There's many issues to be solved and talked about. Technical, legal, emotional and psychological. I don't think there's need to call for one to be more 'important' than any other.
One of my very first comments about self-driving cars on HN was about who's going to be liable in case of a deadly accident, and that we need to establish that very early, ideally before self-driving cars are even allowed on the road.
So far, the police seems to think "no one". I have a big problem with that. Someone should be liable for it, otherwise there will be no accountability when building self-driving systems. Even if we say "the insurance company is liable", that would still be some progress, because then we'd at least know the insurance companies would put much pressure on the self-driving car makers to prove that their systems are safe.
But I'd say it's the car maker that needs to be the most culpable party, because at the end of the day, if there are say broken brakes in a new car, which causes deaths, you don't blame the brake pad maker. You blame the car maker. They are the ones who should make sure everything works and is perfectly safe before selling the product to consumers or putting it on the road.
> if there are say broken brakes in a new car, which causes deaths, you don't blame the brake pad maker.
Yes you do. You blame both. Both have an independent duty to not sell defective products. MacPherson v. Buick Motor Co., 217 N.Y. 382, 111 N.E. 1050 (1916).
I don't think this is accurate. I'm not a lawyer or even very knowledgeable about law, but I would be very surprised if the driver is at fault here. This is anecdotal, but I know someone who was in an eerily similar accident (as the driver) and unfortunately killed a person who walked in front of his car. While tragic, he did nothing wrong and there were no legal repercussions.
I live in NYC, where jaywalking is a way of life, and I've adopted two policies over the last few years:
1. Never jaywalk. Just don't do it. Find the appropriate and safe crosswalk, and wait.
2. Never assume it is safe to cross a street, even when you have the right of way. People laugh at me when I look both ways when we have the right of way, but logic is simple: just because I have the right of way does not mean the drivers coming from either side are aware of that or are paying attention.
I've just approached navigating streets as a pedestrian/cyclist as if I were invisible, because I might as well be to any driver that hits me with their vehicle.
Maybe the took the mid-range from a HDR* video and published it? This way they technically did not add contrast, but just conveniently dropped all the info in the shadows to "make it accessible"
* not tone-mapped, I mean what HDR truly stands for
If Uber's self-driving cars are no better than the visual acuity of humans, they shouldn't be on the road. There is no point to this at all if we can't make robots better than humans.
Self-driving cars should be able to use sensors that pick up stuff better than human eyes.
I can't believe that radar and LIDAR were completely unable to see this woman until the visual cameras picked her up. This seems like either a serious flaw with the sensors or with software. The driver, despite looking down at a phone, still seemed to react faster than the computer!
I'm not that comfortable with Uber doing self-driving cars. I don't really consider them a true tech company. They are not built on world-class engineering and design. They are largely a company built by getting around legal regulations and getting rid of staff workers. They've done incredible legal work with regulatory environments.
A company like Google, I think they have the talent and the culture to build something really good here. A company like GM would understand the stakes at hand here and would be cautious. Uber just doesn't seem to have the talent, mission or ethics to be in the self-driving car business. It's no surprise they are the first company to kill a pedestrian.
If self driving cars are equally good at driving as humans then there is no trade off. It's the same level of safety with more convenience. That doesn't mean we shouldn't strive for greater safety, but I think the original poster might have been a bit over zealous with his words.
You original statement is that they are "no better". If the automated solution is just as good as humans with regard to safety, then there is no trade-off being made.
Given how many things we do that make us unsafe drivers (being distracted by music/passengers/anger/other, driving while tired or otherwise inattentive, ...) I don't see it being long before we can claim automated cars are as safe as those driven by humans, especially in some conditions. I can't say we are there yet, but things are getting pretty good.
I don't understand why people call it unsafe because there are more than zero accidents. By the same measure humans should not be allowed to drive either. Even when people have slightly more reasonable expectations they compare the automation to the best drivers, not any average or bad driver.
Perhaps there are conditions where the automated systems are still going to be worse than the average human, perhaps night conditions like this are included in that. The solution there is to make the automation refuse to continue (refuse to start if not yet en-route, ask the user to take over if so, find a safe place to park if the user can't or doesn't take over when requested) in conditions where that might be an issue. This still allows automation in the majority of driving conditions.
I think we should try finding solutions for the bad drivers that text, drive drunk or speed on the roads, is not impossible to fix that and it will not cost lives.
Do we have any proof that with our current hardware and software we can make a self driving car that is better then the average driver (also this score should be adjusted for the location where the car is used, I do not want the claim to be that this car drives better then average drivers from X where X is a location with tons of bad drivers)
I think we are too optimists that we can train this cars on the roads and by the magic of neural networks and computer vision we get better then average driver. The human brain has evolved to detect moving objects very well, add on top of that intuition where you can anticipate some situation and the capability to adapt to new things.
So we would need some numbers to measure this AIs, but I do not see anyone trying to do this measurements, creating tests courses, checking the sensors quality for this cars, checking the reaction times of this cars,
politicians just approved them to test before making any minimal checks.
> I think we should try finding solutions for the bad drivers that text, drive drunk or speed on the roads, is not impossible to fix that and it will not cost lives.
In the time it took you to write that, some drunk person statistically has already killed an innocent person while driving. "It will not cost lives" is incorrect because even the time it takes to discuss this has cost us lives in the drunk driving debate.
It is impossible to fix, in my opinion. People will drive drunk no matter what. They love to do it and there is absolutely nothing that we can do to make them stop, except from eventually eliminating all human drivers licenses.
The punishments/consequences for drunk driving are already very high and they do not stop people from driving drunk and killing someone.
Do you know what "Is impossible means?
It is possible to eliminate most of the drunk drivers, but here are some ideas:
1 put an AI with sensors and cameras pointed at the driver,
do not allow the car to start if driver is drunk, if he starts texting or sleeping turn of the car, you can add a bypass for emergency,
If you put self driving cars on the road you still have many years until everyone is forced to use this cars so you still have drunk drivers.
So are not companies working on a similar idea like I suggested, because there are not money to be made, cars would be expensive, drivers would not like it, but similar with the airbags, helmets if it will be mandatory to have such a system in the car the safety will increase.
Btw there is a very improbable solution but not an impossible one that would eliminate all drunk drivers, don't allow alcohol production, qed is not impossible
This is more an argument for better land-use policy (allowing more density), more public transportation (proven technology) and more remote work. All of these much better solve giving humans more time than self-driving cars.
Self-driving cars don't promise to save time. It's still driving. Their promise -- their only real promise -- is to save lives. And I think saving tens of thousands of American lives every year is a worthwhile goal. Worldwide, more than 1 million people die in car crashes every year. Humans have proven that they can't drive cars well, but we have put up with it because it is convenient.
I know some people will say that they could work on their way to work, but I can't do work in a car. Staring at a laptop in a car gives me motion sickness, along with a lot of other people. And, of course, this would only apply to people who do computer work in the first place.
As someone who has been in serious car crashes and lost friends to them, I for one am all for self-driving cars. It's going to be a revolution, but we shouldn't be testing them on public roads if they are this bad.
> Self-driving cars don't promise to save time. It's still driving.
Not having to drive the car from and to the parking lot saves a lot of time.
Also many people can get work or other things done while being driven to their destination, for example doing phone calls.
>I know some people will say that they could work on their way to work, but I can't do work in a car.
But some can, and they would benefit from a self-driving car that can drive _at least as safe_ as themselves. Granted, it'd be great if they could improve on safety, but I think GP's point is that this is not their only possible use.
But why we should allow such a car that is not safe, it could be 10 times faster but 10 times less safe, for years we are trying to make transport safer, this included making cars more expensive so why should we go back in our investment in safer cars because company X wants to be the first on the market.
I mean, Germany does this on the autobahn. I can't really find any mortality data for the autobahn though. Also, German driving laws are fairly more strict than in the US.
Germany: About 1 fatality per 10,000 cars per year, 4.1 fatalities per billion km driven per year, 38 fatalities per million people per year (2017). Relatively few accidents on the Autobahn, but the proportion of fatal accidents is higher. All in all, the Autobahn is fairly safe, in particular in relation to distance traveled.
USA: About 106 fatalities per million people per year (2013).
I suspect the difference is due to more public transport, less distance driven per person, and possibly more stringent driver education in Germany.
That would increase accident rates. Parent was positing a scenario where self-driving cars don't cause fewer accidents than humans. This could mean a more or less equal amount.
>How about saving human time that would otherwise be spent on driving?
There may be solutions for your wasted time driving that not costs more human lives, like public transport.
An AI is will never have the intuition of a human brain, so you can try compensating with better sensors and faster reaction time, but if you have worse sensors and reaction time then a human then you do not have a self driving car yet, such toy projects were made by students years ago.
Wow, what the hell happened here? Parent suggested that the only benefit of self-driving cars would be if it prevented accidents. This is just wrong, and I don't think you all are helping the discussion by denying this and downvoting me for pointing out that other benefits exist too, even if self-driving cars will be just as unreliable as human drivers.
We can already do that. It's called public transport. The idea of self-driving cars is they can go places public transport doesn't go but the incident happened on a major highway. When I sit on a train I can work without even thinking about where I'm going. It really doesn't look like we're anywhere near that.
I don’t think Google is a good example of a company that would understand the dangers involved, given how flippantly they’ve responded to government action in the past. GM I would trust more solely because at least the engineering of cars goes through strict and government-mandated testing.
The ignition switch scandal is horrible and disgusting, but it helps that Google hasn’t before had to build any products that could have life or death impacts on their users. I suppose the closest you can get is when the revelation came out that Google had been uploading locations of Android phones even when the settings were flipped off, people criticized the danger this posed to domestic-abuse victims.
What I do know is that Uber, a company with virtually no scruples, is very far down the list of company's that I would trust to develop their own self-driving technology. It makes total sense for them to use self-driving cars to replace human drivers, but I'd feel a lot better about this if they leased it from another company.
I agree with much of what you said, but GM understands the stakes? Are you forgetting the GM ignition switch controversy?
I recall one of my favorite scenes from Fight Club wherein Norton talks about the cost of lawsuit*the likelihood of crash vs the cost of a recall.
I'm all about Tech companies getting into self-driving cars, less excited about car companies with shoddy track records on safety and no history of disruption. There's a reason Cruise Automation (the self driving start up that GM bought) insisted on remaining separate from GM and not being absorbed into GM's culture.
>>I can't believe that radar and LIDAR were completely unable to see this woman until the visual cameras picked her up.
The worst part is, that the car didn't react at all - if it started breaking when the cameras saw her, it would still be better than what happened. But there was no reaction. Nothing from LIDAR, nothing from Radar, nothing from visual recognition - which to me, suggests only one thing: she was picked up by all three sensors, and then Uber's algorithm decided she wasn't an actual obstacle and could be safely ignored. Like a leaflet in the wind, or an immobile traffic sign next to the road. That's far worse than a straight up hardware malfunction.
That seems like a correct read of this. They didn't react at all. If the data shows that all three systems picked her up just fine and no reaction was made, that's really damning about the quality of the software that Uber has.
This reminds me about the Tesla incident with the stoped firetruck, the software was programmed to ignore static o objects as an optimization, so we could have the software ignoring the hardware detection, I hope we can find soon what the sensors detected.
From what I remember reading it there are many static objects so they are having a hard time deciding what is safe to ignore,AFAIK Tesla is extra problematic since they want to use only cameras and not other sensors
Not just that, it seems to imply that Uber either aren't maintaining the vehicles correctly, or that their modifications are rendering them unsafe on the road.
The Volvo XC90 is an exceedingly safe car already. With the LED lighting, it shouldn't be possible for the headlamps to illuminate that poorly - it's as if the auto-levelling circuitry thinks the car is riding nose-up.
The XC90 also comes with a factory auto-brake that is supposed to prevent the vehicle from having this type of accident - their stated goal is zero fatalities. Did Uber not buy models with this included, or did they de-activate this feature? You'd think an independent system like that would actually be seen as a positive feature...
Isn't the auto brake only active at lower speeds? (<50 km/h is a figure that rings a bell). The car was driving 61 km/h which is higher than normal "city speeds".
I don't know about that specific model of Volvo, but the emergency brake works up to much higher speeds in other cars/brands - it's just that above 50km/h it's not guaranteed to slow down to zero or prevent impact, just reduce the speed. I suspect that system was disabled as it would interfere with whatever driving tech tech have though.
Or did they put a couple hundred kilos of computing and sensing equipment and batteries into the car and change the weight distribution such that the headlights now point up?
three people in the back seat are in front of the rear wheels/basically over the rear suspension. Its a different moment arm to have it in the trunk or on the roof.
If you taught Dynamics you would know that the compression of the rear springs caused by most of the weight being over the back wheels would make the lights point higher.
Tesla is showing a video, clearly speed up, on their site and it looks like autopilot slowed down as they drove by a pedestrians on the side of the road. Around 1:30 https://www.tesla.com/videos/autopilot-self-driving-hardware... it looks like it is very cautious at stops and when there are things roadside, almost too cautious
I tend to agree with you, I’ll even give their culture a tiny bit of slack if they are simply a ride share/taxi company (very tiny, you shouldn’t abuse or harass people ever) and I think it would make a ton of sense for them to partner with self-driving partners but it seems flawed that they made it into an existential issue that they do their own. It’s ironic, ride share has provided a very reasonable option to driving after a bar visit, with measurable results in some places, safety is something they could sell on.
I think once they get to 'as good as people' we should let them on the road, because they are 'as good as people' but never get tired, angry, and/or drunk :)
I agree, I don't know how we could objectively measure this but I do not see anyone trying to measure this, just selling hopes that soon the AI will be better then humans.
We would need to create some tests and simulations, passing this tests will not imply that the systems are safe but failing them would prevent any broken attempt to be approved for testing.
IF the sensors would be standard maybe we could create fake inputs and test different situation in a simulation, have some basic unit tests, now I am afraid that this companies are just tweaking things, something that today works tomorrow after an update may not work.
Isn't this quite the issue with machine learning algos. There is no 'standard' as the machine is always learning and getting more data in to match against. Testing such a system would mean that you have to 'freeze' the learning portion, something many will not like to do.
I do not know how this systems work, but I assume is not a complex NN but instead is made from layers, with an expert system and the NN would be used only in some sections like identify the objects in an image.
Even if the NN evolve if it recognizes a bike in image X it should still recognize it after it learns more or is updated with new hardware that supports more neurons.
It's fairly easy to measure, fatalities per million km driven. My understanding is that the industry aims to achieve 10x lower fatality rate before releasing self driving cars.
That way of measuring is terrible, any student then can put his own system on streets, kill 100 people, then after 1 million km some agency decide he needs to try again next year.
We need a better measurement that would not involve killing people, at least as a lower bar before accepting this cars for testing.
This is exactly what I was thinking as well, even if it's only as good as a human, humans comes with a whole host of other problems which impair our driving abilities.
A problem I foresee is that even if we could determine them to be as good as a human, they will still be different from a human. People will live that would otherwise have died; people will die that would otherwise have lived. Members of the second set will tend to get far more emotionally involved.
As good as humans implies that over one million people would be killed by robots every year. Some of the families, loved ones and attorneys of these people would be able to make very convincing cases that they would not have died if a human were driving. This is a hard problem to overcome.
Yes we shouldn't accept robots killing people as a fact of life, otherwise we're going to be really unprepared when they decide to do so intentionally...
I was thinking more they intentionally installed cheap dashcams so that they can release poor quality videos that would favor their narrative of events and get ahead of the story/blame.
> If Uber's self-driving cars are no better than the visual acuity of humans, they shouldn't be on the road.
This is part of a test... how can you determine that these cars are better than humans if you can't try them? Put more regulation on the tests, sure, but don't stop them all, otherwise, we wouldn't be able to do any of it. Considering that woman didn't have all her focus on the road, already we see that they didn't put the right person there.
>> I can't believe that radar and LIDAR were completely unable to see this woman until the visual cameras picked her up.
Well, the LIDAR doesn't control the car. It's just a sensor. The car is controlled by an AI system that takes information from the sensors and makes decisions based on that information (and according to its training).
So the Lidar might very well have "seen" the woman, but if the car's AI didn't recognise her as a human in the middle of the road, or didn't know that it had to stop before hitting her, then the car wouldn't stop.
I'm sure the Uber footage is either extremely low dynamic range (and not similar to human vision) or was modified to make it look darker.
However, the driving footage is from after release of the story. The moon on the 21st had illumination of ~17%. The night of the accident was the day after the new moon with an illumination of 1%. Someone needs to film the same section sometime between April 15 - April 17 to get a more accurate estimate of the light on that night. I expect video will still show much lighter conditions than that shown on the suspiciously dark video, but without the same moon lighting conditions we are comparing apples and oranges.
EDIT: masklinn pointed out the accident occured before the stoplight in the much more lighted area, not later area where the news crew was filming and was commented on in the second video. It was near the stoplight with the parking deck in the background rather than only the long spaced overhead lights. Incredibly misleading video from the dashcam. Maybe good to know for defense of the true conditions, but it probably won't make that much difference. look at the slideshow
I'm almost certain that the footage was modified to ride the public opinion of "it's too dark to see her, so it was her fault!" even though the car should be able to operate in complete darkness thanks to LIDAR and radar sensors. Uber never plays by the rules, I wouldn't expect them to suddenly be super honest about this.
I'd be surprised if it was actually modified seems too big of a liability.
If it was, someone should be going to jail as a result, since that's the footage they seemingly handed over to the police as evidence in the investigation.
I'm pretty sure the multiple streetlights present on that section of the street are the main thing illuminating the road; the phase of moon is an insignificant factor.
> However, the driving footage is from after release of the story. The moon on the 21st had illumination of ~17%. The night of the accident was the day after the new moon with an illumination of 1%.
Most of the illumination from the driving footage clearly comes from the street lights, not from the moon. In the Uber dashcam footage, the street lights barely illuminate their own foot, some seem to not even reach the ground. The phase of the moon is essentially a non-factor.
Not to mention LIDAR is not affected by external light source. So the phase of the moon is doubly irrelevant.
Most, but not all of the illumination. Especially between street lights. That first video is from a section of the road where the accident did not happen. You have to go past the stop light where the first video ends to get to the section where the accident happened (second video).
I understand LIDAR should have handled it, and visible spectrum may not be much different with and without moon lighting. But I would like to see a similar conditions comparison to see exactly how bad the video is compared to actual lighting conditions.
One video pointed out a black splotch over the pedestrian. If you pause and the video you can easily see it. A very unusual artifact to say the least.
EDIT: see masklinn reply -- the accident occured before the stoplight in the much more lighted area, not later area where the news crew was filming and was commented on in the second video. It was near the stoplight with the parking deck in the background rather than only the long spaced overhead lights. Incredibly misleading video from the dashcam. Maybe good to know for defense of the true conditions, but it probably won't make that much difference. look at the slideshow
> That first video is from a section of the road where the accident did not happen. You have to go past the stop light where the first video ends to get to the section where the accident happened (second video).
You don't need the videos, the slideshow has a comparison of the exact spot on pictures 1 and 2 (you can see the shoulder bend on the right). Picture 3 (a few frames back) shows the victim was struck pretty much under the streetlight.
> One video pointed out a black splotch over the pedestrian. If you pause and the video you can easily see it. A very unusual artifact to say the least.
Standard artefact from the sensor bottoming out, as soon as you're below threshold there's literally no signal anymore.
Oh damn. Thank you for pointing out the slideshow. I hadn't really noticed it. It was in the well lighted area before the stoplight, not in the darker area afterward where the news crew was filming and comment was made on the second video.
Not only does it have brighter street lights, there are illuminated signs and parking deck lights in the background.
LIDAR is about the self-driving technology. The illumination of the road is about the operator being able to react quickly. Agreed about the impact of the moon relative to streetlights, but OP does have a good point simply in regards to proactively addressing any pushback.
It wasn't pushback. It was a reasonable line of argument where the second video claimed the accident occured (in the much less illuminated area after the stop light).
Wish I had read this before I made my edits to the two original posts.
Just to quantify this side of the argument, street lighting is by design 50-1000x brighter than the moon, depending on the kind of street. For example the intersection of two major roads with pedestrians present is supposed to be lighted to about 30 lux, and a full moon is .1 lux.
Moon or not, the Uber video shows the headlights projecting less than half the distance they're rated at. They're functioning so badly, in fact, that it wouldn't be safe to drive with them, human or bot.
Not to mention, the big selling point with AVs is that they use LIDAR which is superior to human vision. So where's the LIDAR data? Why didn't the car see this person using its non visual sensors.
I think its a given that visual only data is pretty lousy for AV purposes. You have too many issues with it.
While I completely agree with your assessment, I feel compelled to point out that LIDAR is an active sensing method and so ambient illuminance actually makes things harder for the sensors.
I really wanna make excuses for the Uber vehicle here because self driving vehicles will be awesome, but in this particular case it just failed. :(
I'm surprised to hear a top police authority say this. I come from a family of police and it's always said that when telling your side of the story in a traffic incident, saying something along the lines of:
"But Officer, they came out of nowhere!"
never really holds up too well for you. It's the same as saying "I flat out didn't see them, so not my fault." It's quite the opposite in fact... and any cool-headed person will pick that up. I'm sure the investigation is well underway in any event.
The car driver was half the time not looking at the road! Who the hell drives like that.
Jaywalking or not, the victim was clearly a pedestrian (not riding her bike), the insurance of Uber (Uber itself?) should compensate the victim's relatives. If it doesn't I expect a huge backslash.
They should release the LIDAR sensor data, so we can learn what the car "saw". This dashcash is basically irrelevant compared to the sensors on the car.
I don't want to sound like a crazy person here, but could it be that the video released was edited to look super dark? The difference between the Uber video and the amateur cellphone videos is striking!
Could it be that the police//Uber employees//municipality have a political goal to make this look like it was unavoidable? It looks so to me.
Is Uber paying off the police chief to blame the victim?
I'm with the majority here -- the LIDAR should have picked the pedestrian up and decelerated. Uber's self-driving tech is less safe than a human driver, and should be shelved.
Why do we only have the dashcam videos? I want to see the LIDAR data. Also, Uber definitively has ways to render all the sensor data into a video which shows the objects which was detected (or lack of detection).
There needs to be a process in place for autonomous cars accidents. Required telemetry and data provided to independent organization (financed by manufacturers). If it deems the car (hardware or software) was at fault, whole fleet needs to be immediately disabled until the issue is resolved.
Dead pedestrian here and there won't hurt the companies financially. There's not much reason for them to push for perfect products. They can just shoot for 99% and account for the rest in damages. Disabling their service will hurt them.
This needs to be treated like planes, trains, or medical devices are. Not like consumer device.
I wonder why they don't want to show its LIDAR "video", I thought that all autonomous vehicle are using it. I agree that a cause of the accident can be that LIDAR had failed but I wonder why in this kind of technologies they don't have redundancies system. :/
When I learned to drive, I was taught to slow down when you see that you’re coming up on lower visibility areas of the road. This could be a turn in the road, a part of the road that reduces slope or a darker part of the road. I still do practice this safe precaution. And while the technology is in its infancy, drivers in an autonomous car should be just as attentive as a driver in any other car.
I think it was extremely irresponsible of the officers on the scene to imply that there was "nothing that could be done" or a lack of fault before a real investigation had even started!
209 comments
[ 4.8 ms ] story [ 259 ms ] threadWill it alter the telemetry, too?
If the Tempe police doesn't get forensics experts on this, they're doing their job wrong. Uber is already known as a company that tends to hide and destroy this sort of evidence.
Not true. There is a lot of upside if Uber is declared blameless and innocent in this accident. It could be the difference between Uber deploying self-driving taxis within a year or four. Uber hasn't shied away from this sort of "risky gambles" before. I mean, their entire business model was based on avoiding or breaking laws in multiple countries.
So don't tell me "it's very unlikely Uber did this". With Uber's track record, I'd rather believe the opposite until proven otherwise. You know the saying: "Fool me once, shame on you, fool me 7 times, and I'm the idiot". I don't like being an idiot.
Given other missteps by Uber's management (which led to a business impact), I think we should not count on them being purely rational actors.
And had already crossed an entire lane, they were struck on the right-hand lane of a two-lanes street.
However, being that the sensors used are supposedly secret and innovative, along with the programs/models, I don't see the motivation for any AV company for releasing the data
In the UK I think the data is released to the coroner (assuming a fatality) but I'd have to research that to be sure.
Not by a long shot. The /r/video thread has a comment with 20k upvotes purporting to demonstrate that the hit was inevitable "doing some basic math": https://www.reddit.com/r/videos/comments/86756p/police_relea...
The first two responses (sorted by "best") respectively blame the safety driver and provide their own anecdote of hitting something at night, you need response #3 (500 comments below if you don't fold subthreads) to see LIDAR mentioned and #4 and #5 to note that the dashcam footage is not representative of real visibility.
"In order for a human driver, or the driver in this car to have avoided this collision by merely hitting the brakes and traveling in a straight line, "as is the reaction when startled by something on the road" there would have needed to be at least another 127.5' of distance between the car and the pedestrian."
Wow, that comment is...bad.
(As an aside, the stopping distance of a car, including reaction time, traveling at 35MPH is about 100 feet.)
https://azgovernor.gov/governor/news/2016/12/governor-ducey-...
Politics can always be an explanation.
So Uber moved to Arizona, where they could kill more freely.
While we should not absolve Uber, including the driver who was too busy looking at his phone to intervene, I do wonder if a standard driver would have also been deemed not at fault by the police. The answer is likely, "yes".
But it seems, we do not for sure, that the sensors completely failed and on top of that the human driver also failed, so the question is :are this tests safe? next time the failure could happen on a crosswalk , IMO the question of who is to blame is not as important then the question if is safe to do this tests on public roads with such poor hardware and software.
I've yet to get any information on one thing though, are safety drivers operating under the assumption that the car works at SAE2 or at SAE3? Because if it's the latter, the driver has no cause to keep looking at their phone. If it's the former, the car should have a deadman's switch to ensure the driver stay alert.
An other thing that is not clear is whether they were looking at their phone or at instrumentation (e.g. telemetry or the like).
Then the car should be equipped with a dead man's switch / vigilance devices to ensure the driver pays attention, trains have been equipped with these equipments for decades.
> Of course, humans are not wired as reliable backups to handle random split-second problems when the system, in fact, works correctly most of the time.
Indeed, but again that is a long-known issue and we've had solutions for a long time.
Add in the characterization of the victim that has been going on. If the car had hit an ASU student it would have been assumed they were drunk/on the phone. If it had been a Mormon missionary (not uncommon in AZ) there would be a lot more focus on the car/driver. Instead, they hit a homeless/low value person. The discourse reflects that.
However the article goes on to explain that dip'd beam headlights don't dramatically increase your vision.
However you are correct its apples to oranges. The Dashcam sensor is had not adjusted correctly to the light conditions, or it isn't sensitive enough. However thats not entirely relevant, as the dashcam doesn't (or at least I hope it doesn't) form part of the sensor array.
https://www.youtube.com/watch?v=1XOVxSCG8u0
Edit: I guess this video is also referenced in the article, after Brian's.
That said, I appreciate a site like Arstechnica doing a great job of dissecting just how badly Uber's tech failed here, legal responsibility or not.
Ideally /u/ghdana would have posted the camera settings which filmed this, so you could get a better understanding of the situation.
Not that I'm defending UBER on this, as others have said LIDAR and IR in addition to the cameras should have picked the woman up.
That is true, but raising the ISO for proper (well 18% grey anyway.) exposure is something pretty much every retail camera made in the past decade does. I doubt UBER is using some experimental esoteric camera here.
I think there are serious questions about why the Uber car didn't detect the victim, given all the sensors it was supposed to be using, but "came from the shadows" is an accurate description based on the Google Street View and overhead views of the crash site.
It helps to read the entire article before commenting.
The question is indeed why the electronics failed to detect her, but there's something really wrong with car lights like that.
Not knowing the law as it applies for (semi-)autonomous vehicle testing in Arizona, I am very curious how this one turns out.
Based on my experience driving at night (In Norway, where it also gets pretty dark), even in rural areas with no street lights, you tend to discover pedestrians -even the ones not wearing safety reflectors- significantly more than a second and a half before you cross paths with them.
So - while the vehicle is driving autonomously, there's still a driver (or whatever we should call it, seeing as they do not drive as such) behind the wheel.
If the driver had paid attention to the road, rather than her cell phone, this accident likely wouldn't have happened.
So - who is to blame, legally speaking? The 'driver' for not paying attention? The coder implementing the control algorithms? The HW engineering team deciding which sensors to use? Someone else?
This will be a lot simpler when vehicles are 'properly' autonomous - but right now, it seems AVs simply give the 'driver' all sorts of incentive for not paying attention, while still not absolving them of responsibility. That's the worst of both worlds.
I think there is a question of why didn't the optical cameras detect her. It doesn't seem poorly lit.
The released video looks like the latter, though it could also just be standalone dashcam footage. It is also pretty mangled by compression, especially the dark parts are just completely blocky, so it is probably not a good indicator of what the vehicle saw.
OTOH, bikes can be notoriously hard on image recognition, for example it is not unheard of that reflective rims are interpreted as continuation of a curb or lane marking. LIDAR and RADAR should see them, though, and they probably did, so it is indeed unclear where it all went wrong.
Remember Google had to deal with bike corner cases a couple of years ago.
"A Cyclist's Track Stand Befuddled One of Google's Self-Driving Cars"
https://gizmodo.com/a-cyclists-track-stand-totally-befuddled...
It doesn't matter if I were to run someone over when it's dark. I would still be accountable.
The discussion about footage removes focus from the actual issue which should be a legal issue, not a technical debugging session.
Edit: Sorry, am not native English speaker. What I meant when I wrote accountable was that I would have to own up to what happened. Not that I would get a sentence if it was an accident.
As long as you weren't drinking or otherwise found to be obviously negligent this is typically not the case.
The interesting part of this to me is that the police are treating this as if it were a human driver and holding the car to that level of responsibility. A human driver would not be charged with anything for a typical accident like this one (hitting a pedestrian in low light conditions outside of a crosswalk on a high speed road).
The court case will happen since this is so high profile, but if an average Joe driving the exact same route home from work had hit her, it likely would not even go that far.
It wasn't low light conditions and it isn't a high speed road. It most definitely could be pushed to court in the case of a human driver.
I thought the whole point was that the average human sucked at driving. If a human driver had LIDAR they would not have crashed into this woman; more precisely, a human driver with LIDAR would have had to have been criminally negligent to have crashed into and then killed this woman.
No one is saying that self-driving cars need to be perfect. People are asking whether it is responsible to deploy unsafe ones on public roads.
Of course in the end self driving cars need to fix the bug that caused this to happen.
I don't know where you live, but in most legal systems I am aware of there are definitely situations where you wouldn't be responsible. If you were not impaired, driving legally and someone suddenly walks in front of your car, why would you be accountable for that?
Time to learn some new favourite german words. The first would be Sichtfahrgebot; you may only drive as fast as you can see. If your stopping distance exceeds your vision, you are driving too fast for conditions. This is doubly obvious with autonomous vehicles.
The other is Betriebsgefahr. It was your decision to drive a multi-ton vehicle with enough power for the electricity needs of a city block. She was walking, you were driving - you introduced the vast vast majority of the risk and it was your choice that is responsible for the lethal injuries. For this mere fact, you bear a significant percentage of the fault, always.
Do remember that on a civil suit culpability is a sliding scale. It's not guilt or not but what percentage of this accident is attributable to you. A driver looking at their phone, with an enhanced sensor safety system (lidar, sonar, nightvision) will get far more blame than a typical driver.
I expect the civil case to be settled quickly and quietly. T
With the interior video, and the guy looking at his phone, it gets a little trickier. I think most prosecutors would let the guy twist in the wind for a few more months and then let him plead to a misdemeanor. A lot depends on the victim's family.
The problem is that the case is political and prosecutors tend to behave badly when media and publicity is involved. Hopefully the Tempe prosecutor isn't a camera hound.
So far, the police seems to think "no one". I have a big problem with that. Someone should be liable for it, otherwise there will be no accountability when building self-driving systems. Even if we say "the insurance company is liable", that would still be some progress, because then we'd at least know the insurance companies would put much pressure on the self-driving car makers to prove that their systems are safe.
But I'd say it's the car maker that needs to be the most culpable party, because at the end of the day, if there are say broken brakes in a new car, which causes deaths, you don't blame the brake pad maker. You blame the car maker. They are the ones who should make sure everything works and is perfectly safe before selling the product to consumers or putting it on the road.
Yes you do. You blame both. Both have an independent duty to not sell defective products. MacPherson v. Buick Motor Co., 217 N.Y. 382, 111 N.E. 1050 (1916).
I live in NYC, where jaywalking is a way of life, and I've adopted two policies over the last few years:
1. Never jaywalk. Just don't do it. Find the appropriate and safe crosswalk, and wait. 2. Never assume it is safe to cross a street, even when you have the right of way. People laugh at me when I look both ways when we have the right of way, but logic is simple: just because I have the right of way does not mean the drivers coming from either side are aware of that or are paying attention.
* not tone-mapped, I mean what HDR truly stands for
Self-driving cars should be able to use sensors that pick up stuff better than human eyes.
I can't believe that radar and LIDAR were completely unable to see this woman until the visual cameras picked her up. This seems like either a serious flaw with the sensors or with software. The driver, despite looking down at a phone, still seemed to react faster than the computer!
I'm not that comfortable with Uber doing self-driving cars. I don't really consider them a true tech company. They are not built on world-class engineering and design. They are largely a company built by getting around legal regulations and getting rid of staff workers. They've done incredible legal work with regulatory environments.
A company like Google, I think they have the talent and the culture to build something really good here. A company like GM would understand the stakes at hand here and would be cautious. Uber just doesn't seem to have the talent, mission or ethics to be in the self-driving car business. It's no surprise they are the first company to kill a pedestrian.
How about saving human time that would otherwise be spent on driving?
Given how many things we do that make us unsafe drivers (being distracted by music/passengers/anger/other, driving while tired or otherwise inattentive, ...) I don't see it being long before we can claim automated cars are as safe as those driven by humans, especially in some conditions. I can't say we are there yet, but things are getting pretty good.
I don't understand why people call it unsafe because there are more than zero accidents. By the same measure humans should not be allowed to drive either. Even when people have slightly more reasonable expectations they compare the automation to the best drivers, not any average or bad driver.
Perhaps there are conditions where the automated systems are still going to be worse than the average human, perhaps night conditions like this are included in that. The solution there is to make the automation refuse to continue (refuse to start if not yet en-route, ask the user to take over if so, find a safe place to park if the user can't or doesn't take over when requested) in conditions where that might be an issue. This still allows automation in the majority of driving conditions.
Do we have any proof that with our current hardware and software we can make a self driving car that is better then the average driver (also this score should be adjusted for the location where the car is used, I do not want the claim to be that this car drives better then average drivers from X where X is a location with tons of bad drivers)
I think we are too optimists that we can train this cars on the roads and by the magic of neural networks and computer vision we get better then average driver. The human brain has evolved to detect moving objects very well, add on top of that intuition where you can anticipate some situation and the capability to adapt to new things.
So we would need some numbers to measure this AIs, but I do not see anyone trying to do this measurements, creating tests courses, checking the sensors quality for this cars, checking the reaction times of this cars, politicians just approved them to test before making any minimal checks.
In the time it took you to write that, some drunk person statistically has already killed an innocent person while driving. "It will not cost lives" is incorrect because even the time it takes to discuss this has cost us lives in the drunk driving debate.
It is impossible to fix, in my opinion. People will drive drunk no matter what. They love to do it and there is absolutely nothing that we can do to make them stop, except from eventually eliminating all human drivers licenses.
The punishments/consequences for drunk driving are already very high and they do not stop people from driving drunk and killing someone.
If you put self driving cars on the road you still have many years until everyone is forced to use this cars so you still have drunk drivers.
So are not companies working on a similar idea like I suggested, because there are not money to be made, cars would be expensive, drivers would not like it, but similar with the airbags, helmets if it will be mandatory to have such a system in the car the safety will increase.
Btw there is a very improbable solution but not an impossible one that would eliminate all drunk drivers, don't allow alcohol production, qed is not impossible
Self-driving cars don't promise to save time. It's still driving. Their promise -- their only real promise -- is to save lives. And I think saving tens of thousands of American lives every year is a worthwhile goal. Worldwide, more than 1 million people die in car crashes every year. Humans have proven that they can't drive cars well, but we have put up with it because it is convenient.
I know some people will say that they could work on their way to work, but I can't do work in a car. Staring at a laptop in a car gives me motion sickness, along with a lot of other people. And, of course, this would only apply to people who do computer work in the first place.
As someone who has been in serious car crashes and lost friends to them, I for one am all for self-driving cars. It's going to be a revolution, but we shouldn't be testing them on public roads if they are this bad.
Not having to drive the car from and to the parking lot saves a lot of time. Also many people can get work or other things done while being driven to their destination, for example doing phone calls.
But some can, and they would benefit from a self-driving car that can drive _at least as safe_ as themselves. Granted, it'd be great if they could improve on safety, but I think GP's point is that this is not their only possible use.
USA: About 106 fatalities per million people per year (2013).
I suspect the difference is due to more public transport, less distance driven per person, and possibly more stringent driver education in Germany.
http://apps.who.int/gho/data/node.main.A997
https://www.destatis.de/DE/PresseService/Presse/Pressemittei...
There may be solutions for your wasted time driving that not costs more human lives, like public transport. An AI is will never have the intuition of a human brain, so you can try compensating with better sensors and faster reaction time, but if you have worse sensors and reaction time then a human then you do not have a self driving car yet, such toy projects were made by students years ago.
What I do know is that Uber, a company with virtually no scruples, is very far down the list of company's that I would trust to develop their own self-driving technology. It makes total sense for them to use self-driving cars to replace human drivers, but I'd feel a lot better about this if they leased it from another company.
https://www.wsj.com/articles/is-uber-a-friend-or-foe-of-carn...
I recall one of my favorite scenes from Fight Club wherein Norton talks about the cost of lawsuit*the likelihood of crash vs the cost of a recall.
I'm all about Tech companies getting into self-driving cars, less excited about car companies with shoddy track records on safety and no history of disruption. There's a reason Cruise Automation (the self driving start up that GM bought) insisted on remaining separate from GM and not being absorbed into GM's culture.
edit: spelling
To anyone interested in this issue, I recommend the following paper.
https://law.vanderbilt.edu/files/archive/212_Corporate-Risk-...
Also, this story about the engineer who investigated the infamous Pinto 'fire after rear end collision' issue, and the decision not to recall. https://www.newyorker.com/magazine/2015/05/04/the-engineers-...
The worst part is, that the car didn't react at all - if it started breaking when the cameras saw her, it would still be better than what happened. But there was no reaction. Nothing from LIDAR, nothing from Radar, nothing from visual recognition - which to me, suggests only one thing: she was picked up by all three sensors, and then Uber's algorithm decided she wasn't an actual obstacle and could be safely ignored. Like a leaflet in the wind, or an immobile traffic sign next to the road. That's far worse than a straight up hardware malfunction.
From what I remember reading it there are many static objects so they are having a hard time deciding what is safe to ignore,AFAIK Tesla is extra problematic since they want to use only cameras and not other sensors
The Volvo XC90 is an exceedingly safe car already. With the LED lighting, it shouldn't be possible for the headlamps to illuminate that poorly - it's as if the auto-levelling circuitry thinks the car is riding nose-up.
The XC90 also comes with a factory auto-brake that is supposed to prevent the vehicle from having this type of accident - their stated goal is zero fatalities. Did Uber not buy models with this included, or did they de-activate this feature? You'd think an independent system like that would actually be seen as a positive feature...
However there might be other systems which break automatically for cars, in the case of a sudden traffic jam for instance.
Or did they darken the video?
Modern cars, including the XC90 have a tilt sensor to adjust the headlights automatically for this.
Source: I teach statics
Does anyone trust Uber to provide all the evidence they have, and not just selectively release bits that support them?
Does anyone trust Uber not to alter the video, say by increasing the contrast to make darker objects invisible?
Tesla is showing a video, clearly speed up, on their site and it looks like autopilot slowed down as they drove by a pedestrians on the side of the road. Around 1:30 https://www.tesla.com/videos/autopilot-self-driving-hardware... it looks like it is very cautious at stops and when there are things roadside, almost too cautious
I tend to agree with you, I’ll even give their culture a tiny bit of slack if they are simply a ride share/taxi company (very tiny, you shouldn’t abuse or harass people ever) and I think it would make a ton of sense for them to partner with self-driving partners but it seems flawed that they made it into an existential issue that they do their own. It’s ironic, ride share has provided a very reasonable option to driving after a bar visit, with measurable results in some places, safety is something they could sell on.
However, that's horrific and implies that the engineers are using something of an evolutionary algorithm. Obviously, this does not cut the cheese.
IF the sensors would be standard maybe we could create fake inputs and test different situation in a simulation, have some basic unit tests, now I am afraid that this companies are just tweaking things, something that today works tomorrow after an update may not work.
Even if the NN evolve if it recognizes a bike in image X it should still recognize it after it learns more or is updated with new hardware that supports more neurons.
We need a better measurement that would not involve killing people, at least as a lower bar before accepting this cars for testing.
As good as humans implies that over one million people would be killed by robots every year. Some of the families, loved ones and attorneys of these people would be able to make very convincing cases that they would not have died if a human were driving. This is a hard problem to overcome.
This is part of a test... how can you determine that these cars are better than humans if you can't try them? Put more regulation on the tests, sure, but don't stop them all, otherwise, we wouldn't be able to do any of it. Considering that woman didn't have all her focus on the road, already we see that they didn't put the right person there.
Well, the LIDAR doesn't control the car. It's just a sensor. The car is controlled by an AI system that takes information from the sensors and makes decisions based on that information (and according to its training).
So the Lidar might very well have "seen" the woman, but if the car's AI didn't recognise her as a human in the middle of the road, or didn't know that it had to stop before hitting her, then the car wouldn't stop.
However, the driving footage is from after release of the story. The moon on the 21st had illumination of ~17%. The night of the accident was the day after the new moon with an illumination of 1%. Someone needs to film the same section sometime between April 15 - April 17 to get a more accurate estimate of the light on that night. I expect video will still show much lighter conditions than that shown on the suspiciously dark video, but without the same moon lighting conditions we are comparing apples and oranges.
(accident) http://www.moongiant.com/phase/3/18/2018
(filmed) http://www.moongiant.com/phase/3/21/2018
EDIT: masklinn pointed out the accident occured before the stoplight in the much more lighted area, not later area where the news crew was filming and was commented on in the second video. It was near the stoplight with the parking deck in the background rather than only the long spaced overhead lights. Incredibly misleading video from the dashcam. Maybe good to know for defense of the true conditions, but it probably won't make that much difference. look at the slideshow
"So what does the uncompressed footage look like?"
Intern lost it.
If it was, someone should be going to jail as a result, since that's the footage they seemingly handed over to the police as evidence in the investigation.
[1] https://www.youtube.com/watch?v=sKBy4_3azBg
Most of the illumination from the driving footage clearly comes from the street lights, not from the moon. In the Uber dashcam footage, the street lights barely illuminate their own foot, some seem to not even reach the ground. The phase of the moon is essentially a non-factor.
Not to mention LIDAR is not affected by external light source. So the phase of the moon is doubly irrelevant.
I understand LIDAR should have handled it, and visible spectrum may not be much different with and without moon lighting. But I would like to see a similar conditions comparison to see exactly how bad the video is compared to actual lighting conditions.
One video pointed out a black splotch over the pedestrian. If you pause and the video you can easily see it. A very unusual artifact to say the least.
EDIT: see masklinn reply -- the accident occured before the stoplight in the much more lighted area, not later area where the news crew was filming and was commented on in the second video. It was near the stoplight with the parking deck in the background rather than only the long spaced overhead lights. Incredibly misleading video from the dashcam. Maybe good to know for defense of the true conditions, but it probably won't make that much difference. look at the slideshow
You don't need the videos, the slideshow has a comparison of the exact spot on pictures 1 and 2 (you can see the shoulder bend on the right). Picture 3 (a few frames back) shows the victim was struck pretty much under the streetlight.
> One video pointed out a black splotch over the pedestrian. If you pause and the video you can easily see it. A very unusual artifact to say the least.
Standard artefact from the sensor bottoming out, as soon as you're below threshold there's literally no signal anymore.
Not only does it have brighter street lights, there are illuminated signs and parking deck lights in the background.
Wish I had read this before I made my edits to the two original posts.
I think its a given that visual only data is pretty lousy for AV purposes. You have too many issues with it.
I really wanna make excuses for the Uber vehicle here because self driving vehicles will be awesome, but in this particular case it just failed. :(
"But Officer, they came out of nowhere!"
never really holds up too well for you. It's the same as saying "I flat out didn't see them, so not my fault." It's quite the opposite in fact... and any cool-headed person will pick that up. I'm sure the investigation is well underway in any event.
Jaywalking or not, the victim was clearly a pedestrian (not riding her bike), the insurance of Uber (Uber itself?) should compensate the victim's relatives. If it doesn't I expect a huge backslash.
Could it be that the police//Uber employees//municipality have a political goal to make this look like it was unavoidable? It looks so to me.
I'm with the majority here -- the LIDAR should have picked the pedestrian up and decelerated. Uber's self-driving tech is less safe than a human driver, and should be shelved.
Dead pedestrian here and there won't hurt the companies financially. There's not much reason for them to push for perfect products. They can just shoot for 99% and account for the rest in damages. Disabling their service will hurt them.
This needs to be treated like planes, trains, or medical devices are. Not like consumer device.