This is for the auto-brake feature in cars presumably? This is the future. We'll modify our road networks to accommodate vehicles with improved features. One day we'll have radio car-info-beacons along the roadway and wonder why we ever considered only visual road signs.
What a weird thing to consider. The farmers' opinion is possibly the least relevant thing here. One might as well argue against environmental regulation on the grounds that you can't convince Volkswagen. "I'd love to see you try and convince a diesel car company executive that electric is the future". Yeah, I'm not going to do that. He's going to play along because the market and I will make him.
I'm not going to convince farmers of anything. It will simply be untenable for future governments to have public roads that are not usable by cars.
Why must the roads adapt to cars rather than cars adapt to roads? It would be enormously expensive to refit every road to account for the ever-changing stupidities of self-driving cars. It also removes responsibility from manufacturers and puts it on the state.
For anyone watching this thinking it's clever and great- it's incredibly dangerous for the cow. Cattle can easily get their feet stuck between the pipes and getting them out is really ugly. Like... torch-cut pipes burning away flesh ugly. If you see this, shoo the cow back in.
A cow that regularly attempts to cross cattle guards is first to sale or slaughter.
Edit: And now I'm that guy on the internet that I hate, who saw an amusing animal gif and immediately went into buzzkill mode :-/
I've never had an adult stuck, but I have had to pull out a few calves. They're curious and when your skin is made of leather and your weigh more than an adult human, you can get pretty brazen. I've only helped with beef cattle though, I've noticed dairy operations universally don't use cattle guards (for a multitude of reasons)
It was on a quick steep incline, with a real fence (with matching slats spacing) that actually was going across the road right up to the dashed center line. They fixed it by regrading the road like they should have done in the first place, as cattle guards like this must be level, and the local codes required it.
> The irony is that this is actually a good reason to drag the car manufacturers to the table and compel them to fix it from their end.
The solution is having the balls to lay down good policy/regulation. There is no point forcing car companies to use a certain algorithm, for example, but it's not unreasonable to build a standard basket of problematic road signs which the car must pass - e.g. a cattle grid isn't too bad, but if a driverless car doesn't stop for a speed limit outside a school the company should have some liability.
I feel the "driver" of the driverless car should be responsible too. We are nowhere near actual full unsupervised autonomy - people should still be actively looking out for the car doing the wrong thing.
You don't get to abdicate responsibility on the road just because you bought in to some corporations marketing.
Absolutely true, although the corporate liability thing really depends on whether the engineering takes after software or aviation (to pick two widely contrasting examples).
And I notice the "phone with wheels" built by mostly software engineers chooses to use words with well known meanings from the aviation industry without a) any of the rigour behind the regulations and certifications or b) much in the way of acknowledging the differences between the challenges of an "autopilot" safely managing to traverse the airspace 37,000 feet above Hawaii, compared to a rural lane in the UK (or 280 between the valley and the city).
(While, sadly, at the same time it seems from the outside that the ACAS code got written by the same outsourced dev team that work on their website the rest of the time. :sigh: )
non autonomous cars have driver assists now that slam on the brakes when they think you’re about to crash. Statistically a huge improvement to active safety, and then there’s cattle grids.
Imagine you’re driving along in full control on an empty country road. Out of nowhere your car panics, slams in the brakes, deploys seatbelt tensioners, and flashes bright warning lights because clearly you fell asleep or something if you cant see that big wall.
Would you react? Maybe swerve coz you’re startled?
You don't get to abdicate responsibility for your vehicle's "autopilot" just because that was only corporate marketing.
Words mean things. You don't get to say they mean different things when you say them just because it helps you sell more cars. Not in the fine print, and not in a court.
There's no way a "driver" can autocorrect for a vehicle that makes an emergency stop out of the blue -- the vehicle is still stopped, and now poses a safety hazard for the surrounding traffic.
I'd go further than that. If a vehicle's automated systems caused or contributed to an accident or a road rule violation, the car manufacturer should be held responsible.
If your company's automated emergency braking, or lane assist, or adaptive cruise control - causes or contributes to a problem, someone from that company should be in court next to the driver facing judgement for the consequences.
(I'd even go so far as to suggest if the existence of a system leads to a situation where an incident occurred because of the driver's assumption that the automated system would prevent it, then the company should need to justify why the driver should be held responsible instead of the software. If your advertising claims "autopilot!", you're going to be apportioned a significant portion of liability even if your fine print says "Drivers must keep alert and ready to take control at all times" - you should be judged based on a reasonable persons expectation from the language you use to promote your vehicles, not some w4easel words in your T&Cs.)
Automation issues may fall into the category of manufacturer/developer negligence; you seem to be suggesting some sort of burden shifting.
When you say that a manufacturer should be held liable if their system '... contributes to a problem...', it seems like you're suggesting adoption of the (notoriously problematic) 'if not but for' test. This sort of test is used in negligence cases, but can be useless, as almost anything satisfies it (i.e. slow service at a coffee shop may have been a necessary condition for the timing of that exact accident).
Unfortunately, pretty much every automobile safety improvement in the last century (which collectively have saved ~millions of lives) also occasionally kills people. Airbags can propel objects into people, seatbelts can trap someone in a burning car, antilock brakes perform worse on sand, etc.
In practice, putting the responsibilty on auto manufacturers for every specific injury or death that happens due to a safety system, without somehow giving them "positive credit" for lives saved, massively delays or entirely prevents these sort of advancements from being made. We definitely need processes to verify that new features are actually functional, typically through recalls and product liability suits, but simply putting blame on manufacturers for all accidents is a terrible idea.
It weirded me out when I moved here, but in practice, it's enforced like anywhere else: at the beginning and end of school days.
Since a large fraction of kids are driven to school (at least in southern California), you know that this is happening because of the massive traffic jam near the school.
Illinois has these "no turn on red when pedestrians present" signs and I have no clue how thats supposed to help anyone because pedestrians usually get hit because they aren't seen by the car.
In Germany, when a speed limit of 30kmh is introduced because of a school, it is usually on applicable for school days and hours (e.g. Mo-Fri 8-18) only. After school hours or non school days you are allowed to drive at normal speed of 50kmh.
> it's not unreasonable to build a standard basket of problematic road signs which the car must pass
I'm wondering how you would get around metaparameter optimization here, or the fact that it's too easy to optimize to a known test procedure rather than the real world?
I prefer proactive policy. As much as I would have liked to see more people go to jail for the 2009 financial crisis, for example, I’d prefer to see smarter regulation to have prevented it in the first place
Regulations are too complex to get right like that. You need to figure out how to enforce following the spirit of them not the letter or you will forever be seeing more crises that you didn't foresee.
I think that’s the ideal, but tends to break down in practice. The problem with expecting the spirit of the law to be the compass is that it opens up a lot for interpretation. The end result is that it incentivizes gamesmanship where the organization with the best lawyers win
While I agree - I reckon it's far past time that we pierce the outdated "corporate personhood" idea, and let Tesla/Uber/Apple/Waymo et al. know that society has chosen to hold their "corporate personhood" responsible in the same way people are held responsible.
If your corporation (via it's self driving software) kills someone, the corporation will "go to jail", a C level exec does the jailtime and the entire company will be unable to carry on any of it's normal business for the duration of the sentence (while still being held responsible for it's obligations to staff/suppliers/cerditors). If that self driving software is judged to be liable for a traffic infringement, the corporation will pay an amount equal to the percentage of income that a human driver would have paid. If it's an infringement with a ~$400 fine, the corporation should pay ~0.5% of their annual income - the same sort of penalty as "an average salary" person.
A couple of $20mil fines and a 6 month total company shutdown would be very "proactive" in encouraging Tesla to get Elon to shut the fuck up with his big bold claims that cannot be backed up. If Uber had been slapped with the "corporate personhood" equivalent of whatever penalty a human driver who'd killed that person crossing the road with a bicycle they'd be far more circumspect about using minimum wage contractors as "safety drivers" in their prototypes.
At a certain scale, it's inevitable that even a responsible company will screw something up. Shutting the company down when that happens is tantamount to banning companies above a certain size
Taken to the extreme, what you're suggesting amounts to "large corporations should be allowed to kill people occasionally".
I can see a wide spectrum between my (admittedly over punitive towards corporations) ideas and this (somewhat intentional misinterpreted) idea of yours. But I do honestly think Uber and Tesla have evaded morally deserved punishments by blaming customers or contract staff who were pretty much doing what everybody would expect to be able to do using the technology as described.
"it's inevitable that even a responsible _person_ will screw something up" - but that doesn't mean we give them a zero consequence break if they kill or injure someone. (We do, however, distinguish between things like murder, manslaughter, and negligence-causing-death. And penalise differently, based at least partly on the ability of the punishment to discourage other people from doing similar things. Perhaps Uber was only guilty of negligence-causing-death for using minimum wage contractors as safety drivers. Perhaps Tesla was guilty of manslaughter by advertising as "autopilot" a thing that drives full speed into offramp dividers?)
We have already accepted this in other areas where low probability meets large applicability. We know probability is non-zero and given a large enough sample size there is an expected adverse outcome quite a bit higher than zero. E.g., a more trivial example is that the allowable number of insect parts in food is greater than zero because it’s impossible to mass produce food and never have contamination.
Would the corollary be that autonomous cars should have a certification for self-driving, similar to the FAA certification? Possibly an extension of their car safety rating?
Yes, except for the sort of “FAA Certification” that let Boeing sign off on their own work and the FAA just rubber stamp it...
I’d kinda like to see Tesla’s reaction if they were told that any “autopilot” crashes could result in the entire fleet of that model car being taken off the road, with Tesla responsible to all the owners for the disruption while a thorough investigation takes place, possibly a year or more.
Without any real insight, my hunch is the FAA did that because they are probably too strapped/underfunded to do the proper due diligence. It’s tough to know enough about multiple complex airframes enough to ferret our interaction failures so they defer to the teams that supposedly do know enough... which creates an inherent conflict of interest.
It will be interesting to see how regulators treat autonomous vehicles. I have a feeling they will defer to the existing recall mechanism rather than completely grounding a fleet
In the small, it's dramatically more cost effective to fix the road. Ignoring cost to develop and test, just the cost to deploy a fix to all the affected cars would probably be a lot more than £70,000.
In the large, it's always a shame to fix a good test case.
Or simply decertify all of the vehicles that behave incorrectly from driving on public roads.
This is the kind of issue which will persist, vehicles which fail to recognize a real world situation shouldn’t get the benefit of changing the world to fit them.
An inquiry should be done into how these vehicles came to fail to recognize this road feature and the upstream cause (not just the bug by the systematic issues in the organizations which produced vehicles with this bug) should be fixed before they’re rectified to operate on roads.
I like the idea of a warning that particular models of cars are not permitted past this point for safety reasons. That by itself is enough for some frustrated driver to see the sign, take a photo and upload it to social media where those car manufacturers can be suitably laughed at and ridiculed.
The good ones would then sort out why their braking system is faulty. Perhaps it gets confused about other things in other serious ways.
I can't tell from the article whether the automatic braking happens when a sudden incline, decline or both is detected in front of the car. It was one of those three, though, that much is clear.
Do you really regard braking in front of a sudden incline/decline as ridiculous?
It seems like there was a steep hill, but the cattle gate was level, and the sudden, extreme difference in gradient is what caused the sudden braking. Although from the photos, it all looks fairly flat to me.
Yes, that's my impression too, and braking seems sensible.
From uphill: "There's something weird in front of the car, best drive very slowly". From downhill: "WTF I can't see the road any more, best drive very slowly." Both seem quite sensible to me, I don't understand why so many people on the thread seem to disagree.
"Cars have come off the road trying to cross a cattle grid after sensors mistook it for a wall and slammed on the brakes."
These grids usually lie flat on the ground and in some areas are commonplace. Seems to me these sensors are still needing work. Perhaps they see horizontal lines and panic?
Cars running off the road for no good reason is ridiculous. It is indefensible to assert this is OK. Denying it or otherwise covering it up is definiteky ridiculous.
Decertifying a self driving car does nothing considering that today the driver is legally 100% responsible for the driving. Regardless of how manufacturers sell their "fully self driving cars", they are all still a "hotdog/not hotdog" type intelligence and the tech is there just to assist the driver, not replace.
I don't believe we'll solve autonomous driving too soon without adapting the infrastructure to it rather than the other way around. At this rate we'll have truly self driving cars (not "straight line, follow the highway lane self driving") right around the time everyone starts taking most personal cars off the road to replace them with something more efficient that better suits cities of millions. But right now too many people and manufacturers are lying to themselves that self driving cars are right around the corner, and even that they're mostly here.
The solution would be to fix the disinformation surrounding self driving tech and fining the hell out of any manufacturer that doesn't make it painfully clear on every channel that the tech is very limited, mentioning also the limits of the system (e.g. types of traffic it was tested on, how many road signs it can recognize out of the total, how much of the regular driving experience do those conditions cover, how well did it do compared to an average human driver). All in "human" language.
But consider this: If a human had the same problem, who would be blamed? If several humans, then the road would be changed, just like this. The process seems sound, but maybe the changes that make it more human-friendly make it more machine-hostile.
New thinking is needed on how we retrain machines after the change, as it necessitates uniformity - the machine would benefit with fewer variations in cattle grids for example. I reckon a simple change to the road markings for all cattle grids would be enough to make them logically identical - be it paint or signs or some other physical indicator.
> Decertifying a self driving car does nothing considering that today the driver is legally 100% responsible for the driving.
I don't know that this is true in this case. It seems like the problem here is the auto-braking feature kicking on, perhaps disorienting the driver or causing a following car to need to panic brake+swerve. This isn't an instance of a driver becoming inattentive because they expect the computer to act for them. Rather it's an ever-present system erroneously deciding to take control of the car - analagous to a mechanical failure. What could a driver do to avoid this behavior, besides knowing to disable that system ahead of the hazard?
Decertifying a self driving car does nothing considering that today the driver is legally 100% responsible for the driving
The owner of the vehicle is also 100% responsible for making sure the vehicle is road-worthy. Decertifying the autonomous-driving feature should probably trigger a manufacturer recall, so the features can be disabled permanently (not just a via software update).
Or alternatively, most countries have a periodic vehicle check (like the MOT test in the UK), and having non-certified features enabled could be a reason for failing the test.
Actually, we hired a £32K/$43K Nissan Qashquai for a week, and came across numerous idiosyncrasies. The "best" one was when we attempted to park against a low hedge, and the car set off all sorts of warning alarms. It then refused to lock the doors, so we had to move to a different part of the car park.
The refusal to lock was probably that you'd not fully switched off the engine. I've had this on my Qashqai. The fuel-saving system that automatically turns off the engine when you're stopped and out-of-gear can be confusing. This behaviour often happens when parking but it seems you still need to press the stop-start button to "completely" turn it off in order to lock the car.
This is the first car I've owned with "modern" driver aids and it is true that they are somewhat temperamental. For example, the auto full-beam headlights get very confused in fog (which can be dangerous!), whilst I often get a warning noise when approaching sharp bends at a perfectly safe speed.
Ah, that sounds interesting - we only had it for a while so didn't get chance to get to know all the quirks... we found the motorway lane wandering feature handy, but we had great fun when we tried to lock the damn thing and didn't move away from the car quick enough: it sensed the key was nearby and "helpfully" unlocked again. After the hire period we were glad to get rid of it!
> we found the motorway lane wandering feature handy
Honest curiosity, but as someone who drives quite a lot (A, B & C licenses) I don't understand why people need help staying in their lane. Flippantly I'd suggest that piloting the car and looking at the road is the absolute bare minimum of 'due care and attention' when driving, though obviously the prevalence of this feature would suggest otherwise.
Depending on the implementation, it trains/forces you to actually signal before changing lanes (I see people getting lazy with that). But while that's good for everyone overall, it's not exactly an explanation why people buy cars with these features.
> piloting the car and looking at the road is the absolute bare minimum of 'due care and attention' when driving
I agree! In this case it was novel to see how it worked (because the facility was there and new to us) but I'm more keen on using my eyes and mirrors. I could see how it could be useful in dense fog though, given many drivers don't bother with lights...
I assumed the intention is to alert a driver who really needs to take a break. I don't see what other use it has, other than to annoy me when overtaking a cyclist.
Of course, all of these cars are chock full of computers, speakers and monitors, but none of them ever tell you what's wrong in a more intelligible way than wild screeching.
I spent half an hour outside in the cold at 2am with a borrowed Nissan Leaf. It only made a loud beeping noise when I tried to lock it. Apparently, this noise means that the trunk isn't properly closed and that it therefore won't lock the car for you.
I think a step between what we have now and full human replacement will be roads that are certified for fully automated driving. Those roads would avoid features that confuse the automation, or have some sort of passive or active assistance installed.
> This is the kind of issue which will persist, vehicles which fail to recognize a real world situation shouldn’t get the benefit of changing the world to fit them.
There are already places that regularly confuse human drivers, and those do get fixed (though usually only after someone dies). I don't see why we couldn't do the same for driving robots, especially when they get more common over time.
>Or simply decertify all of the vehicles that behave incorrectly from driving on public roads.
At least in the US we try to grandfather in old tech because the poors have enough problems without the government telling them they can no longer use their junk.
If a box was checked, a standard was met or a capacity was stated when the vehicle shipped the box remains checked more or less in perpetuity. This makes for some great internet arguments when the pearl clutching crowd tries to appeal to the authority of manufacturer statements with regard to older vehicles.
A way to do this would be to create a class of forced manufacturer recalls to disable the broken feature which if not completed would block yearly registration renewal, like the emissions checks some places require.
Indeed recently while going through the California smog check and registration process, the dealership told me I had open recalls which might block my registration. They didn’t, but clearly something like that mechanism exists.
This doesn’t unfairly harm people who already own vehicles.
For unusual fixed features like this, perhaps car manufacturers should start putting exceptions into their navigation databases, rather than tweaking their automatic detection. You generally don't want to be driving over cattle grids, with their sudden gradient changes, at high speed anyway. So tell the car to slow down as it approaches (or warn the driver loudly to slow down), and briefly disable automatic braking. If it's a rare event, it'll have the drivers' attention.
I see the future of self-driving cars to be one where highway authorities map and sign features in a way that is designed for self-driving cars to be able to parse, rather than requiring them to heuristically sense as much as they do today. This would be a step down the path that's inevitable anyway.
While the question of who pays for it is important, we should be focusing on the best way to get safe driving.
If car sensors and algorithms can get to 99.99% of road features, then changing the remaining 0.01% to work with modern car technologies seems like a reasonable use of resource. No algorithm is ever going to be completely perfect.
What happens if there are bugs or we find code being not up to the mark? Should all the models using that version be disabled till OTA updates are rolled out.
For reference bugs in Boeing's plane resulted in the whole fleet being grounded.
I don't see a problem with autonomous vehicles having a manual control mode, unless we're talking about a scenario some decades in the future where you don't actually need a driver's license for an autonomous vehicle.
To create a "virtually smooth ride across the grid", highways teams spent a month raising about 90ft (27.4m) of road and redesigning the "carriageway approaches".
I assume this is because the metal bars in the cattle-grid appear as a very reflective object in the cars' radar sensors which it interprets as an obstacle?
I can see this would be a problem with cars that use only radar or radar+ultrasonic. Besides Tesla (and Waymo) cars which do sensor-fusion with vision, what other cars attempt to corroborate radar data with confirmation it's an obstacle and not a coincidental highly radar-reflective object?
No, it's because the grid was angled relative to the road (and the surrounding hillside). If you regard the road as level, that grid was a sudden steep incline or decline, and after that the road suddenly returned to level.
They fixed it by reworking the road so get rid of the suddenness.
From the radar's point of view going uphill, the road disappeared. The radar would see a horizon, and empty air in front of the car, like the edge of a cliff.
From its point going downhill it would look like an angled metal wall, or a bump, or something, and then the horizon and empty air beyond it.
I was under the impression that some of the Tesla crashes into highway barriers were exactly because it relied too much on the stereo vision and not enough sensor fusion with RADAR or long range Ultra-sound?
Highway barriers are different problem. RADAR, and I think ultra sound, has very poor angular resolution. So it’s very hard to distinguish between and object directly in front of you, from one just off do the side.
As a result cars ignore RADAR data about stationary objects when the car is moving at speed. The assumption being those stationary objects are things like highway barriers to side of the car. You don’t want the break slamming on just because your driving parallel to the highway barrier.
Consequence is that cars ignore stationary objects like highway barriers, when moving at speed. So if you’re heading straight at it, the car won’t slow down. But then the assumption is that the human really ought to notice they’re driving at a wall.
Probably the bars are square and thus act like corner reflectors (retroreflectors), giving a very bright radar return similar in intensity and position to the right angle at the base of a wall.
My wife and I were out driving looking at Christmas lights a few weeks ago and I tried to reverse the car with no other cars around but the car did an emergency stop because the lights confused the system into thinking someone was behind me or going to cross behind me. Automatic can be good but also too easily fooled.
What is going to stop someone from putting up a sign on the side of a highway that looks like a photo of a red light? Very cheap to deploy and extremely dangerous.
I remember seeing a post on HN (but I can’t find it) where simply flashing such a photo for ~100ms was enough to get cars to come to a sudden stop
Humans would be able to see that it is on a long stretch of straight road with no intersection in sight and so while they might be confused, they would not come to a sudden and instant stop.
The paper showed that self-driving cars are not quite at that level of deductive reasoning
I'm not so sure about that. I think it would take more than flashing a photo for 100ms (or just putting up a photo at all for that matter) to trigger a human to stop. You have to overcome the context and I think that's harder than "one photo on the side of a highway".
Any human who stops because he saw a red light pic should not be driving in the first place. Fortunately I think that fraction is very small.I certainly hope it does not include you.
Why would that be a sudden stop unless it is hidden and pops out at the last second? Simply putting up a photo of a traffic light is going to be detected as, well, a traffic light as the car approaches. Maybe with a blind curve or with some other similar obstruction you could create a situation where the photo of the light isn't seen until the last second but you can do the same with human drivers and a fake stop sign.
What's to stop someone from throwing out a bunch of jagged glass or nails on the road to puncher someone's tire? Very cheap to deploy and extremely dangerous. The thing that stops this is human decency. And yes, that safeguard does fail sometimes. But fortunately, either situation isn't permanent -- as soon as a cop (or any city employee or private citizen) drives by and sees it (especially the fake traffic light) they can call it in and have it removed in short order.
Seems like cars get spooked by odd things in their paths just like cows, huh?
Well, I see 2 aspects that make this a "good" failure scenario, regardless of how they solve it.
First, having the sensor+algorithm interpret something conservatively as "should stop" for it rather than "ok to go through" at possible grave cost, is the right behavior.
Second, this is already (as you can tell by the exceptional nature of the story) a very rural and special case where the driver should be driving alertly (and slowly) anyway. From the photo, this is not something meant to be gone over at high speed blindly.
As for the cars going off the road, that needs more explanation as the only action cars should be applying automatically is braking, not turning.
Oi! Having grown up around there I object to this being described as a special case; it's everyday driving conditions for a lot of people. :) Better built cattle grids appear on main roads with good visibility and it's not unreasonable to cross them at 40mph (thinking about it there may even be the occasional example people cross at 60).
An anecdote from around 15 years back: the government considered closing the driving test centre in Minehead because there was no dual carriageway nearby on which learner drivers could demonstrate correct merging skills. Residents collectively replied that all the centres in London should be closed as they did not qualify drivers to deal with reversing down narrow lanes with sheep and mud on the road (...this is painfully evident if you're stuck behind an urbanite driver anywhere near Minehead in summer). Anyhoo, the test centre still operates today.
Emergency stops can be very dangerous, especially if there are people behind you or if the road is slippery.
What makes you think that this is "very rural" or a special case? Why does it matter if it is rural -- should cars not work in non-urban environments?
I have 100% never been around cattle enough to know for sure, but I was under the impression that cattle crossings didn't so much spook cows as are uncomfortable to cross and therefore they stay away.
When one car is following another, and the front car applies the emergency brakes, and the back car (which doesn't have quite as good tyres so can't emergency brake so quickly) swerves to avoid the front one.
This means you didnt adapt your driving style to the conditions of the road, basically tailgated someone. Dont know about other jurisdictions, but in my EU country law is clear on this one - the gap between yourself and the proceeding car must permit safe stop in case of emergency at all times. If you rear end someone emergency braking (child ran into the road, car caught fire, front window shattered with debris, etc) you are 100% liable and the sole guilty party.
The concept people would vilify automatic braking systems is horrifying. They, like GPS mapping save countless lives.
And we as a society need to change our environments to further optimise them. Which good engineers have been doing forever, this is part of engineering.
It was surprising the BBC article actually was neutral, it was a pleasant surprise. The council realised their interesting mistake and fixed it. But it would be nice to see the BBC going to neutral good and mentioning the lives saved and other measures councils can do to improve these technologies.
This feels like the physical world version of writing at the top of your autopilot function
if(thisSpecificCattleGrid)
return True //Skip the autopilot for this case
Like... yeah, that will stop the autopilot from crashing when you come across that cattle grid. It doesn't change the fact that your autopilot has some known bad behaviour and you've got no idea where else it'll show up.
But the automation is not crashing into anything. Automatic obstacle avoidance systems in current cars are only allowed to engage brakes. Something doesnt add up in that story.
The story doesn't say cars left the road by swerving off the side. I think they left it vertically because it was shaped like a jump and they were just driving too fast. It also doesn't say automatic braking/obstacle avoidance caused them to leave the road. Nor does it say the road was altered because of automatic anything.
This looks like a massive reading comprehension problem that people here are suffering from. Journalists are experts in being pedantically correct in all their facts but leaving the careless reader with a completely wrong idea of what happened.
Wow. I actually didn't see that, even after re-reading it looking specifically for it. I had to ctrl-F to find it. It's in bold under the picture so I probably mistook it for general non-article clutter and ignored it. Reading comprehension problem here too!
It does use that causality avoidance word "after". That's technically honest because the author only knows the sequence of events, not the causality, but it also means we only know that too. In an extreme case, it might not even be the same cars that mistook it for a wall as the ones that came off the road, but it also might not be the autonomy that caused them to come off the road, but the driver's actions in response to the braking.
It shows up in other places as well, The Netherlands has a lot of traffic islands at crossings, they often have a metal "fence" on it around schools to force kids on bikes to zig-zag (slowing them down instead of crossing at speed). It's very manufacturer dependent how these are handled. The systems I have experience with are Volvo, Tesla and BMW and they handle things quite differently.
My girlfriend's Volvo was very trigger happy on the emergency brake feature, it would hit the brakes for traffic islands all the time because it didn't know you were going to steer around it. It nearly caused us to be rear-ended at 50 km/h a couple of times.
My BMW and previous Tesla on the other hand almost completely ignore stationary road-side kind of objects. Both have never braked for a traffic island in the few years that I drove them.
The downside is I guess that there are known cases of Tesla Autopilot driving into things (the famous lane-divider incident comes to mind). BMW doesn't sell their system as autopilot, but their manual also has a big warning to be aware of things blocking the road and not to rely on the car to stop for those. So I expect their system, faced with that same lane-divider would probably also crash into it.
Though I think the fault lies with the car manufacturers, I wish they would find an alternative for these cattle grids because they can be quite hazardous for motorcycles in the wet.
Bicycles and pedestrians too. When I'm out walking I can often hop around the wide of the grid or climb the fence but not always. I never enjoy traversing a cattle grid.
The obvious alternative is a gate. But it's been established now that motorists never have to leave their cars to open gates etc. on the public highway so I can't see that happening (although it does exist in some rare level crossings, for example). Also they probably can't be trusted to close the gate behind them.
The UK already has some of the safest roads in the world. We achieved that by spending money on good road design and education. Now we are having to spend even more money to "fix" the roads for car safety features designed for other countries who didn't spend the money to make their roads safe? How frustrating.
It's not actually stated that automatic braking is the reason for the change. It even gives a different reason: 'To create a "virtually smooth ride across the grid"'. They straightened out a step-shaped road into a straight one which is surely just a better experience for all cars and their occupants. I almost wonder if the vehicles that left the road did so by making a jump using it as a stunt ramp.
Oh wow, you could totally see someone using that as a jump by accident, there’s no visual warning that the gradient is about to change if your approaching the cattle grid from below.
Easy to imagine someone at night, taking liberties with the speed limit, hitting that thing at 70mph and taking off.
[Warning: over-serious response coming.] There's a big "cattle grid" sign a little way before it, so I don't think that's a realistic worry. As an aside, that road is long, but a dead end: it leads up to a beauty spot / hiking point. The road also largely lacks dividing lines (i.e. it's sort-of single track, though often wide enough for two vehicles), and the cattle grid comes not long after a sharp corner, so I can't imagine many vehicles would have been able to get to flying speeds. Besides, that neck of the woods has plenty of interesting driving roads including the main A39 out of Porlock just a couple of miles away, that I imagine are of more interest to speed demons.
which one is the before and which one's the after? Usually it should be before first and after second, but in this case the first picture looks smoother than the second picture?
It would be nice if there was a sign post for "auto drive failure ahead". You could put that a minute up the road, the car could see it, ping at the driver until he take over, cross the grid and then re-engage.
This seems to be the correct solution for 99% of these Edge cases...
It seems it isn't just auto drive. Cars with object collision detection are slamming on the brakes when they come near this cattle grid. Signs wouldn't help in that case.
147 comments
[ 5.8 ms ] story [ 153 ms ] threadSolution: replace spotted mice with spotless mice
I'd love to see you try and convince a farmer that this is an "improvement" and that they should replace their cattle guards
Rationale: 2D is easier to detect and they don't move.
I'm not going to convince farmers of anything. It will simply be untenable for future governments to have public roads that are not usable by cars.
https://cdn.newsapi.com.au/image/v1/1a6d5e7bcd4f55f2ca79de5a...
https://www.cornwalllive.com/news/cornwall-news/satnav-stuck...
https://imgur.com/zTNZEl6
The cows always win in the end
A cow that regularly attempts to cross cattle guards is first to sale or slaughter.
Edit: And now I'm that guy on the internet that I hate, who saw an amusing animal gif and immediately went into buzzkill mode :-/
This is just the cow-equivalent of a bungee jumper or free-solo mountain climber.
https://www.youtube.com/watch?v=-E9PWOAsc9o
Editors getting desperate with their TSLA puts.
But pragmatically, they likely don't answer the phone. Or email.
So, it is still easier to fix the road. And hope the issue doesn't resurface elsewhere and inadvertantly injure or kill too many people.
The solution is having the balls to lay down good policy/regulation. There is no point forcing car companies to use a certain algorithm, for example, but it's not unreasonable to build a standard basket of problematic road signs which the car must pass - e.g. a cattle grid isn't too bad, but if a driverless car doesn't stop for a speed limit outside a school the company should have some liability.
You don't get to abdicate responsibility on the road just because you bought in to some corporations marketing.
(While, sadly, at the same time it seems from the outside that the ACAS code got written by the same outsourced dev team that work on their website the rest of the time. :sigh: )
Imagine you’re driving along in full control on an empty country road. Out of nowhere your car panics, slams in the brakes, deploys seatbelt tensioners, and flashes bright warning lights because clearly you fell asleep or something if you cant see that big wall.
Would you react? Maybe swerve coz you’re startled?
Words mean things. You don't get to say they mean different things when you say them just because it helps you sell more cars. Not in the fine print, and not in a court.
If your company's automated emergency braking, or lane assist, or adaptive cruise control - causes or contributes to a problem, someone from that company should be in court next to the driver facing judgement for the consequences.
(I'd even go so far as to suggest if the existence of a system leads to a situation where an incident occurred because of the driver's assumption that the automated system would prevent it, then the company should need to justify why the driver should be held responsible instead of the software. If your advertising claims "autopilot!", you're going to be apportioned a significant portion of liability even if your fine print says "Drivers must keep alert and ready to take control at all times" - you should be judged based on a reasonable persons expectation from the language you use to promote your vehicles, not some w4easel words in your T&Cs.)
When you say that a manufacturer should be held liable if their system '... contributes to a problem...', it seems like you're suggesting adoption of the (notoriously problematic) 'if not but for' test. This sort of test is used in negligence cases, but can be useless, as almost anything satisfies it (i.e. slow service at a coffee shop may have been a necessary condition for the timing of that exact accident).
In practice, putting the responsibilty on auto manufacturers for every specific injury or death that happens due to a safety system, without somehow giving them "positive credit" for lives saved, massively delays or entirely prevents these sort of advancements from being made. We definitely need processes to verify that new features are actually functional, typically through recalls and product liability suits, but simply putting blame on manufacturers for all accidents is a terrible idea.
This is a particular challenge in California because our school zone signs specify that the limit only applies "when children are present."
Since a large fraction of kids are driven to school (at least in southern California), you know that this is happening because of the massive traffic jam near the school.
I'm wondering how you would get around metaparameter optimization here, or the fact that it's too easy to optimize to a known test procedure rather than the real world?
> So, it is still easier to
... hold them responsible in court for damage/injury/deaths caused by the actions their vehicles take which are not under control of the driver.
If your corporation (via it's self driving software) kills someone, the corporation will "go to jail", a C level exec does the jailtime and the entire company will be unable to carry on any of it's normal business for the duration of the sentence (while still being held responsible for it's obligations to staff/suppliers/cerditors). If that self driving software is judged to be liable for a traffic infringement, the corporation will pay an amount equal to the percentage of income that a human driver would have paid. If it's an infringement with a ~$400 fine, the corporation should pay ~0.5% of their annual income - the same sort of penalty as "an average salary" person.
A couple of $20mil fines and a 6 month total company shutdown would be very "proactive" in encouraging Tesla to get Elon to shut the fuck up with his big bold claims that cannot be backed up. If Uber had been slapped with the "corporate personhood" equivalent of whatever penalty a human driver who'd killed that person crossing the road with a bicycle they'd be far more circumspect about using minimum wage contractors as "safety drivers" in their prototypes.
(I can dream, right?)
I can see a wide spectrum between my (admittedly over punitive towards corporations) ideas and this (somewhat intentional misinterpreted) idea of yours. But I do honestly think Uber and Tesla have evaded morally deserved punishments by blaming customers or contract staff who were pretty much doing what everybody would expect to be able to do using the technology as described.
"it's inevitable that even a responsible _person_ will screw something up" - but that doesn't mean we give them a zero consequence break if they kill or injure someone. (We do, however, distinguish between things like murder, manslaughter, and negligence-causing-death. And penalise differently, based at least partly on the ability of the punishment to discourage other people from doing similar things. Perhaps Uber was only guilty of negligence-causing-death for using minimum wage contractors as safety drivers. Perhaps Tesla was guilty of manslaughter by advertising as "autopilot" a thing that drives full speed into offramp dividers?)
I’d kinda like to see Tesla’s reaction if they were told that any “autopilot” crashes could result in the entire fleet of that model car being taken off the road, with Tesla responsible to all the owners for the disruption while a thorough investigation takes place, possibly a year or more.
It will be interesting to see how regulators treat autonomous vehicles. I have a feeling they will defer to the existing recall mechanism rather than completely grounding a fleet
In the large, it's always a shame to fix a good test case.
This is the kind of issue which will persist, vehicles which fail to recognize a real world situation shouldn’t get the benefit of changing the world to fit them.
An inquiry should be done into how these vehicles came to fail to recognize this road feature and the upstream cause (not just the bug by the systematic issues in the organizations which produced vehicles with this bug) should be fixed before they’re rectified to operate on roads.
The good ones would then sort out why their braking system is faulty. Perhaps it gets confused about other things in other serious ways.
Do you really regard braking in front of a sudden incline/decline as ridiculous?
From uphill: "There's something weird in front of the car, best drive very slowly". From downhill: "WTF I can't see the road any more, best drive very slowly." Both seem quite sensible to me, I don't understand why so many people on the thread seem to disagree.
These grids usually lie flat on the ground and in some areas are commonplace. Seems to me these sensors are still needing work. Perhaps they see horizontal lines and panic?
Cars running off the road for no good reason is ridiculous. It is indefensible to assert this is OK. Denying it or otherwise covering it up is definiteky ridiculous.
I don't believe we'll solve autonomous driving too soon without adapting the infrastructure to it rather than the other way around. At this rate we'll have truly self driving cars (not "straight line, follow the highway lane self driving") right around the time everyone starts taking most personal cars off the road to replace them with something more efficient that better suits cities of millions. But right now too many people and manufacturers are lying to themselves that self driving cars are right around the corner, and even that they're mostly here.
The solution would be to fix the disinformation surrounding self driving tech and fining the hell out of any manufacturer that doesn't make it painfully clear on every channel that the tech is very limited, mentioning also the limits of the system (e.g. types of traffic it was tested on, how many road signs it can recognize out of the total, how much of the regular driving experience do those conditions cover, how well did it do compared to an average human driver). All in "human" language.
As someone who only sells products using ML, I find the discussion here inorganic.
Maybe most of the blame.
But consider this: If a human had the same problem, who would be blamed? If several humans, then the road would be changed, just like this. The process seems sound, but maybe the changes that make it more human-friendly make it more machine-hostile.
New thinking is needed on how we retrain machines after the change, as it necessitates uniformity - the machine would benefit with fewer variations in cattle grids for example. I reckon a simple change to the road markings for all cattle grids would be enough to make them logically identical - be it paint or signs or some other physical indicator.
Though I don't think fully autonomous driving will be possible in near future. There are too many edge cases.
I don't know that this is true in this case. It seems like the problem here is the auto-braking feature kicking on, perhaps disorienting the driver or causing a following car to need to panic brake+swerve. This isn't an instance of a driver becoming inattentive because they expect the computer to act for them. Rather it's an ever-present system erroneously deciding to take control of the car - analagous to a mechanical failure. What could a driver do to avoid this behavior, besides knowing to disable that system ahead of the hazard?
The owner of the vehicle is also 100% responsible for making sure the vehicle is road-worthy. Decertifying the autonomous-driving feature should probably trigger a manufacturer recall, so the features can be disabled permanently (not just a via software update).
Or alternatively, most countries have a periodic vehicle check (like the MOT test in the UK), and having non-certified features enabled could be a reason for failing the test.
This is the first car I've owned with "modern" driver aids and it is true that they are somewhat temperamental. For example, the auto full-beam headlights get very confused in fog (which can be dangerous!), whilst I often get a warning noise when approaching sharp bends at a perfectly safe speed.
Honest curiosity, but as someone who drives quite a lot (A, B & C licenses) I don't understand why people need help staying in their lane. Flippantly I'd suggest that piloting the car and looking at the road is the absolute bare minimum of 'due care and attention' when driving, though obviously the prevalence of this feature would suggest otherwise.
Any thoughts gratefully appreciated.
I agree! In this case it was novel to see how it worked (because the facility was there and new to us) but I'm more keen on using my eyes and mirrors. I could see how it could be useful in dense fog though, given many drivers don't bother with lights...
I spent half an hour outside in the cold at 2am with a borrowed Nissan Leaf. It only made a loud beeping noise when I tried to lock it. Apparently, this noise means that the trunk isn't properly closed and that it therefore won't lock the car for you.
> This is the kind of issue which will persist, vehicles which fail to recognize a real world situation shouldn’t get the benefit of changing the world to fit them.
There are already places that regularly confuse human drivers, and those do get fixed (though usually only after someone dies). I don't see why we couldn't do the same for driving robots, especially when they get more common over time.
At least in the US we try to grandfather in old tech because the poors have enough problems without the government telling them they can no longer use their junk.
If a box was checked, a standard was met or a capacity was stated when the vehicle shipped the box remains checked more or less in perpetuity. This makes for some great internet arguments when the pearl clutching crowd tries to appeal to the authority of manufacturer statements with regard to older vehicles.
Indeed recently while going through the California smog check and registration process, the dealership told me I had open recalls which might block my registration. They didn’t, but clearly something like that mechanism exists.
This doesn’t unfairly harm people who already own vehicles.
I see the future of self-driving cars to be one where highway authorities map and sign features in a way that is designed for self-driving cars to be able to parse, rather than requiring them to heuristically sense as much as they do today. This would be a step down the path that's inevitable anyway.
Next trick: QR codes with cryptographic signatures, and a huge traffic jam when the traffic authorities forget to renew their certificate...
If car sensors and algorithms can get to 99.99% of road features, then changing the remaining 0.01% to work with modern car technologies seems like a reasonable use of resource. No algorithm is ever going to be completely perfect.
For reference bugs in Boeing's plane resulted in the whole fleet being grounded.
To create a "virtually smooth ride across the grid", highways teams spent a month raising about 90ft (27.4m) of road and redesigning the "carriageway approaches".
I can see this would be a problem with cars that use only radar or radar+ultrasonic. Besides Tesla (and Waymo) cars which do sensor-fusion with vision, what other cars attempt to corroborate radar data with confirmation it's an obstacle and not a coincidental highly radar-reflective object?
They fixed it by reworking the road so get rid of the suddenness.
From its point going downhill it would look like an angled metal wall, or a bump, or something, and then the horizon and empty air beyond it.
As a result cars ignore RADAR data about stationary objects when the car is moving at speed. The assumption being those stationary objects are things like highway barriers to side of the car. You don’t want the break slamming on just because your driving parallel to the highway barrier.
Consequence is that cars ignore stationary objects like highway barriers, when moving at speed. So if you’re heading straight at it, the car won’t slow down. But then the assumption is that the human really ought to notice they’re driving at a wall.
Then again this could be the same perpetual 10-15 years away from FSD, so maybe it will actually be 25-50.
I remember seeing a post on HN (but I can’t find it) where simply flashing such a photo for ~100ms was enough to get cars to come to a sudden stop
The paper showed that self-driving cars are not quite at that level of deductive reasoning
Tech aside, the point is that disrupting traffic or causing accidents is nothing new, you can practice it today just as effectively!
https://xkcd.com/1958/
Well, I see 2 aspects that make this a "good" failure scenario, regardless of how they solve it.
First, having the sensor+algorithm interpret something conservatively as "should stop" for it rather than "ok to go through" at possible grave cost, is the right behavior.
Second, this is already (as you can tell by the exceptional nature of the story) a very rural and special case where the driver should be driving alertly (and slowly) anyway. From the photo, this is not something meant to be gone over at high speed blindly.
As for the cars going off the road, that needs more explanation as the only action cars should be applying automatically is braking, not turning.
An anecdote from around 15 years back: the government considered closing the driving test centre in Minehead because there was no dual carriageway nearby on which learner drivers could demonstrate correct merging skills. Residents collectively replied that all the centres in London should be closed as they did not qualify drivers to deal with reversing down narrow lanes with sheep and mud on the road (...this is painfully evident if you're stuck behind an urbanite driver anywhere near Minehead in summer). Anyhoo, the test centre still operates today.
What makes you think that this is "very rural" or a special case? Why does it matter if it is rural -- should cars not work in non-urban environments?
I have 100% never been around cattle enough to know for sure, but I was under the impression that cattle crossings didn't so much spook cows as are uncomfortable to cross and therefore they stay away.
not buying it. Not a single car on the market swerves after detecting an obstacle.
And we as a society need to change our environments to further optimise them. Which good engineers have been doing forever, this is part of engineering.
It was surprising the BBC article actually was neutral, it was a pleasant surprise. The council realised their interesting mistake and fixed it. But it would be nice to see the BBC going to neutral good and mentioning the lives saved and other measures councils can do to improve these technologies.
This looks like a massive reading comprehension problem that people here are suffering from. Journalists are experts in being pedantically correct in all their facts but leaving the careless reader with a completely wrong idea of what happened.
>Cars have come off the road trying to cross a cattle grid after sensors mistook it for a wall and slammed on the brakes.
mean according to you?
It does use that causality avoidance word "after". That's technically honest because the author only knows the sequence of events, not the causality, but it also means we only know that too. In an extreme case, it might not even be the same cars that mistook it for a wall as the ones that came off the road, but it also might not be the autonomy that caused them to come off the road, but the driver's actions in response to the braking.
My girlfriend's Volvo was very trigger happy on the emergency brake feature, it would hit the brakes for traffic islands all the time because it didn't know you were going to steer around it. It nearly caused us to be rear-ended at 50 km/h a couple of times.
My BMW and previous Tesla on the other hand almost completely ignore stationary road-side kind of objects. Both have never braked for a traffic island in the few years that I drove them.
The downside is I guess that there are known cases of Tesla Autopilot driving into things (the famous lane-divider incident comes to mind). BMW doesn't sell their system as autopilot, but their manual also has a big warning to be aware of things blocking the road and not to rely on the car to stop for those. So I expect their system, faced with that same lane-divider would probably also crash into it.
The obvious alternative is a gate. But it's been established now that motorists never have to leave their cars to open gates etc. on the public highway so I can't see that happening (although it does exist in some rare level crossings, for example). Also they probably can't be trusted to close the gate behind them.
This article has before and after pictures from the same angle: https://www.somersetcountygazette.co.uk/farmer/18996741.catt...
Easy to imagine someone at night, taking liberties with the speed limit, hitting that thing at 70mph and taking off.
"Easy to imagine illegal, reckless, dangerous behavior posing a danger."
which one is the before and which one's the after? Usually it should be before first and after second, but in this case the first picture looks smoother than the second picture?
This seems to be the correct solution for 99% of these Edge cases...
http://cms.newindianexpress.com/cities/kochi/2019/jan/17/wan...