* range uncertainty grows quadratically with range
* range uncertainty is unbounded in the case of featureless surfaces or bad illumination
* range from stereo disparity depends on looking at very small differences between two images and may be much more adversely affected by weather (rain drops on the lens) compared to, say, lidar.
This is an advertisement piece written by a NODAR advisor about how cool their "Hammerhead" system is.
TLDR: It's stereo cameras with wide baseline and software compensation for vibrations/camera shake.
That said, I am pretty disappointed that there's no mention of the actual problems that people working with stereo cameras in cars have, which is that optical matching usually doesn't have a unique solution and that reflections ruin everything. The latter especially gets worse the wider you make the stereo baseline, so I'd bet you that NODAR will fail badly when there's ice on the road.
That combination of false correspondences due to reflections and AI in-painting to produce the illusion of a complete dataset when it is impossible to see depth everywhere, might be quite dangerous. We laugh at Skydio drones for crashing into windows, puddles, and lakes because their AI (with 6 cameras) gets confused by reflecting things. I'd find it a lot less funny if that was my car crashing.
From discussions with engineers that work at car manufacturers on technology for self-driving cars, the biggy is really the weather. All these technologies work fine in sunny weather, but it's really adverse weather conditions where things go south.
One of the friends told me that is belief is that proper self-driving is still a long time away, because of this.
>> We humans have good depth perception out to about ten meters due to the roughly six centimeter separation of our eyes.
Depth perception is more complex than eye separation distance. By the above measurement, most ball sports would be nearly impossible. We all still play baseball. The reality is that we use multiple techniques for judging distance. Our heads move, increasing the separation between observations (see any owl scoping out prey before launching). Lens focus ads another flow of information. And we judge distance to distant objects using their apparent positions relative to others closer to us.
Talk to anyone who has lost an eye. They are perfectly capable of driving cars and throwing balls accurately well beyond ten meters.
I've read that nearly half of people have low/no depth perception, and use a variety of other cues to judge distance. Sorry, but I don't have the citation handy.
My mom has amblyopia and is old enough that it didn’t get treated. She can drive just fine, but she can’t grab the clothes line between her fingers. She has to sort of paw at it and catch it with her palm.
And I’d say visual processing is a lot more complex then we give it credit for. We don’t understand what’s really all going on yet. We still are trying to understand microsaccades for example. Ocular processing is extremely fascinating.
I have mostly untreated amblyopia and while mine doesn't sound as bad as your mothers, I've always been terrible at most ball sports despite being reasonably athletic due to poor depth perception.
Not to mention object recognition. If I see something that resembles a car, I know roughly what size I expect a car to be so based on the size it appears, I can reasonably estimate its distance from me. Of course this breaks down when objects don't meet our expectation in terms of size and can be the source of some interesting optical illusions.
Lidar is kind of magic. The cost of tech seems to reduce pretty substantially over time. When folks argue that lidar should not be used - other than cost I don't get it, it seems to give you some great added awareness of environment, especially looking at for example Waymo and some others really advancing things in this space. 5 years from now? Going to be pretty slick?
> “Lidar is a fool’s errand,” Elon Musk said. “Anyone relying on lidar is doomed. Doomed! [They are] expensive sensors that are unnecessary. It’s like having a whole bunch of expensive appendices. Like, one appendix is bad, well now you have a whole bunch of them, it’s ridiculous, you’ll see.”
Except that the appendix is a useful organ. Had it been truly useless it would have evolved away long ago, yet we find dinosaurs had them too. I think Musk is also wrong about lidar. It probably does have a place in this problem even if, like his blind spot re the appendix, Musk doesn't currently understand what that place may be.
"Evolution Of The Human Appendix: A Biological 'Remnant' No More"
At least one purpose of the appendix has been already been found, a long time ago. It serves as an isolated/safe repository for gut bacteria in case an adverse event wipes out the rest of the population.
Yes you could think of an analogous statement being: why do we need lenses? A raw exposed imager is just fine, we can use machine learning to extrapolate what the lens would be providing. And yes that's true but why in the hell would you want to spend time doing that? Self driving is hard enough already.
Why would that be optimal? Sounds like you'd just be replicating humans - who are pretty lousy and non-optimal even in cars designed specifically for them!
I think it might be like ray tracing. It's the holy grail of graphics, which makes engineers very positive about it. Yet it is slow and expensive and most problems do perfectly fine if not better with "conventional graphics".
Lidars are expensive, fairly large, power hungry and depending on what your using them for, noisy.
Musk doesn't like them because they are expensive. Not because they don't work. He's spent billions trying to polish the turd that is their monocular depth estimation + radar system. Sure its impressive, but its nowhere near good enough for safe autonomy. (I am sceptical that they ever will with that sensor layout. Thats neatly avoiding the issue that there is no redundancy)
If you want ground truth data from a moving vehicle, then there is no real alternative to lidar[1]. Anyone who tells you otherwise is either:
1) an idiot
2) a musk fan
3) trying to sell you something.
Now, sure for "autopilot" (cruise control) where there isn't that much to do, radar and basic object detection is good enough. But any sign of danger and it backs off, leaving you no time to unfuck the situation.
[1] I work for a mapping company, There are other ways to get to <50cm accuracy point clouds, but its not fast and requires a boat load of GPU (in the order of thousands of hours per km driven)
The article briefly mentions (but does not do justice) to the fact that identifying small objects at extreme range is something that LIDAR is _also_ really bad at, due to extremely low effective resolutions by camera standards. High-end LIDAR units cap out at ~100 channels (roughly equivalent to vertical pixel size), usually over >100deg vertical field of view, for less than one "pixel" per degree. That's not nearly enough to detect a brick at 100m, and pales in comparison to the angular resultions an HD camera with a decent lens can achieve.
In general, it's not yet clear to me whether LIDAR is all that useful for the hard parts of the self-driving car problem. We know it's extremely useful for the easy parts (not hitting curbs, other vehicles, or walls), but it's not clear whether LIDAR helps that much with small objects, vehicle behavior prediction, or object classification (paper bag vs. dog)
I've worked with a researchers who use LIDAR heavily. Amazing technology, but aside from cost/weight/size/expense there's one big disadvantage. LIDAR doesn't work like eyes.
So in certain conditions LIDAR can see better than human eyes, like say complete darkness. This could lead to a car behaving differently than a human driver which might well cause an accident.
In other conditions LIDAR can see much worse than a human. Depending on the fog, snow, rain, dust, tumbleweeds etc can look more solid to a lidar than a human eye. Which can also cause accidents. A human might recognize a whisp of fog, whose outline and volume look like a fallen tree, but a camera would have a better chance.
So even with a magic LIDAR that costs $100, has good range, solid state, etc. it's not clear what LIDAR offers over a pair of cameras, or other cheap sensors like cameras, mm band (60 GHz) radar, and ultrasound.
Additionally LIDAR systems need cameras anyways. Is that a ambulance or UPS truck? Schoolbus or city bus? Police car or Camry? Does the police car have it's lights on?
Did someone flash their high beams at you? Delivery truck or post office truck? Color is hugely important in predicting what's going to happen. Sure you can add cameras to LIDAR, but then why? Doubly is you have two cameras for distance.
So I'd love to be wrong, and indisputably LIDAR is getting better, but seems like pairs of cameras might well be the best fit for driving like a human. Doesn't seem like it could ever be LIDAR, but LIDAR + cameras could compete, but what does LIDAR give you over a pair of cameras?
It seems like lidar has a good result in certain safety cases. Is that a parked white truck stopped in front of you? I tractor trailor crossing the road? Is that a highway divider I'll be steering you into? These are just some of the accidents I think of with vision that might have been avoided with lidar.
In particular, for pedestrians, bicycles, other vehicles, maybe even children running in street, is the limited resolution enough to still substantially increase safety?
Where I got with this - hitting a brick is bad. Hitting a person is much worse. A fair number of never hit type items seem to still be picked up with Lidar.
Well inherently cameras have much better angular resolution. So for current level of tech you get more pixels per object for any given distance.
Lidar's big advantage was a distance for each "pixel", but that advantage seems to mostly disappear for pairs of cameras, at least for common objects like pedestrians, bicycles, most vehicles, etc. However that does depend on edge detection or enough visual detail to be able to match an object through two different perspectives. A featureless white wall that consumes the entire camera view would result in no distance for a stereo camera, but would still work with lidar. Not sure if that's enough of a limitation to break things with say a large white 18-wheeler crossing the road.
I've heard new interest in mm wave (60 GHz or so) radar, but no idea where it falls compared to lidar and stereo cameras as far as distance and angular resolution and sample rate.
> Given engineering, space, and styling limitations, this typically results in a small unit placed forward of the rear-view mirror with cameras spaced only about 20cm apart
It's startling how hard it is to get a vehicle project to change anything. A non-structural but rigid pressed-steel truss that goes somewhere behind the grille of the vehicle would make this software compromise unnecessary.
I'm aware that Tesla basically took Mobileye's single-camera product off the shelf, but I wish someone in the Model S development team could have added that dual-camera structure as a requirement on Day 1.
One advantage of keeping it behind the windshield (aside from having a higher point of view), is that they get automatic camera cleaning from the windshield wipers (plus defrosting). If they put it lower behind the grill, then they need a completely separate cleaning/defrosting system.
My car's lower bumper mounted radar antenna stops working any time I drive in the snow because it quickly gets coated with a layer of snow.
HN headlines like this don't make me go, "Wow!" They make me think, "So you're saying there's a larger chance that the software is going to mistake that brick for something else because it has that much more granular detail."
It sure is a puff piece. I was going to reject it out of hand. In fact I was in the middle of slating it when I actually looked up how hammerhead sharks see. I learnt that hammerhead sharks have really good binocular vision. (http://news.bbc.co.uk/earth/hi/earth_news/newsid_8376000/837...)
I am wary of the depth map they've provided. It looks suspiciously smooth and stable. Given that roads have little to no features of use, the belief propagation must be very good (or its been tarted up in post.)
I assume they use feature matching to estimate the camera movement, and the those same features are compared from each camera to get the divergence. I would be interested to see the calibration setup, and how it performs at speed.
Question for others on HN: What if you used many cameras in a row, each separated from its neighbour by a single stiff anchor like a chain of Mobileyes? Then perform calculations between each neighbouring camera and gradually build up the necessary correction between the two cameras at the outermost points.
Shouldn't it be able to do that much anyway? Considering potential conditions of highway speed and either heavy rain, snow or black ice... A brick is pretty big object and 150m can be quite close...
40 comments
[ 5.3 ms ] story [ 99.1 ms ] threadThe downsides of stereo are:
* range uncertainty grows quadratically with range
* range uncertainty is unbounded in the case of featureless surfaces or bad illumination
* range from stereo disparity depends on looking at very small differences between two images and may be much more adversely affected by weather (rain drops on the lens) compared to, say, lidar.
TLDR: It's stereo cameras with wide baseline and software compensation for vibrations/camera shake.
That said, I am pretty disappointed that there's no mention of the actual problems that people working with stereo cameras in cars have, which is that optical matching usually doesn't have a unique solution and that reflections ruin everything. The latter especially gets worse the wider you make the stereo baseline, so I'd bet you that NODAR will fail badly when there's ice on the road.
That combination of false correspondences due to reflections and AI in-painting to produce the illusion of a complete dataset when it is impossible to see depth everywhere, might be quite dangerous. We laugh at Skydio drones for crashing into windows, puddles, and lakes because their AI (with 6 cameras) gets confused by reflecting things. I'd find it a lot less funny if that was my car crashing.
One of the friends told me that is belief is that proper self-driving is still a long time away, because of this.
Depth perception is more complex than eye separation distance. By the above measurement, most ball sports would be nearly impossible. We all still play baseball. The reality is that we use multiple techniques for judging distance. Our heads move, increasing the separation between observations (see any owl scoping out prey before launching). Lens focus ads another flow of information. And we judge distance to distant objects using their apparent positions relative to others closer to us.
Talk to anyone who has lost an eye. They are perfectly capable of driving cars and throwing balls accurately well beyond ten meters.
I can turn it off by using normal glasses.
Minor movements reveal depth of most things.
My mom has amblyopia and is old enough that it didn’t get treated. She can drive just fine, but she can’t grab the clothes line between her fingers. She has to sort of paw at it and catch it with her palm.
And I’d say visual processing is a lot more complex then we give it credit for. We don’t understand what’s really all going on yet. We still are trying to understand microsaccades for example. Ocular processing is extremely fascinating.
This is a strawman. Nobody argues that it should not be used, only that driving is demonstrably possible without it.
Except that the appendix is a useful organ. Had it been truly useless it would have evolved away long ago, yet we find dinosaurs had them too. I think Musk is also wrong about lidar. It probably does have a place in this problem even if, like his blind spot re the appendix, Musk doesn't currently understand what that place may be.
"Evolution Of The Human Appendix: A Biological 'Remnant' No More"
https://www.sciencedaily.com/releases/2009/08/090820175901.h...
We are so committed to LIDAR being useless that we're dumping our LIDARs for cheap! Buy them and start your own Self Driving Car company!
https://twitter.com/comma_ai/status/1197323020852793344?lang...
I can find plenty more.
https://techcrunch.com/2019/04/22/anyone-relying-on-lidar-is...
I think it might be like ray tracing. It's the holy grail of graphics, which makes engineers very positive about it. Yet it is slow and expensive and most problems do perfectly fine if not better with "conventional graphics".
Lidars are expensive, fairly large, power hungry and depending on what your using them for, noisy.
Musk doesn't like them because they are expensive. Not because they don't work. He's spent billions trying to polish the turd that is their monocular depth estimation + radar system. Sure its impressive, but its nowhere near good enough for safe autonomy. (I am sceptical that they ever will with that sensor layout. Thats neatly avoiding the issue that there is no redundancy)
If you want ground truth data from a moving vehicle, then there is no real alternative to lidar[1]. Anyone who tells you otherwise is either:
1) an idiot
2) a musk fan
3) trying to sell you something.
Now, sure for "autopilot" (cruise control) where there isn't that much to do, radar and basic object detection is good enough. But any sign of danger and it backs off, leaving you no time to unfuck the situation.
[1] I work for a mapping company, There are other ways to get to <50cm accuracy point clouds, but its not fast and requires a boat load of GPU (in the order of thousands of hours per km driven)
In general, it's not yet clear to me whether LIDAR is all that useful for the hard parts of the self-driving car problem. We know it's extremely useful for the easy parts (not hitting curbs, other vehicles, or walls), but it's not clear whether LIDAR helps that much with small objects, vehicle behavior prediction, or object classification (paper bag vs. dog)
So in certain conditions LIDAR can see better than human eyes, like say complete darkness. This could lead to a car behaving differently than a human driver which might well cause an accident.
In other conditions LIDAR can see much worse than a human. Depending on the fog, snow, rain, dust, tumbleweeds etc can look more solid to a lidar than a human eye. Which can also cause accidents. A human might recognize a whisp of fog, whose outline and volume look like a fallen tree, but a camera would have a better chance.
So even with a magic LIDAR that costs $100, has good range, solid state, etc. it's not clear what LIDAR offers over a pair of cameras, or other cheap sensors like cameras, mm band (60 GHz) radar, and ultrasound.
Additionally LIDAR systems need cameras anyways. Is that a ambulance or UPS truck? Schoolbus or city bus? Police car or Camry? Does the police car have it's lights on? Did someone flash their high beams at you? Delivery truck or post office truck? Color is hugely important in predicting what's going to happen. Sure you can add cameras to LIDAR, but then why? Doubly is you have two cameras for distance.
So I'd love to be wrong, and indisputably LIDAR is getting better, but seems like pairs of cameras might well be the best fit for driving like a human. Doesn't seem like it could ever be LIDAR, but LIDAR + cameras could compete, but what does LIDAR give you over a pair of cameras?
In particular, for pedestrians, bicycles, other vehicles, maybe even children running in street, is the limited resolution enough to still substantially increase safety?
Where I got with this - hitting a brick is bad. Hitting a person is much worse. A fair number of never hit type items seem to still be picked up with Lidar.
But all good points.
Lidar's big advantage was a distance for each "pixel", but that advantage seems to mostly disappear for pairs of cameras, at least for common objects like pedestrians, bicycles, most vehicles, etc. However that does depend on edge detection or enough visual detail to be able to match an object through two different perspectives. A featureless white wall that consumes the entire camera view would result in no distance for a stereo camera, but would still work with lidar. Not sure if that's enough of a limitation to break things with say a large white 18-wheeler crossing the road.
I've heard new interest in mm wave (60 GHz or so) radar, but no idea where it falls compared to lidar and stereo cameras as far as distance and angular resolution and sample rate.
It's startling how hard it is to get a vehicle project to change anything. A non-structural but rigid pressed-steel truss that goes somewhere behind the grille of the vehicle would make this software compromise unnecessary.
I'm aware that Tesla basically took Mobileye's single-camera product off the shelf, but I wish someone in the Model S development team could have added that dual-camera structure as a requirement on Day 1.
My car's lower bumper mounted radar antenna stops working any time I drive in the snow because it quickly gets coated with a layer of snow.
Flagging this.
https://news.ycombinator.com/newsguidelines.html
I am wary of the depth map they've provided. It looks suspiciously smooth and stable. Given that roads have little to no features of use, the belief propagation must be very good (or its been tarted up in post.)
I assume they use feature matching to estimate the camera movement, and the those same features are compared from each camera to get the divergence. I would be interested to see the calibration setup, and how it performs at speed.