I saw a Waymo in Seattle, today. If Waymo can get Seattle right, that gives me a lot of confidence that their stack is very capable of difficult road conditions.
Note: I have not had the pleasure of riding in one yet, but from what my friend in SJ says, it’s very convenient and confidence-inspiring.
Seattle probably isn't any harder than SF, other than the occasional weather event where the hills ice over and we get a bunch of funny (and scary) videos.
@dang .... do these comments seem organic to you? old accounts with almost zero karma going out of their way to use the same verbiage to compliment waymo 18 minutes after an article gets posted? .... dead internet at work.
I'm not well versed into RF physics. I had the feeling that light-wave coherency in lasers had to be created at a single source (or amplified as it passes by). That's the first time I hear about phased-array lasers.
I think about it like a series of waves in a pool. One end has wave generators (the lasers) spaced appropriately such that resulting waves hitting the other end interfere just right and create a unified wavefront (same phase, amplitude, frequency).
Interesting to see the cost curve drop ... this always changes the market.
I have been watching the sensor space for a while. Cheap LIDAR units could open up weird DIY uses and not just cars. ALSO regulatory and mapping integration will matter. I tried to work with public datasets and it's messy. The hardware is only one part! BUT it's exciting to see multiple vendors in the space. Competition might push vendors to refine the software stack as well as the hardware. HOWEVER I'm keeping an eye on how these systems handle edge cases in bad weather. I don't think we have seen enough data yet...
The mind salivates at the idea of sub-$100 and soon after sub-$10 Lidar. We could build spatial awareness into damn near everything. It'll be a cambrian explosion of autonomous robots.
The short-range stuff is already $150-300 per unit. If you're thinking indoor robots that's already technically feasible. Over 25% of all Chinese cars being produced today have LiDAR.
Even mid-range sensors used in ADAS systems only cost $600-750. The long-range stuff that's needed for trucking or robotaxis is $1,500–6,000
Interestingly, there have been people in the LIDAR industry predicting costs like this for many years. I heard numbers like $250 per vehicle back in 2012 [1]
Of course, ambitious pricing like this is all about economies of scale - sensors that are used in production vehicles are ordered by the million, and that lowers the costs massively. When the huge orders didn't materialise, the economies of scale and low prices didn't materialise either.
Well he's also argued that just using CV reduces sensor contention and he claims it improves performance and release velocity, which is why they also got rid of radar and ultrasonic sensors. I am doubtful although it'll be interesting to see regardless
Is this Human safe at these volumes? There was a time you could get your feet sized by putting them into an X-ray box at the shoe store. Removed from stores once the harm was known.
Well, the energy levels used in those devices should be miniscule, and the wavelengths used are well studies. The problem with x-rays - was lack of studies on health effects, and regulations on those effects. I think, since that time, we've studies radiation (be it light, rf or other parts of spectrum) much more.
There is indeed a possibility that we're overlooking some bio-electromagnetic interaction effects; for instance now there is some evidence that led lights might not be harmless - but again, it's not the they affect biological structures somehow, but the lack of spectral components has some effects. It is an interesting topic to research. But, the lidar "should" be safe
There are laser measurers sold for a few buck on Temu. Robot vacuums sold for few hundred dollars have Lidars that map out the room in a seconds.
Is there any actual technical reason why automobile Lidar be expensive? Just combine visual processing with single point sampler that will feed points of interest and accurate model of the surroundings will be built.
Most spinning robovac LIDARs are 2D. Most solid state robovac LIDARs are like 8x8 array of laser pointers.
Automotive LIDARs are like, 128x64[px] for production models or 1920x1080[px] for experimental models with GbE and/or HDMI-equivalents-of-industry outputs. Totally different technologies.
Oh my god so many reasons. I don't feel like getting fully into it but that's kind of like asking why you can't use your kitchen scale to measure highway traffic as it drives over it.
Since lidar has distance information and cameras do not, it was always a ridiculous idea by a certain company to use cameras only. Lidar using cars are going to replace at least the ones that don't make use of this obvious answer to obstacle detection challenges.
The reasoning is cynical but sound. If the system uses only the sensing modes people have, it will make the mistakes people do. If a jury thinks "well I could have done that either!" You win. It doesn't matter if your system has fewer accidents if some of the failure modes are different than human ones, because the jury will think "how could it not figure that out?"
> If a jury thinks "well I could have done that either!" You win
“A federal judge” recently “rejected Tesla's request to overturn a $243 million jury verdict over the 2019 crash of an Autopilot-equipped Model S” [1]. If a human supervising still incurs liability, human-like errors, particularly if Waymo and BYD aren’t making them, is a poor defense.
Yea, even in the case they could match human level stereo depth perception with AI, why would they say "no" to superhuman lidar capabilities. Cost could be a somewhat acceptable answer if there wouldn't be problems with the camera only approach but there are still examples of silly failures of it.
And if I remember correctly they also removed their other superhuman radar in their newer models, the one which in certain conditions was capable of sensing multiple cars ahead by bouncing the signal below other cars.
> Since lidar has distance information and cameras do not, it was always a ridiculous idea by a certain company to use cameras only
Human eyes do not have distance information, either, but derive it well enough from spatial (by ‘comparing’ inputs from 2 eyes) or temporal parallax (by ‘comparing’ inputs from one eye at different points in time) to drive cars.
One can also argue that detecting absolute distance isn’t necessary to drive a car. Time to-contact may be more useful. Even only detecting “change in bearing” can be sufficient to avoid collision (https://eoceanic.com/sailing/tips/27/179/how_to_tell_if_you_...)
Having said that, LiDAR works better than vision in mild fog, and if it’s possible to add a decent absolute distance sensor for little extra cost, why wouldn’t you?
As I understand, lidars don't work well in rain/snow/fog. So in the real world, where you have limited resources (research and production investment, people talent, AI training time and dataset breadth, power consumption) that you could redistribute between two systems (vision and lidar), but one of the systems would contradict the other in dangerous driving conditions — it's smarter to just max out vision and ignore lidar altogether.
considering cameras can create reliable enough distance measurements AND also handle all the color reception needed for legally driving roads it was always a ridiculous idea by a certain set of people that lidar is necessary.
I'll preface by saying lidar should be used with autonomous vehicles.
Individual cameras don't have distance information, but you can easily calibrate a system of cameras to give you distance information. Your eyes do this already, albeit not quantitatively. The quantitative part comes from math our brains aren't setup to do in real time.
I'm not an expert on ML vision, but I do have a Tesla and it seems to be able to tell how far away things are just fine. I'm not sure what would be wrong with the vision system that lidar needs to fix.
Why make things more complicated than they need to be? Humans don't have lidar and we are the only intelligence that can reliably drive. Lidar just seems like feature engineering, which has proven to be a dead end in most other AI applications (bitter lesson).
I wouldn’t take too much issue with the “cameras are enough” claim if cameras actually performed like eyes. Human eyes have high dynamic range and continuous autofocus performance that no camera can match. They also have lids with eyelashes that can dynamically block light and assist with aperture adjustment.
The appeal to human biology and argument against fusion between disparate sensors kinda falls flat when you’re building a world model by fusing feeds from cameras all around the car. Humans don’t have 8 eyes in a 360 array around their head. What they do have is two eyes (super cameras) on ~180 degree swiveling and ~180 degree tilting gimbal. With mics attached that help sense other vehicles via road noise. And equilibrioception, vibration detection, and more all in the same system, all fused. If someone were actually building this system to drive the car, the argument based on “how did you drive here today?” gets a lot stronger. One time I had some water blocking my ear and I drove myself to the hospital to get it fixed. That was a shockingly scary drive — your hearing is doing a lot of sensing while driving that you don’t value until it’s gone.
I find it comical that people continue to go back to this rage well against "a certain company" for their vision-only approach when the truth is they have the best automatic driving system an individual can buy, rivaling Waymo and beating the Chinese brands.
Why are the commenters not pissed at the dozens of other car companies who have done absolutely nothing in this space? Answer: because it's not nearly as fun to be pissed at Kia or Mercedes or whoever. Clearly they are just enjoying the shared anger, regardless of whether it is justified.
Karpathy’s main points:
Extra sensors add cost to the system, and more importantly complexity. They make the software task harder, and increase the cost of all the data pipelines. They add risk and complexity to the supply chain and manufacturing.
Elon Musk pushes a philosophy of “the best part is no part” which can be seen throughout the car in things like doing everything through the touchscreen. This is an expression of this philosophy.
Vision is necessary to the task (which almost all agree on) and it should also be sufficient as well. If it is sufficient, the cost of extra sensors and tools outweighs their benefit.
Sensors change as parts change or become available and unavailable. They must be maintained and software adapted to these changes. They must also be calibrated to make fusion work properly.
Having a fleet gathering more data is more important than having more sensors.
Having to process LIDAR and radar produces a lot of bloat in the code and data pipelines. He predicts other companies will also drop these sensors in time.
Mapping the world and keeping it up to date is much too expensive. You won’t change the world with this limitation, you need to focus on vision which is the most important. The roads are designed to be interpreted with vision.
One camera can't really produce depth/distance information, but two cameras sure can. The eyes in your head don't capture distance information individually, but with two eyes you can infer distance.
It's not that simple. Cameras don't report 3D depth, but these AI models can and do pick up on pictorial depth cues. LiDAR is incredibly valuable for collecting training and validation data, but may also make only an insignificant difference in production inference.
Certain company has 300k subscribers that rely on that ridiculous service.
My father lost vision in 1 eye and 50% in other one something like 20 years ago. He struggles in parking but otherwise doing ok without lidar. Turns out motion vision is more accurate after 10-20 meters than stereoscopic vision.
There are more practical difficulties than just cost. If you have lidar it must be calibrated relative to all other sensors. Bumps in the road, weather, thermals, this all causes drift which is non trivial. Waymos are constantly brought in and recalibrated. The advantage of camera only is you have less moving parts which is not insignificant.
Below is one of the comments poster to original article, reading it makes me think that most of the whole article has been regurgitated by some AI:
>"This misleading article contains numerous factual errors regarding automotive lidar. Here are the most glaring:
There are multiple manufacturers, including Hesai, that use mechanical means for at least one scan axis and are already sold for a fraction of the "$10k - $20k" price noted by the author. Luminar itself built this class of scanners before going bankrupt.
Per Microvision's own website, the Movia-S does not use a phased array and also does not have a range anywhere near 200m.
Velodyne and Luminar do not even exist as companies anymore. Both have gone bankrupt and been acquired by competitors."
82 comments
[ 2.9 ms ] story [ 64.3 ms ] threadNote: I have not had the pleasure of riding in one yet, but from what my friend in SJ says, it’s very convenient and confidence-inspiring.
> phased-array
I'm not well versed into RF physics. I had the feeling that light-wave coherency in lasers had to be created at a single source (or amplified as it passes by). That's the first time I hear about phased-array lasers.
Can someone knowledgeable chime in on this?
NB: just my layman's understanding
I have been watching the sensor space for a while. Cheap LIDAR units could open up weird DIY uses and not just cars. ALSO regulatory and mapping integration will matter. I tried to work with public datasets and it's messy. The hardware is only one part! BUT it's exciting to see multiple vendors in the space. Competition might push vendors to refine the software stack as well as the hardware. HOWEVER I'm keeping an eye on how these systems handle edge cases in bad weather. I don't think we have seen enough data yet...
Even mid-range sensors used in ADAS systems only cost $600-750. The long-range stuff that's needed for trucking or robotaxis is $1,500–6,000
Of course, ambitious pricing like this is all about economies of scale - sensors that are used in production vehicles are ordered by the million, and that lowers the costs massively. When the huge orders didn't materialise, the economies of scale and low prices didn't materialise either.
[1] https://web.archive.org/web/20161013165833/http://content.us...
Glad to see someone lowering the cost of this technology, and hope to see lots of engineers using this tech as a result.
We might even see a boom in LIDAR tech as a result
Is there any actual technical reason why automobile Lidar be expensive? Just combine visual processing with single point sampler that will feed points of interest and accurate model of the surroundings will be built.
Automotive LIDARs are like, 128x64[px] for production models or 1920x1080[px] for experimental models with GbE and/or HDMI-equivalents-of-industry outputs. Totally different technologies.
“A federal judge” recently “rejected Tesla's request to overturn a $243 million jury verdict over the 2019 crash of an Autopilot-equipped Model S” [1]. If a human supervising still incurs liability, human-like errors, particularly if Waymo and BYD aren’t making them, is a poor defense.
[1] https://www.reuters.com/world/us-judge-upholds-243-million-v...
Human eyes do not have distance information, either, but derive it well enough from spatial (by ‘comparing’ inputs from 2 eyes) or temporal parallax (by ‘comparing’ inputs from one eye at different points in time) to drive cars.
One can also argue that detecting absolute distance isn’t necessary to drive a car. Time to-contact may be more useful. Even only detecting “change in bearing” can be sufficient to avoid collision (https://eoceanic.com/sailing/tips/27/179/how_to_tell_if_you_...)
Having said that, LiDAR works better than vision in mild fog, and if it’s possible to add a decent absolute distance sensor for little extra cost, why wouldn’t you?
If this lowers Lidar costs, and Tesla has spent all this time refining the camara technology. Now have both.
Use both.
Individual cameras don't have distance information, but you can easily calibrate a system of cameras to give you distance information. Your eyes do this already, albeit not quantitatively. The quantitative part comes from math our brains aren't setup to do in real time.
Well, you did get a chuckle out of me, so that's something!
https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson...
The appeal to human biology and argument against fusion between disparate sensors kinda falls flat when you’re building a world model by fusing feeds from cameras all around the car. Humans don’t have 8 eyes in a 360 array around their head. What they do have is two eyes (super cameras) on ~180 degree swiveling and ~180 degree tilting gimbal. With mics attached that help sense other vehicles via road noise. And equilibrioception, vibration detection, and more all in the same system, all fused. If someone were actually building this system to drive the car, the argument based on “how did you drive here today?” gets a lot stronger. One time I had some water blocking my ear and I drove myself to the hospital to get it fixed. That was a shockingly scary drive — your hearing is doing a lot of sensing while driving that you don’t value until it’s gone.
Why are the commenters not pissed at the dozens of other car companies who have done absolutely nothing in this space? Answer: because it's not nearly as fun to be pissed at Kia or Mercedes or whoever. Clearly they are just enjoying the shared anger, regardless of whether it is justified.
https://archive.is/PPiVG
And here's one of Elon's mentions (he also has talked about it quite a bit in various spots).
https://xcancel.com/elonmusk/status/1959831831668228450?s=20
Edit: My personal view is that LiDAR and other sensors are extremely useful, but I worked on aircraft, not cars.
Karpathy’s main points: Extra sensors add cost to the system, and more importantly complexity. They make the software task harder, and increase the cost of all the data pipelines. They add risk and complexity to the supply chain and manufacturing. Elon Musk pushes a philosophy of “the best part is no part” which can be seen throughout the car in things like doing everything through the touchscreen. This is an expression of this philosophy. Vision is necessary to the task (which almost all agree on) and it should also be sufficient as well. If it is sufficient, the cost of extra sensors and tools outweighs their benefit. Sensors change as parts change or become available and unavailable. They must be maintained and software adapted to these changes. They must also be calibrated to make fusion work properly. Having a fleet gathering more data is more important than having more sensors. Having to process LIDAR and radar produces a lot of bloat in the code and data pipelines. He predicts other companies will also drop these sensors in time. Mapping the world and keeping it up to date is much too expensive. You won’t change the world with this limitation, you need to focus on vision which is the most important. The roads are designed to be interpreted with vision.
My father lost vision in 1 eye and 50% in other one something like 20 years ago. He struggles in parking but otherwise doing ok without lidar. Turns out motion vision is more accurate after 10-20 meters than stereoscopic vision.
But cost isnt the issue as much.
>"This misleading article contains numerous factual errors regarding automotive lidar. Here are the most glaring:
There are multiple manufacturers, including Hesai, that use mechanical means for at least one scan axis and are already sold for a fraction of the "$10k - $20k" price noted by the author. Luminar itself built this class of scanners before going bankrupt.
Per Microvision's own website, the Movia-S does not use a phased array and also does not have a range anywhere near 200m.
Velodyne and Luminar do not even exist as companies anymore. Both have gone bankrupt and been acquired by competitors."
(Insert old man rant “Why are everyone’s headlights so gosh darn bright these days?!”)