"Our work leads to a disruptive technology that can accelerate the Fourth Industrial Revolution (Industry 4.0) with HADAR-based autonomous navigation and human–robot social interactions."
These industrial revolutions are coming faster and faster, and somehow feel less transformative with each unveiling. I only know arguably two. I have no idea what the third, let alone the fourth was.
First: powered tools, mechanised factories, basic but modern chemical processes.
Second: mass production, interchangeable parts, reliable steel, telegraph and other basic uses of electricity.
Third: computers and everything related to them.
Fourth: all the buzzwords and not much substantial at this time — though when the dust settles, I won't be surprised if at least a few currently popular things are still seen as relevant and not merely flash-in-the-pan cultural artefacts like the 18th Amendment, patent medicine, or Spiritualism.
I don't have access, what's the gist? Heat emission then timing combined with thermal imaging? AFAIK, most *AR methods are active and rely on timing, but the abstract mentions passive detection.
I can't tell what they are doing. It seems like it's merging visible and thermal imaging to improve segmentation and identification, then looking up the identified objects for ranging. No?
The thermal images in their examples seem to be intentionally using a wide temperature gamut to reduce the contrast in the image (and make it look bad). Most thermal imaging solutions will automatically adjust and give you a much better result.
Regardless I can see lots of benefits of integrating the two as basically another color channel. There's a lot of information available.
The short of it is stereovision using hyperspectral thermal cameras, but the long answer is a bit more interesting.
The problem with doing stereoimaging with thermal radiation is that almost all objects are glowing due to radiation emission which blurs out the 3d texture of the object. The example they give is if you have a lighbulb with some texture, you can't see it when it's on(you might experiment with this in various 3d engines). This makes it difficult to do stereoimaging because everything is so blurry.
To do this they develop a way to deconvolute the temperature, emissivity, and light reflected by the object(they call this texture). They assume that the much of the reflected light comes from a couple objects in the scene with known properties such as the sky and ground which allows them to get an estimate of surface normal and do something like inverse ray tracing to estimate the light reflected by the object.
They also estimate the emissivity by comparing the spectra to a library of spectra for different materials. I think they use this as part a second step in their inverse ray tracing to work out reflected light and temperature of objects. Much of this is detailed in the supplementary information[0].
It's interesting because they have a passive sensor for depth which works at night, but also estimates what objects are made of. It may degrade when there are fires which may break some of the assumptions they make to estimate reflection or spills of IR absorbing substances. Gasoline, CO2, methane, and other gases can strongly absorb in the infrared spectrum. Which may interfere with temperature and material estimates.
Most IR cameras work with IR as heat emission. They are useful in the dark, without revealing the camera.
Most optical cameras work with reflected light, not direct emission. The source of light is usually away and at an angle; this allows for shadows, textures, etc.
I don't see why an IR camera can't work the same way as an optical camera, if reflected IR is used; IR photos under e.g. sunlight look fine enough.
Radars or lidars work with their own scanning sources of illumination: microwave, IR, or optical. I wonder if the HADAR uses its own source of IR; if it does, how different it from a LIDAR?
(All these mysteries can be solved for $29, but I'm not yet curious enough.)
Yeah go play with an LWIR camera of you really want to understand the spectrum. For instance, everything becomes a mirror, you can see yourself standing in front of surfaces that have better than ~5um surface finish (quite rough sheet metal and such) and clouds appear hot-ish because while they are themselves quite cold, they're reflecting LWIR from the ground, hence why overcast nights are much warmer.
As a note, try to avoid using IR to describe LWIR, there are many IR sub bands (NIR, SWIR, MWIR, LWIR) and your phone camera can detect at least one of them.
I am very disappointed that unlike radar, lidar, and sonar, this "HADAR" technology is not an active-sensing technology, but just relies on passive IR emissions. I was imagining a car driving around with a big open flame at the front like something out of Mad Max.
All I'm saying is that if we're going to have killer robot cars roaming the streets and interfering with firefighters [1], we might as well do it properly.
IR floodlights mounted to the front of trucks is a thing. Here's an example with photos of infrared lights used on a smuggling vehicle so the driver could turn the headlights off and drive by night vision: https://youtu.be/a8Q9Yibblbc?t=626
Just because nobody made me sign an NDA for this: there are satellite companies using this for finding other satellites in eclipse. It's a pretty brilliant strategy if you want to dock with something and there's no usable visible light. You can make markings that appear clearly in LWIR by making dark parts shiny (see mostly empty space reflected) and light parts matte (see the thermal emissions of the spacecraft)
To me, "ranging" means measuring the range to the target directly, not estimating it from other measurement types (bearings, for example). Can someone explain if that's what the authors are doing here? Is it analogous to stadiametric ranging? I haven't read through this in depth, but I can't figure out what their method is.
29 comments
[ 2.6 ms ] story [ 69.0 ms ] threadReally, though?
Second: mass production, interchangeable parts, reliable steel, telegraph and other basic uses of electricity.
Third: computers and everything related to them.
Fourth: all the buzzwords and not much substantial at this time — though when the dust settles, I won't be surprised if at least a few currently popular things are still seen as relevant and not merely flash-in-the-pan cultural artefacts like the 18th Amendment, patent medicine, or Spiritualism.
It doesn't belong in academic work, no matter how impressive the authors think it may be.
The thermal images in their examples seem to be intentionally using a wide temperature gamut to reduce the contrast in the image (and make it look bad). Most thermal imaging solutions will automatically adjust and give you a much better result.
Regardless I can see lots of benefits of integrating the two as basically another color channel. There's a lot of information available.
The problem with doing stereoimaging with thermal radiation is that almost all objects are glowing due to radiation emission which blurs out the 3d texture of the object. The example they give is if you have a lighbulb with some texture, you can't see it when it's on(you might experiment with this in various 3d engines). This makes it difficult to do stereoimaging because everything is so blurry.
To do this they develop a way to deconvolute the temperature, emissivity, and light reflected by the object(they call this texture). They assume that the much of the reflected light comes from a couple objects in the scene with known properties such as the sky and ground which allows them to get an estimate of surface normal and do something like inverse ray tracing to estimate the light reflected by the object.
They also estimate the emissivity by comparing the spectra to a library of spectra for different materials. I think they use this as part a second step in their inverse ray tracing to work out reflected light and temperature of objects. Much of this is detailed in the supplementary information[0].
It's interesting because they have a passive sensor for depth which works at night, but also estimates what objects are made of. It may degrade when there are fires which may break some of the assumptions they make to estimate reflection or spills of IR absorbing substances. Gasoline, CO2, methane, and other gases can strongly absorb in the infrared spectrum. Which may interfere with temperature and material estimates.
[0]https://www.nature.com/articles/s41586-023-06174-6#Sec24
Most optical cameras work with reflected light, not direct emission. The source of light is usually away and at an angle; this allows for shadows, textures, etc.
I don't see why an IR camera can't work the same way as an optical camera, if reflected IR is used; IR photos under e.g. sunlight look fine enough.
Radars or lidars work with their own scanning sources of illumination: microwave, IR, or optical. I wonder if the HADAR uses its own source of IR; if it does, how different it from a LIDAR?
(All these mysteries can be solved for $29, but I'm not yet curious enough.)
- They use and combine both optical and LWIR data.
- They are able to split the IR signal into emissive and reflected, and thus measure the temperature and extract small variations (texture).
- They can get higher-res images using LWIR, and thus do (more) precise ranging using a stereo pair.
- Using the spectral information + some ML models, they are able to identify materials, and paint / augment pictures using this information.
That's actually impressive.
https://github.com/FanglinBao/HADAR
and videos
https://purdue0-my.sharepoint.com/personal/baof_purdue_edu/_...
[1] https://www.theguardian.com/us-news/2023/jun/23/self-driving...