A lightfield is a way of recording an image with ray angle information as well
This means that you know where the light has come from, which allows your to amongst other things alter the focus/depth of field after the image had been captured.
Because you can refocus and change the depth of field post capture, it allows you in theory to do better image segmentation.
This is very interesting. Segmentation is a very, very hard problem and the video indicates they have made it fast. 4000 images is a lot and at 21GB per model the processing overhead has to be massive. 60-100 minutes is also a pretty long time to recover one model, but not crazy, and will certainly get faster.
I don't see any scale reference markers or other reference material in the scene, so I wonder if it recovers scale from light field depth information alone.
Interestingly from the paper, they aren't actually using a light field camera - but rather creating a light volume through sequentially varying the depth of image taken and modeling the volume:
Key to our method is the dense spatio-angular sampling of video, which results in smoothly varying parallax between successive frames
A hack for sure!
I would be curious to know if using a light field camera would even be useful in this scenario as much of what they do mimics the light field camera processes, only manually.
THIS IS HUGE. They are creating/calculating the missing "dimension" based on statistical input. That is amazing. Physics experiments are all about trying to understand the unknown dimension(s) as well! This D to D+1 dimension transformation is mathematically likely to be solvable for any D value that is an integer too.
8 comments
[ 3.4 ms ] story [ 27.0 ms ] threadHopefully lightfield cameras / camera arrays will become more common in the near future.
This means that you know where the light has come from, which allows your to amongst other things alter the focus/depth of field after the image had been captured.
Because you can refocus and change the depth of field post capture, it allows you in theory to do better image segmentation.
I don't see any scale reference markers or other reference material in the scene, so I wonder if it recovers scale from light field depth information alone.
Interestingly from the paper, they aren't actually using a light field camera - but rather creating a light volume through sequentially varying the depth of image taken and modeling the volume:
Key to our method is the dense spatio-angular sampling of video, which results in smoothly varying parallax between successive frames
A hack for sure!
I would be curious to know if using a light field camera would even be useful in this scenario as much of what they do mimics the light field camera processes, only manually.