Interesting idea, but it seems like diffraction will be a serious problem for this. Any time you have tiny little holes (apertures) you are going to get a lot of diffraction that will result in very blurry images.
You can't have it both ways. Either the mask hole is right by the sensor, in which case there is no angular resolution. Or the mask hole is further away, in which case you get lots of diffraction.
Achievable image/angular resolution. Our current prototypes
have low spatial resolution which is attributed to two
factors. First, it is well known that angular resolution of
pinhole cameras and coded aperture cameras decreases when
the mask is moved closer to the sensor [6]. This results
in an implicit tradeoff between the achievable thickness and
the achievable resolution. "
Also the images speak for themselves - all very blurry.
It seems like this is an abstraction on the idea of a light field camera. (i.e. each hole in the mask acts as a tiny piny hole camera in the composite ...)
It looks like a Lytro camera, but going one step further. In general, the main idea is the same: using mask and then recovering the full light field.
While it looks that Lytro was/is not a great commercial success, I see that the field progresses. And very likely we will see things which can't be done in a different ways (e.g. pseudo-3D camera for videos... or 3d teleconferences).
A Lytro camera uses many small image sensors with many lenses. No mask is involved.
The conventional Lytro cameras are of pretty questionable utility, but light field cameras show some promise for capturing VR video: https://www.lytro.com/immerge
As someone who last few years got into photography, I can't wait for computation to be used more to produce better images and more innovative systems. The one for example is camera with 16 small sensors that will probably be flop, but will open doors to so many others that will come after it.
The mask kinda looks like a cosine filter rather than just a simple grid to make many pinholes. If so, then it might be a small version of heterodyne light field camera.
http://web.media.mit.edu/~raskar/Mask/
ISTM that "experts" have long derided the compound eyes of arthropods, since if they were "advanced" they would be like human eyes. Maybe developing cameras that aren't just direct copies of vertebrate eyes will enable us to perceive the benefits of other eye forms.
I would suggest 'traditional' cameras and telescopes are all about being able to trace back the origin of light. So, if light hit part if your sensor X,Y it must have come from location A,B. Pin hole cameras work just fine for this, all the lenses are just trying to collect more light without creating excessive much distortion.
Computational photography instead focuses on modeling things so if you observed A, B, and C, then Y must be true. Which enables things like CT scans to work where traditional X ray force you to look at shadows.
So - score my understanding of this. A lens performs a fourier transform, right? So the idea here is that instead of doing the transform with a lens, we grab the raw information and do an fft in software?
I'm not sure on the implementation details of this particular method, and not really qualified to score anyone's understanding of anything in this field :-) but the shape of what you say is correct - you're modelling the physics of what happens in a lens system X (X can be a range of systems of course, not necessarily one particular system, but you get the idea), and in an alternate system Y (in this case Y is no lens, but a coded aperture used to add extra contextual information to the captured light) so you can use software to do a virtualised mapping of what your light captured from the scene via system Y would have looked like if had instead passed through system X - in other words, estimating virtually what a photo of the scene would have looked like if you'd used a lens, even though you've actually got no lens. Very clever application indeed!
A lens doesn't compute the fourier transform of the image, it inverts it (geometrically speaking). On a wave level (far-field diffraction) it does something like the fourier transform.
A lens is a convolution. Imagine if a photograph was a 1D signal, then a lens would be a convolution of a signal with a square wave (Lens sliced along one plane). Now, typically when a square wave is convolved with a signal, if you look at the magnitude of the fourier transform, you'll notice that you are multiplying the frequencies of the signal with a sinc function. Sinc functions hit zero amplitudes at various points, thereby destroying information when multiplied. Therefore, if you could create a lens (known as a mask), which didn't destroy signal, but instead was more like a band-pass filter, then you could effectively recover the signal and adjust for amplitude changes digitally. Now take this model to the 3D and you've just created a lightfield.
This is pretty amazing, like 'davnicwil said. Personally though, the coolest thing about this is how easy it is to make if you have some way to make the mask.
It uses a Raspberry Pi Camera Module[1], which is $30, with no modifications other than removing the lens and adding the mask.
I wonder if you could make a mask by machining plastic down very thin, and then punching tiny holes through with the smallest endmill you could find.
Offhand, this seems like a job for a laser cutter more than a milling machine. (But hey, if that's what you've got...) Depending on the cutter, largely its minimum kerf size, a single-pass may produce an entirely acceptable result.
An alternative to direct hole cutting would be to use the laser to ablate an etching mask medium then etch the actual holes into the cut surface, which might produce a better end result. Anyone given this approach a try for high-detail work?
Replace 'endmill' with 'precision aperture laser' and the answer is yes. Back at the Optics lab at USC we used an argon laser with a precision mask/plate to may 1uM holes in a substrate at arbitrary places.
So it is a sheet of pinhole cameras? And the images are focused by selecting from various diameters of pixels behind each pinhole? It looks like a great idea, but some of the descriptions make it seem more than it is.
Similar lens-free "computational photography" techniques have been described for super-resolution microscopy (see below).
In conventional imaging, lenses resolve a sharp image of the target onto the sensor by focusing just the light emitted by the target within the lens' "focal plane" (a volume in front of the lens determined by the width of its aperture and the wavelength of the light passing through it) while blurring light from everywhere else into background noise. One can think of this system as amplifying certain information (the light from the focal plane) while filtering out other information (the light from everywhere else) from the total input light.
In other words, the images formed by lenses are lossy by design. This isn't practically a problem at macroscales, but at microscales the focal plane becomes extremely thin (micron-scale), oftentimes 1/100th as thick as the sample being imaged. To image large cells, for instance, a microscopist might be forced to capture tens or hundreds of images of the sample from different focal lengths (a "z-stack"), then deconvolve them to generate an image in which most/all of the sample is in focus.
Anyway, I think these new techniques are interesting because they fundamentally attempt to capture more of the "information" present in the incident light. The lens does not capture an "image" that can be directly visualized, but rather a more full representation of the wave-field of all the incident light. Generating a conventional image from the raw data requires sophisticated additional processing.
The Ozcan lab has published some really neat (and eminently DIYable) work in this area:
"Lensfree On-chip Tomographic Microscopy Employing Multi-angle Illumination and Pixel Super-resolution", Serhan O. Isikman, Waheb Bishara, and Aydogan Ozcan. J Vis Exp. 2012; (66): 4161. doi: 10.3791/4161. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3487288/
"Existing 3D optical imagers generally have relatively bulky and complex architectures, limiting the availability of these equipments to advanced laboratories, and impeding their integration with lab-on-a-chip platforms and microfluidic chips. To provide an alternative tomographic microscope, we recently developed lensfree optical tomography (LOT) as a high-throughput, compact and cost-effective optical tomography modality. 7 LOT discards the use of lenses and bulky optical components, and instead relies on multi-angle illumination and digital computation to achieve depth-resolved imaging of micro-objects over a large imaging volume. LOT can image biological specimen at a spatial resolution of <1 μm x <1 μm x <3 μm in the x, y and z dimensions, respectively, over a large imaging volume of 15-100 mm3, and can be particularly useful for lab-on-a-chip platforms."
31 comments
[ 3.7 ms ] story [ 86.8 ms ] threadEdit: Man, I guess I should have read the paper!
...
Achievable image/angular resolution. Our current prototypes have low spatial resolution which is attributed to two factors. First, it is well known that angular resolution of pinhole cameras and coded aperture cameras decreases when the mask is moved closer to the sensor [6]. This results in an implicit tradeoff between the achievable thickness and the achievable resolution. "
Also the images speak for themselves - all very blurry.
edit: fix typo
Edit: Example images are shown in the included video.
While it looks that Lytro was/is not a great commercial success, I see that the field progresses. And very likely we will see things which can't be done in a different ways (e.g. pseudo-3D camera for videos... or 3d teleconferences).
The conventional Lytro cameras are of pretty questionable utility, but light field cameras show some promise for capturing VR video: https://www.lytro.com/immerge
I personally can't wait.
Computational photography instead focuses on modeling things so if you observed A, B, and C, then Y must be true. Which enables things like CT scans to work where traditional X ray force you to look at shadows.
It uses a Raspberry Pi Camera Module[1], which is $30, with no modifications other than removing the lens and adding the mask.
I wonder if you could make a mask by machining plastic down very thin, and then punching tiny holes through with the smallest endmill you could find.
[1]: https://www.raspberrypi.org/products/camera-module/
An alternative to direct hole cutting would be to use the laser to ablate an etching mask medium then etch the actual holes into the cut surface, which might produce a better end result. Anyone given this approach a try for high-detail work?
After reading the pattern, it seems to be more than just holes, though, so it's probably a bit harder to fabricate.
Uh oh.
Also... it's just amazing that a college lab can afford to fabricate these prototypes.
In conventional imaging, lenses resolve a sharp image of the target onto the sensor by focusing just the light emitted by the target within the lens' "focal plane" (a volume in front of the lens determined by the width of its aperture and the wavelength of the light passing through it) while blurring light from everywhere else into background noise. One can think of this system as amplifying certain information (the light from the focal plane) while filtering out other information (the light from everywhere else) from the total input light.
In other words, the images formed by lenses are lossy by design. This isn't practically a problem at macroscales, but at microscales the focal plane becomes extremely thin (micron-scale), oftentimes 1/100th as thick as the sample being imaged. To image large cells, for instance, a microscopist might be forced to capture tens or hundreds of images of the sample from different focal lengths (a "z-stack"), then deconvolve them to generate an image in which most/all of the sample is in focus.
Anyway, I think these new techniques are interesting because they fundamentally attempt to capture more of the "information" present in the incident light. The lens does not capture an "image" that can be directly visualized, but rather a more full representation of the wave-field of all the incident light. Generating a conventional image from the raw data requires sophisticated additional processing.
The Ozcan lab has published some really neat (and eminently DIYable) work in this area:
"Lensfree On-chip Tomographic Microscopy Employing Multi-angle Illumination and Pixel Super-resolution", Serhan O. Isikman, Waheb Bishara, and Aydogan Ozcan. J Vis Exp. 2012; (66): 4161. doi: 10.3791/4161. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3487288/
"Existing 3D optical imagers generally have relatively bulky and complex architectures, limiting the availability of these equipments to advanced laboratories, and impeding their integration with lab-on-a-chip platforms and microfluidic chips. To provide an alternative tomographic microscope, we recently developed lensfree optical tomography (LOT) as a high-throughput, compact and cost-effective optical tomography modality. 7 LOT discards the use of lenses and bulky optical components, and instead relies on multi-angle illumination and digital computation to achieve depth-resolved imaging of micro-objects over a large imaging volume. LOT can image biological specimen at a spatial resolution of <1 μm x <1 μm x <3 μm in the x, y and z dimensions, respectively, over a large imaging volume of 15-100 mm3, and can be particularly useful for lab-on-a-chip platforms."
Also see http://spie.org/newsroom/technical-articles-archive/3979-mul...