HN Explorer

The Lytro Camera: How it works (without marketing)

7 points by havemurci ↗ HN
The Lytro Camera's engineering (and especially marketing) teams freely use technical jargon to describe their invention. However, what they do can be distilled into much simpler terms. So after reading through the founder's Stanford thesis, and other online resources, here's my take on how the Lytro works.

Just like any camera, the Lytro has an optical lens that focuses to any distance. It has a relatively large aperture of f/2.0, which allows for a relatively shallow depth of field.

The only bit of hardware that's special or unique about a light-field camera is its 'micro-lens' array. The array is a repeated pattern of 3x3 matrices. Each of these 9 different microlenses focuses the light slightly closer or slightly farther away than the others. When you take a picture, the camera records data on the sensor that is then processed into exactly 9 pictures, each corresponding to a slightly different focal length (and therefore a slightly different band of the photo that’s in focus).

The software then uses contrast detection to make a 20x20 sub-matrix indicating which of the nine images is in focus at the chosen point in the image. When you click on a point on a ‘living image’, it looks up that point and loads the image for which that point is most in focus.

The Lytro contains an 11 megapixel sensor. But because it takes 9 photos at once, the effective resolution of the final photograph is 1080x1080 pixels.

It’s not a very complicated design: A microlens array that adjusts the focus on a small scale to produce 9 different images, each with a slightly different range of focus.

Thinking about ‘rays’ of light isn’t necessary to understand how it works.

Here’s a video of the founder getting tripped up when a reporter pinpoints how his technology works (1:05)

http://video.forbes.com/fvn/sxsw-2012/eric-cheng-lytro-lightfield-camera

6 comments

[ 2.9 ms ] story [ 21.5 ms ] thread
havemurci
edit: The person interviewed isn't the founder; he's the Director of Photography.
69_years_and
Well that makes easy to understand - thanks.
kakali
You have it slightly wrong. You make it sound like there are only 9 focus points. When in fact it can post process focus at any point. It just needs to bundle the pixels (rays) that coincide with a synthetic lens that is focused at that point. The math is easy ... but it is still very important to think of the rays through a thin lens.

It might be helpful to watch this: http://www.youtube.com/watch?v=9H7yx31yslM

You might also want to read about the heterodyning camera. It does the same thing as a plenoptic camera but with a large aperture and a cosine filter.

kyboren
Exactly, just like a synthetic aperture radar.
echeng
This isn't just slightly wrong—it's totally wrong. All you have to do is look at a tear down to see that there are far more than 9 microlenses (which do not correspond to different focal lengths). I highly recommend that you read Ren's dissertation again before you speak authoritatively on the subject again. I'm not sure how you could read it and come to the conclusions in this post.

Also, I'm the person in the video you posted. Please let me how I "tripped up."

havemurci
Hi Eric,

In the interview you didn't answer the question posed. Could you please answer it now?

"A typical digital camera captures one image. Isn't what you're doing, in a sense, capturing a thousand different images with each image?"

Am I wrong in assuming that there is a matrix of 9 images which is repeated across the sensor? It was my impression that each microlens was superimposed over a 10x10 array of pixels, such that the entire matrix covered a 30x30 pixel area, and there are 108x108 microlens arrays, for a total of 104,976 microlenses?

I admit that I was wrong in my choice of words concerning focal length. I meant that within each matrix of 9 microlenses, one of them doesn't bend the light, while 4 of them bring items closer than the unbent light into focus, and the other 4 bring farther items into focus.

With that, your software creates 9 different images that focus on light coming from objects 9 different distances from your camera. Is that correct?

I never said there were only 9 microlenses. In my original post I said there was a repeated array of 9 microlenses. Each type of microlens is processed into one 'layer'. Is that wrong?