the google lightfield display app for steamvr was absolutely mindblowing, and i remember at the time thinking how awesome it would be if somehow they managed to do it with video, and if maybe someday the rendering algorithm/bandwidth available would allow it to be livestreamed. Turns out that day is... now lol.
Honestly, if you have a VR rig give their demo a shot. I can honestly say this is the one project I've seen where I would hands down quit my job immediately to become involved with it somehow if I had the option. It is abundantly clear to me that the future of media consumption and production will go in this direction. Insanely cool.
They could probably cut it down significantly by only streaming a several-inch-sized sphere of free movement, predicting what segment to stream next. There is a limitation to how fast human beings can move/rotate their heads.
On a flatscreen I would say it's just a gimmick. With a VR headset though, since you are constantly moving your head a little I would say it's a massive improvement over 180° 3D videos, and really helps with the feeling of "being there".
20-60x the bandwidth requirements is one of the things that can reasonably be expected to improve with effort. Look at 2D video and how much better h264 is than, say, GIF. And the higher the dimensionality, the more data but also the more low-hanging fruit for compression. We had 100:1 lossy lightfield compression 20 years ago; we can likely put another zero on that, maybe more with machine learning.
That's what the linked paper is about - effective representations for lightfield data. The field is just kicking off.
Impressive. I'm guessing it's not long until this is used for live sporting events and concerts with virtual tickets available for those with VR headsets.
It would still be possible though - it would be more like older movies, where you had to nail a shot in the take, post production was mostly cutting.
Cutting would still be more resource intensive anyways for this new tech, because you would have to evaluate if the scene works from all plausible angles.
They did mention that they are compression the image data with H.265, so depending on the quality level, low-contrast areas might get smeared out. Also, there are some depth resolving errors here and there, which might be solved by using better algorithms. Or this might be an artifact from compressing the layer geometry (they used the Draco library, but did not specify in the paper what parameters they used). The paper mentions some of the limitations they encountered.
19 comments
[ 4.7 ms ] story [ 48.6 ms ] threadHonestly, if you have a VR rig give their demo a shot. I can honestly say this is the one project I've seen where I would hands down quit my job immediately to become involved with it somehow if I had the option. It is abundantly clear to me that the future of media consumption and production will go in this direction. Insanely cool.
That's what the linked paper is about - effective representations for lightfield data. The field is just kicking off.
If they get that working with video, it's absolutely worth it.
And yes, I wholeheartedly recommend giving it a try.
Cutting would still be more resource intensive anyways for this new tech, because you would have to evaluate if the scene works from all plausible angles.
It almost feels like there's an eyelash in my eye...