A good friend of mine at Google is the technical lead of this project (he has a Phd Princeton, and was a professor before joining Google).
I've tried it in person and it was truly amazing. They used some very fancy tech when I saw the demo, so I'm thrilled it is finally being announced and possibly shared with a larger audience.
Explanation of why it is amazing: It totally fools your perception. No glasses or goggles--but rather an 8k display with special glass that allows your different eyes to see different pixels (a light field).
They also optimize the sound, and the rest, so as all the testimonials point out, you actually feel like the person is in front of you.
It also works for the "cube" around them, so if they hold up some object, it also feels like that is in front of you.
It's neat and I'm waiting to see what creative things people come up with for it. A Japanese developer put together a Wizardry/Megaten-style dungeon-crawler demo:
I recall there being glasses-less 3D displays about 10 years ago, when the TV industry was trying to make 3D displays the next new thing. I wonder if this is the same technology.
Do you know if it supports multiple people on "screen" at once? Or does it rely on eye tracking of a single person (plus projection of some sort?) to be able to achieve the 3D effect?
I don't know anything more than what's been released, but from my understanding, light field display don't rely on eye tracking, and you can have multiple people looking from different angles (although the range may be limited to near the center). In the video itself, the camera filming the interaction is moving independently of the person sitting, and unless it was faked, it was able to see its own perspective.
i got curious (since you said you'd seen this years ago) and found my way to your cppcon 2016 talk. good talk but in particular i want to congratulate you on the weaving pun (first multithreaded tech).
Depends on how much bandwidth it needs. I've been video conferencing a lot (who didn't, this past year) and even with people on the other side of the globe I don't recall latency ever being a problem.
yeah, like if your playing keys live for example, you want your audio interface to have around 6ms latency max (the delay time between your finger pressing a key and you hearing the sound from pressing that key) above that and it starts to become detrimental to your playing. above 10 and you can barely play.
If this was a consumer product I would buy two of these today, one for myself, and one for my parents. It is more compelling that any product I have seen for a long time.
I doubt you need gigabit for an 8K stream. The real issue is the abysmal common network infrastructure. Latency will be a bottleneck if most ISPs don't care about treating your packets properly.
Agreed, 10Mbps is likely where it'll be, probably less. If I were to guess, they are going to apply a lot of smarts to texture mapping to avoid needing a lot of bandwidth.
Doubtful, IMO. The fact that you need it to be both extremely low latency and with essentially zero compression artifacts (probably lossless, based on their goal). If the numbers listed here[1] are correct, then the most efficient lossy codec at that time was doing 100 minutes of 8k video in ~37GB of data. From that we can intuit that it was using an average of 50 Mbps for that 100 minutes of video. For the most efficient codec, and I'm not even sure from that whether it was using lossy or lossless numbers, because apparently HVEC can do both (but I would assume lossy, since it's about streaming video in that case).
You can't do weird texture mapping or lossy compression and expect people to really seem like they are there. Even if you don't notice that stuff normally watching a video, I think you'll notice it when you're interacting like someone is really in front of you, and that will throw off the immersion.
HVEC is no longer king of the hill when it comes to compression efficiency. AV1 and the upcoming VVC do better.
That said, my intuition is they aren't doing a pure video encoding solution. The fact that they talk about 3d modeling leads me to believe they are doing a combination of model + texture to get the realistic results. That would significantly decrease the amount of bandwidth and computational power needed. Over a low bandwidth situation you'd simply need to send model updates and do some smart interpolation to determine what things should look like.
Similar to the concept that playing a 3d game requires MB of resources but recording the same game at 8k would require a boatload more memory.
My assumption is they are using LIDAR to get a good model, high quality cameras to texture things, and a nice AI to stitch things together and interpolate when data isn't arriving fast enough.
I think this is what will be done in the future. You will interact with the camera for 15 minutes or so and it will create a custom compreasion algorithm for you.
It heavily depends on what encoder and config you use. In my experience, 20Mbps (or less) HEVC by Turing NVEnc realtime encoder is enough for 1080 60p. Also halving frame rate won't halve bitrate because of how video compressed by reference frame. Also video meeting won't move pictures as much as FPS gaming.
Most internet providers here in Finland provides you fiber connections from 100-1000mbs pretty cheap. And 4g/5g connections from 100-300mbs for cheap prices too. Most of them does not have any data caps.
I have 300/100mbs connection which costs 20 euros in month.
I suspect eventually this will percolate down, somewhat like Facebook's Portal TV (£149) but possibly also integrated with something like a Google Assistant/Siri type smart device to power the software and do other smart-home type things in your home.
Many ordinary folk already have 4k TVs at home and 8k will probably become commonplace in the future. The real bottleneck will be good low-latency broadband both up and down, but fibre to the home should make that easier. I wonder if ISPs in the future will offer QoS guarantees to enable really good videoconferencing?
I mean, Zoom, Meet et al are much better than video conferencing solutions like Webex even from a few years ago, but it's hilarious how much drama there still is around video calls. "oh, sorry, I didn't realise I had muted myself", "can you hear me...?", "I think we just lost Steve", and so on. I'll be glad to see all of that just go away.
I think one of the major aspects of this technology is that the person you are talking with appears in its real size, with the same ratio than in real life. Something you cannot do with a smartphone, except if you are referring to a projector.
This is a 3D monitor. Not a light field display like Starline.
The traditional glasses free 3d monitors rely on special coatings on the glass to "split" the direction of a pixel. Some coatings are electronically controlled (like Sony's), some are physical (lenticular filters) It still displays a 2D projection of a 3D image, but twice.
By contrast, my understanding is lightfield displays projects the entire volume. So you don't have a pixel, you have a voxel. So each eye gets a different bit of the same voxel
The term light field display includes lenticulars and parallax barrier displays (and holograms and volumetric displays). They are all different ways of carving up, sweeping out, or multiplexing a light field.
I've used the Echo Show 10 with the drop-in feature for my mom with vision issues (basically can't use touchscreens) and it works fairly well. That said I fully agree with you that something like this would be really excellent. A full-sized person's face and head in 3D is far easier to recognize and understand than a poorly lit 2D 10" display.
Fascinating. So not only is it feeding it an 8K / 30 (60?) FPS image, it's feeding it numerous incident angle variations and displaying all of them simultaneously?
Is existing Looking Glass Factory tech the same though? Not so sure about that. Those displays are typically monitor-sized at the largest and not really aimed at displaying a live feed of a person. This looks to be a more seamless experience on a larger screen.
I only know what I saw from the IO stream, but I think it might send a compressed 3D mesh + texture across the network and render the light field locally.
I think that is where the custom compression algorithm comes in. If you think the fact that human body and face doesn't change much, and the fact that it's a 3d model based, the compression ratio could be very high.
I think what they are transferring is not a video but 3d model and the skin texture applied on the model (all derived from the realtime video / depth recording on the other side). The receiving and then renders it as a 3d model on the screen.
Do you think it's possible 90% of this is due to the studio quality lighting, large high quality screen, good mic/speakers, and low latency network? It seems like those factors alone would get most of the way there and the 3D aspect is just a bonus. Obviously I haven't used it in person but this was just a thought since most people are used to video calls on their small phone/laptop with poor lighting, mics, etc
I am sure that the immersion of the experience is higher. My question (and perhaps that of GP) is: is this greater immersion actually beneficial to communication?
I think this is cool tech, and valuable. I'm just not sure that it offers a communication benefit over well-lit, well-miced, wired, low latency, 8K videoconferencing.
Maybe there's some 3D emotional perception face processing stuff that we have deep in our brains that can immensely benefit from this, but I'm skeptical. I think simply doing 4k or 8k low latency high quality videoconferencing might be a 90 or 95% solution without needing special cameras/displays.
Being able to feel like another person in the room is enough for me to reconsider working from home. As of right now I strongly have a preference for in person, but I do acknowledge most people prefer commute and cost benefits over productivity.
The state of video conferencing today is a poor one and I'm very excited for something that can change the industry like this.
I'm right there with you, and I use a 4k camera and a boom mic and headphones and wired ethernet to videoconference now: I have been regularly complaining about the low resolution and framerates of current videoconferencing systems (10-15fps, 720p, low bitrate - and that's the highest quality setting available!).
If Google wanted to make me believe they care about videoconferencing quality, they'd have a 4k 60fps option that auto-enables in Meet if it detects everyone on the call is on wired gigabit with a 4k camera.
A lot of residential areas in the US have gigabit options, in some cases symmetric. There are lots that have 1000mbps down/40mbps up cable.
Even 100mbps is sufficient for a 1-on-1 4k video call, as high-bitrate 4k is 30-40mbps. Most commercial office buildings in business districts have it available. Even Starlink (20mbps up) should be sufficient for 1-1 30fps 4k videoconferencing with a lower bitrate.
I think you might be underestimating the value of viewing a 3D model on a no-glasses 3D display. This is one of the basic aspects of in-person communication we take for granted that current 2D technology can't replicate. You can move your head and actually see a different angle of the person in front of you. This can even be subtle, our brain will still pick up the effect, and it makes the experience beyond what we usually consider as "immersive".
Yes, having low latencies and high definition video is an important aspect of this, but the 3D part is no gimmick. Once the technology improves and gets affordable this is a game changer for how we communicate online. The step after that are holographic displays, and since we'd be used to 3D models and smart displays, it probably won't feel like such a big jump.
I'm _super_ excited about this project. Hopefully Google doesn't axe it. (:
I thought that as well watching the video, but have you used a PSVR?
I've got one and the first minute is always noticing how low res the eye screens are, then as soon as the game starts, I've forgotten and I'm _there_. The 3D part makes up for the low quality
C'mon, they're showcasing prototypes or early 1st gen products here. There is some artifacting, true, though not nearly as much as I expected. Kudos to them for choosing to show objects difficult to scan/model accurately and doing a pretty impressive job at it. Under ideal conditions to be sure, but still. It's certain this will improve with future advancements and probably will by the time general consumers get to use it. Unless it never gets a widespread release and ends up as another Google research project ala Google Glass, Project Ara, etc. Hopefully not, but if nothing else it would have served as inspiration for other companies to step in now that we know what's possible.
>I think this is cool tech, and valuable. I'm just not sure that it offers a communication benefit over well-lit, well-miced, wired, low latency, 8K videoconferencing.
>Maybe there's some 3D emotional perception face processing stuff that we have deep in our brains that can immensely benefit from this, but I'm skeptical.
>I think simply doing 4k or 8k low latency high quality videoconferencing might be a 90 or 95% solution without needing special cameras/displays.
From my experience, 4k or 8k doesn't matter. Sound quality actually matters most, really clear low latency audio alone will give you a surprisingly strong sense of presence.
Video quality is important but 1080p is enough, beyond that the lighting and latency matter more.
Equally important from my personal POV is video size - physical size. Take a cheap 65 inch TV, turn it vertically, and talk to someone on that. When your talking to someone that is actually life size the sense of presence is vastly improved, even at the exact same video quality. And TVs are so cheap this doesn't seem like much of a techical barrier.
If you just screen share from your cell phone to your 65 inch TV and video chat -- holding everything else equal for audio and video quality -- it's SO MUCH BETTER.
It looks like it came out of the high-fidelity Immersive Light Field Video presented at SIGGRAPH 2020. Quite impressive that within a year it's now a consumer product
WebAssembly SIMD is coming to Chrome as well. 2D images and video that only consisted of RGB and Alpha channels may appear downright primitive to future generations as depth camera rigs gain distribution ;)
I've spent a decent amount of time and money to make this happen and it helps less than you would hope.
Partially there are just affordance issues of things like eye contact which are physically out of alignment unless you start using two way mirrors [1].
There are technologies that can automatically adjust videos for eye contact today, so I imagine something similar could be implemented for this later on.
Eye tracking is the core feature here--the rendered "hologram" is correct from every possible angle. The things you mention are probably closer to 2% of the final result.
My intuition is that a great lighting+microphone+speaker setup is necessary, but not sufficient, for this demo.
Even from viewing the short demo, the stereo display alone is an entirely new dimension that no amount of studio lighting will recreate. While better lighting and audio setup would certainly improve the average person's videoconferencing experience, this looks to be a genuine step beyond.
That said, we've been seeing holographic-display prototypes for the better part of a decade, and it'll be interesting if this actually pans out or fizzles.
What I think you're trying to describe already exists as a product from Cisco called "telepresence". It is/was insanely expensive, was a permanent installation that only Cisco contracted techs could install, and did what you describe: It is a series of large, curved HD displays with desks at an appropriate distance from the screens/cameras, and copious amounts of indirect lighting from behind the setup to make each party look good.
It seems like the imaging/rendering technology that Google is using is much more advanced.
I’ve used such a Cisco system. Compared to regular video calls the latency and quality was light years ahead, much more natural conversations were possible. By which I mean it was possible to laugh, interject, and generally have a realistic conversation with a colleague in another country without having to compensate for video lag in that very careful way I find necessary on Meet and Zoom.
That said, there was no “emotional connection” like the Google one is described as offering. It was still a video call. There was no forgetting that. I suspect the 3D and the apparent physical closeness to the display add a lot.
Wow I forgot about Telepresence. I used it a decade ago at a Fortune 500 company. With all of the cameras and displays perfectly positions, everyone was life-sized on video, felt like you were sitting around a roundtable. Now I'm imagining that with higher resolution and 3D light field display, wow.
I spent a year using telepresence a few times a week. It was genuinely amazing. Lifesize people 1200 miles away with audio so crisp that one of the guys was idly rubbing the edge of some papers with his thumb and I could hear it.
Low latency is more important than all the other bits. I worked at Bell Communication Research in the nineties and they had an experimental video conferencing system that used analog circuit switched video and it worked really well, mainly because the latency was only a little more than the speed of light.
It looks like they're doing photogrammetry in real time, which is mind boggling. I'm not familiar with this space, but building a 3D model, texturing it with live video, compressing and sending that over the internet, and doing it with minimal latency for it to be believable/enjoyable? Incredible technical achievement if that's the approach. Using state of the art tech, no doubt, and probably lots of ML magic to smooth the rendering. The 3D display is the icing on the cake, it must look amazing in person.
not that hard to do if you have actual depth sensing cameras, and even without those, something like the oculus quest 2 does that exact task (generate a rough 3d volume based on several 2d video feeds) you can see a neat example when you draw your guardian space, and move objects (and notice how it updates the 3d volume representation)
Good point, I haven't followed the latest VR advancements, that does sound neat. Still, Starline's approach is surely much more sophisticated (the hardware obviously has a lot to do with that, these are prototypes of a desk-sized machine vs a headset). The 3D model looks reasonably detailed, and the final render has very few artifacts. Making it all work over a WAN link with latencies critical for teleconferencing is also impressive.
The difficulty completely depends on the level of quality you're after. They're certainly working on cutting-edge-level quality, so it is likely no easy task. Someone else pointed out that they released a paper on related tech last year:
That's not true -- even with depth sensing cameras, it will still be full of artifacts, and things like curly hair or strands of hair will become disastrous because they're not easily geometrically modeled.
The Oculus Quest 2 doesn't do anything like what you're describing -- it essentially just pipes in stereoscopic video from its stereo cameras and stitches them together in a trivial way. It doesn't attempt to build geometric representations of objects in your environment at all.
(For guardian functionality it does very simple things like using the depth cloud to figure out the height of the floor, and if there are points inside the guardian that shouldn't be there, but that doesn't inferring object geometries.)
if you look at the video you can see there are artifacts around the hair. It is likely applying some matting via AI to make it less obvious, but it is still there.
The Oculus Quest 2 (and the Quest 1) infers the geometry of your environment in the same way a Magic Leap does. The Quest uses the mesh to show perspective-correct stereoscopic pass-through views. https://www.youtube.com/watch?v=3V__SEPobM4
I want to try this just to see what happens, if I will have again the feeling something is more real than reality.
Because optometrists are illegal in my country (here only medics can decide what glasses you can use), currently I don't have stereoscopic vision, although my brain CAN do it, if I had the correct images sent to my eyes somehow. (one of my eyes muscles is slightly shorter than the other side thus the images on that eye are shifted unless I had an optometrist design me glasses with a prism).
So when I watched Avatar, an actually well made 3D movie, it literally felt more real than reality, despite it being obvious fantasy with aliens, floating rocks and all that stuff.
EDIT: for those wondering, I am from Brazil, here medical professionals often sue the shit out of anyone offering any service remotely similar to an optometrist, they are quite aggressive about it, some attempted to make even discussion of the subject illegal. And when I was trying to get the prism and asked around my medics about it, one of them went really ballistic, I honestly thought the guy was going to punch me. I believe the reason for that is that for many medics, designing glasses is their only source of income, a guaranteed one, since here is ALSO illegal to buy glasses without a medic desining them for you first, even if the new glasses are supposed to be identical to the old ones!
Shouldn't an ophthalmologist here in Brazil be able to diagnose you and give you a prescription for corrective lenses, or even perform corrective surgery?
I'm asking in case I might need to look for one in the future.
I'm from Brazil too, and my eye glasses are recommended by a optometrist. The only way an optometrist to work is being an ophthalmologist with specialization in "optometry". It is a sad reality from our bureaucratic government that don't give us freedom to innovate.
The profession of optometry came from physics instead of medicine, many of the early ones with physicists specialized in optics that ALSO understood how the eye worked, optics-wise.
In places where they are legal they spend their time learning more and more about optics, physics, math and eye anatomy, they don't study diseases, infections and so on.
As result you can't rely on one to fix certain stuff where you do need a medic, but if you need fancy lenses, they will calculate them, not just use a number they get from their measurement machine like ophtalmologists do.
They also would be helpful to design proper 3D glasses and whatnot, holograms and so on.
Got it, I was a bit confused as I never had heard about the problem you mentioned. Too bad here in Brazil you can't get the specialized care you need for the condition you have.
This is a major screw up and inefficiency from this country.
Thanks for explaining the concept more in-depth too.
Brazilian guy here: my girlfriend just bought two pairs of glasses from China, I don't know from which site exactly, but some AliExpress type of site. She paid 5 dollars each, just telling which degrees she needed for each eye. The quality was the same from the 200 dollars pair she bought with a medical note here in Brazil. Unbelievable. Just telling that, if you want it, you can import it through the mail and it will pass through customs without a problem.
If you search for "myopia glasses" on Aliexpress you'll find prescription glasses starting as low as $2. I just added a pair to my cart and the total was $3.31 including shipping. :O
I have glasses from Zenni. Very cheap, and well made. My current glasses cost ~10% of what the previous pair cost me in NZ, and the lenses are better quality. The glasses cost less than the optometrist visit, and I bought a second identical pair as a backup.
Something used in some countries is a briefcased-sized glasses making kit. Eyeglass lenses have three parameters - spherical radius, cylindrical radius, and axis of the cylindrical curvature. The trick is that for round lenses, you can use the same lens for all axis angles, which reduces the number of combinations to a set you can carry around. Once the right set of lenses has been decided, a notcher is used to cut a small notch on the side of the round lens so it locks into the frame and can't rotate.
>my brain CAN do it, if I had the correct images sent to my eyes somehow.
>when I watched Avatar, an actually well made 3D movie, it literally felt more real than reality
Buy an Oculus Quest 2. (Or any VR headset, but that's the best value at the moment.) It sends seperate images to each eye. You should get that same 'more real than reality' feeling. It may even train your eye muscles to see in 3D, I know I've seen some research into that area. (https://www.seevividly.com/ comes up on a Google search, though it seems to be prescription only)
I'm someone with Amblyopia and would love to hear people's experiences using this. Seems like the project made some waves on reddit and kind of died down. I have bought Oculus Quest last year in anticipation of trying this but the pandemic and work pushed it out somewhat.
Watching 3D movies in the theaters just gives me a headache but Oculus actually works fine, I've played a few games and it was amazing...
I have that too. My right eye keeps steady while my left eye floats
around at random (or at least that's how the linguistic side of my
brain describes it). It would be great if a VR headset could track the
left eye position and translate the image accordingly.
I would've never guessed Brazil (or any country for that matter) would be like that. I guess I can see why if medics depend on that for their income, but I certainly would've never guessed they would act so aggressive about it. It seems like there should be some other way for them to make enough money. There should be somebody who's specialty is optometry. Do you have specialists for other things like a heart doctor, or cancer doctor?
The question for me is how much it matters after the novelty wears off.
I count at least 5 waves of 3D technology starting in 1851 with the Brewster Stereoscope. Each time there's a surge of popularity driven by the legitimately amazing initial experience. And each time people slowly stop caring. People were incredibly excited about Avatar, and many thought it would change the movie industry. But how many people now go out of the way to see something in 3D?
Replicating the experience of in-person communication much more closely than video and 2D displays will ever do. That's a noble research goal if nothing else, I don't get the skepticism.
There are several reasons 3D content and previous generation displays didn't take off, but there's no reason to believe a revolutionary new approach and product couldn't change this (e.g. electric cars were invented in the 19th century and are only now becoming popular). AFAICT the real time photogrammetry they're using here along with the no-glasses 3D display is a major leap forward. If they can get it cheap and reliable enough to mass market, it would be a game changer.
I certainly know what kind of display and teleconferencing software I want when the next pandemic hits, and it's not what we have now.
> Replicating the experience of in-person communication much more closely
That's not a problem people express much, at least not in ways where "3D" points to a solution. When I want to see people in person, it's not because of a lack of stereoscopy. I want to hug them, to break bread with them.
> there's no reason to believe a revolutionary new approach and product couldn't change this
There is indeed! In specific, the many times we have already had revolutionary new approaches and products that were met with great enthusiasm in the market for a few years.
I'd add that the telephone was not only a very successful technology for a century, audio calls still remain very popular. (I'm not sure what your work calls are like these days, but quite a lot of people turn off video in mine.) The lesson I take from that is that people mainly self-generate the feeling of interpersonal connection, and they can do it with very little in the way of cues. To me that's another strong indication that no new 3D technology will make much of a difference.
> When I want to see people in person, it's not because of a lack of stereoscopy.
Of course, I'm not saying this will replace physical communication (I should've said "simulating" instead of "replicating"). But it's a clear step forward for traditional teleconferencing solutions. What do you think is the next leap from 2D video and displays? We're at the point of diminishing returns as far as increasing resolution goes, most consumers don't have a need for 8K or higher res displays. VR/AR is chugging along, but we're still a few generations away from mass market adoption.
> I'd add that the telephone was not only a very successful technology for a century, audio calls still remain very popular.
I don't understand. Video calls were never meant to replace audio calls, they just added a new sensory experience. It's perfectly fine for both technologies to co-exist for different moments and preferences. In a similar way this 3D approach is an extension to traditional 2D video conferencing if people have the equipment and prefer it. Judging by the expressions of the people in the demo and some of the comments here, you're underestimating how impactful this could be, especially if it's polished and cheap enough.
It's not a clear step forward except in technical terms. Those often don't matter. For example, the big revolution in the last 20 years is not faster computers, it's mobile ones.
I don't have much reason to think there is any near term "next leap from 2D video and displays". 2D renderings are more than 40,000 years old. They have improved drastically in resolution and fidelity. Computers added being dynamic and interactive to that. it's really not clear that 3D rendering adds much.
> Judging by the expressions of the people in the demo and some of the comments here, you're underestimating how impactful this could be
I am not, because that kind of novelty-driven excitement has driven every wave of popular 3D rendering technology for 170 years. VR/AR has been close to mass market adoption for 25 years. We've just been through an unprecedented period of demand for at-home entertainment, and the hardware that many said was finally, finally the thing turns out once again not to matter.
People have had those excited faces every time. There were people jazzed about the possible impact every time. The Brewster Stereoscope. The ViewMaster (with the US Defense Department purchasing 6 million reels on the theory it would revolutionize training). 3D movies in the 1950s. VR in the 1980s and 1990s. 3D movies again this century. 3D TV for 2 CESes. And then the latest wave of VR, which you agree is still not there despite fantastic investment from companies floating in cash.
Could it be different this time? Maybe! But if we keep measuring it by novelty effects, we're setting ourselves up for the exact same failure that keeps happening.
> the big revolution in the last 20 years is not faster computers, it's mobile ones
Surely the improvements in manufacturing processes, faster hardware and better screens are partly responsible for that. The iPhone as a concept has existed since the 1980s, and revolutionary ideas like what General Magic tried to produce in the 90s were just too early to be successful. When Apple tried it again in the late 00s it was a massive success, but technology finally reached a point when it was commercially feasible.
So it doesn't take much to push a product to mass adoption. Just the right industry circumstances, a manufacturer willing to take the risk and capable hardware and software existing to make it happen.
> We've just been through an unprecedented period of demand for at-home entertainment, and the hardware that many said was finally, finally the thing turns out once again not to matter.
Are you dismissing the potential of VR/AR as well? The current innovation wave we're on is much bigger than whatever we had before. Headsets are becoming cheaper, more comfortable and accessible, and the visual tech we have now is leaps and bounds better than previous generations. Once we get to being able to put on sunglasses and experience different worlds, though likely sooner than that, the market adoption will likely go through the roof.
> People have had those excited faces every time.
I think it's different this time. It's not just it being 3D, but the merging of new generations of light field cameras, face/eye tracking, powerful ML algorithms, low latency networks and revolutionary displays is miles ahead of previous attempts. You can't just compare this to the ViewMaster and last century VR. The improvements here are much more substantial, and if they can make it cheap and reliable enough it could be a ground breaking product.
Again, you're arguing that the technology might get better. I don't disagree. I'm not comparing the technology of the ViewMaster. I'm comparing the lack of demonstrated demand/utility and the pattern of hype.
Every one of the products I named was greeted at the time exactly like you are now. The new technology was amazing! The potential was unlimited! And for the repeats like 3D movies and VR: It's different this time!
I agree it might be different this time. Nobody's denying that. Aliens might land tomorrow. What I'm saying is that because of the clear pattern of "OMG novelty! OMG possibilty!" around 3D tech that has failed repeatedly for 170 years, you can't just uncritically make the same arguments. If you want to be persuasively realistic, you have to explain why the 3D novelty effect isn't the major driver this time. Because the long evidence is that 3D displays just don't matter enough for people to stick with them.
Just one final comment: I agree with you that there are technical innovations that don't result in mass adoption and ultimately don't matter. Where I think you're mistaken is that the leap from 2D displays to 3D holograms (or 3D displays as an intermediate step) is similar to the leap from black and white TVs to color TVs. It's obvious that it's the next big step since our world is not black and white in the same way that it's not 2D. The potential market for that is global so we've been pushing in that direction for 170 years, as you say (though sources for that claim would be appreciated as I couldn't find any), yet the technology just wasn't there to make it a good product.
Do you remember the Virtual Boy? Or the old cheap red/green paper glasses, and recently plastic glasses that are uncomfortable, darken the picture and give you headaches? These are all issues that better technology can solve, thus reducing the barrier to entry. A display that shows a 3D image without glasses to every viewer with a head tracking effect can potentially solve a lot of them. With similar improvements in camera technology, networks (5G anyone?), ML, etc. and all the pieces are starting to fall into place for what could be a revolution in how we communicate electronically.
Or Google might just axe it as they've done before ¯\_(ツ)_/¯ (Still bummed about Project Ara...)
I agree it's "obvious" that it's the "next big step". And I'm saying that's the problem.
Obviousness is a feeling people have about ideas in their head. Feelings can be useful or misleading. Every single person who got behind previous generations of 3D thought that it was "obviously" the next step. Many thousands of people had that feeling each time, buying in to a new platform. Great sums of money were invested by smart execs. All of those people were wrong each time. All of them. That you have the same feeling is not proof that it will be different this time. Indeed, the history suggests you should distrust that feeling.
Color video is a great example, so thanks for bringing it up. Color TV and color movies were quickly and widely adopted despite the extra cost and complexity. But 3D movies and TV have failed. Wearing a pair of glasses is not a major burden; 64% of Americans do it every day. Millions of people tried 3D movies and gave a collective shrug. The pretty obvious lesson to learn from those waves is that people are drawn to the concept but actually do not care in practice once the novelty wears off.
Another way to look at it is that people don't even care about stereoscopic vision much in actual life. Humans have a lot of mechanisms for extracting spatial information from the world, and the stereo-ness of it doesn't matter much. About 10% of people don't have it; they can still drive just fine. My grandfather, for example, was blinded in one eye as a kid, and nobody ever noticed. You can try it yourself; go out for a walk and keep one eye closed. Your 3D perception will be basically unaffected except for relatively close objects.
So sure, as I've said repeatedly, anything can happen. I'm just saying there is good reason to believe this will not happen, and excellent reason to not just assume it will. To see this not as a technological problem, but a problem of demand.
As to citations, I'm not sure what you're looking for. I've mentioned the Brewster Stereoscope twice in this thread. Ditto the ViewMaster. What do you need that isn't in the first page of Google results for those?
"Hard tech" often only matures after several hype cycles. Sometimes cool tech demos can be produced a century or more beforehand. If you were in the 1980s and people were talking about video calls being the next big thing, you might point out that people have been working on video calls since the 1930s[0], and it hasn't caught on in every one of the hype cycles that have followed, so that's an indication that it won't catch on in the future. Video calls have caught on now though - especially as they've reached mobile devices instead of requiring a literal booth in your house, as with AT&T's initial "Picturephone" tech in the 70s.
I will say though that people tend to assume that each new technology will replace the preceding technology (text->audio->video->VR/light-field->...), but in fact it tends to end up just supplementing the existing tech.
Wow, this is a pretty ableist take. Deaf people or hard of hearing exist.
I've got a hearing problem where I struggle to make out what people are saying on a phone but with a video call I can add lip-reading and visual cues which helps me keep on thread.
Which is excellent, and I totally support that. But that doesn't change the market dynamics that I'm describing. We probably should live in a world where what drove the adoption of video calling was supporting the hard of hearing. But we don't, so it's not a relevant factor for the market analysis of what will drive the adoption of 3D video.
> (e.g. electric cars were invented in the 19th century and are only now becoming popular)
I hate this example, and it's like one of the most common ones on HN.
As said by thousands of people and many documentaries before me , the electric car had numerous real conspiracies working against it, some of which were the most powerful financial groups in the world.[0]
The 'electric car' wasn't made popular and possible by recent technological strides -- although it was made better.
The success and popularity of the electric vehicle was made possible by financial shifts away from petroleum exploration, facilitated by dwindling profits and increased scarcity of oil, and encouraged by a movement towards sustainability both from the social culture of the world and the various actions of government from country to country.
Yes, range has improved. Yes, the cars are more intelligent and better to drive -- but these improvements have been seen across the automotive industry since its' inception with ICE based vehicles included.
The real motivating factor behind the electric car is the environment that now exists that allows such endeavors to be profitable -- an environment that not only includes technological improvements like you hint towards, but more importantly it's an environment that fosters development of such things due to the existence of a profit incentive and increased governmental-body cooperation.
All that said, unless Cisco is even more evil than I realized (woah..), I have a hard time presuming that video conferencing has been held back by the same sort of conspiratorial under-handed back-office dealings that slowed the progress of EV adoption.
Why does it matter which circumstances allowed the electric car to become popular only now? Whether that's because of major industry and consumer shifts, or because the technology matured enough for mass market, they have the same effect. It's certainly a combination of both, and we shouldn't downplay the advances in battery technology alone.
I mentioned that example because the previous two comments were dismissing this attempt at 3D teleconferencing on the grounds of it being old technology with past failures. But I think we agree that it takes a certain industry environment along with a technical leap to make a technology truly popular. Even if that never ends up happening in this case and it remains a niche product, we should applaud the technical merits here instead of being dismissive.
For that matter video teleconferencing period has only just really hit critical mass even though there were videophones at the NY World's Fair in the 1960s and camera systems have been around in conference rooms for a few decades.
What really happened was that it became more or less accessible to anyone with a laptop and an even marginal network connection for basically no cost. And, oif course, the last 18 months really pushed it over the finish line if it wasn't already.
Exactly. Video calls have become successful because remote work has become successful. The goal was not to make video calls. The goal was to further improve a team that was already remote. And I think you're right that the low/no cost hardware for most adopters has been key. Which I think is further proof that the demand is really pretty modest. I was just on a work meeting where half the people had their cameras off, and where I often didn't have the Zoom window on top because the video was very much secondary to what they were saying.
Yes? Other than not being hip, how is Instagram better than Myspace? WhatsApp at least added features over SMS, although it sacrificed interoperability and went all-in on a closed system to get there. Newer is not always better.
If you talk to daily Instagram users, you'll find out what problem it solves. In particular, it was the lucky winner in the crowd of early photo-sharing-on-mobile apps. But the demand for that was proven by early photo sharing successes like Flickr. Realizing people would want to do that on their pocket camera device is not a big leap given what people were already doing to share photos from their mid-aughts cameraphones phones.
There's no such plausible story for 3D video calls. It's not like people are already demanding 3D displays for any of their other 3D stuff. The 3D first-person shooter, for example, has been around for decades. But 3D displays have never been popular despite being available for at least a decade.
That's an interesting hypothesis, but I'd need to see some data. Since you haven't experienced it, you would be buying based on hype, on the concept of 3D. As I said, I'm not arguing that the novelty is appealing. I'm saying that once people actually experience it and the novelty wears off, people stop caring.
Another issue here is that this is being sold as like "being there", but it's more like "being there at a jail" where you can see person but can't get close to them, can't touch them, can't hand them anything. I have immigrant friends who do calls with their parents basically daily. They do it with mid-grade consumer tech, even though they could easily afford big screens and high-res cameras. That suggests to me that image size and video quality are not as important for this market as one might think at first blush.
You're not winning anything with casting Skype on TV, except messing with another remote for audio controls (which is in no way immersive or often not even high quality).
And yes OF COURSE I predicate buying on it actually delivering to the extent people describe it in the marketing video, it's ridiculous I have spell it out.
Casting Skype to a TV is not the only possibility Right now with off the shelf hardware one could make a great video-call station. 4k screen, 4k camera, high-quality mics and speakers. But approximately nobody does it because laptops and iPads and phones are generally good enough for them.
That to me demonstrates that, contra your initial assertion, there isn't a big market for this.
As to the last part, you've gone from "I will get it" no questions asked to what sounds like "I will get it if it checks out". But that's a big jump. You've gone from an early adopter to a mainstream purchaser. From one of those people that buys things on Kickstarter to the much, much larger group who want to see proof of value before they buy.
I think that's very reasonable, but it's exactly the kind of reasonable behavior that has killed 3D over and over in the past. By definition novelty doesn't last, so by the time mainstream purchasers might be ready, the social proof just isn't there.
It depends on what you mean by want. Are you having feelings of desire? Sure. That's the point of demos and commercials. I fully believe you have those feelings, and trust you to be an expert on them.
But I've done a lot of customer development over the years. People say all sorts of things. The question when doing market analysis is what they'll actually do. And the better guide there is what they're actually doing , not what they say they would do.
So when you say that "everyone in immigrant communities" will buy it, I'm going to be skeptical because what people are actually doing is nothing like that. They could already move in this direction with existing tech. As far as I can tell, they aren't. If you have evidence otherwise, I'd love to see it.
I also can't find evidence of third parties competing with shared higher-quality video call setups, which is what we'd expect to see if the demand were there but the price hadn't fallen enough yet. That's the pattern we saw with video arcades and internet cafes/wangbas, for example. Wangbas are still getting by because they've shifted to gamers, who are willing to pay up for better hardware and connections (and room for team play). But I can't find mention of any similar shift for video calls. E.g. India's PCO network seems to be in rapid free-fall, not reinventing themselves around high-quality video calls. That suggests what all the other market data suggests: to the extent people want video calling, relatively low-quality gear like smartphones and laptops are in practice sufficient.
>Tell me a problem WhatsApp solves. It's just a fancy SMS.
I gather it mostly solves that SMS is expensive in a lot of contexts. Personally I never use it because most of the people I text with have US phones. And the one person who doesn't, we use Facebook.
Oh, I'd bet that team did a great job with 3D. (For those who didn't see it, it was a movie with great visual effects designed to blow the viewer's mind.) But you and that movie are the exceptions that prove the rule: for most people and for most movies, it just doesn't add much. The reason people go to see something on the stage is never stereoscopy!
I honestly think it's amazing and I'm sure the novelty would wear off, but it would still be useful. One of the things that stood in the way of 3D viewing was dedicated hardware, which is the same issue here. Although the need to wear glasses is gone, you still need to buy some serious equipment for it. Maybe at some point this will be bundled into a normal TV setup and people will just take it for granted that it's there
Anything's possible. But I want the people saying 3D TV/monitors failed because the glasses were just too burdensome to argue things out with the VR people who say that 3D is so amazing that the (much heavier!) facehugger units will take over the world.
I'm a regular in the XR space. A popular running theory is that wearing something is the barrier right now. Often what's brought up is that the only wearable to make it to the prime time is the smartwatch--and even then it's a very slow uptake. People absolutely do not want to wear something to make marginal gains in their viewing experience.
In that respect I'm very excited for Starline and related technologies--"trying them" will finally just involve being in the line of sight of one rather than having an attendant fit some goggles onto your head.
I have wondered in the past if a similar result could be achieved using a 3d headset with some tracking/cameras and removing the headset from the view through a real-time deep fake that could be achieved through a short scan before the call. Would this even be feasible?
I've seen 3d screens in Japan 20 years ago. TVs, ad displays, mobile phones. It works really well, and it's actually very simple technology. Never understood why that never took off.
This is different. It's a light field display, so you can view things at different angles as you move around, rather than having to sit in a very exact position and only seeing one angle.
No, this sends a different image to each eye. It's not actually a transparent piece of glass with an image on it[0] - I think they just try to make it seem as if it is a transparent pane you're looking through.
The marketing for it is pretty terrible though, because I initially came away with the same impression as you.
In other words, you pay market value for it, it doesn't include advertising, and Google is contractually obligated not to snoop on your content.
Which is why even Google competitors use this hardware, because the legal and technological protections are strong enough they don't worry about Google stealing their IP.
So in my company, I've used various avatar's of Cisco TelePresence systems over many years.
The big commercial need it turns out isn't so much realism as it is flexibility to accommodate people dialing in from different systems - phones, laptops, different types of telepresence setups from small single room to big conference rooms or even telephone connections etc.
Many years ago, we were all wowed by the life-size realism and had people come into offices for it . Nowadays these meetings have lots of people crammed in across multiple screens dialing in as they please and all the better for everyone :-)
Over a decade ago I used a holographic video conferencing system that used kinect for depth mapping, it was very convincing. It would also reorient the display based on your head position. so you could look "around" the avatar.
There are demos of binocular 3d conferencing done with a lenticular display (lookingglass), although those large displays are extortionately priced (1/4 of the size of this google one is $3000...) - keeping them out of the hands of most devs.
No doubt google are doing the same, but can afford these larger displays.
You can easily find examples (and research papers) by googling for the relevant terms. Google claiming they have invented a "new technology" just shits on all those folk who dont have the publicity/funding of google.
Just very practically, is this a $xxxx.xx improvement over simple video chat with a large screen and simple camera? Will this even work without low-latency high-bandwidth connection up and down?
It seems more like an art project than a tech usable in the coming decade.
Can you tell your friend at Google that their website weights whooping 89 MB, most of which are huge animated GIF-s? This is insane! Doesn't Google know about HTML5 video yet? https://www.smashingmagazine.com/2018/11/gif-to-video/
Interesting thing to note is that they don't show the participants touching the glass pane 'separating' them, whereas for that kind of situation it would be very very natural thing to do.
I guess doing so (reaching towards the 'glass pane') would make the imagery distort/degrade real fast as you would start going out of camera's FoV, which that would break the magic.
A bit ironic that this promo video maxxes out in 1080p, despite YouTube supporting 4K and this tech pumping 8K+ res.
There is so much more we can do in terms of quality and immersion that we're not doing simply because bandwidth and connectivity are so low-quality and overpriced at most of our leaf nodes in the USA.
I get similar vibes to demoing VR the first time. My first thought was, "My tech-illiterate dad would have his mind _blown_ by this."
Unfortunately, even an already-set-up VR experience was too strange/unnatural for him so he never got to experience it. However, this looks easy/natural to use and set up and feels like it'd have a similar mind-blowing effect for many of the older generations, which I think is a good indicator of being revolutionary tech (assuming it can be made available/cheap enough for most people to try it out).
I got my Dad to play SUPERHOT VR for a bit, he liked it, but I suspect the "Shoot yourself in the head to start the demo" bit was possibly a bit on the nose considering he spent quite a few years manning suicide helplines.
The "wow it's really 3D" effect without glasses is what people experience everytime I've shown Looking Glass to someone, glad to see devices with higher resolution are coming and more people in the market, end users will benefit from more concurrence to drive the quality up.
This reminds me of the devices described in the short story "The Story of Your Life", which is the basis of the movie Arrival. Excellent, excellent story and very different from the movie, if anyone's into that kind of thing.
Seconded, I’ve really enjoyed all of Ted Chiang’s books and wish he was more prolific as I’ve read them all! An excellent writer and I think his style would appeal to the HN crowd as it’s kind of sci-fi, kind of based in science, and many of his stories are quite thought provoking and unique. Also they’re all short stories so great if you struggle finding time to take on a whole book or whatever!
I recently read Exhalation, and was blown away by how engaging and thought provoking the stories were. Would highly recommend it to anyone with even a passing, high-level interest in sci-fi.
Same here. I’m quite picky when it comes to sci-fi, a lot of it I don’t really like, but he nails a really unique sweet spot where sci-fi is just the framing device used for some beautiful stories. Might reread his books next actually!
Edit: I’d love any recommendations of other authors people think of when discussing his books. Doesn’t necessarily have to be the same style but more just that level of quality and uniqueness.
The first story in Exhalation is written in the style of the Arabian Nights (Thousand and One Nights). Get the unabridged, original translation by Richard Burton - it is some of the most beautiful literature ever written - comes in 16 volumes so there will be no shortage of reading material!
Came here to say that.
Asimov was one of the few SF authors that not only imagined advanced technology, but also could see what it would do to society.
Is it not the case that having access to a machine capable of running either Minecraft or Linux in the early days of each (if not now) means you are (or were) fairly affluent?
It depends on what you mean by the early days of each - in the really early days of Linux (1992) computers were probably going to cost north of a few thousand dollars and the type of computer Linux was designed for (a multi-user system for dumb terminals) would cost tens of thousands of dollars. By the time linux became a thing more than a handful of people in any given state knew about you could probably run it on a machine costing somewhere around 300$.
Minecraft has never been demanding resource wise, I'm sure early versions had serious performance issues but running it on a cheapo laptop has always been totally reasonable - it's quite accessible (it was written in java even!)
I disagree - I would agree that Minecraft wasn't a novel technology, just like Linux wasn't a novel technology - it was an alternative version of Minix.
Additionally Zoom isn't a novel technology, it isn't even particularly interesting technically when compared to other video conferencing solutions - but over the past year it's been incredibly important to a number of people.
I think the OC slightly missed the mark in mentioning "important technologies" instead of something closer to "technologically innovative" technologies or, more accurately (but less interesting of a statement) "expensive to develop technologies". Things that are expensive to develop generally aren't cheap to begin with, while things that are cheap to develop need to be cheap to compete with other market entrants and clones. Additionally hardware (a limiting factor on cost for a lot of technology) tends to get cheaper over time and that rate of change is accelerated by a large market of interest (leading to more folks deciding to try and iterate new designs).
I think Minecraft and Linux are more like the content produced on top of the technology that is computers. It's like if a new book is written it's quickly available to everyone in the market who can afford a book. The book isn't really technology, but the printing, publishing, and distribution is and it's been around long enough to be distributed.
Software seems less like technology and more like writing. The distribution cost, once the systems are in place, is marginal. The technology part is creating the systems that enable the software.
IBM in the 90's strongly agrees with you - this software stuff is never going to be profitable and everything people pay for will always end up going through us!
More seriously, I disagree about software being less important because there have been very real innovations for tooling accomplished in software alone. Email is a pretty classic example - but a more modern one might be Google Cardboard which can turn your smart phone into a rather underwhelming VR headset. There are plenty of hardware alternatives but the same basic functionality was accomplished on generalized hardware.
Additionally, all this technology is only really possible due to other technology - we don't discount a new shiny computer just because it's just a dumb oddly shaped box if you can't supply it with electricity - but the costs to develop software are generally lower than hardware so I think it's fair to have a general notion that hardware is more innovative - it's just that you're conflating two different variables - cost and medium.
I think you're conflating technology with profitable or important. That is, you see me saying that software isn't technology and think I'm saying that software isn't profitable or important. That's not at all what I'm saying though. I likened software to writing. Writing can be important and it can be profitable, it's just not technology.
Maybe we could agree on email as a technology. Maybe. I think it's a stretch. I hope we could both agree that the nth email client isn't technology though. It's not adding a new capability to humanity which is how I tend to think about technology. Refrigerator - keep stuff cold. Electricity - power to operate machines and light. Computers - organize, access, modify information. etc. New JavaScript library or new game... Not so much technology.
I think that's fair yea - it might just be a matter of semantics. If you think software is included in technology then I stand by my point but, if your view of technology excludes software then you're quite correct.
I think they meant hardware. For software, yes there are tons of examples (open source software is usually free, games are usually cheap).
But for hardware, it's almost always some expensive thing first. The internet was once very expensive to access, cell phones were initially very expensive, DVD players were initially very expensive, computers in general, etc.
Penicillin. But then, he wasn't trying to make money off of it.
And basically anything Nintendo pushes as a console gimmick. It's not that the tech immediately goes from research to broadly accessible, but rather that they tend to take old tech that no one saw as having profitable consumer applications and find one for it. In that way, as far as consumers are concerned, it goes from unknown to widely-used without making a stopover in early-adapter purgatory.
This may be a little unfair, but I do wonder if there isn't a tendency to consider a technology to be widely available when it becomes available to you and the folks farther back in the line don't count or aren't relevant.
To say that a single company's products being used by even a single digit percentage of the world population doesn't meet the requirements to ve considered "widely available" is a stretch.
In any case, you said "important," not "widely available," and yes, Nintendo's products are hugely important. Many of today's technological advancements can be traced back to their proving that a given use case for a primitive version of a given technology was viable.
Whether it's a single company (or product) or multiple is beside the point. If a technology is only available to (say) 1% of the population, I don't think that qualifies as being widely available.
I will also note that my original comment was in response to someone who is "more interested in tech for the other 95%".
Nah, if Google actually went for it for years it would have went down the wealth requirement. The problem is that Google has demonstrated time and time again that it cannot do reiteration grind.
And yeah, insane prices at the start funds development for everyone else, I remember when the first 4K screens came out and they where exotic, now they are just normal.
Same thing happened with phones and hell computers, I was the first kid I knew with a computer back in the 80s and we where not wealthy, that think cost more than my dad made in a week, an actual IBM PC was unattainable til I was I was 10.
Now I have hilariously more powerful single board computers shoved in a drawer because I can’t find the time to do anything with them.
New tech typically debuts at a high price point only wealthy people can afford, then as it get commoditized it becomes more affordable. Wealthy early adopters willing to pay high prices for novelty or business applications are what enable the fast pace of innovation that we have become accustomed to.
I was travelling a few years ago. Even Russian bus ladies (they collect your fare) and Mongolians living in traditional huts in the middle of nowhere (and without any signal!) have smartphones now. I found the idea funny that the "To plug in a USB cable, you need 3 tries" experience was maybe universal.
So, 12 years (back then) after the iPhone, it's reached a lot more than 5% of the world.
Also, microsoft did this a couple years ago, without fancy volumetric display, just face tracking on their expensive 8k(?) tv for meeting rooms, and it was a complete flop.
Same. I suppose they kept the decor and backgrounds stark to showcase what it's doing. But it does have that feel, as-is. At least nobody was wearing orange :)
It looks like magic, when I saw the video I briefly thought it was a joke (ha-ha, they're actually in the same room!). I wonder how much of the magic is software and how much is hardware?
The video certainly plays up the software, but I've never used zoom or FaceTime in an 8k video call booth before, so I suppose I don't have a point of comparison.
I'm not sure it's the parent's point (I suspect it is), but mine would be that it might be you get 80% of the "it's like they're there in the room" effect by having 8k; good lighting; well-separated, good quality, stereo sound; a purpose made space without distractions; etc.?
If you've only had 480/720 on a small screen, with a mono microphone with poor lighting, lag, dropouts, then just fixing those things and making it 2k+ might be surprisingly good (ie will have an emotional impact due to the truer representation)?
HP's Halo conferencing system had several studio quality cameras that would change the video feed based on who was talking. There were also specially placed mics on the conference table to keep audio crisp. The video quality was impressive, however you had the same issues that you have with current web-cam based systems. Eye-contact was non-existent. While the camera placement was above the middle-top of the LCD screen, you'd never feel like the person was making eye-contact with you unless they looked directly at the camera. I had several team meetings in Halo rooms, and for the time, it was the next best thing to having a meeting in person. It also worked well for groups of 4 on each side of the link.
From watching the video, Google's conferencing setup is creating a 3D rep of the people talking and adjusting rendered view based on where the participants are seated. This is blending AR with videoconferencing. It would be interesting to see how their conferencing system works with multiple-people on each side. I know the video had a mother and baby in one segment, however is the 3D rendering based on the eye-level of the main participants?
Why does it appear similar to that tech other than you get a 3d image? As far as I can tell there's no info on the display other than it uses "Light field technology" which would make it different than the parallax barrier display on the 3DS.
The term light field technology is broad enough to cover lenticular arrays in displays. The 3DS had the major limitation that it only rendered two views. If you increase the resolution to be able to display more views for more viewing directions within a wider cone, you already have a light field display - simply because all these views combined form a sampled light field representation by definition. This is exactly the same as glassless 3d displays for multiple viewers of decades past. But advances in display pixel density and computing power apparently make the resulting illusion much more convincing these days.
Isn't it clear glass and not a layer over a traditional display like the 3DS? Glasses would either actively time slice with shutters, or spectrum slice with passive filters on the lens and of course you need the glasses.
How could any of those technologies be what is used here?
E: Looking again, perhaps it could be some layer over a traditional screen. You see through some of the broadcasts but that could just be the digital far plane that shows through.
I'm not sure I'm following. If this is based on a flat panel display, there must be a lens array in front of it. There is no other way to achieve this effect without requiring the users to wear glasses. The lens array can be covered by a protective flat glass pane without issue.
Track both eyes, and then project an image to each eye based on its image in the room. The part I don't really understand is how it's possible to target the image to each eye. Maybe we have displays now which are like the 3DS screen, but with variable focal locations?
Use a demo mode to disable eye tracking and follow your ARcore/Vive puck located camera ?
Or just ask the guy to close his eyes and put googly eyes on your camera...
From this Wired story, link found in another post above https://www.wired.com/story/google-project-starline, there is this passage: Move to the side just a few inches and the illusion of volume disappears. Suddenly you’re looking at a 2D version of your video chat partner again.
This implies, AFAIK, that it either uses lenticular lenses (which is the tech 3D-cards typically use), or a parallax barrier (screen tech from 3DS). There are a thus sectors from the screen to the viewer, and you need to have your head placed so that your one eye sees one sector, and the other eye sees another. What the reporter describes is when both her eyes end up in the same sector, which immediately makes the result 2D. Note that there might be more than two sectors, so that you can move further sideways and get a realistic view, but each eye must all the time be in a different sector. It can also use head tracking to achieve such correction of your view wrt. movement of your head, since it evidently constructs a full 3D scene of you and the other side, it can render that from any angle.
Much simpler idea but I think I’d get a good sense of being together to play table tennis over the quest with a friend. (Sadly none of my friends will buy a quest.)
I agree I love my Quest, and the sense of presence is amazing. Can't wait until more of my friend social network has them so we can meet up and do things like table tennis.
It seems like this could be used in the future dystopian economy. Workers sleeping in little capsules next to a screen where they can choose to see their partner in another factory or AI driven escort.
On a serious note likely everyone is going to have at least one at home at some point to use for work meetings, pairing or for home inspections by the government.
I wonder what the lag is like. I can imagine that's one thing that would break the illusion. I know with something like Zoom I've gotten used to managing the lag over time by taking turns with the other person. However, with the "live" feel of this, there could be an uncanny valley effect if the lag is subtle, but perceptible.
Another thought: this is being presented as ongoing research. I wonder what the corporate thinking is in presenting it now when it's still being tried out. Does Google want capitalize on remote meetings while it's still hot? If the pandemic wanes and we have more in-person meetings, this might not make as big a splash. I remember when I worked at Microsoft, I often noticed research announcements we'd make in public often wouldn't translate to actual product, so I got a bit jaded on any cool new thing that was announced without a product timeline.
The longer the project exists and the more people there are who get to try it, the more likely it is to leak.
To speculate, here are some reasons why keeping it secret longer might be hard:
- They're going to do wider testing within Google.
- They're going to start bringing outside testers in to try it.
- They're going to start working with manufacturers.
- Some newspaper got wind of it and is about to publish a story. (I think this happened with driverless cars?)
Apple is better at keeping secrets, and even they leak.
Also, it's nice for the people working on it when they no longer have to keep what they do a secret.
Edit: although, in this case, the timing mostly has to do with Google I/O starting today.
It's advantageous to announce when you have a good working prototype so that when a competitor (e.g. Apple) announces something similar, the world is less impressed/amazed.
Zoom lag is an issue with Zoom, not the network generally. If you actually do wired, p2p on the same side of the same continent you can get rid of most of the lag. Current lag comes from services that aren't p2p and bad networks (e.g. wifi).
Yeah, I honestly hate how much Zoom has won because I've found it's the worst for latency. I have a mumble server running on a pi that blows the doors off Zoom for audio-quality and latency but it's unusable for casual groups because nobody wants to wear a headset and feedback destroys it.
Mumble beats nearly all service‘s in latency and audio quality because these two things are the problems it‘s trying to solve.
Did you ever play around with Teamspeak? It‘s a bit worse on the latency side but more mainstream friendly.
Personally I run a mumble server as well because it solves our problems but TS may be worth looking into for you.
If they wanted to capitalise on remote meetings, I assume they would have shown one in the demo video. Instead they focused on family members reconnecting.
With the amount of data needed, one on one is probably easiest. We saw teams and zoom struggle to support more people in a call last year. It’s also nice to have them be true to size. Mini people might be somewhat uncomfortable.
GOOG 411 was free slave labor for voice sample
Captcha is free OCR training first and now road sign detection and image training
AMP so that google can capture the web flow in the name of improving performance
This will be data for AI models specializing in video features of humans:
"AI" generated realistic human expressions which can then be used to do so many things
AI gait detection
aI partial face reconstruction
These all already have further applications which are harmful and will lead to tie in with Google
And in none of the other cases above has google published their compiled dataset. They "own that"
Looks like tonari[0] will have some heavy competition sooner than expected. This seems to have a lot more attention paid to the sensation of depth than the tonari offering. Could see this being popular at high end senior care facilities.
At Atlassian, they built a Portal inspired portal that connects the Sydney and SF office during one hackathon. Basically a camera behind a 2way mirror (and decorated with blue/orange LEDs in an oval frame). Tonari looks a lot more like that than Starline.
This is getting downvoted but it really is a valid concern. Google’s been losing users on each discontinuation and only the most pro-Google techies I know are still jumping in to something new, especially given the trend towards one cool feature and a bunch of “QA is boring” stuff which will take a year to get fixed. Given their past reputation, it’s really cautionary as a shareholder to hear C-level questions about services like GCP questioning the risks of not going with AWS/Azure and getting stranded.
That’s a big deal for anything which requires hardware you wouldn’t otherwise own. Once you hear “custom-built hardware and highly specialized equipment” that sounds like something you really want a commitment to not just begrudgingly patch but to continue to seriously invest in the product.
It's getting downvoted because it is repetitive, with thousands of effectively identical copies being posted on HN yearly. It is not insightful. It is not clever. It's not reacting to the story, except for lazy pattern matching. It is just a tired meme that's been flogged all the way down to the bone by now.
(It's probably also not true, when compared to Google's peers. Amazon and Microsoft similarly throw a bunch of stuff on the wall, and unceremoniously kill the failures, but neither their launches nor cancellations get this reaction.)
I imagine it will be like Google Glass. Tons of hype with the initial announcement, sobering realization that it's too expensive for mass market adoption, and its eventual settling into a niche market.
My dad used to tell a story about my grandfather's first experience with television in the late forties. There was a buxom woman on the screen, and he walked up to it, trying to look down her cleavage. It didn't work.
Does Starline give you a different view when you change your perspective? It looks like it does to some extent. If so, it might work before long, but grandpa died about fifty years too soon for it.
That would really take it to the next level if it had that feature, even with very few degrees of freedom.
The (New) Nintendo 3DS has head tracking, but it doesn't change your perspective into the view port, which gives a very dizzying effect when you deliberately test the feature.
I would imagine its possible to extrapolate perspective if they had an array (N > 2) of cameras.
This is super cool tech, and can't wait to see an array of these installed in the secret undisclosed board rooms for the illuminati.
My understanding of the existing tech that's out (3d displays) is that it allows for such on the X axis (different view per-eye) but not on the Y, so a scenario such as you described (looking downwards) might still remain sci-fi for the moment. Perhaps this does something entirely new/novel with display tech but there's nothing to suggest that at the moment.
On the topic of buxom women, my first thought about this tech was, it'll be the next frontier for OnlyFans/porn accounts on Snapchat.
Before you laugh or be prude, porn content was what made VHS and BluRay succeed (or are these urban legends?) and they were pioneers in stuff like online video streaming.
Ah yes, the people whose first thought is not how they can use this to commodify women’s bodies are “prudes”. Yes of course. Women: become literal prostitutes for the only fans corporation. You could be RICH! It’s so fun! They have t-shirts! Remember how I said you could be RICH?!
I don’t know. You should ask some of the abuse victims coming out of the prostitution industry why they’re so upset, and remind them that their job is basically the same as working at google.
The fact that you instantly associate what OP mentioned to women specifically is quite disgusting, not going to lie
FYI More people than just women suffer through that and taking the oxygen of such a discussion via weaponizing linguistics is just nasty and a horrible way to spend the opportunity cost of it
Not sure my comment will bring anything to change your opinion, you seem to be already a hardcore antiporn person, and I can't defend sex work either because yeah I know it's got a lot of seedy aspects to it; but no, they're not literal prostitutes, cam girls and OnlyFans models have the advantage that they can earn money without having to be in contact with other bodies and potential diseases they bring.
I know some girls who are prostitutes. They seem normal enough. Maybe they're wearing a facade and have hidden PTSDs, I don't know, but if that's the case, I think you might want to extend your crusade against the military.
I'm no expert but if it simulates a window, no normal cameras can capture the field of view as effectively as a lightfield. Google also had a proprietary method of transferring data, not sure if that's mentioned in the article.
This is actually groundbreaking technology. If this gets widely deployed and then evolves we're looking at telepresence as the next step in ambient computing. When the technology fades into the background that's when you know things are going to be remarkably different. This is quite honestly going to be as close to teleportation as we get.
Asim, are you familiar with how Stephen Baxter portrayed the use of telepresence in his science fiction novels? If not, check it out. But if you are, please explain it to me! It always confused the hell out of me.
It's interesting to see the compression artifacts affecting hair specifically.
I guess we're going to get used to different kinds of compression artifacts in the coming years because we're switching to spatial information being transmitted as opposed to just pixels. Hair is so much harder to get right than a face.
I feel like this is the kind of thing I'm going to have to experience in person to appreciate. The video just isn't conveying anything meaningful to me.
Still, the concept is exciting, and if the execution is there, it'll be one of the most important leaps in communications technology in decades.
And I'm looking forward to a company named something like InstaPresence (TM) applying filters and making us all photorealistic cat people.
Am I the only one that thinks one-on-one video is already fine, and what we actually need is to improve the experience so that a group of people feel like they're meeting in the same place?
538 comments
[ 3.8 ms ] story [ 234 ms ] threadI've tried it in person and it was truly amazing. They used some very fancy tech when I saw the demo, so I'm thrilled it is finally being announced and possibly shared with a larger audience.
Explanation of why it is amazing: It totally fools your perception. No glasses or goggles--but rather an 8k display with special glass that allows your different eyes to see different pixels (a light field).
They also optimize the sound, and the rest, so as all the testimonials point out, you actually feel like the person is in front of you.
It also works for the "cube" around them, so if they hold up some object, it also feels like that is in front of you.
Amazing...
https://lookingglassfactory.com/product/portrait
https://twitter.com/mizzmayo/status/1394171128491487234
https://lookingglassfactory.com/8k
If it's as seamless as it looks in the video that would be truly novel and exciting.
You sit down and you forget that there's technology happening. The person is there, in front of you. I don't know how else to describe it.
The testimonials in the video aren't exaggerating compared to my experience.
I mean, not in the same room, but down the hall or so.
You can have the most perfect rendering in the world and 100ms of latency will be enough to make the experience miserable.
https://www.google.com/search?q=speed+of+light+*+10ms
You can't do weird texture mapping or lossy compression and expect people to really seem like they are there. Even if you don't notice that stuff normally watching a video, I think you'll notice it when you're interacting like someone is really in front of you, and that will throw off the immersion.
1: https://www.quora.com/In-regards-to-filesize-how-big-is-1-mi...
That said, my intuition is they aren't doing a pure video encoding solution. The fact that they talk about 3d modeling leads me to believe they are doing a combination of model + texture to get the realistic results. That would significantly decrease the amount of bandwidth and computational power needed. Over a low bandwidth situation you'd simply need to send model updates and do some smart interpolation to determine what things should look like.
Similar to the concept that playing a 3d game requires MB of resources but recording the same game at 8k would require a boatload more memory.
My assumption is they are using LIDAR to get a good model, high quality cameras to texture things, and a nice AI to stitch things together and interpolate when data isn't arriving fast enough.
https://blogs.nvidia.com/blog/2020/10/05/gan-video-conferenc...
https://ai.googleblog.com/2021/02/lyra-new-very-low-bitrate-...
https://aibots.my/officialBlog/deepminds-ai-agent-muzero-cou...
I have 300/100mbs connection which costs 20 euros in month.
Many ordinary folk already have 4k TVs at home and 8k will probably become commonplace in the future. The real bottleneck will be good low-latency broadband both up and down, but fibre to the home should make that easier. I wonder if ISPs in the future will offer QoS guarantees to enable really good videoconferencing?
I mean, Zoom, Meet et al are much better than video conferencing solutions like Webex even from a few years ago, but it's hilarious how much drama there still is around video calls. "oh, sorry, I didn't realise I had muted myself", "can you hear me...?", "I think we just lost Steve", and so on. I'll be glad to see all of that just go away.
$20,000 is a bit out of the reach of most people, and reserves this for business use or desperate need.
I could see a bunch of execs getting this installed in their home offices as a company perk, and then using it for personal reasons too.
Even for business use a price with 5 digits would make it confined to a few executive offices.
The traditional glasses free 3d monitors rely on special coatings on the glass to "split" the direction of a pixel. Some coatings are electronically controlled (like Sony's), some are physical (lenticular filters) It still displays a 2D projection of a 3D image, but twice.
By contrast, my understanding is lightfield displays projects the entire volume. So you don't have a pixel, you have a voxel. So each eye gets a different bit of the same voxel
Sounds like a monster data rate.
> the 8K is their Input Resolution. > That resolution is then divided into the 45 viewing directions:
It is impossible for me to explain how/why it works so well.
I think this is cool tech, and valuable. I'm just not sure that it offers a communication benefit over well-lit, well-miced, wired, low latency, 8K videoconferencing.
Maybe there's some 3D emotional perception face processing stuff that we have deep in our brains that can immensely benefit from this, but I'm skeptical. I think simply doing 4k or 8k low latency high quality videoconferencing might be a 90 or 95% solution without needing special cameras/displays.
The state of video conferencing today is a poor one and I'm very excited for something that can change the industry like this.
If Google wanted to make me believe they care about videoconferencing quality, they'd have a 4k 60fps option that auto-enables in Meet if it detects everyone on the call is on wired gigabit with a 4k camera.
Even 100mbps is sufficient for a 1-on-1 4k video call, as high-bitrate 4k is 30-40mbps. Most commercial office buildings in business districts have it available. Even Starlink (20mbps up) should be sufficient for 1-1 30fps 4k videoconferencing with a lower bitrate.
Yes, having low latencies and high definition video is an important aspect of this, but the 3D part is no gimmick. Once the technology improves and gets affordable this is a game changer for how we communicate online. The step after that are holographic displays, and since we'd be used to 3D models and smart displays, it probably won't feel like such a big jump.
I'm _super_ excited about this project. Hopefully Google doesn't axe it. (:
I've got one and the first minute is always noticing how low res the eye screens are, then as soon as the game starts, I've forgotten and I'm _there_. The 3D part makes up for the low quality
>Maybe there's some 3D emotional perception face processing stuff that we have deep in our brains that can immensely benefit from this, but I'm skeptical.
>I think simply doing 4k or 8k low latency high quality videoconferencing might be a 90 or 95% solution without needing special cameras/displays.
From my experience, 4k or 8k doesn't matter. Sound quality actually matters most, really clear low latency audio alone will give you a surprisingly strong sense of presence.
Video quality is important but 1080p is enough, beyond that the lighting and latency matter more.
Equally important from my personal POV is video size - physical size. Take a cheap 65 inch TV, turn it vertically, and talk to someone on that. When your talking to someone that is actually life size the sense of presence is vastly improved, even at the exact same video quality. And TVs are so cheap this doesn't seem like much of a techical barrier.
If you just screen share from your cell phone to your 65 inch TV and video chat -- holding everything else equal for audio and video quality -- it's SO MUCH BETTER.
https://augmentedperception.github.io/deepviewvideo/
WebAssembly SIMD is coming to Chrome as well. 2D images and video that only consisted of RGB and Alpha channels may appear downright primitive to future generations as depth camera rigs gain distribution ;)
https://www.chromestatus.com/feature/6533147810332672
Partially there are just affordance issues of things like eye contact which are physically out of alignment unless you start using two way mirrors [1].
https://hackaday.com/2020/05/29/two-way-mirror-improves-vide...
Even from viewing the short demo, the stereo display alone is an entirely new dimension that no amount of studio lighting will recreate. While better lighting and audio setup would certainly improve the average person's videoconferencing experience, this looks to be a genuine step beyond.
That said, we've been seeing holographic-display prototypes for the better part of a decade, and it'll be interesting if this actually pans out or fizzles.
It seems like the imaging/rendering technology that Google is using is much more advanced.
That said, there was no “emotional connection” like the Google one is described as offering. It was still a video call. There was no forgetting that. I suspect the 3D and the apparent physical closeness to the display add a lot.
> project (he has a Phd Princeton
Why is this part relevant? Was the PhD in light field technology?
https://augmentedperception.github.io/deepviewvideo/
Then it would be done already.
The Oculus Quest 2 doesn't do anything like what you're describing -- it essentially just pipes in stereoscopic video from its stereo cameras and stitches them together in a trivial way. It doesn't attempt to build geometric representations of objects in your environment at all.
(For guardian functionality it does very simple things like using the depth cloud to figure out the height of the floor, and if there are points inside the guardian that shouldn't be there, but that doesn't inferring object geometries.)
Because optometrists are illegal in my country (here only medics can decide what glasses you can use), currently I don't have stereoscopic vision, although my brain CAN do it, if I had the correct images sent to my eyes somehow. (one of my eyes muscles is slightly shorter than the other side thus the images on that eye are shifted unless I had an optometrist design me glasses with a prism).
So when I watched Avatar, an actually well made 3D movie, it literally felt more real than reality, despite it being obvious fantasy with aliens, floating rocks and all that stuff.
EDIT: for those wondering, I am from Brazil, here medical professionals often sue the shit out of anyone offering any service remotely similar to an optometrist, they are quite aggressive about it, some attempted to make even discussion of the subject illegal. And when I was trying to get the prism and asked around my medics about it, one of them went really ballistic, I honestly thought the guy was going to punch me. I believe the reason for that is that for many medics, designing glasses is their only source of income, a guaranteed one, since here is ALSO illegal to buy glasses without a medic desining them for you first, even if the new glasses are supposed to be identical to the old ones!
I'm asking in case I might need to look for one in the future.
In places where they are legal they spend their time learning more and more about optics, physics, math and eye anatomy, they don't study diseases, infections and so on.
As result you can't rely on one to fix certain stuff where you do need a medic, but if you need fancy lenses, they will calculate them, not just use a number they get from their measurement machine like ophtalmologists do.
They also would be helpful to design proper 3D glasses and whatnot, holograms and so on.
This is a major screw up and inefficiency from this country.
Thanks for explaining the concept more in-depth too.
Something used in some countries is a briefcased-sized glasses making kit. Eyeglass lenses have three parameters - spherical radius, cylindrical radius, and axis of the cylindrical curvature. The trick is that for round lenses, you can use the same lens for all axis angles, which reduces the number of combinations to a set you can carry around. Once the right set of lenses has been decided, a notcher is used to cut a small notch on the side of the round lens so it locks into the frame and can't rotate.
Back then people even were saying it was pseudoscience or a scam, but seemly there is ongoing research that shows that the article wasn't lying.
Maybe it depends on the exact condition you have. Is it Binocular Vision Dysfunction?
>my brain CAN do it, if I had the correct images sent to my eyes somehow.
>when I watched Avatar, an actually well made 3D movie, it literally felt more real than reality
Buy an Oculus Quest 2. (Or any VR headset, but that's the best value at the moment.) It sends seperate images to each eye. You should get that same 'more real than reality' feeling. It may even train your eye muscles to see in 3D, I know I've seen some research into that area. (https://www.seevividly.com/ comes up on a Google search, though it seems to be prescription only)
Watching 3D movies in the theaters just gives me a headache but Oculus actually works fine, I've played a few games and it was amazing...
I count at least 5 waves of 3D technology starting in 1851 with the Brewster Stereoscope. Each time there's a surge of popularity driven by the legitimately amazing initial experience. And each time people slowly stop caring. People were incredibly excited about Avatar, and many thought it would change the movie industry. But how many people now go out of the way to see something in 3D?
Replicating the experience of in-person communication much more closely than video and 2D displays will ever do. That's a noble research goal if nothing else, I don't get the skepticism.
There are several reasons 3D content and previous generation displays didn't take off, but there's no reason to believe a revolutionary new approach and product couldn't change this (e.g. electric cars were invented in the 19th century and are only now becoming popular). AFAICT the real time photogrammetry they're using here along with the no-glasses 3D display is a major leap forward. If they can get it cheap and reliable enough to mass market, it would be a game changer.
I certainly know what kind of display and teleconferencing software I want when the next pandemic hits, and it's not what we have now.
That's not a problem people express much, at least not in ways where "3D" points to a solution. When I want to see people in person, it's not because of a lack of stereoscopy. I want to hug them, to break bread with them.
> there's no reason to believe a revolutionary new approach and product couldn't change this
There is indeed! In specific, the many times we have already had revolutionary new approaches and products that were met with great enthusiasm in the market for a few years.
I'd add that the telephone was not only a very successful technology for a century, audio calls still remain very popular. (I'm not sure what your work calls are like these days, but quite a lot of people turn off video in mine.) The lesson I take from that is that people mainly self-generate the feeling of interpersonal connection, and they can do it with very little in the way of cues. To me that's another strong indication that no new 3D technology will make much of a difference.
Of course, I'm not saying this will replace physical communication (I should've said "simulating" instead of "replicating"). But it's a clear step forward for traditional teleconferencing solutions. What do you think is the next leap from 2D video and displays? We're at the point of diminishing returns as far as increasing resolution goes, most consumers don't have a need for 8K or higher res displays. VR/AR is chugging along, but we're still a few generations away from mass market adoption.
> I'd add that the telephone was not only a very successful technology for a century, audio calls still remain very popular.
I don't understand. Video calls were never meant to replace audio calls, they just added a new sensory experience. It's perfectly fine for both technologies to co-exist for different moments and preferences. In a similar way this 3D approach is an extension to traditional 2D video conferencing if people have the equipment and prefer it. Judging by the expressions of the people in the demo and some of the comments here, you're underestimating how impactful this could be, especially if it's polished and cheap enough.
I don't have much reason to think there is any near term "next leap from 2D video and displays". 2D renderings are more than 40,000 years old. They have improved drastically in resolution and fidelity. Computers added being dynamic and interactive to that. it's really not clear that 3D rendering adds much.
> Judging by the expressions of the people in the demo and some of the comments here, you're underestimating how impactful this could be
I am not, because that kind of novelty-driven excitement has driven every wave of popular 3D rendering technology for 170 years. VR/AR has been close to mass market adoption for 25 years. We've just been through an unprecedented period of demand for at-home entertainment, and the hardware that many said was finally, finally the thing turns out once again not to matter.
People have had those excited faces every time. There were people jazzed about the possible impact every time. The Brewster Stereoscope. The ViewMaster (with the US Defense Department purchasing 6 million reels on the theory it would revolutionize training). 3D movies in the 1950s. VR in the 1980s and 1990s. 3D movies again this century. 3D TV for 2 CESes. And then the latest wave of VR, which you agree is still not there despite fantastic investment from companies floating in cash.
Could it be different this time? Maybe! But if we keep measuring it by novelty effects, we're setting ourselves up for the exact same failure that keeps happening.
Surely the improvements in manufacturing processes, faster hardware and better screens are partly responsible for that. The iPhone as a concept has existed since the 1980s, and revolutionary ideas like what General Magic tried to produce in the 90s were just too early to be successful. When Apple tried it again in the late 00s it was a massive success, but technology finally reached a point when it was commercially feasible.
So it doesn't take much to push a product to mass adoption. Just the right industry circumstances, a manufacturer willing to take the risk and capable hardware and software existing to make it happen.
> We've just been through an unprecedented period of demand for at-home entertainment, and the hardware that many said was finally, finally the thing turns out once again not to matter.
Are you dismissing the potential of VR/AR as well? The current innovation wave we're on is much bigger than whatever we had before. Headsets are becoming cheaper, more comfortable and accessible, and the visual tech we have now is leaps and bounds better than previous generations. Once we get to being able to put on sunglasses and experience different worlds, though likely sooner than that, the market adoption will likely go through the roof.
> People have had those excited faces every time.
I think it's different this time. It's not just it being 3D, but the merging of new generations of light field cameras, face/eye tracking, powerful ML algorithms, low latency networks and revolutionary displays is miles ahead of previous attempts. You can't just compare this to the ViewMaster and last century VR. The improvements here are much more substantial, and if they can make it cheap and reliable enough it could be a ground breaking product.
Every one of the products I named was greeted at the time exactly like you are now. The new technology was amazing! The potential was unlimited! And for the repeats like 3D movies and VR: It's different this time!
I agree it might be different this time. Nobody's denying that. Aliens might land tomorrow. What I'm saying is that because of the clear pattern of "OMG novelty! OMG possibilty!" around 3D tech that has failed repeatedly for 170 years, you can't just uncritically make the same arguments. If you want to be persuasively realistic, you have to explain why the 3D novelty effect isn't the major driver this time. Because the long evidence is that 3D displays just don't matter enough for people to stick with them.
Do you remember the Virtual Boy? Or the old cheap red/green paper glasses, and recently plastic glasses that are uncomfortable, darken the picture and give you headaches? These are all issues that better technology can solve, thus reducing the barrier to entry. A display that shows a 3D image without glasses to every viewer with a head tracking effect can potentially solve a lot of them. With similar improvements in camera technology, networks (5G anyone?), ML, etc. and all the pieces are starting to fall into place for what could be a revolution in how we communicate electronically.
Or Google might just axe it as they've done before ¯\_(ツ)_/¯ (Still bummed about Project Ara...)
Obviousness is a feeling people have about ideas in their head. Feelings can be useful or misleading. Every single person who got behind previous generations of 3D thought that it was "obviously" the next step. Many thousands of people had that feeling each time, buying in to a new platform. Great sums of money were invested by smart execs. All of those people were wrong each time. All of them. That you have the same feeling is not proof that it will be different this time. Indeed, the history suggests you should distrust that feeling.
Color video is a great example, so thanks for bringing it up. Color TV and color movies were quickly and widely adopted despite the extra cost and complexity. But 3D movies and TV have failed. Wearing a pair of glasses is not a major burden; 64% of Americans do it every day. Millions of people tried 3D movies and gave a collective shrug. The pretty obvious lesson to learn from those waves is that people are drawn to the concept but actually do not care in practice once the novelty wears off.
Another way to look at it is that people don't even care about stereoscopic vision much in actual life. Humans have a lot of mechanisms for extracting spatial information from the world, and the stereo-ness of it doesn't matter much. About 10% of people don't have it; they can still drive just fine. My grandfather, for example, was blinded in one eye as a kid, and nobody ever noticed. You can try it yourself; go out for a walk and keep one eye closed. Your 3D perception will be basically unaffected except for relatively close objects.
So sure, as I've said repeatedly, anything can happen. I'm just saying there is good reason to believe this will not happen, and excellent reason to not just assume it will. To see this not as a technological problem, but a problem of demand.
As to citations, I'm not sure what you're looking for. I've mentioned the Brewster Stereoscope twice in this thread. Ditto the ViewMaster. What do you need that isn't in the first page of Google results for those?
I will say though that people tend to assume that each new technology will replace the preceding technology (text->audio->video->VR/light-field->...), but in fact it tends to end up just supplementing the existing tech.
[0] https://en.wikipedia.org/wiki/History_of_videotelephony
I've got a hearing problem where I struggle to make out what people are saying on a phone but with a video call I can add lip-reading and visual cues which helps me keep on thread.
I hate this example, and it's like one of the most common ones on HN.
As said by thousands of people and many documentaries before me , the electric car had numerous real conspiracies working against it, some of which were the most powerful financial groups in the world.[0]
The 'electric car' wasn't made popular and possible by recent technological strides -- although it was made better.
The success and popularity of the electric vehicle was made possible by financial shifts away from petroleum exploration, facilitated by dwindling profits and increased scarcity of oil, and encouraged by a movement towards sustainability both from the social culture of the world and the various actions of government from country to country.
Yes, range has improved. Yes, the cars are more intelligent and better to drive -- but these improvements have been seen across the automotive industry since its' inception with ICE based vehicles included.
The real motivating factor behind the electric car is the environment that now exists that allows such endeavors to be profitable -- an environment that not only includes technological improvements like you hint towards, but more importantly it's an environment that fosters development of such things due to the existence of a profit incentive and increased governmental-body cooperation.
All that said, unless Cisco is even more evil than I realized (woah..), I have a hard time presuming that video conferencing has been held back by the same sort of conspiratorial under-handed back-office dealings that slowed the progress of EV adoption.
[0]: https://en.wikipedia.org/wiki/Who_Killed_the_Electric_Car%3F
I mentioned that example because the previous two comments were dismissing this attempt at 3D teleconferencing on the grounds of it being old technology with past failures. But I think we agree that it takes a certain industry environment along with a technical leap to make a technology truly popular. Even if that never ends up happening in this case and it remains a niche product, we should applaud the technical merits here instead of being dismissive.
For that matter video teleconferencing period has only just really hit critical mass even though there were videophones at the NY World's Fair in the 1960s and camera systems have been around in conference rooms for a few decades.
What really happened was that it became more or less accessible to anyone with a laptop and an even marginal network connection for basically no cost. And, oif course, the last 18 months really pushed it over the finish line if it wasn't already.
Tell me a problem WhatsApp solves. It's just a fancy SMS.
Tell me a problem X solves. It's just a fancy Y.
Etc etc
There's no such plausible story for 3D video calls. It's not like people are already demanding 3D displays for any of their other 3D stuff. The 3D first-person shooter, for example, has been around for decades. But 3D displays have never been popular despite being available for at least a decade.
Another issue here is that this is being sold as like "being there", but it's more like "being there at a jail" where you can see person but can't get close to them, can't touch them, can't hand them anything. I have immigrant friends who do calls with their parents basically daily. They do it with mid-grade consumer tech, even though they could easily afford big screens and high-res cameras. That suggests to me that image size and video quality are not as important for this market as one might think at first blush.
And yes OF COURSE I predicate buying on it actually delivering to the extent people describe it in the marketing video, it's ridiculous I have spell it out.
That to me demonstrates that, contra your initial assertion, there isn't a big market for this.
As to the last part, you've gone from "I will get it" no questions asked to what sounds like "I will get it if it checks out". But that's a big jump. You've gone from an early adopter to a mainstream purchaser. From one of those people that buys things on Kickstarter to the much, much larger group who want to see proof of value before they buy.
I think that's very reasonable, but it's exactly the kind of reasonable behavior that has killed 3D over and over in the past. By definition novelty doesn't last, so by the time mainstream purchasers might be ready, the social proof just isn't there.
But I've done a lot of customer development over the years. People say all sorts of things. The question when doing market analysis is what they'll actually do. And the better guide there is what they're actually doing , not what they say they would do.
So when you say that "everyone in immigrant communities" will buy it, I'm going to be skeptical because what people are actually doing is nothing like that. They could already move in this direction with existing tech. As far as I can tell, they aren't. If you have evidence otherwise, I'd love to see it.
I also can't find evidence of third parties competing with shared higher-quality video call setups, which is what we'd expect to see if the demand were there but the price hadn't fallen enough yet. That's the pattern we saw with video arcades and internet cafes/wangbas, for example. Wangbas are still getting by because they've shifted to gamers, who are willing to pay up for better hardware and connections (and room for team play). But I can't find mention of any similar shift for video calls. E.g. India's PCO network seems to be in rapid free-fall, not reinventing themselves around high-quality video calls. That suggests what all the other market data suggests: to the extent people want video calling, relatively low-quality gear like smartphones and laptops are in practice sufficient.
I gather it mostly solves that SMS is expensive in a lot of contexts. Personally I never use it because most of the people I text with have US phones. And the one person who doesn't, we use Facebook.
Personally I though VR glasses would take off when I had a go with them in the late 90's.
Today with remote working I am on the end of a microphone without a picture of myself or my colleagues in the chat.
Yet I am looking forward to being in the office.
I see what Google are trying to do but we have wave after wave of this. VR is a classic, if only we can solve the motion sickness!
On the family level those zoom calls with my niece are now plain telephone calls. Or WhatsApp messages. We stopped caring.
In that respect I'm very excited for Starline and related technologies--"trying them" will finally just involve being in the line of sight of one rather than having an attendant fit some goggles onto your head.
https://en.wikipedia.org/wiki/Pepper%27s_ghost
The marketing for it is pretty terrible though, because I initially came away with the same impression as you.
[0] See image here: https://www.engadget.com/google-project-starline-191228699.h...
How does this relate to advertising and the necessary surveillance to support it.
Will this "product" be set up to phone home and "update" by default. Will the price be "free".
Commercial viability. How much would someone pay for this.
It would presumably work exactly the same way as Google's existing videoconferencing hardware:
https://workspace.google.com/products/meet-hardware/
In other words, you pay market value for it, it doesn't include advertising, and Google is contractually obligated not to snoop on your content.
Which is why even Google competitors use this hardware, because the legal and technological protections are strong enough they don't worry about Google stealing their IP.
The big commercial need it turns out isn't so much realism as it is flexibility to accommodate people dialing in from different systems - phones, laptops, different types of telepresence setups from small single room to big conference rooms or even telephone connections etc.
Many years ago, we were all wowed by the life-size realism and had people come into offices for it . Nowadays these meetings have lots of people crammed in across multiple screens dialing in as they please and all the better for everyone :-)
There are demos of binocular 3d conferencing done with a lenticular display (lookingglass), although those large displays are extortionately priced (1/4 of the size of this google one is $3000...) - keeping them out of the hands of most devs.
No doubt google are doing the same, but can afford these larger displays.
You can easily find examples (and research papers) by googling for the relevant terms. Google claiming they have invented a "new technology" just shits on all those folk who dont have the publicity/funding of google.
It seems more like an art project than a tech usable in the coming decade.
https://www.youtube.com/watch?v=Q13CishCKXY
Interesting thing to note is that they don't show the participants touching the glass pane 'separating' them, whereas for that kind of situation it would be very very natural thing to do.
I guess doing so (reaching towards the 'glass pane') would make the imagery distort/degrade real fast as you would start going out of camera's FoV, which that would break the magic.
There is so much more we can do in terms of quality and immersion that we're not doing simply because bandwidth and connectivity are so low-quality and overpriced at most of our leaf nodes in the USA.
Unfortunately, even an already-set-up VR experience was too strange/unnatural for him so he never got to experience it. However, this looks easy/natural to use and set up and feels like it'd have a similar mind-blowing effect for many of the older generations, which I think is a good indicator of being revolutionary tech (assuming it can be made available/cheap enough for most people to try it out).
Edit: I’d love any recommendations of other authors people think of when discussing his books. Doesn’t necessarily have to be the same style but more just that level of quality and uniqueness.
It is 6552 words long.
Minecraft has never been demanding resource wise, I'm sure early versions had serious performance issues but running it on a cheapo laptop has always been totally reasonable - it's quite accessible (it was written in java even!)
Additionally Zoom isn't a novel technology, it isn't even particularly interesting technically when compared to other video conferencing solutions - but over the past year it's been incredibly important to a number of people.
I think the OC slightly missed the mark in mentioning "important technologies" instead of something closer to "technologically innovative" technologies or, more accurately (but less interesting of a statement) "expensive to develop technologies". Things that are expensive to develop generally aren't cheap to begin with, while things that are cheap to develop need to be cheap to compete with other market entrants and clones. Additionally hardware (a limiting factor on cost for a lot of technology) tends to get cheaper over time and that rate of change is accelerated by a large market of interest (leading to more folks deciding to try and iterate new designs).
No, Linux differed from Minix in utterly fundamental ways outlined in the correspondence between Linus Torvalds and Andrew Tenenbaum.
Software seems less like technology and more like writing. The distribution cost, once the systems are in place, is marginal. The technology part is creating the systems that enable the software.
More seriously, I disagree about software being less important because there have been very real innovations for tooling accomplished in software alone. Email is a pretty classic example - but a more modern one might be Google Cardboard which can turn your smart phone into a rather underwhelming VR headset. There are plenty of hardware alternatives but the same basic functionality was accomplished on generalized hardware.
Additionally, all this technology is only really possible due to other technology - we don't discount a new shiny computer just because it's just a dumb oddly shaped box if you can't supply it with electricity - but the costs to develop software are generally lower than hardware so I think it's fair to have a general notion that hardware is more innovative - it's just that you're conflating two different variables - cost and medium.
Maybe we could agree on email as a technology. Maybe. I think it's a stretch. I hope we could both agree that the nth email client isn't technology though. It's not adding a new capability to humanity which is how I tend to think about technology. Refrigerator - keep stuff cold. Electricity - power to operate machines and light. Computers - organize, access, modify information. etc. New JavaScript library or new game... Not so much technology.
But for hardware, it's almost always some expensive thing first. The internet was once very expensive to access, cell phones were initially very expensive, DVD players were initially very expensive, computers in general, etc.
And basically anything Nintendo pushes as a console gimmick. It's not that the tech immediately goes from research to broadly accessible, but rather that they tend to take old tech that no one saw as having profitable consumer applications and find one for it. In that way, as far as consumers are concerned, it goes from unknown to widely-used without making a stopover in early-adapter purgatory.
And I see that Nintendo has apparently sold an extremely impressive number of consoles (https://www.nintendolife.com/news/2019/11/nintendo_has_now_s...). But even if everyone only bought one console each, that's only about 10% of world population.
This may be a little unfair, but I do wonder if there isn't a tendency to consider a technology to be widely available when it becomes available to you and the folks farther back in the line don't count or aren't relevant.
In any case, you said "important," not "widely available," and yes, Nintendo's products are hugely important. Many of today's technological advancements can be traced back to their proving that a given use case for a primitive version of a given technology was viable.
I will also note that my original comment was in response to someone who is "more interested in tech for the other 95%".
Quite a bit of the technology used there seems destined to get more affordable as it's getting more widely adopted.
And yeah, insane prices at the start funds development for everyone else, I remember when the first 4K screens came out and they where exotic, now they are just normal.
Same thing happened with phones and hell computers, I was the first kid I knew with a computer back in the 80s and we where not wealthy, that think cost more than my dad made in a week, an actual IBM PC was unattainable til I was I was 10.
Now I have hilariously more powerful single board computers shoved in a drawer because I can’t find the time to do anything with them.
So, 12 years (back then) after the iPhone, it's reached a lot more than 5% of the world.
Also, microsoft did this a couple years ago, without fancy volumetric display, just face tracking on their expensive 8k(?) tv for meeting rooms, and it was a complete flop.
The video certainly plays up the software, but I've never used zoom or FaceTime in an 8k video call booth before, so I suppose I don't have a point of comparison.
If you've only had 480/720 on a small screen, with a mono microphone with poor lighting, lag, dropouts, then just fixing those things and making it 2k+ might be surprisingly good (ie will have an emotional impact due to the truer representation)?
https://www.networkworld.com/article/2258553/inside-the-halo...
https://www.youtube.com/watch?v=TC4sNztp8dk
From watching the video, Google's conferencing setup is creating a 3D rep of the people talking and adjusting rendered view based on where the participants are seated. This is blending AR with videoconferencing. It would be interesting to see how their conferencing system works with multiple-people on each side. I know the video had a mother and baby in one segment, however is the 3D rendering based on the eye-level of the main participants?
Potentially a solved problem, just fix it in post. :)
https://www.zdnet.com/article/windows-10-microsoft-finally-l...
I wonder how well it works, and how much latency (if any) it adds to the feed.
I'm also sad it isn't rolled out more generally, a very strange feature to lock behind a small-ish volume hardware product.
How could any of those technologies be what is used here?
E: Looking again, perhaps it could be some layer over a traditional screen. You see through some of the broadcasts but that could just be the digital far plane that shows through.
Track both eyes, and then project an image to each eye based on its image in the room. The part I don't really understand is how it's possible to target the image to each eye. Maybe we have displays now which are like the 3DS screen, but with variable focal locations?
See: https://storage.googleapis.com/gweb-uniblog-publish-prod/ori...
Notice how the angle of her face changes as the camera moves: first you only see her left ear, at the end of the animation you only see her right ear.
This implies, AFAIK, that it either uses lenticular lenses (which is the tech 3D-cards typically use), or a parallax barrier (screen tech from 3DS). There are a thus sectors from the screen to the viewer, and you need to have your head placed so that your one eye sees one sector, and the other eye sees another. What the reporter describes is when both her eyes end up in the same sector, which immediately makes the result 2D. Note that there might be more than two sectors, so that you can move further sideways and get a realistic view, but each eye must all the time be in a different sector. It can also use head tracking to achieve such correction of your view wrt. movement of your head, since it evidently constructs a full 3D scene of you and the other side, it can render that from any angle.
I wonder what the lag is like. I can imagine that's one thing that would break the illusion. I know with something like Zoom I've gotten used to managing the lag over time by taking turns with the other person. However, with the "live" feel of this, there could be an uncanny valley effect if the lag is subtle, but perceptible.
Another thought: this is being presented as ongoing research. I wonder what the corporate thinking is in presenting it now when it's still being tried out. Does Google want capitalize on remote meetings while it's still hot? If the pandemic wanes and we have more in-person meetings, this might not make as big a splash. I remember when I worked at Microsoft, I often noticed research announcements we'd make in public often wouldn't translate to actual product, so I got a bit jaded on any cool new thing that was announced without a product timeline.
To speculate, here are some reasons why keeping it secret longer might be hard:
Apple is better at keeping secrets, and even they leak.Also, it's nice for the people working on it when they no longer have to keep what they do a secret.
Edit: although, in this case, the timing mostly has to do with Google I/O starting today.
Zoom has won in the sense that MySpace had won, or perhaps in the sense that Facebook has won.
This too shall pass.
GOOG 411 was free slave labor for voice sample Captcha is free OCR training first and now road sign detection and image training AMP so that google can capture the web flow in the name of improving performance
This will be data for AI models specializing in video features of humans:
"AI" generated realistic human expressions which can then be used to do so many things AI gait detection aI partial face reconstruction
These all already have further applications which are harmful and will lead to tie in with Google
And in none of the other cases above has google published their compiled dataset. They "own that"
Google is cancer
[0]https://tonari.no
That’s a big deal for anything which requires hardware you wouldn’t otherwise own. Once you hear “custom-built hardware and highly specialized equipment” that sounds like something you really want a commitment to not just begrudgingly patch but to continue to seriously invest in the product.
(It's probably also not true, when compared to Google's peers. Amazon and Microsoft similarly throw a bunch of stuff on the wall, and unceremoniously kill the failures, but neither their launches nor cancellations get this reaction.)
And its inevitable discontinuation.
Does Starline give you a different view when you change your perspective? It looks like it does to some extent. If so, it might work before long, but grandpa died about fifty years too soon for it.
The (New) Nintendo 3DS has head tracking, but it doesn't change your perspective into the view port, which gives a very dizzying effect when you deliberately test the feature.
I would imagine its possible to extrapolate perspective if they had an array (N > 2) of cameras.
This is super cool tech, and can't wait to see an array of these installed in the secret undisclosed board rooms for the illuminati.
Before you laugh or be prude, porn content was what made VHS and BluRay succeed (or are these urban legends?) and they were pioneers in stuff like online video streaming.
The fact that you instantly associate what OP mentioned to women specifically is quite disgusting, not going to lie
FYI More people than just women suffer through that and taking the oxygen of such a discussion via weaponizing linguistics is just nasty and a horrible way to spend the opportunity cost of it
I know some girls who are prostitutes. They seem normal enough. Maybe they're wearing a facade and have hidden PTSDs, I don't know, but if that's the case, I think you might want to extend your crusade against the military.
[0] https://www.google.com/amp/s/techcrunch.com/2020/06/23/googl...
The device knows where the viewer's eyes are in 3D, so can display an image of the other end, as viewed from that point (within some constraint).
I guess we're going to get used to different kinds of compression artifacts in the coming years because we're switching to spatial information being transmitted as opposed to just pixels. Hair is so much harder to get right than a face.
Still, the concept is exciting, and if the execution is there, it'll be one of the most important leaps in communications technology in decades.
And I'm looking forward to a company named something like InstaPresence (TM) applying filters and making us all photorealistic cat people.