I think it's safe to say we won't be seeing truly immersive VR any time soon either. At least any that doesn't make you want to blow chunks, or has a compelling and broad raisondetre.
It's interesting to me since I've followed VR/AR for 40 years myself, and what I can say is if you look at the money and hype that have surrounded HMD's (especially in the last couple of years), the technology is still just not there, and neither are the applications.
That said, a lot of very bright people are doing very cool things with technology in the name of VR/AR. That can only be a good thing.
> won't be seeing truly immersive VR any time soon either.
Have you demoed VR on a PC, or at least PSVR? I've heard this far too often from people who have only seen a phone HMD, which are complete and utter crap (apart from the mobility aspects - which has recently been solved for Vive).
I get that there is an immediate "Wow" factor, but then I wander off and look at the real world, and think "Holy crap!".
They are good, I get that... Not great but something. I don't understand the purpose though.
The best virtual reality experiences I've ever had have been between the pages of a book, or induced by psychotropic substances.
If they can fix the latency, resolution and FOV issues, reduce the size of the headsets and make them cost $150, I can see that a certain sector of people might sit at home on the couch and do whatever they do. But moving around with those things on seems like an accident waiting to happen, even more so if they get the display technology right.
For AR, I can see it as a technology used in business a lot more than for the consumer.
That said, as one of the comments in the article mentioned... would be great to ditch my monitors and have an ultra wide panoramic coding experience. But again, that is more of a business application.
I really struggle with the fatigue aspects.
And yes you are right, the phone stuff is abysmal.
> For AR, I can see it as a technology used in business a lot more than for the consumer.
I'll be the first to criticize VR for being expensive, but the Hololens is absurd - you'd think that Microsoft would have paid attention to the Glass stillbirth. Oculus pulled off $600 devkits, which gave VR a fighting chance. AR has no fighting chance right now, not at that price, not with so few devs making apps (and many of the issues in the article still apply).
It's hardly fair to compare the technology in the Oculus to what's in HoloLens. The HoloLens is an x86 PC that fits on your head and has 5 cameras, 4 microphones and a depth sensor and lasts several hours on a single charge (which is important because it doesn't need to be plugged into a $2000 PC.)
I also haven't seen anyone get sick from HoloLens or trip over anything while using one.
I wouldn't be surprised if the price point of HoloLens is absurdly high for a very specific reason: to stop people from thinking it's consumer technology - it isn't - not yet. And if that's Microsoft's strategy then I'd say it's a smart move. As a HoloLens developer you have to focus your efforts on enterprise and practical applications. Not gimmicks, because there's no money in that.
Hololens is not for consumers. It's for businesses which benefit from any off-the-shelf AR, e.g. construction, and Hololens is very much appreciated in that field. From my POV in providing software for construction industry Hololens has huge potential. Will it become a just a fad, or an irreplaceable tool, remains to be seen.
Personally, I don't look at current consumer VR as expensive.
But this is because I have the perspective of having played around with DIY VR back in the mid-1990s. Back then, the only way you could have gotten anything -close- to what we have now, would have been to drop about $50k on hardware. Even then, it still would have been lacking.
Today, you can get a much better experience in your home for under $5k. To me, that's astounding.
Sadly, we still don't know what to do with it - beyond games and a few other gimmicky things. There has yet to be that "must have" application for VR - we don't have a "photoshop" or "desktop publishing" or whatever you want to call it application yet - at least for the consumer.
For the consumer, it's mainly about video games, and maybe some kind of "social VR" - but that's about it, for now. Those applications (except for the "hardcore") don't justify the price. I'm not sure there is an app that could.
Tiltbrush on the Vive absolutely blew my mind. I could paint in the air and use the wide brushstrokes as an immediate and intuitive 3D modeling tool. And then walk around my scuplture, etc.
I would have to categorize the experience to the same slot where building snowmen, igloos, painting, etc goes - ie. creative fun with aesthetically pleasing results. The difference is that modeling on on the Tiltbrush/Vive was so effortless I literally felt I was operating at the speed of thought and intuition, instead of figuring out what to do, then work the aspect for several minutes, then move onwards.
Consumer side I agree, I've not seen anything I would call transformative.
What do you mean that it "blew your mind?" Do you experience life differently than you did before? I hear this a lot but when I relate it to my experience of the same thing (in this case VR) I would never explain it as "mind blowing."
What demo/experience were you shown? In my case it was Elite Dangerous and Universe Simulator 2. The latter is particularly mind-blowing - within the same spectrum as "the pale blue dot" but to a far, far, greater degree. Tilt Brush has an extreme fun factor - if you own a Vive then you probably want Tilt Brush too, but I'd hardly call it mind-blowing.
The trend I've personally found with demoing VR is that the less interactive it is, the more likely it is to "wow" a newcomer.
"mind blowing" in the sense that I could sculpt a life size figure in front of me with mere digital brush strokes. To produce anything like it using analogue methods (snow, clay, cardboard, wood, etc.) would have been painstaking and would have taken hours, and would have required raw materials and dedicated workspace.
This is specific to Vive, with the short latency and the fact that I could walk around my sculpture. This is a major difference to VR platform such as Oculus where the view point is fixed.
The only time anything digitally produced has felt as tangible has been when I've ordered 3D prints of my models.
I think you need to create something on the medium to experience this exhilaration. Not just walk around someone else's scene.
Or, take the general complaint that learning 3D modeling is really hard. Well, with this paradigm is completely changed with Tiltbrush. That said, it's not yet a production ready modeler.
"Do you experience life differently than you did before?"
I think the expression "Life changing" is usually used to describe traumatic experiences more than anything positive. That said I've felt a moderate urge to purchase a VR rig ever since.
Hmm, I tried it a few months ago. Maybe the system has evolved. Did you try it on the Vive?
May I ask, what sort of 3D modeling packages are you used to? Did you try the "paint a picture in the air" approach or did you attempt to paint 3D meshes? I went first with the "painting in the air" approach but when I realized the system was effectively a hard surface modeler as well I changed the brush width to really wide and then could just paint forms. I enjoyed the fact that I could use same speed as sketching, but could actually think and create in 3D. I could paint the anatomy of a humanoid body in air. With traditional 2D medium I have to project the 3D shape to a 2D canvas, while with most 3d modeling packages the laying out of shapes is more like meticulous mechanical engineering more than intuitive creation of forms. The tilt brush system combined the fluidity of 2d medium sketching with the pleasurable sensation of actually working in 3d form.
Packages such as zbrush and Blender's sculpt mode enable fluid sculpting - but having the capability to create continuous 3D free flowing curves with the full muscular control of my entire body was such a kick. Maybe it's just me - I don't know.
Yep, tried on the Vive and my experience mirrored what I've seen since. There was another paint/modeling program I played with and it was similar. I think it was Sketchfab.
I'm just not impressed with most of my VR experiences to the point that they made a real impact on my thinking. Maybe it's cause I'm an AR developer.
Ah, perhaps this is the difference. I'm not evaluating the Tiltbrush+Vive from the point of view of experience or 3D graphics but as a tool for art - I'm categorizing it among tools such as Blender, regular pen and paper, clay, etc, and not as a "transformative experience". Thus my statement "mindblowing" should be taken in same context as "mindblowingly good pen", not "a mindblowingly new view of the world".
Thanks for clarifying. I think that's probably true for specialists generally when they encounter a new tool. I'm always curious what a "lay person" says when they mean that.
I think motion controllers and room scale make the difference. Usually it's Google Tiltbrush where it clicks for people. I think Valve's Robot Repair also did it for me. Walking over and slowly opening a drawer was really immersive.
I still think it's missing the killer app that'll make it stick. It's too low resolution for a virtual monitor and the fatigue would set in well before lunchtime.
I've seen a few very niche businesses where VR will probably change things. Room scale VR really helps with visualizing something; scale and shape. One of the car companies is using VR to prototype interiors--this replaces foam core, glue, and paint. I also think it'll do well in architectural visualization if the tools and talent to generate content are reasonably priced.
"If an elderly but distinguished scientist says that something is possible, he is almost certainly right; but if he says that it is impossible, he is very probably wrong." - Arthur C. Clarke
It's similar to the moving AI goalpost. At the end of the road, it's Matrix level wetware, though I'm sure at that point there will still be people waiting for "really, actually truly immersive".
Actually the two senses of "goalpost moving" are opposite. The first AI researchers imagined we'd be having pleasant philosophical debates with our robot butlers by now. Instead we have a network of big iron that can simulate ten chess moves more quickly than well-practiced humans can.
I think my Vive is immersive, with the definition that it tricks my brain into believing I'm somewhere else to the extent any technology short of a direct neural interface could.
You're never going to "truly" be immersed. You feel your carpet under your feet. You feel the weight of the headset. You don't smell anything. Nothing you pick up weighs anything. The list goes on. Every item breaks "true" immersion to a certain extent.
But your brain really loves to be tricked, and the visual does this rather easily. When I'm playing Valve's Xortex 26xxx, I'm in a futuristic room fighting orbs that are shooting at me. My hand is that spaceship. I don't even really care that my FOV is smaller than normal, or that it's a little blurry due to the low resolution. It's good enough that my brain is fooled. When I take off the headset, I generally have NO CLUE where I am in my room because for the entire time my brain didn't know or care.
And does it need to be "truly immersive"? No. It just needs to offer experiences that you can't find elsewhere. Try coming even close to the awesomeness of Google Earth VR, for instance.
> You're never going to "truly" be immersed. You feel your carpet under your feet. You feel the weight of the headset. You don't smell anything. Nothing you pick up weighs anything. The list goes on. Every item breaks "true" immersion to a certain extent.
Let's go with "its going to be a while" instead of never
I'm going to hold out that a Ready Player One full body haptic suit will be possible someday. Perhaps someday instead of asking about the resolution of our screens we will be talking the latest in "resolution" of haptic feedback.
Plus smell-o-vision surely will someday work.. right? :)
Uncanny valley of immersion is specifically talking about. Which is probably okay for a lot of stuff, but personally I find it "uncanny" and disorienting.
If you haven't tried the Vive, try it. Room-scale is everything. Also don't do anything in VR that forces artificial movement without employing some technique to counter the nausea (Google Earth VR does this well, for instance).
It will never be true immersion. There will always be some level of unreality to it until we figure out how to plug a computer directly into our nervous system.
Since SciFi in relation to AR is mentioned I though a recent episode of Black Mirror I watched was pretty interesting ("Men Against Fire", S3E5). I won't spoil it but the general idea was pretty interesting/food for though with regards to uses of AR.
This episode was really depressing. I am glad that this show highlights potential abuses of technology to the extent highlighted in this episode as this was not something I even considered.
Reading this article, it seems like we should be on the cusp of realizing this technology. Does anyone know if this will be hitting the mainstream in the next couple of years?
There's a section about that further down, it argues the issue then is focusing on a screen 1" from your eye and the real world at the same time isn't possible.
Not really. At least not in the simple way of having an LCD screen somewhere near the eye blocking out light from an object in a certain direction by interposing a black pixel. It seems like it should work, but it would be exactly like putting some paint on a camera lens and expecting it to block out that exact part of the picture; it won't - the lens will simply get dimmer, or if the area of paint is large, there would be darkening - vignetting - of a part of the image, with a diffuse boundary; but nothing like a sharp mask.
You can easily convince yourself that this won't work for an eye. Of course, you can use your finger to block out any object - as long as you hold your finger well away from your eye. Now take a match, and try to block out objects by holding it a centimeter in front of your eye; it doesn't work - you will see a diffuse slightly darkened area, but all objects will be visible.
But we've seen "soft" AR and it appears to be fairly popular - Pokemon Go and Ingress. If you ask players of either of those games, if they would like a "harder" - i guess this means more immersive - experience while playing these games, they'll bite your hand off.
There's also father.io, which looks pretty fun, but I haven't tested that out yet.
For me, in my uneducated almost ignorant understanding of the situation, AR has a massive advantage over VR in that it tends to not make you quite as motion sick.
But every player of Pokémon Go that really tries to play the game not just for 30 minutes disables the AR camera first to save battery and because it doesn't add value. People using Pokémon Go with the AR cam enabled are young kids (it's funny) and Asian tourists snapping photos with Pokémon in front of landmarks and tourist spots.
I second your remark on motion sickness. And VR cuts the user/player from the real world, which is a social annoyance outside your own living room.
VR also doesn't tend to make the vast majority of people "quite motion sick". Instances of simsickness are widely over blown and overeported.
We've run thousands of VR demos, both in the road and at our dedicated "arcade" space in DC. Not one person has reported anything more than mild discomfort, and that had been a rare minority.
Simsickness is anti-VR FUD. I don't know why, but there is a small cohort of AR advocates and VR detractors who would be gleeful to see the demise of VR.
The Magic Leap is rumoured to be able to create pure black:
"Such may be used to cancel light from the planar waveguides with respect to light from the background or real world, in some respects similar to noise canceling headphones." http://gpuofthebrain.com/blog/2016/7/22/how-magic-leap-will-...
Realize, that site is pure speculation and nothing has been shown or demonstrated by ML or otherwise that can "display black."
In theory it's possible to build a standing wave on the retina, but to do it with pixel precision with a vrd/fsd is beyond difficult compounded worse by form factor.
I disagree with the article that rendering black is necessary for wide consumer adoption of AR.
What makes you disagree with it? I haven't re-read this post from Abrash since it was written (so I don't remember the arguments), but I think telling us more about why you disagree can give valuable input to those of us who are not in the field. :)
To be clear his definition of Hard AR is different from consumer adoption. So really my statement should say "I don't think we need 'Hard AR'" for AR to win with consumers.
I disagree with it based on my interactions with naiive (not an epithet) consumers who are playing in AR. They are excited enough about HMD AR when shown that shadow casting and an AR version of "presence" isn't as much of a consideration whether they would adopt or not. It comes down to "does this provide value." Shadows create realism and people are flexible with realism.
That said, "casting shadows" would be a huge benefit and I think it will get there eventually but there isn't a roadmap to it yet.
The requirements for "Hard AR" are the same as those for "good VR" becasue it needs to use video so we need e.g. exact eye-driven focusing and super low latency.
make it small enough, process fast enough, and know every light source in the frame are engineering problems... ok so why is drawing black not? seems like a total engineering problem to me, one very easily solved by having lcd embedded in the display, or e-ink, or well the host of things we've already done to solve this problem. https://www.amazon.com/Hanslin-LCD-Transparent-Alarm-Clock/d...
I'm still having a hard time understanding why adding a black pixel doesn't work. I understand 0x008000 + 0x000000 = 0x008000. But if I were to add a drop of black paint to a drop of green paint it would surely change the color. So not sure where that analogy breaks down.
58 comments
[ 0.26 ms ] story [ 114 ms ] threadIt's interesting to me since I've followed VR/AR for 40 years myself, and what I can say is if you look at the money and hype that have surrounded HMD's (especially in the last couple of years), the technology is still just not there, and neither are the applications.
That said, a lot of very bright people are doing very cool things with technology in the name of VR/AR. That can only be a good thing.
So keep at it...
Nice article, thanks for resurfacing it.
Have you demoed VR on a PC, or at least PSVR? I've heard this far too often from people who have only seen a phone HMD, which are complete and utter crap (apart from the mobility aspects - which has recently been solved for Vive).
I get that there is an immediate "Wow" factor, but then I wander off and look at the real world, and think "Holy crap!".
They are good, I get that... Not great but something. I don't understand the purpose though.
The best virtual reality experiences I've ever had have been between the pages of a book, or induced by psychotropic substances.
If they can fix the latency, resolution and FOV issues, reduce the size of the headsets and make them cost $150, I can see that a certain sector of people might sit at home on the couch and do whatever they do. But moving around with those things on seems like an accident waiting to happen, even more so if they get the display technology right.
For AR, I can see it as a technology used in business a lot more than for the consumer.
That said, as one of the comments in the article mentioned... would be great to ditch my monitors and have an ultra wide panoramic coding experience. But again, that is more of a business application.
I really struggle with the fatigue aspects.
And yes you are right, the phone stuff is abysmal.
I'll be the first to criticize VR for being expensive, but the Hololens is absurd - you'd think that Microsoft would have paid attention to the Glass stillbirth. Oculus pulled off $600 devkits, which gave VR a fighting chance. AR has no fighting chance right now, not at that price, not with so few devs making apps (and many of the issues in the article still apply).
I also haven't seen anyone get sick from HoloLens or trip over anything while using one.
I wouldn't be surprised if the price point of HoloLens is absurdly high for a very specific reason: to stop people from thinking it's consumer technology - it isn't - not yet. And if that's Microsoft's strategy then I'd say it's a smart move. As a HoloLens developer you have to focus your efforts on enterprise and practical applications. Not gimmicks, because there's no money in that.
But this is because I have the perspective of having played around with DIY VR back in the mid-1990s. Back then, the only way you could have gotten anything -close- to what we have now, would have been to drop about $50k on hardware. Even then, it still would have been lacking.
Today, you can get a much better experience in your home for under $5k. To me, that's astounding.
Sadly, we still don't know what to do with it - beyond games and a few other gimmicky things. There has yet to be that "must have" application for VR - we don't have a "photoshop" or "desktop publishing" or whatever you want to call it application yet - at least for the consumer.
For the consumer, it's mainly about video games, and maybe some kind of "social VR" - but that's about it, for now. Those applications (except for the "hardcore") don't justify the price. I'm not sure there is an app that could.
Tiltbrush on the Vive absolutely blew my mind. I could paint in the air and use the wide brushstrokes as an immediate and intuitive 3D modeling tool. And then walk around my scuplture, etc.
I would have to categorize the experience to the same slot where building snowmen, igloos, painting, etc goes - ie. creative fun with aesthetically pleasing results. The difference is that modeling on on the Tiltbrush/Vive was so effortless I literally felt I was operating at the speed of thought and intuition, instead of figuring out what to do, then work the aspect for several minutes, then move onwards.
Consumer side I agree, I've not seen anything I would call transformative.
The trend I've personally found with demoing VR is that the less interactive it is, the more likely it is to "wow" a newcomer.
This is specific to Vive, with the short latency and the fact that I could walk around my sculpture. This is a major difference to VR platform such as Oculus where the view point is fixed.
The only time anything digitally produced has felt as tangible has been when I've ordered 3D prints of my models.
I think you need to create something on the medium to experience this exhilaration. Not just walk around someone else's scene.
Or, take the general complaint that learning 3D modeling is really hard. Well, with this paradigm is completely changed with Tiltbrush. That said, it's not yet a production ready modeler.
"Do you experience life differently than you did before?"
I think the expression "Life changing" is usually used to describe traumatic experiences more than anything positive. That said I've felt a moderate urge to purchase a VR rig ever since.
May I ask, what sort of 3D modeling packages are you used to? Did you try the "paint a picture in the air" approach or did you attempt to paint 3D meshes? I went first with the "painting in the air" approach but when I realized the system was effectively a hard surface modeler as well I changed the brush width to really wide and then could just paint forms. I enjoyed the fact that I could use same speed as sketching, but could actually think and create in 3D. I could paint the anatomy of a humanoid body in air. With traditional 2D medium I have to project the 3D shape to a 2D canvas, while with most 3d modeling packages the laying out of shapes is more like meticulous mechanical engineering more than intuitive creation of forms. The tilt brush system combined the fluidity of 2d medium sketching with the pleasurable sensation of actually working in 3d form.
Packages such as zbrush and Blender's sculpt mode enable fluid sculpting - but having the capability to create continuous 3D free flowing curves with the full muscular control of my entire body was such a kick. Maybe it's just me - I don't know.
I'm just not impressed with most of my VR experiences to the point that they made a real impact on my thinking. Maybe it's cause I'm an AR developer.
I still think it's missing the killer app that'll make it stick. It's too low resolution for a virtual monitor and the fatigue would set in well before lunchtime.
I've seen a few very niche businesses where VR will probably change things. Room scale VR really helps with visualizing something; scale and shape. One of the car companies is using VR to prototype interiors--this replaces foam core, glue, and paint. I also think it'll do well in architectural visualization if the tools and talent to generate content are reasonably priced.
And it really is a shame you haven't tried Vive roomscale. 9/10 people I've demoed it to all agreed that it was immersive as hell.
>The best virtual reality experiences I've ever had have been [...] induced by psychotropic substances.
https://www.reddit.com/r/Drugs/comments/3nxyei/oculus_rift_f...
;)
Please, keep this in mind... ;-)
A statement like that is so subjective. What is "truly immersive"?
I believe it is soon here, at least from my definition of truly immersive.
You're never going to "truly" be immersed. You feel your carpet under your feet. You feel the weight of the headset. You don't smell anything. Nothing you pick up weighs anything. The list goes on. Every item breaks "true" immersion to a certain extent.
But your brain really loves to be tricked, and the visual does this rather easily. When I'm playing Valve's Xortex 26xxx, I'm in a futuristic room fighting orbs that are shooting at me. My hand is that spaceship. I don't even really care that my FOV is smaller than normal, or that it's a little blurry due to the low resolution. It's good enough that my brain is fooled. When I take off the headset, I generally have NO CLUE where I am in my room because for the entire time my brain didn't know or care.
And does it need to be "truly immersive"? No. It just needs to offer experiences that you can't find elsewhere. Try coming even close to the awesomeness of Google Earth VR, for instance.
Let's go with "its going to be a while" instead of never
I'm going to hold out that a Ready Player One full body haptic suit will be possible someday. Perhaps someday instead of asking about the resolution of our screens we will be talking the latest in "resolution" of haptic feedback.
Plus smell-o-vision surely will someday work.. right? :)
I'll keep trying whenever I get opportunities.
It will never be true immersion. There will always be some level of unreality to it until we figure out how to plug a computer directly into our nervous system.
Reading this article, it seems like we should be on the cusp of realizing this technology. Does anyone know if this will be hitting the mainstream in the next couple of years?
You can easily convince yourself that this won't work for an eye. Of course, you can use your finger to block out any object - as long as you hold your finger well away from your eye. Now take a match, and try to block out objects by holding it a centimeter in front of your eye; it doesn't work - you will see a diffuse slightly darkened area, but all objects will be visible.
It seems then that the best option would be a device which affects your neural cortex more directly.
There's also father.io, which looks pretty fun, but I haven't tested that out yet.
For me, in my uneducated almost ignorant understanding of the situation, AR has a massive advantage over VR in that it tends to not make you quite as motion sick.
I second your remark on motion sickness. And VR cuts the user/player from the real world, which is a social annoyance outside your own living room.
We've run thousands of VR demos, both in the road and at our dedicated "arcade" space in DC. Not one person has reported anything more than mild discomfort, and that had been a rare minority.
Simsickness is anti-VR FUD. I don't know why, but there is a small cohort of AR advocates and VR detractors who would be gleeful to see the demise of VR.
In theory it's possible to build a standing wave on the retina, but to do it with pixel precision with a vrd/fsd is beyond difficult compounded worse by form factor.
I disagree with the article that rendering black is necessary for wide consumer adoption of AR.
I disagree with it based on my interactions with naiive (not an epithet) consumers who are playing in AR. They are excited enough about HMD AR when shown that shadow casting and an AR version of "presence" isn't as much of a consideration whether they would adopt or not. It comes down to "does this provide value." Shadows create realism and people are flexible with realism.
That said, "casting shadows" would be a huge benefit and I think it will get there eventually but there isn't a roadmap to it yet.