I remain sceptical about how ergonomic this is. I also wonder how much processing power they need to run this in real-time. Such virtual reality simulations are often still somewhat computationally expensive and sophisticated. But it actually looks quite exciting.
For me, a non-gamer, this is the killer app for any of the new head-mounted displays. I've been waiting for years for some sort of goggles I can plug into my dev machine, put on to block out all visual distractions, and get work done.
I've been waiting for the same thing. I'd also like to have N virtual monitors that I could either switch through or see all at once (like a big wall of screens).
I think strapping gear onto our heads and flying around the Gibson like Zero Cool has been a dream of many of us for a long time. Too bad the actual ergonomics of such displays, even for small trivial tasks like "get this part, it's on your left, now get this part, it's on your right and just above your head", are still shit, making it an out-of-reach fantasy.
Wait until you can start hanging with your friends when you're halfway around the world. The HTC Vive is amazing in positional fidelity and in pixel density. I tried it out.
Article is really thin on details, I'm skeptical. Last I checked, pixel density was still a big issue with HMD's and text. Contrast, as well is a problem with AR style HMD's.
I'd love it if the tech really was there though, it would be awesome. I actually think using some flavor of DLP tech and a foveated display with eye tracking might allow for phenomenal pixel density, wonder if anyone is working on that yet.
SMI are working on fundamental technology for foveated displays with eye tracking. There's also an HMD from FOVE on Kickstarter that intends to ship to backers in "Fall 2016".
John Carmack suggested on Twitter that foveated rendering won't be a performance win until a few more generations, when higher-resolution panels are more commonplace. Right now the necessary tracking and layering could actually reduce performance compared to rendering the full scene.
Additionally, every time you move your eyes, the high-detail area has to shift with them. If you did foveated rendering without low latency and high refresh rate, you're going to get visible pop-in of render detail levels.
Haven't tried it, but I'd hazard a guess that it would be really uncomfortable.
There is foveated rendering: a rendering optimization technique where you render at higher resolutions where the eye is looking at due to higher visual acuity in a small ~5-degree circle thanks to a large concentration of cones in the fovea of the eye.
This has nothing to do with the actual pixel density of HMDs. Physical pixel density remains an issue, and foveated rendering does not solve that.
edit: Perhaps I misunderstood, are you talking about retinal laser projected displays and concentrating the beam on the fovea?
This article is probably referring to Robert Scoble's live stream from Meta's office, you can see the whole video here. There's an engineer who claims to have replaced his multi-monitor setup with an AR device by Meta demoed at TED. The TED video will be released this Wednesday.
I've used http://vrdesktop.net/ with the Oculus DK2, but, the resolution was still a little low for getting real work done. I'm skeptical that this HMD would have better resolution/tracking than Oculus.
I'm hopeful the CV1 will make it at least bearable, because there are so many possible benefits (custom environments, isolating distractions, privacy, ...)
You could add multiple virtual monitors as long as Windows believes they exist (there is http://virtualmonitor.github.io/ for XPDM, but i'm not sure how well that would work nowadays).
>And even that won't even get close to 3 physical monitors in terms of resolution, simple because the geometric transforms kill your perfect pixels.
I have 3 physical monitors at $DAYJOB and i don't think i see 1:1 pixels on all three simultaneously - certainly i do for the monitor i'm focusing on, but less so on the others, or maybe only peripheral changes. So i think the HMD could get by with a slightly reduced resolution as long as the FOV accommodated peripheral vision.
I can think of a few professions that need multi at the same time, but since yoy can only interact with one at a time, most probably wouldnt be helped by this, including programmers.
Of course you can interact with two monitors at once. I do it every day.
My current standing desk setup is an MBPR15 on top of a wine case on my desk, with a ViewSonic 24" 4K display to the left, hanging in portrait mode on the Amazon Basics version of an Ergotron arm. Best dev rig I've ever had!
Quite often I'll put docs and references on the external monitor while I write code in Visual Studio or IntelliJ on the Mac's display. This works especially well with the monitor in portrait mode.
Then I leave the mouse cursor over on the external display and use two-finger scrolling there, while using the keyboard in the editor window.
If this isn't interacting with two monitors at once, I don't know what is. And it's a huge improvement over a single monitor.
I'm interacting with 3 monitors right now (well, was, then I got distracted and started reading HN).
I think the more important way to unpack that assertion though, is that if you watch people studying using pens and paper, almost everyone has a specific way they end up laying out papers and books on their desk - and the reason we still do it that way, by and large, is because you get a lot more desk real estate then you get screen real estate.
You can only interact with one at a time, but you can interact with that one while looking at another one. You don't have to look at your fingers to type, and you don't have to look at the screen you're typing into while you do it.
That was the previous version of the glasses. These are supposed to be much better, but we won't see specs until at least Wednesday, when the press embargo lifts.
Not interested. Not until they fix the eyehurt problem. AR glasses have this problem wherein they don't detect where your eyes are or should be focusing. So they throw an image up at some default focal depth, and if that image happens to overlay some of my surroundings at a different focal depth, then my eyes' focusing muscles start "hunting" and that can lead to significant eyestrain.
I experienced this while trying out some state-of-the-art Sony goggles one of their sales reps was demoing at my workplace.
>And I've solved it (you'll have to wait awhile to hear about that).
Why tease it like that? Why not just say how you solved it? There are a handful of people who have published options [1] and nobody has ripped them off.
In fact we have a whitepaper that does the same and I'm happy to share our approach which is also a VRD system based on pinlight display with some persistence of vision tricks. Quick, everyone sign an NDA!
You're right. It was an asshole thing for me to say, and I apologize. We haven't filed a patent yet, and my team has agreed on the advise of a lawyer that it would be best not to disclose the technical details. The principal on how it works is very similar to Hong Hua's fused depth plane technique[0], except we've found a way to do it that's compact and light enough to put in a headset. I've used that pinlight display prototype before, and it's one of the best AR displays I've tried. I'm really interested in what your team is working on. Is it something you plan on commercializing?
Our approach uses a separated LED array at high refresh rate, with each point source serving as something like a raster display source. Each point source is "injected" via fiber into a slotted waveguide which collimates of the light with a form of the pinlight system - though MEMS is actually a better way to go however more difficult to build. Thats the short version.
Actually we might not to go forward with commercialization - though we are working with some DoD labs so they might take it on. Hardware is a bitch, especially high risk hardware like this, and our business goals don't really align with it. We also think we have a bigger potential market with some of the other stuff we are doing with mapping and tracking within the AR metaverse (silly word but makes sense).
"The keyboard is the only thing I need, except for the glasses."
I think this loses a lot of appeal until we can throw away the keyboard. We should be able to do better. Hopefully, Google ships this silicon this year:
Now that's a step forward. Gloves in VR without force feedback have sucked. The insight here is that there are new kinds of gestures, such as touching fingers together and rubbing them sideways. They're able to sense those gestures with Doppler radar. They probably don't even try to resolve the gestures into positions; they may just throw a machine learning algorithm at the problem of recognizing the gestures from the Doppler radar outputs.
This starts to make VR look useful for something other than roller coaster simulators.
Google has a nice demo of a volume control knob. Will this scale up to a virtual mix board?
On my X Box I don't care if the image resolution is low cause it doesn't matter that much for games or video. But for work I need lots of pixels. It's going to be a few years before that will fit in VR goggles.
If I understand this correctly you can also see the real world because the image is overlayed. This would be too distracting for me. The opposite, a pair of glasses that totally blacked out the rest of the world, would be ideal for me. Put on some sound-cancelling headphones and I'm in programming heaven.
Related: Some editor users set up their background color as partially transparent and see other browser windows, etc., behind the text they are editing. This would drive me crazy, not to mention that my eyes are bad enough to begin with.
58 comments
[ 3.4 ms ] story [ 128 ms ] threadI'd love it if the tech really was there though, it would be awesome. I actually think using some flavor of DLP tech and a foveated display with eye tracking might allow for phenomenal pixel density, wonder if anyone is working on that yet.
John Carmack suggested on Twitter that foveated rendering won't be a performance win until a few more generations, when higher-resolution panels are more commonplace. Right now the necessary tracking and layering could actually reduce performance compared to rendering the full scene.
Haven't tried it, but I'd hazard a guess that it would be really uncomfortable.
There is foveated rendering: a rendering optimization technique where you render at higher resolutions where the eye is looking at due to higher visual acuity in a small ~5-degree circle thanks to a large concentration of cones in the fovea of the eye.
This has nothing to do with the actual pixel density of HMDs. Physical pixel density remains an issue, and foveated rendering does not solve that.
edit: Perhaps I misunderstood, are you talking about retinal laser projected displays and concentrating the beam on the fovea?
It's a puff piece. According to their website, they're launching something in 3 days, this is probably just a PR thing to drum up interest.
Does anyone know more about this tech?
Video stream link here: https://www.facebook.com/RobertScoble/videos/101539286551246...
Edited for link.
I'm hopeful the CV1 will make it at least bearable, because there are so many possible benefits (custom environments, isolating distractions, privacy, ...)
You could add multiple virtual monitors as long as Windows believes they exist (there is http://virtualmonitor.github.io/ for XPDM, but i'm not sure how well that would work nowadays).
And even that won't even get close to 3 physical monitors in terms of resolution, simple because the geometric transforms kill your perfect pixels.
I have 3 physical monitors at $DAYJOB and i don't think i see 1:1 pixels on all three simultaneously - certainly i do for the monitor i'm focusing on, but less so on the others, or maybe only peripheral changes. So i think the HMD could get by with a slightly reduced resolution as long as the FOV accommodated peripheral vision.
Turning your head is not going to be faster than Meta-Tabbing to another virtual desktop.
My current standing desk setup is an MBPR15 on top of a wine case on my desk, with a ViewSonic 24" 4K display to the left, hanging in portrait mode on the Amazon Basics version of an Ergotron arm. Best dev rig I've ever had!
Quite often I'll put docs and references on the external monitor while I write code in Visual Studio or IntelliJ on the Mac's display. This works especially well with the monitor in portrait mode.
Then I leave the mouse cursor over on the external display and use two-finger scrolling there, while using the keyboard in the editor window.
If this isn't interacting with two monitors at once, I don't know what is. And it's a huge improvement over a single monitor.
I think the more important way to unpack that assertion though, is that if you watch people studying using pens and paper, almost everyone has a specific way they end up laying out papers and books on their desk - and the reason we still do it that way, by and large, is because you get a lot more desk real estate then you get screen real estate.
I experienced this while trying out some state-of-the-art Sony goggles one of their sales reps was demoing at my workplace.
But it's not actually as big of a deal as it sounds, at least for most people. Crosstalk is a more common source of eyestrain and luckily easy to fix.
Why tease it like that? Why not just say how you solved it? There are a handful of people who have published options [1] and nobody has ripped them off.
In fact we have a whitepaper that does the same and I'm happy to share our approach which is also a VRD system based on pinlight display with some persistence of vision tricks. Quick, everyone sign an NDA!
[1] https://www.youtube.com/watch?v=P407DFm0PFQ
[0] https://www.osapublishing.org/jdt/abstract.cfm?uri=jdt-10-4-...
Our approach uses a separated LED array at high refresh rate, with each point source serving as something like a raster display source. Each point source is "injected" via fiber into a slotted waveguide which collimates of the light with a form of the pinlight system - though MEMS is actually a better way to go however more difficult to build. Thats the short version.
Actually we might not to go forward with commercialization - though we are working with some DoD labs so they might take it on. Hardware is a bitch, especially high risk hardware like this, and our business goals don't really align with it. We also think we have a bigger potential market with some of the other stuff we are doing with mapping and tracking within the AR metaverse (silly word but makes sense).
I don't mention half the crap I've "solved" in various handwritten pages and folios over the years, why?
Because I have not solved anything till Joe Average can just get it tomorrow on Amazon Prime or instantly via some web page. It's rude to be a tease.
https://www.youtube.com/watch?v=8hLzESOf8SE
http://i.imgur.com/MREkcju.jpg
I think this loses a lot of appeal until we can throw away the keyboard. We should be able to do better. Hopefully, Google ships this silicon this year:
https://www.youtube.com/watch?v=0QNiZfSsPc0
I'd rather have those monitors and no keyboard.
This starts to make VR look useful for something other than roller coaster simulators.
Google has a nice demo of a volume control knob. Will this scale up to a virtual mix board?
Related: Some editor users set up their background color as partially transparent and see other browser windows, etc., behind the text they are editing. This would drive me crazy, not to mention that my eyes are bad enough to begin with.