This is brilliant. I imagine remote collaboration will be very exciting with VR technology. Since I haven't tried Oculus yet, how is text rendered on the screen? Is it crisp? Or at least readable?
I tried one yesterday for the first time. It was a DK2, it was very low resolution, and you could clearly see the colored pixels of the display.
Text had to be very large, so only three or four lines fit on the screen at a time. After hearing people praise the DK2, I thought it would be basically seamless, but it was jarringly low-res.
The experience was great, very natural, but the pixel density left a lot to be desired.
That hmd has only a 40 degree field of view, as opposed to the rift's 100 degrees.
FOV is normally diagonal, so for 800x600 we have 1000 "diagonal" pixels, and for the rift's 960x1080 (per eye) we have 1444 "diagonal" pixels.
So that works out to:
1000/40 = 25.0 pixels per degree for the z800
1444/100 = 14.4 pixels per degree for the rift
So the z800 would have almost 4 pixels for every 1 pixel of the Rift
In fact, it's even less for the rift because the DK2 doesn't have a standard RGB display but pentile pixels, which means you only have the full 1920x1080 for the green subpixels, the red and blue are reduced since humans are less sensitive to those colours
Sorry, I only tried it for half an hour or so and saw that it only displayed three or four lines. 16 lines is probably doable, I just meant it can't display an entire Web page, like a screen can.
I own a DK1 and the low resolution makes text reading incredibly difficult. DK2 is a step up but text reading still strains my eyes. I think the 'sweet spot' for us coders will be when the DK4 hits market late next year (DK3's supposed to be out early/mid 2015).
OP here. The text is comfortable to read. i.e. I made it big enough to be comfortable. I haven't experienced any of the eyestrain that others have mentioned, despite spending up to 2 consecutive hours playing with this app.
However, in addition to the low resolution in the current Oculus dev kits, text rendering is definitely hard to do in VR. How do you do crisp sub-pixel rendering when your face is ever so slightly tilted? Something that will have to be solved in future software or by sheer pixel density.
That said, it's early days for VR and the future is bright!
It should be pretty simple to add collaboration with node and something like socket.io. We are working on a project using threejs, node, and socket, just no Oculus support (yet.)
Perhaps when the Oculus HD comes out. At the moment the text would be in such a low resolution that it would make it not worth the time. Pre-emptively creating an app in preparation for the Oculus HD might not be a bad idea however.
Wouldn't it be the exact same as outside the Rift, assuming the head tracking is implemented correctly? Outside the Rift, movement changes perspective. Similarly so inside the Rift.
Look into the Bates method, it's natural vision research from an ophthalmologist in the 20s. The argument is that animals (including us) naturally move neck and eyes in exactly tiny little movements literally all the time, non-stop. And that is healthy because constant movement promotes relaxation, whereas keeping your neck (and eyes) fixed leads to tension and over time is what actually increases our prescription, which he argues is just like being inflexible (that is, it becomes your new normal), but in the eye and neck muscles.
So I'm actually pretty excited for something that needs me to look around all the time. I've still gotta try making my text editor twitch about all the time. A moving target makes more sense for our predator eyes, so maybe VR will finally allow people who work with computers not to have disproportionally more need for glasses.
So I guess I'll end up with a standing/treadmill workstation because sitting is unatural, and now also twitching/slightly evasive windows because staring at a fixed spot is unnatural. I guess the next thing after that is an input device which you have to bounce on and off the screen, in order to mimic those primal hunter-gatherer spear throwing techniques.
This is absolutely what I wanted an DK2 for. To see exactly how far we are away from this application, and think about improvements for it.
My dream is a world where we eliminate physical displays from our computers altogether.
But we're definitely not there yet - it's going to 8, maybe 12K displays in a Rift to get the necessary resolution. But once it's viable, yeah, I hope to never buy another computer monitor again.
i wonder if something like this can be achieved on the opensim platform, does anyone know? I know a webkit browser can be accessed in the sim, but that's on a 2D surface. You want to 'rev' the 3d them live and in game.
https://www.youtube.com/watch?v=ubYYVnfLULI
This is why I bought a DK2, I believe VR IDE's are going to be a thing, and they're going to be great. The fonts aren't very crisp in this video, but once we get some nice editors in there it's going to rock, the new display has a high enough resolution to make it look great, and I bet the consumer version is going to have a retina display which will just make any discomfort vanish.
Note that in the video there's just one editing window, in an IDE obviously there could be editing windows and references all around you. You would never need a mouse as your head movements convey positional information, though you can use a mouse of course, and with good keyboard controls I think you can have an editing space ten times as big as the traditional dual 27" monitors give you.
No more hiding buffers in tabs, no more disruptions of flow as you're scrolling through your buffers looking for the right one. You just access the visual memory in your brain to intuively remember where you left the buffers you use.
Also, the idea that this would somehow be bad for your neck sounds ludicrous to me, how is staring for hours at a stationary rectangle with miniature information more natural and easy for your neck muscles than having information spread around you in even spacing at a comfortable virtual distance? I think it's going to be a relief for nearly everyone.
> This is why I bought a DK2, I believe VR IDE's are going to be a thing, and they're going to be great. The fonts aren't very crisp in this video, but once we get some nice editors in there it's going to rock
To a degree. But I when I work I have two WQHD monitors (2560x1440) with one turned on its side so I have one text editor with a vertical resolution of 2560. When ever I work on a laptop, even an HD laptop, I miss this setup so much as it feels like I am drinking through a straw.
Because of this I am unsure if VR, which is going to be much less than what I am working with now, is going to be a serious replacement.
> The display technology and software is still a long way away however.
OP here. Well, I'm a bit more optimistic. Oculus now has ties with Samsung's display manufacturing, relationships with Nvidia and Facebook's cash. I wouldn't be surprised if they become the driving force behind high-density 5" displays and graphics horsepower in the next few years.
John Carmack trying to figure out how to get Samsung to believe him on display technology fills me with confidence. That dude is clearly thinking very hard about how to make everyone happy to encourage technology to push forward.
At the moment, the effective vertical resolution is about 3240p, and the horizontal I'd guess would be around 6000p. And that's just the technology preview, I don't see how you think VR is going to be less than your setup, when it's already more..
Those are very strange numbers - Are you getting those figures from the ability to look around? So in terms of the 180 degrees from straight up to straight down it's 3240p?
I think pixel density is what people want - to make full use of that 3240p you would have to move your head quite considerably, to make use of a large monitor from 75cm away doesn't need nearly so much head movement.
It's what I want and am hoping for, but the current DK2 is in no way usable for this
OP here. Your numbers are definitely off. The effective resolution of the DK2 is more like 320p or something. Definitely lots of room for the tech to improve there but I'm hopeful. Despite this limitation, the experience is pretty darn good if I do say so myself.
Why 320p? I say 3240 because the height res is 1080p, and I think you can comfortably tilt your head to get three times that height resolution. The horizontal guesstimate I made similarily.
edit: ah I see, I hadn't factored in the warping of the optics, if you're correct then the effective resolution would only be around 900p so I'm being too optimistic :)
Oh you're factoring in head movement. I see. Well you still haven't accounted for the warping that the optics do. They stretch that 1080 pixels across a 100 degree vertical field of view, so the pixels directly in your sight (not including peripheral vision) is really only about 320.
I'm happy with a single 24" monitor with one split screen.
There is a virtual space 10 times bigger than 27" monitor, but it's only in my head, and I have no need to see it in 3D space.
But that's because I work in this space every day, I know all filenames by memory, I always know exactly where I wanna go, and type cmd+t followed by filename automatically.
This VR idea is like flying through places, but you don't need that if you can teleport.
I can't imagine this being useful in day-to-day programming, but definitely would use it for discovering an unknown project.
> This VR idea is like flying through places, but you don't need that if you can teleport.
It's not, didn't you watch the video? You don't fly, you sit still. Even if you would want to move your virtual presence, I'll guarantee you that the computer can teleport you faster than your brain can, not that I think that would be comfortable..
Anyway, I'm glad that you're happy, so by all means stay happy.
I question the assertion that the computer could teleport you somewhere faster than your brain could. I mean, I agree if you want complete sensory teleportation. At least my brain does a poor job of transporting my eyes and ears someplace. However, if I just want to think about one thing while I am doing another, it is hard to beat just thinking about it.
And then there is the problem of you brain can think about things that aren't actually done, yet. Computers can certainly help in this regard, but they are more about sharing something that was done in a different way. Consider walking through a house. Sure, a computer can let you walk through one that hasn't been built physically, yet. I'd wager skilled architects can imagine walking through one that hasn't even been built virtually yet.
And, finally, there is the problem of thinking about data structures that are literally gigabytes in size. I have a hard enough time thinking about ones that are megabytes in size. I'm not entirely sure accurately visualizing large instances would really even help.
Lol bro I think you missed the point he's making with flying. It's an analogy.
once you're familiar with your dev environment, you don't need to have it floating around in VR space. Its all memorized pretty much, instant access, L1 cache. so to look at something in VR you turn your head and the view pans. To look at it in your brain, its just there, instant lookup.
one requires a transition in space to access information (flying) the other does not (teleportation)
> how is staring for hours at a stationary rectangle with miniature information more natural and easy for your neck muscles than having information spread around you in even spacing at a comfortable virtual distance?
Low-pixel displays can't replace high-pixel displays (yet). It's all about the speed your head can move, vs. the speed your eyes can move, vs. the speed your display can refresh.
As a simple test, try holding your head still while moving your eyes back and forth ten times, from one extreme of your FOV to the other. Then try moving your head back and forth ten times to see things beyond your FOV on each side. Two observations when I try this: (1) the eye movement was much faster than the head movement; (2) the head movement was much more tiring.
In fact, eye movement is the fastest thing we can do, and even uses special circuits to reduce latency:
"Saccades are the fastest movements produced by the human body. The peak angular speed of the eye during a saccade reaches up to 900°/s in humans; in some monkeys, peak speed can reach 1000°/s. ... Under certain laboratory circumstances, the latency of, or reaction time to, saccade production can be cut nearly in half (express saccades). These saccades are generated by a neuronal mechanism that bypasses time-consuming circuits and activates the eye muscles more directly."[1]
Head movement apparently maxes out at 500°/s or so, and that's not comfortable to keep up for long. So it's at least twice as fast to access high-resolution information by eye movement as to access the same information at lower res by turning your head. That's the tradeoff between using a fixed monitor at your max comfortable resolution, and using a lower-res head-mounted display.
At the same time, a fixed display (for now) allows you to move your head more rather than less while reading comfortably. A head-mounted display has to update the display in response to your head movement, and a fixed display doesn't.
As a test here, see how fast you can move your head back and forth while reading text on your fixed display. Then see how fast you can move back and forth while reading on your DK2. The second will be slower, because the Oculus has to impersonate a pixel on your virtual display by juggling which physical pixels are used to render it as your head turns. In addition to further throwing away resolution by aliasing your virtual display onto your real display (the virtual text has to look sharp regardless of how it happens to overlap with physical pixels), this means the refresh rate is a limiting factor on how fast you can turn your head and still read. The text which was crisp while your head was still is physically blurred before it even gets to your eye, because the Oculus can't quite keep up with where it should be. That stops being a factor at, say, 1000hz, but we're not there yet.[2]
So on a fixed display, you can actually move and stretch your neck and turn your head without affecting the reading experience; on a virtual display, those movements will blur the text you're trying to read.
I'm very excited about all of the things VR can be -- just saying that one thing it can't be, yet, is a replacement for a high-resolution monitor when you're trying to quickly access lots of text.
I was just thinking of something like this the other day. I was kind of lamenting the fact that my phone has the same resolution display as my laptop (1080p), and newer phones have even higher resolution screens, yet I can't really get much work done on one because I can't physically display much text (though I have actually written a decent amount of code from my phone over SSH). Then it occurred to me that if we have VR mounts like Samsung is doing for the Note 4 with the Gear VR we could actually use the super high density screens on our phones for productivity.
Though as the other guy brought up, I guess it may not be the most natural way to read text if you have to actually move your neck, but it may be decent enough that you could adapt to it. And thus, you could write code on the train/bus to work from your phone (at least, until someone mugs you and steals your expensive phone VR setup heh).
While there's a lot of talk here about the future of IDEs, difficulties with current hardware and other challenges, I'd like to take a moment to talk about what's actually in the video.
This fills me with almost childish glee. This is a virtual reality, where you can pop up a "god window" and create/mess with a world around you. Even just what's in the video, making some cubes move and change colour is wonderful. It's fun!
Remember the early bits of amazement at programming? Where you could make the computer say your name or draw a square? How much cooler would that be in VR?
Did you ever catch a look at (or play with) OpenCroquet (which then morphed into OpenCobalt and... kinda stalled, from what I can gather).
Croquet was this (but without the Oculus) in 2007. I miss it, but it went nowhere, but.. that's Smalltalk for you - seems to scare people for some reason.
The thing is though you can do this today without the VR glasses. Sure the VR glasses offer a coolness factor, and I am excited for that, but I don't see the revolution here yet.
My problem is that it lacks imagination. Let's just take our text editor over to the VR world. Not come up with a new method of interacting with code utilizing VR.
> The thing is though you can do this today without the VR glasses.
Yes, and I think live coding is pretty cool.
> but I don't see the revolution here yet.
Neither do I, but I don't think it needs to be. It certainly didn't seem to be sold as that. That was part of the reason for my post actually, that people are alternately seeing this as the next big thing or falling short of some incredible goal. It's neither, it's just cool.
> My problem is that it lacks imagination.
Well feel free to create something else incredible, and I'll be happy to find that awesome as well. This doesn't hold anyone back, it doesn't stop anything and it isn't being billed as TheNewWave(tm), it's just an interesting experiment combining a few different bits of tech together.
It's a drastic shame to see people poke holes in something just because it isn't world changing.
You don't need to down vote if you disagree. I was contributing to the conversation. Either way, in my opinion it's valuable feedback. If I was working on something like this, I'd want to hear what people really thought. Not useless cheerleading. I did not poke holes - I gave suggestions. Look for alternate methods of input not just rehashing the same methods from a desktop.
Also I am working on something on my own. Which is why I thought I'd give feedback. I encountered the same issues myself.
I'm not sure if that was aimed at me, but if so you can't downvote replies to your own comments (at least I can't, maybe there's a higher karma threshold where you can but I doubt it).
If it wasn't you I apologize. I have no problems being down-voted if I am out of line but it is annoying when people just down-vote because they disagree.
I agree with your overall sentiment, but I also have an observation: I have the DK2 -- and while I haven't had a chance to play with this yet, I have been playing a little bit of the Elite: Dangerous beta.
The revelation there, is that working headtracking makes working with a huge "display" feel very natural. You can only look at one (or a few) things at once anyway -- but with head-tracking you don't have to worry about lining up five 30" screens in order to keep track of everything -- having it all (sanely) mapped out in 3d actually works just as mundanely and boring and well as you'd expect.
And that in itself is kind of a revelation. Suddenly you have almost unlimited screen real estate, that is even more naturally and intuitively accessible than multiple desktops (and I'm generally a great fan of xmonad-style tiling, no-full window managers with easily accessible multiple workspaces).
Another "wow" factor for Elite was the fact that what seems like a kind of silly gimmick on a single screen -- that "holograms" pop up with info when you look at them -- actually works in VR.
The one thing they did botch in Elite in the previous beta, was font-size -- the DK2 leaves "half-hd" per eye -- and that's not really enough to read "fine print". They sort-of fixed it in the new Beta just out -- but now the UI for space docks is "too wide" -- the text is more readable, but the "holo" should've been placed "further away" from you, so you don't have to move your head from side to side to read a line of text. I've yet to try it plain 2d -- I suspect it works fine on a regular screen.
Anyway, the point is that even a completely boring an unimaginative use of VR for programming can actually be a minor revolution. Fully working gloves/hand tracking would be better -- maybe with just a "dummy" keyboard for touch input (I'm not convinced "drum on a tabletop, visual-tracking keyboard" is actually that usable for touch-typing -- at least not when your back-tick is at shift-two-places-right-of-zero and opening-curly-brace is at meta-7 etc -- as it is on a Norwegian layout).
Finally -- immediate feedback certainly is nice -- and this is where the "big screen" effect of VR can really help. You can be effortlessly aware of both your evolving 3d model, and the text-representation of your code -- without having to have two screens, or switch between virtual workspaces.
Thanks for this. I am really excited for the DK2 to arrive. I ordered it a bit late but it should ship soon. I did try one at PAX Prime, it was outstanding.
You make some good points and I'll have to investigate further once I get the headset. Currently using the SDK and it just isn't the same.
This is pretty much how I imagined Notch's sadly abandoned space game project that had a programmable CPU as the heart of your ships. I hope someone takes over his torch on this, I'd love a fully fleshed out programmer's game. Imagine using such a thing in programming class as an introduction.
Reminds me of Worldbuilder [1], a scifi short about a VR interface for editing the world around you.
I kinda like the "grab object & kiss to save" gesture: it wouldn't make sense in 2D but feels natural in VR, although "grab & push object in your brain" might be easier implement to represent a save function.
For a year or so I worked as a contract programmer on the inworld content team for Second Life. I didn't have VR, and the language I programmed in (LSL) was even worse than Javascript, but it was still incredibly fun to use code to shape the world around me. It's even more fun when you can do that in a shared space and see others interacting with your creations.
Someday there will be something like Second Life with Oculus support and a decent programming environment, and that will be awesome.
InWorldz is working on it, or at least talking about eventually adding both Oculus support and another language. They've already made considerable improvements to LSL and to the grid and viewer software.
Second life looks like a great game that I would never play. Most people on it seem to be only interested in the social aspect, but it's pretty awesome that you can create programmable objects and sell them.
While watching the video, I kept hoping he'd look down at the cubes and "jump down" to them. Like, enable gravity and jump down onto the cubes as a player. That'd have been awesome.
So cool! Great work. Might be nice to keep the window as more of a HUD with some transparency so you can move around and still see the results while you code.
This is really impressive and I'm interested in trying this.
But... the video shows a very specific example of programming where you are basically designing a map or a level. As soon as you are programming something else than a simple 3d game, it becomes less useful.
The major problem we are trying to solve here is managing windows, but I argue that normal screens already do that and you don't need VR.
A large screen or two plus using alt-tab and virtual desktops can completely and better replace virtual screens in VR. You have to move in the VR world to access information. On the other hand using a normal desktop where the information is represented abstractly( where in VR it is a physical object ) can be accessed instantly with a simple and fast key-combination. I simply don't see which programming activity could you do better in VR than in a normal desktop.
OP here. Yeah, I was definitely not aiming for a full IDE here (obviously). It more like a playground environment. Something like CodePen for VR with a focus on the live-coding experience.
Using this app is comfortable but VR will definitely need higher resolutions for anything more serious. It's not too far in the future though. If things go well, I expect VR will be the driving force behind ultra-high-density 5" screens in the next few years. Or some entirely new display tech (retinal projectors?) will replace it.
I agree with Brian on this one, like I told him recently, IDEs were definitely next on my side project radar. I tried out the Gear VR and it's only about 1.5 times DK2's resolution (in both vertical and horizontal resolution) and it made a huge difference. At Oculus Connect I felt like a lot of the talks were about exactly this that we finally have a use case for way higher res screens instead of incremental demand that is only slowly cranked up by vested companies like Hardware Manufacturers and really early adopters.
I don't think this will replace your big screen(s) setup any time soon. That said, think about the mobility of a setup like this. Your psuedo-physical dev environment can move with you. I don't know if that's a good thing or not, but I'm certainly interested to see how it evolves.
It is trivial to implement that functionality in today's desktop mode without separating you from the rest of the system.
I'm trying to think of a good example where VR could be used but just can't.
It will probably be fantastic for stock brokers where their information relies on quick glances over a lot of realtime graphs/tables and if you look at their monitor setup today( +4 monitors ) the extra space will work for them. But when you code you are heavily centered on a single window and the rest doesn't really matter.
"""But when you code you are heavily centered on a single window and the rest doesn't really matter."""
For times like this you might want a completely black/white background, a forest clearing with very gentle soundscape or a quietly psychedelic lightshow/music all centered around your code window.
Some people's VR world will be serene and spare and some people's will be chaotic and full of the energy of a city street. It will be up to the user.
Some people will use the fact that everyone's brain benefits from both of those extremes at different times and use that to improve their cognition.
Your example is very idealistic. As a programmer you have to know how to concentrate, it is true that things like music( especially ) and surroundings help, but VR is overkill.
A company would never buy an VR rig for every employee just to maybe boost their productivity by an unknown margin. Also the learning curve would set them back by an unknown margin and some might have problems adapting.
Using your example, VR is useful, but only as an edge case where VR measurably help a person.
Only if you include the learning time and the lost productivity in that cost. In reality it will never happen.
Think of it as switching every keyboard in the company for a Dworak version. That will not meaningfully affect your average programmer's productivity, as their typing speed gained from a faster layout gives only diminishing results. Dworak only makes sense for a programmer that really benefits from >100 words per minute, and most really don't. The bottleneck it the programmer, not the input. The same goes for VR, unless you have a special case as I have mentioned before.
Let's say you type 300 lines of 80 characters of bug-free(!) code per working day of 8 hours. That is an pessimistic estimate( for my theory ), average is much lower.
Let's assume the average typing speed is 60 words per minute. That gives you 83% of the time not spent typing(!). Doubling the speed to a very fast of 120 wpm, only gives you 92%.
Here is one example: eye tracking could completely eliminate the need for alt tab or switching desktops. Put your keyboard's focus in a window by looking at it, and have it automatically recenter into your natural field of view. Still doable in 27" but not nearly as useful.
Just guestimating, I would need 11 or 12 27" screens to fully utilize the area I can see without moving my head. With gentle neck motion included I think that number triples or quadruples.
Let's assume; VR is much better than alt-tab and clicking at the taskbar. It will reduce the time taken by 95%. Assume 5 seconds average for non-VR method, which is a conservative estimate, this gives us 0.15 seconds for VR.
If you switch windows every 60 seconds, this gives us an improvement of ~8%.
Table:
300s -> 1.5%
120s -> 4%
60s -> 8%
30s -> 16%
15s -> 32%
I have no idea what my average window switch time is. I should measure it sometime. I doubt it is under 30 seconds where the VR actually makes a meaningful improvement.
If you take an average of 2 seconds for non-VR and a skilled user with keyboard shortcuts and virtual desktops, VR advantage becomes insignificant.
Table:
300s -> 0.6%
120s -> 1.5%
60s -> 3%
30s -> 6%
15s -> 12%
If you compare a specialized window managing software versus the VR equivalent then I don't see any benefit towards the latter. The latency is the same. The only difference is the abstract windows management versus spacial of VR.
I think programmers can handle the non-VR solution just
fine. I really hope it becomes better though; the resolution is currently just too small.
I don't know I could see a scenario where you have an object moving around in space and you grab a special developer handle and a code window pops up with its behaviors. then you modify it and it starts moving around again.
Right, that falls into the very specific case which is the level/map making for 3d worlds, which usually isn't programming but more design and scripting.
Any suggestions for actual programming? You know, like; classes, boring code, loops, i/o, algorithms?
This is really good because I have a stomach condition that means being in a seated position is very painful for me, but using this I can stretch out on the bed, with a keyboard resting on me, and still code :)
if it's < 10 minutes you could remove your headset temporarily, or you could leave some space between your face and the oculus rift to glance down. You can also mount a fisheye webcam on your head and transparently overlay it on the world when needed.
OP here. You definitely need to be very familiar with your keyboard for this to be a fun experience. Even switching to a slightly different key layout would throw me off. As long as you're a proficient touch-typist, it's really not a problem.
To see your keyboard, I can imagine a few hacks but all of them are a bit awkward:
- Have a "keyboard cam" that only shows you the keyboard (and your hands when they're over it) displayed at some natural spot in the VR.
- Have a keyboard with capacitive sensing keys, reproduced in the VR with keys highlighted as a function of the proximity or contact with your fingers.
- Something like this [1]. Typing in thin air or on a table probably wouldn't work very well, but if the leap motion senses the keyboard as well as your hands, and the whole thing is reproduced in the VR, that might work.
Some combination of voice recognition, gestures (e.g. space, tab, return), and a virtual keyboard is needed. Don't think people want to be confined to sitting in front of a keyboard at a desk. It's not going to be easy.
It doesn't yet support "live programming" like this, but the hope is the framework would eventually be used to easily make these sorts of things.
So far, the focus has mostly been on the user input side. There is a constraint-based system for defining input values from different "devices", i.e. keyboard, mouse, gamepad, head mounted display motion tracking, arm motion tracking, etc. There is even a rudimentary gesture system, where things like a nodding gesture or head shaking gesture can be wired up and used to activate commands just as easily as a mouse click or key press.
The framework also supports multi-user environments, and I've already started building the text input controls (somewhat similar to what is shown here) to do things like this (right now, it's only useful for live chat). Eventually, I'll have buttons and text boxes and togglers, etc.
If you're interested in this sort of stuff, please consider joining me.
OP here. I like where you're going with that. If you weren't already aware, you definitely need to integrate WebVR. A couple of devs from Mozilla and Google are adding VR APIs to Firefox and Chrome.
I haven't seen this mentioned in the comments here: I think this is huge for CAD/CAE. I can see this disrupting current [Digital Mockup](http://en.wikipedia.org/wiki/Digital_mockup) offerings.
Everything from design, analyses, testing moves into the VR environment. Already CAD/CFD/CAE computational models are saving huge costs to companies. Throw in VR and we are that much closer to a tangible, collaborative environment. Very very excited!
Are there any players in the high end VR market for that sort of work that look like they'll be better than the consumer oculus? I'm thinking that a very high resolution screen would be the main thing a high end device could buy. Maybe good inertial measurement units as well but I'm not sure how much that would help over oculus style positioning.
Eye tracking would be a sweet enhancement to an interface like this. If you could just target something by looking at it (without having to turn your head) press a key to use/select it and then use some other keys to jump through the various pieces of code that created it, I couldn't even imagine how much time would be saved in a large project.
be careful getting excited over demos of technology that add a layer of abstraction from the tech direct
ever seen a photo of a painting?
ever read a book you loved and your friends hated?
i'd suggest you try the oculus as you would use it: in your home on your box; before spending any money on the device
also, you are blind to the keyboard with the headset on
i feel oculus caries a mythos around it that is very difficult to obviate if you have yet to have worn one..
i'll link to an old comment i posted to carmacks keynote at the first ocucon : https://news.ycombinator.com/item?id=8347450
essentially i have stopped referring to oculus as vr,
oculus is digital stereoscopy,
and being stereoscopy the oculus is exploiting an optical illusion and as such, i believe, a flawed direction for the touted outcome: virtual reality
the experience failed to deliver on my expectations,
though at the time i thought my expectations were practical and rooted in evidence,
of course now i realise those expectations were irrational and that evidence was secondary being derived from this video demo
You have commented elsewhere in this thread with this same rant but I simply don't understand the point you're trying to make.
Of course stereoscopy is part of what the oculus does (although there's also head tracking, so talking only about the stereoscopy is misleading). You say it's a flawed direction, so what's the right direction then?
it is my belief that oculus will sell a lot of devices largely by manipulating the ad copy,
resulting in frustrated people out a lot of money,
i suppose i am attempting to include another perspective on the device,
one that i failed to see before i bought one,
and have yet to see since,
but feel confident in my understanding of it
i suppose it depends on your definition of virtual reality..
but i think the verbiage of vr is very misleading
my position is that if oculus was called: oculus digital stereoscopy; the tech would lose a lot of its hype
as for the tracking it is certainly the best feature of the headset,
again with the goal of calling the thing what it does i refer to the 'head tracking' as 'inferred spherical screen encapsulation'
what the head tracking is doing is windowing your vision to the size of a fisheyed cell phone screen and then inferring your position in a 3d space and displaying the contents of that field of view on the screen,
refreshing the content of the image in regard to the movement of your neck,
giving you the impression that you are in a sphere of flat displays looking through fisheye glasses with blinders with a forced forward stare
when i put it on the restricted periphery and the inability for me to use my eyes to look around was the most dramatic realisation
but that is just dismantling the vr hype,
actual stereoscopy is flawed in its own ways..
peoples eyes differ in distance apart,
the oculus is solid plastic,
certainly the error is measured in mm to fractions of mm but that is still error that your brain now is required to correct,
also the goggles press against your face tightly to seal out distracting ambient light,
this hurts after some time,
and in the case of my watching a film,
completely depleted my eyes of blood leaving white bags under my eyes
that is the hardware
as for the software if you read the comment i linked in the comments you are referring to i talk about an experiment i performed when i first received the oculus that shows that the brain error corrects the inconsistencies between the eye inputs at a pretty substantial distance,
talking here of the blue and pink box experiment,
in the experiment only the pink box moves,
but when i showed it to a friend thae asked why when the two boxes began overlapping did the blue one start moving as well?
i watching this all cloned on the screen in front of me explained that it still was the case that only the pink one moved and the blue one is being moved around by thaer brain,
so if a simple experiment has my brain working overtime shifting still objects it would seem reasonable that a lot of the 'jitter' that is so often talked about when displaying entire worlds is your brain organising its inputs
as for what i believe is the right direction?
if we are talking about visual input manipulation.. i'd think projection, perhaps onto a dome or disc encapsulating the entire eye range including periphery of the eyes even with the pupil extended to the outer most regions of the eye
or one fantasy device i dreamed up used this recent research: http://newsoffice.mit.edu/2014/magnetic-hair-directs-water-f... ; if you watch the video, at the end, 57s in, you can see these hairs bending optics,
what you'd do is have two rows of these hairs, one for each eye and each row bends to follow the movement of a pupil and each hair acts as a pixel
this way simulates two photons bouncing off a single object and reaching the eye, that would be wild
but mind that i referred to the effort as visual input manipulation, if you want to talk vr i think the only possible direction is bypassing our sensors and interacting directly with the processors, a la the suit in Zero Theorem : http://cdn-static.denofgeek.com/sites&#...
I don't understand your argumentation against stereoscopy. It is how you see the world. Even your suggested "dream device" would be stereoscopic device. You can't run away from it. Besides even if the device something like you suggested will ever be developed, there will be those few guys who will complain how it is not real VR and how you need to send signals directly to the optical nerve as well as spinal chord and all other senses in order to achieve "real VR". These kind of talks are unproductive. Better try to develop your version of VR instead of complaining about current best.
If anyone played Myst, you may be tickled to realize that this is in fact what Myst was about.
The hand-wavy "this might look like a book with a video in it and some neat drawings, but it actually contains a magical language that describes the world you're in" ...
... you're looking at a book written in a magical language that describes the world it's in.
128 comments
[ 5.3 ms ] story [ 183 ms ] threadText had to be very large, so only three or four lines fit on the screen at a time. After hearing people praise the DK2, I thought it would be basically seamless, but it was jarringly low-res.
The experience was great, very natural, but the pixel density left a lot to be desired.
FOV is normally diagonal, so for 800x600 we have 1000 "diagonal" pixels, and for the rift's 960x1080 (per eye) we have 1444 "diagonal" pixels.
So that works out to:
So the z800 would have almost 4 pixels for every 1 pixel of the RiftIn fact, it's even less for the rift because the DK2 doesn't have a standard RGB display but pentile pixels, which means you only have the full 1920x1080 for the green subpixels, the red and blue are reduced since humans are less sensitive to those colours
However, in addition to the low resolution in the current Oculus dev kits, text rendering is definitely hard to do in VR. How do you do crisp sub-pixel rendering when your face is ever so slightly tilted? Something that will have to be solved in future software or by sheer pixel density.
That said, it's early days for VR and the future is bright!
So I'm actually pretty excited for something that needs me to look around all the time. I've still gotta try making my text editor twitch about all the time. A moving target makes more sense for our predator eyes, so maybe VR will finally allow people who work with computers not to have disproportionally more need for glasses.
My dream is a world where we eliminate physical displays from our computers altogether.
But we're definitely not there yet - it's going to 8, maybe 12K displays in a Rift to get the necessary resolution. But once it's viable, yeah, I hope to never buy another computer monitor again.
Note that in the video there's just one editing window, in an IDE obviously there could be editing windows and references all around you. You would never need a mouse as your head movements convey positional information, though you can use a mouse of course, and with good keyboard controls I think you can have an editing space ten times as big as the traditional dual 27" monitors give you.
No more hiding buffers in tabs, no more disruptions of flow as you're scrolling through your buffers looking for the right one. You just access the visual memory in your brain to intuively remember where you left the buffers you use.
Also, the idea that this would somehow be bad for your neck sounds ludicrous to me, how is staring for hours at a stationary rectangle with miniature information more natural and easy for your neck muscles than having information spread around you in even spacing at a comfortable virtual distance? I think it's going to be a relief for nearly everyone.
To a degree. But I when I work I have two WQHD monitors (2560x1440) with one turned on its side so I have one text editor with a vertical resolution of 2560. When ever I work on a laptop, even an HD laptop, I miss this setup so much as it feels like I am drinking through a straw.
Because of this I am unsure if VR, which is going to be much less than what I am working with now, is going to be a serious replacement.
The display technology and software is still a long way away however.
OP here. Well, I'm a bit more optimistic. Oculus now has ties with Samsung's display manufacturing, relationships with Nvidia and Facebook's cash. I wouldn't be surprised if they become the driving force behind high-density 5" displays and graphics horsepower in the next few years.
I think pixel density is what people want - to make full use of that 3240p you would have to move your head quite considerably, to make use of a large monitor from 75cm away doesn't need nearly so much head movement.
It's what I want and am hoping for, but the current DK2 is in no way usable for this
edit: ah I see, I hadn't factored in the warping of the optics, if you're correct then the effective resolution would only be around 900p so I'm being too optimistic :)
There is a virtual space 10 times bigger than 27" monitor, but it's only in my head, and I have no need to see it in 3D space.
But that's because I work in this space every day, I know all filenames by memory, I always know exactly where I wanna go, and type cmd+t followed by filename automatically.
This VR idea is like flying through places, but you don't need that if you can teleport.
I can't imagine this being useful in day-to-day programming, but definitely would use it for discovering an unknown project.
It's not, didn't you watch the video? You don't fly, you sit still. Even if you would want to move your virtual presence, I'll guarantee you that the computer can teleport you faster than your brain can, not that I think that would be comfortable..
Anyway, I'm glad that you're happy, so by all means stay happy.
And then there is the problem of you brain can think about things that aren't actually done, yet. Computers can certainly help in this regard, but they are more about sharing something that was done in a different way. Consider walking through a house. Sure, a computer can let you walk through one that hasn't been built physically, yet. I'd wager skilled architects can imagine walking through one that hasn't even been built virtually yet.
And, finally, there is the problem of thinking about data structures that are literally gigabytes in size. I have a hard enough time thinking about ones that are megabytes in size. I'm not entirely sure accurately visualizing large instances would really even help.
once you're familiar with your dev environment, you don't need to have it floating around in VR space. Its all memorized pretty much, instant access, L1 cache. so to look at something in VR you turn your head and the view pans. To look at it in your brain, its just there, instant lookup.
one requires a transition in space to access information (flying) the other does not (teleportation)
Low-pixel displays can't replace high-pixel displays (yet). It's all about the speed your head can move, vs. the speed your eyes can move, vs. the speed your display can refresh.
As a simple test, try holding your head still while moving your eyes back and forth ten times, from one extreme of your FOV to the other. Then try moving your head back and forth ten times to see things beyond your FOV on each side. Two observations when I try this: (1) the eye movement was much faster than the head movement; (2) the head movement was much more tiring.
In fact, eye movement is the fastest thing we can do, and even uses special circuits to reduce latency:
"Saccades are the fastest movements produced by the human body. The peak angular speed of the eye during a saccade reaches up to 900°/s in humans; in some monkeys, peak speed can reach 1000°/s. ... Under certain laboratory circumstances, the latency of, or reaction time to, saccade production can be cut nearly in half (express saccades). These saccades are generated by a neuronal mechanism that bypasses time-consuming circuits and activates the eye muscles more directly."[1]
Head movement apparently maxes out at 500°/s or so, and that's not comfortable to keep up for long. So it's at least twice as fast to access high-resolution information by eye movement as to access the same information at lower res by turning your head. That's the tradeoff between using a fixed monitor at your max comfortable resolution, and using a lower-res head-mounted display.
At the same time, a fixed display (for now) allows you to move your head more rather than less while reading comfortably. A head-mounted display has to update the display in response to your head movement, and a fixed display doesn't.
As a test here, see how fast you can move your head back and forth while reading text on your fixed display. Then see how fast you can move back and forth while reading on your DK2. The second will be slower, because the Oculus has to impersonate a pixel on your virtual display by juggling which physical pixels are used to render it as your head turns. In addition to further throwing away resolution by aliasing your virtual display onto your real display (the virtual text has to look sharp regardless of how it happens to overlap with physical pixels), this means the refresh rate is a limiting factor on how fast you can turn your head and still read. The text which was crisp while your head was still is physically blurred before it even gets to your eye, because the Oculus can't quite keep up with where it should be. That stops being a factor at, say, 1000hz, but we're not there yet.[2]
So on a fixed display, you can actually move and stretch your neck and turn your head without affecting the reading experience; on a virtual display, those movements will blur the text you're trying to read.
I'm very excited about all of the things VR can be -- just saying that one thing it can't be, yet, is a replacement for a high-resolution monitor when you're trying to quickly access lots of text.
[1] http://en.wikipedia.org/wiki/Saccade [2] http://blogs.valvesoftware.com/abrash/down-the-vr-rabbit-hol...
need some new/better input devices in addition to jey oard to manipulate the objects in the space.
does the oculus have a "axis lock" feature of some sort? foe instance, lock X so that any head movements only change the users camera along x?
Though as the other guy brought up, I guess it may not be the most natural way to read text if you have to actually move your neck, but it may be decent enough that you could adapt to it. And thus, you could write code on the train/bus to work from your phone (at least, until someone mugs you and steals your expensive phone VR setup heh).
You'll soon build a counter strike level by walking inside it and sculpting the world around you. How cool !
[rp1]: https://en.wikipedia.org/wiki/Ready_Player_One
This fills me with almost childish glee. This is a virtual reality, where you can pop up a "god window" and create/mess with a world around you. Even just what's in the video, making some cubes move and change colour is wonderful. It's fun!
Remember the early bits of amazement at programming? Where you could make the computer say your name or draw a square? How much cooler would that be in VR?
Croquet was this (but without the Oculus) in 2007. I miss it, but it went nowhere, but.. that's Smalltalk for you - seems to scare people for some reason.
https://www.youtube.com/watch?v=FfxG_uB66vI
i argue against people calling the oculus virtual reality (i)
that child like glee is warranted in a vr scenario, but what you are seeing demo'd here is digital stereoscopy
.
(i) https://news.ycombinator.com/item?id=8347450
My problem is that it lacks imagination. Let's just take our text editor over to the VR world. Not come up with a new method of interacting with code utilizing VR.
Yes, and I think live coding is pretty cool.
> but I don't see the revolution here yet.
Neither do I, but I don't think it needs to be. It certainly didn't seem to be sold as that. That was part of the reason for my post actually, that people are alternately seeing this as the next big thing or falling short of some incredible goal. It's neither, it's just cool.
> My problem is that it lacks imagination.
Well feel free to create something else incredible, and I'll be happy to find that awesome as well. This doesn't hold anyone back, it doesn't stop anything and it isn't being billed as TheNewWave(tm), it's just an interesting experiment combining a few different bits of tech together.
It's a drastic shame to see people poke holes in something just because it isn't world changing.
Also I am working on something on my own. Which is why I thought I'd give feedback. I encountered the same issues myself.
I'm not sure if that was aimed at me, but if so you can't downvote replies to your own comments (at least I can't, maybe there's a higher karma threshold where you can but I doubt it).
I agree with your overall sentiment, but I also have an observation: I have the DK2 -- and while I haven't had a chance to play with this yet, I have been playing a little bit of the Elite: Dangerous beta.
The revelation there, is that working headtracking makes working with a huge "display" feel very natural. You can only look at one (or a few) things at once anyway -- but with head-tracking you don't have to worry about lining up five 30" screens in order to keep track of everything -- having it all (sanely) mapped out in 3d actually works just as mundanely and boring and well as you'd expect.
And that in itself is kind of a revelation. Suddenly you have almost unlimited screen real estate, that is even more naturally and intuitively accessible than multiple desktops (and I'm generally a great fan of xmonad-style tiling, no-full window managers with easily accessible multiple workspaces).
Another "wow" factor for Elite was the fact that what seems like a kind of silly gimmick on a single screen -- that "holograms" pop up with info when you look at them -- actually works in VR.
The one thing they did botch in Elite in the previous beta, was font-size -- the DK2 leaves "half-hd" per eye -- and that's not really enough to read "fine print". They sort-of fixed it in the new Beta just out -- but now the UI for space docks is "too wide" -- the text is more readable, but the "holo" should've been placed "further away" from you, so you don't have to move your head from side to side to read a line of text. I've yet to try it plain 2d -- I suspect it works fine on a regular screen.
Anyway, the point is that even a completely boring an unimaginative use of VR for programming can actually be a minor revolution. Fully working gloves/hand tracking would be better -- maybe with just a "dummy" keyboard for touch input (I'm not convinced "drum on a tabletop, visual-tracking keyboard" is actually that usable for touch-typing -- at least not when your back-tick is at shift-two-places-right-of-zero and opening-curly-brace is at meta-7 etc -- as it is on a Norwegian layout).
Finally -- immediate feedback certainly is nice -- and this is where the "big screen" effect of VR can really help. You can be effortlessly aware of both your evolving 3d model, and the text-representation of your code -- without having to have two screens, or switch between virtual workspaces.
You make some good points and I'll have to investigate further once I get the headset. Currently using the SDK and it just isn't the same.
I kinda like the "grab object & kiss to save" gesture: it wouldn't make sense in 2D but feels natural in VR, although "grab & push object in your brain" might be easier implement to represent a save function.
https://www.youtube.com/watch?v=VzFpg271sm8
Someday there will be something like Second Life with Oculus support and a decent programming environment, and that will be awesome.
But... the video shows a very specific example of programming where you are basically designing a map or a level. As soon as you are programming something else than a simple 3d game, it becomes less useful.
The major problem we are trying to solve here is managing windows, but I argue that normal screens already do that and you don't need VR.
A large screen or two plus using alt-tab and virtual desktops can completely and better replace virtual screens in VR. You have to move in the VR world to access information. On the other hand using a normal desktop where the information is represented abstractly( where in VR it is a physical object ) can be accessed instantly with a simple and fast key-combination. I simply don't see which programming activity could you do better in VR than in a normal desktop.
Using this app is comfortable but VR will definitely need higher resolutions for anything more serious. It's not too far in the future though. If things go well, I expect VR will be the driving force behind ultra-high-density 5" screens in the next few years. Or some entirely new display tech (retinal projectors?) will replace it.
I'm trying to think of a good example where VR could be used but just can't.
It will probably be fantastic for stock brokers where their information relies on quick glances over a lot of realtime graphs/tables and if you look at their monitor setup today( +4 monitors ) the extra space will work for them. But when you code you are heavily centered on a single window and the rest doesn't really matter.
For times like this you might want a completely black/white background, a forest clearing with very gentle soundscape or a quietly psychedelic lightshow/music all centered around your code window.
Some people's VR world will be serene and spare and some people's will be chaotic and full of the energy of a city street. It will be up to the user.
Some people will use the fact that everyone's brain benefits from both of those extremes at different times and use that to improve their cognition.
A company would never buy an VR rig for every employee just to maybe boost their productivity by an unknown margin. Also the learning curve would set them back by an unknown margin and some might have problems adapting.
Using your example, VR is useful, but only as an edge case where VR measurably help a person.
Think of it as switching every keyboard in the company for a Dworak version. That will not meaningfully affect your average programmer's productivity, as their typing speed gained from a faster layout gives only diminishing results. Dworak only makes sense for a programmer that really benefits from >100 words per minute, and most really don't. The bottleneck it the programmer, not the input. The same goes for VR, unless you have a special case as I have mentioned before.
Let's say you type 300 lines of 80 characters of bug-free(!) code per working day of 8 hours. That is an pessimistic estimate( for my theory ), average is much lower.
Let's assume the average typing speed is 60 words per minute. That gives you 83% of the time not spent typing(!). Doubling the speed to a very fast of 120 wpm, only gives you 92%.
Just guestimating, I would need 11 or 12 27" screens to fully utilize the area I can see without moving my head. With gentle neck motion included I think that number triples or quadruples.
If you switch windows every 60 seconds, this gives us an improvement of ~8%.
Table:
300s -> 1.5%
120s -> 4%
60s -> 8%
30s -> 16%
15s -> 32%
I have no idea what my average window switch time is. I should measure it sometime. I doubt it is under 30 seconds where the VR actually makes a meaningful improvement.
If you take an average of 2 seconds for non-VR and a skilled user with keyboard shortcuts and virtual desktops, VR advantage becomes insignificant.
Table:
300s -> 0.6%
120s -> 1.5%
60s -> 3%
30s -> 6%
15s -> 12%
If you compare a specialized window managing software versus the VR equivalent then I don't see any benefit towards the latter. The latency is the same. The only difference is the abstract windows management versus spacial of VR.
I think programmers can handle the non-VR solution just fine. I really hope it becomes better though; the resolution is currently just too small.
Any suggestions for actual programming? You know, like; classes, boring code, loops, i/o, algorithms?
This is really good because I have a stomach condition that means being in a seated position is very painful for me, but using this I can stretch out on the bed, with a keyboard resting on me, and still code :)
- Have a "keyboard cam" that only shows you the keyboard (and your hands when they're over it) displayed at some natural spot in the VR.
- Have a keyboard with capacitive sensing keys, reproduced in the VR with keys highlighted as a function of the proximity or contact with your fingers.
- Something like this [1]. Typing in thin air or on a table probably wouldn't work very well, but if the leap motion senses the keyboard as well as your hands, and the whole thing is reproduced in the VR, that might work.
[1]: https://www.youtube.com/watch?v=TRhoQ6o4mI8#t=142
It doesn't yet support "live programming" like this, but the hope is the framework would eventually be used to easily make these sorts of things.
So far, the focus has mostly been on the user input side. There is a constraint-based system for defining input values from different "devices", i.e. keyboard, mouse, gamepad, head mounted display motion tracking, arm motion tracking, etc. There is even a rudimentary gesture system, where things like a nodding gesture or head shaking gesture can be wired up and used to activate commands just as easily as a mouse click or key press.
The framework also supports multi-user environments, and I've already started building the text input controls (somewhat similar to what is shown here) to do things like this (right now, it's only useful for live chat). Eventually, I'll have buttons and text boxes and togglers, etc.
If you're interested in this sort of stuff, please consider joining me.
Join the mailing list: https://mail.mozilla.org/listinfo/web-vr-discuss
More info: http://blog.bitops.com/blog/2014/06/26/first-steps-for-vr-on... http://blog.tojicode.com/2014/07/bringing-vr-to-chrome.html
Everything from design, analyses, testing moves into the VR environment. Already CAD/CFD/CAE computational models are saving huge costs to companies. Throw in VR and we are that much closer to a tangible, collaborative environment. Very very excited!
this demo is cool but i'd argue impractical
be careful getting excited over demos of technology that add a layer of abstraction from the tech direct
ever seen a photo of a painting? ever read a book you loved and your friends hated?
i'd suggest you try the oculus as you would use it: in your home on your box; before spending any money on the device
also, you are blind to the keyboard with the headset on
i feel oculus caries a mythos around it that is very difficult to obviate if you have yet to have worn one.. i'll link to an old comment i posted to carmacks keynote at the first ocucon : https://news.ycombinator.com/item?id=8347450
essentially i have stopped referring to oculus as vr, oculus is digital stereoscopy, and being stereoscopy the oculus is exploiting an optical illusion and as such, i believe, a flawed direction for the touted outcome: virtual reality
this demo: http://www.phoronix.com/scan.php?page=news_item&px=MTcyMTI ; is what encouraged me to shell out for the dk2
the experience failed to deliver on my expectations, though at the time i thought my expectations were practical and rooted in evidence, of course now i realise those expectations were irrational and that evidence was secondary being derived from this video demo
Of course stereoscopy is part of what the oculus does (although there's also head tracking, so talking only about the stereoscopy is misleading). You say it's a flawed direction, so what's the right direction then?
i suppose it depends on your definition of virtual reality..
but i think the verbiage of vr is very misleading
my position is that if oculus was called: oculus digital stereoscopy; the tech would lose a lot of its hype
as for the tracking it is certainly the best feature of the headset, again with the goal of calling the thing what it does i refer to the 'head tracking' as 'inferred spherical screen encapsulation'
what the head tracking is doing is windowing your vision to the size of a fisheyed cell phone screen and then inferring your position in a 3d space and displaying the contents of that field of view on the screen, refreshing the content of the image in regard to the movement of your neck, giving you the impression that you are in a sphere of flat displays looking through fisheye glasses with blinders with a forced forward stare
when i put it on the restricted periphery and the inability for me to use my eyes to look around was the most dramatic realisation
but that is just dismantling the vr hype, actual stereoscopy is flawed in its own ways..
peoples eyes differ in distance apart, the oculus is solid plastic, certainly the error is measured in mm to fractions of mm but that is still error that your brain now is required to correct,
also the goggles press against your face tightly to seal out distracting ambient light, this hurts after some time, and in the case of my watching a film, completely depleted my eyes of blood leaving white bags under my eyes
that is the hardware
as for the software if you read the comment i linked in the comments you are referring to i talk about an experiment i performed when i first received the oculus that shows that the brain error corrects the inconsistencies between the eye inputs at a pretty substantial distance, talking here of the blue and pink box experiment, in the experiment only the pink box moves, but when i showed it to a friend thae asked why when the two boxes began overlapping did the blue one start moving as well? i watching this all cloned on the screen in front of me explained that it still was the case that only the pink one moved and the blue one is being moved around by thaer brain, so if a simple experiment has my brain working overtime shifting still objects it would seem reasonable that a lot of the 'jitter' that is so often talked about when displaying entire worlds is your brain organising its inputs
as for what i believe is the right direction?
if we are talking about visual input manipulation.. i'd think projection, perhaps onto a dome or disc encapsulating the entire eye range including periphery of the eyes even with the pupil extended to the outer most regions of the eye
or one fantasy device i dreamed up used this recent research: http://newsoffice.mit.edu/2014/magnetic-hair-directs-water-f... ; if you watch the video, at the end, 57s in, you can see these hairs bending optics, what you'd do is have two rows of these hairs, one for each eye and each row bends to follow the movement of a pupil and each hair acts as a pixel
this way simulates two photons bouncing off a single object and reaching the eye, that would be wild
but mind that i referred to the effort as visual input manipulation, if you want to talk vr i think the only possible direction is bypassing our sensors and interacting directly with the processors, a la the suit in Zero Theorem : http://cdn-static.denofgeek.com/sites&#...
Then I can legitimately wear an Oculus Rift at work!
I think what would work is some sort of hologram reality agumentation that is able to project 3d images in the environment you are in.
Again, this ergonomic aspect of the Oculus might be improved but my primary concern is the vision health aspect of it.
Its sort of like wondering about the health effects of wearing glasses.
The hand-wavy "this might look like a book with a video in it and some neat drawings, but it actually contains a magical language that describes the world you're in" ...
... you're looking at a book written in a magical language that describes the world it's in.