Yeah, Haskell was an interesting choice. I started working on it _because_ it was a Haskell project.
Overall it was a pretty rough road, but someone has to go first and I'd really like for Haskell graphics to take off.
I ended up writing bindings for Godot so we didn't have to deal with graphics stuff from scratch (like in the first few iterations).
While this lost us some of Haskell's benefits and it's not super functional-y, it's still nice to work with and most of the issues we encounter are related to the bindings being unpolished.
If you have any questions about doing this kind of project in Haskell, hit me up.
That’s weird, wasn’t Simula 2 the first OO language? This Simula doesn’t seem to be related at all. I get name reuse, but this Simula is one of those huge milestones in computer science.
What is the reason you went with the name in the first place? Simula is the first part of simulation (which Simula was specialized in expressing), but I'm not seeing a simulation aspect here. Maybe an inside design inspiration relating to VR and simulating reality? Or maybe named after a village in Estonia?
"Simula" reminded me of the word "Simulation" or "Simulate", and had a good Matrixy feel to it. We're open-minded to a name reboot as we get further off the ground.
Right now we're working on perfecting Linux VR Desktop (for 2D apps), but the long-term goal is to bring forth killer office VR apps (for productivity, not gaming/entertainment) that wouldn't otherwise be possible in normal 2D Linux. These apps will likely take advantage of VR as an amazing "Simula[ting]" environment.
Yes, it was introduced in 1967 as a successor to Simula 1 and the language was called Simula 67. It influenced subsequent OO languages like CLU, C++, Java, and Smalltalk.
Barbara Liskov lent me a book that described it in 1973 and recommended that I read about Simula 67 before starting to work on a small part of the the CLU compiler design. I believe the book was Structured Programming by Dijkstra, Hoare, and Dahl. It had a chapter by Dahl describing Simula 67.
Packaging this was an absolute nightmare due to various distributions shipping incompatible library versions and so on.
Nix, despite issues with everything OpenGL related, was the only thing that actually worked. And Cachix made installation really fast.
PS: if it starts building for some reason during the install process, let us know. We've tested it on various machines but due to some peculiarities specific configurations might require a rebuild even with Cachix. We can add that to the repo so that future installs and updates are faster.
That's a bit like saying "Just got a monitor. What's good to look at on it?" ;-)
The world of VR is so diverse it's tricky to recommend things without knowing you.
Personally I'm not much of a gamer so I tend to be interested in narrative stuff, abstract visuasation and geometry stuff or just exploring interesting environments.
And the games I do like tend to have one or more of the above qualities.
You might be an RPG fan, a strategy gamer, an art aficionado , looking for productivity or educational tools or any number of things.
We don't actually use X as the framework. The underlying code is Wayland, and then Xwayland as a compatibility layer. It's just that most apps we test against are X11.
In case people are curious what it's like to work in VR, here's a demo of me using Simula: https://youtu.be/FWLuwG91HnI
I would argue that working in VR is fundamentally superior to working in PCs/laptops. It's basically 10x the screens, focus, and work immersion. Simula's text quality is also very good (getting around the eye fatigue present in older VR Desktops).
Well looking at the video, I mean, I can have dozens workspaces with hundreds of windows (if I need to) and navigate them with moving just my fingers, whereas you are apparently breaking your neck to navigate more than 3 windows. I think what you're doing is cool and has to be done for the sake of exploration, but I would really, really focus on navigation. Also, how are you supposed to focus on text, when reading or reading or writing when it's wiggling all the time. It seems to me that VR really is for games, but for working with text I think humans are better off with stationary screens.
On a second thought: if you would make it easier to "fix" window, such that one can focus on it and work. As when you are "placing" a new window in your video, that could make it a workable solution.
Navigation is one of the things we definitely need to improve upon.
I think the wiggling is actually kind of natural -- you don't notice your head or eye movements when you do them automatically, but it might be exacerbated in VR.
The fixing is an interesting idea, we'll try it out. I'm a bit worried it might feel weird in VR.
Yes, there is a thing called corollary discharge in the human visual system that keeps the world steady as one glances around (in the real world). Changing the direction one is looking causes the image to shift rapidly on the retina, yet perceptually we see a world that is not wiggling.
Corollary discharge[1] causes the brain to “subtract” out the motion of the image on the retina. It happens when our nervous system instructs the eyes to look in another direction. You can have someone else move your eyes or even do it yourself by pressing gently on the side of the eyeball with a fingertip on top of the lid. Wiggling one eye this way is not supported by the hardware of the eye and produces a very noticeable wiggly view of the real world.
I mentioned subtracting out the motion going on in the visual system. Experiments suggest that something like this is happening. If a muscle paralyzing drug is injected into the eye’s muscles it causes a wiggly world because the intent to glance somewhere causes both an ineffective command to move the eye, producing no real movement of the image in the retina while the triggered corollary discharge subtracts the intended motion from the still image and causes a perceived motion (in the opposite direction).
YouTube has support for VR videos. If you could record & render such that people with headsets could watch a session, that might help with interest and adoption.
I think navigation needs eye tracking, which sadly no headset currently supports. Focus-follows-gaze would be a game changer.
Uploading a YouTube VR video of a work session is a potentially fantastic idea. Thanks for the suggestion.
Right now, Simula uses "dumb" eye tracking, in the sense that windows receive keyboard and cursor focus when the user's forward eye gaze intersects a window. We also have it so that users can control the cursor focus with their forward gaze (presently binded to `Super + Apostrophe`); similarly, users can drag windows around by holding `Super + Alt` and looking around. The experience adds up to something quite productive once you learn all the keyboard shortcuts (your fingers don't need to leave the keyboard).
I bought a Vive Pro Eye specifically to try and develop eye tracking navigation for a system like this. I haven’t done any actual digging yet bc it turns out the NDAs for their SDK are...unfortunate. Definitely an opportunity for a free/open alternative.
I haven't used VR devices too much but I believe the wiggling that you see in VR demos is not present when actually using the device. The device is simply adjusting the perspective to match your head motion and so the actual user is not seeing the wiggling but in the demo it appears that the view is wiggling to the watcher.
But I agree that wiggling is off putting. I think VR demos should low-pass filter the output to some degree to give a more realistic perception of what VR is like.
I wonder if a separate camera for pancake with a larger FOV would help, with the translation (e: and rotation) matrix lowpass-filtered from user orientation.
But I feel its mostly rotational motion that is perceptible in VR demos rather than translational motion. I suspect that's because a small rotational motion results in a large shift in perspective for objects in the world that are some distance away from the camera while small translational movements don't really result in a significant perspective shift.
Would be interested in seeing what research into human head motion while sitting/standing/walking/etc. shows though.
Virtual Desktop already has many customizable features that allow you to work the way you're describing. I've worked in it before, and it was very immersive, though there was of course some fatigue.
You can have just one window at a time, and with a good headset, the wiggle is not significant. You can even adjust it so that your workspace follows you when your head turns.
I haven't used Windows' Virtual Desktop since the early days, but I suspect our text enhancements have improved upon the situation dramatically (when keeping hardware constant).
There is no wiggling. That wiggling you see in videos is your head movements not being in sync with the wearer’s head movements. You’re doing it right now, your brain is just anticipating and compensating for it.
> It seems to me that VR really is for games, but for working with text I think humans are better off with stationary screens.
Have you actually tried VR? The fact that you think there's wiggling makes me think you haven't. Yes, videos make it look that way but I can guarantee you that's not reflected in the actual experience. VR is notoriously bad to watch on video and doesn't capture the experience at all.
Actually the problem is low resolution and FOV of VR headsets. VR require at least 72Hz framerate at 4Kish which result in 2K 72Hz 90° ish stereo and that’s hard already even for moderate gaming PCs.
As for pinned windows: fixed HUD is a no-no in VR games because it burns in same spot of your eyesight. When a HUD is desired, players are usually given a virtual or symbolic helmet that allow display to lag behind head movement.
How is the eye fatigue in general? How long can you go in a VR work session?
For VR gaming (Oculus Rift), I start to experience mild discomfort after about 30min, which grows severe by 60min. It's hard to envision using VR for a full work day. (My eyes are fairly light-sensitive from laser-eye surgery; not sure how much is saccading, vs. bright light directly in front of the eyes.)
Many early VR Desktops (and VR games) didn't go to special effort to optimize text quality, and I think this has given people a lower than warranted impression of what is possible in VR today. There's no doubt that VR hardware is going to get exponentially lighter/sharper over time, and that these improvements will be welcome, but things are good enough now to put in 1hr+ long sessions without eye strain (at least in my experience as a daily user).
It really depends on the task - I (currently) have an OG Vive, and its absolutely fine for games with good anti-aliasing (no discomfort for me for basically any length of time, although I mainly play beatsaber and you don't really focus on the blocks beyond blinks of light and muscle memory), but it's hopeless if you want to watch a movie let alone program. The kind of programming I tend to do is generally fiddly enough that I've got a book open or some kind of datasheet etc. (I think - even good - VR would be a hindrance here)
I think the Index might be OK but I couldn't comment.
I would not be able to work like that, it would lead to neck strain or wrist pain, if you think about it comes naturally to reposition your body when you are looking in a new direction there is a reason for this. This hurts me because the keyboard locks the shoulders and it's only my heavy head that moves, this is not ergonomical.
So being able to fetch up windows so it's perfectly centered above the keyboard would make it easier for me to use.
An interesting potential for VR would be alternative portable form factors. The laptop form factor is mostly dictated by the shape and size of the screen; since with VR the screen is moved to the HMD, this would no longer be a restriction.
The webcam view is also interesting; if it were "inverted" so that, instead of being confined to a window, it was outside of all windows (that is, the background "wallpaper"), it would work even better as a laptop replacement (that is, the screen doesn't cover your whole view, allowing you to keep aware of your surroundings).
If could contribute an idea, I wonder if one can make the camera watch your surroundings but with people recognizer that would notify you if someone is approaching with the intent to interact with you, and maybe show a notification or an approaching avatar. It would be neat in an office scenario. The next step would be a screen in the real world telling your visitor whether you're busy or interruptible, and a button the visitor can push to "knock on your door".
So that's the future, maybe: cubicle dwellers get their own VR offices...
This seems like the job for another program that you run simultaneously, not the window manager. Unless there are problems with having multiple programs make use of the camera or something.
Productivity use for VR is way more compelling to me than any game I’ve ever seen. Maybe that’s why it’s never taken off. We probably need to have an email client people want to use in VR and an IDE before we worry about gamers.
Totally agreed. Our project's "secret": most people think that the future of VR is in games & entertainment, but it's actually in office work.
I never would have even purchased a VR headset had it not been for the promise to start working on a productivity environment. But before we can start building the killer VR office apps, we have to get the basic 2D apps working crisply/perfectly, IMO.
I came here to say this. Never in my life have I wanted a VR headset. I've used others' to play games (Beatsaber in particular is really fun) but I don't have time in my daily life to allow for a regular gaming habit or indulging that kind of money on it.
In 2017, I had a conversation with a co-worker, where we talked about how badly we wanted a VR desktop environment with infinite screen-space. There's VRDesktop, but that isn't this. And then again the same conversation last year with another friend.
After seeing the demo video, I was compelled to go look at Vive prices (~$1,000, ouch).
I feel like my productivity would skyrocket with this, mostly due to lack of background distractions. Imagine wearing noise-cancelling headphones. I want this so bad.
It’d be awesome if you could do this natively on the Quest (sideloaded). Native gesture control, inside-out tracking, wireless, almost twice the resolution, and wouldn’t even need to be connected to a PC at all so could be used anywhere (with Bluetooth keyboard and mouse).
Quest would be amazing because it would truly demonstrate a completely untethered, self contained portable full desktop system that you could take anywhere.
Having said that, getting even a standard desktop setup on Android to work well is iffy so doing it in VR does seem like a pretty immense challenge. But it would be a breakthrough in what it could demonstrate in terms of the concept.
I love the idea of a VR workspace. There are so many
limitations in our current workflows and tools that mainly just arise from being restricted to a small flat surface. Even just having your perfect setup anywhere you go would be amazing. All you need is a headset and a small portable cube computer - basically a laptop without a screen.
---
But after watching your video, I feel like this really needs some tiling window manager (i3 etc) inspired way of easily creating workspaces with tiles, and effortlessly navigating between them. With relatively minimal head movement.
Also, does Simula support curved displays? It would seem very natural to arrange different curved screens around you instead of flat ones, which also potentially waste quite a bit of useful space.
I use a curved monitor (Samsung CRG9) myself as my daily driver, and it appears curved when I use desktop view in SteamVR, so I definitely see the appeal. Just not sure how nice it looks on smaller windows.
Cool demo. But wouldn't it be better (for your neck) to move the screen instead of moving your head? Or both? I guess sooner or later it's going to hurt the way it is implemented now.
Agreed that VR isn't comparable to 4K screens (yet), but for what it's worth I regularly put in 1hr+ long SimulaVR sessions with the good ol' 2016 HTC Vive.
Here are a couple of links showing off Simula's text quality:
Many early VR Desktops (and VR games) didn't go to special effort to optimize text quality, and I think this has given people a lower than warranted impression of what is possible in VR today (even on old headsets).
I think VR desktops look really cool, but I'm not sure I'd want to have to wear a VR headset every time I used my computer. Monitors will surely stick around in parallel.
I have Rift not a Quest so I don't know if they're the same but you can pull your desktop and every window in your desktop up in VR on the default Rift OS, even in the middle of running another VR app. I had a browser open in the middle of playing Half Life: Alyx. Did the same on No Man's Sky VR
Someone asked if I could watch all the Star Wars movies at the same time. I got 6 trailers running at once in 6 separate virtual monitors with the default built in OS feature.
Yes, but there's the PC link option for the Quest, so if you have the link do you get the same experience as a Rift including being able to pull all your Windows desktop windows individually into VR?
About eye-strain, how is focusing in a VR headset? Is it always a fixed point given it's really just flat screens, or does one have to focus differently when things are "far away".
I’m pretty sure you never focus on anything up close in VR. It’s all “far away”, as someone who is near sighted I absolutely have to use glasses in VR. I suspect but am not certain some far sighted people might not need correction, as something may seem close up but really it’s just “big and far away”.
> It’s all “far away”, as someone who is near sighted I absolutely have to use glasses in VR
Yeah, me too. As far as my eye is concerned, most things are "far away". Interestingly though, I can read things in VR "farther" than I can in real life. I am not exactly sure what the convergence looks like.
Also worth noting the foam guard on your headset can be adjusted or replaced for a better/different fit. I have three foam guards I alternate between (because people get sweaty when I have them over for Beat Saber and it's easy to swap in a clean one for the next person) and one of them didn't fit my glasses until I cut out some clearance for them. Now all three work well, and I wear glasses in VR 100% of the time.
From a practical point of view, at least for me, "it just works like normal".
As I understand the biology of it, there are really two different relevant mechanisms for "focus".
The lens that is your eyeball changes shape to focus on things closer/father away in real life. In VR this doesn't happen, there's a fixed focal plane a few meters away. I know this, but I don't notice it at all. Everything is always in focus and my brain doesn't seem to have any trouble dealing with it.
There is also the problem of "vergence", your eyes point at the thing you are looking at, which means they point "closer to eachother" when looking at something close, and in parallel lines when looking at something at infinite distance. VR headsets implement this reasonably well. If you hold a finger a few inches in front of your nose and focus on something a few meters away you should be able to see two copies of your finger (or at least I do, I'm told that other people have to try to see this). This works in VR as well.
As an aside. Trying to play shooters in VR has been surprisingly hard for me because of the vergence issue. I'm not use to aiming a real weapon, and it turns out I have trouble figuring out which near-field image corresponds to the eye I'm trying to aim with while I focus on the target in the distance. I'm not sure if I'd find it easier in real life.
Looks cool and will try it out. That said, these things always feel like old paradigm stuffed in to new technology. Bit similar like when film was new people kept using it to record theater. It took years to come up with editing, not having to start scene by people entering the stage etc. Now that we have unlimited spatial space to manipulate the thing we come up with is to fill it with 2d screens.
There aren't a lot of jobs that can really use the 'space' provided by VR. I think the adoption here would have to drastically outpace multiple monitors and OS workspaces and keyboard shortcuts. I can easily get to 10-20 tabs in VS Code or a browser, but my attention is only really ever on one at a time. If I have to search through them, I'll have to do that on a monitor or in VR. I might be able to organize better in 3D: HTML files on the left, CSS files on the right, JS in the middle, backend reference above, tools and console below (like 5 monitors in a plus pattern).
I'd need probably a dozen or so channels of information that I had to cycle between quickly to convince me, given I've gotten used to multi-monitors, workspaces, and keyboard shortcuts. Maybe some futuristic high level analyst job that needs to look through lots of different kinds of information (geographical, text, videos, etc) at once.
I can imagine a web editor with some "3d space" flair. You have your HTML/JS/CSS editor in front of you, but imagine strands connecting CSS rules to the HTML elements in the preview window, which is floating slightly to your right. Or the page being previewed can even be the size of a 5-storey building which seems to be standing across the street.
For another interesting use case, how about a 3d debugger that lets you follow a program flow over multiple classes and stacks...
I end up needing a lot of screen real estate whenever I work on some EE project --- datasheets, schematic, PCB editor, calculator, notes. Additionally, being able to e.g. manipulate the enclosure and PCB to do a test fit and immediately edit things would be super cool.
This looks very nice! I've been thinking about something like this for quite a while. May I suggest two additional features that I would love to see in such a window manager:
1. A way to neatly arrange all the windows on a virtual sphere that surrounds the user, possibly arranging them automatically in a similar manner as a tiling window manager.
2. A way to rotate the before-mentioned sphere around you without forcing the user to rotate it's head. This would avoid much of the neck strain. It could be done by, for example, holding a button on the keyboard while moving the mouse or by a simple keyboard shortcut to rotate the sphere by X degrees in any direction.
This concept could also be extended to virtual desktops where each desktop is a sphere around the next, like an onion, with the ability to "zoom" in to the next desktop.
> 2. A way to rotate the before-mentioned sphere around you without forcing the user to rotate it's head. This would avoid much of the neck strain. It could be done by, for example, holding a button on the keyboard while moving the mouse or by a simple keyboard shortcut to rotate the sphere by X degrees in any direction.
one more thing for the author to play around with (I don't have the hardware to try it myself) is experiment with speed/acceleration -- the mouse pointer movement has speed and acceleration parameters that affect how it moves, so that if you want you only have to move your mouse very little in real dimensions to move it thousands of pixels.
It might cause motion sickness, but maybe you can get away with pitching/yawing 5 degrees for every 1 degree of head movement, so to look "straight up" you only have to tilt your head up 18 degrees. Hopefully you'd still have the illusion of being oriented in a space.
> It might cause motion sickness, but maybe you can get away with pitching/yawing 5 degrees for every 1 degree of head movement, so to look "straight up" you only have to tilt your head up 18 degrees.
A tip for finding out the email of a Github user: choose any of their commits, open it on Github and add ".patch" at the end of the URL, like this [1]. There's a good chance they use a real email to sign their commits.
Seems like this doesn't have any relation to xrdesktop[1], which doesn't have mouse support! Really glad to see they have it here, any VR workspace is completely unusable if I have to use motion controls to simulate a mouse. Will definitely be trying this out tonight.
xrdesktop has a very talented team of hackers working on it (so I have no doubt they'll fix the mouse issue!).
Our approach with mouse cursors has been to assign every window its own mouse cursor. This is possible in Simula since the active window is the one that the user is currently gazing at, so that -- for example -- if you gaze at one window and move its cursor, it won't affect the other windows' cursors. We also allow users to control mouse cursors with their eye gaze (presently bounded to `Super + Apostrophe`), which is a good productivity boost.
In fact, you can do everything in Simula with just the keyboard (no mouse or VR controllers needed): move windows, move the mouse cursor, click the mouse cursor, etc. Once you learn the shortcuts, it's very quick.
Very clever idea actually, major props. Unfortunately I get really sick from VR so it's not gonna be for me but I can definitely see how someone can take advantage of that. In the WFH era for people who have tiny laptops, this could do. I personally hook mine to the tv as a second screen but still... Very clever!
Do you get sick in all VR? It was my understanding that VR generally only gets people sick when there is motion. So any stationary 6 DOF VR experience should generally not make anyone sick.
This looks absolutely awesome, but I think there's a bit more to be done RE neck strain and general ergonomics for the vast majority of people.
If you are at a desk and/or without a swivel chair, you're pretty limited to a grid of ~9 screens directly in front of you, and even then it looks like you're turning your head a TON to see the outer layer.
Some ideas that might help:
1) To use an analogy, could you increase the "mouse speed" of moving your head? Right now it's 1:1 with the virtual space and makes geometric "sense", but it might be nice to be able to focus on screens on the left/right by moving your head less and having your focus move just as fast. For example, moving your head 50% of what you do now to zoom over to the same place would reduce neck strain when looking at those windows, but also open up a bit more window space if, for example, you could still move your neck 100% of what you're doing and be able to "look over" far enough to fit another set of screens even further out. I don't know if this disconnect in pan speed would be disorienting, but it's worth trying. :) c.f. https://i.imgur.com/0KCo9hG.png
2) Keyboard gestures to "re-center" your focus might also go a long way. Perhaps instead of craning your head to look at a window that isn't "straight ahead" for a long time (as long as you're working in that window), you could do a quick glance at that window and hit a keyboard shortcut that would recenter wherever you're looking to be your "straight ahead", so you could then straighten your neck back out and continue working in that window while in a more comfortable position. c.f. https://i.imgur.com/u6SI284.png
3) Likewise, we don't all have amazing headset resolutions and there's a lot of potentially wasted space whenever focusing on a particular window that's not "full-screened". Perhaps a keyboard shortcut to expand whatever screen you're looking at to the full size of your view would be helpful for eye strain. Even something that could temporarily "pin" a window to your view (full screen and moving with your gaze) might go a long way toward minimizing unused focus/pixels. c.f. https://i.imgur.com/yA1twKf.png
My immediate thought watching this was to mount a keyboard on a swivel chair, but I think requiring end-users to get customized hardware to take full advantage of your environment might be the wrong way to go. :)
As another option, perhaps having it passively reorient whenever you look at the same location for a long time. If you stare at a window, then it slowly adjusts that window to be straight ahead. If done slowly enough, you might be able to trick the wearer's brain into thinking that it hasn't actually moved.
Having spent some time in VR in the past, remapping inputs like "the mouse speed of your head" can be super disorienting when you return to the "real world," perhaps dangerously so if spent the majority of your workday in VR.
Your brain adapts, and starts to believe at a primal level that "things fall this speed when I drop them," "I move this fast when I walk", "when I point over there, it's at this angle," etc.
If anybody remembers playing GTA for too long, and later walking outside with the feeling like you ought to be able to jump in the nearest car and drive it a way, you have an inkling of how your brain gets remapped.
Remarkably quickly, moving around in the real-world can feel "off" to the same degree that it felt "off" when you first entered the VR world when you start messing with these primal feedback loops. Get ready to bump into things, and don't operate any heavy machinery if you've been in a non-1:1 VR mapping.
After playing half-life: alyx for a few hours straight the other day, my mind kept wanting to hit the teleport button to move around my house instead of just walking. It was a very strong reaction.
It would require Apple it care about VR and gaming and VR capable GPUs. Except for a MacPro, no Apple devices are powerful enough for VR. Even the top end MacBookPro at $19000 still has an underpowered GPU below the lowest VR specs.
Sorry, bad memory. Still that's beside the point. The point is Apple so far is neither game nor VR friendly. I hope they change that stance. I'd prefer to be on Mac 100% of the time but I have a Windows machine as well because that's basically where all VR is happening.
That top end MBP has four thunderbolt 3 ports; have you tried any of the egpu setups? (I haven't, but I've been thinking about it for my 5th gen x1 carbon with thunderbolt 3)
How does it get the image of an application inside of SimulaVR? Is it something weird like grabbing a graphics buffer? Or is it launching the applications as a child process so it can access its viewport? Or is it asking the windows manager? I'm not sure how this part works and am curious.
Been thinking about VR workspaces since sometime in the '80's. Broadly, they suck. It seems so cool but in practice VR adds pointless overhead to efficient UI. Windows with affine transformations suck at their one job. Very few people use Second Life as an IDE.
The big win, as far as I can tell, would be to engage the user's spacial memory. (There is a small but non-zero niche for 3D visualization of complex systems: weather, large molecule, etc.) You're going to want to combine "memory palace" with "zooming" UI in a kind of pseudo-3D (I think of it as 2.618...D but the exact fractal dimension isn't important I don't think.) Then infuse with Brenda Laurel's "Computers as Theatre"...
They might suck right now, but this is a relatively nascent application of the technology.
Wait until resolution improves and we break out of the "desktop" paradigm. We could have a collection of unlimited windows and tabs that exist in a continuum around us, and we could use gestures to organize and surface the contextually relevant ones.
We won't need a bulky multi-monitor setup, and we could work remotely nearly anywhere. Imagine carrying your workspace with you.
> The big win, as far as I can tell, would be to engage the user's spacial memory.
Absolutely! Physical workspaces and work benches are incredibly functional because we are spatial animals. Breaking out of the limitations of using a screen could unlock more of our senses for use in problem solving.
I'm extremely excited about this technology. It will be great for software engineers, creatives (2d and 3d artists), mechanical engineering, CAD, ... you name it.
I really hope this keeps getting pushed forward. While I'm using all of my spare cycles on a tangentially-related problem domain, I'd be more than happy to donate money and ideas. This technology will be a dream come true if it continues to mature.
> Wait until resolution improves and we break out of the "desktop" paradigm. We could have a collection of unlimited windows and tabs that exist in a continuum around us, and we could use gestures to organize and surface the contextually relevant ones.
You can get 90% there by just using multiple desktops IMHO, at least that's my experience.
I think in the mid-term, the big win would be ability to have 4-10 large monitors but with a cheaper, mobile, and compact solution with the eyes focused further away.
Headsets have improved almost ~2x in resolution and have halved in price and some have become wireless with better optics. A long ways from 4-5 years ago, but still need another doubling of resolution (or maybe more) and an increase in wearing comfort (lighter, more compact) plus an improvement in wireless latency and maybe a reduction from $400 to $300, and you’re looking at something that would be useful just as a replacement for multiple monitors.
Plus probably improvement with registering where your laptop and mouse are automatically. In principle that could be done with a software update to inside-out tracking software.
Additionally, some improvement is possible with similar to current resolution but with improved subpixel rendering and RGB pixel layout.
Seeing what has been done with the Oculus Quest since I last checked out VR like 3 years ago has left me pretty impressed. A lot of this stuff with multiple windows in this demo could be done natively and wirelessly with the Quest (which runs a kind of Linux). The inside-out tracking is impressively good. If combined with an insert you can put your tracked controllers on Bluetooth mouse and keyboard (so the Quest can register their positions in3D space to allow proper rendering in-headset), it could give you a high productivity workstation experience just about anywhere with WiFi (could be through phone). Hand-tracking (Which works already) could even allow gestures, although I’m not sure how important that is. Can the Quest do subpixel text rendering like ClearType but 3D?
This. I can't see moving my head and making gestures with my hands ever beating tiled windows and good keyboard shortcuts. I don't want a VR enabled workspace (ie I don't want my windows to move when my head moves), but I might prefer a sufficiently high resolution headset to multiple monitors if the software support was reasonably seamless.
@echelon: This is exactly along the dimension we were thinking: VR as working environment for problem solving & creativity (via killer VR office apps that have yet to be invented). If you're curious, we have mapped out our long-term ideas in writing/deck format elsewhere. If you email me, I can send it over to you: george.w.singer [at] gmail.com.
This actually used to be the default behavior in Simula a few months ago, and worked really well. There's a few use cases where it imposes a tradeoff though (specifically: it makes it hard hold windows in a specific orientation in space when you might accidentally look at them), so we now use a key binding to do this instead.
> it makes it hard hold windows in a specific orientation in space
I don't know if you're still watching this thread, but it would be interesting to hear an example of when one might care to lock a window's orientation.
Great ideas! The Method of Loci is a very powerful concept, that takes excellent advantage of how human memory works, and works nicely with zooming user interfaces, and is a great way to support user-defined editable pie menus that you can easily navigate with gestures.
I've experimented with combining the kinesthetic advantages of pie menus and gesture with the method of loci and zooming interfaces, including a desktop app called MediaGraph for arranging and navigating music, and an iPhone app called iLoci for arranging notes and links and interactive web based applets.
>MediaGraph Music Navigation with Pie Menus. A prototype developed for Will Wright’s Stupid Fun Club.
>This is a demo of a user interface research prototype that I developed for Will Wright at the Stupid Fun Club. It includes pie menus, an editable map of music interconnected with roads, and cellular automata.
It uses one kind of nested hierarchical pie menu to build and edit another kind of geographic networked pie menu.
>iPhone iLoci Memory Palace App, by Don Hopkins @ Mobile Dev Camp.
A talk about iLoci, an iPhone app and server based on the Method of Loci for constructing a Memory Palace, by Don Hopkins, presented at Mobile Dev Camp in Amsterdam, on November 28, 2008.
DonHopkins 81 days ago | parent | favorite | on: Nototo – Build a unified mental map of notes
>Great idea, I totally get it! Your graphics are beautiful, and the layering and gridding look helpful.
It reminds me of some experimental user interfaces with pie menus I designed for creating and editing memory palaces: "iLoci" on the iPhone for notes and pictures and links and web browser integration in 2008, and "MediaGraph" on Unity3D for organizing and playing music in 2012, both of which I hope will inspire you for ideas to implement (like pie menus, and kissing!) or ways to explain what you've already created.
>A memory map editor can not only benefit from pie menus for editing and changing properties (like simultaneously picking a font with direction, and pulling out the font size with distance, for example), but it's also a great way for users to create their own custom bi-directionally gesture navigable pie menus by dragging and dropping and "kissing" islands together against each other to create and break links (like bridges between islands). (See the gesture navigation example at the end of the MediaGraph demo, and imagine that on an iPad or phone!) [...]
>I like the idea of moving away from hierarchal menu navigation, towards spatial map navigation. It elegantly addresses the problem of personalized user created menus, by making linking and unlinking locations as easy as dragging and dropping objects around and bumping them together to connect and disconnect them. (Compare that to the complexity of a tree or outline editor, which doesn't make the directions explicit.) And it eliminates the need to a special command to move back up in the menu hierarchy, by guaranteeing that every na...
Video with z-depth, maybe computed from stereoscopic video. The complexity added would then be similar to an alpha channel. This would break video out of the rectangles we see in all UI design.
Wikipedia articles follow a format which makes it easy to find information. Perhaps site maps for a VR web could be arranged according to a convention so that users can easily figure out where to look for specific information. Early VR we web tried to do this by arranging content like rooms in a building, and streets, but that is cumbersome to use in practice. I think the wiki format suggests a much better approach can be created.
Very cool work, particularly important is the text quality improvements. Could you talk about the approach to improving text quality is? Are you implementing projected font hinting by any chance?
The core issue with surface (text) quality is essentially that classic texture filtering fails for surfaces here.
So we set a higher DPI than usual, and then use a supersampling algorithm with a suitable kernel as a lowpass filter. This allows us to avoid artifacts while maintaining sharpness.
179 comments
[ 4.8 ms ] story [ 219 ms ] threadOverall it was a pretty rough road, but someone has to go first and I'd really like for Haskell graphics to take off.
I ended up writing bindings for Godot so we didn't have to deal with graphics stuff from scratch (like in the first few iterations).
While this lost us some of Haskell's benefits and it's not super functional-y, it's still nice to work with and most of the issues we encounter are related to the bindings being unpolished.
If you have any questions about doing this kind of project in Haskell, hit me up.
Right now we're working on perfecting Linux VR Desktop (for 2D apps), but the long-term goal is to bring forth killer office VR apps (for productivity, not gaming/entertainment) that wouldn't otherwise be possible in normal 2D Linux. These apps will likely take advantage of VR as an amazing "Simula[ting]" environment.
Barbara Liskov lent me a book that described it in 1973 and recommended that I read about Simula 67 before starting to work on a small part of the the CLU compiler design. I believe the book was Structured Programming by Dijkstra, Hoare, and Dahl. It had a chapter by Dahl describing Simula 67.
Nix, despite issues with everything OpenGL related, was the only thing that actually worked. And Cachix made installation really fast.
PS: if it starts building for some reason during the install process, let us know. We've tested it on various machines but due to some peculiarities specific configurations might require a rebuild even with Cachix. We can add that to the repo so that future installs and updates are faster.
We initially tried to get a static binary with AppImage and even nix-bundle, but those ended up being failed/painful paths.
It's not like Alyx is the only reason to own a VR headset.
PavlovVR is CS:GO in VR and done really well, hilariously good fun.
The world of VR is so diverse it's tricky to recommend things without knowing you.
Personally I'm not much of a gamer so I tend to be interested in narrative stuff, abstract visuasation and geometry stuff or just exploring interesting environments.
And the games I do like tend to have one or more of the above qualities.
You might be an RPG fan, a strategy gamer, an art aficionado , looking for productivity or educational tools or any number of things.
I would argue that working in VR is fundamentally superior to working in PCs/laptops. It's basically 10x the screens, focus, and work immersion. Simula's text quality is also very good (getting around the eye fatigue present in older VR Desktops).
I feel we are just scratching the surface of what's possible. Yes, multiple (arbitrarily sized) windows are neat. But we can probably do more.
Even without any changes though, I just wish my VR headset was lighter. I can go more than 8h straight in VR... playing games. Not sure about working.
Will try this :)
SteamVR does this, but I've found it less useful than movable/resizable windows.
> I feel we are just scratching the surface of what's possible.
Definitely. This is one of the first viable iterations, but there's lots and lots of improvement potential everywhere.
> Even without any changes though, I just wish my VR headset was lighter.
Yeah, agreed. But I think future hardware will improve on that.
On a second thought: if you would make it easier to "fix" window, such that one can focus on it and work. As when you are "placing" a new window in your video, that could make it a workable solution.
I think the wiggling is actually kind of natural -- you don't notice your head or eye movements when you do them automatically, but it might be exacerbated in VR.
The fixing is an interesting idea, we'll try it out. I'm a bit worried it might feel weird in VR.
Corollary discharge[1] causes the brain to “subtract” out the motion of the image on the retina. It happens when our nervous system instructs the eyes to look in another direction. You can have someone else move your eyes or even do it yourself by pressing gently on the side of the eyeball with a fingertip on top of the lid. Wiggling one eye this way is not supported by the hardware of the eye and produces a very noticeable wiggly view of the real world.
I mentioned subtracting out the motion going on in the visual system. Experiments suggest that something like this is happening. If a muscle paralyzing drug is injected into the eye’s muscles it causes a wiggly world because the intent to glance somewhere causes both an ineffective command to move the eye, producing no real movement of the image in the retina while the triggered corollary discharge subtracts the intended motion from the still image and causes a perceived motion (in the opposite direction).
[1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2807735/
I think navigation needs eye tracking, which sadly no headset currently supports. Focus-follows-gaze would be a game changer.
Right now, Simula uses "dumb" eye tracking, in the sense that windows receive keyboard and cursor focus when the user's forward eye gaze intersects a window. We also have it so that users can control the cursor focus with their forward gaze (presently binded to `Super + Apostrophe`); similarly, users can drag windows around by holding `Super + Alt` and looking around. The experience adds up to something quite productive once you learn all the keyboard shortcuts (your fingers don't need to leave the keyboard).
But I agree that wiggling is off putting. I think VR demos should low-pass filter the output to some degree to give a more realistic perception of what VR is like.
But I feel its mostly rotational motion that is perceptible in VR demos rather than translational motion. I suspect that's because a small rotational motion results in a large shift in perspective for objects in the world that are some distance away from the camera while small translational movements don't really result in a significant perspective shift.
Would be interested in seeing what research into human head motion while sitting/standing/walking/etc. shows though.
Yeah, both rotation and translation would be filtered in the idea I had, so that small wiggling motions aren't as visible.
You can have just one window at a time, and with a good headset, the wiggle is not significant. You can even adjust it so that your workspace follows you when your head turns.
https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/d... https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/d...
I haven't used Windows' Virtual Desktop since the early days, but I suspect our text enhancements have improved upon the situation dramatically (when keeping hardware constant).
Have you actually tried VR? The fact that you think there's wiggling makes me think you haven't. Yes, videos make it look that way but I can guarantee you that's not reflected in the actual experience. VR is notoriously bad to watch on video and doesn't capture the experience at all.
As for pinned windows: fixed HUD is a no-no in VR games because it burns in same spot of your eyesight. When a HUD is desired, players are usually given a virtual or symbolic helmet that allow display to lag behind head movement.
For VR gaming (Oculus Rift), I start to experience mild discomfort after about 30min, which grows severe by 60min. It's hard to envision using VR for a full work day. (My eyes are fairly light-sensitive from laser-eye surgery; not sure how much is saccading, vs. bright light directly in front of the eyes.)
https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/d... https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/d...
Many early VR Desktops (and VR games) didn't go to special effort to optimize text quality, and I think this has given people a lower than warranted impression of what is possible in VR today. There's no doubt that VR hardware is going to get exponentially lighter/sharper over time, and that these improvements will be welcome, but things are good enough now to put in 1hr+ long sessions without eye strain (at least in my experience as a daily user).
I think the Index might be OK but I couldn't comment.
So being able to fetch up windows so it's perfectly centered above the keyboard would make it easier for me to use.
Where does Haskell fit in with Godot and Simula?
The webcam view is also interesting; if it were "inverted" so that, instead of being confined to a window, it was outside of all windows (that is, the background "wallpaper"), it would work even better as a laptop replacement (that is, the screen doesn't cover your whole view, allowing you to keep aware of your surroundings).
The AR idea is really interesting. We'll try replacing the background with "reality" as seen from the HMD cameras.
So that's the future, maybe: cubicle dwellers get their own VR offices...
Would allow you to spin the windows around without having to spin yourself, for prolonged periods of "switching over to the right" for instance.
Could use a different gesture to move around the 3D space too allowing you to navigate to different pockets of work
Holy crap, I never thought of it that way!
Anyone know if this is possible with macOS?
It can create extra displays, but you have to buy an Elite license for that.
I never would have even purchased a VR headset had it not been for the promise to start working on a productivity environment. But before we can start building the killer VR office apps, we have to get the basic 2D apps working crisply/perfectly, IMO.
In 2017, I had a conversation with a co-worker, where we talked about how badly we wanted a VR desktop environment with infinite screen-space. There's VRDesktop, but that isn't this. And then again the same conversation last year with another friend.
After seeing the demo video, I was compelled to go look at Vive prices (~$1,000, ouch).
I feel like my productivity would skyrocket with this, mostly due to lack of background distractions. Imagine wearing noise-cancelling headphones. I want this so bad.
Have you thought about an Android/Quest version?
I'll take a look at Quest docs and see if it's doable, but I don't have high hopes for it.
Having said that, getting even a standard desktop setup on Android to work well is iffy so doing it in VR does seem like a pretty immense challenge. But it would be a breakthrough in what it could demonstrate in terms of the concept.
---
But after watching your video, I feel like this really needs some tiling window manager (i3 etc) inspired way of easily creating workspaces with tiles, and effortlessly navigating between them. With relatively minimal head movement.
Also, does Simula support curved displays? It would seem very natural to arrange different curved screens around you instead of flat ones, which also potentially waste quite a bit of useful space.
I'll put curved windows onto our todo list. Not sure if they're a good idea for all applications but at least some would benefit.
I won't get to try out Simula until Sunday, but based on the video, my intuition says the apps being curved would look more natural / immersive.
Now you need something like a knob or dial to rotate the background, so you don't need to move your chair or turn 360.
Here are a couple of links showing off Simula's text quality:
https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/d... https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/d...
Many early VR Desktops (and VR games) didn't go to special effort to optimize text quality, and I think this has given people a lower than warranted impression of what is possible in VR today (even on old headsets).
Disclaimer: I worked at its company at that time
A cross-platform possibility would be if it served up a WebVR page that could be accessed with Firefox Reality.
We'd have to implement a Godot VR interface for the Quest. If they have Linux support at all, this should be doable.
Someone asked if I could watch all the Star Wars movies at the same time. I got 6 trailers running at once in 6 separate virtual monitors with the default built in OS feature.
There's prescription lenses for headsets, but they're fairly expensive and IMO only worth it if you use VR a lot.
Try it!
*IANA optician
Yeah, me too. As far as my eye is concerned, most things are "far away". Interestingly though, I can read things in VR "farther" than I can in real life. I am not exactly sure what the convergence looks like.
From a practical point of view, at least for me, "it just works like normal".
As I understand the biology of it, there are really two different relevant mechanisms for "focus".
The lens that is your eyeball changes shape to focus on things closer/father away in real life. In VR this doesn't happen, there's a fixed focal plane a few meters away. I know this, but I don't notice it at all. Everything is always in focus and my brain doesn't seem to have any trouble dealing with it.
There is also the problem of "vergence", your eyes point at the thing you are looking at, which means they point "closer to eachother" when looking at something close, and in parallel lines when looking at something at infinite distance. VR headsets implement this reasonably well. If you hold a finger a few inches in front of your nose and focus on something a few meters away you should be able to see two copies of your finger (or at least I do, I'm told that other people have to try to see this). This works in VR as well.
As an aside. Trying to play shooters in VR has been surprisingly hard for me because of the vergence issue. I'm not use to aiming a real weapon, and it turns out I have trouble figuring out which near-field image corresponds to the eye I'm trying to aim with while I focus on the target in the distance. I'm not sure if I'd find it easier in real life.
I'd need probably a dozen or so channels of information that I had to cycle between quickly to convince me, given I've gotten used to multi-monitors, workspaces, and keyboard shortcuts. Maybe some futuristic high level analyst job that needs to look through lots of different kinds of information (geographical, text, videos, etc) at once.
For another interesting use case, how about a 3d debugger that lets you follow a program flow over multiple classes and stacks...
1. A way to neatly arrange all the windows on a virtual sphere that surrounds the user, possibly arranging them automatically in a similar manner as a tiling window manager.
2. A way to rotate the before-mentioned sphere around you without forcing the user to rotate it's head. This would avoid much of the neck strain. It could be done by, for example, holding a button on the keyboard while moving the mouse or by a simple keyboard shortcut to rotate the sphere by X degrees in any direction.
This concept could also be extended to virtual desktops where each desktop is a sphere around the next, like an onion, with the ability to "zoom" in to the next desktop.
one more thing for the author to play around with (I don't have the hardware to try it myself) is experiment with speed/acceleration -- the mouse pointer movement has speed and acceleration parameters that affect how it moves, so that if you want you only have to move your mouse very little in real dimensions to move it thousands of pixels.
It might cause motion sickness, but maybe you can get away with pitching/yawing 5 degrees for every 1 degree of head movement, so to look "straight up" you only have to tilt your head up 18 degrees. Hopefully you'd still have the illusion of being oriented in a space.
That would cause extreme motion sickness.
You could reach me at: http://tbf-rnd.life/contact/
Or if you have some ways of contacting you I'd be very happy to do so
[1] https://github.com/SimulaVR/Simula/commit/19cf46894cae1962e9...
[1] https://gitlab.freedesktop.org/xrdesktop/xrdesktop
Our approach with mouse cursors has been to assign every window its own mouse cursor. This is possible in Simula since the active window is the one that the user is currently gazing at, so that -- for example -- if you gaze at one window and move its cursor, it won't affect the other windows' cursors. We also allow users to control mouse cursors with their eye gaze (presently bounded to `Super + Apostrophe`), which is a good productivity boost.
In fact, you can do everything in Simula with just the keyboard (no mouse or VR controllers needed): move windows, move the mouse cursor, click the mouse cursor, etc. Once you learn the shortcuts, it's very quick.
If you are at a desk and/or without a swivel chair, you're pretty limited to a grid of ~9 screens directly in front of you, and even then it looks like you're turning your head a TON to see the outer layer.
Some ideas that might help:
1) To use an analogy, could you increase the "mouse speed" of moving your head? Right now it's 1:1 with the virtual space and makes geometric "sense", but it might be nice to be able to focus on screens on the left/right by moving your head less and having your focus move just as fast. For example, moving your head 50% of what you do now to zoom over to the same place would reduce neck strain when looking at those windows, but also open up a bit more window space if, for example, you could still move your neck 100% of what you're doing and be able to "look over" far enough to fit another set of screens even further out. I don't know if this disconnect in pan speed would be disorienting, but it's worth trying. :) c.f. https://i.imgur.com/0KCo9hG.png
2) Keyboard gestures to "re-center" your focus might also go a long way. Perhaps instead of craning your head to look at a window that isn't "straight ahead" for a long time (as long as you're working in that window), you could do a quick glance at that window and hit a keyboard shortcut that would recenter wherever you're looking to be your "straight ahead", so you could then straighten your neck back out and continue working in that window while in a more comfortable position. c.f. https://i.imgur.com/u6SI284.png
3) Likewise, we don't all have amazing headset resolutions and there's a lot of potentially wasted space whenever focusing on a particular window that's not "full-screened". Perhaps a keyboard shortcut to expand whatever screen you're looking at to the full size of your view would be helpful for eye strain. Even something that could temporarily "pin" a window to your view (full screen and moving with your gaze) might go a long way toward minimizing unused focus/pixels. c.f. https://i.imgur.com/yA1twKf.png
My immediate thought watching this was to mount a keyboard on a swivel chair, but I think requiring end-users to get customized hardware to take full advantage of your environment might be the wrong way to go. :)
2) I believe we already have that (via Super-')
3) Good idea. Being able to instantly fullsize and "pin" a window sounds good, maybe even as a toggle.
Your brain adapts, and starts to believe at a primal level that "things fall this speed when I drop them," "I move this fast when I walk", "when I point over there, it's at this angle," etc.
If anybody remembers playing GTA for too long, and later walking outside with the feeling like you ought to be able to jump in the nearest car and drive it a way, you have an inkling of how your brain gets remapped.
Remarkably quickly, moving around in the real-world can feel "off" to the same degree that it felt "off" when you first entered the VR world when you start messing with these primal feedback loops. Get ready to bump into things, and don't operate any heavy machinery if you've been in a non-1:1 VR mapping.
X11 apps are supported via Xwayland.
[0]https://github.com/oakes/vim_cubed
The big win, as far as I can tell, would be to engage the user's spacial memory. (There is a small but non-zero niche for 3D visualization of complex systems: weather, large molecule, etc.) You're going to want to combine "memory palace" with "zooming" UI in a kind of pseudo-3D (I think of it as 2.618...D but the exact fractal dimension isn't important I don't think.) Then infuse with Brenda Laurel's "Computers as Theatre"...
https://en.wikipedia.org/wiki/Second_Life
https://en.wikipedia.org/wiki/Method_of_loci
https://en.wikipedia.org/wiki/Zooming_user_interface
https://en.wikipedia.org/wiki/Brenda_Laurel - https://www.goodreads.com/book/show/239018.Computers_as_Thea...
https://web.archive.org/web/20111102002100/http://nooface.ne...
Wait until resolution improves and we break out of the "desktop" paradigm. We could have a collection of unlimited windows and tabs that exist in a continuum around us, and we could use gestures to organize and surface the contextually relevant ones.
We won't need a bulky multi-monitor setup, and we could work remotely nearly anywhere. Imagine carrying your workspace with you.
> The big win, as far as I can tell, would be to engage the user's spacial memory.
Absolutely! Physical workspaces and work benches are incredibly functional because we are spatial animals. Breaking out of the limitations of using a screen could unlock more of our senses for use in problem solving.
I'm extremely excited about this technology. It will be great for software engineers, creatives (2d and 3d artists), mechanical engineering, CAD, ... you name it.
I really hope this keeps getting pushed forward. While I'm using all of my spare cycles on a tangentially-related problem domain, I'd be more than happy to donate money and ideas. This technology will be a dream come true if it continues to mature.
Eight hours+ of using gestures a day sounds like hell.
You can get 90% there by just using multiple desktops IMHO, at least that's my experience.
Headsets have improved almost ~2x in resolution and have halved in price and some have become wireless with better optics. A long ways from 4-5 years ago, but still need another doubling of resolution (or maybe more) and an increase in wearing comfort (lighter, more compact) plus an improvement in wireless latency and maybe a reduction from $400 to $300, and you’re looking at something that would be useful just as a replacement for multiple monitors.
Plus probably improvement with registering where your laptop and mouse are automatically. In principle that could be done with a software update to inside-out tracking software.
Additionally, some improvement is possible with similar to current resolution but with improved subpixel rendering and RGB pixel layout.
Seeing what has been done with the Oculus Quest since I last checked out VR like 3 years ago has left me pretty impressed. A lot of this stuff with multiple windows in this demo could be done natively and wirelessly with the Quest (which runs a kind of Linux). The inside-out tracking is impressively good. If combined with an insert you can put your tracked controllers on Bluetooth mouse and keyboard (so the Quest can register their positions in3D space to allow proper rendering in-headset), it could give you a high productivity workstation experience just about anywhere with WiFi (could be through phone). Hand-tracking (Which works already) could even allow gestures, although I’m not sure how important that is. Can the Quest do subpixel text rendering like ClearType but 3D?
There's no reason you can't make the windows always perpendicular to the line of sight while still fixing their centers in space.
I don't know if you're still watching this thread, but it would be interesting to hear an example of when one might care to lock a window's orientation.
I've experimented with combining the kinesthetic advantages of pie menus and gesture with the method of loci and zooming interfaces, including a desktop app called MediaGraph for arranging and navigating music, and an iPhone app called iLoci for arranging notes and links and interactive web based applets.
https://medium.com/@donhopkins/mediagraph-demo-a7534add63e5
>MediaGraph Music Navigation with Pie Menus. A prototype developed for Will Wright’s Stupid Fun Club.
>This is a demo of a user interface research prototype that I developed for Will Wright at the Stupid Fun Club. It includes pie menus, an editable map of music interconnected with roads, and cellular automata. It uses one kind of nested hierarchical pie menu to build and edit another kind of geographic networked pie menu.
MediaGraph Demo Video:
https://www.youtube.com/watch?v=2KfeHNIXYUc
https://medium.com/@donhopkins/iphone-app-iloci-by-don-hopki...
>iPhone iLoci Memory Palace App, by Don Hopkins @ Mobile Dev Camp. A talk about iLoci, an iPhone app and server based on the Method of Loci for constructing a Memory Palace, by Don Hopkins, presented at Mobile Dev Camp in Amsterdam, on November 28, 2008.
iLoci Demo Video:
https://www.youtube.com/watch?v=03ddG3jWF98
Here's some more discussion about window managers, the Method of Loci, MediaGraph and iLoci, and pie menus:
https://news.ycombinator.com/item?id=22089694
DonHopkins 81 days ago | parent | favorite | on: Nototo – Build a unified mental map of notes
>Great idea, I totally get it! Your graphics are beautiful, and the layering and gridding look helpful. It reminds me of some experimental user interfaces with pie menus I designed for creating and editing memory palaces: "iLoci" on the iPhone for notes and pictures and links and web browser integration in 2008, and "MediaGraph" on Unity3D for organizing and playing music in 2012, both of which I hope will inspire you for ideas to implement (like pie menus, and kissing!) or ways to explain what you've already created.
>A memory map editor can not only benefit from pie menus for editing and changing properties (like simultaneously picking a font with direction, and pulling out the font size with distance, for example), but it's also a great way for users to create their own custom bi-directionally gesture navigable pie menus by dragging and dropping and "kissing" islands together against each other to create and break links (like bridges between islands). (See the gesture navigation example at the end of the MediaGraph demo, and imagine that on an iPad or phone!) [...]
>I like the idea of moving away from hierarchal menu navigation, towards spatial map navigation. It elegantly addresses the problem of personalized user created menus, by making linking and unlinking locations as easy as dragging and dropping objects around and bumping them together to connect and disconnect them. (Compare that to the complexity of a tree or outline editor, which doesn't make the directions explicit.) And it eliminates the need to a special command to move back up in the menu hierarchy, by guaranteeing that every na...
Volumetric video, spatial website navigation design guidelines, and augmented reality are high on m wish-list.
Wikipedia articles follow a format which makes it easy to find information. Perhaps site maps for a VR web could be arranged according to a convention so that users can easily figure out where to look for specific information. Early VR we web tried to do this by arranging content like rooms in a building, and streets, but that is cumbersome to use in practice. I think the wiki format suggests a much better approach can be created.
So we set a higher DPI than usual, and then use a supersampling algorithm with a suitable kernel as a lowpass filter. This allows us to avoid artifacts while maintaining sharpness.