I've done the block demo (or a block demo) on an Oculus with LeapMotion and it is truly amazing. It made those generally well liked Vive controllers feel downright clunky. Of course there'll be situations where buttons and feedback are appropriate, but even in its early form, Leap Motion (after the Orion update) was quick, accurate, and enough to convince me that hand tracking is an essential part of VR. It's so immersive and really feels like it's your hands in there.
> Leap Motion (after the Orion update) was quick, accurate (...)
but not stable. 10% of the time one hand goes away, gets out of range.. It's really good, but not enough for mass -or even eventual- consumption, imho.
Orion must have had a bit of a placebo effect or something. When I first installed it, I had a sense "this is better!" But after trying to write applications with it and use it more frequently, I found I was hitting the same old problems, just as often as before.
Let's also not forget that Orion came out a little over a year ago now. Still in beta. Still pretty janky. I just don't have a lot of confidence that Leap will clean up their issues and make a better product.
There is another device, Occipital Structure Sensor (https://structure.io/), that I'm keen to try out. It's essentially a micro Kinect you can strap your face. Supposedly enables room-scale, inside-out tracking and pass-through AR on smartphones. They support desktop devices, but frankly, if I'm on desktop, I want an OpenVR-based system. For mobile devices, the company officially only supports iOS, which is complete balls. However, I have heard tale that some people from parts unknown have gotten it to work on Android using the now-ancient OpenNI 2 (https://github.com/occipital/OpenNI2).
There's an additional problem with all of these devices, though: they only face front and they aren't able to track the body's full range of motion. It's severely limiting for what you can do.
The structure sensor is okay but it still has far too much latency to be acceptable. So far the only convincing inside out tracking I've seen has been the Hololens and even that isn't quite perfect.
Yeah, I frequently tell people that the Hololens is amazing from a technology standpoint, hot garbage from an actual usefulness one. For my purposes, the Vive is the only system that does a thing that is usefully different enoug from standard WIMP systems--and does it well enough--to bother putting up with its respective defects.
But if we consider what the "holy grail" of MR might be--an unobtrusive device that conforms to the user rather than force contrivances on them--that foretold device is more likely to be in the HoloLens' lineage than the Rift or Vive.
The problem with Orion is that any tracking in VR needs to be 100% accurate, or nearly. Orion is awesome when it works, but the amount of time it fails to work is a disorienting, frustrating, critical dealbreaker for now.
I love the idea of a prototype-ish headset with work in progress stuff the industry is working on. Inside out tracking, eye tracking, finger tracking, etc. They should make it super modular and very interoperable in the software so that third parties can integrate their own stuff, like haptics, individually tracked modules, extra cameras, depth cameras, FIR cameras.
Reminds me of the earlier Oculus headsets when it was more engineer friendly.
Because as an industry we are still in the exploration phase, let's keep experimenting with everything and see what works and what doesn't.
there was a video on here a few days ago of Gabe talking about Valves VR strategy.
he kept reiterating how important it was for them to keep their ecosystem open because it helps their chances to succeed either on the software or hardware side of things.
now there may have been some lip service, but he did seem to genuinely believe that all VR developers working independently has a greater chance chance to hurt actors in the industry more than help.
Yes, and I think the tracking pucks are a prime example of this. We don't exactly know what people are going to do with them, they put it out there for people to figure it out. Track a baseball bat, track your pet, track a camera, track an RC vehicle...
Valve is in a uniquely comfortable situation: they don't have to win VR, they only need to avoid losing. They can be open to cooperation because cooperating with them implies accepting that a huge chunk of monetization is already spoken for.
yes, Valve does have an advantage- which might suggest other companies could not copy their playbook.
but, i think the point we can generalize is that in an underdeveloped space that requires a lot of innovation, any company creating a walled garden is putting themselves at an explicit disadvantage by not contributing to the landscape at large.
I was struggling to understand the meaning of the world 'Foveated', thinking it was Field-of-View-something-something, but the Fovea is actually the back part of your eyeball:
I like this. I think there is definitely room in the VR space (no pun intended), for variants of headsets and hardware. In the same way we have laptops for portable workers, and high end towers for dedicated gamers, right down to just smartphones and tablets for those people who don't actually need a 'computer - we will hopefully have different VR equipment for different use purposes. The more well-heeled among us might even invest in a few headsets, each for a different requirement.
As much as I was woefully underwhelmed by Ready Player One - the haptic VR suits are definitely something I'd like to see developed.
I think it's amazing that they are pushing this tech out without waiting for it to be in a shipped device.
I think that the non-gloved hand tracking is amazing(as an entry level device) although for me I'd defiantly want haptic feedback gloves.(Ed:at home)
While I don't own a hmd,I do have friends that do and have tried a few times.
I cannot explain how excited I am for things like this(specifically the wireless hand tracking), as while gaming is great(I am a huge gamer), I can see how I could use this for training and apprentices.
Example: I have been a chef for a long time, if I want to portion a filet of beef, i could almost do it with my eyes closed.
But, if I want to show someone, it takes quite a few sides for them to begin to understand why I'm doing what. There are many cuts I make because I know what is underneath, and I can get a feel for the filet due to experience.
If I were to try and show that to them, I would have to stop before every cut and rotate the meat and try to show why, and sometimes it's impossible until after the cut. This is what I currently do.
If they had the option to review what I had done with full freedom of rotation?(yes I figure the camera requirement would be high, but I have taught so many people, just doing it once with 30+ cameras would be worth it), it would decrease my training time by a lot.
Also consider portioning. If I'm portioning waygu, 15 grams off is BAD(I aim for 5 grams at my level). I can do that, but I'd never let an apprentice touch it. Meat isn't the same thickness throughout the cut.
BUT if they could practice with hand tracking? They would learn without costing me a fortune. Just training hours(which for portioning it would be about 100 units of waygu, will be cheaper compared to a slip which could cost you 30-50 grams on a 150gram portion).
As a techie and a chef, I am super excited about these sorts of things.
Edit:fixed fomatting, spelling mistakes and better explanations.
Replying for a side note. If anyone is working in this space, or is trying, drop me an email(in profile), I'd love to be involved. While I'm quite busy, it's something I'd love to put my name to :-D
In a similar vein, can someone build a machine that teaches people how to ski or snowboard? The balancing is the key, so the simulator can't just be goggles on your eyes, it needs to make you feel gravity, sliding, falling sideways, recovering... Any ideas?
As an avid snowboarder, I think we are a long way from building a machine to teach someone to do this. It's quite a bit more complicated than just balance - edge control IMHO was the hardest thing to learn.
I'm not suggesting it's impossible to create a machine to do so, but I would put it on the same difficulty level as building haptic gloves. (Creating haptic "boots" with separate platforms maybe?)
CASI instructor turned AR/VR developer here and I'll second this - balance is only a part of the equation. The model used to train instructors includes in brief;
- Flexion / extension of the legs
- Forward / aft pressure
- Subtle steering movement with the knees
- Fine control over the angles formed by both knees and ankles
All of the above vary depending what point of a given turn you're currently in and need to be coordinated in a fairly intricate fashion to get perfect pencil lines and turn "properly". I'll leave properly in quotes there because YMMV and you don't need these techniques to snowboard; having spent 3 months full time learning and using them I can say they make a big difference to how far you can push yourself though.
A significant proportion of the sensation also comes from the inertia that you carry whilst on an edge, the resistance from which would be near impossible to convey without doing it for real.
I think for both examples VR becomes an immersion add on to a physical machine. Both examples are interesting because they involve much larger forces than haptics will provide.
For example you can imagine a ski trainer like an artificial wave so you get the real forces and the sense of exposure you get looking down a large mountain. For cutting meat you need a simulation medium that can be cut with a knife and change density somehow on command.
Foveated rendering is going to be the most significant step towards making mobile VR viable, so this is very good news.
I'm curious as to how they will handle eye tracking. I hope it uses capacitive lenses that can track your pupil bulges instead of unweildly and power hungry internal cameras + CV processing.
29 comments
[ 3.1 ms ] story [ 85.2 ms ] threadbut not stable. 10% of the time one hand goes away, gets out of range.. It's really good, but not enough for mass -or even eventual- consumption, imho.
Let's also not forget that Orion came out a little over a year ago now. Still in beta. Still pretty janky. I just don't have a lot of confidence that Leap will clean up their issues and make a better product.
There is another device, Occipital Structure Sensor (https://structure.io/), that I'm keen to try out. It's essentially a micro Kinect you can strap your face. Supposedly enables room-scale, inside-out tracking and pass-through AR on smartphones. They support desktop devices, but frankly, if I'm on desktop, I want an OpenVR-based system. For mobile devices, the company officially only supports iOS, which is complete balls. However, I have heard tale that some people from parts unknown have gotten it to work on Android using the now-ancient OpenNI 2 (https://github.com/occipital/OpenNI2).
There's an additional problem with all of these devices, though: they only face front and they aren't able to track the body's full range of motion. It's severely limiting for what you can do.
But if we consider what the "holy grail" of MR might be--an unobtrusive device that conforms to the user rather than force contrivances on them--that foretold device is more likely to be in the HoloLens' lineage than the Rift or Vive.
Reminds me of the earlier Oculus headsets when it was more engineer friendly.
Because as an industry we are still in the exploration phase, let's keep experimenting with everything and see what works and what doesn't.
he kept reiterating how important it was for them to keep their ecosystem open because it helps their chances to succeed either on the software or hardware side of things.
now there may have been some lip service, but he did seem to genuinely believe that all VR developers working independently has a greater chance chance to hurt actors in the industry more than help.
but, i think the point we can generalize is that in an underdeveloped space that requires a lot of innovation, any company creating a walled garden is putting themselves at an explicit disadvantage by not contributing to the landscape at large.
https://www.youtube.com/watch?v=GKR8tM28NnQ
It should really help to reduce the GPU requirements for super high-res VR screens.
https://en.wikipedia.org/wiki/Fovea_centralis
As much as I was woefully underwhelmed by Ready Player One - the haptic VR suits are definitely something I'd like to see developed.
I think that the non-gloved hand tracking is amazing(as an entry level device) although for me I'd defiantly want haptic feedback gloves.(Ed:at home)
While I don't own a hmd,I do have friends that do and have tried a few times.
I cannot explain how excited I am for things like this(specifically the wireless hand tracking), as while gaming is great(I am a huge gamer), I can see how I could use this for training and apprentices.
Example: I have been a chef for a long time, if I want to portion a filet of beef, i could almost do it with my eyes closed.
But, if I want to show someone, it takes quite a few sides for them to begin to understand why I'm doing what. There are many cuts I make because I know what is underneath, and I can get a feel for the filet due to experience.
If I were to try and show that to them, I would have to stop before every cut and rotate the meat and try to show why, and sometimes it's impossible until after the cut. This is what I currently do.
If they had the option to review what I had done with full freedom of rotation?(yes I figure the camera requirement would be high, but I have taught so many people, just doing it once with 30+ cameras would be worth it), it would decrease my training time by a lot.
Also consider portioning. If I'm portioning waygu, 15 grams off is BAD(I aim for 5 grams at my level). I can do that, but I'd never let an apprentice touch it. Meat isn't the same thickness throughout the cut.
BUT if they could practice with hand tracking? They would learn without costing me a fortune. Just training hours(which for portioning it would be about 100 units of waygu, will be cheaper compared to a slip which could cost you 30-50 grams on a 150gram portion).
As a techie and a chef, I am super excited about these sorts of things.
Edit:fixed fomatting, spelling mistakes and better explanations.
I'm not suggesting it's impossible to create a machine to do so, but I would put it on the same difficulty level as building haptic gloves. (Creating haptic "boots" with separate platforms maybe?)
- Flexion / extension of the legs - Forward / aft pressure - Subtle steering movement with the knees - Fine control over the angles formed by both knees and ankles
All of the above vary depending what point of a given turn you're currently in and need to be coordinated in a fairly intricate fashion to get perfect pencil lines and turn "properly". I'll leave properly in quotes there because YMMV and you don't need these techniques to snowboard; having spent 3 months full time learning and using them I can say they make a big difference to how far you can push yourself though.
A significant proportion of the sensation also comes from the inertia that you carry whilst on an edge, the resistance from which would be near impossible to convey without doing it for real.
For example you can imagine a ski trainer like an artificial wave so you get the real forces and the sense of exposure you get looking down a large mountain. For cutting meat you need a simulation medium that can be cut with a knife and change density somehow on command.
Produce comes in amazingly messy and different from one week to another, learning how to prepare itnwith very little waste takes a long time.
(https://store.thundercomm.com/en/svrproducts)
EDIT: That's ^ the old version. New one with the 835 is coming out in Q2. Dev signup is here: https://developer.qualcomm.com/
I'm curious as to how they will handle eye tracking. I hope it uses capacitive lenses that can track your pupil bulges instead of unweildly and power hungry internal cameras + CV processing.
This is what contrast-preserving foveated rendering looks like in VR: https://youtu.be/lNX0wCdD2LA?t=63