> For those hoping to add HaptX to their dream gaming rig, we have some bad news for you: The company is currently developing the technology specifically for industrial, manufacturing, and government. While the technology has a wide range of applications in these sectors, the biggest one is training.
I imagine a lot of things tested at the USAF base aren't the "most advanced in the world"—and frankly the assumption that they would be is somewhat odd. There's only so much throwing wads of cash at contractors can solve.
Nah, HaptX genuinely is on the leading edge of this industry. No one else is as advanced or has the same quality of haptics. Which is why their price is so incredibly high.
Nifty, but the article should have elaborated on the use cases. It suggested that training is the biggest one, but are these really a substitute for actually feeling real-world things? Seems like they’d have to be almost magically precise to, say, simulate the difference between quality and unacceptable fabric.
Yeah, if this article was more balanced and less sensational they might have touched on(heh) how incredibly sensitive our sense of touch/proprioception in our fingers and hands is, especially if trained[1].
I work on VR, for training various types of mechanics and machine operators. I've tried every handtracking/glove/touch system under the sun, and tested integrating them with my product when possible. I've tried the HaptX gloves at various trade shows, and even had them to my office to demo their newest stuff personally.
There is no use case for HaptX. They're not a substitute for anything. They're not magically precise. They're nothing.
The technology to simulate texture with that amount of detail does not exist yet. HaptX has a few dozen tiny pneumatic pistons poking your hand in various places and that's about the best thing available today.
The industry has really only found use in training so far. But it's pretty promising in my opinion.
As to efficacy, I don't know if there have been rigorous studies yet, the industry is still pretty new. But anecdotally we do see that users build better muscle memory that reduces training time with real world equipment.
The idea is generally to have users do their first round of training virtually instead of with real equipment. Then they require less time with the real equipment and don't make newbie mistakes that break things.
You'd be surprised at just how poor quality the haptics can be and still be convincing. VR has some interesting psychological effects that make your perception of the haptics much better than it actually is. What you see can actually override your proprioception to a large enough degree that we can get away with not being so precise.
I've tried these before. They use directed air onto your hands hence the backpack. They do a pretty okay job at providing a sense of texture. So virtual objects have a surface feeling, but still no physical shape or resistance. The demo has a bunch of feely textured objects you can handle: steam, dripping water, a cat, a rope, etc. You still have to pretend that you "stop" at the volume of whatever object as the system still doesn't provide any physical resistance. Some of the demo surfaces were kind of "meh" but a few, like the rope felt spot on to what a rough hemp rope being touched through a glove might feel like.
To me the biggest problem outside of that was that the system can't really provide the sense of touch of a static object. Like resting your hands on the thorns of a cactus. Because it uses streams of air bubbles, it has to be a continuously changing surface, so rubbing your hand or finger on something works better.
Is it more immersive? Yeah, to a point. I personally think that a feedback system that let you feel the volume or shape of gripped objects would be better, but these are very hard problems.
One can imagine fanciful solutions, but people tend to get hurt in VR space. It would be very, very hard not injure the humans in meat space with tech that is sufficiently rigid to let them also physically interact with vr objects.
I'm reminded of the description in "Rainbow End?" where the guy doesn't have a VR setup yet and sees people reaching out and touching shapes held in the air by mechanical arms. Basically, there are 'self-arranging blobs' that arrive 'just in time' to your interaction, changing their tactile surface to reflect what you expect. The user sees none of this, and only what they see in their interface.
I can't imagine it working perfectly. Say, pulling a book out of a shelf should have exactly zero resistance (other than the force of gravity pulling the book down). Having near zero latency between action and the 'tactile interface' moving in exactly the right direction would be a very hard problem...
Not sure how dangerous it has to get. Just having some basic motors that counteract the force you exert with a fast response time probably couldn’t do much damage. An intelligent system to also check that you maintain your balance (or only doing this seated) probably takes away a bunch of risk.
Well first are you wearing just haptic gloves or are you also wearing a mask? In the first case nothing would happen obviously. In both cases the system should be able to distinguish you exerting a force and matching a counteracting force to give a sense of pressure vs VR contact that can be simulated through significantly less force (eg a haptic buzz).
I imagine a force-feedback exo-suit that turns you into a mime, but with sensation. It won’t necessarily give you a punch in the face, but could afford resistance and virtual heft to some virtual objects, and would have been an interesting scene in an Iron Man movie.
I work on gloves like this, and that's really not what we see in the real world.
The active force applied to the fingers is actually pretty small. The geometry of a flexed hand gives huge mechanical advantage. The key is a large holding force. It's easy to prevent you from closing your hands further without needing to actually overpower you.
You don't need to pull hard, you just have to be able to stop hard. No real risk there.
See the seldon_mallow comment with the link to https://www.fluidreality.com/, they do this with small devices that don't need stored air. The "breakthrough" here feels like the connection between the haptic array and the software, if that is the case, then the implementation can be done in lots of ways.
HaptX does have a force feedback system, maybe it just wasn't on that day?
> the system can't really provide the sense of touch of a static object.
Yeah, that's hard because of how you perceive weight. We'd have to be able to pull your forearm down relative to the elbow, and that's not exactly easy to do elegantly. I've thought a lot about it and haven't been able to come up with something that isn't a ridiculous exoskeleton for your arms and shoulders.
> I personally think that a feedback system that let you feel the volume or shape of gripped objects would be better, but these are very hard problems.
I agree! It really does make the interaction more convincing. I've been working on this problem for the last year or so and it is indeed quite hard. But we're nearly there. Maybe this year ;)
I was deep in this space for a while. For those interested in digging deeper, It's a few years old but one of the better and more comprehensive documents that came out at the time (2021) was this one. Not sure how much has changed since then
A Systematic Review of Commercial Smart Gloves: Current Status and Applications
I work for a competitor, but I do know a fair bit about HaptX's system.
The force feedback is driven with a complex exotendon (cable) system that can give dynamic resistance as well as just pulling on you.
The tactile haptics are much more interesting. It's a grid of piston actuators in several places along the palm and fingers. They need the huge backpack and separate gigantic floor unit with a compressor to generate the pressure required to push those pistons fast and hard enough.
They also have their own hand and finger tracking. It's actually a pretty cool system.
As an engineer, I respect the hell out of what they're doing. The haptics really are the best thing on the market right now. Unfortunately the cost is so high and the use case so narrow that it just isn't worth it.
So far, VR haptics has really only found use in training and simulation. There's real potential there, but it's a very small market right now. But we do see that it genuinely does help shorten training times when we convince someone to use it.
I wonder if you couldn’t just have gloves with small cushions inflated with air along the fingers and the palm, to simulate light physical resistance. Obviously you could overpower those easily, so it wouldn’t work for some scenarios, but still — shouldn’t that allow for a cheaper way than having huge setup like this?
The problem is latency. Your actuator needs to be able to move from 0 to 100% actuation within a half second or so. Ideally as fast as possible. If there's significant delay between visually seeing your hand contact an object and feeling the haptics, it becomes worse than useless.
To make pneumatic actuators go really fast, you need a lot of pressure. The reason the HaptX system is so big is that they have dozens of individual actuators. You need a lot of valves, which is what's in the backpack. The compressor is a separate unit that sits on the floor. It's also huge.
But this is very much an area of active research. There's a lot of interesting actuators out there. Overall, people seem to be moving away from pneumatics because the cost and complexity is so high.
Awesome, thank you for the thorough explanation. I wasn’t aware that pressure was so big of an engineering challenge here, but that makes perfect sense.
Could there be an application for this to be used in conjunction with an exoskeleton suit (to accommodate the giant backpack)?
Another comment mentioned that these gloves have an application in NN training. Warehouse workers could wear the exoskeleton “bottoms” while providing training data for manipulating various objects.
I'm not sure what the gloves add to this scenario. The point of haptic gloves is to give the sensation of virtual objects. It sounds like this idea would work much better by simply pointing an Ultraleap camera at a human doing the task. You want hand tracking data from a human doing a real thing. Haptic feedback isn't helpful here.
Besides, I don't think that training a neural network on hand motions is actually that useful. What you're talking about is a kinematics problem that is totally tangential. You'd want the robot to understand how to manipulate objects, not hands, and not a human body. Watching how a human moves doesn't tell you how a robot moves. The neural network would need to learn how to manipulate arbitrary objects with a robot body. Capturing human motion doesn't help much.
Would love to try one of these :) With regards 'only', only can get quite big, 'The global Military Simulation and Training market size was valued at USD 9815.49 Million in 2022 and will reach USD 11603.49 Million in 2028, with a CAGR of 2.83% during 2022-2028.' (from https://www.linkedin.com/pulse/military-simulation-training-...)
These are too complex and expensive for consumer use now and for the foreseeable future. However there is an important application where these may end up being key: teleoperation of robot hands to produce training data for neural nets.
We will probably need millions of hours of data of robots doing everyday tasks to train our neural net models. Collecting that data might mean humans puppeteering robots very precisely but also very quickly. Without haptic gloves, human operators would be attempting tasks with no sense of touch at all, and their movements would be unnatural and inefficient, ultimately resulting in the trained robot moving inefficiently too.
This is already happening in some ways: https://www.thehandy.com/ (NSFW!), it replicates the motions exhibited in VR videos on the user's "body part" ;)
A glove for more interactivity would be a lot more difficult and expensive but really cool :)
If people are wondering about applications, giving blind people the opportunity to unlock their design visions in a CAD environment would be a good one.
About 2 decades ago, I visited an international school for the blind to talk about technology needs and this was at the top of their list. A lot of blind people would like to be engineers, but they lack the ability to use CAD to be competitive in industry.
49 comments
[ 1.3 ms ] story [ 118 ms ] threadI want to buy one anyway, even if it's $20,000.
[1] https://www.universityofcalifornia.edu/news/amazing-sensitiv...
There is no use case for HaptX. They're not a substitute for anything. They're not magically precise. They're nothing.
The industry has really only found use in training so far. But it's pretty promising in my opinion.
As to efficacy, I don't know if there have been rigorous studies yet, the industry is still pretty new. But anecdotally we do see that users build better muscle memory that reduces training time with real world equipment.
The idea is generally to have users do their first round of training virtually instead of with real equipment. Then they require less time with the real equipment and don't make newbie mistakes that break things.
You'd be surprised at just how poor quality the haptics can be and still be convincing. VR has some interesting psychological effects that make your perception of the haptics much better than it actually is. What you see can actually override your proprioception to a large enough degree that we can get away with not being so precise.
To me the biggest problem outside of that was that the system can't really provide the sense of touch of a static object. Like resting your hands on the thorns of a cactus. Because it uses streams of air bubbles, it has to be a continuously changing surface, so rubbing your hand or finger on something works better.
Is it more immersive? Yeah, to a point. I personally think that a feedback system that let you feel the volume or shape of gripped objects would be better, but these are very hard problems.
One can imagine fanciful solutions, but people tend to get hurt in VR space. It would be very, very hard not injure the humans in meat space with tech that is sufficiently rigid to let them also physically interact with vr objects.
I can't imagine it working perfectly. Say, pulling a book out of a shelf should have exactly zero resistance (other than the force of gravity pulling the book down). Having near zero latency between action and the 'tactile interface' moving in exactly the right direction would be a very hard problem...
Seated is kind of stupid imo.
The active force applied to the fingers is actually pretty small. The geometry of a flexed hand gives huge mechanical advantage. The key is a large holding force. It's easy to prevent you from closing your hands further without needing to actually overpower you.
You don't need to pull hard, you just have to be able to stop hard. No real risk there.
> the system can't really provide the sense of touch of a static object.
Yeah, that's hard because of how you perceive weight. We'd have to be able to pull your forearm down relative to the elbow, and that's not exactly easy to do elegantly. I've thought a lot about it and haven't been able to come up with something that isn't a ridiculous exoskeleton for your arms and shoulders.
> I personally think that a feedback system that let you feel the volume or shape of gripped objects would be better, but these are very hard problems.
I agree! It really does make the interaction more convincing. I've been working on this problem for the last year or so and it is indeed quite hard. But we're nearly there. Maybe this year ;)
A Systematic Review of Commercial Smart Gloves: Current Status and Applications
(weblink) https://www.mdpi.com/1424-8220/21/8/2667
(pdf link) https://mdpi-res.com/d_attachment/sensors/sensors-21-02667/a...
The force feedback is driven with a complex exotendon (cable) system that can give dynamic resistance as well as just pulling on you.
The tactile haptics are much more interesting. It's a grid of piston actuators in several places along the palm and fingers. They need the huge backpack and separate gigantic floor unit with a compressor to generate the pressure required to push those pistons fast and hard enough.
They also have their own hand and finger tracking. It's actually a pretty cool system.
As an engineer, I respect the hell out of what they're doing. The haptics really are the best thing on the market right now. Unfortunately the cost is so high and the use case so narrow that it just isn't worth it.
So far, VR haptics has really only found use in training and simulation. There's real potential there, but it's a very small market right now. But we do see that it genuinely does help shorten training times when we convince someone to use it.
The problem is latency. Your actuator needs to be able to move from 0 to 100% actuation within a half second or so. Ideally as fast as possible. If there's significant delay between visually seeing your hand contact an object and feeling the haptics, it becomes worse than useless.
To make pneumatic actuators go really fast, you need a lot of pressure. The reason the HaptX system is so big is that they have dozens of individual actuators. You need a lot of valves, which is what's in the backpack. The compressor is a separate unit that sits on the floor. It's also huge.
But this is very much an area of active research. There's a lot of interesting actuators out there. Overall, people seem to be moving away from pneumatics because the cost and complexity is so high.
Another comment mentioned that these gloves have an application in NN training. Warehouse workers could wear the exoskeleton “bottoms” while providing training data for manipulating various objects.
Besides, I don't think that training a neural network on hand motions is actually that useful. What you're talking about is a kinematics problem that is totally tangential. You'd want the robot to understand how to manipulate objects, not hands, and not a human body. Watching how a human moves doesn't tell you how a robot moves. The neural network would need to learn how to manipulate arbitrary objects with a robot body. Capturing human motion doesn't help much.
You want pressure and force information too, for helping train the robot to be dextrous.
like, have you found if a slight counter-pressure makes for a more significant/faster feeling physical pressure?
We will probably need millions of hours of data of robots doing everyday tasks to train our neural net models. Collecting that data might mean humans puppeteering robots very precisely but also very quickly. Without haptic gloves, human operators would be attempting tasks with no sense of touch at all, and their movements would be unnatural and inefficient, ultimately resulting in the trained robot moving inefficiently too.
Tesla has a motion capture suit that can be seen here: https://youtu.be/XiQkeWOFwmk?t=33 In this video they aren't doing teleoperation but in this other video demonstrating teleoperation you can briefly see a hand in the bottom right corner which looks like it could be wearing a haptic glove: https://twitter.com/elonmusk/status/1746964887949934958
A glove for more interactivity would be a lot more difficult and expensive but really cool :)
[1] https://www.aptx.com/about-aptx
About 2 decades ago, I visited an international school for the blind to talk about technology needs and this was at the top of their list. A lot of blind people would like to be engineers, but they lack the ability to use CAD to be competitive in industry.