This seems like it could go well with AR glasses. Could even be extended to scan the mouth and recognize subvocalizations so you could talk to it without emitting sound or opening your mouth. I.e. a general purpose input device.
That would be a game changer, if feasible. I used to wonder why no one tried building a transceiver that picked up speech vibrations from skull perhaps through the ear cavity.
Great idea! Does anyone know what the current state of brainwave navigation is? I saw a flurry of investment in new startups over the last 5 years but it's all gone a bit quiet
It turns out that the promised innovations never came, not without having to put in way more work training the brain than buyers of a toy wanted to put in.
You can buy a Neurosky off eBay to play with for < $200 if you're really interested in trying to do said training.
I am half way through "The Battle for Your Brain: Defending the Right to Think Freely in the Age of Neurotechnology" by Nita A. Farahany [1] and it is not a matter of if; but of when. I can only guess that the same advances in transformers etc that are buzzfeeding the LLM space right now will also accelerate BMI systems. Introducing new input and output sensory pathways between the brain and the world is yet another potentially transformational species defining tech for society to watch for on the radar screen as if there was not enough going on in that department.
Unfortunately, Steven Hawking's muscular dystrophy affected his tongue as well so he was unable to use it, but it was in that context that I heard about it.
It requires custom hardware to fit each user's mouth properly, and as a medical device, a doctor's note is required. Or at least way back when. The FDA may have moved on that, enabling this device this time around.
My first impression was What the actual F**, but after watching the video I'm so blown away and impressed by how this can make people's life better. it's really exciting.
I played around with Tobii eye-tracking hardware half a decade ago and was pretty amazed by its precision. I understand now it also has Glasses (vs. the model I tried, a tracker you put below your monitor usually). Marrying eye-tracking glasses with this mouthpad might be a winning combination: eye-tracking being the mouse to navigate in xy-space, and mouthpad for clicks.
Kudos to this group for putting out a finished product though, can't wait to see it used and iterated more out in the field.
I built a little voice front-end for my aging father to interact with GPT. He has solved a few issues that would be easy with google, but he won't use google on his own. He likes that he can talk to an 'AI'.
So it literally is already unblocking some computing avenues, for some people. This will grow a lot in the near future, as real professionals do serious versions of my toy.
I'd like to see more comments from the target audience, at the moment the comment section is full of abled people who speculate on the needs and experiences of the target audience.
I immediately sent this to my tetraplegic brother in-law. He cannot move his fingers so he uses a special device to precisely point on a touchscreen using his fist. It makes sense to me: other than the fingers, the tongue is probably the most precise output device on the body.
I have had periods of time where I couldn't use my hands to use the computer due to bad rsi. I used an eye tracker for controlling the mouse and a voice recognition tool (talon) for typing. However, I couldn't use these for very long because I got a sore throat from speaking in a way the computer understands, and focusing places with my eyes all the time gave me headaches.
While a mouthpad seems slighly superior to an eye tracker, the real benefit lies in having more alternatives in my opinion. So you can use your tongue when your eyes are tired and your eyes when your tongue is tired.
+1 on the complementary potential of the MouthPad^.
As you mention, the MouthPad^ would serve as an awesome tool in tandem with gaze-based cursor control. I’d like to point out that there are also other interface combinations that could work well, too. For example, we have tested using our device in tandem with voice-assistant systems and found preliminary success as our device does not prevent speech. In the end, everyone has different needs and preferences, and each interface has their quirks, so we aim to create a better option by leveraging the power of the tongue (comprised of 8 different muscles, and intrinsically dexterous).
I'm a c-5 incomplete quadriplegic with hemispheric stroke-like paralysis that bothers my body all over. Some of it is just weakness, some of it is incomplete sensation, some of it is total paralysis. I have one weak-yet-functioning hand, and one hand that works only via the tone present in the hand as a rough pointing-poking tool. I use a manual wheelchair. At my injury level I am exceedingly fortunate to have decent trunk and upper body mobility.
I have found setups that work for me, and i'm comfortable now -- but I wish I had something like this years ago when I had to make accommodations.
My main worry in seeing this -- like all accessibility equipment -- is that it will be outrageously priced or locked up behind some kind of insurance coding so as to capture that segment of the market; most medical groups aim for VFW-equivalent equipment coverage on insurance, and I have no-where near that.
That said : I would love this as a device for TIG-torch actuation. That's one nut that I haven't yet been able to crack, and I was seriously considering the idea of tinkering up some kind of mouth-actuated wireless solution for welding. I can imagine this as a PC bridge to an interface that could help realize a fine control for the torch.
> I would love this as a device for TIG-torch actuation
Thanks for the chuckle.
My mom works with people with disabilities and all of her clients that I've met are like this--determined and capable.
I imagine there's a lot of people who need help in getting these tech setups going with a bunch of trial and error and then after that they can independently operate. Are there any volunteer groups that help with that?
>I imagine there's a lot of people who need help in getting these tech setups going with a bunch of trial and error and then after that they can independently operate. Are there any volunteer groups that help with that?
I'm pretty out-of-touch with the accessibility/advocacy world, but should such a group exist I think that it'd be a blast to work for/with them. Giving someone back an ability is something that may be minor for some, but life-changing for others. I would imagine that it would be immensely gratifying work.
I know of groups like AbleGamers that have helped with gaming accessibility as well as providing equipment for people on a limited-basis. I think they were also one of the forces behind the XBox Adaptive Controller, which was a boon for some.
>My main worry in seeing this -- like all accessibility equipment -- is that it will be outrageously priced or locked up behind some kind of insurance coding so as to capture that segment of the market; most medical groups aim for VFW-equivalent equipment coverage on insurance, and I have no-where near that.
Our trains of though crossed paths here. I was thinking about how useful it might be to an able bodied person, if it would be fun or even beneficial to use. Like having a third hand in gaming would be nice sometimes. Then it immediately called to mind how outrageously expensive aid devices like this tend to be. For example an actually good hearing aid is like $2,000 for some reason. And a really nice robotic wheelchair can be like $40,000. So with disappointment I assume this thing, even if it makes it to scaled up commercial production is going to be out of my price range as someone who doesn't actually need it but wouldn't mind helping to fund the project.
Is the apparent price gouging because insurance might pay for it? Or because some people don't have a choice but to buy it?
I have joint+nerve issues and struggle using a computer -- but a mouse in particular -- for extended periods. Something like this would be game-changing for me; it would be awesome to have another input with "resolution"/accuracy anywhere close to what the mouse can do. Eye tracking isn't quite it for me, trackballs are very uncomfortable (thumbs are one of the biggest joint issues for me), ergonomic mice that meet my needs are hard to find with left-handed design, keyboard as mouse input works terribly, ... so a device like this would be wonderful.
It's interesting but I wouldn't use it, partly because I got some hand functions still as a quadriplegic, I'm typing this on a keyboard with only my thumbs and I can type quite fast. I also have my mouse clicks switched as I cannot move my index finger, so I click with my middle finger. Other quadriplegics have it worse so things like this would be dope for them. At the end of the day, we will adapt to anything regardless.
Which is somewhat unfortunate. Having the device go mainstream is one way to reduce the cost per unit, or even subsidize the cost for people who really need it. Many accessibility technologies end up being an improvement for everyone.
I would certainly be interested in the device purely from a keeping-both-hands-on-the-keyboard perspective.
Although our promotional video features people with quadriplegia, the MouthPad^ is designed for everyone and anyone interested in using their tongue and mouth to control their personal devices. We believe strongly in universal design, and how designing for extremes can create an improved experience for all users, and this has led us to work closely with the disability community. A classic example of the benefits of universal design can be seen in curb cuts, which were initially built into our sidewalks to help accommodate the needs of people living with mobility issues, but ended up providing value to many other groups, such as mothers with strollers, workers with carts, travelers with luggage, runners, skateboarders, etc.
I realise this is intended for accessibility (for which is looks amazing), but I can also see this gaining widespread abled [0] use for things like gaming, and making mobile computing easier.
I can see myself pulling out the iPad on the train, and popping in the mouthpad to make navigation through Mails, Procreate, etc. easier.
[0] I tried "non-disabled" here, but that just seemed too clumsy. Am I missing an obvious term?
"able-bodied" is a common one, but doesn't quite fit here. I think "widespread" on its own is good enough to get across that you mean expanding to outside the market of disabled people
Yes, while we all tend to jump to permanent disabilities as the justification for accessibility, one can have situationally or otherwise temporarily reduced ability. Examples that jump to mind:
* broken fingers
* wearing gloves at a cold station
* a parent holding a bottle while feeding a newborn baby
That depends if the 3D intraoral scan can be done by running an app and then putting your (open) mouth over the rear camera with the LED on, and moving it around until it beeps. Or if you have to go into a doctor's office to get it.
The custom design is simply a matter of business process on top of a standardized scan.
Could be, but I would imagine if they were able to certify this as a medical device that it could be covered in part or in whole by insurance in some (non-US I would imagine) countries.
Dental offices all over the country have scanners that use regular cameras + photogrammetry to create 3d models of your teeth. They've become pretty widespread with the rise of Invisalign, Smile Direct, and others. These scans can be pretty inexpensive, like a few hundred dollars or less.
If this one is pretty inexpensive and can be bought without a doctor prescription, this would be a great improvement.
Out of curiosity, I had looked into getting a tongue-controlled device some years ago (to complement an eye tracking interface I had made for playing a specific game without keyboard/controller as a challenge), but had run into these two blockers.
Unfortunately lots of ergonomic or accessible tools are very expensive. I guess one part is that they're niche. But I think it's mainly that it's often companies buying them (often because they need to make the workplace accessible by law), and then the makers can charge what they want.
I think this is quite cool! Even for abled people, I can use a bluetooth trackpad and doing other things with both hands. And I think it's a good way to interact with TV.
However, for me it seems that I can't eat things when wearing it, or it can be hard to clean.
What a great idea, I suffered from really debilitating rsi for a few years which affected my ability to use a keyboard and mouse. Knowing theres options out there really kept me hopeful.
182 comments
[ 1.9 ms ] story [ 448 ms ] threadDisadvantages are that you need to fully speak (whispering and sub-vocalizing doesn't work), and your voice sounds kind of funny.
Keywords is "bone conduction microphone" or "ear bone microphone".
https://openbci.com/community/synthetic-telepathy-with-subvo...
mouthpad + subvocalisation....keyboards and mice are DONEZO.
very cool shit.
You can buy a Neurosky off eBay to play with for < $200 if you're really interested in trying to do said training.
[1] https://www.goodreads.com/en/book/show/60784561
And tongue controlled controllers aren't a novelty, they are used sometimes, but again - they are mostly one-time devices.
https://iopscience.iop.org/article/10.1088/0960-1317/8/2/028...
Unfortunately, Steven Hawking's muscular dystrophy affected his tongue as well so he was unable to use it, but it was in that context that I heard about it.
There's a clue that starts at 0:00 in the video.
I played around with Tobii eye-tracking hardware half a decade ago and was pretty amazed by its precision. I understand now it also has Glasses (vs. the model I tried, a tracker you put below your monitor usually). Marrying eye-tracking glasses with this mouthpad might be a winning combination: eye-tracking being the mouse to navigate in xy-space, and mouthpad for clicks.
Kudos to this group for putting out a finished product though, can't wait to see it used and iterated more out in the field.
https://www.youtube.com/watch?v=vJ0UngGRsck
—edit—
Mavis Beacon Teaches… umm, got nothing.
Ok, I’m done.
So it literally is already unblocking some computing avenues, for some people. This will grow a lot in the near future, as real professionals do serious versions of my toy.
https://www.bloomberg.com/news/features/2023-04-24/a-high-sc...
While a mouthpad seems slighly superior to an eye tracker, the real benefit lies in having more alternatives in my opinion. So you can use your tongue when your eyes are tired and your eyes when your tongue is tired.
As you mention, the MouthPad^ would serve as an awesome tool in tandem with gaze-based cursor control. I’d like to point out that there are also other interface combinations that could work well, too. For example, we have tested using our device in tandem with voice-assistant systems and found preliminary success as our device does not prevent speech. In the end, everyone has different needs and preferences, and each interface has their quirks, so we aim to create a better option by leveraging the power of the tongue (comprised of 8 different muscles, and intrinsically dexterous).
I have found setups that work for me, and i'm comfortable now -- but I wish I had something like this years ago when I had to make accommodations.
My main worry in seeing this -- like all accessibility equipment -- is that it will be outrageously priced or locked up behind some kind of insurance coding so as to capture that segment of the market; most medical groups aim for VFW-equivalent equipment coverage on insurance, and I have no-where near that.
That said : I would love this as a device for TIG-torch actuation. That's one nut that I haven't yet been able to crack, and I was seriously considering the idea of tinkering up some kind of mouth-actuated wireless solution for welding. I can imagine this as a PC bridge to an interface that could help realize a fine control for the torch.
My mom works with people with disabilities and all of her clients that I've met are like this--determined and capable.
I imagine there's a lot of people who need help in getting these tech setups going with a bunch of trial and error and then after that they can independently operate. Are there any volunteer groups that help with that?
I'm pretty out-of-touch with the accessibility/advocacy world, but should such a group exist I think that it'd be a blast to work for/with them. Giving someone back an ability is something that may be minor for some, but life-changing for others. I would imagine that it would be immensely gratifying work.
I know of groups like AbleGamers that have helped with gaming accessibility as well as providing equipment for people on a limited-basis. I think they were also one of the forces behind the XBox Adaptive Controller, which was a boon for some.
Our trains of though crossed paths here. I was thinking about how useful it might be to an able bodied person, if it would be fun or even beneficial to use. Like having a third hand in gaming would be nice sometimes. Then it immediately called to mind how outrageously expensive aid devices like this tend to be. For example an actually good hearing aid is like $2,000 for some reason. And a really nice robotic wheelchair can be like $40,000. So with disappointment I assume this thing, even if it makes it to scaled up commercial production is going to be out of my price range as someone who doesn't actually need it but wouldn't mind helping to fund the project.
Is the apparent price gouging because insurance might pay for it? Or because some people don't have a choice but to buy it?
A device like this makes a lot of sense for wearable computing. I wouldn't assume the target market are people with disabilities.
I would certainly be interested in the device purely from a keeping-both-hands-on-the-keyboard perspective.
Although our promotional video features people with quadriplegia, the MouthPad^ is designed for everyone and anyone interested in using their tongue and mouth to control their personal devices. We believe strongly in universal design, and how designing for extremes can create an improved experience for all users, and this has led us to work closely with the disability community. A classic example of the benefits of universal design can be seen in curb cuts, which were initially built into our sidewalks to help accommodate the needs of people living with mobility issues, but ended up providing value to many other groups, such as mothers with strollers, workers with carts, travelers with luggage, runners, skateboarders, etc.
I can see myself pulling out the iPad on the train, and popping in the mouthpad to make navigation through Mails, Procreate, etc. easier.
[0] I tried "non-disabled" here, but that just seemed too clumsy. Am I missing an obvious term?
* broken fingers
* wearing gloves at a cold station
* a parent holding a bottle while feeding a newborn baby
* a chef working with busy/messy hands
* a computer with a dodgy mouse and no trackpad
> Custom-designed MouthPad^ device
This seems expensive. I hope the future brings a more general version.
The custom design is simply a matter of business process on top of a standardized scan.
Out of curiosity, I had looked into getting a tongue-controlled device some years ago (to complement an eye tracking interface I had made for playing a specific game without keyboard/controller as a challenge), but had run into these two blockers.
However, for me it seems that I can't eat things when wearing it, or it can be hard to clean.
Jokes aside, it's really great to see innovations like this in human-machine interfaces.
Throat cancer