HN Explorer

Show HN: Talk to your computer without talking out loud (subvocal.company)

3 points by aarushyadav ↗ HN
Hi HN, we're Aarush and Arnav from The Subvocal Company (YC F26). We’re building smart glasses that let you talk to your computer without talking out loud. You can mouth words silently and discreetly and the glasses decode them to text at the speed of speech.

Here's a video of David Lieb from YC trying it out: https://youtube.com/shorts/-L1Qb5oINK4

Arnav and I were studying CS at Georgia Tech at the beginning of the year, with no experience in silent speech or sensing. We were both AI-native builders and power users of Wispr Flow. What bugged us was not being able to use these tools in the library (or an office) or in public. At Stanford’s TreeHacks in February, we decided to build the most technically ridiculous thing we could think of. We’d read about AlterEgo, a silent speech system out of MIT’s Media Lab, and wanted to recreate it in 36 hours. We built a crude prototype, sticking EMG electrodes onto my face:

https://drive.google.com/file/d/1mwhv5HZMz3UTevdrgwS5O7g2jWV...

We got it to kind of work at a letter level in a very janky way. It wasn’t good enough to win the hackathon, but convinced us to go all in. The first time I typed a sentence without anyone noticing, I felt the magic of the technology.

We raised our pre-seed from Afore Capital and went all-in! We quickly realized EMG (reading muscle electrical signals) wouldn’t work: - Signal changes due to sweat - Electrical interference - Placement variability - Most importantly, no one wants to stick electrodes onto their face!!

So we pivoted to RF and built the glasses you see now. They emit radio waves, similar to Bluetooth, that pass through your face. As they cross the tongue, lips and soft palate, they’re modulated by silent mouthing. We train foundation models on this data to decode these perturbations to text.

Despite prior academic research, getting this to work was a huge engineering challenge. First we built an extremely low noise RF sensing stack. Then we had to place antennas so their fields overlapped maximally with the articulators. Our first attempt on glasses failed completely, so we reverted to old habits and taped antennas on our face:

https://drive.google.com/file/d/1ks0dS0eZiTV9EqSFsyB_w5BBjpc...

Since taping antennas consistently in the same spot and angle was hard, we created this weird but effective head rig to hold them in place:

https://drive.google.com/file/d/1rhzNFpdDzr7oHk0isFY70dkoV4C...

This worked, but obviously wasn’t a product yet. So we worked deep in simulations to design glasses that could hold the antennas.

That evolved into our design now, with antennas embedded within the frames. Once we established there was a signal, the next hurdle was cross-user generalization. Everyone’s facial anatomy, and hence RF signal patterns, are vastly different. We tried algorithmic workarounds but realized the only lever was to massively scale pretraining data.

So that’s what we did. We set up booths in our office and paid people to read prompts wearing our glasses, to pretrain our models on. This produced a beautiful scaling curve, with errors falling quickly as we scaled our dataset and cross-user performance becoming genuinely compelling (as you see in the video above; Dave only had a short 20-sentence calibration).

We always heard about building things from first principles but never understood it before this. We tried tackling it top down but completely failed. Only when we broke eac...

277 comments

[ 0.42 ms ] story [ 7.9 ms ] thread
[deleted]
(comment deleted)
hmokiguess
EDIT: I find talking in that manner in a public setup to be awkward and unpleasant, maybe that's just me.

That said, I think the idea of having a private walkie-talkie that connects on its own frequency for large crowded spaces to be very cool

s3graham
Neat! I don't like talking to my computer (audibly or not), but that's a cool demo.

I wonder if we could eventually end up with a "throat muscle" version where people learn to talk a new way without having to move the visible mouth muscles too.

mcbishop
I've thought about learning sign language in order to communicate with the computer in a not-super-annoying way. Your solution... much more realistic. Cheers.
paisawalla
How have the glasses been received? I don't understand why they're required, is it only to have a consistent mount point for the radar sensor, relative to the face? Have you thought about a throat mic?
kennywinker
It looked like there was a throat mic in the first shot... it seems counterintuitive that you'd get better performance out of lidar on face muscles than on a throat mic.
aarushyadav
No throat mic, the first image uses EMG (muscle electrical signal) sensors. A throat mic only works if you're still actually whispering (not fully silent) and is also a weird form factor for a consumer device, which is why we didn't go with it!
paisawalla
I'll defer to your expertise on what people find weird, but it seems like you should be able to have a throat-based device that could hide under a shirt collar. I have the Meta Ray Bans and even though I love the functionality, I don't care for the bulky frames. And those are designed by Ray Ban, meaning they're presumably at the upper end of what can be done stylishly.
kennywinker
> Meta Ray Bans and even though I love the functionality

Which functionality do you love? Is it the nonconsensual recording? Or maybe the giving away your own privacy and the privacy of everyone around you to one of history's least-trustworthy companies?

fowlie
Talk about over engineering a problem! If typing speed is an issue guess what, humans can train themselves to type really fast on a keyboard.
waltbosz
I think the problem to solve here is repetitive stress injury caused by typing
rootusrootus
They are claiming 200+ wpm on a prototype. Half that speed would put you in the top tier of typists, and the number of people that get anywhere close to 200 is vanishingly small.
stubish
I can't type while simultaneously using another control like a mouse, joystick, tablet. I guess I could overengineer the problem by growing more arms. And even that doesn't solve the problem of mouse pointers having to constantly jump between UI elements and a canvas because they can only be in one place at a time.

I look forward to your typing speed tutorials for arthritis sufferers.

thisisauserid
I'm more looking for the Jane from Speaker for the Dead version. In-ear device that can hear and feel the subvocalization.
40four
I had to look it up, I forgot he communicated with her through sub-vocalizations. I remember the jewel earpiece, but in my head he just communicated through thoughts or something.

Good reference! That series of books Is one of my all time favorites.

frenchie4111
First, this my favorite book of all time so I love seeing it shouted out

Second, I agree with exactly this. Something shaped like an AirPod

vessenes
I mean we're getting pretty close here -- you could have these + your AirPods in transparency mode up and you're not far off. Also Claude won't currently ice you out for going dark, unlike Jane.
PebblesRox
Very cool! I was curious to hear the details of how it worked but couldn’t find them on your landing page. I’m glad to see them here and you might want to include them more prominently on the page itself. I didn’t even realize it was a pair of glasses til I read the above.
PebblesRox
E.g. your FAQ says it doesn’t use cameras but does not say what it does use. And when I got to the footer and clicked the “how it works” link, it seemingly took me back to the top of the page I had just read.
OhMeadhbh
Spent 30 seconds scrolling down to try to figure out what this is. Eventually clicked away from the site because it was taking too long to get to the point. I'm sure it's great and I understand it's something to do with glasses and subvocalizations, but consider a one sentence contextualization at the top of the page.
LatencyKills
I was just trying to help a vibe coder and immediately noticed that their README.md was over 2,000 lines long. I asked, "Why is your readme so long?" and their response was, "This is just a really complicated application."

The app essentially manages a bunch of TODO items written in markdown.

aarushyadav
Yup that makes sense. Thanks for the feedback, we'll make some changes shortly!
jasonlotito
> Talk without talking. > Input text at the speed of speech, silently and invisibly.

I understood from just this. But the demo video impressed even further.

> Spent 30 seconds scrolling down to try to figure out what this is.

Did you try reading the site? Or maybe doing something other than just scrolling?

Does it need some form of subway surfer to get you to pay attention?

> Eventually clicked away from the site because it was taking too long to get to the point.

So yeah... subway surfer. This generation is cooked.

jstummbillig
Category: A product, that's obviously useful, as long as it will work.
tomhow
I'm curious about whether this could be done with the hardware mounted on top of the laptop screen or on a stand on the desk, rather than the glasses form factor. Have you explored this?
adamsmark
I use the Vekta. I tried whispering and it doesn't feel right.

Throat mics don't work at all - see reviews online.

This is a different approach, so I am cautiously optimistic.

TechSquidTV
I would be fairly interested if this works without opening my mouth. Like, if you can nail _micro_ tongue movements, it would feel pretty close to thought.
hk__2
But how would you distinguish all the sounds that don’t require any tongue movement, like (most of?) the vowels?
jefftk
Vowels definitely require tongue movement! Leave your mouth closed and whisper "a-e-i-o-u" and you can feel your tongue moving.

Vowels change the shape of your mouth's cavity to change which frequencies to emphasize ("formants") and we hear the differences in vowels by analyzing those formant patterns.

popap
That is fascinating. How do the radio waves get modulated? Do you think it is possible to use this technique for spying technology, with remote transmitters and receivers intersecting a person's facial anatomy to pick up the details of even a whispered or muttered conversation?
ciefa
God do I hate websites that make you scroll to go through an animation, why do people still do this
cebert
It is frustrating and makes me close a lot of sites
uhutiik
Because they don’t even understand their own product well enough to communicate the benefits
hippycruncher22
What's wrong with human connection and speaking. :( I feel like the isolation of this modern world is lame.
temp123477
This is more a keyboard killer than a dinner party killer.
ShinTakuya
I can see this being useful on public transport where you don't want to bother people while taking notes verbally. It's not all about avoiding connection.
augment_me
Maybe you don't want strangers around you hearing you say "Hey Claude wish my grandma happy birthday" as it's not yet morally okay to delegate such social tasks to the models
233mhz
Aha what a loser thing to do, I personally have a fleet of 347 agents running 24/7 talking to my parents, siblings, grandparents, friends, asking them how they're doing and wishing them a happy birthday when the time comes. I can finally focus on the important things!
barbazoo
There is no human connection typing words into a box, that's seems like what it's mostly replacing. Talking to your email client, talking to your AI agent. Lots of us are much faster speaking than typing and especially with AI, it doesn't matter a whole lot in my experience how polished the output is, the benefit from ending up with a lot more text because one can just talk freely is very beneficial.
nice_byte
if you talk to your computer in my vicinity, i will politely ask you to shush.
johnhamlin
With so many people singing the praises of voice to text, I keep thinking about how most sci-fi futures feature subvocalization as a prerec for that kind of adoption. Always love to see sci-fi come to life!
bluegatty
Why do you destroy communicating your very cool product with an annoying and counter intuitive scroll experience?

And after some time I could not figure out what this is?

Put this part: " building smart glasses that let you talk to your computer without talking out loud. You can mouth words silently and discreetly and the glasses decode them to text at the speed of speech."

right at the top and show someone doing that instead of hiding behind cryptic marketing.

Also, looks great.

Get your words out of the way 'say what it is'.

avree
Do most people talk faster than they can type? It's the opposite for me - I also read aloud slower than I can read in my head.
wongarsu
While sitting at the desk, I type about as fast as I talk. While unloading the dishwasher I'm a much faster talker than typer
ronyeh
On my phone, I do. I still like to tap with my thumbs, but my iPhone keyboard sucks so when I need to type a lot, I’d rather dictate.
SkyPuncher
I talk faster, but beyond the speed, I find it allows me to explain things to the LLM much more naturally.
mirmor23
The tech sounds cool, but my guess is this product might work for anyone whose text to command ratio is very high. If the formatting and commanding are the main tasks, then this product would be more of a distraction for a fast typist.

Regarding the website, you should lead with product, its details, rather than founders shown in 360 degrees from various vantage points. It is distracting and frankly irritating.

newsy-combi
Unless they're more interested in selling themselves (getting bought by one of the big smart glasses producers), then the webdesign would make sense, but I would never insinuate such a thing.
sourpanda
I think this is incredibly cool and straight out of a sci-fi novel! I think this is phenomenal for accessibility and for hands-free computing in general. I also agree with other comments that the website should have more detail on how it works, such as what you included in the post body. Amazing work guys!
themanmaran
Yea in one of the further Ender's Game novels they talk about this tech a lot for characters silently mumbling to the AI.
joshgachnang
Ahhhh exactly! I've wanted this since I was a kid and read those books.
spiderice
CMD+F Ender

That's exactly where my mind went. Something I've wanted ever since I read that book a couple decades ago.

viraptor
If I remember the details correctly, the AI-like entity optimised the communication to subvocal, but his wife felt disconnected, noticed him turned away in bed whispering to the AI.

There's some lesson there...

Infiniti20
Which of the books is this? I only ever read the first
viraptor
It's from Xenocide. It was between Ender and Novinha.
garrettgarcia
I've been wanting to have a subvocal interface to a voice assistant ever since reading about Ender talking to the AI Jane as a kid. After Alexa came out, I encouraged my employer to develop the technology, but they didn't. I'm so happy to see people are working on this finally! Scott Card was a prophet.
ElijahLynn
I second that. I've been talking and talking to my phone and AI even more and more since it keeps getting better and is generally pretty accurate these days. And there are times when I end up typing because of environmental reasons.

This is just really groundbreaking and I know the research has been out there for a little while but to actually implement is a whole nother ball game.

I think the use cases for this will expand as the product ships and gets out there!

_jholland
Agree, incredibly cool, and I could definitely see myself using one, but it does give off slightly dystopian vibes, making me think of the speakwrite in 1984:

"In the long, windowless hall, with its double row of cubicles and its endless rustle of papers and hum of voices murmuring into speakwrites, there were quite a dozen people whom Winston did not even know by name"

wiml
Subvocal interfaces have been a staple of SF since probably the 1960s, but what this really reminds me of is the "ensemble coding" used by kids in the book Rainbows End.
gh5000
Cue the resurgence of lip reading-based espionage. Imagining having to scan the room for people watching subvocalizers. Will subvocalizers start holding up a clipboard over their face a la NFL coaches?
0x4e
Thought of the same! Cue George Orwell facepalming in his grave.