It is just a proof of concept, but they released the source code so others can build on it. Hopefully someone will create something cool, but not charge a ridiculous markup.
This could actually be really helpful to me, as I have trouble hearing someone speaking in a busy room because my mind is trying to pick up everything (I think this is because of my ADHD). Having a way to significantly quiet out other noises aside for the voice of the person I'm speaking with would be amazing.
I'm having the same problem, my hearing is fine but talking to people in busy clubs or cafe's is next to impossible for me. This feature would be a blessing for me!
Ditto. I would pay big money for this if it came in an inconspicuous form factor like airpods. Hopefully it's just a matter of time before Airpods themselves can do this.
i don't know much about adhd/autism, but i'm pretty sure i'm somewhat autistic and have this problem really really bad. i score fine on hearing tests where i just have to listen for quiet beeps but have a lot of trouble processing what people are saying especially in a crowded setting. my dad also has this issue
A potential feature I didn't know I needed. Have headphones with ANC on around home all the time, would be really useful if it auto passthrough my partners voice.
We watch ads, not by force, but voluntarily for free services or currencies in mobile games. Politicians can do basically whatever they want. We have cameras and even glasses that record everything we do. V-Tubers are more popular by the minute. People get blackmailed for their online activities, wrong as they might be. Kids walk around with parent-forced app's to track their location and online life. Robot dogs are being sold to the public and being used by the military. People care more about filming something or the documentary than the event itself.
Just to nitpick one point, I personally think we've passed the peak of popularity for vtubers and it will settle down to a slightly lower level. Having been one myself, I've seen the other side and I think a combination of lockdown and the launch of Holo-EN made a lot of people try it, before realising it didn't work for them
I would be happy with ANC with a doorbell passthrough. Missed a few package deliveries this way. But maybe that could also be achieved with a desktop notification.
lol. My partner and I share a home office and I've worn ear plugs + my range headphones (or my bose 700s) and I can still hear her clacking away and talking on meetings. I'm sure I'm some kind of spaz but god I wish I had something that could completely mute all sounds except my rain sounds. lol
You want iems. The same kind of earphones musicians wear on sets. The cons is that they can be uncomfortable for long periods. I’m right next to a night club and I’m glad I have a pair lying around.
Highly doubtful, it's just a microphones and the speakers that emit inverted sound waves.
This is one of the safest technologies I can imagine.
It's more likely that the radio waves from wireless communication (phones, Bluetooth headphones etc) will have negative impact, but even that's unlikely at this point, considering how widespread their use is and no statistically significant link exists.
Sure but it's combined noise on top of existing sound waves that gets me. They don't cancel out midair, they cancel out in your brain right?
And yeah, I also don't trust all these radio waves we have going everywhere and wonder if big events with hundreds of thousands of phones are doing at least some damage.
Things seem unlikely til we realise. Like plague spreading through bad smells. We were close but dismissive.
Anyway I hope I'm not right, but it still plays in my mind.
> don't cancel out midair, they cancel out in your brain right?
No, they cancel out in front of your ears. If you imagine a bowl of water which has waves on it with a small buoy, then the noise cancelling is another wave emitter placed in front of the buoy that cancells out the Waves that are about to hit the buoy.
It's a very physical technology. The amazing/magical part of it is that the microphones pick up the Waves quickly enough for the headphones to emit the inverted wave.
Okay I just assumed it was both waves coming to shore and they fell in such a way that lessened both of them. But if they're colliding outside the ear then I feel a bit better now.
I thought so, but I live in pretty quiet neighbourhood with ~8 hours of ANC which enables quietter playback volume, versus growing up in a very loud metropolis where bustle was non stop and blasting headphones in before ANC days.
TBH at this point, I wouldn't even object to losing my hearing to have forever ANC (hearing loss) and turning up the hearing aid.
E: no offense to those with hearing loss in this thread
ANC does not block voices. It's probably passive sound protection in your headphones that causes your partner's voice to sound weak and not go through to your ear. Or plain and simple, just the music you listen to that masks the voice.
The only case ANC would block your partners voice would be if it is about as high/low level as the background noise/sound so that it is all mixed into a white or colored noise which ANC can suppress.
I used to think of building something related to let a mic pick up a single person to handle questions from the audience, during presentations. Will save the hassle of passing around mics.
This looks like it could do just that with the headphones feeding directly into the mixer and behaving like a focused mic.
As somebody who is hearing impaired, a feature like this would be a Godsend for me! This feature should be integrated into hearing-aids ASAP! Shut up - no, actually - keep talking and take my money!
I have sensoneural hearing loss as well and fyi Bose Hearphones do have something a little like this with directional noise cancellation that helps a lot. They are discontinued but you can find them refurbished.
My phonak HAs have some directional noise cancellation (or biasing at least; I don't have rigorous definitions for these terms)... It helps but isn't great.
Has a problem that I think the AI headphones wouldn't solve either: in a (non-quiet) group setting you still need to anticipate who's going to speak when and look at them for best results.
The direction bit is just biasing to preferring forward stuff (via two mics on each ear's HA).
Sadly, no backwards bias option for overhearing people behind you ;)
It's a bit physically trickier than that due to curved tubing and ear molds... but I could totally "try" it with friends, e.g. rotating the BTE hearing aid 180 so it's forward, and they'd have fun too.
And we almost all will be where you are now, if we live long enough.
If you can pick out audio from individuals, you could also send it through speech recognition and subtitle real world conversations for when hearing is worse or not there at all.
But it really needs mobile devices capable of doing the processing locally, as I think round trips to the cloud would make it less useful or potentially useless.
How much latency is acceptable? If you're off in the woods somewhere far from the cloud, sure, but it's less than 10ms to ping Google.com for me, and if the speech-to-text engine runs faster than realtime, I don't see why processing remotely is a problem. 10 ms is nothing.
Still, the transcription part is already here today. The Google Translate app has a transcribe app that does this (runs locally; does not do magic AI "pick voice out from crowd"). My father-in-law has been using it for years. When I'm in a loud environment, the app I use on iOS is called Big, which just displays large text on the screen.
10ms on a wifi connection is exceptional; on a cellular connection it's unheard of. I normally get 70-80ms on 5G, which is well past the threshold for realtime—and that's with a solid connection.
> If you're off in the woods somewhere far from the cloud, sure, but it's less than 10ms to ping Google.com for me
I'm in one of the biggest cities of my current country, and the RTT to google from me is 87-91ms. Well over 4 million people live within 100km of me, so I suspect they see similar latencies. On my cell, I see 191-207ms.
Also in a big city in the Netherlands, but I just blame ziggo. We're getting fiber in my neighborhood ... "soon" ... so we'll see how it is once that happens.
Looks like a good 60ms is nothing but buffer-bloat in the router, as when pinging directly from the router, the RTT is much less.
"But it really needs mobile devices capable of doing the processing locally"
I would think this shouldn't be a problem as the correct hardware gets adopted in phones. As it stands now, you could probably run it on a Coral USB accelerator and battery run Pi (just an example of hardware, obviously we don't have the code).
You should look into Luxottica's efforts in this category.
Wearable glasses are quite promising for the use case you mentioned, as they avoid the bulk and impoliteness of wearing headphones while talking to someone.
This but more advanced would quite nicely help with my tinnitus. I hear fine when one person is speaking (even softly and at a distance), but multiple or with music, I hear nothing.
In the same boat. I have some tinnitus (low frequency, radio static like noise) and struggle with conversation in loud places with lots of background noise and conversation. If I sit in a loud bar it is hopeless hearing what anyone but the closest two persons are saying. Conversation in normal settings are mostly no issue.
So something that would enhance the speech of whomever I'm looking at would be super cool. Apple AirPods already have some sound shaping abilities to react to environment and mode. They also support specific voice enhancement if you put your phone down in front of the person speaking. If they ever support directional voice enhancement, like in this research, directly in the AirPods it would help me so much with social interactions in loud places.
This is what this post (or rather just the title, tbh) immediately reminded me of.
I remember learning about it in the early 2000s. It was considered a very challenging problem, with very important applications, most notably speech recognition in natural settings.
I wonder what is the current status on this. Is this considered solved nowadays?
How does it solve the problem of humans being able to detect that someone's looking at us? We tend to stop talking when we sense someone's staring at us.
You don’t need to stare at them. You only have to look at them for a moment to tell the system who you want to listen to. Then you can look away and it keeps listening to them.
Actually from the description it sounds (no pun intended!) like you wouldn’t even have to look at them. You just would need to be facing their direction. You could be looking at something else in the direction, like the ground in front of you.
When you tell it to start listening to the person you are looking at what it really does is start listening to the person whose sound is coming from that direction, which it figures out from arrival times at both ears.
This could easily hold a library of voices that you interact with (e.g at a bigger table of friends and family) and let you toggle in and out voices that are relevant. Apple please include this feature for your Airpods, thanks! :)
Imagine it helping people with Autism and ADHD! ADHD people have hard time listening to 1 person because part of the brain tries to listen to all other conversations going around.
I have high octane AD[H]D and as my hearing "goes" so does my ability to listen to people with noise around. For at least 12 years i've had to warn my wife and children if they're not facing me when they talk i can't hear them - but i was recently let in on the fact that other people consider them mumblers, so. I have a heard time hearing most people on the phone, especially call centers.
Prior to what i would describe as idiotic practices in my 20s and 30s, i could focus rapt attention on as few or as many people as required in nearly any situation, parties, bars, whatever. Music venues obviously not, but looking at someone and talking loudly near their ear (vice versa) worked fine.
My hearing issues are similar to my FIL's, who is 50% deaf in one ear and 95%+ in the other. if you're sittin on the wrong side, you're getting a lot of smiles and nods, because "eh?" gets old. real. fast. However, i can hear a raccoon messing around outside, and no one else seems to hear it; also a phone notification going off in the next room, as examples. my hearing "feels" fine, except in the very specific circumstance of human speech recognition. I also have tinnitus sometimes - it comes and goes, and if i concentrate i can focus it to the point of wincing, sometimes.
interestingly i have to turn down the master volume of games when i am voice chatting. any amount of noise from the game will interfere with my ability to process people speaking, especially if the game has a lot of talking. Additionally, i cannot watch any tv or film produced after about 2000 or so without subtitles, regardless of how fancy the "5.1 surround" center channel DSP is.
sorry for rambling, i suppose i don't think ADHD has anything to do with my hearing loss or issue; I should have worn hearing protection more when i was younger and i'm not regretting it much now, but when i can't hear music i enjoy i'll be pretty sad about that.
> suppose i don't think ADHD has anything to do with my hearing loss or issue;
I wouldn't be so quick to dismiss that as a potential factor, auditory processing issues are known to something that can be connected to ADHD in some people. (Also mentioned by a few other people in the comments.)
> Additionally, i cannot watch any tv or film produced after about 2000 or so without subtitles, regardless of how fancy the "5.1 surround" center channel DSP is.
IMO that's potentially more because audio engineers for home movie releases are... ok, well, let's just say, "have different opinions on how to mix audio with dialogue than me". :) It's a very common compliant.
> I should have worn hearing protection more when i was younger
Shouldn't we all? :) Still worth trying to protect what you have now--I've used the "Etymotic" brand ear plugs mentioned elsewhere in the comments which are intended to more evenly reduce sound levels without just "muffling" everything.
> but when i can't hear music i enjoy i'll be pretty sad about that.
I use active muffs if i need to hear, and howard leight earplugs and plain howard leight muffs contemporaneously for exploding things. A skeet shooting film grip recommended this after (nearly?) having her eardrum broken by a long tube hitting a tower leg on a set next to her head. Specifically the brand, but also the doubling up if there's guaranteed risk of loud noises.
active muffs are so nice i regret waiting a decade to actually buy some. I'll check out the etymotic e*20** at some point, although i'm hesitant about how useful they'll actually be relative to active muffs.
Curious what sort of processing power or chipsets the 'onboard embedded computer' needs. Could this be an iPhone app? Or is this going to require new, specialized hardware to commoditize?
451 comments
[ 3.3 ms ] story [ 310 ms ] threadTurn anything into a mirror, or something like that.
We watch ads, not by force, but voluntarily for free services or currencies in mobile games. Politicians can do basically whatever they want. We have cameras and even glasses that record everything we do. V-Tubers are more popular by the minute. People get blackmailed for their online activities, wrong as they might be. Kids walk around with parent-forced app's to track their location and online life. Robot dogs are being sold to the public and being used by the military. People care more about filming something or the documentary than the event itself.
If I'm being dismissive it's because these things can be dismissed as revolutionary.
https://support.apple.com/guide/iphone/use-sound-recognition...
This is one of the safest technologies I can imagine.
It's more likely that the radio waves from wireless communication (phones, Bluetooth headphones etc) will have negative impact, but even that's unlikely at this point, considering how widespread their use is and no statistically significant link exists.
And yeah, I also don't trust all these radio waves we have going everywhere and wonder if big events with hundreds of thousands of phones are doing at least some damage.
Things seem unlikely til we realise. Like plague spreading through bad smells. We were close but dismissive.
Anyway I hope I'm not right, but it still plays in my mind.
No, they cancel out in front of your ears. If you imagine a bowl of water which has waves on it with a small buoy, then the noise cancelling is another wave emitter placed in front of the buoy that cancells out the Waves that are about to hit the buoy.
It's a very physical technology. The amazing/magical part of it is that the microphones pick up the Waves quickly enough for the headphones to emit the inverted wave.
I should watch a "how it works" video on this.
TBH at this point, I wouldn't even object to losing my hearing to have forever ANC (hearing loss) and turning up the hearing aid.
E: no offense to those with hearing loss in this thread
Much better to wear headphones than to need hearing aids (also: some forms of hearing loss aren’t helped by hearing aids. Mine, for example).
The only case ANC would block your partners voice would be if it is about as high/low level as the background noise/sound so that it is all mixed into a white or colored noise which ANC can suppress.
This looks like it could do just that with the headphones feeding directly into the mixer and behaving like a focused mic.
Has a problem that I think the AI headphones wouldn't solve either: in a (non-quiet) group setting you still need to anticipate who's going to speak when and look at them for best results.
The direction bit is just biasing to preferring forward stuff (via two mics on each ear's HA).
Sadly, no backwards bias option for overhearing people behind you ;)
Put them on "backwards"; left cup on right ear and vice versa: forward facing mics now face backwards ;)
If you can pick out audio from individuals, you could also send it through speech recognition and subtitle real world conversations for when hearing is worse or not there at all.
But it really needs mobile devices capable of doing the processing locally, as I think round trips to the cloud would make it less useful or potentially useless.
Still, the transcription part is already here today. The Google Translate app has a transcribe app that does this (runs locally; does not do magic AI "pick voice out from crowd"). My father-in-law has been using it for years. When I'm in a loud environment, the app I use on iOS is called Big, which just displays large text on the screen.
https://apps.apple.com/us/app/make-it-big/id479282584
I'm in one of the biggest cities of my current country, and the RTT to google from me is 87-91ms. Well over 4 million people live within 100km of me, so I suspect they see similar latencies. On my cell, I see 191-207ms.
Looks like a good 60ms is nothing but buffer-bloat in the router, as when pinging directly from the router, the RTT is much less.
And given the US healthcare system, somebody is gonna take all our money too, one way or another. :P
I would think this shouldn't be a problem as the correct hardware gets adopted in phones. As it stands now, you could probably run it on a Coral USB accelerator and battery run Pi (just an example of hardware, obviously we don't have the code).
Or for when you don't speak that language.
>> https://www.cnet.com/health/medical/what-did-you-say-these-e...
So something that would enhance the speech of whomever I'm looking at would be super cool. Apple AirPods already have some sound shaping abilities to react to environment and mode. They also support specific voice enhancement if you put your phone down in front of the person speaking. If they ever support directional voice enhancement, like in this research, directly in the AirPods it would help me so much with social interactions in loud places.
But this is very useful for people like me who don't hear well in the high frequencies.
I remember learning about it in the early 2000s. It was considered a very challenging problem, with very important applications, most notably speech recognition in natural settings.
I wonder what is the current status on this. Is this considered solved nowadays?
The only behavior close to that I can think of is when someone is looking at someone expectantly, and trying to break in.
Actually from the description it sounds (no pun intended!) like you wouldn’t even have to look at them. You just would need to be facing their direction. You could be looking at something else in the direction, like the ground in front of you.
When you tell it to start listening to the person you are looking at what it really does is start listening to the person whose sound is coming from that direction, which it figures out from arrival times at both ears.
Prior to what i would describe as idiotic practices in my 20s and 30s, i could focus rapt attention on as few or as many people as required in nearly any situation, parties, bars, whatever. Music venues obviously not, but looking at someone and talking loudly near their ear (vice versa) worked fine.
My hearing issues are similar to my FIL's, who is 50% deaf in one ear and 95%+ in the other. if you're sittin on the wrong side, you're getting a lot of smiles and nods, because "eh?" gets old. real. fast. However, i can hear a raccoon messing around outside, and no one else seems to hear it; also a phone notification going off in the next room, as examples. my hearing "feels" fine, except in the very specific circumstance of human speech recognition. I also have tinnitus sometimes - it comes and goes, and if i concentrate i can focus it to the point of wincing, sometimes.
interestingly i have to turn down the master volume of games when i am voice chatting. any amount of noise from the game will interfere with my ability to process people speaking, especially if the game has a lot of talking. Additionally, i cannot watch any tv or film produced after about 2000 or so without subtitles, regardless of how fancy the "5.1 surround" center channel DSP is.
sorry for rambling, i suppose i don't think ADHD has anything to do with my hearing loss or issue; I should have worn hearing protection more when i was younger and i'm not regretting it much now, but when i can't hear music i enjoy i'll be pretty sad about that.
I wouldn't be so quick to dismiss that as a potential factor, auditory processing issues are known to something that can be connected to ADHD in some people. (Also mentioned by a few other people in the comments.)
Couple of links that go into more details:
* https://en.wikipedia.org/wiki/Selective_auditory_attention
* https://en.wikipedia.org/wiki/Auditory_processing_disorder
* https://en.wikipedia.org/wiki/Cocktail_party_effect
> Additionally, i cannot watch any tv or film produced after about 2000 or so without subtitles, regardless of how fancy the "5.1 surround" center channel DSP is.
IMO that's potentially more because audio engineers for home movie releases are... ok, well, let's just say, "have different opinions on how to mix audio with dialogue than me". :) It's a very common compliant.
> I should have worn hearing protection more when i was younger
Shouldn't we all? :) Still worth trying to protect what you have now--I've used the "Etymotic" brand ear plugs mentioned elsewhere in the comments which are intended to more evenly reduce sound levels without just "muffling" everything.
> but when i can't hear music i enjoy i'll be pretty sad about that.
Indeed. Understandably.
I use active muffs if i need to hear, and howard leight earplugs and plain howard leight muffs contemporaneously for exploding things. A skeet shooting film grip recommended this after (nearly?) having her eardrum broken by a long tube hitting a tower leg on a set next to her head. Specifically the brand, but also the doubling up if there's guaranteed risk of loud noises.
active muffs are so nice i regret waiting a decade to actually buy some. I'll check out the etymotic e*20** at some point, although i'm hesitant about how useful they'll actually be relative to active muffs.