The paper explains why basic pitch shifting in hearing aids is not localisable,
> A less drastic version of such processing is utilised by certain hearing aids, where high frequencies that are audible to normal-hearing subjects, but inaudible to those who are hearing-impaired, are shifted down to the audible range of the wearer. However, such an approach is problematic in terms of the preservation of directional cues, since the acoustic filtering effects due to the diffraction caused by the head and pinna vary substantially with frequency. Therefore, the directional cues delivered by the pitch-shifted signals do not carry the appropriate information related to the source direction that a human subject would have evolved and grown accustomed to.
I wonder if this would apply to mixing in a frequency that would result in an audible heterodyne mix (think like active noice cancelling but using a waveform that is designed to downconvert vs cancel).
I do believe there is a method to use this to give blind folks an ambient doppler tone from objects in their vicininity. I just don't have the chops or time to try it out.
I know someone missing a lot of the high end, and he has really enjoyed the pandemic because all of his social interactions are online, where everyone is wearing headphones, and everyone is suffering from the same handicap:
No conferencing software that I'm aware of has solved the cocktail party effect. Since nobody can triangulate sounds properly, nobody can filter out side conversations or external noises. Everyone has to take turns to be understood, and if they don't, the guy with hearing aids doesn't get singled out by having to bring it up.
It can be done with a normal PC but I loved meeting people in a VR environment where you could walk around and form groups and you only heard audio from nearby people. They also had a movie theater with a movie everyone could watch at the same time.
But any interface where you can move a character in a world would be enough to make this work. It’s lovely because you can drop in and listen to a convo and walk away to another if you’d like.
Mumble does have positional audio but it has to have a game/input source that supports reporting the position of the player - always wanted to test it one day.
Pretty much all the VR ones do this, as you have a spacial representation to work from, from what I've seen. And there have been some recent-ish things like https://gather.town/ that should be capable as well, though I'm not sure if they do. But yeah, Zoom doesn't.
Cool thing about VR is when you move your head the position of the speaker stays exactly where it is in relative space thanks to motion tracking which lets you essentially turn towards people you wanna listen/talk to. At some point we need some sort of positional awareness in our headphones to add some resemblance of that.
For a hackathon last year, I patched Jitsi to play each person's audio from a different angle using the web audio spatialization APIs[1], and it really does help the cocktail party effect tremendously- people laughing or talking over each other (extremely common in physical conversations) don't make it impossible to understand the other person like when everything is mixed to a mono channel as it is with most video conferencing software.
That's an awesome idea & project, but mightn't it feel jarringly like being surrounded by people talking at you?
I think I'd at least want them all in the, say, central 160deg in front of me. At which point I wonder how good the 'spatialisation' is, how many people could be meaningfully supported.
Because you can't just have alternate speakers spaced out for maximal distinction - the same speaker suddenly coming from somewhere else would be even more jarring.
Apple just announced this as a feature for FaceTime at WWDC (though not out until the fall, which seems poorly timed). I’ve wondered for the past year why no major conferencing software has implemented that.
I heard about VocalZoom at the Embedded Online Conference, who are trying a hardware approach to solving the cocktail party effect.
Basically they use lasers and directional microphones. Seems like it could work for a conference call in a meeting room, but not sure how helpful it would be for a party.
I'm more interested in the SRAY: a directional loudspeaker, which could avoid the need for so much ambient background noise. Also useful for alarm clocks, I suppose.
I'm deaf on one side, so issues like this tend to be topics that I've already given a lot of thought to.
A counter I employ to the cocktail-party effect is to pitch my voice very low. Sometimes the person I'm talking to picks up on this (consciously or not!) and imitates my low pitch. Despite others talking over us, it's easier to focus in on a lower pitch and let the brain filter out the noise.
I wonder if you could build some sort of deliberate reflector into a microphone housing to generate comb filtering that an analytic technique (or ML) could derive spatial data from. I know in the limit this is possible, since there's dummies used for binaural recording or testing earbuds, but could an intentionally designed mechanism accomplish the same thing in a smaller package?
It'd be neat if you could then encode the spatial information and use an HRTF on the destination receiver to appropriately place the sound in the listener's soundstage.
Frankly, it just allows me to hear more high pitched squeals and "animal repellent" devices. My ability to interpret normal sounds and speech seems average.
I was the same way. At 58, my hearing is still "perfect", but I assume that assessment takes my age into account. I could always hear the whine of computer monitors, and was very happy when LCD monitors came out. I could even "hear" Ethernet traffic (10Base-T), but I never understood how or why. It must have been sub-harmonics and/or the PHY coupling transformers emitting ultrasonics.
Clickbait :( It's a device that plays back - through headphones - specific normal-range sound which is crafted in such as way as to help you locate a source of another (ultra)sound which you normally wouldn't hear. Nothing in this thing enables people to hear ultrasonic sources. They still hear sound from normal range.
I agree, and real time pitch shifting (as is done with the bat sounds in this work) is itself a non-trivial procedure where certain weigh-offs have to be made, so even pitched shifted it is not an accurate (or even easily invertible) representation that you can mentally pitch up. (this is not a problem for non-realtime pitch shifting where you would adjust the tempo accordingly).
One could argue that people don't "hear" anything but the electrical signals transmitted from their ears to their brain. If an ultrasonic signal causes someone to perceive a noise--any noise, even if waveform-translated from the original, but in a predictable fashion, isn't it "hearing"? E.g. if we set up a chime that went "bing bong" every time the cat went through the cat door, after a while wouldn't it be appropriate to ask your roommate, "Did you hear the cat come in?"
If that's the case, I could write an article "Company X developed technology that enables people to see individual atoms and hear the explosions on the other side of the Earth."
If it was realtime, then sure, as it's kind of implied in that phrasing. What are our senses other than machines that detect phenomenon and turn it into electrical signals for realtime processing for the brain? Do people with cochlear implants not hear?
> hear the explosions on the other side of the Earth
It is quite hard to enable them to 'hear' where on earth it is, not sure how you could hear it properly. (Language is not hearing although hearing is involved)
What's an example where it's not clickbait with a sense that's been 'enabled'?
For this article I'd consider it not clickbait if all the 'information' from ultrasonic is transferred to the ear real time or close to.
I wonder if you could hear bats directly by having a headset / amplifier that contacts your skull rather than putting a speaker over your ear. It’s been reported that humans can hear ultrasonic frequencies via bone contact. (I’d speculate identifying the source direction doesn’t work though.) https://en.wikipedia.org/wiki/Ultrasonic_hearing
There are definitely bats that produce chirps (partially) in the human-audible range. You don't need above-average hearing to notice 15 kHz sounds, which appears to be the lower end listed in some source. Most echolocation happens way beyond human hearing range though.
Ha, would be.. fun? interesting? something? to have an app or little device that's sort of an 'equaliser for your ears'.
Presumably there's nothing technical stopping that for those with hearing aids fitted? Just insufficient use case / desire for such granular control. I'd be surprised if you can't get them with something more dumbed down like a 'quieten crowds' or 'loudness' switch.
"Humans can observe what and where something happens around them with their hearing, as long as sound frequencies lie between 20 Hz and 20 000 Hz."
I wish. This might be true if you are very young, but deteriorates rather fast.
In a related vein, I was curious whether this would be something miraculous that made humans hear the actual frequencies over 20kHz somehow, but it's of course the much more likely interpretation: High frequencies are aliased down into lower frequencies where you can then hear them normally.
Still seems to have cool applications, and if I understand correctly, it's going through great length to make the sounds still intuitively localizable, which must make for a neat experience.
>rather fast
In countries where they don't regulate noise polution, especially in bars where self-tought "audio engineers" will try hacking it by turning the amp up to 11.
It actually turned out even cooler than I expected. A lot of things emit ultrasound that I didn't realise, like jingling keys, walking on dry grass, and all kinds of things in urban environments. And of course bats sound really neat also!
44 comments
[ 0.22 ms ] story [ 101 ms ] thread> A less drastic version of such processing is utilised by certain hearing aids, where high frequencies that are audible to normal-hearing subjects, but inaudible to those who are hearing-impaired, are shifted down to the audible range of the wearer. However, such an approach is problematic in terms of the preservation of directional cues, since the acoustic filtering effects due to the diffraction caused by the head and pinna vary substantially with frequency. Therefore, the directional cues delivered by the pitch-shifted signals do not carry the appropriate information related to the source direction that a human subject would have evolved and grown accustomed to.
https://www.nature.com/articles/s41598-021-90829-9
I do believe there is a method to use this to give blind folks an ambient doppler tone from objects in their vicininity. I just don't have the chops or time to try it out.
No conferencing software that I'm aware of has solved the cocktail party effect. Since nobody can triangulate sounds properly, nobody can filter out side conversations or external noises. Everyone has to take turns to be understood, and if they don't, the guy with hearing aids doesn't get singled out by having to bring it up.
But any interface where you can move a character in a world would be enough to make this work. It’s lovely because you can drop in and listen to a convo and walk away to another if you’d like.
1. https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_A...
I think I'd at least want them all in the, say, central 160deg in front of me. At which point I wonder how good the 'spatialisation' is, how many people could be meaningfully supported.
Because you can't just have alternate speakers spaced out for maximal distinction - the same speaker suddenly coming from somewhere else would be even more jarring.
Really cool though.
Basically they use lasers and directional microphones. Seems like it could work for a conference call in a meeting room, but not sure how helpful it would be for a party.
https://patents.google.com/patent/US8286493
I'm more interested in the SRAY: a directional loudspeaker, which could avoid the need for so much ambient background noise. Also useful for alarm clocks, I suppose.
https://www.indiegogo.com/projects/sray-world-s-smallest-dir...
A counter I employ to the cocktail-party effect is to pitch my voice very low. Sometimes the person I'm talking to picks up on this (consciously or not!) and imitates my low pitch. Despite others talking over us, it's easier to focus in on a lower pitch and let the brain filter out the noise.
It'd be neat if you could then encode the spatial information and use an HRTF on the destination receiver to appropriately place the sound in the listener's soundstage.
Frankly, it just allows me to hear more high pitched squeals and "animal repellent" devices. My ability to interpret normal sounds and speech seems average.
It is quite hard to enable them to 'hear' where on earth it is, not sure how you could hear it properly. (Language is not hearing although hearing is involved)
What's an example where it's not clickbait with a sense that's been 'enabled'?
For this article I'd consider it not clickbait if all the 'information' from ultrasonic is transferred to the ear real time or close to.
There might be blind people who orient themselves by echolocation - maybe not ultrasonic but still pretty cool
https://www.discovermagazine.com/mind/the-blind-individuals-...
Presumably there's nothing technical stopping that for those with hearing aids fitted? Just insufficient use case / desire for such granular control. I'd be surprised if you can't get them with something more dumbed down like a 'quieten crowds' or 'loudness' switch.
I wish. This might be true if you are very young, but deteriorates rather fast.
In a related vein, I was curious whether this would be something miraculous that made humans hear the actual frequencies over 20kHz somehow, but it's of course the much more likely interpretation: High frequencies are aliased down into lower frequencies where you can then hear them normally.
Still seems to have cool applications, and if I understand correctly, it's going through great length to make the sounds still intuitively localizable, which must make for a neat experience.
https://www.eio.com/products/velleman-k8118-stereo-ultrasoni...
It actually turned out even cooler than I expected. A lot of things emit ultrasound that I didn't realise, like jingling keys, walking on dry grass, and all kinds of things in urban environments. And of course bats sound really neat also!