The answer is in the comments at the posted link, and is exactly what I thought -- the funding that makes the service possible is designed to help people with hearing disabilities, they don't want to pay additional for others to use the service who don't need it.
> It's common for the relay agent to use their human intelligence to navigate a phone menu or get the right party on the line, to rephrase and clarify, since typing is so slow.
I'd be tempted to use such a service simply for having someone else deal with phone menus and similar bullshit at the other end of the call.
Deaf here. In some contexts yes. In many, nope. Whisper can't talk to the other person. AI transcription systems on the phone are very awkward for both parties. Slow. Long pauses. Asking for clarification is a pain since the turn taking is so slow. The error rate is also way too high in some contexts (some heavily accented speakers for example).
Transcription services are still horrible if you have (a) some kind of accent, (b) non-standard terms, or (c) noise. They can be better than nothing if you want some meeting notes and no one is taking them, but they're definitely not something you want to rely on for an important conversation.
A human in a conversation of this type has the major advantage that, when they're not sure they've understood a word, they can ask the other party to repeat it.
There are (still) actually good transcriptionists who will deliver a meticulous transcript with all your weirdly abbreviated jargon thoroughly researched (which can be a feat of truly impressive "detective" work at times), multiple speakers discerned, expert punctuation choices that fit the tone and meaning, etc. But it costs way more than most people can/will pay. I worked at a "real" (non-sweatshop) transcription outfit ca. 2012. Judging by my pay rate of $75 per audio hour at the time, I would guess that the really good operations these days must be charging close to $200 per audio hour. A quick google found one anecdatum corroborating this guesstimate: http://www.transprof.com/prices-faq
Whisper transcription is extremely good. For most real-time transcription tasks I do think it performs as well or better than the average human at this point. In this case it sounds like the task is more active and involves actually thinking and talking to the other party, which makes it much harder. But for simply transcribing phone calls, the state of the art is very very good.
Deaf relay is done by transcriptionists. The end user relies on deaf relay in a way where automatic systems are not really an option - possibly in a life and death way if someone is calling their pharmacy or doctor or whatever. It's common for the relay agent to use their human intelligence to navigate a phone menu or get the right party on the line, to rephrase and clarify, since typing is so slow. That's actually a great benchmark for when AI will be ready. Can it replace humans for that task? At present absolutely not. (Trust me I've tried the various speech recognizers. Add a little noise or a mumbly person and it can degrade into uselessness. And the AI isn't going to interrupt someone immediately and then to repeat the last phrase.
Anyway. You need to register for relay service with the provider. Using without would be unauthorized.
Speaking as someone who's deaf and uses these services a lot: for speech to text, the AI stuff is getting rather good.
I'm not saying it's perfect for every situation, but I have a very high success rate using InnoCaption[0] for captioned phone calls, including to places like restaurants with a lot of noise going on in the background. InnoCaption does both live person and AI-based captioning; since they started offering the AI-based option I've left that on, and I've never had to switch to human operators to continue a conversation.
That said - I'm not deaf from birth (lost my hearing in elementary school), so I voice for myself and that does simplify the process. I have used the old school text-only relay services and that was always such a miserable experience for me that I would crawl over broken glass to avoid making phone calls, especially going through phone trees. That's one area that relay operators still have a major advantage on. IIRC, Google's Pixel phones are supposed to be able to navigate phone trees for you, but since I use iOS I have no personal experience there.
The phone tree stuff on Pixel is decent but nowhere near 100% reliable or robust.
If it hears and understands an automated system speaking out a phone tree, it will start to list the options and you can tap on them. Usually works but often doesn't recognize that a phone tree is happening. Other times it recognizes the phone tree, but mistranscribes the options.
As a non-deaf person, it's a handy UX improvement. But I wouldn't recommend that anyone rely on it.
These services are indeed great for those that need them. I received one or two years ago when I worked at a computer shop. Unfortunately they were always scammers, abusing the system.
I can't really understand speech these days without the captions to go with it. But I encounter discrepancies with AI generated captions very often. As in, I heard something and from context I know I'm right and the AI is wrong. With Whisper and other deep learning based speech systems in particular - they can generate very plausible misinterpretations - sounds similar and is grammatically plausible - but not what was said. Of a kind that a person with semantic understanding of what's going on would not make. So I am a little leery of them for that reason. I rely on it every day for generating captioning to video and so on. I don't find any iteration I've tried reliable or comfortable for interactive use.
> I encounter discrepancies with AI generated captions very often. As in, I heard something and from context I know I'm right and the AI is wrong.
I've been noticing this as well. It's becoming a common problem. Also, many times I've noticed that if I hadn't heard the speech being captioned and only had the captioning to go by, I would have had little chance of correctly understanding what was actually said.
[Applause] on YouTube transcripts, short two or three syllable sentence fragments, and absolute nonsense are the only ones I’d be able to reliably detect sans audio. But doubt YouTube captions are state of the art given how poor it is.
I’ve received one call in my life from such a relay service. I worked the phones at a retail shop selling specialty sports equipment. It was a normal call aside from some delays due to the typing and a brief introduction to explain that the person on the call was a middle man. It’s great that these services exist.
A lot of stores don't accept them anymore because of scumbag scammers abusing the system, leaving us dependent on a friend or family member to make our calls.
It is. But when the front line person has received a bunch of spam calls they start hanging up when they hear the relay disclaimer at the start of the call, whatever the law says.
I don't know if you're outside of US that's done differently but in US, it's not called "deaf relay", it's just relay services.
The relay service is not for deaf people, it's also for people with speech issues, deafblind, blind, and so on.
> transcriptionists.
Also, not as such in US. It's interpreter (usually for ASL/video call) or operator. Relay service relays messages in two ways; voice the message from the end user and then type from voice to the end user.
AI has to be absolutely perfect to replace humans for these relay calls because as you said, when it comes to medical situations, or legal issues; who would be at fault for miscommunication that led to some issues?
In US, you must also verify your identity to the relay service as federal regulations has step up the identity requirements recently. I had to do a video call with VRS to show my state ID that I'm in the state and registered with the state's relay services.
I do appreciate GP using the term "deaf relay", as this confirmed my understanding of the StackExchange post. The term "relay" is so broad that, without context, I'd assume "relay service" in telecom is whatever relays your call across vendor/state/national service borders, or something.
Just look at the automatic closed captions on any news or sports broadcast, or enable them on a YouTube video. They are pretty good, but there are still some obvious errors that you'll see within just a few minutes.
My first thought after reading the post was also more or less "hang on - practically usable, local, realtime speech recognition functionality in a 20-30 year old phone? Where is the error?"
The FCC foots the bill, via your phone bill(cell and land line). They don't want to foot the bill for every person on the planet, or even every person in the USA, just for those of us that are deaf/hard of hearing and need the service.
If you need the service, it's free. If you don't need the service then you can pay for it yourself if you want it. Or do what most do these days and use AI and hope for the best.
Yes the AI captioning stuff is improving, but they regularly break all the time. In my last call with AI captioning, the AI kept talking about lawyers. That was not a topic of discussion at all during the call. I was of course confused as all get out, and the hearing people had to type in what they actually meant.
34 comments
[ 3.2 ms ] story [ 85.1 ms ] thread> It's common for the relay agent to use their human intelligence to navigate a phone menu or get the right party on the line, to rephrase and clarify, since typing is so slow.
I'd be tempted to use such a service simply for having someone else deal with phone menus and similar bullshit at the other end of the call.
I would wager whisper is more accurate than said human, and definitely cheaper.
A human in a conversation of this type has the major advantage that, when they're not sure they've understood a word, they can ask the other party to repeat it.
Anyway. You need to register for relay service with the provider. Using without would be unauthorized.
I'm not saying it's perfect for every situation, but I have a very high success rate using InnoCaption[0] for captioned phone calls, including to places like restaurants with a lot of noise going on in the background. InnoCaption does both live person and AI-based captioning; since they started offering the AI-based option I've left that on, and I've never had to switch to human operators to continue a conversation.
That said - I'm not deaf from birth (lost my hearing in elementary school), so I voice for myself and that does simplify the process. I have used the old school text-only relay services and that was always such a miserable experience for me that I would crawl over broken glass to avoid making phone calls, especially going through phone trees. That's one area that relay operators still have a major advantage on. IIRC, Google's Pixel phones are supposed to be able to navigate phone trees for you, but since I use iOS I have no personal experience there.
[0] https://www.innocaption.com/
If it hears and understands an automated system speaking out a phone tree, it will start to list the options and you can tap on them. Usually works but often doesn't recognize that a phone tree is happening. Other times it recognizes the phone tree, but mistranscribes the options.
As a non-deaf person, it's a handy UX improvement. But I wouldn't recommend that anyone rely on it.
I've been noticing this as well. It's becoming a common problem. Also, many times I've noticed that if I hadn't heard the speech being captioned and only had the captioning to go by, I would have had little chance of correctly understanding what was actually said.
A lot of stores don't accept them anymore because of scumbag scammers abusing the system, leaving us dependent on a friend or family member to make our calls.
Similar issue here in Canada.
The relay service is not for deaf people, it's also for people with speech issues, deafblind, blind, and so on.
> transcriptionists.
Also, not as such in US. It's interpreter (usually for ASL/video call) or operator. Relay service relays messages in two ways; voice the message from the end user and then type from voice to the end user.
AI has to be absolutely perfect to replace humans for these relay calls because as you said, when it comes to medical situations, or legal issues; who would be at fault for miscommunication that led to some issues?
In US, you must also verify your identity to the relay service as federal regulations has step up the identity requirements recently. I had to do a video call with VRS to show my state ID that I'm in the state and registered with the state's relay services.
I do appreciate GP using the term "deaf relay", as this confirmed my understanding of the StackExchange post. The term "relay" is so broad that, without context, I'd assume "relay service" in telecom is whatever relays your call across vendor/state/national service borders, or something.
Your post explains it very well...
If you need the service, it's free. If you don't need the service then you can pay for it yourself if you want it. Or do what most do these days and use AI and hope for the best.
Yes the AI captioning stuff is improving, but they regularly break all the time. In my last call with AI captioning, the AI kept talking about lawyers. That was not a topic of discussion at all during the call. I was of course confused as all get out, and the hearing people had to type in what they actually meant.