Anyone who's tried to explain to another person how to do something on a computer (or what to do with a computer) and eventually, in exasperation, said "here just let me drive", knows that voice UI is not the (only) UI of the future.
As I work with schoolkids, I've found the best way is to direct with long thin stick like a baton. You can keep your hand well out of their vision and indicate where you want the mouse to go. If you drive for them, they will not know what you did. (to be fair, when I point and tell, they don't always know what they did either)
Voice as a user interface doesn't scale. If you have a lot of people in a small space (an office, a coffeeshop, a metro train, etc) then voice just won't work. From a technical perspective it could work with good direction microphones to isolate the user, and excellent voice recognition to eliminate accidental activations from other people, but the cost of those things would be prohibitive, and it'd still fail if the background noise level was too high. Where voice really fails is the simple fact that a lot of people would have issues dictating a sensitive or intimate message with their voice in public.
There will always need to be a second screen- or keyboard-based UI. That doesn't mean voice won't be hugely important, but it does mean it'll never be the whole story. Consequently, I don't see it being 'the future' of UIs.
I don't think it'll ever be the dominant UI. It works brilliantly in some situations (your house, your car) but there are more situations where it doesn't (everywhere else).
they work brilliantly in dedicated offices and even in cubicles if there are isolated enough, just like people mumble into their telephone at work and in trains today.
Around here, if somebody is talking into a phone on the train, they get dirty looks. It really is not at all common and pretty awkward. I have never witnessed anybody give a voice command to their phone.
At work, we isolate the frequent-phoners from those who have to concentrate for prolonged periods of time. If more poeple start talking at work, the distractions are going to get worse.
Actually, only those who are talking loudly into a sensitive microphone held right next to their mouth get dirty looks. Plenty of people sit there having a quiet conversation and even from the seat in front or behind you can't hear them speaking
Indeed, back in my Google Glass days, I could "ok glass" a voice command, but heck if I ever actually wanted to. It's also super awkward in public... and I still look at people strangely today when they "ok google" something or the like.
In a home, I do totally get it. If it wasn't for the always listening microphones, I'd be up for it. But even then... I might ask my house to do something, I'm still going to manually want to type this comment.
Voice recognition when I’m out and about don’t want to get my phone out it’s for the best use for it for me but as this text shows voice recognition isn’t very good you had even indoors
Would love to see ultra precise and reliable eye tracking that positions the cursor on screen. Left and right clicking could be implemented with hand gestures.
This would eliminate the bandwidth and privacy issues a voice UI has while allowing to keep the hands on the keyboard for working the cli.
I think google agrees with you, you can type or speak to its assistant. I really only use Google's assistant to set alarms, it's a lot easier to say, wake me up in 20 minutes (or just typing the same query if someone else is sleeping), than looking for the "Clock" app, then clicking the add button in the "alarms" tab, then thinking what time should I set my alarm in order to sleep 20 mins.
My point is that there are a lot of really good use cases for Voice or Type commands, but a lot of them are really inconsistent or unpolished, I love how I can say "Play HipHop" and magically it will look for my Spotify's HipHop playlist and play it, but it's unpolished because it'll play the songs in a different order than how it would be played by opening the playlist manually, and it's inconsistent because if I ask the same command in the search bar it will look on google's "Play Music" instead of Spotify. It feels like they are working really hard in the backend, but too little in the frontend (if we can call it like that).
Fully agree. People who work in this space, need to spend time with Deaf or hard-of-hearing users to fully understand the day to day issues. Otherwise there is a plethora of 'cool' highly controlled demonstrated use cases that haven't scaled.
Indeed, privacy! My doctor called the other day needing some private information while I was in the office with other people nearby, about to start a meeting. I couldn't leave the room to answer her and with my umming and ahhing she cleverly surmised I was unable to talk. Wanting the information quickly she turned to yes/no questions and got the info she needed. There's simply no way I'm gonna talk into my phone "Hey Google, renew my prescription for highly sensitive medication on the bus or anywhere outside the privacy of my own home.
Actually, it depends how much you want to talk to your device.
If you really want this to work, we can install infrastructure at public places that can cooperate with your personal devices to improve your experience dramatically.
Think directional microphones and user tracking cameras that listens to you and watch your surroundings for you.
That way, you can have actual assistant as the infrastructure can be very knowledgable about the environment that's installed. For example, it can know how full the trains in the tube are and direct you to the optimal door, it an know the exact exit, it can know the wending machine and what it offers.
Suddenly, you will have voice UI to an intelligent assistant. Probably will feel as if it is a general purpose AI since it will be specialised for every location and occasion.
Maybe it can works as a subscription service where you gain advantage in crowded places against those who don't have a subscription or maybe it can be a public service and optimize the crowds movements and actions to improve productivity. Either model sounds dystopian but I would argue this is the direction we are headed to.
I think gestures with combination of buttons has more potential to solve most interfacing challenges than voice. Unless a use-case where the instruction scope is complex enough to want the use of human language, voice as an interface seems more like a fun/cool feature rather than a utilitarian one.
It would be interesting if someone could come up with a list of interface challenges that voice could solve best.
I wonder, if using camera to read cues from lips and check if user watches on screen (as in addressing device) as a secondary channel to mic could improve situation.
While this might be a set of problems, I don't think it is the current bottleneck. There are plenty of opportunities for VUI in quite/private settings, if it was a better way of interacting with machines.
IDK what the eventual VUIs look like, but I have a feeling were still at the discovery phase. Even operating GPS/calls/music via voice while driving is still awkward. In a car you have privacy, quite and a need to keep hands and eyes free. It should be the ideal environment for VUI.
Personally, I'm curious to see more voice used as an additional input to existing UIs, the way mouse, keyboard & touch are currently used simultaneously.
Agreed. I’d like to have full voice control in the car but it’s just not sophisticated enough to do anything other than a few simple rote commands without taking my eyes off the road and/or hands off the wheel.
I can’t just ask it what the last couple episodes of a podcast are, have it tell me, and say OK play the second one. Or transparently even handle nav without fiddling.
Voice recognition has gotten decent—though it’s better with Alexa’s directional microphones. NLP and taking associated actions is still pretty bad.
I suspect you’re right. More conversational NLP would help. But, as a practical matter, we probably also need applications that are explicitly designed to work in concert with voice UIs rather than simply using voice as an alternative for filling in a search box or whatever. For example, you need apps that expect a certain back and forth to resolve an action.
While I agree about voice (and that even though I've been working on TTS for 6 years now ;)), I don't think screens and keyboards should be the ultimate goal, but rather something like brain-computer interfaces.
Maybe not exactly voice, but natural language UI is definitely the future. It would be awesome if I could talk to git and tell it to do exactly what I want in English. Instead of rote-learning arcane incantations.
It depends on the complexity and dangers of the command. Git actually seems like a very poor candidate for natural language.
Consider: Legal documents initially began as natural language evaluated by very effective natural language processors (other people). Over time, as the edge cases and ambiguities built up, the language of law diverged to the point that legalese is hardly considered "natural" today.
> It would be awesome if I could talk to git and tell it to do exactly what I want in English.
Considering that even high-IQ native English speakers have huge, huge problems "just doing what you want", even after explicit and unambiguous instructions -- don't hold your breath. (Maybe after the flying moon car.)
I don't think that natural language is a useful interface for tools like Git. Natural language is inherently verbose and ambiguous. That's precisely what you don't want when using tools like that.
Natural language has a low signal to noise ratio compared to explicit verbs and keywords.
Git just has less than stellar usability when it comes to performing common actions. Most of that probably is intentional because on the other hand it affords you a great deal of flexibility. If you want to improve the usability of Git standard UI patterns like providing better keywords and shortcuts to frequently used features will suffice. It's just a matter of the design philosophy behind the tool.
I'm less interested in voice assistants and more interested in how speech recognition will interact with existing interfaces. One advantage GUI has over CLIs is the "graphical" component. Graphics can output to the user so much more information in so much less time than text (colloquially "a picture says a thousand words"). Text-to-speech output has the bandwidth problem a CLI has and it isn't persistent. If you didn't hear the voice assistant the first time, you have to make it repeat the entire sentence.
What I think is really the killer feature of speech recognition is its role as an input. Anecdotally, I find the only times I use any voice UI is for setting alarms (because iOS's time input makes you scroll thru every minute) and asking a sufficiently long question to Google (because touchscreen keyboards are still imprecise and I've got fat fingers). Even though both Android and iOS have speech recognition built into their keyboards I think there's something someone is missing on the input side.
Couldn't agree with you more. So much so we actually are building a company to help developer do just that - http://slanglabs.in :) Provide tools to developers to build a voice interface on top of existing mobile apps that in our mind would marry the best of voice (as an input mechanism) with GUI (as an output mechanism).
Voice UI without general AI is not the future. Forget about it.
Even for long form text entry it's tiresome (ever tried speaking for 1 hour?) -- never mind the noise (e.g. in an office), privacy issues, discoverability of options, and all that.
TFA briefly mentions it but I can't imagine how "Voice UI" can be even decent without strong AI. In order to be more efficient than inputting commands with a keyboard or mouse you want an "assistant" that will be able to infer what you want from context. And of course you want to be reasonably certain that it'll get it right without having to double check everything all the time.
Also, make it available offline. I don't want $bigdata to know everything I do on my computer. Amazon Echo & friends are way too orwellian for my taste. I don't know if the general public will care about that however.
Wouldn't having a strong AI require lots of user data?
And having the option to make it work offline means letting go of all the possible precious tidbits that could make your AI strong.
Maybe, but I don't really care to use Big Brother AI. I mean, you and I are pretty clever and yet we don't require an internet connection to function, do we? Maybe it'll take longer to perfect but I hope we'll get there eventually when it comes to artificial intelligence.
Voice UI has some use cases where it is great and some use cases where it is terrible.
"Fire and forget" commands are great. "Hey alexa, set an alarm for 8am". Once you trust alexa to do these tasks you don't need any feedback.
But tasks where you need constant feedback are not well suited to voice UI. If you accidentally blast music through your hi-fi asking alexa to "turn it down a bit" 4 times and then "turn it up" finally to get to a nice listening volume takes much much longer than turning a dial while listening to the auditory feedback you get.
There should be a keyboard with integrated voice recognition, and a "voice assist" button.
Remove any superfluous keys (anything you might have to look down for) and just keep the typewriter keys.
and add a voico-correct to operating systems, not "auto". you mistype and you can dictate the correct word for it to fix for you.
once we have this paradigm the os will evolve for it.
I think FULL voice has privacy issues, and annoyance too. I want to quietly instruct my pc, not talk to it all day (and I don't want to have to hear someone else do that either)
It won't be for awhile yet. Add to all the other issues mentioned here its drastic reduction in accuracy when dealing with accents, let alone non English languages. We're far, far from it working universally. And that's before you even get to semantics or whether it even make sense as a UI.
No it isn't. Huge screens are, whether they come as standard monitors, virtual reality headsets, or 3D displays. People consume orders of magnitude more information that they produce. When compared to large display, voice output is just painful.
Furthermore, moving eyes is way faster than any kind of navigation input, even voice. That's why displays contain way more information than what people are able to process at any given moment.
And then there's this old principle of GUIs: making choices visible. Screen can list all available options. Voice UI cannot. Even command-line UIs support tab-complete for this reason.
If I were to be very cynical, I would say that voice UIs shine when people are illiterate enough to avoid typing and reading. A less cynical answer is that voice UI fits where big screen doesn't.
51 comments
[ 4.1 ms ] story [ 109 ms ] threadThere will always need to be a second screen- or keyboard-based UI. That doesn't mean voice won't be hugely important, but it does mean it'll never be the whole story. Consequently, I don't see it being 'the future' of UIs.
At work, we isolate the frequent-phoners from those who have to concentrate for prolonged periods of time. If more poeple start talking at work, the distractions are going to get worse.
In a home, I do totally get it. If it wasn't for the always listening microphones, I'd be up for it. But even then... I might ask my house to do something, I'm still going to manually want to type this comment.
Voice recognition when I’m out and about don’t want to get my phone out it’s for the best use for it for me but as this text shows voice recognition isn’t very good you had even indoors
This would eliminate the bandwidth and privacy issues a voice UI has while allowing to keep the hands on the keyboard for working the cli.
My point is that there are a lot of really good use cases for Voice or Type commands, but a lot of them are really inconsistent or unpolished, I love how I can say "Play HipHop" and magically it will look for my Spotify's HipHop playlist and play it, but it's unpolished because it'll play the songs in a different order than how it would be played by opening the playlist manually, and it's inconsistent because if I ask the same command in the search bar it will look on google's "Play Music" instead of Spotify. It feels like they are working really hard in the backend, but too little in the frontend (if we can call it like that).
If you really want this to work, we can install infrastructure at public places that can cooperate with your personal devices to improve your experience dramatically.
Think directional microphones and user tracking cameras that listens to you and watch your surroundings for you.
That way, you can have actual assistant as the infrastructure can be very knowledgable about the environment that's installed. For example, it can know how full the trains in the tube are and direct you to the optimal door, it an know the exact exit, it can know the wending machine and what it offers.
Suddenly, you will have voice UI to an intelligent assistant. Probably will feel as if it is a general purpose AI since it will be specialised for every location and occasion.
Maybe it can works as a subscription service where you gain advantage in crowded places against those who don't have a subscription or maybe it can be a public service and optimize the crowds movements and actions to improve productivity. Either model sounds dystopian but I would argue this is the direction we are headed to.
It would be interesting if someone could come up with a list of interface challenges that voice could solve best.
One that I would like to have is a writing platform/software that is entirely functional with voice. Something of a similar nature: http://harrypotter.wikia.com/wiki/Quick-Quotes_Quill
IDK what the eventual VUIs look like, but I have a feeling were still at the discovery phase. Even operating GPS/calls/music via voice while driving is still awkward. In a car you have privacy, quite and a need to keep hands and eyes free. It should be the ideal environment for VUI.
Personally, I'm curious to see more voice used as an additional input to existing UIs, the way mouse, keyboard & touch are currently used simultaneously.
I can’t just ask it what the last couple episodes of a podcast are, have it tell me, and say OK play the second one. Or transparently even handle nav without fiddling.
Voice recognition has gotten decent—though it’s better with Alexa’s directional microphones. NLP and taking associated actions is still pretty bad.
Voice recognition and very simple NLP have gotten good pretty fast, and is still improving fast. That's necessary but not sufficient for a good UI.
The interaction model just isn't good enough yet. We probably need a podcatcher app to be built for voice-first, in order to discover the right UI.
https://en.wikipedia.org/wiki/Hype_cycle
Consider: Legal documents initially began as natural language evaluated by very effective natural language processors (other people). Over time, as the edge cases and ambiguities built up, the language of law diverged to the point that legalese is hardly considered "natural" today.
Considering that even high-IQ native English speakers have huge, huge problems "just doing what you want", even after explicit and unambiguous instructions -- don't hold your breath. (Maybe after the flying moon car.)
Natural language has a low signal to noise ratio compared to explicit verbs and keywords.
Git just has less than stellar usability when it comes to performing common actions. Most of that probably is intentional because on the other hand it affords you a great deal of flexibility. If you want to improve the usability of Git standard UI patterns like providing better keywords and shortcuts to frequently used features will suffice. It's just a matter of the design philosophy behind the tool.
What I think is really the killer feature of speech recognition is its role as an input. Anecdotally, I find the only times I use any voice UI is for setting alarms (because iOS's time input makes you scroll thru every minute) and asking a sufficiently long question to Google (because touchscreen keyboards are still imprecise and I've got fat fingers). Even though both Android and iOS have speech recognition built into their keyboards I think there's something someone is missing on the input side.
No it isn't.
Even for long form text entry it's tiresome (ever tried speaking for 1 hour?) -- never mind the noise (e.g. in an office), privacy issues, discoverability of options, and all that.
Because most of my days when I am tired, the last thing i want to do is to open my month.
Also, make it available offline. I don't want $bigdata to know everything I do on my computer. Amazon Echo & friends are way too orwellian for my taste. I don't know if the general public will care about that however.
A tangential question. What steps can an organisation take to assuage your worries about your data being mis-used?
"Fire and forget" commands are great. "Hey alexa, set an alarm for 8am". Once you trust alexa to do these tasks you don't need any feedback.
But tasks where you need constant feedback are not well suited to voice UI. If you accidentally blast music through your hi-fi asking alexa to "turn it down a bit" 4 times and then "turn it up" finally to get to a nice listening volume takes much much longer than turning a dial while listening to the auditory feedback you get.
I have yet to see a classic gui with a decent NN- which learns your default behaviour in every step, applying those by ctrl+left-click.
There should be a keyboard with integrated voice recognition, and a "voice assist" button.
Remove any superfluous keys (anything you might have to look down for) and just keep the typewriter keys.
and add a voico-correct to operating systems, not "auto". you mistype and you can dictate the correct word for it to fix for you.
once we have this paradigm the os will evolve for it.
I think FULL voice has privacy issues, and annoyance too. I want to quietly instruct my pc, not talk to it all day (and I don't want to have to hear someone else do that either)
Furthermore, moving eyes is way faster than any kind of navigation input, even voice. That's why displays contain way more information than what people are able to process at any given moment.
And then there's this old principle of GUIs: making choices visible. Screen can list all available options. Voice UI cannot. Even command-line UIs support tab-complete for this reason.
If I were to be very cynical, I would say that voice UIs shine when people are illiterate enough to avoid typing and reading. A less cynical answer is that voice UI fits where big screen doesn't.