I thought this was a great moment to take stock of these voice assistants and do a postmortem on why the platform failed to live up to expectations. Something telling from the article is that Alexa is used to play music and check the weather and not much else, no matter how hard the division tries to get people to order Ubers or toothpaste.
I have been wondering if a next gen “GTP-4” like language model could take these AI assistants from being jukeboxes to actual assistants.
Some of the problem is not the A.I. technology but the model of the user interface. The model of Alexa is
[voice input] -> [action]
but to do more complex interactions it has to be more like
[input 1] -> [ask for clarification] -> [input 2] -> [propose action, ask for confirmation ] -> [action]
from the viewpoint of developing a training set this is exponentially more complicated. Maybe it is like playing chess or learning Atari games by reinforcement learning, but those games involve a small number of elements that combine in complex ways, whereas a conversation involves a space of millions of concepts. (e.g. "20 questions")
It's not so much that they need any specific feature ("buy toothpaste") that they need some way to recover the cost, for instance being more tied to the Amazon ecosystem, more likely to subscribe to Prime, etc.
It's not easy though. With roughly 70 million users they would need to generate about $150 a user in profit to cover $10 billion. At 2% margin they'd need to sell $7500 worth of stuff that they wouldn't otherwise sell.
What I don't get is where the $10 billion went: how much of it subsidizing the device? (If they sold 10 million devices last year it would be a $1000 subsidy which is incredible.) are they paying AWS prices to run the backend service? how much are they spending in software development, model training, etc?
And in those more complex interactions there is not enough adjustment capability. I often want Alexa to speak faster or just shorter (reduce filler words and pauses). There's no way (I am aware of) to make it more efficient for the determined user.
My biggest problem with the flow nowadays is instead of getting briefer and simpler as you and the device learn each other better, it stays just as onerous all the way through--and, in fact, becomes more so as they introduce more "By the way, you can...[new feature discovery I did NOT want forced down my throat when trying to reset my alarm while half asleep]..." asides.
It's like every lesson about not disrupting GUI flow with modal banners, and about creating shortcuts and other accelerated paths for advanced users was forgotten in VUI. At best you get a one-size-fits-all brief mode that usually isn't very brief, and doesn't reward learning so doesn't increase engagement.
VUI vendors also plainly have no clue how to enable discovery without either fucking up your home screen with some ugly "things you can do" list or being annoying about volunteering stuff when responding to your commands. It's as if they might need to *gasp* document their features instead because voice devices aren't web apps you can just fiddle with.
The net result is I actually get way more frustrated with these devices now that I'm super-experienced with the tech, not less. That's the wrong direction.
That anti-pattern of interrupting you to tell you about a new feature is common and it's the worst. All the time I feel like I am behind the 8-ball and I really need to finish some small task right now but Firefox, Photoshop and seemingly every other product wants to get in my way of doing the task.
It's a deep problem though matching the expectations of a user for a VUI with what it can so, Alexa's greatest triumph is that it had a domain that it seemed pretty complete over and that mostly people could "get" what it was capable of doing.
I feel like there are just not many truly game changing use-cases for a voice-activated assistant.
They only ever seem to be useful if you do not have free hands (e.g. cooking) which is why they are almost exclusively used as timers (at least in my household). I have tried getting it to read a recipe but that's just cumbersome (I often want to have a quick glance at ingredients, not have someone list all of them in slow succession).
Other actions such as "play stranger things on netflix" seem like useful actions at first, but again only really make sense if you are truly busy with your hands (which you would not be if you are on the couch with the remote near-by). Using the remote to turn on the tv and play something on netflix is often not more than two or three button presses (assuming the show is in your 'continue watching' list, which it usually is if it's something you specifically want to watch).
And turning on lights? Again, a novelty. Unless your house is designed in a weird way where your light switches are on opposite ends of the room you are walking into, it's always faster to just flip the switch when walking into the room than shouting out and waiting for your assistant to do it for you (never mind the fact that shouting to turn on the lights during a bathroom run at night would be highly annoying to other household members).
I like my routine that turns off my entire house when I go to sleep at night, because I'm ADHD and I won't flip off lights or the TV leaving a room as a general rule--especially in the evenings when I'm low-meds. With the recent heat-sensing models, I could even automate them to be presence detecting lights and fix that issue room by room.
I also like being able to sequence the set of several lighting, audio, and video operations that sets my theater room up for various uses. I like being able to quickly check the house temp and tweak the thermostat from the other side of the house. And I like being able to turn whole rooms on and off at once, or to set their light dimness and color for my use of the moment without getting up or fiddling with my phone, etc.
I especially like the routine that does a sunrise wake on my bedroom lights; turns off my brown noise machine a minute before my alarm since sometimes that will peacefully wake me without having to hear the beep; chatters jokes, news, etc, incessantly at me in the morning once I dismiss my alarm to make sure I actually get up; then automatically shifts into whole-house Sirius XM to play hard rock as I start my day.
And say what you want, I like being able to ask the thing over my shoulder what my office fan 5 feet behind me is set to and change that while coding and still looking at the screen, rather than stopping, screwing around with a cheap remote control and trying to see a little LCD from across the room. Like I said, I'm ADHD. If I stop to do that, who knows when I get back on task again.
I agree for simple "turn on a switch" tasks they're way overkill, but for actual automation (and perhaps accommodation) these devices are useful, especially once you start replacing ALL your environmental devices with controllable ones. It's more a question as to whether you can leverage the automation.
Are you using Home Assistant? You seem to be all in and I've started doing the research to get my place setup. I think I would like voice commands for quick actions but would like to avoid giving my voice data to Amazon, Google, Apple, etc. There seem to be some limited open source models, but their support currently seems limited.
I have not tried Home Assistant, no, but I'm keeping an eye on what happens with Matter. If I were able to consolidate the interaction to a neutral third party that could control the other stuff I already have, I'd love to do that.
LLM chaining seems like it works pretty incredibly - between the newly released Whisper Model and chaining building up a more sophisticated assistant than the current open source ones seems extremely plausible even for smaller teams.
That is an insane demo and really makes me think that building Alexa was not wrong, just early. I don’t know if that next-level capability would have been more useful to consumers or able to be monetized, but I suppose we will have to wait a few years to find out. Best of luck in your future endeavors.
Sample size of 1, but I use HomeAssistant a lot more than I ever used Alexa/Google Home. The voice interface might be interesting for the next generation, but having grown with computer/smartphones it just feels more natural to flip a switch or to start a timer on my phone by clicking than just uttering "Alexa turn on the coffee machine".
I see the value of a smart home, I just don't think Alexa is the best interface for it when we all carry a perfect little tablet that has every switch/cameras on it.
I see voice assistant being useful when trying to get stuff done while driving or otherwise occupied. And I want to be able to speak with it interactively much as I would another person. Being able to spin up new tasks and have it appropriately begin them and query for needed info or on blockers like a person would
I'm not sure if it is what it seems. That is, they maybe didn't lose $10 billion last year, the year before that, and the year before that, etc.
In accounting there is all sorts of wiggle room. They can "write off" losses that haven't been yet been realized, claim that they are paying the same rates AWS would charge you to run the back end service, add some what you spend licensing music as an expense, not give it credit for me keeping Prime or ordering another thing on Amazon (Alexa's notifications it gave when I received Amazon packages were really welcome given that I have a 1/8 mile long driveway)
The motive of overstatement is murder.
Alexa is a beloved brand, it's a bit like Disney killing off Mickey Mouse or Anheuser-Busch InBev SA/NV killing off Budweiser. Google doesn't care if they look heartless, but Amazon is laying out the business case to maintain the image they are in control.
There's this really interesting dynamic here, I would never buy an Alexa- why? Because it's an Amazon machine in your home optimized to bankrupt you. We all kind of know it, the aim is for Alexa to figure out ways to monetize you. But having said that, a smart speaker should make money! It's a bit of low margin consumer tech, massive volume. The only way you lose money on that is by.... hiring a load of AI engineers to figure out how to exploit your customers. It's also kind of funny, you sold all these devices at cost.... but why? Apple's homepod sucks, and you (hope) you're clearly differentiated against Google. So why not actually just make money selling it.
I'm just mad once again, that I didn't get the benefit of venture subsidized consumer products.
Voice assistants are just too unsophisticated, lacking required context. Recently during a drive I decided to give the Google Assistant another try - Google Maps was being utilized as the navigation but screen was too bright for my liking - it was not possible to accomplish this with voice only, yes, screen got dimmed but in the process nav got into this small floating window mode and no possible voice command was able to restore it to fullscreen - maximize, enlarge, open, fullscreen, fullsize - nope.
I was disappointed, such simple, basic yet usefull use case but it just didn't work
The Thermostat, Air Purifier also seem like loss leaders. There was like 20-30 devices that were introduced a year or two ago which also included a microwave.
Especially the thermostat, it was a joint development with Honeywell and is like 1/2 the cost of other Smart Thermostats.
When I had an Echo it was only used to set alarms :$
Also, last time I checked for the Air Purifier, you had to do ~5 taps to get to the Air Purifier Data.
I worked at Alexa. I'm not surprised by the massive loss - R&D was substantial, and nobody will ever be comfortable buying via voice (I own multiple Echo devices, use them extremely frequently, but disabled voice purchasing the first day I set them up). I did see multiple attempts at alternative monetization, like special voices via neural TTS translation (which work awesome by the way) but the price point will never come close to recouping even the trained model GPU investment I'd bet.
Ambient intelligence is science fiction level cool - especially if it follows you around throughout multiple contexts like Alexa does - but if Amazon's not willing to take the hit on it this halting further inflow seems like a prudent move.
I actually reorder consumables I've purchased before by voice all the time (e.g. Alexa, order more cat litter) since Alexa is very good at that one particular thing and has an efficient flow for it. I keep a PIN on it, of course, re: comfort.
I cannot imagine actually shopping with one, mind you.
In addition to the R&D, the average Amazon Device (Echo, FireTV, Fire Tablet, Kindle) is sold at a loss. Amazon doesn’t make a gross profit on their main branded devices. Ring, eero, blink are slightly different. Kindle is the closest to break even.
The article doesn't divulge actual usage details, but Alexa does have incredible market share given that both google and apple probably are counting android/ios users as users. I have never seen anyone use google voice assistant ever.
Many of us are familiar with the “unhelpful valley” where an incompetent assistant is way more of a drain on your time than having no assistant and simply doing everything yourself. We are decades away from when a computer algorithm can do the job.
Alexa barely understands semantic English. I don’t have time for an illiterate assistant to “help” me with anything more than setting timers or reciting the weather.
It's bad for civilization for everyone to have far-field microphones in their residences. With cell phones, the microphones are a problem but the upside is compelling enough to put in the effort of mitigating some of the risk and accept the rest. With Alexa/Google Home/etc, on the other hand, the upside isn't there. The state of cybersecurity is simply not good enough for a company to credibly promise that it only listens after a wakeword.
Not only won't I allow Alexa or its competitors into my home, if I visit someone and see that they have it, I'll tell them that's a faux pas. I suspect many others do the same.
Not to be too pointed, but if I invited you into my house and you told me my choice of tech was a faux pas, I would invite you out again.
I have a number of Alexa and Siri (and Ring and Nest, for that matter) devices -and- I'm aware of the far-field microphones -and- I'm aware of the surveillance potential, -and- I have chosen not to care. That's in no small part because the options for monitoring me go further than that, and they will be used if someone has that much interest. So I'm not going to limit my own convenience just to make it trivially harder for someone to do something they're unlikely to do anyway.
As far as being a guest in such a house, like most other places you don't personally own or control, I think that's something you either just have to roll with, or choose not to come by--for your own sensibilities. If you were my guest and told me you cared, I'm sure we'd just stick to the back yard, but that would be because you have a strong preference, not because I'm somehow "wrong."
To be clear, I would not expect you to agree with my stance at all, but if you actually told me to my face I was somehow breaking a social standard that only exists for a very niche set of people, or implied whatsoever that I was doing it out of naivete, I think that would be very offensive. I hope that is not something you really do in real life.
And no, in ten years of having a house full of microphones and various security geek friends in and out, nobody has ever told me the same. That's because it's fairly normal nowadays (and damned near unavoidable with some devices such as, for example, your phone or tablet, or your console's game controller, or your TV's remote control) to have a single one, and once there's a single one you may as well have as many as you need.
If anything, I think them consolidating to Alexa, Google, and Apple is probably a good thing. For awhile there, we were starting get microphones in all the individual devices too, and then you really have no control.
>Not only won't I allow Alexa or its competitors into my home, if I visit someone and see that they have it, I'll tell them that's a faux pas. I suspect many others do the same.
I suspect that with a large enough sample size, you could find "many others" that do the same.
The majority of people don't care, though. I can count on one hand the number of homes I have been in over the past 5 years that don't have an Echo, HomePod, or Nest, and those with them would be near or in the triple digits.
I'm not providing a value judgment on whether or not people should own these, but in my anecdotal experience, it might be a faux pas to you, but not to society on the whole.
Maybe showing people that you are essentially building a spy agency in order to sell them knick knacks slightly more effectively wasn't the brightest long term business plan?
41 comments
[ 1.5 ms ] story [ 104 ms ] threadI have been wondering if a next gen “GTP-4” like language model could take these AI assistants from being jukeboxes to actual assistants.
It's not so much that they need any specific feature ("buy toothpaste") that they need some way to recover the cost, for instance being more tied to the Amazon ecosystem, more likely to subscribe to Prime, etc.
It's not easy though. With roughly 70 million users they would need to generate about $150 a user in profit to cover $10 billion. At 2% margin they'd need to sell $7500 worth of stuff that they wouldn't otherwise sell.
What I don't get is where the $10 billion went: how much of it subsidizing the device? (If they sold 10 million devices last year it would be a $1000 subsidy which is incredible.) are they paying AWS prices to run the backend service? how much are they spending in software development, model training, etc?
Don’t forget that between all those steps there’s a “store everything you say in order to sell you stuff later” step
It's like every lesson about not disrupting GUI flow with modal banners, and about creating shortcuts and other accelerated paths for advanced users was forgotten in VUI. At best you get a one-size-fits-all brief mode that usually isn't very brief, and doesn't reward learning so doesn't increase engagement.
VUI vendors also plainly have no clue how to enable discovery without either fucking up your home screen with some ugly "things you can do" list or being annoying about volunteering stuff when responding to your commands. It's as if they might need to *gasp* document their features instead because voice devices aren't web apps you can just fiddle with.
The net result is I actually get way more frustrated with these devices now that I'm super-experienced with the tech, not less. That's the wrong direction.
Sometimes it seems like this is the style manual for software today
https://tvtropes.org/pmwiki/pmwiki.php/Funny/Idiocracy
That anti-pattern of interrupting you to tell you about a new feature is common and it's the worst. All the time I feel like I am behind the 8-ball and I really need to finish some small task right now but Firefox, Photoshop and seemingly every other product wants to get in my way of doing the task.
It's a deep problem though matching the expectations of a user for a VUI with what it can so, Alexa's greatest triumph is that it had a domain that it seemed pretty complete over and that mostly people could "get" what it was capable of doing.
They only ever seem to be useful if you do not have free hands (e.g. cooking) which is why they are almost exclusively used as timers (at least in my household). I have tried getting it to read a recipe but that's just cumbersome (I often want to have a quick glance at ingredients, not have someone list all of them in slow succession).
Other actions such as "play stranger things on netflix" seem like useful actions at first, but again only really make sense if you are truly busy with your hands (which you would not be if you are on the couch with the remote near-by). Using the remote to turn on the tv and play something on netflix is often not more than two or three button presses (assuming the show is in your 'continue watching' list, which it usually is if it's something you specifically want to watch).
And turning on lights? Again, a novelty. Unless your house is designed in a weird way where your light switches are on opposite ends of the room you are walking into, it's always faster to just flip the switch when walking into the room than shouting out and waiting for your assistant to do it for you (never mind the fact that shouting to turn on the lights during a bathroom run at night would be highly annoying to other household members).
I also like being able to sequence the set of several lighting, audio, and video operations that sets my theater room up for various uses. I like being able to quickly check the house temp and tweak the thermostat from the other side of the house. And I like being able to turn whole rooms on and off at once, or to set their light dimness and color for my use of the moment without getting up or fiddling with my phone, etc.
I especially like the routine that does a sunrise wake on my bedroom lights; turns off my brown noise machine a minute before my alarm since sometimes that will peacefully wake me without having to hear the beep; chatters jokes, news, etc, incessantly at me in the morning once I dismiss my alarm to make sure I actually get up; then automatically shifts into whole-house Sirius XM to play hard rock as I start my day.
And say what you want, I like being able to ask the thing over my shoulder what my office fan 5 feet behind me is set to and change that while coding and still looking at the screen, rather than stopping, screwing around with a cheap remote control and trying to see a little LCD from across the room. Like I said, I'm ADHD. If I stop to do that, who knows when I get back on task again.
I agree for simple "turn on a switch" tasks they're way overkill, but for actual automation (and perhaps accommodation) these devices are useful, especially once you start replacing ALL your environmental devices with controllable ones. It's more a question as to whether you can leverage the automation.
I have not tried Home Assistant, no, but I'm keeping an eye on what happens with Matter. If I were able to consolidate the interaction to a neutral third party that could control the other stuff I already have, I'd love to do that.
[0] https://twitter.com/hwchase17/status/1589648226449514496
I see the value of a smart home, I just don't think Alexa is the best interface for it when we all carry a perfect little tablet that has every switch/cameras on it.
On the other hand I can just set an album going and not have to touch anything for 50 minutes.
In accounting there is all sorts of wiggle room. They can "write off" losses that haven't been yet been realized, claim that they are paying the same rates AWS would charge you to run the back end service, add some what you spend licensing music as an expense, not give it credit for me keeping Prime or ordering another thing on Amazon (Alexa's notifications it gave when I received Amazon packages were really welcome given that I have a 1/8 mile long driveway)
The motive of overstatement is murder.
Alexa is a beloved brand, it's a bit like Disney killing off Mickey Mouse or Anheuser-Busch InBev SA/NV killing off Budweiser. Google doesn't care if they look heartless, but Amazon is laying out the business case to maintain the image they are in control.
I'm just mad once again, that I didn't get the benefit of venture subsidized consumer products.
The Thermostat, Air Purifier also seem like loss leaders. There was like 20-30 devices that were introduced a year or two ago which also included a microwave.
Especially the thermostat, it was a joint development with Honeywell and is like 1/2 the cost of other Smart Thermostats.
When I had an Echo it was only used to set alarms :$
Also, last time I checked for the Air Purifier, you had to do ~5 taps to get to the Air Purifier Data.
Ambient intelligence is science fiction level cool - especially if it follows you around throughout multiple contexts like Alexa does - but if Amazon's not willing to take the hit on it this halting further inflow seems like a prudent move.
I cannot imagine actually shopping with one, mind you.
Alexa barely understands semantic English. I don’t have time for an illiterate assistant to “help” me with anything more than setting timers or reciting the weather.
Not only won't I allow Alexa or its competitors into my home, if I visit someone and see that they have it, I'll tell them that's a faux pas. I suspect many others do the same.
I have a number of Alexa and Siri (and Ring and Nest, for that matter) devices -and- I'm aware of the far-field microphones -and- I'm aware of the surveillance potential, -and- I have chosen not to care. That's in no small part because the options for monitoring me go further than that, and they will be used if someone has that much interest. So I'm not going to limit my own convenience just to make it trivially harder for someone to do something they're unlikely to do anyway.
As far as being a guest in such a house, like most other places you don't personally own or control, I think that's something you either just have to roll with, or choose not to come by--for your own sensibilities. If you were my guest and told me you cared, I'm sure we'd just stick to the back yard, but that would be because you have a strong preference, not because I'm somehow "wrong."
To be clear, I would not expect you to agree with my stance at all, but if you actually told me to my face I was somehow breaking a social standard that only exists for a very niche set of people, or implied whatsoever that I was doing it out of naivete, I think that would be very offensive. I hope that is not something you really do in real life.
And no, in ten years of having a house full of microphones and various security geek friends in and out, nobody has ever told me the same. That's because it's fairly normal nowadays (and damned near unavoidable with some devices such as, for example, your phone or tablet, or your console's game controller, or your TV's remote control) to have a single one, and once there's a single one you may as well have as many as you need.
If anything, I think them consolidating to Alexa, Google, and Apple is probably a good thing. For awhile there, we were starting get microphones in all the individual devices too, and then you really have no control.
I suspect that with a large enough sample size, you could find "many others" that do the same.
The majority of people don't care, though. I can count on one hand the number of homes I have been in over the past 5 years that don't have an Echo, HomePod, or Nest, and those with them would be near or in the triple digits.
I'm not providing a value judgment on whether or not people should own these, but in my anecdotal experience, it might be a faux pas to you, but not to society on the whole.