If they had actually built an Ai model that could interact with the UI of any application by looking at the underlying UI components, the product might have been super cool.
That they didn't do this is rather unfortunate, because it should in theory be quite possible. Heck given recent advances in small parameter models, it might even be possible to run locally with some custom HW.
A voice only interface capable of running any android app would be pretty cool. I doubt Google will ever release that (I'd love to be proved wrong) if simply because it risks destroying the display ads market.
The main problem with their system is not that it doesn't work with every app out there, the main problem is that voice control is just not that desirable for most apps. It has its place in applications where you're already using your hands, such as when driving or when cooking or something like that, but even there you can have much richer, faster and more private interaction with a (touch)screen than would be possible with voice controls. Imagine if everyone in the entire office is constantly responding to whatsapp messages by saying "reply to xyz", then speaking the entire message out loud. Or in a restaurant, in the subway or at a crowded concert.
The other glaring problem being that you'd have to entrust this shady company with keys to your digital kingdom so that they could perform actions on your behalf from their cloud.
When they inevitably get hacked and your money gets stolen, you won't have any recourse because you authorized them to act on your behalf.
Of course this could be solved if their models ran locally on your phone and interacted with target apps through accessibility APIs, but then they're just another app.
Every part of their business is either not practical or downright insane if you think about it for more than a minute.
I'm not so sure about that - voice control is super useful when it works. Walking down the street for example it would be far easier to reply without having to use both hands, look down at the phone and type.
The problem is that it has to work _all of the time_ and be perfectly analogous to the non-voice control equivalent.
Otherwise, even in basic things like dictation it fails, because if you message a friend using dictation, the punctuation, capitalization, mumbles and other things make it seem jarring to them as if someone else is suddenly speaking.
I have one on my desk unopened. I’m in the UK and by the time it finally arrived all the stories had come out about how lame it was. Just have not invested any time on it.
"LLM + speech" was such an obvious and inevitable idea, so their only advantage was rushing to sell to the early adopters. It could have been an app, but the margin on a device is higher. But early adopters with novice coding ability realized the openAI API is just a single curl command. I like the free voice call modes the chatgpt and gemini apps have now, but I'm still using my homemade slapped-together phone and laptop apps (if you can call a few lines of shell script an app) since the groqcloud API is soooo much faster than openAI (instant really) and has `whisper-large-v3` & `llama-3.1-70b-versatile`.
26 comments
[ 3.8 ms ] story [ 68.2 ms ] threadEDIT: it's found a home!
[1] https://x.com/snazzylabs/status/1838770446801735798?s=46
I got one as a gift - been trying to sell it unsuccessfully for months now…
That they didn't do this is rather unfortunate, because it should in theory be quite possible. Heck given recent advances in small parameter models, it might even be possible to run locally with some custom HW.
A voice only interface capable of running any android app would be pretty cool. I doubt Google will ever release that (I'd love to be proved wrong) if simply because it risks destroying the display ads market.
When they inevitably get hacked and your money gets stolen, you won't have any recourse because you authorized them to act on your behalf.
Of course this could be solved if their models ran locally on your phone and interacted with target apps through accessibility APIs, but then they're just another app.
Every part of their business is either not practical or downright insane if you think about it for more than a minute.
The problem is that it has to work _all of the time_ and be perfectly analogous to the non-voice control equivalent.
Otherwise, even in basic things like dictation it fails, because if you message a friend using dictation, the punctuation, capitalization, mumbles and other things make it seem jarring to them as if someone else is suddenly speaking.
So instead of pulling out your phone to use it, you have this additional device you bought for 200 bucks.
... and the word 'people' in the title was just being politically correct.
(It also kind of ironic rewatching that video now considering all the bad press Brownlee is getting for is $50 per year wallpaper app [2].)
[1] https://www.youtube.com/watch?v=ddTV12hErTc [2] https://www.theverge.com/2024/9/24/24253023/mkbhd-panels-wal...
What big tech companies? Rabbit doesn't have any competition.
It still feels to me like there has to be an angle that I'm missing.