Still woefully unfinished compared to the marketing/demo hype.
And unable to answer well the question “shouldn’t my phone do this?”
But all in all I’m excited about the many attempts at hardware and interfaces — chat or voice I refuse to believe are “the best” interaction models for agents/llms.
> And unable to answer well the question “shouldn’t my phone do this?”
Exactly this.
So much of the functionality of the Humane gadget, and this, overlap directly with a mobile phone. In Humane's case, due to the removal of a screen (and the projector not being a suitable replacement) it was either level (in some cases) with your phone's abilities, or a lot worse in others, and at great cost. This, at least, is cheaper and has a better screen...
Big picture, all these manufacturers seem to be taking advantage of is that neither Android or iOS have baked AI into the system, yet. When (e.g.) Siri becomes the interface for a strong LLM, both gadgets lose any differentiation they ever had and just become immediately irrelevant.
The weird thing with Humane is that they decide to make it a totally different device: different data plan, different phone number, different storage etc. etc. etc. It sounds so stupid to for the user to do that. I get that they wanted to have people to use it INSTEAD of their phones, but let's face it: nobody is gonna do that. If you paired it with your phone instead, you could share the data plan, share the data (your phone also has knowledge about you), use your own phone number etc.
Was it determined if the "leaked source code" for this thing was real? If so, it showed that the Rabbit R1 didn't have any AI powering it -- all rule-based heuristics (which is... fine, it's not uncommon for rule-based hueristics to be labeled as AI these days).
So the "leak" is a bit of a nothingburger, containing partial code for the relatively boring process of letting users authenticate with online services through a sandboxed browser session, from which auth tokens etc. can be extracted. You can't infer anything about how LAM does or doesn't work from this.
They likely used "kiosk escape" tricks to get code exec within the box that runs the browser. Assuming their sandboxing is all set up correctly, this isn't particularly concerning, but it does expose the code that runs within the sandbox for analysis. That's what we appear to have here.
The attacker left behind a file named cdk.log, which is an artifact of https://github.com/cdk-team/CDK/, a container pentesting tool. They were clearly trying to escape the sandbox and pivot to somewhere more interesting, but I don't think they managed it. I think "part 2" is a bluff, this is all they have (feel free to prove me wrong, lol).
But that doesn't mean there's nothing here. Lets look at what we do have.
The most interesting detail to me is a package name list in repo/ typescript/common/base-tsconfig.json
[...]
The only code actually present is for q-web-minion-
What follows is my speculation based on the names alone:
"q" seems like a codename for the rabbit device (so q-hole rabbit hole). Q might stand for "quantum".
The problem with trying to log into and interface with consumer-facing services from 'the cloud" is that you'll get IP rate limited, blocked as a bot, etc. It would make sense to proxy traffic back out through the user's device, and that's what I'd hope q-proxy is about. The big downside with this is that it ~doubles latency and halves available bandwidth, magnifying any deficiencies of a flaky 4G connection. This is perhaps partly why their doordash demo chugged so hard. (protip to the team; use a caching proxy, with SSL, MitM. Detect CDN URLs and don't proxy those.)
This is a total stab in the dark but my guess is that bunny-host is where the LAM action happens, and bunny-builder is for LAM training.
cm-quantum-peripheral-common might be the wrist-mounted device teased in the launch event.
Addendum:
It's also possible there were some juicy credentials accessible within the container. But if there were, they aren't in this leak. In particular, it looks like they're using GCP "service account keys' (/credentials/ cm-gcp-service-account-quantum-workload/gcp-service-account- quantum-workload.json), which according to google's docs "create a security risk and are not recommended. Unlike the other credential file types, compromised service account keys can be used by a bad actor without any additional information".
There isn't enough information here (and/or my analysis isn't deep enough - "cloud" is not my forte) to determine if that'll cause any issues in practice, but if there really is a "part 2" leak, I'd guess this is how they got it."
I OCRed two screenshots that I took, so there may be errors.
I’m sure more stuff will drop throughout the day but in case folks wanted to see a hands on.
Right now I don’t see it doing anything I can’t do from my phone already.
I’m very unclear on why these (rabbit and humane) aren’t just apps. I just don’t see people carrying these in addition to their phones and dealing with the split interaction ecosystem.
I also don’t understand why neither made their products into a watch form factor. That at least solves the majority of their issue of “having it on you in a convenient way”
Also as an aside, while I think the teenage engineering design is gorgeous, I’m disappointed that the packaging is so plastic heavy. A plastic cartridge like case wrapped in plastic. Neither of which will be used again.
The real tragedy of this era of technology is that even if an important new invention did work as well as imagined in sci fi, it would be so compromised by surveillance capitalism and patterns of enshittification that no one paying attention would want to come to rely on it.
If I wanted a second phone with an always on microphone why would I buy a weird looking orange potato instead of just buying a phone that supported it?
Yes, this is the reason why the products exist. The harder question is why people are hyping them up. My guess is a combination of gadget lust and a general unease about how central smartphones have become in our lives.
The short answer is that they are paid to hype it. They launched at CES (paid) with a splashy video (paid). They got lots of media coverage (paid and paid in kind by bringing in ad revenue). They were talked-up on X by all the VCs (paid in kind by hyping the segment they invest in) and the AI influencers (paid by ads).
> Right now I don’t see it doing anything I can’t do from my phone already.
To start, I think these things are toys and think they are an opportunity to explore one potential use of the tech, probably not the right one.
But… I do understand the vision of having a device that intentionally adds friction to my interaction with digital tools, in theory, with minimal compromise in capability.
> I’m very unclear on why these (rabbit and humane) aren’t just apps.
That's the one thing that seemed really clear to me. People's phones are too locked down so these can't be apps, you'd either hit Apple restrictions right away, or would inevitably hit them as you developed further. (The first time you, for example, wanted to always listen transparently for a wake word the way an iPhone or HomePod does for Siri, and Apple blocked you for it. Or the first time you wanted to show your UI while the screen is locked, Apple is allowed do do this for Maps, but you definitely aren't allowed to in your own app).
At this point, if you want to do anything big on mobile, you have to also re-invent (or copy a pre-OEM android) smartphone, just for the chance to try it out. Just like Rabbit does here.
> Points at sign saying New York Stock Exchange, "what's this", "it's the NYSE"
These are impressive in that we did not have software that could do these sorts of things a few years ago, but they are functionally useless examples. People do not do this beyond writing tech reviews or a tech demo for a friend.
I'd really like to see useful demonstrations, of things that I actually want to know. For example I saw a tower on the horizon ~15 miles away and wanted to know what tower it was, I pointed my phone at it and zoomed in extremely far, and asked Google Lens, and it gave me the wikipedia page. This is materially better in two ways: first I can zoom in past all the other towers in my field of view, and second I get the wikipedia page, on a screen, I don't know what information I need about it, it sort of depends on what it is, and that page is a good place to go for that.
What other demos do we have for these devices (R1, AI Pin) of real world scenarios? So far the reporting is lacking on these, and where they have been demoed (e.g. AI Pin translation) it has been with poor results.
What about mutts? Would be cool if dog physiognomy is so advanced it could identify distinct breed percentages. Then we could apply it to humans. That would be useful!
Alternatively you could ask the owner holding the dog.
I don't disagree that knowing the breed might be nice, but again I feel like it lacks a real-world use-case. How many dogs do I see in day to day life that are unaccompanied? None.
100%. The iOS vision accessibility APIs are top notch and consistent throughout the system. A simple example, Siri describes the image in the notification if you have announce messages turned on
Accessibility is the biggest use-case I can see for all this stuff, and if that's what we get from this wave of AI, then great. But that's not how any of it is being pitched, and the tech itself is not particularly accessibility friendly in other ways so I don't think it's properly on the radar of these companies.
> These are impressive in that we did not have software that could do these sorts of things a few years ago, but they are functionally useless examples.
I disagree.
Here's a thing I'd like to do that I can't do today:
I go out for a 15 minute walk. I turn my device on. When I get back, I have a text file about my walk annotated with what I saw, landmarks, weather, perhaps an unusual car, and what I bought at the coffeeshop, all taken from the camera alone.
That's useful, doesn't exist now, but could exist in the future, using technology like this.
I'm interested in what your use-case is. I understand the feature request, but I don't understand why a text file of those things is something I'd want.
I think there are some very interesting accessibility use-cases here, but that's not how the tech is being pitched.
personally, i’d love this, as long as i know everything is staying local and not being sold/shared for a profit.
i regularly find myself thinking things like “what movie was i talking about with that guy in the gym?” or “what did i tell my wife we needed to get at the drug store?”
while i have gotten into a habit of keeping a long running note in my phone, there are still lots of blind spots in my memory that leave these fragmented mini-mysteries. having a machine that i could ask “what was that song playing in the supermarket i wanted to download?” and get a useful answer would be a huge boon in my life, if only to quiet the anxiety of leaving these threads hanging in my mind.
Fwiw, “what did i tell my wife we needed to get at the drug store?” is actually our household's main use case for Amazon Alexa.
Granted, you have to be home for that, but "Alexa add pickles to the shopping list" is something we're all accustomed to saying now, to the device in the kitchen. Then whoever does the shopping checks that list whenever they go out.
Wouldn't any of these devices be better paired with some sort of app-connection to a phone? A standalone Rabbit device just seems to be under-powered and certainly improvable and capable of more by being connected to a phone with more resources...
Why would I want a Rabbit if I could get a Rabbit that also connects to my mobile computing device that is engineered for performance?
I have a feeling that the founders of this and the Humane pin know deep down that if they allowed any app-connection to a phone, it would make it even more obvious to the users that the product is useless as the phone can do everything and far better. So they have actively decided to not allow it.
The concept (if I understand it correctly) is that it can remote-control (or maybe rather local-control) your apps. An app couldn’t do that at least on iOS, it would have to be integrated into the OS.
https://github.com/OpenAdaptAI/OpenAdapt runs locally on your machine to automate repetitive tasks in desktop apps. We have plans to build a mobile interface as well.
Rabbit R1 sounds much more useable and better than the significantly over-priced + subscription Humane AI Pin scam that the VCs and paid promoters were screaming about it as one of the "Best Inventions of 2023". [0]
Even if Rabbit R1 is a scam, it looks like a very good and cheap scam device for a first version that "actually works" and is more polished than Humane's AI pin.
This tells me that Humane's AI Pin performed worse than the Rabbit R1 and Humane is no where near worth >$1BN and after 5 years this is what the result of all of that is.
But so far overall, there is nothing that these devices can do better than a mobile device and an app at this time.
There's no way I'd trust any company like this with all these credentials (however they spin it) for such a marginal convenience improvement – maybe – that I have to carry around another device for. I extremely don't get it. I am in no way a tech minimalist, but everything I have has its place, and if it gets in my way then it's shoved somewhere else very quickly. I cannot possibly imagine how this would earn its place.
The Meta Ray-Bans have multi-modal AI built in (yes, vision that it can very quickly look and explain you see, is ridiculous how good it is), can interact/notifications with your phone, and is extremely fast and reliable.
They also don't look clunky at all anymore.
I seriously wear them all the time and would never want to carry a Rabbit with me along with my phone.
I'm a huge fan. You wear it seamlessly and AI becomes true extension in your ear. Putting in ear phones seems off now.
For vision utility, mostly identification:
- look and tell me info about what is this mushroom while hiking
- asked it to explain facts about the artist of a painting at a museum
- what is this PDF on my computer screen generally about
Other utility is simply discussion. Ask questions while cooking or if you you have a random idea. For example, I used it to see ingredient alternatives while cooking home-made pasta. I also used it to brainstorm related gift ideas for my wife while purchasing a present for my wife.
It also has real-time integration (weather, news, etc.).
Finally, I don't use social media. I have a private Instagram however to post pictures and videos since the footage and auto-stability features is truly next level.
My friends mostly use for documenting their kids lives privately. It makes that Vision Pro ad where the dad is taking photos of the kids with the giant goggles a hilarious joke.
Edit: Probably the only reason it hasn't taken off is because "Meta". It's probably the best new gadget I've bought in 5+ years.
Sorry these sound like marketing brochure answers not real life experience.
I believe you that you own and love this device but be honest about what you really used it for, not what you could use it for.
It's a phone, without the phone bits.
Another device to carry, charge, upgrade every few years? It's not a hard no, but the barrier is also very high. Smart watches at least mostly solve the "carry" problem, as you wear it in a very normal and socially acceptable way.
Nothing about the demo addressed something that I felt was a pressing need. Neat tech concept, but not seeing the market.
72 comments
[ 4.6 ms ] story [ 132 ms ] threadStill woefully unfinished compared to the marketing/demo hype.
And unable to answer well the question “shouldn’t my phone do this?”
But all in all I’m excited about the many attempts at hardware and interfaces — chat or voice I refuse to believe are “the best” interaction models for agents/llms.
Exactly this.
So much of the functionality of the Humane gadget, and this, overlap directly with a mobile phone. In Humane's case, due to the removal of a screen (and the projector not being a suitable replacement) it was either level (in some cases) with your phone's abilities, or a lot worse in others, and at great cost. This, at least, is cheaper and has a better screen...
Big picture, all these manufacturers seem to be taking advantage of is that neither Android or iOS have baked AI into the system, yet. When (e.g.) Siri becomes the interface for a strong LLM, both gadgets lose any differentiation they ever had and just become immediately irrelevant.
EDIT: Wayback Machine got a snapshot of it: https://web.archive.org/web/20240424095544/https://github.co...
"tl;dr: The "leak" seems real, but doesn't prove any of the claims made in the readme.
This statement from Peiyuan Liao, the rabbit CTO, is consistent with what I'm seeing here: https://twitter.com/liaopeiyuan/status/ 1782922595199033662
So the "leak" is a bit of a nothingburger, containing partial code for the relatively boring process of letting users authenticate with online services through a sandboxed browser session, from which auth tokens etc. can be extracted. You can't infer anything about how LAM does or doesn't work from this.
They likely used "kiosk escape" tricks to get code exec within the box that runs the browser. Assuming their sandboxing is all set up correctly, this isn't particularly concerning, but it does expose the code that runs within the sandbox for analysis. That's what we appear to have here.
The attacker left behind a file named cdk.log, which is an artifact of https://github.com/cdk-team/CDK/, a container pentesting tool. They were clearly trying to escape the sandbox and pivot to somewhere more interesting, but I don't think they managed it. I think "part 2" is a bluff, this is all they have (feel free to prove me wrong, lol).
But that doesn't mean there's nothing here. Lets look at what we do have.
The most interesting detail to me is a package name list in repo/ typescript/common/base-tsconfig.json
[...]
The only code actually present is for q-web-minion-
What follows is my speculation based on the names alone:
"q" seems like a codename for the rabbit device (so q-hole rabbit hole). Q might stand for "quantum".
The problem with trying to log into and interface with consumer-facing services from 'the cloud" is that you'll get IP rate limited, blocked as a bot, etc. It would make sense to proxy traffic back out through the user's device, and that's what I'd hope q-proxy is about. The big downside with this is that it ~doubles latency and halves available bandwidth, magnifying any deficiencies of a flaky 4G connection. This is perhaps partly why their doordash demo chugged so hard. (protip to the team; use a caching proxy, with SSL, MitM. Detect CDN URLs and don't proxy those.)
This is a total stab in the dark but my guess is that bunny-host is where the LAM action happens, and bunny-builder is for LAM training.
cm-quantum-peripheral-common might be the wrist-mounted device teased in the launch event.
Addendum:
It's also possible there were some juicy credentials accessible within the container. But if there were, they aren't in this leak. In particular, it looks like they're using GCP "service account keys' (/credentials/ cm-gcp-service-account-quantum-workload/gcp-service-account- quantum-workload.json), which according to google's docs "create a security risk and are not recommended. Unlike the other credential file types, compromised service account keys can be used by a bad actor without any additional information".
There isn't enough information here (and/or my analysis isn't deep enough - "cloud" is not my forte) to determine if that'll cause any issues in practice, but if there really is a "part 2" leak, I'd guess this is how they got it."
I OCRed two screenshots that I took, so there may be errors.
https://youtu.be/W2pYTRdX5LA?si=giLseweTP0zWf-M-
I’m sure more stuff will drop throughout the day but in case folks wanted to see a hands on.
Right now I don’t see it doing anything I can’t do from my phone already.
I’m very unclear on why these (rabbit and humane) aren’t just apps. I just don’t see people carrying these in addition to their phones and dealing with the split interaction ecosystem.
I also don’t understand why neither made their products into a watch form factor. That at least solves the majority of their issue of “having it on you in a convenient way”
Also as an aside, while I think the teenage engineering design is gorgeous, I’m disappointed that the packaging is so plastic heavy. A plastic cartridge like case wrapped in plastic. Neither of which will be used again.
the real end game for AI devices is an always on microphone which is impossible on iOS.
"Activate to launch a command" is not very compelling, the real magic comes from some combination of:
- is aware of the environment
- can talk to like a person
- can act on the world
Before cost, capability, privacy, and expectations on how to "use" a "tool" get in the way.
Try to get a VC to give you $100M for an app
To start, I think these things are toys and think they are an opportunity to explore one potential use of the tech, probably not the right one.
But… I do understand the vision of having a device that intentionally adds friction to my interaction with digital tools, in theory, with minimal compromise in capability.
That's the one thing that seemed really clear to me. People's phones are too locked down so these can't be apps, you'd either hit Apple restrictions right away, or would inevitably hit them as you developed further. (The first time you, for example, wanted to always listen transparently for a wake word the way an iPhone or HomePod does for Siri, and Apple blocked you for it. Or the first time you wanted to show your UI while the screen is locked, Apple is allowed do do this for Maps, but you definitely aren't allowed to in your own app).
At this point, if you want to do anything big on mobile, you have to also re-invent (or copy a pre-OEM android) smartphone, just for the chance to try it out. Just like Rabbit does here.
They’re all push to talk. They could literally be an app that you launch from the shortcut button and provide the same functionality.
> Points at sign saying New York Stock Exchange, "what's this", "it's the NYSE"
These are impressive in that we did not have software that could do these sorts of things a few years ago, but they are functionally useless examples. People do not do this beyond writing tech reviews or a tech demo for a friend.
I'd really like to see useful demonstrations, of things that I actually want to know. For example I saw a tower on the horizon ~15 miles away and wanted to know what tower it was, I pointed my phone at it and zoomed in extremely far, and asked Google Lens, and it gave me the wikipedia page. This is materially better in two ways: first I can zoom in past all the other towers in my field of view, and second I get the wikipedia page, on a screen, I don't know what information I need about it, it sort of depends on what it is, and that page is a good place to go for that.
What other demos do we have for these devices (R1, AI Pin) of real world scenarios? So far the reporting is lacking on these, and where they have been demoed (e.g. AI Pin translation) it has been with poor results.
That's a lot of hardware and compute just to have a gadget that helps identify dog breeds, assuming the dog is purebred to begin with.
"Can I eat this?"
Pic of random foliage > is this safe to eat
I don't disagree that knowing the breed might be nice, but again I feel like it lacks a real-world use-case. How many dogs do I see in day to day life that are unaccompanied? None.
https://time.com/3667058/frisbee-history/
Part of that is because it’s sold as accessibility or out of the way enough that people don’t encounter them regularly.
But even then, a lot of the stuff is right there in the photos app (what plant/bug/animal is this) or sign reading and OCR
I disagree.
Here's a thing I'd like to do that I can't do today:
I go out for a 15 minute walk. I turn my device on. When I get back, I have a text file about my walk annotated with what I saw, landmarks, weather, perhaps an unusual car, and what I bought at the coffeeshop, all taken from the camera alone.
That's useful, doesn't exist now, but could exist in the future, using technology like this.
I think there are some very interesting accessibility use-cases here, but that's not how the tech is being pitched.
i regularly find myself thinking things like “what movie was i talking about with that guy in the gym?” or “what did i tell my wife we needed to get at the drug store?”
while i have gotten into a habit of keeping a long running note in my phone, there are still lots of blind spots in my memory that leave these fragmented mini-mysteries. having a machine that i could ask “what was that song playing in the supermarket i wanted to download?” and get a useful answer would be a huge boon in my life, if only to quiet the anxiety of leaving these threads hanging in my mind.
Granted, you have to be home for that, but "Alexa add pickles to the shopping list" is something we're all accustomed to saying now, to the device in the kitchen. Then whoever does the shopping checks that list whenever they go out.
Google lens has been doing this for ages.
Why would I want a Rabbit if I could get a Rabbit that also connects to my mobile computing device that is engineered for performance?
Even if Rabbit R1 is a scam, it looks like a very good and cheap scam device for a first version that "actually works" and is more polished than Humane's AI pin.
This tells me that Humane's AI Pin performed worse than the Rabbit R1 and Humane is no where near worth >$1BN and after 5 years this is what the result of all of that is.
But so far overall, there is nothing that these devices can do better than a mobile device and an app at this time.
[0] https://time.com/collection/best-inventions-2023/6327143/hum...
The Meta Ray-Bans have multi-modal AI built in (yes, vision that it can very quickly look and explain you see, is ridiculous how good it is), can interact/notifications with your phone, and is extremely fast and reliable.
They also don't look clunky at all anymore.
I seriously wear them all the time and would never want to carry a Rabbit with me along with my phone.
For vision utility, mostly identification:
- look and tell me info about what is this mushroom while hiking
- asked it to explain facts about the artist of a painting at a museum
- what is this PDF on my computer screen generally about
Other utility is simply discussion. Ask questions while cooking or if you you have a random idea. For example, I used it to see ingredient alternatives while cooking home-made pasta. I also used it to brainstorm related gift ideas for my wife while purchasing a present for my wife.
It also has real-time integration (weather, news, etc.).
Finally, I don't use social media. I have a private Instagram however to post pictures and videos since the footage and auto-stability features is truly next level.
My friends mostly use for documenting their kids lives privately. It makes that Vision Pro ad where the dad is taking photos of the kids with the giant goggles a hilarious joke.
Edit: Probably the only reason it hasn't taken off is because "Meta". It's probably the best new gadget I've bought in 5+ years.
- identification of things (for me, plants)
- asking questions, facts about things (history)
- asking random questions / brainstorming / bouncing ideas (cooking thing real, gift idea real)
- notifications and phone integration
- really excellent private photos/videos dump my dog and wife (friends use for kids)
Nothing about the demo addressed something that I felt was a pressing need. Neat tech concept, but not seeing the market.