Eh, who knows really? Having something in the lab that works and getting a product team to understand how to make it useful to customers aren’t the same problem.
In some sense I’ve always thought Apple is focused so much on reliability and deterministic approaches that wrapping their heads around probabilistic outcomes is harder than for other companies.
At a big company, unless you have some serious lubrication, the first 12-18 months are spent landing. Usually you have to land your big changes in a planning cycle which are usually 6-12 months. This often means, for your first ~2 years, you're not able to do much.
That, plus the lost years of productivity during COVID -- I'd say it's okay to forgive for not turning the ship.
We are talking about 5 years, which is longer than the average tenure at most tech companies.
I also don't buy any Covid-related excuse with this one. If anything, I personally saw at least a couple companies make good use of the "never let a good crisis go to waste" mantra, using the general chaos of the early months of Covid to make some long-standing necessary-yet-painful changes - that is, changes that pretty much everyone knew were the way forward in the long term, but had risk of hits to revenue or much higher expenses in the short term. Using the pandemic as an excuse to go all-in on AI/Siri (e.g. "People are spending more time at home/on screens and we need a better voice interface") would have been the perfect approach IMO.
Apple sometimes does incremental improvements, and sometimes does an entirely new release.
Five years to improve? Sure, probably could've done a lot. Five years to start completely new? Not enough time to make it land with a bang compared to the existing Siri.
Yeah when it's a new product it makes sense getting it right before releasing. But leaving their customers with crap for 5 years does not make ton of sense.
Siri is honestly so bad, I don't use it. Every time I try to schedule a meeting it tries to incorporate it in the calendar and looks for a contact. If I say "put meet with Mike on my calendar for 2 PM on Wednesday" it'll come back with "I can't find Mike on your contacts." Unless that's changed. Then when you ask it a question its 50/50 if it answers or just gives you useless "this is what I found on the web."
I've started using the ChatGPT app with Siri and it feels like how it should. Only problem is it can't do the scheduling or other useful things.
If they'd hire the right person, they could have Siri in the top 3 of assistants within the timespan of a perhaps few months. It's not rocket science anymore and in fact most of the underlying tech is already open source.
Products at Apple have to meet a high bar for all of: Privacy Compliance,Public Relations+Marketing alignment,World Class R&D,Focus Testing,UAT and Deployment on billions of devices. I don't see how one person could make that big of an impact. And in a few months? Somehow I don't think you thought this statement through very well.
I agree, but we’re still talking about five years for a huge company with colossal cash reserves. Surely one person won’t cut it to do all the work, but I’d wager that the person overseeing the Siri department isn’t doing their job very well. Sounds more like there are managerial/political issues rather than technical ones.
> I’m sure he’s a smart man, but considering where Siri is, and the fact he’s been there for 5 years, I’m not holding my breath.
This is the standard Apple playbook. Five years is nothing for a new Apple product or foundational technology. The M1 was based on key ARM advances that Apple started building 10 years before it was introduced. Apple worked on the project that became the Vision Pro for 16 years.
> Can I conclude - it got eliminated from his biography on secrecy grounds?
It'd be more accurate to say that biographers would not have heard anything (from Steve or any other in-the-know Apple employees) about any projects that hadn't already been unveiled by Apple.
How much does a typical Apple programmer make a month?
Salary no doubt, but I’m not aware that a team is operating at a million+ a day. Red Bulls don’t cost that much, and I doubt any of them are taking yoga classes during their shift lol
I suspect most of that figure is in machine time, but otherwise they pay engineers a lot and Apple likewise occupies a non-trivial % of all the commercial real estate in Cupertino, employees security guards, other staff, etc.
Apple pulls in hundreds of millions per day, I'd think that a lot of them should be going to AI if they want to remain relevant.. it's not hard to imagine Google, Facebook, or maybe even a seriously retooled Microsoft coming out with an earbud that acts as a smart assistant and pretty much replaces the modern smartphone
Machine time was a metric that never made any sense to me. I should’ve lead off with machine time excluded that millions a day statement seems hyper inflated.
> helmed by John Giannandrea, Apple’s head of AI, who was hired in 2018 to help improve Siri.
In the age of ChatGPT, Siri not only hasn’t been improved but has been getting even dumber lately with no significant announcements towards its improvements of understanding and doing more tasks in any WWDC recently. I would take what he does with a grain of salt.
It feels like it's the economics holding it back at this point.
I cobbled together my own smart home voice assistant on a weekend a few weeks ago, sitting on top of the OpenAI APIs (Whisper, GPT-4), and of course using porcupine for wake word detection.
It can do things I could never get the commercial products to do properly, for example I gave it a memory: When a user command comes in, I have GPT-4 evaluate whether it can be executed immediately or requires later follow-up. When a sensor event happens, the machinery re-prompts GPT-4 with the user command backlog, the sensor backlog and the current state, and it figures things out. That way, things like "Please turn of the lights after I leave the room" now work just fine, and all it takes is an afternoon of hacking and a PIR sensor on my little DIY Homebrew-lexa wood boxes. And of course it's also much better at interpreting natural language commands "in spirit" or "creatively".
I'm sure Amazon, Google & Apple have made all of these tinkering experiments, too, but deploying LLM-backed voice services to tens of millions just isn't affordable yet, especially when you factor in risk and liability.
I bought little wooden boxes with a hinged lid, stained them in a lovely walnut color, put ReSpeaker 2.0 USB mic arrays under round speaker grilles on top of the lids, two 3w/8ohm speakers into the sides of the box, and a RasPi4 with a WM8960 sound card / speaker driver on the inside.
The boards are raised off the wood surfaces by PCB spacers (I embedded M2.5 threaded sockets into the wood) and I bought speaker grilles that bulge out a little at edge of the cylinder, so that the 4-mic array would remain fully exposed also laterally. I covered the side speakers and the mic array speaker grille with acoustic textile.
The onboard code is written in very pedestrian Python and uses porcupine and the OpenAI APIs.
It roughly works like this:
1. Capture audio frames and run overlapping frames by porcupine to perform hot word detection (overlapping avoids the problem of the hotword falling inbetween frames, at a cost to latency)
2. Once the hot word has been detected, buffer all audio frames into a command buffer until silence is detected as a stop (detecting "silence" is a bit involved, taking noise levels into account, and a few other tricks, more below)
3. The command buffer is sent to Whisper for transcription
4. GPT-4 is prompted with a system message steering it's behavior, the user command transcription and a JSON print out of the state of all devices (e.g. lights and Sonos speakers) in the home, grouped by rooms
5. Following the system message, GPT-4 replies with a JSON structure of changes it would like to make to the device state, omitting unchanged bits from the original
6. Add the sensor event and memory system described above
There's a few other tricks. To improve the audio capture, I take note of spatially where the hot word is detected (i.e. which mic in the array gets the best signal) and then capture the rest & perform the silence detection with a corresponding bias.
This is actually done in a distributed fashion over the network, so if two of the AI speakers hear the same command, only one of them will end up processing it.
They end up making mainly HTTP calls to APIs that already exist around my house. I have a second RasPi in my LED shelf (another old project, https://github.com/eikehein/hyelicht/) that doubles as a Philips Hue bridge with a zigbee dongle. That's what the DIY AI speakers interact with when making changes to the lighting.
I will say: Depending on the user command and the weather in the cloud, it's pretty slow. I've tried my best to optimize the client side for perceived user latency, but there's no way around the GPT-4 API just being pretty slow, even if it's amazingly low-friction and reliable otherwise. And 3.5-turbo just doesn't cut it for what I'm trying to do.
I'd like to get all of this out of the cloud entirely. I predict the next generation of my home NAS will have a GPU in it and try to run things like fine-tuned llama2 for the home.
Have you looked at finetuning GPT3,5? I’ve heard anecdotally that it can significantly improve its ability to handle correctly formatting outputs like JSON, and the increased speed and significantly reduced cost would make it much more appealing.
Also I hope you consider posting more about your home setup as I’d love to see more.
Huge models are running on stock laptops. There is no need to send it to cloud. They had no problem e.g. sound recognition (reacting to alarms, cough, cry etc.) running only on selected devices. IPads have M1/M2 chips. And home assistant model does not need detailed understanding of neuroscience, best haskell patterns etc.
But all transformers development is pretty fresh corpo-wise. I think having a good safe dataset, which is not infringing any copyrights etc. is really hard. And they probably have to be very careful about it since it's not "only" about getting sued, but also potentially damaging partnerships they need for tv/books/music.
Btw have you tried some locally running models instead of GPT-4 for your automation? I don't want my HA touching the Internet unless necessary for 3rd party integrations but GPT-4 sets bar pretty high.
> Btw have you tried some locally running models instead of GPT-4 for your automation?
Not yet, but the desire is there, especially because the generic GPT-4 API is quite slow in responding to more complex prompts, not to mention the privacy concerns. I think the next version of my home NAS is likely to have a GPU in it and will run things like an appropriately tuned llama2 or similar to be the brain backbone of my smart home. Feels like an obvious direction for commercial NAS to go in as well.
Can't wait for a future where we can buy a Rasberry Pi 7 with a little analog compute ASIC running local inference with ease ... intelligent controllers everywhere!
It seems like coral.ai is a glimpse into this future you’re waiting for. I dont know exactly what types of AI it actually speeds up on an RPi but it seems promising…
Indeed. If you see all devices sold by a company like Amazon as storefronts, I can't help but wonder if they monetize all that well. How much they're willing to subsidize them likely also reads on whether it's financially worth it to upgrade them to the latest LLM-based stuff and on what timeline.
This is very very cool I’d be super interested if you had any repos or code snippets to share!
You make a very valid point about economics. My naive point of view is: one would think a cash cow like Apple could afford it even to a limited extent, but then again iCloud free tier is still restricted to just 5GB so they have never been too generous with their cloud offerings.
I still can't even tell it to turn the bedroom and living room lights on in one go. Bite-sized scripted chunks only. And half of the time it thinks I want to play some stupid song. I don't have my homepods for music, I even tried to remove all songs from my account but that free U2 album keeps coming back.
The only reason I use Siri is that it's the only one where I can turn off recording my voice.
Who cares ? Apple at this point as like infinite money. Plus they need to integrate more and more AI to improve their products. They already have the Hardware, now they need the SW and AI to make use of them. Same deal regarding the Apple Online Services.
Precisely. No idea why you are downvoted. Apple has been in the highly profitable position in the billions to be able to sit out and wait for years, rather than hastefully join in the AI race to zero.
So why race when you are already at the finish line? This is Apple's case and anyone who owns the hardware that they are making or creating and releasing $0 free AI models out there.
This whole article and its comments is a great big storm in a tea cup.
A co-worker wanted to show me a specific vacation photo and just wrote what was in it and the photo came up out of the hundreds he had like magic so that was kinda nifty.
Sure but why not give me file name matches first instead of random pictures, since I started the search from Finder itself? Why try to outsmart the user?
I have no problem with searching pictures using keywords if it’s limited to the Photos app.
It's still super fast for me, but now I have about 10x more internal storage in my machine than in the Tiger days (which is filled with random downloaded PDF files and other crap), so there are way more results, requiring a much more intelligent sorting algorithm than it ever did when it was new.
My team does ML for a Big Tech co. Considering how much we spend on hardware, and multiplied by all the teams doing ML in the company, we are easily spending millions a day training AI just on business as usual. This seems reasonable for any big tech.
This sounds like non-news to me. Maybe a fluff piece for Apple?
'Millions of dollars per day' is about $1 billion a year? Considering Apple's annual revenue is about $400 billion and its net income is about $100 billion, it's more than reasonable for apple to spend millions per day beefing up their AI.
I guess 'millions of dollars per day' gets more clicks than 'apple spends less than .25% of its revenue on AI'.
This does have one interesting consequence though. If Apple spends millions a day on AI training, then only similar mega-corps will be able to complete.
AI is not a democratising force, it is a capital concentrator.
Even if they spent 100 million per day it could arguably be money well spent and they would still be the 3rd or 4th most profitable company in the world.
I think it was hacker news that compared voice assistants. It was Google assistant > Alexa > dog shit > Siri. Siri is such a joke that I do stuff physically instead. But on the flip side, how often do people choose a phone based on the quality of the voice assistant?
I'm honestly not sure why Siri sucks so much. Apple could probably hire 10,000 employees to manually train for various scenarios brute-force (10,000 x $200K (salary + admin) = $2B), and it would still be chump change to them.
I use Siri to set timers, like for the pasta that I am cooking. It does pretty well with that. Every once in a while I get a 30 minute timer instead of a 13 minute timer.
The iPhone was in development for like 8 years before it saw the light of day and everyone shat all over it and it still redefined mobile phones. Apples Silicon was in development for like 15 years and now laptop makers don’t even have a leg to stand on.
Apple isn’t going to release a half cocked ai project to the world. Apple isn’t Google who hasn’t released anything of note in like 10+ years. They aren’t Meta pouring billions into a metaverse no one wants.
Releasing something top-notch and then letting it whither away is a sad Apple tradition indeed. They've done this with the Mac pro multiple times. First the trash can, then the cheese grater.
At least Google kills something when they lose interest.
Apple can afford to experiment and waste their money in the millions, because they are profitable in the billions. AI doesn't affect Apple's core central products at all. It's complementary.
Unlike the majority of so-called ChatGPT-wrapper and VC fuelled LLM companies burning millions a day on training and scrambling to compete with $0 on-device AI models.
Apple does not need to compete with OpenAI (Although Apple has the hardware and software to do so) and can do so at any time. Google is the one that needs to. So all of the "But but but Siri", "Siri is garbage", "Apple isn't doing anything with Siri", etc isn't even the point.
On-device LLMs or AI is the thing that matters to Apple and Siri will likely transition to a hybrid offline / online system eventually. Apple is in no rush to compete.
Apple has a huge opportunity for AI + personal privacy if they can get an LLM on the phone that doesn’t need a cel connection to be good and they can somehow promise won’t phone home your info. I would recommend they spend more than this.
84 comments
[ 6.0 ms ] story [ 151 ms ] threadI’m sure he’s a smart man, but considering where Siri is, and the fact he’s been there for 5 years, I’m not holding my breath.
In some sense I’ve always thought Apple is focused so much on reliability and deterministic approaches that wrapping their heads around probabilistic outcomes is harder than for other companies.
That, plus the lost years of productivity during COVID -- I'd say it's okay to forgive for not turning the ship.
I also don't buy any Covid-related excuse with this one. If anything, I personally saw at least a couple companies make good use of the "never let a good crisis go to waste" mantra, using the general chaos of the early months of Covid to make some long-standing necessary-yet-painful changes - that is, changes that pretty much everyone knew were the way forward in the long term, but had risk of hits to revenue or much higher expenses in the short term. Using the pandemic as an excuse to go all-in on AI/Siri (e.g. "People are spending more time at home/on screens and we need a better voice interface") would have been the perfect approach IMO.
Apple sometimes does incremental improvements, and sometimes does an entirely new release.
Five years to improve? Sure, probably could've done a lot. Five years to start completely new? Not enough time to make it land with a bang compared to the existing Siri.
Siri is honestly so bad, I don't use it. Every time I try to schedule a meeting it tries to incorporate it in the calendar and looks for a contact. If I say "put meet with Mike on my calendar for 2 PM on Wednesday" it'll come back with "I can't find Mike on your contacts." Unless that's changed. Then when you ask it a question its 50/50 if it answers or just gives you useless "this is what I found on the web."
I've started using the ChatGPT app with Siri and it feels like how it should. Only problem is it can't do the scheduling or other useful things.
This is the standard Apple playbook. Five years is nothing for a new Apple product or foundational technology. The M1 was based on key ARM advances that Apple started building 10 years before it was introduced. Apple worked on the project that became the Vision Pro for 16 years.
So during Steve Jobs tenure? Can I conclude - it got eliminated from his biography on secrecy grounds?
It'd be more accurate to say that biographers would not have heard anything (from Steve or any other in-the-know Apple employees) about any projects that hadn't already been unveiled by Apple.
Additionally, it seems likely that biographers would've been required to not ask about possible future projects hinted at by public resources like patents: https://www.patentlyapple.com/2023/08/apple-won-a-patent-tod...
Salary no doubt, but I’m not aware that a team is operating at a million+ a day. Red Bulls don’t cost that much, and I doubt any of them are taking yoga classes during their shift lol
Shots fired but you’re not wrong. Wild how that doesn’t get more attention.
These numbers seem right to me.
In the age of ChatGPT, Siri not only hasn’t been improved but has been getting even dumber lately with no significant announcements towards its improvements of understanding and doing more tasks in any WWDC recently. I would take what he does with a grain of salt.
I cobbled together my own smart home voice assistant on a weekend a few weeks ago, sitting on top of the OpenAI APIs (Whisper, GPT-4), and of course using porcupine for wake word detection.
It can do things I could never get the commercial products to do properly, for example I gave it a memory: When a user command comes in, I have GPT-4 evaluate whether it can be executed immediately or requires later follow-up. When a sensor event happens, the machinery re-prompts GPT-4 with the user command backlog, the sensor backlog and the current state, and it figures things out. That way, things like "Please turn of the lights after I leave the room" now work just fine, and all it takes is an afternoon of hacking and a PIR sensor on my little DIY Homebrew-lexa wood boxes. And of course it's also much better at interpreting natural language commands "in spirit" or "creatively".
I'm sure Amazon, Google & Apple have made all of these tinkering experiments, too, but deploying LLM-backed voice services to tens of millions just isn't affordable yet, especially when you factor in risk and liability.
The boards are raised off the wood surfaces by PCB spacers (I embedded M2.5 threaded sockets into the wood) and I bought speaker grilles that bulge out a little at edge of the cylinder, so that the 4-mic array would remain fully exposed also laterally. I covered the side speakers and the mic array speaker grille with acoustic textile.
The onboard code is written in very pedestrian Python and uses porcupine and the OpenAI APIs.
It roughly works like this:
1. Capture audio frames and run overlapping frames by porcupine to perform hot word detection (overlapping avoids the problem of the hotword falling inbetween frames, at a cost to latency)
2. Once the hot word has been detected, buffer all audio frames into a command buffer until silence is detected as a stop (detecting "silence" is a bit involved, taking noise levels into account, and a few other tricks, more below)
3. The command buffer is sent to Whisper for transcription
4. GPT-4 is prompted with a system message steering it's behavior, the user command transcription and a JSON print out of the state of all devices (e.g. lights and Sonos speakers) in the home, grouped by rooms
5. Following the system message, GPT-4 replies with a JSON structure of changes it would like to make to the device state, omitting unchanged bits from the original
6. Add the sensor event and memory system described above
There's a few other tricks. To improve the audio capture, I take note of spatially where the hot word is detected (i.e. which mic in the array gets the best signal) and then capture the rest & perform the silence detection with a corresponding bias.
This is actually done in a distributed fashion over the network, so if two of the AI speakers hear the same command, only one of them will end up processing it.
They end up making mainly HTTP calls to APIs that already exist around my house. I have a second RasPi in my LED shelf (another old project, https://github.com/eikehein/hyelicht/) that doubles as a Philips Hue bridge with a zigbee dongle. That's what the DIY AI speakers interact with when making changes to the lighting.
I will say: Depending on the user command and the weather in the cloud, it's pretty slow. I've tried my best to optimize the client side for perceived user latency, but there's no way around the GPT-4 API just being pretty slow, even if it's amazingly low-friction and reliable otherwise. And 3.5-turbo just doesn't cut it for what I'm trying to do.
I'd like to get all of this out of the cloud entirely. I predict the next generation of my home NAS will have a GPU in it and try to run things like fine-tuned llama2 for the home.
Also I hope you consider posting more about your home setup as I’d love to see more.
Huge models are running on stock laptops. There is no need to send it to cloud. They had no problem e.g. sound recognition (reacting to alarms, cough, cry etc.) running only on selected devices. IPads have M1/M2 chips. And home assistant model does not need detailed understanding of neuroscience, best haskell patterns etc.
But all transformers development is pretty fresh corpo-wise. I think having a good safe dataset, which is not infringing any copyrights etc. is really hard. And they probably have to be very careful about it since it's not "only" about getting sued, but also potentially damaging partnerships they need for tv/books/music.
Btw have you tried some locally running models instead of GPT-4 for your automation? I don't want my HA touching the Internet unless necessary for 3rd party integrations but GPT-4 sets bar pretty high.
Not yet, but the desire is there, especially because the generic GPT-4 API is quite slow in responding to more complex prompts, not to mention the privacy concerns. I think the next version of my home NAS is likely to have a GPU in it and will run things like an appropriately tuned llama2 or similar to be the brain backbone of my smart home. Feels like an obvious direction for commercial NAS to go in as well.
Can't wait for a future where we can buy a Rasberry Pi 7 with a little analog compute ASIC running local inference with ease ... intelligent controllers everywhere!
You make a very valid point about economics. My naive point of view is: one would think a cash cow like Apple could afford it even to a limited extent, but then again iCloud free tier is still restricted to just 5GB so they have never been too generous with their cloud offerings.
But Bard is actually pretty good.
I still can't even tell it to turn the bedroom and living room lights on in one go. Bite-sized scripted chunks only. And half of the time it thinks I want to play some stupid song. I don't have my homepods for music, I even tried to remove all songs from my account but that free U2 album keeps coming back.
The only reason I use Siri is that it's the only one where I can turn off recording my voice.
So why race when you are already at the finish line? This is Apple's case and anyone who owns the hardware that they are making or creating and releasing $0 free AI models out there.
This whole article and its comments is a great big storm in a tea cup.
I have no problem with searching pictures using keywords if it’s limited to the Photos app.
This sounds like non-news to me. Maybe a fluff piece for Apple?
I guess 'millions of dollars per day' gets more clicks than 'apple spends less than .25% of its revenue on AI'.
- size of apple
- size of apple r&d
- size of near future opportunity and value that might be gained
"Millions" just means AI is more than 3% of R&D.
AI is not a democratising force, it is a capital concentrator.
They're making $350 billions in yearly revenues. They're not taking AI seriously.
Must be a fair few for the amount of money Apple makes on every new release.
For example, imagine if self-driving was everything that was ever promised. Sales of cars without it would plummet.
A good chunk of the S&P can’t compete with that, let alone startups.
“Wah Siri sucks” “wahh 5 years”
The iPhone was in development for like 8 years before it saw the light of day and everyone shat all over it and it still redefined mobile phones. Apples Silicon was in development for like 15 years and now laptop makers don’t even have a leg to stand on.
Apple isn’t going to release a half cocked ai project to the world. Apple isn’t Google who hasn’t released anything of note in like 10+ years. They aren’t Meta pouring billions into a metaverse no one wants.
Sorry what again? What is Siri then? That's even less than a half cooked product
While I agree it hasn’t improved over the years. Doesn’t change The fact it was good when it came out.
At least Google kills something when they lose interest.
Unlike the majority of so-called ChatGPT-wrapper and VC fuelled LLM companies burning millions a day on training and scrambling to compete with $0 on-device AI models.
Apple does not need to compete with OpenAI (Although Apple has the hardware and software to do so) and can do so at any time. Google is the one that needs to. So all of the "But but but Siri", "Siri is garbage", "Apple isn't doing anything with Siri", etc isn't even the point.
On-device LLMs or AI is the thing that matters to Apple and Siri will likely transition to a hybrid offline / online system eventually. Apple is in no rush to compete.