In this post, we show how to run the latest frontier-class model — Muse Spark Open, with nearly a trillion parameters — right from your desk, using the latest version of MLX, released today. On a cluster of four Mac Studios, Muse Spark Open generates over 36 tokens per second (see Figure 1), well above average human reading speed. Beginning on a single Mac Studio (up to 512GB of unified memory), then scaling out across multiple Macs with the RDMA over Thunderbolt driver, below we share an overview of the range of models you can now run on Mac Studio.
I find it bizarre of Apple to do anything publicly in partnership with Meta. I always thought the reasons Apple snubbed Facebook were out of values and principles not business. That could be wrong. Now that Meta has a line of business that Apple can finally charge a platform tax on there is new reason to start sleeping with them.
> I always thought the reasons Apple snubbed Facebook were out of values and principles not business.
Apple is showing a masterclass of how to tow the line perfectly. Or more candily "have the cake and eat it too".
If you ask anyone, what's one company that'll stand up to government requests, most reasonable people would say Apple. Yet, Apple is extremely strategic when it comes to China. It thrives in that country, without being associated with CCP.
Same here - they are happy to partner with Meta, without getting any of the bad "aura".
To be clear, all companies do this just that Apple alone has a successful strategy here. Tim Cook personally gifted the current adminstration with a gold plaque, something none of the other "dirty" CEOs did. Such a public gesture, yet nobody would think that Apple is in bed with the current administration. Its marvelous to see this happen in real time.
> If you ask anyone, what's one company that'll stand up to government requests, most reasonable people would say Apple. Yet, Apple is extremely strategic when it comes to China. It thrives in that country, without being associated with CCP.
You can stand up to government requests when it is legal to do so. Eventually you have to decide to abide by the law or leave the market.
Tim Cook, smiled. with OpenAI and Google a smile is easy when it isn’t costing you anything on the Apple side so John can smile away in addition, the AI models are getting smaller and more powerful by the day something else to smile about if you’re Apple.
I think they're sandbagging Qwen pretty heavily by benchmarking Meta's model at 4-bit, against Qwen only at native BF16 (and nothing else shown).
I get that this is cross-brand marketing for Apple's partner, Meta; but really Muse Spark isn't even in the top *ten* models I'd consider if I had a 512 GB M5 in my paws. Or a cluster of them.
Partner!? I don't see where you get that from the article, and Apple & Facebook have traditionally not been best of friends (see everything iOS implemented to limit Facebook's overreach).
I'm guessing they chose Muse because 1. It's not Chinese so it'll look good for Trump's AI cronies (Apple is in the administration's spotlight) and 2. It's fairly recent and thus on people's mind.
I mean that they're doing joint marketing on this particular thing. Muse Spark marketing features Mac Studios quite prominently, and Apple's material spotlights Muse Spark.
Eh, Apple spent years suing Samsung while also buying parts from them for their products. You can be a partner and an enemy at the same time at the high levels of the game.
Most of the following tables show relative speedups; not tok/s numbers.
I agree no home AI users should really use bf16^; but I see two defensible justifications: (1) showing that their new hardware's speedups scale to full precision too [crucially; NVIDIA is focusing more and more on lower quants], and (2) trying to sell to the crowd running IQ3XXS or something on 16GB.
^q8 ~= bf16 is a good general rule of thumb. And friends don't let friends quantize KV cache.
For what it’s worth, Muse Glimmer is by far the most usable model on my 32GB M1 Max. It gets stuff done very competently while Qwen never gets done reasoning.
If used correctly, I find the em dash one of the most elegant and reader-friendly ways to structure certain parts of a text — that it got such a bad reputation lately really is a shame!
> In this post, we will show how to run the latest frontier-class model - Muse Spark Open, with nearly a trillion parameters - right from your desk, using the latest version of MLX, released today.
You will get
> In this post, we will show how to run the latest frontier-class model – Muse Spark Open, with nearly a trillion parameters – right from your desk, using the latest version of MLX, released today.
Not everything with em dashes is the product of generative AI... some of it is the word processor correctly following the rules of grammar and punctuation.
So, Meta will spend millions/billions on building open-weight models and then people will send Apple $20,000 to run them. Seems like a good deal for Apple.
idk but naive me sees the article say the model needed 158GB of RAM so I guess 10x 16GB GPUs? Apparently retail price is $1k per 5080 16GB but they're actually selling at $1.5k so I guess $15k for just the GPUs.
Google, who is far more qualified, and is getting a measly 1 billion dollar per year refund on that $20 billion dollars they currently paying Apple for a default search position on Mac OS. Meta, whose software prowess is questionable, when compared to Google or Microsoft isn’t even in the game they are getting how should I put this Apples charity/tolerance which is all OpenAI and Google are currently getting.
One is getting a $1 billion refund from what they already pay Apple, and the other two are getting nothing from Apple.
Meta won’t be getting anything from Apple in the future, other than tolerance/charity.
I think the bet is that it will get bigger, both at the consumer level, and with orgs eventually discovering ready-made solutions for having their own on-prem inference.
100,000 units a year would be $2 billion annually.
And usually, most of a company's revenue comes from the heaviest spenders.
So figure there are some out there who will by tens of these because they have some "idea to change the world". (Whether true or not.) Some startups, etc.
I'd wager, across planet earth right now, it should be trivial to push 100,000 units per year. Probably closer to 500,000 units per year. So that's an easy $10 billion a year.
$2-10 billion is meaningful for any company. Even one as large is Apple.
Especially if it's a first step towards a stronger foothold in the AI inference space.
They can then expand down market to grow the pie further.
For example, with truly dedicated inference boxes.
And given the demand for $7,000 Sparks and sold out $5,000 GPUs, the market for these $20,000 machines is likely much higher than my 100,000 unit lower bound. (Earth is a big place. It doesn't take much excitement to generate massive sales in a space with currently insatiable demand.)
The only question you need to ask is whether or not Apple will do anything to design and engineer memory in-house, the money, and the capability is there that is the challenge. because you don’t need to select $20,000 computer if you get that under control by bringing it in-house.
In short, Apple needs to roll up there sleeves and get busy the three hyenas of memory and that nameless memory company in China will not sell you memory when you need it at a reasonable price. In the past Motorola said no, IBM said no, and Intel said no. This is another one of those times for Apple to adapt by designing and engineering around the competition.
The memory constraint isn’t design. It’s fab capacity. Apple has exactly 0 fabs. By the time they hired domain experts and built fabs and tuned to the point of reasonable yields, the crunch would have been over for 5 years, and apple would have wasted 20 billion with no return.
And the Mac Studio can continue to be used as a regular computer. (Hilariously over-qualified, I suppose. But, no, I got a maxed out, 128 GB Mac Mini earlier in the year—when you still could—and I find it a snappy daily driver in my 'man cave' downstairs.)
Apple playing 3d chess here -- they can just absolutely kill it in making the HW available for everyone to run models at home, and will do a better job than Nvidia especially when M7 comes out which is specifically designed for AI workloads.
While right now Mac Studio is not a budget device and it certainly costs more than the subsidised subscription, the promise that one day we can run these models on commodity hardware is exciting.
Muse is significantly faster and more token efficient. That is pretty impressive considering how much larger it is. There is a lot of potential even though this release won't shake up the standings.
It's a good article which showcases direction of travel, i am sure mac studio will get more and more powerful in coming generations but for now it's way to expensive for local setup.
FWIW, single M5 Ultra with 256GiB runs DeepSeek 4.1 Flash 0910 just fine (~3200 tok/s prefill, ~45 tok/s). M5 Max w/ 128GiB runs GLM 5.3 Flash also fine.
I also benchmarked both on quality benchmarks (MathArena Apex / GPQA Diamond etc) to make sure the quality is what you get from the official numbers.
I wouldn’t know since I haven’t found a model to test it on, but in my experience with smaller MoE models (35b A3b), they are the only choice for my M3 Max because prefill times are reasonable (I don’t even need to quant my KV cache, which would mess up MTP anyways). The only reason dense models (3.8 27b) are unusable for me are the high prefill times (mitigated if prefix caching is working correctly).
> peak power consumption is about 200W for the Mac Studio with M3 Ultra, and 300W for the Mac Studio with M5 Ultra — a fraction of the power used by a common toaster.
Apple has too many arbitrary, pointless grudges for Ternus to take them all seriously.
Google partnership for Apple Intelligence, Meta partnership for local models, and allegedly NVLink Fusion support for M8 GPUs. Maybe the White House told him that it's time to start burying hatchets. I don't get the impression that Apple's prior AI strategy under Cook made any real progress.
Not pointless, all of that led to Apple Silicon, and you seem to be ignoring the fact that it’s not costing Apple any money. None of their hookups. This so-called cooperation is costing Apple a dime. But it is costing the other side billions of dollars.
Smile, the models are getting smaller and better, every generation of Apple Silicon is getting better in the opposite direction. The only problem is the cost of memory and that is within the design engineering ability in-house it is also within their financial capacity, so if I’m John or Tim, I too would wake up with a smile because I can fix the one thing that’s giving us trouble memory because we have the talent and the money to fix it in the next four years.
Apple Silicon copied AMD's mistake. RDNA and Apple Silicon are both the slowest GPU architecture for AI because they spent 6 years optimizing raster performance over compute. Now there's 4 generations of Apple Silicon Macs that rely on compute shaders for any kind of efficient inference. Even the M5 GPU is less programmable than Nvidia's ARM GPUs from 2017.
ARM datacenters were a trillion-dollar market that Apple made ~$0 off of. The Mac Pro could have fit that niche, but Tim Cook chose to depreciate it instead of letting the PCIe connectors be used with Nvidia cards. He cut off Apple's nose to spite Nvidia's face. It's nonsense, especially since they're crawling back to Nvidia to license their interconnect, in tacit recognition that it wasn't a real grudge in the first place. My guess is that Ternus doesn't have Tim's money allergy, maybe at the White House's request. In any case, the grudge is over, and Apple forfeit ARM HPC altogether.
> because I can fix the one thing that’s giving us trouble memory because we have the talent and the money to fix it in the next four years.
By all means, continue. Which X account or YouTube channel convinced you that Apple could replace SK Hynix in 4 years?
> The Mac Pro could have fit that niche, but Tim Cook chose to depreciate it instead of letting the PCIe connectors be used with Nvidia cards. He cut off Apple's nose to spite Nvidia's face.
Apple Silicon only has support for 16k pages via DART, rather than the 4k pages typically used by AMD/Nvidia. The requirement to coordinate a major change of drivers to understand 16k pages is why both PCIe GPU and eGPU do not work (or rather, did not work - TinyGPU is a compatibility layer that adds eGPU support and is used for AI workflows).
The Mac Pro's biggest redeeming feature for a data center was that it was rack-mountable. The ARM version was still mostly an empty case. It remained mostly for its ability to support PCIe cards for video and audio production use cases, such as live broadcasting.
> especially since they're crawling back to Nvidia to license their interconnect
Licensing the interconnect for broader compatibility. They support other interconnects as well.
They went with several open weight models. Presumably they did that because they wanted their research to be reproducible (once the 512 GB M5 Ultra actually hits the market). Using say a Gemini model they have access to only due to their partnership with Google would not be as useful for people to understand what Apple did.
Agent swarms have figured out how to mostly work around context window limitations. There's no reason the same ideas can't be applied on a smaller scale, at least in principle.
To be clear, at 0.75 words per token, that's about ~6,100 words (roughly ~12 pages), which I suppose could be useful, but as other commenters have pointed out, you'd need some kind of orchestrated swarm/hive that could smartly fit payload to within that envelope
Am I right that investing in your own ML hardware only makes sense when you are dealing with private data and confidential sessions, right? (Not counting development or tinkering work).
If you think of it as API access vs temporary hosting running the same open model, it makes sense to run the hardware yourself whenever you are sure it can be fully utilized.
90 comments
[ 1.0 ms ] story [ 11.2 ms ] threadThe datacenters are not full of Mac studios but Nvidia hardware.
They are known for macbooks.
And they are not willing to build proper apple servers for all the build systems people have.
Apple is showing a masterclass of how to tow the line perfectly. Or more candily "have the cake and eat it too".
If you ask anyone, what's one company that'll stand up to government requests, most reasonable people would say Apple. Yet, Apple is extremely strategic when it comes to China. It thrives in that country, without being associated with CCP.
Same here - they are happy to partner with Meta, without getting any of the bad "aura".
To be clear, all companies do this just that Apple alone has a successful strategy here. Tim Cook personally gifted the current adminstration with a gold plaque, something none of the other "dirty" CEOs did. Such a public gesture, yet nobody would think that Apple is in bed with the current administration. Its marvelous to see this happen in real time.
You can stand up to government requests when it is legal to do so. Eventually you have to decide to abide by the law or leave the market.
The title (“Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio”) doesn’t even mention Muse.
It’s more like an advertorial for their tech.
...and Zuck and Ternus high-fiving each other here: https://x.com/finkd/status/2110778025629347999
"We're excited for mommy and daddy to be playing nice again," Meta AI chief Alexandr Wang told Axios here: https://www.axios.com/2026/10/15/apple-and-meta-bury-the-hat...
I get that this is cross-brand marketing for Apple's partner, Meta; but really Muse Spark isn't even in the top *ten* models I'd consider if I had a 512 GB M5 in my paws. Or a cluster of them.
Partner!? I don't see where you get that from the article, and Apple & Facebook have traditionally not been best of friends (see everything iOS implemented to limit Facebook's overreach).
I'm guessing they chose Muse because 1. It's not Chinese so it'll look good for Trump's AI cronies (Apple is in the administration's spotlight) and 2. It's fairly recent and thus on people's mind.
Most of the following tables show relative speedups; not tok/s numbers.
I agree no home AI users should really use bf16^; but I see two defensible justifications: (1) showing that their new hardware's speedups scale to full precision too [crucially; NVIDIA is focusing more and more on lower quants], and (2) trying to sell to the crowd running IQ3XXS or something on 16GB.
^q8 ~= bf16 is a good general rule of thumb. And friends don't let friends quantize KV cache.
> In this post, we will show how to run the latest frontier-class model - Muse Spark Open, with nearly a trillion parameters - right from your desk, using the latest version of MLX, released today.
You will get
> In this post, we will show how to run the latest frontier-class model – Muse Spark Open, with nearly a trillion parameters – right from your desk, using the latest version of MLX, released today.
Not everything with em dashes is the product of generative AI... some of it is the word processor correctly following the rules of grammar and punctuation.
As open weight models continue to match frontier models, the race to $0 will continue to benefit supporters of running open weight models locally.
One is getting a $1 billion refund from what they already pay Apple, and the other two are getting nothing from Apple.
Meta won’t be getting anything from Apple in the future, other than tolerance/charity.
And usually, most of a company's revenue comes from the heaviest spenders.
So figure there are some out there who will by tens of these because they have some "idea to change the world". (Whether true or not.) Some startups, etc.
I'd wager, across planet earth right now, it should be trivial to push 100,000 units per year. Probably closer to 500,000 units per year. So that's an easy $10 billion a year.
$2-10 billion is meaningful for any company. Even one as large is Apple.
Especially if it's a first step towards a stronger foothold in the AI inference space.
They can then expand down market to grow the pie further.
For example, with truly dedicated inference boxes.
And given the demand for $7,000 Sparks and sold out $5,000 GPUs, the market for these $20,000 machines is likely much higher than my 100,000 unit lower bound. (Earth is a big place. It doesn't take much excitement to generate massive sales in a space with currently insatiable demand.)
In short, Apple needs to roll up there sleeves and get busy the three hyenas of memory and that nameless memory company in China will not sell you memory when you need it at a reasonable price. In the past Motorola said no, IBM said no, and Intel said no. This is another one of those times for Apple to adapt by designing and engineering around the competition.
While right now Mac Studio is not a budget device and it certainly costs more than the subsidised subscription, the promise that one day we can run these models on commodity hardware is exciting.
This is still a welcome model, even if I won't be running it at home
Local LLM's really sharpen your senses to model size. You could fit a car in the price difference!
I also benchmarked both on quality benchmarks (MathArena Apex / GPQA Diamond etc) to make sure the quality is what you get from the official numbers.
I'm no expert but it seems the consensus is you get a suboptimal experience under 4-bit. Please correct me if I'm mistaken here
(Emphasis added.)
Google partnership for Apple Intelligence, Meta partnership for local models, and allegedly NVLink Fusion support for M8 GPUs. Maybe the White House told him that it's time to start burying hatchets. I don't get the impression that Apple's prior AI strategy under Cook made any real progress.
openai and anthropic are too chaotic to bet on would be my guess
Smile, the models are getting smaller and better, every generation of Apple Silicon is getting better in the opposite direction. The only problem is the cost of memory and that is within the design engineering ability in-house it is also within their financial capacity, so if I’m John or Tim, I too would wake up with a smile because I can fix the one thing that’s giving us trouble memory because we have the talent and the money to fix it in the next four years.
ARM datacenters were a trillion-dollar market that Apple made ~$0 off of. The Mac Pro could have fit that niche, but Tim Cook chose to depreciate it instead of letting the PCIe connectors be used with Nvidia cards. He cut off Apple's nose to spite Nvidia's face. It's nonsense, especially since they're crawling back to Nvidia to license their interconnect, in tacit recognition that it wasn't a real grudge in the first place. My guess is that Ternus doesn't have Tim's money allergy, maybe at the White House's request. In any case, the grudge is over, and Apple forfeit ARM HPC altogether.
> because I can fix the one thing that’s giving us trouble memory because we have the talent and the money to fix it in the next four years.
By all means, continue. Which X account or YouTube channel convinced you that Apple could replace SK Hynix in 4 years?
Apple Silicon only has support for 16k pages via DART, rather than the 4k pages typically used by AMD/Nvidia. The requirement to coordinate a major change of drivers to understand 16k pages is why both PCIe GPU and eGPU do not work (or rather, did not work - TinyGPU is a compatibility layer that adds eGPU support and is used for AI workflows).
The Mac Pro's biggest redeeming feature for a data center was that it was rack-mountable. The ARM version was still mostly an empty case. It remained mostly for its ability to support PCIe cards for video and audio production use cases, such as live broadcasting.
> especially since they're crawling back to Nvidia to license their interconnect
Licensing the interconnect for broader compatibility. They support other interconnects as well.
Has anyone adapted the open Muse model to work like a decision model?
Sort of like Laya but based on muse spark