Apple recently introduced rdma support in mac os. They are probably trying to push those people buying the 512gb configuration towards buying more of the 256gb configuration and clustering them together.
I'm trying to work out if I should buy a 48GB M4 Pro Mac Mini now, or wait for M5 Pro ones later this year. For AI/ML purposes, mostly. As far as I can tell, the new M5 MacBooks didn't go up much or any for the same amount of RAM?
They may be trying to sell through the existing CPU before a launch (soft or not) of the M5-based versions (though I've heard the rumor is there will be no M5 Ultra and we might be looking at an M6 Ultra later in the year).
I think it's unlikely that Apple is paying the spot price for memory. They almost certainly negotiate delivery/price contracts in advance. Maybe the contract for the chips used in the 512GB model will expire soon?
The story here is about what lesson was learned by the DRAM cartel after they got busted and hit with large fines. One might hope the lesson learned would be, "we should not fix prices", but what got them in trouble was colluding secretly. What if we just did it via earnings reports, press releases, and other public statements?
While there is some market variance like the 2022 to 2023 glut, DRAM prices haven't fallen in real terms in over 15 years. This was all done by controlling supply, and it was all done in public. It starts with one of the big three putting out a statement like, "Samsung is considering reducing DRAM wafer output due to softness in the mobile PC segment." The actual reason varies and often makes little sense.
This is followed by similar public statements from the other large vendors expressing a willingness to reduce supply. Once everyone commits in this way, the companies follow up with announcements of actual supply reductions. You can watch this happen any time prices start to dip.
My bet is if the DOJ investigates, they will not find the same sort of embarrassing smoking gun emails between representatives of Micron, Hynix, and Samsung. The collusion was all done in public. The companies will claim it is just good business management, a strategy known as "conscious parallelism." They used this exact defense to get a 2022 antitrust lawsuit dismissed.
That said, their goal seemed to be just keeping prices fixed. They wanted to avoid boom and bust cycles, keep profits high, and keep prices stagnant. A massive price hike invites investigations and creates problems. If DRAM prices just never fall, they can enjoy healthy profits with little risk.
But what happens when your intentionally constrained supply hits a sudden large spike in demand? Prices skyrocket, everyone gets mad, and demands investigations. My guess is instead of being thrilled with the price spike, the executives at the large DRAM manufacturers are very worried someone put something incriminating in a document somewhere that can be subpoenaed ("how we're going to fix prices in public and get away with it").
Now how am I supposed to develop Electron apps and use Chrome?
In all seriousness, though, as one of the uninitiated, what would be the value of hosting LLMs on a machine like this that has a lot of memory that you pay for up front versus some sort of VPC-based approach?
The rumor from Gurman is that the M5 Ultra Mac Studio ships in the first half of this year.
This may just be a sign that the M5 Ultra Mac Studio is shipping sooner rather than later, as it's common for Apple to push out ship dates for soon to be replaced products.
We do have leaked benchmarks showing that the M5 Max outperforms the M3 Ultra currently shipping in the Mac Studio, so buying an M3 Ultra Studio right now would be a terrible idea.
More likely that the M5 Max Studio is coming out. the M5 Max Macbook Pros just came out.
Also the 512 GB ssd version has a slower SSD than anything 1 TB and up. The new SSDs on the M5 I believe are much faster and what's coming likely will receive that.
There's no doubt there's a ram shortage, and price increases, and the biggest companies in the world lock in their pricing well in advance, and the remaining leftovers are where the consumers experience shortages.
IMO its more nuanced. They're likely in production ramp-up of the M5 Ultra Mac Studio, for release in the next ~3 months; they have pre-purchased bins of memory from the supply-constrained major memory supplies; and they need as much as they can get because they want to push an M5 Ultra config to 768gb to continue the "you can run local models" story that the M5 Max Macbook Pro started telling last week.
Going beyond 512gb and into 768gb memory is something of a threshold that will allow Apple to claim local capability for significantly more models. Qwen3-235B, Minimax M2.5, and GLM 4.7 could kind of run with no quantization on 512gb, but they'll comfortably run at 768gb. DeepSeek-V3.2 and GLM 5 may also work at some level of quantization.
31 comments
[ 2.2 ms ] story [ 68.0 ms ] threadAmazon is selling 128GB memory kits @ 5600MHZ for $3k.
I think there might be a market failure guys.
https://www.youtube.com/watch?v=jVzeHTlWIDY
While there is some market variance like the 2022 to 2023 glut, DRAM prices haven't fallen in real terms in over 15 years. This was all done by controlling supply, and it was all done in public. It starts with one of the big three putting out a statement like, "Samsung is considering reducing DRAM wafer output due to softness in the mobile PC segment." The actual reason varies and often makes little sense.
This is followed by similar public statements from the other large vendors expressing a willingness to reduce supply. Once everyone commits in this way, the companies follow up with announcements of actual supply reductions. You can watch this happen any time prices start to dip.
My bet is if the DOJ investigates, they will not find the same sort of embarrassing smoking gun emails between representatives of Micron, Hynix, and Samsung. The collusion was all done in public. The companies will claim it is just good business management, a strategy known as "conscious parallelism." They used this exact defense to get a 2022 antitrust lawsuit dismissed.
That said, their goal seemed to be just keeping prices fixed. They wanted to avoid boom and bust cycles, keep profits high, and keep prices stagnant. A massive price hike invites investigations and creates problems. If DRAM prices just never fall, they can enjoy healthy profits with little risk.
But what happens when your intentionally constrained supply hits a sudden large spike in demand? Prices skyrocket, everyone gets mad, and demands investigations. My guess is instead of being thrilled with the price spike, the executives at the large DRAM manufacturers are very worried someone put something incriminating in a document somewhere that can be subpoenaed ("how we're going to fix prices in public and get away with it").
In all seriousness, though, as one of the uninitiated, what would be the value of hosting LLMs on a machine like this that has a lot of memory that you pay for up front versus some sort of VPC-based approach?
This may just be a sign that the M5 Ultra Mac Studio is shipping sooner rather than later, as it's common for Apple to push out ship dates for soon to be replaced products.
We do have leaked benchmarks showing that the M5 Max outperforms the M3 Ultra currently shipping in the Mac Studio, so buying an M3 Ultra Studio right now would be a terrible idea.
Also the 512 GB ssd version has a slower SSD than anything 1 TB and up. The new SSDs on the M5 I believe are much faster and what's coming likely will receive that.
There's no doubt there's a ram shortage, and price increases, and the biggest companies in the world lock in their pricing well in advance, and the remaining leftovers are where the consumers experience shortages.
I guess it gives more credence to stitch a few more of them togther in the meantime.
Going beyond 512gb and into 768gb memory is something of a threshold that will allow Apple to claim local capability for significantly more models. Qwen3-235B, Minimax M2.5, and GLM 4.7 could kind of run with no quantization on 512gb, but they'll comfortably run at 768gb. DeepSeek-V3.2 and GLM 5 may also work at some level of quantization.