I’ve not kept up with Intel in a while, but one thing that stood out to me is these are all E cores— meaning no hyperthreading. Is something like this competitive, or preferred, in certain applications? Also does anyone know if there have been any benchmarks against AMDs 192 core Epyc CPU?
I've seen scenarios where HT doesn't help, iirc very CPU-heavy things without much waiting on memory access. Which makes sense because the vcores are sharing the ALU.
Also have seen it disabled in academic settings where they want consistent performance when benchmarking stuff.
These sorts of core-density increases are how I win cloud debates in an org.
* Identify the workloads that haven't scaled in a year. Your ERPs, your HRIS, your dev/stage/test environments, DBs, Microsoft estate, core infrastructure, etc. (EDIT, from zbentley: also identify any cross-system processing where data will transfer from the cloud back to your private estate to be excluded, so you don't get murdered with egress charges)
* Run the cost analysis of reserved instances in AWS/Azure/GCP for those workloads over three years
* Do the same for one of these high-core "pizza boxes", but amortized over seven years
* Realize the savings to be had moving "fixed infra" back on-premises or into a colo versus sticking with a public cloud provider
Seriously, what took a full rack or two of 2U dual-socket servers just a decade ago can be replaced with three 2U boxes with full HA/clustering. It's insane.
Back in the late '10s, I made a case to my org at the time that a global hypervisor hardware refresh and accompanying VMware licenses would have an ROI of 2.5yrs versus comparable AWS infrastructure, even assuming a 50% YoY rate of license inflation (this was pre-Broadcom; nowadays, I'd be eyeballing Nutanix, Virtuozzo, Apache Cloudstack, or yes, even Proxmox, assuming we weren't already a Microsoft shop w/ Hyper-V) - and give us an additional 20% headroom to boot. The only thing giving me pause on that argument today is the current RAM/NAND shortage, but even that's (hopefully) temporary - and doesn't hurt the orgs who built around a longer timeline with the option for an additional support runway (like the three-year extended support contracts available through VARs).
If we can't bill a customer for it, and it's not scaling regularly, then it shouldn't be in the public cloud. That's my take, anyway. It sucks the wind from the sails of folks gung-ho on the "fringe benefits" of public cloud spend (box seats, junkets, conference tickets, etc...), but the finance teams tend to love such clear numbers.
It seems a lot of people have forgotten how BigCorp IT used to work.
- request some HW to run $service
- the "IT dept" (really, self-interested gatekeeper) might give you something now, or in two weeks, or god help you if they need to order new hardware then its in two months, best case
- there will be various weird rules on how the on-prem HW is run, who has access etc, hindering developer productivity even further
- the hardware might get insanely oversubscribed so your service gets half a cpu core with 1GB RAM, because perverse incentives mean the "IT dept" gets rewarded for minimizing cost, while the price is paid by someone else
- and so on...
The cloud is a way around this political minefield.
>Realize the savings to be had moving "fixed infra" back on-premises or into a colo versus sticking with a public cloud provider
As other people have pointed out: what happens when the PSU or mobo shits itself, what happens when the new version of vmware or docker (or whatever) shits itself, etc
I don't post much on HN, but this topic is near and dear to my heart, so here we go.
Context: been helpdesk, sysadmin & network admin, DevOps, Site Reliability Engineer, in that progression, starting in the 90's. Max on-prem was 40 racks, scaled up and down over years.
Many comments talking about how staffing is a key element of this equation that can't be overlooked, but I decided to reply to the root comment, which doesn't say whether/how it considers staffing.
This is a complex equation - and it is relatively easy to present an incomplete or misleading picture management to push the move into the cloud.. or out of the cloud.
Some factors, in no particular order:
1) Scaling: it is self-evident that pulling a single physical server worth out of the cloud is not worth it.. even for 288 cores. Or perhaps 1152 for 4xXeon in a single server. Still likely not worth it. Why? Because a single server is never just that. Someone has to swap components when it goes down. When it goes down.. ALL 1152 cores are down, along with everything they are doing. Is that acceptable for all applications running on all those cores? It is also appropriate supporting infrastructure - power, cooling, physical space. The "fairly obvious" minimum scaling is "enough servers that one can be entirely down for maintenance while keeping everything else running." But now you're paying for some overhead. At 2 servers, you're buying 2x what you need, half that capacity is idle all the time. And so on.
On this point - I think the other comments talking about "each SRE managing 4 (or 5, or 7)" racks missed the point entirely. SRE's should be doing scalable work, whether in the cloud, or on-prem. And they should NOT be swapping failed hard drives and power supplies. Designing a larger-than-one rack install is probably worth hiring consultants for if you don't have that expertise in-house, though the SREs that would be supporting it would need to supply lots of input. To some extent, server & network equipment vendors can also help. It is not trivial as the scale goes up. But then it should run for some years, with relatively unskilled people handling hardware failures and you can re-engage consultants if necessary to do upgrades as hardware and needs evolve.
But your SREs should be on-staff, and probably on-call to handle the software running on that hardware.. and to some extent to call the remote hands to deal with hardware failures.
2) Business needs: does the business need the tech skills that self-hosting requires for the core business? For example - if the business itself is cloud SAAS, maybe DIY-ing at least some of your infrastructure is right in your wheelhouse. If so - a modest increase in staff could mean a huge cost savings. But if not, all the cost of skilled staff to run it is simply part of the cost of in-housing this stuff.
3) Staffing: the people that swap broken hardware are not the same people that respond to pages because the business critical application crashed due to a bug. You can pay a colo facility for all this, typically by the hour - but it isn't cheap and you've got to supply all the spares etc. Is that part of your budget for on-prem?
4) On-call: maybe your self-hosted ERP system can be down every night and weekend without issues.. and even business hours can tolerate 98% uptime. But that doesn't mean you can get away without having someone on call - presumably you're hosting more than just this lowish-requirement ERP system. I'll disagree with other comments - the "no burnout" number of on-call staff you need is 6-7, not 4! Remember people take vacations too. This is well studied, and established, I'll reference Tom Limoncelli's books. This could be relatively cheap and require fewer staff with geo-distributed staff, and it would tend to overlap with staff you already use to provide on-call for anything you host on the cloud - so maybe for your situation it is close to a ...
Core density plus power makes so many things worthwhile. Generally human cost of managing hardware scales with number of components under management. CPUs very reliable. So once you get lots of CPU and RAM on single machine you can run with very few.
But right pricing hardware is hard if you’re small shop. My mind is hard-locked onto Epyc processors without thought. 9755 on eBay is cheap as balls. Infinity cores!
Problem with hardware is lead time etc. cloud can spin up immediately. Great for experimentation. Organizationally useful. If your teams have to go through IT to provision machine and IT have to go through finance so that spend is reliable, everybody slows down too much. You can’t just spin up next product.
But if you’re small shop having some Kubernetes on rack is maybe $15k one time and $1.2k on going per month. Very cheap and you get lots and lots of compute!
Previously skillset was required. These days you plug Ethernet port, turn on Claude Code dangerously skip permissions “write a bash script that is idempotent that configures my Mikrotik CCR, it’s on IP $x on interface $y”. Hotspot on. Cold air blowing on face from overhead coolers. 5 minutes later run script without looking. Everything comes up.
Still, foolish to do on prem by default perhaps (now that I think about it): if you have cloud egress you’re dead, compliance story requires interconnect to be well designed. More complicated than just basics. You need to know a little before it makes sense.
Feel like reasoning LLM. I now have opposite position.
> Previously skillset was required. These days you plug Ethernet port, turn on Claude Code dangerously skip permissions “write a bash script that is idempotent that configures my Mikrotik CCR, it’s on IP $x on interface $y”. Hotspot on. Cold air blowing on face from overhead coolers. 5 minutes later run script without looking. Everything comes up.
Last time I tried to do anything networking with Claude it set up route preference in opposite order (it thought lower number means more preferred, while it was opposite), fucking it up completely, and then invented config commands that do not exist in BIRD (routing software suite).
Then I looked at 2 different AIs and they both hallucinated same BIRD config commands that were nonexistent. And by same I mean they hallucinated existence of same feature.
> If your teams have to go through IT to provision machine and IT have to go through finance so that spend is reliable, everybody slows down too much. You can’t just spin up next product.
The time of having to order a bunch of servers for new project is long over. We just spun k8s cluster for devs to self-service themselves and the prod clusters just have a bit of accounting shim so adding new namespace have to be assigned to a certain project so we can bill client for it.
Also you're allowed to use cloud services while you have on-prem infrastructure. You get best of both, with some cognition cost involved.
As a Yocto enthusiast, I am curious as to how much elapsed realtime would be needed for a clean Yocto build. Yocto is thread heavy, so with 288, it oughta be good.
My Yocto build times on a 32-core AMD are negligible, <2 minutes for a full distro, IIRC. I suspect higher core counts have diminishing returns, especially since most dev builds are heavily cached.
With packages like this (lots of cores, multi-chip packaging, lots of memory channels), the architecture is increasingly a small cluster on a package rather than a monolithic CPU.
I wonder whether the next bottleneck becomes software scheduling rather than silicon - OS/runtimes weren’t really designed with hundreds of cores and complex interconnect topologies in mind.
So, they're selling this as an AI accelerator, with drop in compatibility with existing boards, and no boost to RAM bandwidth.
As I understand things, it would be extremely unusual to ship a chip that was bound by floating point throughput, not uncached memory access, especially in the desktop/laptop space.
I haven't been following the Intel server space too carefully, so it's an honest question: Was the old thing compute and not bandwidth limited, or is this going to be running inference at the same throughput (though maybe with lower power consumption)?
Meanwhile, somebody put 8192 arm cores on a chip and ran a risc-v emulator on top of that which emulated a 6502 which then emulated a 288 core xeon and it used 0.01% of the power and outperformed the Intel chip in every other metric 10:1, probably.
Helped a friend make a difficult career decision (cozy job vs something hard and new + moving to a new city) that ultimately ended up with him working on the project. Glad that happened. I love to see people grow.
A bad moment to have a make-or-break moment for your CPU business - a lot of customers will probably hold off purchases right now because of the RAM prices, no matter how good your CPU might be.
I think everyone's focusing on the core count, but the packaging story is way more interesting here. This thing is 12 separate chiplets on 18A stacked on base dies made on Intel 3, connected to I/O tiles on Intel 7. Three different process nodes in one package, shipping at volume. That's nuts.
And it's clearly an IFS play too. Intel Foundry needs a proof point — you can publish PDKs all day, but nothing sells foundry credibility like eating your own cooking in a 288-core server part at 450W. If Foveros Direct works here, it's the best ad Intel could run for potential foundry customers.
The chiplet sizing is smart for another reason nobody's mentioned: yield. 18A is brand new, yields are probably rough. But 24 cores per die is small enough that even bad yields give you enough good chiplets. Basically AMD's Zen playbook but with a 3D twist.
Also — 64 CXL 2.0 lanes! Several comments here are complaining about DDR5 prices, which is fair. But CXL memory pooling across a rack could change that math completely. I wonder if Intel is betting the real value isn't the cores but being the best CXL hub in the datacenter.
The ARM competition is still the elephant in the room though. "Many efficient cores" is what ARM has always done natively, and 17% IPC uplift on Darkmont doesn't close that gap by itself.
Agree entirely with your take. The packaging story is awesome, I wish there were more details on the stacking used on this one.
But I am at a loss to how Intel are really going to get any traction with IFS. How can anyone trust Intel as a long-term foundry partner. Even if they priced it more aggressively, the opportunity cost in picking a supplier who decides to quit next year would be catastrophic for many. The only way this works is if they practically give their services away to someone big, who can afford to take that risk and can also make it worth Intel's continued investment. Any ideas who that would be, I've got nothing.
> From a cache hierarchy standpoint, the design groups cores into four-core blocks that share approximately 4 MB of L2 cache per block. As a result, the aggregate last-level cache across the full package surpasses 1 GB, roughly 1,152 MB in total.
If cores are grouped into four-core blocks, and each block has 4MB of cache… isn’t that just 1MB per core? So 288MB total?
> these processors pack in up to 288 of the little guys as well as 576MB of last-level cache, 96 PCIe 5.0 lanes, and 12-channel DDR5-8000.
> The Xeon 6+ processors each have up to 12 compute tiles fabbed on 18A, all of which have six quad-core modules for a total of 24 cores per tile. There are also three 'active' base tiles on Intel 3, so-called because the base tiles include 192MB of last-level cache, which is so-called because each compute tile has 48MB of L3 cache.
So maybe 1MB per core L2, then 192MB of basically-L4 per base tile, then 48MB of L3 per compute tile? 192*3+48*12 gets me to the 1152, maybe that’s it.
Anyway, apparently these things will have “AMX” matrix extensions. I wonder if they’ll be good number crunchers.
I honestly just want Intel to fail. I believe they have done more anticompetitive harm than good these past years. Datacenter needs to move to ARM so Intel can finally go home.
More Intel vaporware. Seriously, their other 18A product, panther lake, supposedly "launched" January 18th. It's been 1.5 months and I still can't go and buy any panther lake laptop except from dell.com. Why are they like this?
I'll believe it when I see it.
Also about "make-or-break": they've been saying this for all of Intel's products since at least 2022 *yawn*
37 comments
[ 3.4 ms ] story [ 59.6 ms ] threadAlso have seen it disabled in academic settings where they want consistent performance when benchmarking stuff.
* Identify the workloads that haven't scaled in a year. Your ERPs, your HRIS, your dev/stage/test environments, DBs, Microsoft estate, core infrastructure, etc. (EDIT, from zbentley: also identify any cross-system processing where data will transfer from the cloud back to your private estate to be excluded, so you don't get murdered with egress charges)
* Run the cost analysis of reserved instances in AWS/Azure/GCP for those workloads over three years
* Do the same for one of these high-core "pizza boxes", but amortized over seven years
* Realize the savings to be had moving "fixed infra" back on-premises or into a colo versus sticking with a public cloud provider
Seriously, what took a full rack or two of 2U dual-socket servers just a decade ago can be replaced with three 2U boxes with full HA/clustering. It's insane.
Back in the late '10s, I made a case to my org at the time that a global hypervisor hardware refresh and accompanying VMware licenses would have an ROI of 2.5yrs versus comparable AWS infrastructure, even assuming a 50% YoY rate of license inflation (this was pre-Broadcom; nowadays, I'd be eyeballing Nutanix, Virtuozzo, Apache Cloudstack, or yes, even Proxmox, assuming we weren't already a Microsoft shop w/ Hyper-V) - and give us an additional 20% headroom to boot. The only thing giving me pause on that argument today is the current RAM/NAND shortage, but even that's (hopefully) temporary - and doesn't hurt the orgs who built around a longer timeline with the option for an additional support runway (like the three-year extended support contracts available through VARs).
If we can't bill a customer for it, and it's not scaling regularly, then it shouldn't be in the public cloud. That's my take, anyway. It sucks the wind from the sails of folks gung-ho on the "fringe benefits" of public cloud spend (box seats, junkets, conference tickets, etc...), but the finance teams tend to love such clear numbers.
cloud = opex
The accounting dept will always win this debate.
- request some HW to run $service
- the "IT dept" (really, self-interested gatekeeper) might give you something now, or in two weeks, or god help you if they need to order new hardware then its in two months, best case
- there will be various weird rules on how the on-prem HW is run, who has access etc, hindering developer productivity even further
- the hardware might get insanely oversubscribed so your service gets half a cpu core with 1GB RAM, because perverse incentives mean the "IT dept" gets rewarded for minimizing cost, while the price is paid by someone else
- and so on...
The cloud is a way around this political minefield.
As other people have pointed out: what happens when the PSU or mobo shits itself, what happens when the new version of vmware or docker (or whatever) shits itself, etc
Context: been helpdesk, sysadmin & network admin, DevOps, Site Reliability Engineer, in that progression, starting in the 90's. Max on-prem was 40 racks, scaled up and down over years.
Many comments talking about how staffing is a key element of this equation that can't be overlooked, but I decided to reply to the root comment, which doesn't say whether/how it considers staffing.
This is a complex equation - and it is relatively easy to present an incomplete or misleading picture management to push the move into the cloud.. or out of the cloud.
Some factors, in no particular order:
1) Scaling: it is self-evident that pulling a single physical server worth out of the cloud is not worth it.. even for 288 cores. Or perhaps 1152 for 4xXeon in a single server. Still likely not worth it. Why? Because a single server is never just that. Someone has to swap components when it goes down. When it goes down.. ALL 1152 cores are down, along with everything they are doing. Is that acceptable for all applications running on all those cores? It is also appropriate supporting infrastructure - power, cooling, physical space. The "fairly obvious" minimum scaling is "enough servers that one can be entirely down for maintenance while keeping everything else running." But now you're paying for some overhead. At 2 servers, you're buying 2x what you need, half that capacity is idle all the time. And so on.
On this point - I think the other comments talking about "each SRE managing 4 (or 5, or 7)" racks missed the point entirely. SRE's should be doing scalable work, whether in the cloud, or on-prem. And they should NOT be swapping failed hard drives and power supplies. Designing a larger-than-one rack install is probably worth hiring consultants for if you don't have that expertise in-house, though the SREs that would be supporting it would need to supply lots of input. To some extent, server & network equipment vendors can also help. It is not trivial as the scale goes up. But then it should run for some years, with relatively unskilled people handling hardware failures and you can re-engage consultants if necessary to do upgrades as hardware and needs evolve.
But your SREs should be on-staff, and probably on-call to handle the software running on that hardware.. and to some extent to call the remote hands to deal with hardware failures.
2) Business needs: does the business need the tech skills that self-hosting requires for the core business? For example - if the business itself is cloud SAAS, maybe DIY-ing at least some of your infrastructure is right in your wheelhouse. If so - a modest increase in staff could mean a huge cost savings. But if not, all the cost of skilled staff to run it is simply part of the cost of in-housing this stuff.
3) Staffing: the people that swap broken hardware are not the same people that respond to pages because the business critical application crashed due to a bug. You can pay a colo facility for all this, typically by the hour - but it isn't cheap and you've got to supply all the spares etc. Is that part of your budget for on-prem?
4) On-call: maybe your self-hosted ERP system can be down every night and weekend without issues.. and even business hours can tolerate 98% uptime. But that doesn't mean you can get away without having someone on call - presumably you're hosting more than just this lowish-requirement ERP system. I'll disagree with other comments - the "no burnout" number of on-call staff you need is 6-7, not 4! Remember people take vacations too. This is well studied, and established, I'll reference Tom Limoncelli's books. This could be relatively cheap and require fewer staff with geo-distributed staff, and it would tend to overlap with staff you already use to provide on-call for anything you host on the cloud - so maybe for your situation it is close to a ...
But right pricing hardware is hard if you’re small shop. My mind is hard-locked onto Epyc processors without thought. 9755 on eBay is cheap as balls. Infinity cores!
Problem with hardware is lead time etc. cloud can spin up immediately. Great for experimentation. Organizationally useful. If your teams have to go through IT to provision machine and IT have to go through finance so that spend is reliable, everybody slows down too much. You can’t just spin up next product.
But if you’re small shop having some Kubernetes on rack is maybe $15k one time and $1.2k on going per month. Very cheap and you get lots and lots of compute!
Previously skillset was required. These days you plug Ethernet port, turn on Claude Code dangerously skip permissions “write a bash script that is idempotent that configures my Mikrotik CCR, it’s on IP $x on interface $y”. Hotspot on. Cold air blowing on face from overhead coolers. 5 minutes later run script without looking. Everything comes up.
Still, foolish to do on prem by default perhaps (now that I think about it): if you have cloud egress you’re dead, compliance story requires interconnect to be well designed. More complicated than just basics. You need to know a little before it makes sense.
Feel like reasoning LLM. I now have opposite position.
Last time I tried to do anything networking with Claude it set up route preference in opposite order (it thought lower number means more preferred, while it was opposite), fucking it up completely, and then invented config commands that do not exist in BIRD (routing software suite).
Then I looked at 2 different AIs and they both hallucinated same BIRD config commands that were nonexistent. And by same I mean they hallucinated existence of same feature.
> If your teams have to go through IT to provision machine and IT have to go through finance so that spend is reliable, everybody slows down too much. You can’t just spin up next product.
The time of having to order a bunch of servers for new project is long over. We just spun k8s cluster for devs to self-service themselves and the prod clusters just have a bit of accounting shim so adding new namespace have to be assigned to a certain project so we can bill client for it.
Also you're allowed to use cloud services while you have on-prem infrastructure. You get best of both, with some cognition cost involved.
What are the dimensions and dynamics here vs EPYC?
I wonder whether the next bottleneck becomes software scheduling rather than silicon - OS/runtimes weren’t really designed with hundreds of cores and complex interconnect topologies in mind.
So on the OS side we might already have the needed tools for these CoC (cluster on chip ;))
As I understand things, it would be extremely unusual to ship a chip that was bound by floating point throughput, not uncached memory access, especially in the desktop/laptop space.
I haven't been following the Intel server space too carefully, so it's an honest question: Was the old thing compute and not bandwidth limited, or is this going to be running inference at the same throughput (though maybe with lower power consumption)?
Because if this is not thunder Intel will default.
I promise you. Heard it from some youtuber as well, trust me.
And it's clearly an IFS play too. Intel Foundry needs a proof point — you can publish PDKs all day, but nothing sells foundry credibility like eating your own cooking in a 288-core server part at 450W. If Foveros Direct works here, it's the best ad Intel could run for potential foundry customers.
The chiplet sizing is smart for another reason nobody's mentioned: yield. 18A is brand new, yields are probably rough. But 24 cores per die is small enough that even bad yields give you enough good chiplets. Basically AMD's Zen playbook but with a 3D twist.
Also — 64 CXL 2.0 lanes! Several comments here are complaining about DDR5 prices, which is fair. But CXL memory pooling across a rack could change that math completely. I wonder if Intel is betting the real value isn't the cores but being the best CXL hub in the datacenter.
The ARM competition is still the elephant in the room though. "Many efficient cores" is what ARM has always done natively, and 17% IPC uplift on Darkmont doesn't close that gap by itself.
That the CPU cores are low frequency cores probably helps with yield as well.
But I am at a loss to how Intel are really going to get any traction with IFS. How can anyone trust Intel as a long-term foundry partner. Even if they priced it more aggressively, the opportunity cost in picking a supplier who decides to quit next year would be catastrophic for many. The only way this works is if they practically give their services away to someone big, who can afford to take that risk and can also make it worth Intel's continued investment. Any ideas who that would be, I've got nothing.
> From a cache hierarchy standpoint, the design groups cores into four-core blocks that share approximately 4 MB of L2 cache per block. As a result, the aggregate last-level cache across the full package surpasses 1 GB, roughly 1,152 MB in total.
If cores are grouped into four-core blocks, and each block has 4MB of cache… isn’t that just 1MB per core? So 288MB total?
HotHardware reports
https://hothardware.com/news/intel-clearwater-forest-xeon-6-...
> these processors pack in up to 288 of the little guys as well as 576MB of last-level cache, 96 PCIe 5.0 lanes, and 12-channel DDR5-8000.
> The Xeon 6+ processors each have up to 12 compute tiles fabbed on 18A, all of which have six quad-core modules for a total of 24 cores per tile. There are also three 'active' base tiles on Intel 3, so-called because the base tiles include 192MB of last-level cache, which is so-called because each compute tile has 48MB of L3 cache.
So maybe 1MB per core L2, then 192MB of basically-L4 per base tile, then 48MB of L3 per compute tile? 192*3+48*12 gets me to the 1152, maybe that’s it.
Anyway, apparently these things will have “AMX” matrix extensions. I wonder if they’ll be good number crunchers.
Also about "make-or-break": they've been saying this for all of Intel's products since at least 2022 *yawn*