One big difference between CPUs and GPUs is memory bandwidth.
Dumb question, but why don't Intel/AMD massively increase memory bandwidth by adding more memory lanes? For example instead of the usual 4 slots on a desktop motherboard you would have 16 slots surrounding the CPU on all sides.
Is it that the CPU would be unable to effectively use all that bandwidth due to less parallelism compared with a GPU?
The standard answer is cost, but I'm not sure how significant that actually is in this day and age. Maybe it's just artificial market segmentation at this point.
There's nothing artificial or imaginary about the extra costs of using a CPU socket with a 512-bit memory bus rather than a 128-bit bus. Bigger sockets and more memory slots make motherboards significantly more expensive, and the wider memory controller and faster interconnects makes a CPU more expensive and power-hungry. Those costs affect the whole platform, including entry-level systems that don't have enough memory to populate all those channels and ones that don't run workloads which would benefit from the extra bandwidth.
- which are mostly portable and have a low thermal budget
- which are by most users hardly ever run on full throttle
- a well documented tendency to proclaim any hardware failure as user fault
makes this quite viable for them
but if you run systems with 4TiB of RAM and 12k+$ CPU which is most time running full throttle potential for days at a time with a higher thermal budged and customers which might have docents or even thousands of such CPUs and will be very pushy about reclaiming any failure they think isn't their fault then having to replace over 20-50k$ just isn't viable (instead of ~1-3k...)
It's very rare that software is able to use all the memory bandwidth on a CPU. Something well written with multiple cores is where you will see it happen and that's why higher core counts can often have more memory lanes.
I'd say that bandwidth matters less than latency for performance. Both are good, but this is part of the reason why we see ever-increasing cache sizes, and memory physically closer to the CPU.
Edit: Of course, it depends on the benchmark. Latency is critical for programs that access many memory addresses randomly, not for data processing applications that scan memory linearly (and GPU use-cases often fall into that category).
With CPUs the bottleneck isn't just memory bandwidth only but also memory latency. If prefetch fails to get the correct data loaded into the cache, the CPU just stalls for thousands of cycles waiting for RAM to deliver the data.
If I'm correct, this is the difference between DDR RAM and GDDR RAM - the latter has higher bandwidth but also higher latency, which is great for graphics with its high bandwidth requirements and very predictable memory access patterns (except when you do ray tracing) which allows you to hide the latency, but you don't want to use GDDR for system RAM if you value performance.
Here’s an unusual motherboard salvaged from Xbox Series X which combines AMD Zen2 CPU cores, GDDR6 memory, where the integrated GPU is disabled due to defects: https://www.tomshardware.com/news/4800s-xbox-chip-shows-us-w... The performance is not great.
GPUs are good at hiding the latency due to high degree of parallelism. GPU cores switch to another thread instead of waiting for the data. Each CPU core only runs 2 threads, it can’t quite do the same thing.
GPU memory access patterns are only great for well written compute shaders (or CUDA kernels) when the developer who wrote that shader managed to do fully coalesced loads and stores. Memory access patterns for pixel shaders are not that great, especially when doing trilinear or anisotropic sampling in these shaders.
Which is why I always found it funny to see consoles using unified GDDR for CPU and GPU.
But you can optimize around the limitation (within reason) and don't have to content with a general non-specified OS.
This is why I do wonder about the path not taken with the 1T-SRAM that powered the Gamecube/Wii. It was designed purely with low latency in mind but I don't think it scaled well.
> Is it that the CPU would be unable to effectively use all that bandwidth due to less parallelism compared with a GPU?
Yes, but also it's due to simple physical limitations. If you look at how close CPUs are to RAM as well as the number of PCB lines coming out of CPUs, it becomes pretty readily apparent that there's not much room for additional lines.
You have to recognize that each of those lines is creating a magnetic field, the closer you stack them, the more likely cross talk is.
We could continue to lower the voltage on the memory lanes so you can pack them closer together, but then you have to start worrying about environmental interference on the lines. It likely won't be too long in the future (if it's not already here) that we'll start seeing shielding for the memory lanes so they can be packed closer together and more can be added.
It wouldn't shock me if this became a part of DDR6.
It also wouldn't shock me to start seeing more complex memory controllers doing things like lz4 compression to eck out even higher bandwidths (particularly when the memory is highly uniform). We are sort of seeing the precursor of this in DDR5 with commands to set whole blocks to a given value.
I'm curious if anytime soon we'll start seeing optical interconnections between CPUs and on-device / same-chasis memory.
I'm not sure what that would add for latency and cost, but I'm guessing that with optical wave guides, crosstalk isn't a concern. At least for the optical part of the data path.
The speed of light in an optical fiber is still only about 2*10^8 m/s - ddr5 6000's 3ghz clock means it still only travels 6.7cm/clock - so there'll still be difficulties around latency if it's not "close", and that's ignoring any issues of actually getting optical transceivers on the die itself.
It blew my mind when I learned that even if you could hand wire conceptually perfect switches and wires, once your board was over a few inches, the laws of physics (e.g., the speed of light) would mean you still couldn’t beat a modern CPU for clock speed.
It really is insane the level at which current hardware works.
Yeah. Any time I wait for my computer to do something I think about this. In the 4 seconds discord took to open, my computer could do what, 5 billion operations per second per core? Even single threaded that’s 20 billion steps? And my internet connection could have delivered 600 million bytes of data - enough for a complete copy of windows xp. It’s wild.
They are coming yes. They may not bring much in term of bandwidth per line, but will allow much more lines and entirely different geometries for the cpus allowing chips to be cooled on both sides etc.
> You have to recognize that each of those lines is creating a magnetic field, the closer you stack them, the more likely cross talk is.
PC desktops have 2 memory channels. EPYC servers have 12 channels per socket. It's not the physics, it's that ~8-core processors generally aren't bounded by memory bandwidth and it's not worth the cost of increasing the number of pins.
The server sockets do it because they have 128-core processors.
If this changes what we may start seeing is CPUs (or iGPUs) with a few GB of HBM as L4 cache.
Intel is doing this today in the Xeon MAX line, up to 64gb of hbm ram, you don't even necessarily need ddr5 on them. serve the home has a good article about them but it's all very new so it's hard to say how it'll shake out.
because it makes it much harder and in turn more costly to the more channels you have to have high quality reliable connections
server RAM is for example also slower to compensate this e.g. recent EPYC processors run at up to 4800MT/s but a Ryzen 9 runs at up to 5200MT/s (without overclocking, with running at 6000MT/s isn't that uncommon)
> server RAM is for example also slower to compensate this e.g. recent EPYC processors run at up to 4800MT/s but a Ryzen 9 runs at up to 5200MT/s (without overclocking, with running at 6000MT/s isn't that uncommon)
Zen3 EPYC has 8 channels per socket and uses the same 3200MT/s DDR4 as Zen3 desktops, so this little explains why desktops couldn't have e.g. 4 channels if there was a need for it. They already commonly have 4 DIMM slots.
The rated speed for DDR5 is 8% higher for desktops, but the additional channels don't seem to be why. Intel uses 8 channels of DDR5 rather than 12 and it's still 4800MT/s.
It isn't so much the physical limits but more the cost of manufacturing a physically more complex board. This is why most low end GPU's typically have their memory bus width cut down to save on PCB and RAM complexity costs.
An extreme example, the phone I am using has RAM running at 1.8Ghz but on a 16bit bus. Anything to keep costs down.
Server sockets have more memory channels and server motherboards usally have more memory slots. Layout limitations probably make ram on four sides not very feasible.
But, more memory channels means more pins and larger sockets, and that adds cost and layout issues for motherboards. HBM ram that attaches on top of the cpu addresses the layout issue (and the bandwidth issue), but interferes with heat removal and removes expandability. Apple makes it work, but they limit clock speeds and were never big on expandability. Intel and AMD have a hard time in the market when their top consumer chips don't clock to the moon, even if the last 10% of clock speed takes 50% of the power and delivers 5% of performance (percentages made up). Server chips don't need to clock so high, so AMD can release clock limited compact cores that have about the same IPC in a bit more than half the area, and the difference in clocks for a max-cores regular zen4 and a max-cores zen4c isn't too much (L3/core is half for zen4c though, so may not be a benefit depending on your working set)
> but interferes with heat removal and removes expandability
Eh. RAM generally consumes a single digit number of watts and will do so regardless of where it's physically located in the system.
If you put HBM on the CPU socket then you have to replace them together, but you can still replace them. It could also make sense to do both: If you have 16GB of HBM and 16GB of DDR5, you have 32GB of RAM and half of it is faster. Or 80GB of RAM and 16GB of it is faster. Cache hierarchies are a good way to strike a balance between performance and cost, and then the latter can still be expanded independent of the CPU.
Two reasons: First, it's a lot of wiring which makes everything more expensive, complicated, power hungry, etc. Mobile SoCs (and Apple's chips) sidestep this issue by packaging the memory with the CPU so distances are short and the issues are avoided. Second, increasing memory bandwidth wouldn't be that much of a performance win because as you point out there isn't that much parallelism and normal software would benefit much more from lower latency than higher bandwidth. Together that means a many-slots design would cost a lot but not be proportionally faster.
Tradeoffs are different for server boards (many cores in the CPU) and on platforms with integrated GPUs (becoming more common) so none of this is static. I suspect in-package RAM will become more common even for platforms that still have a couple channels of slotted memory, that would create a pool of faster memory available for graphics etc while still leaving room for more if you need to have a lot of Chrome tabs open or whatever.
On a GPU, the memory access patterns are often very wide & easily controlled, such as fetching a block of texture memory. This is like streaming reads for a disk, where you can actually just throw essentially RAID 0 at the problem to scale up really easily. So scaling all the way up to ridiculously wide 2048-bit bus widths (ex. AMD Vega 64) is completely viable since you're working with relatively bulk array data almost constantly. Now most GPUs aren't going quite that wide to hit the speeds they want, but consider that 192-bit is still considered "low-end" here and that's already a lot wider than you'll find on most CPUs.
On a CPU, especially in languages like Java, JavaScript, etc..., memory access patterns are often very random and very small. Consider just something like a simple boolean accessor call `if (foo->isEnabled()) doThing();` - you're reading let's say 8 bytes (foo pointer) to in turn access 1 byte (a boolean). Now the CPU is already reading in chunks of 64-bytes typically (cache line width), so you're already blowing 2 memory fetches to get 128-bytes of data just to read 9 bytes total. Going wider, which is how GPUs scale bandwidth, is utterly pointless here. You'd just be fetching even more data you didn't want in the first place just to occupy precious cache space on something you're probably not going to use anyway. In fact this is why DDR5 actually essentially shrinks the bus-width, from 64-bit in prior DDR generations to 2x32-bit in DDR5
> especially in languages like Java, JavaScript, etc
Not only there is sequential array in both the language, it is very often used. Things like iterating the array is one of the most common operation and is almost always memory bandwidth bound even for interpreted language.
>Not only there is sequential array in both the language, it is very often used
In Java only primitive arrays are memory-efficient for iteration; ArrayLists are not because every element is boxed, so even when iterating sequentially a lookup of some random memory address is needed for every element.
a major part of JVM, JS Engines and even memory allocators, pools and similar is to make this patters less random, more predictable and more local.
For example a common optimization for lisp like languages it to layout a linked list in a continuous way in memory.
Stuff like that is needed even if you have more memory bandwidth as sometimes even more important then memory bandwidth is cache locality, which is WAY harder to scale then adding more memory channels. And branch prediction a major performance factor for modern CPUs also benefits quite a lot from cache locality.
Furthermore having more bandwidth also only helps if you can bring the bandwidth to each core as needed. But if you ad more memory bandwidth it still has a limit per memory region which can be an issue.
So in practice nearly all applications benefit from more bandwidth but the degree they do benefit is limited outside of highly paralelizable use-cases (like some graphics and AI use-case).
So for most typical consumer use cases it's not worth the additional (high) cost this would impose to various aspects (especially motherboards).
And for pro-sumer/server CPUs _we do have more channels_!
But always a balance between added cost and expected need for more bandwidth.
> In fact this is why DDR5
AFIK the reason they DDR5 has a split channel is way more complicated then that
> a major part of JVM, JS Engines and even memory allocators, pools and similar is to make this patters less random, more predictable and more local.
They don't, though. The JVM memory model doesn't even really allow it. This is a big reason for value types, but that keeps getting perpetually delayed...
> On a CPU, especially in languages like Java, JavaScript, etc..., memory access patterns are often very random and very small. Consider just something like a simple boolean accessor call `if (foo->isEnabled()) doThing();` - you're reading let's say 8 bytes (foo pointer) to in turn access 1 byte (a boolean). Now the CPU is already reading in chunks of 64-bytes typically (cache line width), so you're already blowing 2 memory fetches to get 128-bytes of data just to read 9 bytes total. Going wider, which is how GPUs scale bandwidth, is utterly pointless here. You'd just be fetching even more data you didn't want in the first place just to occupy precious cache space on something you're probably not going to use anyway. In fact this is why DDR5 actually essentially shrinks the bus-width, from 64-bit in prior DDR generations to 2x32-bit in DDR5
I think that you are mixing ram channels with ram channel width. Channels, could be fetching totally different address at same time. On a multi core world this is very helpful. One core could being fetching a single 32 bit integer, and another core could being fetching a a struct of 128 bytes from another address.
Also, server CPUs usually have many more RAM lanes that desktop CPUs . For example, EPYC server CPU could have 12 RAM channels!
I don't know how often it's a problem, but I work for a company doing software video encoding, and we always fill up all the dimm slots on servers to have as much bandwidth as possible, even if we have only really use maybe 1/4 of the RAM.
Depends on the task you are concerned with; see Roofline model [1]. It boils down to "how many ops will you do for each value you load from RAM to cache".
I don't know about typical consumer workloads, but e.g. for physics simulations, any time you need to solve a system where the value in a point depends on a lot of other points you are bandwidth limited. Like solving the pressure equation in fluid simulations. Also I believe a lot of traditional database workloads fall into this category.
You can measure memory bandwidth with the STREAMS benchmark.
lets say instead of infinite you get 16 instead of 2 channels
then while probably all applications will profit
for most consumer applications you might not notice much of a difference
through you can measure some difference in benchmarks
one exceptions would be various graphic editing software if you do stuff like editing 8k multi channel video content
For compiling you might feel a noticeable difference. It won't be 8x but if you pair it with enough cores and things which have tons of parallel compilation parts it will really feel noticeable faster.
But for most gaming you instead of getting 8x the bandwidth you probably benefit more from a speed improvement e.g. running 6000MT/s DDR5.
Anyway you can get it today ;)
The M2 Ultra has a pretty insane memory bandwidth (and so had the M1 Ultra).
I had a Tseng ET4000 I think that had socketed memory, and has some Trident TVGA which I beefed up from 512k to 2MB to get 65k colours and 800x600 maybe...
I had a GUS soundcard for a while and it also had an EDO slot I think.
Aside from what others have said about greater bit width only being useful for large fetches, there's also circuitry necessary to allow more parallel fetches from the L1 to other parts of the memory subsystem (for Intel, these are called line fill buffer) and they're the real limiting factor for single threaded memory bandwidth (at least on that platform). For whole chip bandwidth this isn't as much of an issue, but the things others have said are true there still.
The number of slots doesn't matter but the number of channels, i.e. x86 board have 4 slots but only 2 channels (through nomenclature is a bit of a mess as as depending on context channel might mean something else).
More memory bandwidth increases the price/size of the IO module on the CPU but I don't think that's a major reason.
One of the major reason AFIK is to keep the number of channels low is that it directly maps to the number of pins a socket need to has and the number of high quality lanes which need to be routed independently equally long and chided from inference from e.g. power lines to the CPU.
So especially e.g. for server design chip vendors balance the benefit of more memory channels with the cost of it increasing socket size/pins, complexity of mother boards etc.
And while on the paper doubling the memory channels doubles the memory bandwidth it doesn't at all means you RAM is twice as fast. It's similar to the difference between duplicating the number of cores vs. duplication the per-core IPC. While all applications profit from more bus width the degree might differ hugely. Oversimplified super highly parallelizeable tasks (e.g. much of GPU tasks and many AI tasks) tend to also be better at taking advantage of more bandwidth through more channels but many other taks less so. This means that the degree to which more channels will yield benefits highly depend on the task the CPU vendor expect is majorly run on it. And the tasks which most benefit from it already tend to be run on GPUs anyway.
Apples M1,M2 CPUs one the other hand have a much much smaller cost problem wrt. more memory channels due to how they stack ram on top of the CPU (I think) and they happen to also have much more memory bandwidth by having more bus width.
But stacking RAM on top of CPUs has it's own issues. Like for example if the RAM dies the whole CPU is dead and you not being able to upgrade the RAM. Doesn't matter that much for apple, but matters a lot for AMD/Intel which core focus is the server center, not desktop CPUs. Having to replace a 10k CPU and multiple TiB of RAM because a single ram segment fails just isn't acceptable. Additionally for some use cases instead of stacking RAM on the CPU you get as much or more benefit by stacking tons of L3 cache on it (AMD 3D V-cache). There also seems to be in general be more issues with thermal budges, not an issue for apple of which most devices are mostly passive cooled. Issues with manufacturing cost making it harder to provide many CPU spec+RAM amount variants (again doesn't matter for apple but does for Intel/AMD). Potential cost issues making it only suitable for "premium" products (again not an issue for Apple but for Intel/AMD). And maybe more.
So basically:
- on server (or pro-sumer,i.e. Threadripper) more channels are already used, but it's always a cost benefit balance where there could be even more channels
- on desktop, at least currently, it's not seen to be worth the price/complexity increase
Through Apples is putting a bit of pressure on the market, but outside of the high end segment it still makes little sense and it's probably not worth it to have it just for the "high end but not pro-sumer" use-case, that market it too small.
I would predict prosumer devices like Threadrippers to grow the number of memory channels with every now socket they have.
And I do predict a major technology change for "normal desktop computers" 3-6 years down the line which probably will bring more channels to desktop computers. But only then. And it's very speculative.
In addition to what everyone has already said, memory controllers are relatively big and expensive. System integrators often ship systems <~$1000 with only a single channel of memory. It’d probably be like pulling teeth to get them to actually ship a 4-channel configuration. Putting in unused memory controllers is a big waste.
People who have many threaded workloads which can actually use all that memory bandwidth tend to get workstation or server platforms.
Apple uses LPDDR, which is what's used in all smartphones and most really thin laptops. LPDDR has the exact opposite design goal from HBM: LPDDR is optimized for high bandwidth per pin to enable low pin counts, while HBM relies on advanced silicon interposer packaging to enable extremely high pin counts allowing for modest bandwidth per pin. The only similarity is that Apple puts the LPDDR in the package with the CPU, but it's an entirely different and more mundane packaging technology compared to what is used for HBM.
AMD seems not interested. Intel will go the Apple way though, simpler cores and trading die area for ever larger caches, also on-die memory, downside you can't upgrade memory, upside it's faster and more power efficient.
[Edit] Or someone could have said: I think you meant processor not die, and I would have said: Yes sure, not sure what was on my mind.
It becomes more of a consideration when accounting for the fact that you can buy a Macbook that ships with an un-upgradeable 8 GB of RAM, which, sadly, is deficient for even moderate tasks these days.
Upgradable memory also makes things more flexible for OEMs.
Most OEMs continue to sell systems built around last years processor (or even a couple year old processors), and they can increase the memory in those systems as needed. Maybe 4GB was enough when they came out, but now you need 8GB or 16GB to sell a system with an older processor. Not a big deal with socketed RAM, not even a huge deal with soldered RAM, but the processors with 4GB on package become very hard to sell.
Apple's cores are very far from simple. They are much larger than the other ARM CPUs on the market, and this is also why they are faster than other ARM CPUs on the market.
> trading die area for ever larger caches
Apple has big L1 comparatively, but L2 & L3 are nothing special. Certainly nowhere close to the massive caches that AMD's X3D offers.
> also on-die memory
Apple doesn't have on-die memory. AMD is the only company to have shipped on-die memory to consumers (they were also the first to ship it), and they only did so on GPUs so far. Nvidia & Intel are playing with it for datacenter products, though, but that's about it (H100 & Xeon Max respectively)
> Apple doesn't have on-die memory. AMD is the only company to have shipped on-die memory to consumers
What distinction are you drawing such that Fury X interposers count as “on-die memory” but apple’s literal stacking of the LPDDR on the actual die doesn’t?
Apples technique is more advanced than fury actually: fury is just on-package, not on-die, but apple is literally doing v-cache style stacking of the whole memory subsystem right on top of the die.
Of course fury x was earlier, but die stacking is a more advanced technology, there is a reason AMD didn’t do that for 5+ years after fury x. And apple actually beat AMD to market with on-die stacking, by several years (epyc v-cache was 2022 or 2021) with a more advanced implementation (the whole memory subsystem rather than just v-cache).
(People literally are in such a race to pick a fight and defend their best friend AMD they forgot to make the rest of the post lol. Simply say the most mild thing and AMD people trip over themselves to do this stuff constantly, it’s funny.)
Do you think OP is referring to something other than fury x/Vega when they said “gpu shipped to consumers”? That’s the only thing that comes to mind, literally there are only less than a half dozen consumer gpus from AMD in the stacking era so I don’t know what else it would be.
If you mean true memory in the die, not stacked, not just a cache - AMD has never shipped that to consumers (or professionals) in a gpu. Nobody puts memory in the gpu because it would be a waste of die space.
But like, gotta defend AMD, better hammer that downboat and drop a pithy six-word/single-sentence reply. Social media is constantly seething with the AMD defense force looking for fights, gets super old.
Like, it was literally less than 30 seconds from clicking post on my comment to seeing your debate determinating cliche. What fine discourse we have here.
AMD’s consumer gpus don’t use v-cache. Apple is using the same memory-on-package technique that fury x and Vega use. Rdna3 cache is not v-cache, just another flavor of mcm.
AFAIK Apple only uses a silicon interposer to connect the CPUs. The memory is on standard interconnects through the package like AMD's chiplet approach or the ancient Intel Clarksdale.
It's very obviously not on die. On package maybe, but definitely not on-die.
Since you're talking about die stacking i assume you're now talking about their "low end" phone parts. That's a quite different technology from V-cache, foveros, or even Apple's "UltraFusion." It's also not at all a unique technology to Apple, it's very common in the phone SoC space where power consumption is so limited that the thermal issues are mostly negated.
Clearly! I don’t know if I’d say it’s “on par” if an Apple A-series part (i.e., low power enough to be shipped in iPhones and not in any of their “serious” computers) is actually getting slightly higher numbers than the fastest, most power hungry core that AMD has right now!
> I don’t know if I’d say it’s “on par” if an Apple A-series part (i.e., low power enough to be shipped in iPhones and not in any of their “serious” computers) is actually getting slightly higher numbers than the fastest, most power hungry core that AMD has right now!
It isn't. The A17 Pro gets 7199 on GB6 multi-thread compared to 18778 for the 7950X, and multi-thread workloads are the only thing that will cause the 7950X to use its full TDP. If you give the Apple chip a desktop's power budget, the single-thread number barely changes because that isn't what sets the TDP.
And Zen4 is punching above its weight because it's on TSMC 5nm compared to the A17 Pro on TSMC 3nm. Try comparing the chips that use the same process.
No, that is clearly not the case given that the TDP of the A17 Pro (10W) is 17 times less than the TDP of the 7950X (170W) while the multi-core performance of the 7950X is only 2.6x that of the A17 Pro.
The reason it uses more power is that it's less efficient. The reason it has higher multi-core performance is that it has four times the number of cores.
I dunno, I haven't seen much evidence that 3nm is as big a leap as hoped/claimed, the A17 pro is certainly impressive but in most cases it is not more efficient for its power gains.
Interesting that you‘re writing about serious computers followed by referring to running games at 200 fps. It seems like you yourself don’t know anything about actual “serious“ use-cases.
Here's what I did years ago, and will never look back:
1. Get a usb extender and a fiber optic displayport extender.
2. Stick your computer in the garage, run the cables to whatever room and hook up your monitor and peripherals.
3. Enjoy as perfect silence and low heat generation as can be possibly be achieved. (Monitors generate heat and sometimes electrical noise, and mice and keyboards are what they are)
yup. if you are picky about mobo selection, you can also do it all over one fiber thunderbolt cable; I have my desktop rackmounted and could go up to 165ft away for the desk.
It’s a problem with laptops especially, where massive coolers aren’t a possibility (at least if you want the laptop to be more than just technically “portable”). Even worse, tiny whiny fans are the standard there which means anything with much power at all will have times where its fans are screaming. Really annoying.
That is like saying every single Intel CPU cant be overclocked because they were not designed to do so. There certainly is a limit ( It definitely cant do 5Ghz ), but I am willing bet a 20% is easily achievable with enough cooling.
The A17 is already on the TSMC 3nm process, while the 7950X is on TSMC 4nm. If anything, the A17 has the advantage here and the real question is, where the AMD chip would land if it was produced with the same technology.
I have seen this over and over again and it is tiring. We are discussing ISA, uArch ( and possible Node as well ) in a ChipandCheese article. Not a CPU comparison / gamer review site.
If I wanted to build a home server with very good single core performance but very low power consumption (storage and upgradability aside), what are the best options right now? Probably a Mac Mini M2 haha
Depends on your definition of "very low power consumption". The M2 Mac Mini will max out at 50w[0] from the wall, which is several times higher than a Raspberry Pi under load. You genuinely might have better luck with a laptop board in a mini-PC, depending on your particular constraints.
For a home server, idle power consumption is probably more important than efficiency under load. Ryzen desktop processors are certainly better than most server platforms, but their chiplet-based design is quite bad for idle power. AMD has sold laptop SoCs packaged for their desktop sockets, but I don't think those come with ECC capability enabled.
Comparing the mobile 7840U[0] to the M2 Mac Mini[1] suggests it's kinda a wash. Both idle around 6-7w, and under load they both max out around 50w.
There are definitely less efficient chips you could bring into the comparison fold, but the M2 is a mobile chipset. You need another mobile chip if you want to draw more interesting power consumption comparisons.
I think you lost the plot. Bringing up AMD's low-end mobile processors in a discussion of whether their desktop processors are well-suited for a home server with ECC is pretty far off topic.
I have been interested in the low power home server space for a while.
What I ultimately want, is a collection of SBCs that are able to be remotely booted so I can have Intel and AMD x86 systems, ARM, Risc-5, Apple that can spin up on demand.
If you're willing to look at 1L form factors (think Mac Mini), you do have options there.
Intel has vPRO and AMD has AMD DASH for remote power on. A Mac Mini won't have remote power on, but if you're willing to just keep it slept, Wake-On-LAN is an option.
If your goal is to get the best single core performance at a given wattage: pick any high end option (13900k, 7950x, M2) and just limit the power target/available performance to the power level you want. I.e. there is a massive amount of power variance in getting dozens of cores to eek out slightly more multi-core performance but the power variance for single core performance at a given wattage is really not that much.
If your goal is to go cheap but still have decent performance it depends on what exactly you need it to perform at but mini PCs like Beelink usually give otherworldly performance vs what you'd expect in the 100-200 dollar category.
I can probably squeeze a bit more out if I boot to linux and cap the TDP for the MT tests.
I'm not saying the Apple silicon isn't impressive, but the 7640U (or any Phoenix silicon) isn't positioned as premium silicon like Apple's stuff. AMD would be using a wider bus, more on-die L3, faster packaged RAM, their TSV L3 or something like that if it was.
Its undervolted, but it doesn't make a difference in ST tests because (without bus overclocking) its hard capped at 5GHz.
I am running good RAM, and some laptops run soldered LPDDR5X which is also very good. DDR5 SODIMMs are unfortunately not so good, as they need 1.35V to run at awful timings/speed.
Honestly I hope they come to desktops and servers too. The performance of full size ddr5 dimms seems OK for now, but it must be close to its limit if SODIMMs are already past their limit.
Oh wow, LPCAMM sounds like a dream!
According to the article this form factor is incompatible with normal CAMMs, let's hope that gets figured out and standardized before long. (And actually gets applied throughout the market!)
Look: Apple fabs good cores. As a matter of fact, they're pretty much the only company even attempting to make ARM cores. They have lots of money, and good designs on dense silicon usually equals fast chips.
This is a brutally facetious comparison though. Nevermind the obvious ARM/x86 disparity, or even the node difference in fabrication; you're not even comparing mobile cores to each other. What kind of serious comparison takes desktop wattages and compares them to a smartphone SOC?
If you take the chip the parent mentioned (7640U) instead of picking a random desktop option, the gap closes considerably. 7640U scores 2559 in single-core, which scales very well for multicore workloads. Given that it's a 6-core chip with a 30w max TDP, a single core probably couldn't draw more than 5w either.
I don't think anything will be able to compete with Apple as long as they continue to buy out all the capacity of the latest node offered by TSMC. Unless Intel finally gets their act together.
Now that op fusion has been implemented for this idiom, the cost of a new instruction does not seem worth the benefit.
The benefit would be only a tiny improvement in code size, and that's restricted to when the compiler can be absolutely certain that the target CPU is going to be modern enough to support it. So, in practice, for many, many years, this benefit would be almost never reaped.
This is not a new topic in the design of ISAs and their implementations in silicon. Idiom recognition and optimization has long existed in x86, ARM, and I'm sure other old ISA implementations. RISC-V can avoid it by adding dedicated instructions because it's just getting started (relatively speaking), but even so, I'm sure there will be opportunities for idiom recognition and optimization that RISC-V implementers will take.
On RISC-V's fusion idioms, it's not just recognition; they're designed in. Compilers are meant to try and produce them reliably, as if they were a dedicated single opcode.
For x86? Yes. There are many holes in the opcode space that are regularly filled. The bespoke encoding comes from there not being (really any) holes in the beginning, and the designers trying to work around it.
At the expense of code density, and thus hurting PPA (Power, Performance and Area).
Wildly variable opcode length does also complicate the decoder considerably, as it has to find where instructions start and end, an inherently serial operation, and yet decode enough of them to feed an n-wide pipeline.
Yet, it does not push macro-op fusion in implementations. It can either be done, or not done. But RISC-V creates opportunities where it is possible and not harmful to do so, while increasing code density.
>Exploiting this fact, the RV64G and RV64GC effective instruction count can be reduced by 5.4% on average by leveraging macro-op fusion.
It is up to the architects behind implementations to decide when and if to take advantage of the opportunities created by RISC-V's design to do macro-op fusion.
> Yet, it does not push macro-op fusion in implementations
Yes it does. It pushes it as an alternative to adding instructions to the ISA. It says we don’t need these extra instructions because architects can use macro-op fusion to remove the penalty associated with having two instructions rather than one.
>It says we don’t need these extra instructions because architects can use macro-op fusion to remove the penalty associated with having two instructions rather than one.
They are correct in saying that. x86 didn't have the chance to do so, because back when it was designed, macro-ops were not considered.
Yet whether to fuse or not is still up to microarchitecture architects. RISC-V doesn't push it. Only creates the opportunity where sensible, while increasing code density which benefits everyone.
In the same sense that a CPU not supporting hard floats gives software the 'opportunity' to implement soft-floats.
With RISC-V's current design, you can either not do fusion and eat a penalty, or you can do fusion.
Different ISAs may have a greater or lesser need for fusion, and greater or lesser associated penalty. It seems a little obtuse to insist the ISA doesn't encourage fusion, when other ISAs would avoid a structural penalty for not having fusion, that RISC-V intentionally doesn't try to avoid at all.
>With RISC-V's current design, you can either not do fusion and eat a penalty, or you can do fusion.
Only a small portion of existing RISC-V hardware uses fusion.
This should tell you all you need to know about this mistakenly inferred "penalty" and whether RISC-V forces macro-op fusion into anybody.
As per why fusion helps even when not leveraged: Not allocating encoding space to a unified opcode does, in average over a large body of code, increase code density, as shown in the paper.
This means more code fits in cache, and in hardware this does weight far more than instruction count, which remains low and highly competitive even where fusion isn't leveraged.
Well, that existing RISC-V hardware is also significantly slower than modern x86/ARM, so judging the importance of fusion from them is pretty meaningless. Surprise, there's a bunch of x86 & ARM hardware without fusion (or very little of it) too!
Of course, a slow core can easily afford to not have any fusion, but that's a boring question with an obvious answer. The only situation where it makes sense to ask is in the context of a fast core. I think it's pretty clear that competitively-fast RISC-V cores will need significant amounts of fusion.
Anyway, in the context of the original post of cmp+branch, this doesn't even matter, as "separate cmp & branch, fused together if perf desired" and "separate add & load, fused together if perf desired" are the exact same kind of thing, so there's no reason to call one "accumulated cruft" and the other good design. (ok, there are some major problems with x86's flags, but that's primarily just because many operations needlessly update them; other than that, they're just an alternative way to achieve a thing)
>Your definition of ‘push’ here is make compulsory
Macro-op fusion is entirely optional, leveraged by few hardware implementations; Fusion is useful mostly (i.e. exceptions exist) only on mid-range cores.
High end cores prefer to split everything up (unnecessary on RISC-V), whereas low end cores prefer simpler hardware.
Net win in code density weights more in hardware than instruction count, which remains excellent in RISC-V even when the studied microarchitecture does not implement fusion.
Not a CPU designer by any means, but I imagine mundane RMW ops on regular registers, like the second half of LUI+ADD, are easier to handle than a flags register that is implicitly threaded through the entirety of the program. So the argument against the CMP+BEQ design could be all the other possible occurrences of BEQ the architecture is required to handle, not the one that can be fused (or not) with a preceding instruction.
(Then RISC-V goes and introduces a flags register for floating-point ops...)
A flags register is "threaded through the entirety of the program" just as much as all other registers. Though lui+add does have the benefit that the add should write over the lui result, whereas cmp+branch needs to both update the flags & do the branch (but x86 cores should have no problem with that, as every single add & xor & and & inc & load & etc also write to flags already).
Flags for FP aren't particularly nice, but without them you just can't implement <fenv.h> which software may be expecting. (of course using a global flag for semantics isn't particularly nice either, but hey that's what's been standardized in C)
Dubious, as that's up to implementation either way.
A microarchitecture might split it into a load and an add, eliminating any instruction count reduction advantage.
Whereas another microarchiture might take a separate load and add and fuse them, which would free opcode space to be used more effectively elsewhere, resulting in overall code size reduction.
Can I ask why more manufacturers don't implement apple's strategy with soc with ram and all and offer option to add additional classic ram as a slower cache buffer between soc&ssd? This should both solve the lack of upgradability and unlock more performance per watt?
Intel did this is the Sapphire Rapids xeons via 64 GB HBM built in and DDR 5 slots. L4 cache is also not unheard of, having been tried a few times for well of a decade. The downsides are cost/complexity for relatively little gain.
Your comment is ambiguous, but if you’re implying that Apple does this then it’s incorrect. They just have “unified memory” (i.e. the high bandwidth SoC substrate memory).
Yes, and I'm asking why not allow to add extra ram through slots as a slower cache between unified mem and ssd? And why other manufacturers are not doing this to achieve apple lvl of performanca and also allowing upgradability
I thought slots don't really slow things down much and aren't the primary source of latency, isn't it more about cost and power? Adding slots back in would remove the cost and power advantages wouldn't they? And now using more SRAM to track occupancy of an extra layer of cache, or if not SRAM for that something adding more latency.
One of the neighboring comments citing Intel’s HBM memory I guess does the 2-layer memory thing. The reason we probably don’t see these dual-level memory solutions is that the fast-access part of the memory of HBM is really expensive to implement on the same substrate as the SoC. For example, part of the reason why top-end GPUs are expensive (besides “pricing power in the age of AI”) is that they all have HBM memory.
DDR based memories are super-cheap in comparison to implement in hardware. Also another aspect is that the SoC needs to support HBM memory, most chips of yesteryears simply have DDR<n> support only.
168 comments
[ 3.1 ms ] story [ 228 ms ] threadDumb question, but why don't Intel/AMD massively increase memory bandwidth by adding more memory lanes? For example instead of the usual 4 slots on a desktop motherboard you would have 16 slots surrounding the CPU on all sides.
Is it that the CPU would be unable to effectively use all that bandwidth due to less parallelism compared with a GPU?
Yes. That is why you only see more memory channels for higher CPU count.
given there use case of
- premium consumer only products
- which are mostly portable and have a low thermal budget
- which are by most users hardly ever run on full throttle
- a well documented tendency to proclaim any hardware failure as user fault
makes this quite viable for them
but if you run systems with 4TiB of RAM and 12k+$ CPU which is most time running full throttle potential for days at a time with a higher thermal budged and customers which might have docents or even thousands of such CPUs and will be very pushy about reclaiming any failure they think isn't their fault then having to replace over 20-50k$ just isn't viable (instead of ~1-3k...)
Edit: Of course, it depends on the benchmark. Latency is critical for programs that access many memory addresses randomly, not for data processing applications that scan memory linearly (and GPU use-cases often fall into that category).
If I'm correct, this is the difference between DDR RAM and GDDR RAM - the latter has higher bandwidth but also higher latency, which is great for graphics with its high bandwidth requirements and very predictable memory access patterns (except when you do ray tracing) which allows you to hide the latency, but you don't want to use GDDR for system RAM if you value performance.
Here’s an unusual motherboard salvaged from Xbox Series X which combines AMD Zen2 CPU cores, GDDR6 memory, where the integrated GPU is disabled due to defects: https://www.tomshardware.com/news/4800s-xbox-chip-shows-us-w... The performance is not great.
GPUs are good at hiding the latency due to high degree of parallelism. GPU cores switch to another thread instead of waiting for the data. Each CPU core only runs 2 threads, it can’t quite do the same thing.
GPU memory access patterns are only great for well written compute shaders (or CUDA kernels) when the developer who wrote that shader managed to do fully coalesced loads and stores. Memory access patterns for pixel shaders are not that great, especially when doing trilinear or anisotropic sampling in these shaders.
But you can optimize around the limitation (within reason) and don't have to content with a general non-specified OS.
This is why I do wonder about the path not taken with the 1T-SRAM that powered the Gamecube/Wii. It was designed purely with low latency in mind but I don't think it scaled well.
Yes, but also it's due to simple physical limitations. If you look at how close CPUs are to RAM as well as the number of PCB lines coming out of CPUs, it becomes pretty readily apparent that there's not much room for additional lines.
You have to recognize that each of those lines is creating a magnetic field, the closer you stack them, the more likely cross talk is.
We could continue to lower the voltage on the memory lanes so you can pack them closer together, but then you have to start worrying about environmental interference on the lines. It likely won't be too long in the future (if it's not already here) that we'll start seeing shielding for the memory lanes so they can be packed closer together and more can be added.
It wouldn't shock me if this became a part of DDR6.
It also wouldn't shock me to start seeing more complex memory controllers doing things like lz4 compression to eck out even higher bandwidths (particularly when the memory is highly uniform). We are sort of seeing the precursor of this in DDR5 with commands to set whole blocks to a given value.
I'm not sure what that would add for latency and cost, but I'm guessing that with optical wave guides, crosstalk isn't a concern. At least for the optical part of the data path.
It really is insane the level at which current hardware works.
Isn't that kind of what those APUs sold by AMD are? And SoCs? I could be misunderstanding you, though.
PC desktops have 2 memory channels. EPYC servers have 12 channels per socket. It's not the physics, it's that ~8-core processors generally aren't bounded by memory bandwidth and it's not worth the cost of increasing the number of pins.
The server sockets do it because they have 128-core processors.
If this changes what we may start seeing is CPUs (or iGPUs) with a few GB of HBM as L4 cache.
no physics matter
because it makes it much harder and in turn more costly to the more channels you have to have high quality reliable connections
server RAM is for example also slower to compensate this e.g. recent EPYC processors run at up to 4800MT/s but a Ryzen 9 runs at up to 5200MT/s (without overclocking, with running at 6000MT/s isn't that uncommon)
server also tend to always use EEC memory
so it's a benefit/cost balance calculation
Zen3 EPYC has 8 channels per socket and uses the same 3200MT/s DDR4 as Zen3 desktops, so this little explains why desktops couldn't have e.g. 4 channels if there was a need for it. They already commonly have 4 DIMM slots.
The rated speed for DDR5 is 8% higher for desktops, but the additional channels don't seem to be why. Intel uses 8 channels of DDR5 rather than 12 and it's still 4800MT/s.
> so it's a benefit/cost balance calculation
But that was my point?
An extreme example, the phone I am using has RAM running at 1.8Ghz but on a 16bit bus. Anything to keep costs down.
But, more memory channels means more pins and larger sockets, and that adds cost and layout issues for motherboards. HBM ram that attaches on top of the cpu addresses the layout issue (and the bandwidth issue), but interferes with heat removal and removes expandability. Apple makes it work, but they limit clock speeds and were never big on expandability. Intel and AMD have a hard time in the market when their top consumer chips don't clock to the moon, even if the last 10% of clock speed takes 50% of the power and delivers 5% of performance (percentages made up). Server chips don't need to clock so high, so AMD can release clock limited compact cores that have about the same IPC in a bit more than half the area, and the difference in clocks for a max-cores regular zen4 and a max-cores zen4c isn't too much (L3/core is half for zen4c though, so may not be a benefit depending on your working set)
Eh. RAM generally consumes a single digit number of watts and will do so regardless of where it's physically located in the system.
If you put HBM on the CPU socket then you have to replace them together, but you can still replace them. It could also make sense to do both: If you have 16GB of HBM and 16GB of DDR5, you have 32GB of RAM and half of it is faster. Or 80GB of RAM and 16GB of it is faster. Cache hierarchies are a good way to strike a balance between performance and cost, and then the latter can still be expanded independent of the CPU.
Tradeoffs are different for server boards (many cores in the CPU) and on platforms with integrated GPUs (becoming more common) so none of this is static. I suspect in-package RAM will become more common even for platforms that still have a couple channels of slotted memory, that would create a pool of faster memory available for graphics etc while still leaving room for more if you need to have a lot of Chrome tabs open or whatever.
On a GPU, the memory access patterns are often very wide & easily controlled, such as fetching a block of texture memory. This is like streaming reads for a disk, where you can actually just throw essentially RAID 0 at the problem to scale up really easily. So scaling all the way up to ridiculously wide 2048-bit bus widths (ex. AMD Vega 64) is completely viable since you're working with relatively bulk array data almost constantly. Now most GPUs aren't going quite that wide to hit the speeds they want, but consider that 192-bit is still considered "low-end" here and that's already a lot wider than you'll find on most CPUs.
On a CPU, especially in languages like Java, JavaScript, etc..., memory access patterns are often very random and very small. Consider just something like a simple boolean accessor call `if (foo->isEnabled()) doThing();` - you're reading let's say 8 bytes (foo pointer) to in turn access 1 byte (a boolean). Now the CPU is already reading in chunks of 64-bytes typically (cache line width), so you're already blowing 2 memory fetches to get 128-bytes of data just to read 9 bytes total. Going wider, which is how GPUs scale bandwidth, is utterly pointless here. You'd just be fetching even more data you didn't want in the first place just to occupy precious cache space on something you're probably not going to use anyway. In fact this is why DDR5 actually essentially shrinks the bus-width, from 64-bit in prior DDR generations to 2x32-bit in DDR5
Not only there is sequential array in both the language, it is very often used. Things like iterating the array is one of the most common operation and is almost always memory bandwidth bound even for interpreted language.
In Java only primitive arrays are memory-efficient for iteration; ArrayLists are not because every element is boxed, so even when iterating sequentially a lookup of some random memory address is needed for every element.
a major part of JVM, JS Engines and even memory allocators, pools and similar is to make this patters less random, more predictable and more local.
For example a common optimization for lisp like languages it to layout a linked list in a continuous way in memory.
Stuff like that is needed even if you have more memory bandwidth as sometimes even more important then memory bandwidth is cache locality, which is WAY harder to scale then adding more memory channels. And branch prediction a major performance factor for modern CPUs also benefits quite a lot from cache locality.
Furthermore having more bandwidth also only helps if you can bring the bandwidth to each core as needed. But if you ad more memory bandwidth it still has a limit per memory region which can be an issue.
So in practice nearly all applications benefit from more bandwidth but the degree they do benefit is limited outside of highly paralelizable use-cases (like some graphics and AI use-case).
So for most typical consumer use cases it's not worth the additional (high) cost this would impose to various aspects (especially motherboards).
And for pro-sumer/server CPUs _we do have more channels_!
But always a balance between added cost and expected need for more bandwidth.
> In fact this is why DDR5
AFIK the reason they DDR5 has a split channel is way more complicated then that
They don't, though. The JVM memory model doesn't even really allow it. This is a big reason for value types, but that keeps getting perpetually delayed...
I think that you are mixing ram channels with ram channel width. Channels, could be fetching totally different address at same time. On a multi core world this is very helpful. One core could being fetching a single 32 bit integer, and another core could being fetching a a struct of 128 bytes from another address.
Also, server CPUs usually have many more RAM lanes that desktop CPUs . For example, EPYC server CPU could have 12 RAM channels!
Or to put it another way, if I had infinite memory bandwidth, how much faster would browsing/gaming/compiling become?
I'm not sure any of the standard Linux tools can show you memory bandwidth usage easily (maybe perf), I know we use Intel PCM (https://github.com/intel/pcm) and AMDuProfPCM (https://www.amd.com/en/developer/uprof.html)
I don't know about typical consumer workloads, but e.g. for physics simulations, any time you need to solve a system where the value in a point depends on a lot of other points you are bandwidth limited. Like solving the pressure equation in fluid simulations. Also I believe a lot of traditional database workloads fall into this category.
You can measure memory bandwidth with the STREAMS benchmark.
[1] https://en.m.wikipedia.org/wiki/Roofline_model
then while probably all applications will profit
for most consumer applications you might not notice much of a difference
through you can measure some difference in benchmarks
one exceptions would be various graphic editing software if you do stuff like editing 8k multi channel video content
For compiling you might feel a noticeable difference. It won't be 8x but if you pair it with enough cores and things which have tons of parallel compilation parts it will really feel noticeable faster.
But for most gaming you instead of getting 8x the bandwidth you probably benefit more from a speed improvement e.g. running 6000MT/s DDR5.
Anyway you can get it today ;)
The M2 Ultra has a pretty insane memory bandwidth (and so had the M1 Ultra).
I had a GUS soundcard for a while and it also had an EDO slot I think.
e.g. if we speak about how many memory channels a x86 desktop, prosumer or server CPU have they count them as a single channel
main reason for this is that DDR5 didn't split the channel instead of adding a new channel
More memory bandwidth increases the price/size of the IO module on the CPU but I don't think that's a major reason.
One of the major reason AFIK is to keep the number of channels low is that it directly maps to the number of pins a socket need to has and the number of high quality lanes which need to be routed independently equally long and chided from inference from e.g. power lines to the CPU.
So especially e.g. for server design chip vendors balance the benefit of more memory channels with the cost of it increasing socket size/pins, complexity of mother boards etc.
And while on the paper doubling the memory channels doubles the memory bandwidth it doesn't at all means you RAM is twice as fast. It's similar to the difference between duplicating the number of cores vs. duplication the per-core IPC. While all applications profit from more bus width the degree might differ hugely. Oversimplified super highly parallelizeable tasks (e.g. much of GPU tasks and many AI tasks) tend to also be better at taking advantage of more bandwidth through more channels but many other taks less so. This means that the degree to which more channels will yield benefits highly depend on the task the CPU vendor expect is majorly run on it. And the tasks which most benefit from it already tend to be run on GPUs anyway.
Apples M1,M2 CPUs one the other hand have a much much smaller cost problem wrt. more memory channels due to how they stack ram on top of the CPU (I think) and they happen to also have much more memory bandwidth by having more bus width.
But stacking RAM on top of CPUs has it's own issues. Like for example if the RAM dies the whole CPU is dead and you not being able to upgrade the RAM. Doesn't matter that much for apple, but matters a lot for AMD/Intel which core focus is the server center, not desktop CPUs. Having to replace a 10k CPU and multiple TiB of RAM because a single ram segment fails just isn't acceptable. Additionally for some use cases instead of stacking RAM on the CPU you get as much or more benefit by stacking tons of L3 cache on it (AMD 3D V-cache). There also seems to be in general be more issues with thermal budges, not an issue for apple of which most devices are mostly passive cooled. Issues with manufacturing cost making it harder to provide many CPU spec+RAM amount variants (again doesn't matter for apple but does for Intel/AMD). Potential cost issues making it only suitable for "premium" products (again not an issue for Apple but for Intel/AMD). And maybe more.
So basically:
- on server (or pro-sumer,i.e. Threadripper) more channels are already used, but it's always a cost benefit balance where there could be even more channels
- on desktop, at least currently, it's not seen to be worth the price/complexity increase
Through Apples is putting a bit of pressure on the market, but outside of the high end segment it still makes little sense and it's probably not worth it to have it just for the "high end but not pro-sumer" use-case, that market it too small.
I would predict prosumer devices like Threadrippers to grow the number of memory channels with every now socket they have.
And I do predict a major technology change for "normal desktop computers" 3-6 years down the line which probably will bring more channels to desktop computers. But only then. And it's very speculative.
People who have many threaded workloads which can actually use all that memory bandwidth tend to get workstation or server platforms.
The new Intel Xeon Max processors have 64GiBytes of HBM in the package, as well as offboard DDR5.
Apple A17 Pro, 3.8Ghz, Single Core GB6 Score @ 2914
I dont think asking for an 20% increase in IPC is too much at this point.
[Edit] Or someone could have said: I think you meant processor not die, and I would have said: Yes sure, not sure what was on my mind.
Most OEMs continue to sell systems built around last years processor (or even a couple year old processors), and they can increase the memory in those systems as needed. Maybe 4GB was enough when they came out, but now you need 8GB or 16GB to sell a system with an older processor. Not a big deal with socketed RAM, not even a huge deal with soldered RAM, but the processors with 4GB on package become very hard to sell.
Hmm, curious what the "Apple way" is here..
> simpler cores
Apple's cores are very far from simple. They are much larger than the other ARM CPUs on the market, and this is also why they are faster than other ARM CPUs on the market.
> trading die area for ever larger caches
Apple has big L1 comparatively, but L2 & L3 are nothing special. Certainly nowhere close to the massive caches that AMD's X3D offers.
> also on-die memory
Apple doesn't have on-die memory. AMD is the only company to have shipped on-die memory to consumers (they were also the first to ship it), and they only did so on GPUs so far. Nvidia & Intel are playing with it for datacenter products, though, but that's about it (H100 & Xeon Max respectively)
0: https://www.intel.com/content/www/us/en/developer/articles/t...
What distinction are you drawing such that Fury X interposers count as “on-die memory” but apple’s literal stacking of the LPDDR on the actual die doesn’t?
Apples technique is more advanced than fury actually: fury is just on-package, not on-die, but apple is literally doing v-cache style stacking of the whole memory subsystem right on top of the die.
Of course fury x was earlier, but die stacking is a more advanced technology, there is a reason AMD didn’t do that for 5+ years after fury x. And apple actually beat AMD to market with on-die stacking, by several years (epyc v-cache was 2022 or 2021) with a more advanced implementation (the whole memory subsystem rather than just v-cache).
What isn’t how what works? Use your words.
(People literally are in such a race to pick a fight and defend their best friend AMD they forgot to make the rest of the post lol. Simply say the most mild thing and AMD people trip over themselves to do this stuff constantly, it’s funny.)
Do you think OP is referring to something other than fury x/Vega when they said “gpu shipped to consumers”? That’s the only thing that comes to mind, literally there are only less than a half dozen consumer gpus from AMD in the stacking era so I don’t know what else it would be.
If you mean true memory in the die, not stacked, not just a cache - AMD has never shipped that to consumers (or professionals) in a gpu. Nobody puts memory in the gpu because it would be a waste of die space.
But like, gotta defend AMD, better hammer that downboat and drop a pithy six-word/single-sentence reply. Social media is constantly seething with the AMD defense force looking for fights, gets super old.
Like, it was literally less than 30 seconds from clicking post on my comment to seeing your debate determinating cliche. What fine discourse we have here.
Thanks for the second sentence.
AMD’s consumer gpus don’t use v-cache. Apple is using the same memory-on-package technique that fury x and Vega use. Rdna3 cache is not v-cache, just another flavor of mcm.
There's pictures of Apple's silicon all over the Internet, didn't think to bother even looking at one?
https://cdn.mos.cms.futurecdn.net/EWuFBHNeGSfhjJfoWFxL7D-102...
It's very obviously not on die. On package maybe, but definitely not on-die.
Since you're talking about die stacking i assume you're now talking about their "low end" phone parts. That's a quite different technology from V-cache, foveros, or even Apple's "UltraFusion." It's also not at all a unique technology to Apple, it's very common in the phone SoC space where power consumption is so limited that the thermal issues are mostly negated.
It's alreay on par.
p.s.: I don't care about power consumption :)
Clearly! I don’t know if I’d say it’s “on par” if an Apple A-series part (i.e., low power enough to be shipped in iPhones and not in any of their “serious” computers) is actually getting slightly higher numbers than the fastest, most power hungry core that AMD has right now!
It isn't. The A17 Pro gets 7199 on GB6 multi-thread compared to 18778 for the 7950X, and multi-thread workloads are the only thing that will cause the 7950X to use its full TDP. If you give the Apple chip a desktop's power budget, the single-thread number barely changes because that isn't what sets the TDP.
And Zen4 is punching above its weight because it's on TSMC 5nm compared to the A17 Pro on TSMC 3nm. Try comparing the chips that use the same process.
But how many cores does the A17 have versus the 7950X?
The reason it uses more power is that it's less efficient. The reason it has higher multi-core performance is that it has four times the number of cores.
Apple A17 Pro: 6 cores (4 efficiency and 2 performance)
Refs: https://www.techpowerup.com/cpu-specs/ryzen-9-7950x.c2846 https://en.wikipedia.org/wiki/Apple_A17
There is no 'serious' computers coming out of Apple for quite some time now - just glorified consumption devices :)
Can Apple sillicon run Dota2@200fps or Visual Studio (Not CODE!) 2022+ as good as an x86? Can I put a 7900XTX on a Mac and run Windows natively ?
I know how to cool CPUs such that they can run under load for days and weeks and be relatively quiet.
The point is: the m series chips get you a crap ton of performance with utter silence. The m2 pro Mac minis — amazing. No fans whatsoever.
Some professionals have workloads that require more multi core perf (not to mention proper Linux support). And running those dead silent is trivial.
1. Get a usb extender and a fiber optic displayport extender.
2. Stick your computer in the garage, run the cables to whatever room and hook up your monitor and peripherals.
3. Enjoy as perfect silence and low heat generation as can be possibly be achieved. (Monitors generate heat and sometimes electrical noise, and mice and keyboards are what they are)
At the end chose the 7950X, super happy about it.
Absolute beast.
The 7950X, though, is on 5nm.
You should, considering it's really the only bottleneck to getting more perf
I have seen this over and over again and it is tiring. We are discussing ISA, uArch ( and possible Node as well ) in a ChipandCheese article. Not a CPU comparison / gamer review site.
[0] https://support.apple.com/en-us/HT201897
There are definitely less efficient chips you could bring into the comparison fold, but the M2 is a mobile chipset. You need another mobile chip if you want to draw more interesting power consumption comparisons.
[0] https://www.notebookcheck.net/Framework-Laptop-13-5-Ryzen-7-...
[1] https://support.apple.com/en-us/HT201897
What I ultimately want, is a collection of SBCs that are able to be remotely booted so I can have Intel and AMD x86 systems, ARM, Risc-5, Apple that can spin up on demand.
Intel has vPRO and AMD has AMD DASH for remote power on. A Mac Mini won't have remote power on, but if you're willing to just keep it slept, Wake-On-LAN is an option.
If your goal is to go cheap but still have decent performance it depends on what exactly you need it to perform at but mini PCs like Beelink usually give otherworldly performance vs what you'd expect in the 100-200 dollar category.
https://browser.geekbench.com/v6/cpu/2986403
I can probably squeeze a bit more out if I boot to linux and cap the TDP for the MT tests.
I'm not saying the Apple silicon isn't impressive, but the 7640U (or any Phoenix silicon) isn't positioned as premium silicon like Apple's stuff. AMD would be using a wider bus, more on-die L3, faster packaged RAM, their TSV L3 or something like that if it was.
I am running good RAM, and some laptops run soldered LPDDR5X which is also very good. DDR5 SODIMMs are unfortunately not so good, as they need 1.35V to run at awful timings/speed.
https://www.anandtech.com/show/21069/modular-lpddr-becomes-a...
Honestly I hope they come to desktops and servers too. The performance of full size ddr5 dimms seems OK for now, but it must be close to its limit if SODIMMs are already past their limit.
Zen 4, AMD 7800X3D, 5.04Ghz, Single Core GB6 Score @ 2833.
Apple A17 Pro, 3.8Ghz, Single Core GB6 Score @ 2914
The reason why Phoenix were chosen was because of power saving and TDP figures. If 7800X3D were using 28W on single core, the A17 Pro was using ~5W.
Look: Apple fabs good cores. As a matter of fact, they're pretty much the only company even attempting to make ARM cores. They have lots of money, and good designs on dense silicon usually equals fast chips.
This is a brutally facetious comparison though. Nevermind the obvious ARM/x86 disparity, or even the node difference in fabrication; you're not even comparing mobile cores to each other. What kind of serious comparison takes desktop wattages and compares them to a smartphone SOC?
If you take the chip the parent mentioned (7640U) instead of picking a random desktop option, the gap closes considerably. 7640U scores 2559 in single-core, which scales very well for multicore workloads. Given that it's a 6-core chip with a 30w max TDP, a single core probably couldn't draw more than 5w either.
Zen 4, AMD Ryzen 5 7640U, TSMC 4nm, Single Core GB6 Score 2340
Apple A17 Pro, TSMC 3nm, Single Core GB6 Score 2914
https://news.ycombinator.com/item?id=37715082
It's not relevant for single threaded code, but then neither is doubling the core count from 4 to 8 or 16.
One cause may be that with just two threads the chance that both get blocked on a memory access is pretty good.
x86 shows its age and accumulated cruft here.
RISC-V has test and jump in a single instruction, doing away with flags.
The benefit would be only a tiny improvement in code size, and that's restricted to when the compiler can be absolutely certain that the target CPU is going to be modern enough to support it. So, in practice, for many, many years, this benefit would be almost never reaped.
This is not a new topic in the design of ISAs and their implementations in silicon. Idiom recognition and optimization has long existed in x86, ARM, and I'm sure other old ISA implementations. RISC-V can avoid it by adding dedicated instructions because it's just getting started (relatively speaking), but even so, I'm sure there will be opportunities for idiom recognition and optimization that RISC-V implementers will take.
On RISC-V's fusion idioms, it's not just recognition; they're designed in. Compilers are meant to try and produce them reliably, as if they were a dedicated single opcode.
x86 encoding space already has a lot of clutter in it.
The cost of low code density is that less code fits in cache. This cost is paid in performance, power, area or a combination of them.
x86 is, by many metrics, not a good ISA.
Its value is derived from the software moat, accumulated since the late 1970s.
Thus it isn't immutable. Opcodes can be added and removed.
Yet breaking compatibility with old code would destroy x86's value.
At the expense of code density, and thus hurting PPA (Power, Performance and Area).
Wildly variable opcode length does also complicate the decoder considerably, as it has to find where instructions start and end, an inherently serial operation, and yet decode enough of them to feed an n-wide pipeline.
RISC-V is newer than macro-op fusion, and could account for it possibly being present in implementations.
I do not remember ever seeing the designers of RISC-V actually push macro-op fusion.
The Renewed Case for the Reduced Instruction Set Computer: Avoiding ISA Bloat with Macro-Op Fusion for RISC-V
https://people.eecs.berkeley.edu/~krste/papers/EECS-2016-130...
Yet, it does not push macro-op fusion in implementations. It can either be done, or not done. But RISC-V creates opportunities where it is possible and not harmful to do so, while increasing code density.
>Exploiting this fact, the RV64G and RV64GC effective instruction count can be reduced by 5.4% on average by leveraging macro-op fusion.
It is up to the architects behind implementations to decide when and if to take advantage of the opportunities created by RISC-V's design to do macro-op fusion.
Yes it does. It pushes it as an alternative to adding instructions to the ISA. It says we don’t need these extra instructions because architects can use macro-op fusion to remove the penalty associated with having two instructions rather than one.
They are correct in saying that. x86 didn't have the chance to do so, because back when it was designed, macro-ops were not considered.
Yet whether to fuse or not is still up to microarchitecture architects. RISC-V doesn't push it. Only creates the opportunity where sensible, while increasing code density which benefits everyone.
With RISC-V's current design, you can either not do fusion and eat a penalty, or you can do fusion.
Different ISAs may have a greater or lesser need for fusion, and greater or lesser associated penalty. It seems a little obtuse to insist the ISA doesn't encourage fusion, when other ISAs would avoid a structural penalty for not having fusion, that RISC-V intentionally doesn't try to avoid at all.
Only a small portion of existing RISC-V hardware uses fusion.
This should tell you all you need to know about this mistakenly inferred "penalty" and whether RISC-V forces macro-op fusion into anybody.
As per why fusion helps even when not leveraged: Not allocating encoding space to a unified opcode does, in average over a large body of code, increase code density, as shown in the paper.
This means more code fits in cache, and in hardware this does weight far more than instruction count, which remains low and highly competitive even where fusion isn't leveraged.
Of course, a slow core can easily afford to not have any fusion, but that's a boring question with an obvious answer. The only situation where it makes sense to ask is in the context of a fast core. I think it's pretty clear that competitively-fast RISC-V cores will need significant amounts of fusion.
Anyway, in the context of the original post of cmp+branch, this doesn't even matter, as "separate cmp & branch, fused together if perf desired" and "separate add & load, fused together if perf desired" are the exact same kind of thing, so there's no reason to call one "accumulated cruft" and the other good design. (ok, there are some major problems with x86's flags, but that's primarily just because many operations needlessly update them; other than that, they're just an alternative way to achieve a thing)
I’ll leave others to judge if the quoted paper pushes it and move on.
Macro-op fusion is entirely optional, leveraged by few hardware implementations; Fusion is useful mostly (i.e. exceptions exist) only on mid-range cores.
High end cores prefer to split everything up (unnecessary on RISC-V), whereas low end cores prefer simpler hardware.
Net win in code density weights more in hardware than instruction count, which remains excellent in RISC-V even when the studied microarchitecture does not implement fusion.
(Then RISC-V goes and introduces a flags register for floating-point ops...)
Flags for FP aren't particularly nice, but without them you just can't implement <fenv.h> which software may be expecting. (of course using a global flag for semantics isn't particularly nice either, but hey that's what's been standardized in C)
Sure.
>doing away with temporary register usage
Dubious, as that's up to implementation either way.
A microarchitecture might split it into a load and an add, eliminating any instruction count reduction advantage.
Whereas another microarchiture might take a separate load and add and fuse them, which would free opcode space to be used more effectively elsewhere, resulting in overall code size reduction.
It's a careful balance.
Sounds reductive, but I think oems don’t see much demand for an architecture that implements both. So they’ll go with one or the other.
DDR based memories are super-cheap in comparison to implement in hardware. Also another aspect is that the SoC needs to support HBM memory, most chips of yesteryears simply have DDR<n> support only.