"Which is funny because that describes AI and you’d be doing this kind of work on a GPU which leans entirely into these advantages and not a CPU anyway"
IPC has definitely not flatlined. Zen 4 can do something like 50-70% more IPC than a CPU from 10-15 years ago. Zen 5 is capable of ~16% more IPC than Zen 4.
There's some sleight of hand being perpetrated here through the use of an inappropriate scale to trick the reader into thinking that NVMe latency is roughly zero when it's not. Let's just stipulate that you have an unreleased, hypothetical NVMe device with consistent 10µs read latency. The same device will have a write latency 10-100x higher, which it hides, under transient conditions, with DRAM. It's a very poor mental model to assume that "I/O" latency has gone to zero. It only looks that way if you're polluting your chart with disk access times from 1980, and it assumes a false symmetry between "I" and "O".
The data does not support the premise much less the grandiose conclusions at the end of the article. By pure construction memory is always going to be faster than disk, right up to the point where disk is inplemented as memory which makes this a pointless semantic discussion. Such a fact would hardly invalidate the core pillars as computer science as the conclusions imply. Waste of time.
We are seeing more SRAM in the form of CPU caches, this is one of the things that is actually scaling still.
Are you asking why not use SRAM in something like a DIMM? You could do this. Here's why I wouldn't advocate for this. Assume you had zero latency SRAM in your DIMM. It still takes ~40ns to get out of the processor by the time you go through all the memory controller and phy. So you'd have an incredibly expensive but small DIMM taking up limited pins on the processor package/die. Even then you'd only cut the memory latency in half, and we'd still be stuck at a new lower flatline.
Incorporating the SRAM in die is different story, you get to scale the latency and the bandwidth closer to the other capabilities of the cores.
In this context CXL is kinda storage and kinda higher latency RAM. It's latency is worse than standard DDR5 DIMMs but the bandwidth is on the same trajectory as storage/networking.
I'm inclined to think of it like storage in this context. It's scaling, but it will require new thinking to take full advantage of.
Ai generated slop. Constantly summarising various parts of the memory hierarchy, graphs with no x axis, bad units, no real world examples, the final conclusion doesn't match the previous 10 summaries.
The big problem is that it misses a lot of nuisance. If actually try to treat an SSD like ram and you randomly read and or write 4 bytes of data that isn't in a ram cache you will get performance measured in the kilobytes per second, so literally 1,000,000 x worse performance. The only way you get good SSD performance is reading or writing large enough sequential chunks.
Generally random read/write for a small number of bytes is similar cost to a large chunk. If you're constantly hammering an SSD for a long time, the performance numbers also tank, and if that happens your application which was already under load can stall in truly horrible ways.
This also ignores write endurance, any data that has a lifetime measured in say minutes should be in ram, otherwise you can kill an SSD pretty quick.
> Data used for the charts was researched and compiled by ChatGPT, I spot checked it and found it was accurate enough to for the narrative.
Last I checked, that wasn't how citations worked.
I only sampled one data point (2017 AMD EPYC Rome Clock rate) which was significantly off, because in 2017 it was the Naples chipset that was released, and, unless 2017 was desperate enough to clock from 2.2Ghz to 3Ghz on the regular (Boost up to 3.2Ghz), the 'research' was a fair bit off...
Doesn't undermine or contradict the authors (bots?) point, but a strange way to provide 'evidence' for an argument.
This was largely a waste of time to read. It doesn't really support its own point or honestly even make a point or provide anything actionable. I'm not even sure I can trust the numbers given it's been vibes-researched.
Relevant to this: memory cost per GB dropped exponentially every year from the days of the earliest computers until 2010 or so, reaching $4/GB in 2011. A decade and a half later it's still in the $2-$4/GB range.
Note also that SSDs started out only slightly cheaper per GB than DRAM - the 80GB Intel X25-M had a list price of about $500 when it was released in 2008, and references I find on the net show a street price of about $240 for the next-gen 80GB device in 2009. Nowadays you can get a 1TB NVMe drive for about the cost of 16GB of RAM, although you might want to spend a few more bucks to get a non-sketchy device.
Disk hardware may be faster relative to RAM, but if you're using typical serverless PaaS offerings to run a hosted application at modest operational scale, it's a heck of a lot cheaper to get a node with "normal" RAM and CPU, than it is to get decent IOPS. If you're a big iron company with big iron problems, you may need to think in different scaling terms, but for the SaaS hoi polloi the economics of hosting have somewhat preserved traditional notions of performance.
23 comments
[ 5.3 ms ] story [ 51.8 ms ] threadDamn thats harder to parse than it needed to be.
IPC has definitely not flatlined. Zen 4 can do something like 50-70% more IPC than a CPU from 10-15 years ago. Zen 5 is capable of ~16% more IPC than Zen 4.
Do I need exponential improvements and vector operations in a text editor though?
Are you asking why not use SRAM in something like a DIMM? You could do this. Here's why I wouldn't advocate for this. Assume you had zero latency SRAM in your DIMM. It still takes ~40ns to get out of the processor by the time you go through all the memory controller and phy. So you'd have an incredibly expensive but small DIMM taking up limited pins on the processor package/die. Even then you'd only cut the memory latency in half, and we'd still be stuck at a new lower flatline.
Incorporating the SRAM in die is different story, you get to scale the latency and the bandwidth closer to the other capabilities of the cores.
I'm inclined to think of it like storage in this context. It's scaling, but it will require new thinking to take full advantage of.
The big problem is that it misses a lot of nuisance. If actually try to treat an SSD like ram and you randomly read and or write 4 bytes of data that isn't in a ram cache you will get performance measured in the kilobytes per second, so literally 1,000,000 x worse performance. The only way you get good SSD performance is reading or writing large enough sequential chunks.
Generally random read/write for a small number of bytes is similar cost to a large chunk. If you're constantly hammering an SSD for a long time, the performance numbers also tank, and if that happens your application which was already under load can stall in truly horrible ways.
This also ignores write endurance, any data that has a lifetime measured in say minutes should be in ram, otherwise you can kill an SSD pretty quick.
Last I checked, that wasn't how citations worked.
I only sampled one data point (2017 AMD EPYC Rome Clock rate) which was significantly off, because in 2017 it was the Naples chipset that was released, and, unless 2017 was desperate enough to clock from 2.2Ghz to 3Ghz on the regular (Boost up to 3.2Ghz), the 'research' was a fair bit off...
Doesn't undermine or contradict the authors (bots?) point, but a strange way to provide 'evidence' for an argument.
[1] https://www.karlrupp.net/2018/02/42-years-of-microprocessor-...
Note also that SSDs started out only slightly cheaper per GB than DRAM - the 80GB Intel X25-M had a list price of about $500 when it was released in 2008, and references I find on the net show a street price of about $240 for the next-gen 80GB device in 2009. Nowadays you can get a 1TB NVMe drive for about the cost of 16GB of RAM, although you might want to spend a few more bucks to get a non-sketchy device.