The OS I work on currently supports a maximum of 254 logical cpus. If we ran on the 9754 and had to ignore two entire threads, we'd be giving up a maximum of 0.78% performance. I remember being excited about the first dual core systems not that long ago... Makes me wonder how long this trend of just packing more cores into the chip can continue, and where it will go after that. There seemed to be a fundamental shift between the Pentium 4 era and multi-core. Then between 8 core and many core. Big/Little also working its way in there. What's the next major innovation that's going to drive this ever-increasing performance limit? Or is there a theoretical maximum a single chip can do?
The cuda cores of Nvidia GPUs are closer to fp32 units in vector ALUs than CPU cores capable of operating independently in parallel. Following that definition a modern CPU core would have dozens of "cuda cores" as well (although far fewer than GPUs optimized for that kind of workload)
More comparable would be the ~130 streaming Multiprocessors of a H100.
The fact that each cuda core also has its own instruction pointer is quite misleading. You would think that this lets it run different instructions per cuda core but the opposite is the case. The driver uses these instruction pointers for finer scheduling granularity. That is cool but is not the same.
In the NVIDIA peculiar language, which seems to have been chosen with the only purpose of causing confusion, because all the names invented by NVIDIA are applied to things that already had traditional names for many decades, warp means thread and thread means SIMD lane.
NVIDIA has never given a good explanation about what they mean by the "instruction pointer" that belongs to each NVIDIA "thread". It certainly does not mean what in means normally, i.e. a special register that contains the address from where the next instruction will be fetched for execution. I believe that this "instruction pointer" refers to a register where the actual instruction pointer is saved when a "thread" is stalled because it has diverged into two branches after a condition test and only one of the branches continues to be executed, while the other branch must be executed later, with the complementary predicate.
These saved instruction pointers are presumably used for scheduling the "threads" to be executed by the SIMD lanes provided by the hardware, in such a way as to satisfy the cross-lane dependencies.
> Or is there a theoretical maximum a single chip can do?
There's a point where you won't be able to pack more transistors on a chip, but if Cerebras' work is an indication, chips themselves can get quite large.
But remember these chips are not designed with desktop users in mind. These chips are more similar to the ones inside very large computers that are built to host hundreds of simultaneous virtual machines on behalf of hundreds of users.
Sadly, we won't see desktops based on these unless we custom order them.
I can't help but think this will be king of Virtual Server hosting for the next couple years. At that point, who knows what comes next, I know Intel is betting on custom accelerators and NVidia is pushing for GPU-like AI parallelism. Between AMD and other ARM options, who knows in a few years, but this is a pretty great option in the near term.
One thing we can be sure is that, if there's money to be made, someone will try to make it. And Intel will eventually get the process house in order, so we can expect them to come up with something.
I'd also love to see software like Firefox trying to leverage these abundant puny cores, because the average corporate laptop now does 8 threads or more and that number is only going to get higher, with mixed beefy/puny cores.
AFAIK, the Zen4c are full cores, just smaller node size, not big/little per Intel. I'm not sure where the limit will hit on core utilization in most common applications, which are relatively entrenched with legacy code bases.
Zen 5c, although the ~25% increase in per Core improvement doesn't seems that interesting in these kind of workload. Until AMD could fit 256 Core in a Single Socket. Definitely achievable with 2nm, although we are looking at 2026+.
I mean in terms of a "best" option for many/most data center applications. Right now, these 128 Zen 4c server processors are actually really close to ARM in terms of per core power usage, so who knows if ARM will still even have that advantage in production in 3-5 years as these servers start to cycle out.
Also worth noting that AMD could drop in ARM chiplets as well as x86, so there is some flexibility there as well if there's enough demand. As it stands, this will bring a lot of compute density for a lot less power draw than previous generations of servers.
14 comments
[ 2.9 ms ] story [ 51.0 ms ] threadLatest GPUs have 10,000+ CUDA Cores, so, if you can paralyze the work, evidently pretty damned far.
Of course we'll need ever faster versions of PCI-E to come out to feed these beasts with enough data they don't sit idle most of the time.
More comparable would be the ~130 streaming Multiprocessors of a H100.
https://stackoverflow.com/questions/58071834/why-does-each-t...
NVIDIA has never given a good explanation about what they mean by the "instruction pointer" that belongs to each NVIDIA "thread". It certainly does not mean what in means normally, i.e. a special register that contains the address from where the next instruction will be fetched for execution. I believe that this "instruction pointer" refers to a register where the actual instruction pointer is saved when a "thread" is stalled because it has diverged into two branches after a condition test and only one of the branches continues to be executed, while the other branch must be executed later, with the complementary predicate.
These saved instruction pointers are presumably used for scheduling the "threads" to be executed by the SIMD lanes provided by the hardware, in such a way as to satisfy the cross-lane dependencies.
It really depends on the cache and memory architecture. You need to be able to feed data to the CPU efficiently.
There's a point where you won't be able to pack more transistors on a chip, but if Cerebras' work is an indication, chips themselves can get quite large.
But remember these chips are not designed with desktop users in mind. These chips are more similar to the ones inside very large computers that are built to host hundreds of simultaneous virtual machines on behalf of hundreds of users.
Sadly, we won't see desktops based on these unless we custom order them.
And STH recently posted an article about how they're used: https://www.servethehome.com/100m-usd-cerebras-ai-cluster-ma...
I'd also love to see software like Firefox trying to leverage these abundant puny cores, because the average corporate laptop now does 8 threads or more and that number is only going to get higher, with mixed beefy/puny cores.
Zen 5c, although the ~25% increase in per Core improvement doesn't seems that interesting in these kind of workload. Until AMD could fit 256 Core in a Single Socket. Definitely achievable with 2nm, although we are looking at 2026+.
Also worth noting that AMD could drop in ARM chiplets as well as x86, so there is some flexibility there as well if there's enough demand. As it stands, this will bring a lot of compute density for a lot less power draw than previous generations of servers.