We're two data center networking engineers who've spent years designing and operating data center infrastructure for governments, telcos and banks in West Africa. This piece came out of our work on a new AI architecture based on associative memory rather than transformers. The GPU-free argument here is something we think about the next phase of AI networking. Happy to discuss further about it.
PS: Taking a look at our manifesto (https://almartis.xyz/) can help with more context.
Yes, read that. What these people are talking about seems to replacing training of NNs by something else entirely. The big question is, does that work? At all?
It's premature to discuss network architecture until that basic question is answered.
> For the past few decades, building a datacenter has been a well-understood, predictable exercise in utility engineering.
> In modern AI clusters, the network is no longer just infrastructure sitting beneath compute
It always make me smile when someone is presenting these kind of topologies as "New", "Modern A.I" or anything remotely "Revolutionary".
The HPC domain and any decent supercomputers have been doing RDMA networking centered around "all-to-all" and "all-reduce" operations for at least 3 fucking decades now.
They are the main reasons supercomputers are almost always constructed around stupidly complex Torus or Dragonfly network topologies.
MPI itself has these primitives defined from v1.
The only difference now is that it switch from "This niche thing 3 nerds were using for weather simulations" to "this cool thing any hyperscaler NEED to have for *A.I*"
Datacenters are being built for AI. What happens when you remove the AI workload?
Don't get me wrong. I don't mind when some tech bros burn billions of venture capital & nothing much (?) comes out of it.
But those datacenters embody a lot of resources. Raw materials, complex/resource heavy manufacturing processes for IC's, servers, networking gear, etc etc.
I sure hope that doesn't go to waste when the AI bubble pops. Datacenter stuffed with AI optimized hardware any good for general engineering? Science projects? Weather prediction? Web hosting? ...??
>This allows us to flatten the physical datacenter into a GPU-free, non-blocking, 1-tier full mesh architecture built around high-density CPU nodes and 51.2Tb silicon switching fabric.
I wonder how many CPUs will be required to do the job of one Nvidia H200 GPU.
9 comments
[ 2.8 ms ] story [ 23.9 ms ] threadPS: Taking a look at our manifesto (https://almartis.xyz/) can help with more context.
It's premature to discuss network architecture until that basic question is answered.
> In modern AI clusters, the network is no longer just infrastructure sitting beneath compute
It always make me smile when someone is presenting these kind of topologies as "New", "Modern A.I" or anything remotely "Revolutionary".
The HPC domain and any decent supercomputers have been doing RDMA networking centered around "all-to-all" and "all-reduce" operations for at least 3 fucking decades now.
They are the main reasons supercomputers are almost always constructed around stupidly complex Torus or Dragonfly network topologies.
MPI itself has these primitives defined from v1.
The only difference now is that it switch from "This niche thing 3 nerds were using for weather simulations" to "this cool thing any hyperscaler NEED to have for *A.I*"
Don't get me wrong. I don't mind when some tech bros burn billions of venture capital & nothing much (?) comes out of it.
But those datacenters embody a lot of resources. Raw materials, complex/resource heavy manufacturing processes for IC's, servers, networking gear, etc etc.
I sure hope that doesn't go to waste when the AI bubble pops. Datacenter stuffed with AI optimized hardware any good for general engineering? Science projects? Weather prediction? Web hosting? ...??
I wonder how many CPUs will be required to do the job of one Nvidia H200 GPU.