No. Some 4090s had issues with power connectors. The 4090 does not support NVLink (though there is evidence that they were designed with the capability).
They're not using the same PCBs because 4090 has the blow-through cooler and so a downsized pcb but the DC cards have big fanless heatsinks for cooling by chassis fans.
3090 had nvlink so it was probably considered for 4090 but eventually cut for some reason
I imagine the reason was that hardly anyone used it, and those that did use it were the same group of clients who would traditionally buy Quadro cards.
Is the white paper out and precise specs available for L40? Last time I looked nvlink was still with a question mark on L40? Doesn't make sense to remove the nvlink feature from the A40 line that had some nvlink, that would piss off a bunch of users hoping to upgrade to not-H100...
Your question is understandable given the hysteria about those connectors, but they're power connectors and it seems there were only ~50 cases in total, mostly down to the connector not being inserted fully (with a handful due to manufacturing defects).
NVSwitch looks around 1/4th the size of a contemporary Ethernet switch so it may be only ~100 W which is pretty small relative to the GPUs it's connecting.
I think NVLink is a memory interconnect more like NUMAlink, UPI, or Infinity Fabric. AFAIK GPUs can transparently access each other's memory over NVLink (probably at cache line granularity) while RDMA protocols require explicitly copying memory (usually in larger chunks) before accessing it.
I would have compared with PCIe bandwidth, which seem to drag behind actual network speeds these days. In the PCIe 3 era, Mellanox/NVIDIA sold (still sell) 200g (2x100 mostly) boards using 2 PCIe3 16x ports, linked by a specific cable (the whole montage is called SocketDirect? or sometimes PCIe 32x?).
When PCIe 4 started getting standardized you could already saturate the bus with one 200G NIC. Now ConnectX 7 is out, should be able to saturate PCIe 5 but most constructors haven't got their motherboards out. SFP112 is out and you can already saturate PCIe 6 with 2 QSFP112 (or equivalent 2x4x100G). Still need CPUs with support or GPUs with PCIe5 or 6 support, which I guess are coming or you can stream from one NIC to several GPUs.
It might be more interesting for nvidia in the mid to long run to not wait for PCIe standards to settle and to allow streaming to/from their NICs to their GPUs (or CPUs, see Grace Hopper white paper) using nvlink, never touching the PCIe bus...
RDNA3 has memory/cache chiplets providing 3.5Tbps. The 0.9Tbps of much further distance chip to chip conmevtivity here is highly impressive. Interesting to note that RDNA3 links are only 10Gbps, they just have extreme interposer design with hundreds upon hundreds of incredibly small wires (resulting in stupendously high bit-efficiency); not something a multi-GPU design can do, and hence Nvidia doing 100Gbps links.
Given where ethernet is (on the way to terabit ethernet) Im not surprised Nvidia is here, but it's still cool to see.
The onboard SHARP processor on the NVswitch is very cool to read about. Given how much ML involves crunching many matrixes/tensors then adding the results, being able to parallelize the crunching and have a central aggregator makes sense. In general I hope we see an emerging presence of extremely high-fabric connectivity cpus & gpus that can serve as DPUs like this, as central coordinators. Thinking of this less as a switch (which it certainly is too, with it's vast bank of PHYs and massive crossbar) but more as an io cemtric computer is, I think, deserved.
23 comments
[ 2.6 ms ] story [ 74.3 ms ] thread3090 had nvlink so it was probably considered for 4090 but eventually cut for some reason
When PCIe 4 started getting standardized you could already saturate the bus with one 200G NIC. Now ConnectX 7 is out, should be able to saturate PCIe 5 but most constructors haven't got their motherboards out. SFP112 is out and you can already saturate PCIe 6 with 2 QSFP112 (or equivalent 2x4x100G). Still need CPUs with support or GPUs with PCIe5 or 6 support, which I guess are coming or you can stream from one NIC to several GPUs.
It might be more interesting for nvidia in the mid to long run to not wait for PCIe standards to settle and to allow streaming to/from their NICs to their GPUs (or CPUs, see Grace Hopper white paper) using nvlink, never touching the PCIe bus...
Given where ethernet is (on the way to terabit ethernet) Im not surprised Nvidia is here, but it's still cool to see.
The onboard SHARP processor on the NVswitch is very cool to read about. Given how much ML involves crunching many matrixes/tensors then adding the results, being able to parallelize the crunching and have a central aggregator makes sense. In general I hope we see an emerging presence of extremely high-fabric connectivity cpus & gpus that can serve as DPUs like this, as central coordinators. Thinking of this less as a switch (which it certainly is too, with it's vast bank of PHYs and massive crossbar) but more as an io cemtric computer is, I think, deserved.