(not a hardware engineer) This looks like Google's version of Infiniband, which is basically mandatory for large scale ML-training servers due to issues with Ethernet, as described in the paper. Really cool stuff, wonder if we'll see traditional network hardware manufacturers copying this design.
The latency numbers they state seem achievable or beatable with Infiniband, Amazon's EFA, or TCPDirect. 2us round-trip is achievable for very simple systems. If this kind of networking sounds good to you, you can buy it today! It's even available on AWS, Azure and Oracle Cloud (but not GCP yet AFAIK).
Latency measurements are tricky, the usual benchmarks kind of suck and aren't predictive of actual performance in real systems under load.
Given that the entire Myrinet team went to work for Google, and the InfiniPath microarchitecture can be discovered by reading the device driver and some open source code, I'm pretty sure Google's team was well aware of what has been done in the recent past.
13 comments
[ 3.2 ms ] story [ 40.7 ms ] threadGiven that the entire Myrinet team went to work for Google, and the InfiniPath microarchitecture can be discovered by reading the device driver and some open source code, I'm pretty sure Google's team was well aware of what has been done in the recent past.
I remember the Aquila TV series, in the UK, fondly. Enough that I tracked down the books to read.
It’s bizarre how much I remember of that show, none of the Latin though.
There's also a city with the name Aquila, or "L'Aquila", where L' equates to "the", in the Abruzzo region. [0]
[0]: https://en.wikipedia.org/wiki/L%27Aquila
1. https://en.wikipedia.org/wiki/Aquila_(TV_series)