Rail-Only: A Low-Cost High-Performance Network for Training LLMs with T Params

Please check this HN post on the similar subject by Meta [1].

Previous paper by the same team from Meta and MIT but with Billions instead of Trillions of parameters [2].

[1] A RoCE network for distributed AI training at scale:

[2] Optimized Network Architectures for Training Large Language Models With Billions of Parameters [PDF]:

Rail-Only: A Low-Cost High-Performance Network for Training LLMs with T Params (arxiv.org)