Deep dive into Azure Maia 300 architecture (azure.microsoft.com)

As soon as you scale out to multiple machines, you're already thinking about non-uniform access. If you just push that down one layer further, I don't see all that much of a difference if you're already having to do the work in your software to deal with it.