Google AI Infrastructure Supremacy: Systems Matter More Than Microarchitecture (semianalysis.com)

Really good article. Interesting to see how even Google is scrambling to keep TPU architectures up to date enough to be relevant for the latest model types.