The article was a bit light on the interconnect information: It's called the TH Express-2 (proprietary) and from this report[1]:
> The TH Express-2 uses a fat tree topology with 13 switches each of 576 ports at the top level. This is an optoelectronics hybrid transport technology. Running a proprietary network. The interconnect uses their own chip set. The high radix router ASIC called NRC has a 90 nm feature size with a 17.16x17.16 mm die and 2577 pins. The throughput of a single NRC is 2.56 Tbps. The network interface ASIC called NIC has the same feature size and package as the NIC, the die size is 10.76x10.76 mm, 675 pins and uses PCI-E G2 16X. A broadcast operation via MPI was running at 6.36 GB/s and the latency measured with 1K of data within 12,000 nodes is about 9 us.
Other Chinese computers were plagued by reliability issues and poor IO performance, mainly due to unwarranted focus on LINPACK performance. We shall see if they fixed these problems.
The network looks relatively anemic at first glance. I would guess that this will be a LINPACK monster, but it'll be difficult to scale applications that aren't pure linear algebra up to the point that justifies a single machine this size.
Interesting that the Chinese have adopted x86 for their supercomputers; I remember a few years ago they were experimenting with MIPS and there was a huge effort to make it the "national ISA" - and lots of tablet and media player SoCs came out with MIPS cores (still many today, but they seem to be gradually shifting into ARM instead.)
Tianhe 2 has some Chinese SPARC cores in it too but it looks like they're just being used for control/management functions.
According to wikipedia [1], they planned to use their Loongson MIPS chips for some of the Dawning supercomputers, but later reconsidered. I'd guess using Loongson MIPS didn't make any sense given its poor performance compared to Intel CPUs.
It will take quite some time to get to a modern CPU design that can compete with Intel but it seems Loongson development continues even without being able to produce comptetive products for now. For example, Lemote.com seems to be selling Loongson 3A server blades [2]. For HPC applications Loongson 3B would have much better performance due to implementing 256-bit vector instructions (maybe similar to intel AVX), I wonder whether these can be readily bought anywhere in china. Loongson 3C would bring quite some performance boost by finally switching to a more modern process technology (28nm instead of 65nm), but google doesn't show any results that'd indicate it's produced already.
One really cool part of this supercomputer that is left out of this article is that it uses the Intel Larabee/Phi co-processor, at least 48000 of them.
6 comments
[ 12.0 ms ] story [ 155 ms ] thread> The TH Express-2 uses a fat tree topology with 13 switches each of 576 ports at the top level. This is an optoelectronics hybrid transport technology. Running a proprietary network. The interconnect uses their own chip set. The high radix router ASIC called NRC has a 90 nm feature size with a 17.16x17.16 mm die and 2577 pins. The throughput of a single NRC is 2.56 Tbps. The network interface ASIC called NIC has the same feature size and package as the NIC, the die size is 10.76x10.76 mm, 675 pins and uses PCI-E G2 16X. A broadcast operation via MPI was running at 6.36 GB/s and the latency measured with 1K of data within 12,000 nodes is about 9 us.
[1]: http://www.netlib.org/utk/people/JackDongarra/PAPERS/tianhe-...
Tianhe 2 has some Chinese SPARC cores in it too but it looks like they're just being used for control/management functions.
It will take quite some time to get to a modern CPU design that can compete with Intel but it seems Loongson development continues even without being able to produce comptetive products for now. For example, Lemote.com seems to be selling Loongson 3A server blades [2]. For HPC applications Loongson 3B would have much better performance due to implementing 256-bit vector instructions (maybe similar to intel AVX), I wonder whether these can be readily bought anywhere in china. Loongson 3C would bring quite some performance boost by finally switching to a more modern process technology (28nm instead of 65nm), but google doesn't show any results that'd indicate it's produced already.
[1] https://en.wikipedia.org/wiki/Dawning_Information_Industry#D...
[2] http://www.lemote.com/products/computer/hongri/346.html
[3] http://www.eetimes.com/document.asp?doc_id=1257218
http://en.wikipedia.org/wiki/Larrabee_(microarchitecture)