While this will probably cause a certain amount of hand-wringing in the US, it's nothing to be too concerned about. The bragging rights for "World's Fastest Computer" are pretty cheap, even cheaper than the bragging rights for "World's Tallest Building", so you can expect 'em both to be fought over by secondary countries with something to prove.
All I know about supercomputers based on GPUs is that they're very cheap, very theoretically fast, and very hard to program for effectively. Any experts in here who can enlighten us?
They're also not amenable to all problem types. If, for example, your problem requires performing relatively light computations on large amounts of data, there is not much data reuse, and you constantly require new answers, GPUs aren't a good fit [1].
One simple question to ask is: how many computations am I going to perform on each element I transfer? For example, if you're doing a matrix multiplication of NxN matrices, each element will be used in 2N computations. (I think. Someone check my mental analysis.) So, matrix multiplication is a pretty good fit. But what if you're just summing a vector of size N? Then each element will be used once. Probably not a good fit.
GPUs are good for algorithms with simple control flow (take conventional rendering where you basically does simple processing of huge amounts of vertices and pixels). If you have an algorithm with complex control flow (e.g. ray tracing, most problems in science) you will have to code really hard to achieve even 50% of peak. You can usually do something with an algorithm which is good for GPU but not optimal overall. So you have 30x speedup on GPU compared to CPUs but the scaling of this algorithm is worse then its more complex variant on CPU (e.g. O(n) vs O(log(n))). In the end many of these speedups are just marketing.
Moreover GPUs have problems with double precision floating point numbers which are a must for most scientific problems (though sometimes you can utilize mixed precision approaches -> estimate in single precision, then get a correction in double).
Another problem is having ECC (error correction). You don't want to run a computation for a week to find out that it crashed because of data corruption. Latest generation of Teslas has ECC, consumer cards don't.
I work in an HPC facility. We are all a bit skeptical of LINPACK running on a hybrid system.
LINPACK in general is an interesting benchmark. We recently bought a new cluster. We were faced with an interesting decision. Get 4,000 cores with Infiniband, giving us a spot on the top500 list (We've been on there for the past 5 years) or get a 6,000 cores with gigabit ethernet, losing our spot on the top500. (We get %90 of our theoretical max over infiniband. Gigabit drops that down to 50%). We did a survey among our users and very few of them ran multinode jobs. Most used 8 or 12 threads, which runs fine on a dual socket hex core westmere. So we dropped infiniband. So even though we technically have a slower cluster than we could have had, we have more efficient utilization and are able to offer our users more resources.
Since very few comercial applications can currently take advantage of a hybrid system, it seems to me this cluster was built with the top500 in mind, rather than being efficiently used. As far as useful results, I wouldn't be surprised if it only accomplished 2 or 3 times what we do, and we're a 10,000 core shop that probably won't be on the next top500.
Adding GPUs seems a no-brainer in retrospect. Have top500 institutions outside China been trying this approach?
Except one thing. Until recently GPUs have only done single-precision calculations. Double precisions has been added with the latest architectures but double is, I believe, still very slow. So this means that single precision results are sufficient for the top500 tests as well as the particular applications (in the Tianhe’s case: weather and oil exploration)?
Very cool, and good on China for developing this. However, I would imagine that it matters less in the modern era of distributed computing how powerful the biggest computer is, but how powerful the biggest group of computers is. Isn't folding@home up to about 6 petaflops now?
This list and China's place on it are both irrelevant for the following reasons:
1. Any entrant that extensively relies on GPU's for its performance should not even be on the list or at least appear with an asterisk. The "G" in GPU does not stand for general. Consequently this machine is principally applicable to specific problem sets that are designed to take advantage of the processing power provided by the GPU.
2. Primary attention in the supercomputing space has shifted focus from using pure flops as a metric to flops per watt as the primary concern. Beyond that, focus has also shifted towards interconnect technology to keep all those flops well fed with data.
Just like the Chinese to be a few steps behind the thought leaders but front and center for biggest and most easily duplicated.
25 comments
[ 3.2 ms ] story [ 49.8 ms ] threadhttp://news.ycombinator.com/item?id=1843248
http://news.ycombinator.com/item?id=1841807
http://news.ycombinator.com/item?id=1844338
Edit: According to the PCMag article they're Xeon processors.
All I know about supercomputers based on GPUs is that they're very cheap, very theoretically fast, and very hard to program for effectively. Any experts in here who can enlighten us?
Then any moronic theoretical physicists/geophysicist/industrial chemist can just use MPI, LINPACK,Nag to do useful work on it
One simple question to ask is: how many computations am I going to perform on each element I transfer? For example, if you're doing a matrix multiplication of NxN matrices, each element will be used in 2N computations. (I think. Someone check my mental analysis.) So, matrix multiplication is a pretty good fit. But what if you're just summing a vector of size N? Then each element will be used once. Probably not a good fit.
[1] http://people.cs.vt.edu/~scschnei/papers/debs2010.pdf
Moreover GPUs have problems with double precision floating point numbers which are a must for most scientific problems (though sometimes you can utilize mixed precision approaches -> estimate in single precision, then get a correction in double).
Another problem is having ECC (error correction). You don't want to run a computation for a week to find out that it crashed because of data corruption. Latest generation of Teslas has ECC, consumer cards don't.
LINPACK in general is an interesting benchmark. We recently bought a new cluster. We were faced with an interesting decision. Get 4,000 cores with Infiniband, giving us a spot on the top500 list (We've been on there for the past 5 years) or get a 6,000 cores with gigabit ethernet, losing our spot on the top500. (We get %90 of our theoretical max over infiniband. Gigabit drops that down to 50%). We did a survey among our users and very few of them ran multinode jobs. Most used 8 or 12 threads, which runs fine on a dual socket hex core westmere. So we dropped infiniband. So even though we technically have a slower cluster than we could have had, we have more efficient utilization and are able to offer our users more resources.
Since very few comercial applications can currently take advantage of a hybrid system, it seems to me this cluster was built with the top500 in mind, rather than being efficiently used. As far as useful results, I wouldn't be surprised if it only accomplished 2 or 3 times what we do, and we're a 10,000 core shop that probably won't be on the next top500.
Lawrence Livermore is already planning the next "World's Fastest Computer"
Except one thing. Until recently GPUs have only done single-precision calculations. Double precisions has been added with the latest architectures but double is, I believe, still very slow. So this means that single precision results are sufficient for the top500 tests as well as the particular applications (in the Tianhe’s case: weather and oil exploration)?
I'm sure Google could claim to have the largest "computer" if they wanted to, or if the definition of "supercomputer" were stretched a bit.
Yeah, I only use bittorrent to download linux distros as well.
1. Any entrant that extensively relies on GPU's for its performance should not even be on the list or at least appear with an asterisk. The "G" in GPU does not stand for general. Consequently this machine is principally applicable to specific problem sets that are designed to take advantage of the processing power provided by the GPU.
2. Primary attention in the supercomputing space has shifted focus from using pure flops as a metric to flops per watt as the primary concern. Beyond that, focus has also shifted towards interconnect technology to keep all those flops well fed with data.
Just like the Chinese to be a few steps behind the thought leaders but front and center for biggest and most easily duplicated.