Pretty good; 8gb memory and impressive transfer speeds. The 980 Ti is pretty common in the Deep Learning community. Only the Titan X is more popular, and that thing is far outside any average person's budget. I'm definitely looking to pick up one of these soon.
Isn't the Titan X also about RAM? It has 12GB to the GTX 1080's 8GB. At the low end you could just buy more than one GTX 1080, so it looks like a good deal there, but at the top end you are running out of slots for cards.
Larger learning sets, essentially. I'm a neural net novice, but I think the general perspective seems to be that 8 vs 12 is not a big deal; either you are training something that's going to have to get split up into multiple GPUs anyway, or there's probably a fair amount of efficiency you can get in your internal representations shrinking RAM usage.
One thing not mentioned in this gaming-oriented press release is that the Pascal GPUs have additional support for really fast 32 bit (and do I recall 16 bit?) processing; this is almost certainly more appealing for machine learning folks.
On the VR side, the 1080 won't be a minimum required target for some time is my guess; the enthusiast market is still quite small. That said, it can't come too quickly; better rendering combined with butter-smooth rates has a visceral impact in VR which is not like on-screen improvements.
8 vs 12 is a really big difference - especially with state of the art deep learning architectures. The problem isn't the size of the dataset but of the model. Large Convolutional Neural Networks often need the whole 12 GB VRAM of a Titan card for themselves and distributing over multiple cards/machines adds a lot of complexity.
Can someone provide a quick overview of the current GPU landscape?
There seems to be Nvidia's pascal, gtx, titan, etc. Something called geforce. And I believe these are just from Nvidia.
If I'm interested in building out a desktop with a gpu for:
1. Learning computation on GPU (matrix math such as speeing up linear regression, deep learning, cuda) using c++11
2. Trying out oculus rift
Is this the right card? Note that I'm not building production models. I'm learning to use gpus. I'm also not a gamer, but am intrigued by oculus. Which GPU should I look at?
Buy a GTX 1070 if you are on a "budget" (around $350ish, comes out June 10th) or a GTX 1080 if not ($600ish, comes out may 27th). Do not buy a Titan model unless you feel you really need to work with 64 bit numbers on the GPU (32 bit floats should cover most of your cases).
Do you need FP64? If so, right now the OG GTX Titan is the default choice - it offers full double-precision performance with 6 GB of VRAM. There's nothing better south of $3000.
If not, the 980 Ti or Titan X offer excellent deep learning performance, albeit only at FP32. And their scheduling/preemption is not entirely there, they may not be as capable of Dynamic Parallelism as Kepler was. The 780 Ti is actually a more capable card in some respects.
The new consumer Pascal cards will almost certainly not support FP64, NVIDIA has considered that a Quadro/Tesla feature since the OG Titan. If DP performance is a critical feature for you and you need more performance than an OG Titan will deliver, you want the new Tesla P100 compute cards, and you'll have to convince NVIDIA you're worthwhile and pay a 10x premium for it if you want it within the next 6 months. But they probably will support compute better, although you should wait for confirmation before spending a bunch of money...
For VR stuff or deep learning, the consumer Pascal cards sound ideal. Get a 1070 or 1080, definitely. The (purportedly) improved preemption performance alone justifies the premium over Maxwell, and the FP16 capability will significantly accelerate deep learning (FP16 vs FP32 is not a significant difference in overall net output in deep learning).
The AMD Pro Duo seems aimed at the workstation/rendering market rather than the compute market.
NVIDIA is totally dominant in the compute market. They have an enormous amount of software running on CUDA that you would be locking yourself out of with AMD, and since NVIDIA has such a dominant share of the compute hardware you would also be tuning for what amounts to niche hardware.
AMD has recently been working on getting a CUDA compatibility layer working, hopefully this will improve in the future.
If you have any interest in deep learning, NVIDIA is your only option. They are miles ahead of everyone else. For deep learning the 1080 is the best option. You'll be waiting for models to train and the extra money is well worth the many hours it will save you. If you aren't serious about deep learning then get the 1070, it's more than enough for VR and CUDA experimentation. Or, if you just can't wait until June (or July/August with supply constraints probably), get a 970 now (min spec for VR).
The 1080 will probably be the best card available until next year, when HBM2 cards (~3x memory bandwidth) reach general availability. I'm hoping for a 1080 Ti or a new Titan then.
What are the characteristics of NVIDIA GPUs that make them superior for deep learning applications?
Phrased another way, if you're designing a card specifically to be good for training deep neural nets, how does it come out differently from cards designed to be good at other typical applications of GPGPU?
Nvidia's main advantage over AMD for deep learning is software, not the hardware itself. Nvidia's software advantage has lead to network effects where almost all GPU supporting deep learning libraries build on top of Nvidia. There's no Tensorflow/Theano/Torch/Keras/etc for AMD/OpenCL for the most part. Nvidia also releases high performance kernels for common neural net operations (such as matrix multiply and convolution).
On the hardware end, Nvidia has slightly superior floating point performance (which is the only thing that matters for neural nets). Pascal also contains 16 bit floating point instructions, which will also be a big boost to neural net training performance.
I would be interested to hear about the difference in floating point performance. I would have guessed that, at this point, pretty much every chip designer in the world knows equally well how to make a floating-point multiplier. So it must be that nvidia vs amd have made different trade-offs in when it comes to caching memory near the FP registers or some such thing?
I'm unsure about the floating point performance differences, but 2 other reasons for potential differences are (1) number of floating point units and (2) different clock speeds.
- GTX: label for GeForce graphics cards meant for gaming and other demanding tasks (see AMD Radeon "R9").
- Titan: special name for super high-end GeForce graphics cards. (For AMD, see "Fury".)
You might also encounter Nvidia Quadro graphics cards (comparable to AMD FirePro), which are meant for professional workstations, and Nvidia Tesla graphics cards, which target high performance computing.
I watched the livestream, looked good and love the performance/watt. 180watt card. Way more GPU power than I need professionally or for fun though. I'm actually all-in on the new Intel gaming NUCs. Skull Canyon looks fantastic and has enough GPU performance for the gaming I do anymore. Mostly older games, some CS:Go (my friends and I play at 1280x1024 anyway since it's competitive gaming) and League of Legends.
It's also nice to have an all Intel machine for Linux. I'd use a lowend NV Pascal to breathe new life into an older desktop machine since NV seems to always have a bit better general desktop acceleration that really helps out old CPUs. If building a high end gaming rig I'd probably wait for AMD's next chip. I've liked them more on the high end for a few generations now. Async compute and fine-grained preemption, generally better hardware for Vulkan/DX12. AMD is also worth keeping an eye on for their newfound console dominance, subsequent impact if they push dual 'Crossfire' GPUs into XboxOne v2, the PS4K and Nintendo NX. That would be a master stroke to get games programmed at a low-level for their specific dual GPUs by default. Also, the removal of driver middleware mess with the return of low-level APIs to replace OGL/DX11 will remove the software monkey off AMD's back. That always plagued them and the former ATI a bit.
I'll probably buy the KabyLake 'Skull Canyon' NUC revision next year and if I end up missing the GPU power, hook up the highest end AMD Polaris over Thunderbolt. Combining the 256MB L4 cache that Kabylake-H will have with Polaris will truly be next-level. Kaby also has Intel Optane support in the SODIMM slots, it's possible we'll finally see the merge of RAM+SSDs into a single chip.
But more than anything, I want Kabylake because it's Intel's last 14nm product so here's to hoping they finally sort out their electromigration problems. Best to take a wait and see on these 16nm GPUs for the same reason. I'm moving to a 'last revision' purchase policy on these <=16nm processes.
I'd assume it's because higher resolutions would offer an advantage due to simply having more pixels to aim at, and possibly higher FOV if you're using a widescreen resolution.
The player hitbox's aren't defined by what the player can see though right? So having those chunkier pixels will be less representative of what your bullets will actually hit no?
Back in the CS < 1.3 days everyone used 640x480 but I never really understood why.
it's not what your bullet hits, but how easy for the eyes to recognise that your crosshair is already aiming the right pixels. some players even use a stretched display mode (4:3 render displayed fully on 16:9 monitors) to further use this advantage.
1280x1024 is less resolution than 1080p, thus render faster, also it is the "recommended" resolution of many high end CRTs, some that can run that resolution at 160 fps ( or more even... ), with virtually zero latency, thus essentially allowing people with fast enough reflexes to use their full potential.
It's common for pros and old timer Counterstrike players. My friends and I have been PC gaming since the mid-80s so we've been in on CS since the earliest days. Here's a spreadsheet of some pro player's setups.
A little bit of everything but note how common 1024x768 is. It looks a little sharper, enough that it doesn't need AA (which I always felt introduced some strange input lag).
Guarantees good performance, makes the models slightly bigger. Gives you a slight edge. You can also run a much slower GPU, that's how I'm getting away with using stuff like Intel Iris Pro. I'd run 1280x1024 with all options on lowest even if I had a Fury X. Other people I know have GF780s and do the same. I only turn up graphics options in single player games, at which point I don't care about FPS dips. When playing competitively I want every edge I can get.
> When playing competitively I want every edge I can get.
Jesus christ, I can't even imagine the leaps in skill I'd have to make before the difference between 50fps and 150fps had any impact on my performance. Watching gaming at that level feels like being a civilian in Dragonball Z, just seeing a bunch of blurs zipping in the sky while wondering if I'm about to become obsolete.
The jump from 60 to 100 fps (even on 60Hz monitors) really does make a difference. I haven't experienced playing on 120/144Hz, but I guess it would be even better. Pros demands 300+ fps on top championships (played on 120/144Hz monitors). For me it's more on the confidence that I would not have frame drops (smoke graphics on CS:GO is bad, I drop from 90ish fps to 30-45 fps on the edges of the smoke), perceived smoothness, and input lag. The actual game itself (timeline on the servers) is only played at 64-ticks (64Hz) online or 120-ticks (120Hz) on select community servers and competitive matches.
> I can't even imagine the leaps in skill I'd have to make before the difference between 50fps and 150fps had any impact on my performance
Keep in mind fps are normally given in averages. You can go up to a wall and have nothing on screen and get an insane FPS or you can back out and take in all of the area with max view distance and see a lower FPS. Add in players/objects/explosions etc and your FPS dips and dives.
Having an average of 50FPS means you likely ARE going to notice it at certain dips while at 150 average would be much less likely to get a perceivable fps dip
My parent has been downvoted, can someone explain to me why you downvoted me? What was so offensive? Is it because I'm not a huge NV fan and would only use their cards on the lowend?
Worth noting - IIRC the stream mentioned that it could do up to 16 simultaneous projections at little additional performance cost. This is important for VR...a big part of the cost, when you are dumping many vertices to the GPU, is performing a transform on each vertex (a four component vector multiplied by a 4x4 matrix) +. Even bigger cost comes from filling the resulting polygons, which if done in two passes (as is fairly common) results in something that violates cache across the tiles that get filled. So, in other words, its expensive to render something twice, as is needed for each eye in VR - from what they have shown, their new architecture largely reduces this problem.
+ This is a "small" part of the cost, but doing 5m polygons at 60 fps can result in about 30 GFLOPS of compute for that single matrix operation (in reality, there are many vertex operations and often many more fragment operations).
I was thinking about this earlier - the transition from a warp of 32 to a warp of 64 that Pascal supposedly made sounds exactly like how you would accelerate multiple projections (by at least a factor of 4).
edit: apparently Pascal still has a warp-size of 32.
A major concept in GPGPU programming is "warp coalescing".
Threads are executed an entire warp at a time (32 or 64 threads). All threads execute all paths though the code block - eg if ANY thread executes an if-statement, ALL threads execute an if-statement. The threads where the if-statement is false will execute NOP instructions until the combined control flow resumes. This formula is followed recursively if applicable (this is why "warp divergence/branch divergence" absolutely murders performance on GPUs).
When threads execute a memory load/store, they do so as a bank. The warp controller is designed to combine these requests if at all possible. If 32 threads all issue a request for 32 sequential blocks, it will combine them into 1 request for 32 blocks. However, it cannot do anything if the request isn't either contiguous (a sequential block of the warp-size) or strided (a block where thread N wants index X+0, N+1 wants thread X+0, X+2N, etc). In other words - it doesn't have to be contiguous, but it does have to be uniform. The resulting memory access will be broadcast to all units within the warp, and this is a huge factor in accelerating compute loads.
Having a warp-size of 64 hugely accelerates certain patterns of math, particularly wide linear algebra.
edit: apparently Pascal still has a warp-size of 32.
Wow, memory access on NVidia is pretty bad. AMD has a separate unit that coalesces memory requests and goes through cache so if you do strided loads, for example, the next load will likely be reading data cached by the previous one and it does not matter how many lanes are active. AMD has 64-wide "wraps" btw and it does not seem superior to NV in computation on the same number of ALUs.
I did my grad research on disease transmission simulations on GPUs, so this is super interesting to me. Could you please hit me with some papers or presentations?
The NVIDIA memory model also goes through L1 cache - but that's obviously not very big on a GPU processor (also true on AMD IIRC). Like <128 bytes per thread. It's great if your threads hit it coalesced, otherwise it's pretty meaningless.
At glance there is a lot of legacy stuff so I'd look at anything related to GCN, Sea Islands and Southern Islands. Evergreen, R600-800 etc are legacy VLIW ISA as far as I know.
A friend of mine will be starting an epidemiology grad program this fall. Do you have some good basic pointers on the use of GPUs in the field? Also .. what is your opinion of the field in general?
the hardware performs some coalescing, but it's complicated...
memory accesses in a warp do not necessarily have to be contiguous, but it does matter how many 32 byte global memory segments (and 128 byte l1 cache segments) they fall into. the memory controller can load 1, 2 or 4 of those 32 byte segments in a single transaction, but that's read through the cache in 128 byte cache lines.
thus, if every lane in a warp loads a random word in a 128 byte range, then there is no penalty; it's 1 transaction and the reading is at full efficiency. but, if every lane in a warp loads 4 bytes with a stride of 128 bytes, then this is very bad: 4096 bytes are loaded but only 128 are used, resulting in ~3% efficiency.
Thanks, this is about how I imagined all GPUs work. With most games using interleaved vertex buffers it would be a very strange decision to penalize this access pattern.
It might, but most likely it won't: splitting position data from the rest of vertex had been around since, at least, PS3 (for the similar reasons of culling prims) and yet how many games have done the Edge's tri-culling in that generation? And even if you go ahead and split off position the rest of the vertex is still better be interleaved.
It wouldn't because as you say the ALUs are the same, it just would change the unit that the warp scheduler deals with (and relevant changes to kernel occupancy that it would entail).
the warp scheduler can dual-issue instructions; it has to because otherwise 192 fp32 alus can't be used by the 4 warp schedulers (4 x 32) effectively. increased ipc can do this for you. this was maxwell though; the ratio of resources to schedulers to sm has changed a lot in pascal.
That's exciting as well, but for whatever reason I've always personally found multi-monitor gaming to be a little annoying (even with compensated projection). Probably ok if you have super small monitor bezels though.
I'm hopeful this will provide a significant speed up for my 3D modeling / rendering. The number of cuda cores is only slightly higher than the 780. I'll definitely wait for more benchmarks.
There are data-dependencies in most AES modes (except ecb) which prevent parallelization.
If you however run 2k separate AES encryption streams you could expect a nice performance increase.
But for a single stream there is little performance to be had.
We run tens of thousands of streams in parallel.. So that's not a problem. :)
I'm just realizing that GPUs could be a viable alternative to specialized (eg, expensive) encryption offload engines or very high end (eg, expensive) v4 Xeons with many, many cores. However, it is rather hard to find data on the bandwidth at which GPUs can process a given cipher. I'm just now starting to look into this, so I could very well be mis-understanding something,
Throughput 100% depends on how hard they need to hit memory. GPUs love stuff that they can do entirely within registers. Having to hit (the limited quantity of) shared memory slows things down much more. Inefficient/non-coaslesced use of shared memory or global memory too often can trash performance. Hitting host/CPU memory in any non-trivial amount usually dooms a program.
With each step you not just cut down your bandwidth but also you increase your latency and that has a huge impact.
Also would data-dependencies be different on a GPU than with parallelization on a CPU? Because modes like GCM (i.e. CTR) both the encryption and decryption are parallelizable (with CBC it's just decryption).
I wonder if AMD GPU's would be better suited to the task than Nvidia's (at least their existing GPU's) provided you could actually saturate the GPU with enough work to make it worth it.
Bit math (bfe.*, logical bit ops, ...) are nowhere near as fast as float32 ops on their GPUs by a factor of 2-4x; there are many fewer functional units for them. Maxwell doesn't have support for many 64-bit word bit ops as well (they get emulated in SASS using the operations that they do have; e.g. bfe bitfield extract), so you'd have to do it in uint32.
Issue rate is in the CUDA documentation for Maxwell (cc 5.2) at least, presumably they'll update with Pascal soon:
Where they would win is in memory bandwidth for intermediate results (e.g., in/out of registers or smem or even global memory), assuming that you aren't ultimately bound by I/O over the bus from the CPU.
Towards the end of the article in the table overview, they list a `NVIDIA GP100` model with a memory bus width of 4096 bit. It is still shared memory, but considering bcrypt only requires a 4k memory working set, that now fits into 8 cycles instead of the 128 of 256 bit bus architectures...
Am I wrong to think this card could really shake bcrypt GPU cracking up?
This has a similar leap to the one I felt when the 3Dfx Voodoo 2 SLI came out. The possibilities seem pretty amazing.
I'm interested to know how quickly I can plug in a machine learning toolkit, it was bit finicky to get up and running on a box with a couple of 580GTs in it but that might just be because it was an older board.
On paper, the Fastra II would still be a fair bit faster, and would have more RAM. However. in a standoff, I'd put my money on the 1080 simply because of the overhead of wrangling all those GPUs to work together.
> The performance of NVIDIA’s GeForce GTX 1080 is just off the charts. NVIDIA’s CEO, Jen-Hsun, mentioned at the announcement that the GeForce GTX 1080 is not only faster than one GeForce GTX 980 but it crushes two 980s in SLI setup.
A big part of what you're paying for with the workstation-grade cards (EG Quadro/FirePro) is certification. If you're using many "serious" applications like 3D CAD, you must be using a certified (e.g. not consumer grade) card to be fully supported. They run fine on the consumer cards - but don't expect support.
And as it happens, the companies that specialize in cad support will use every opportunity to shove these overpriced cards down your throat. As far as I've understood their business model, they earn almost nothing from selling licenses, so they live on their maintenance contracts and selling Quadro/Fire cards.
Meanwhile we have no problems with an office running on 950Ms and 780/980 GTXes, except for some weird driver bugs that only occur on the single Quadro machine that we keep for reference.
I know that synthetic tests only mean so much, but are these scorings roughly linear? I'm just kind of bluesy away at a ten times difference in power over my laptop's card. I guess it's been a while since I've had a desktop, or bought a demanding game for that matter.
I wonder what the idle power usage of one of these would be? I wish my motherboard allowed me to turn off the dedicated card and fall back to the Intel chipset when I didn't need the performance.
From the slides in Nvidia's presentation, it looks like the max power consumption is about 180 W. That puts it squarely between the GTX 980 (165 W) and the GTX 980 Ti (250 W).
Idle power consumption is an order of magnitude less. For the 980 Ti, you're looking at about 10 W of power consumption while running outside of a gaming application. Maybe 40 W if you're doing something intensive. [1]
My laptop does that, but it's frequently a pain. Pull up a demo of OpenGL ES stuff on Chrome and it won't switch to to the dedicated card, even if I ask it to since Chrome 'isn't a graphics application'. Plus Linux drivers for it are hit-or-miss (it's improved since launch). Still a good idea though and it keeps the fan from running much of the time.
I am just curious (and a total machine learning novice). If you were to experiment with ML, what are the benefits to getting a fast consumer card like this or use something like AWS GPU instances (or some other cloud provider) ?
Or phrased differently: When does it make sense to move from a local GPU to the Cloud and vice versa ?
AWS would make sense if you don't yet know if your ML algo scales well on a GPU: you can try the hardware without losing much money.
Consumer cards are a good deal if you know your algo works well on GPUs in single (32-bit) floating point and you don't mind dicking around with hardware configurations to save money.
I'm a relative novice also, but we have run an EC2 8 way GPU instance. Basically the GPUs are a bit slower and they only have 4 GB of ram. Also getting everything set up and uploading a data set is tedious. Costs weren't too bad. I could see AWS being used for training smaller problems and learning. If you have a serious project it probably won't fit the bill.
Image analysis is cumbersome on the cloud, copying data back and forth gets tedious really quick. I would use the cloud only when I was confident as what I was doing.
The AWS Linux GPU instances only have 4GB of memory which means processing can be slow (memory swaps) if your datasets is large.
Generally, it makes sense to have a local GPU to try out models, and move to the cloud for large scale computing, once you have a better idea of the model.
It's about how much machine learning you're currently doing and how much you expect to spend.
E.g. if you play around for a little bit, you could end up spending $50 on the cloud. By contrast, a card will set you back $350+.
If you're doing lots of machine learning, the cloud will quickly rack up.
There's other factors. E.g. the cloud's performance can vary depending on the time of day (and whose using the application).
Don't buy a graphics card unless you're certain you're going to do lots of ML. Or buy one if you think that'll motivate you to do justify the purchase by doing lots of ML.
The lower TDP is just as significant as speed. I've got a pair of GTX 480 that I can use to heat my office with a light workload. How many 1080s could run in a single workstation?
Haswell-E chips have 40 PCIe Lanes. If you give 8 to each then 4 is a go although cooling that will be fun. Now that already exceeds the ATX 7 slots but many cases (especially those that offer SSI EEB compatibility) offers 8 slots. Four such GPUs, a Haswell-E, the motherboard -- it'd be challenging to assemble this below $3K but I think $4k is a reasonable target.
Going beyond that is not impossible but requires server grade hardware. For example, the Dell R920/R930 has 10 PCIe slots so does the Supermicro SuperServer 5086B. The barebone for the latter is above $8K. You need to buy Xeon E7 chips for these and those will cost you more than a pretty penny. I do not think $20K is unreasonable to target.
Not enough? A single SGI UV 300 chassis (5U) provides 4 Xeon E7 processors, up to 96 DIMMs, and 12 PCIe slots. You can stuff 8 Xeon E7 CPUs into the new HP Integrity MC990 X and have 20 (!) PCIe slots. How much these cost? An arm and two legs. I can't possibly imagine how such a single workstation would be worth it instead of a multitude of cheaper workstations with with just 4 GPUs (you'd likely end up with a magnitude more GPUs in this case -- E7 CPUs and their base systems are hideously expensive) but to each their own.
Wow thanks for the detailed response on hardware. I was aware of the Supermicro beast.
Setting aside this 1080, just on per CUDA core prices, cheaper workstations with pairs of 970s are much better deals, because everything gets cheaper, not just the GPUs.
I left out something: the Supermicro 6037R-TXRF Server is dual socket and has ten slots and since it's an older dual socket even with CPUs and RAM it can be had for <$3K so together with five GPUs you can have it for less than $6K. It's a whole another question whether almost double the price worths adding another GPU (no).
Not so specific to this new card but wtf are cpus at 65 TDP and gpus at 200+? It just seems crazy that we have all other elements of computing heavily pushing for lower power use for many years now and graphics rips the other way.
Note that in terms of pure compute performance the new 16nm Nvidia GTX 1080 (2560 shaders at up to 1733 MHz = 8873 SP GFLOPS) barely equals the performance of the previous-generation 28nm AMD Fury X (4096 shaders at up to 1050 MHz = 8602 SP GFLOPS). Of course the 16nm chip does so at a significantly lower TDP (180 Watt) than the 28nm chip (275 Watt), so it will be interesting to see what Nvidia can achieve at a higher thermal/power envelope with a more high-end card... I am waiting impatiently to see how AMD's upcoming 14nm/16nm Polaris chips will fare, but from the looks of it it seems like Polaris will beat Nvidia in terms of GFLOPS per Watt.
Yeah, it will be interesting to see how good AMD's Polaris chips will be, especially when it comes to energy efficiency. Fury was a step in the right direction (I'm wondering how much of this can be attributed to HBM's lower voltage etc.). But I'm even more interested in AMD Vega - the high end competitor to Nvidias GP 100 that is equipped with HBM2. Especially if AMD has an advantage over Nvidia when it comes to implementing HBM, since they developed it together with SK Hynx and have previous experience with Fury, or if this doesn't make a difference.
Vega is also exiting because it should be released when AMD's Zen CPU's will be released, and there were rumors about a HPC Zen APU with Vega and HBM.
Yes, Zen for the desktop (Summit Ridge) is even listed for end of 2016 [1], followed by first Zen APU's (Raven Ridge) sometime 2017 and Vega early 2017. [2]
If we're discussing future GPUs, does anyone know what exactly AMD means by "next-gen memory" when it talks about Navi?
At first I thought it means that HBM2 would be available across their whole lineup, but I think if that was the case they would've just said so.
I've also read a bit about how Nvidia wanted to go with HMC (Hyper Memory Cube) instead of HBM, but it seems it's still at least twice as expensive/GB, and they needed to go with high-end GPUs that have at least 16GB. HMC is not even at 8GB yet. Intel also seems to have adopted HMC for some servers.
So is the "next-gen" memory HMC, or something else? AMD is supposed to come out with it in 2018, hopefully at 10nm.
Was the difference between HMC and HBM not basically HMC had good latency comparable to normal main RAM like DDR3 plus good bandwith (up to 400 GB/s), while HBM had worse latency than DDR3 (like GDDR5) with a large bus and really good bandwith (1 TB/s)? Will indeed be interesting to see what AMD means with next gen memory.
No. HMC is fully buffered and provides a packet-based interface to the CPU it's connected into. This means that the CPU doesn't need to know anything about the memory it uses, and saves die space on the CPU side, but it adds latency. HBM has lower latency than HMC.
Also, GDDR5 does not have higher latency than DDR3. The memory arrays themselves are the same and exactly as fast, and the bus doesn't add any extra latency. GDDR5 doesn't trade latency for bandwidth, it trades density for bandwidth.
Things built to use GDDDR5 often have much higher latencies than things built to use DDR3, but this has nothing to do with the memory and instead about how the memory controllers in GPUs delay accesses to merge ones close to each other to optimize for bandwidth.
AMD chips have almost always had a lead over Nvidia from a per-flops perspective, and AMD's architecture is definitely ahead of Nvidia as far as supporting integer operations and bitwise ops efficiently(see their dominance of cryptocurrency mining). However, Nvidia's software tooling has always been superb, and they basically own the GPGPU-for-HPC space because of it. On top of that, one of the biggest growth areas of consumers of GPGPU in recent years has been "deep learning", for which Nvidia has embraced and has great tooling and software for their hardware. AMD has potential to compete with Nvidia in this area, but they don't seem to have the resources to develop optimized Neural Network software to run on their GPUs like Nvidia does, so they are way behind. Basically, GFLOPS per Watt is nice and all, but if nothing you care about runs on the hardware, you won't buy it. AMD seems to have been stuck in this position for HPC, and it doesn't seem to be changing sadly.
I think the near future in compilers is going to be dominated by accelerator capable LLVM variants (such as NVVM). Ideally, all Nvidia, AMD and Intel would and should need to do is to provide LLLVM based backends. Frontends just need a sane way of specifying SIMT. And no, loop + directive based stuff like OpenMP and OpenACC is IMO just a stopgap measure in order to serve higher ups the lie that portation is going to be swift and easy. There's basically one sane way to deal with data parallel problems, and that's to treat what's going on in one core as a scalar program with precomputed indices (i.e. the CUDA / OpenCL way).
Python has some good support for this already. I haven't done a project like this yet, but if setting up something fresh I'd immediately jump on NumPy interacting with my own kernels and some Python glue for the node level parallelism.
Alternatively they could make their own API, independent from OpenCL. I have written a small amount in each languages (a rainbow table generator), and found the CUDA version much more pleasant to use. CUDA made it really easy to get a good idea of the underlying hardware, and make architectural changes in order to make the code faster.
OpenCL (using an ATI card) was much harder to program, since the abstraction level was much higher. Writing two separate kernels and have them each be faster than a generic version that ends up compromising for compatibility.
The OpenCL one ended up being faster, but I suspect that's due to ATI hardware being superior at the time.
AMD are supporting open standards like OpenCL, which is effectively their answer to CUDA. The problem is that Nvidia refuse to keep their drivers up to date with the latest OpenCL capabilities. The latest version nVidia supports is OpenCL 1.1, but its now up to OpenCL 2.2. nVidia are years behind on OpenCL. This is pretty much the primary reason why CUDA seems so far ahead.
nVidia released new features and capabilities on CUDA that later versions of OpenCL also supported, but were not available on nVidia cards due to lack of OpenCL driver support. Hence, lots of people developed libraries and tools for CUDA, with great encouragement and support from nVidia (which attracted developers). These tools were useful and powerful, encouraging others to also use nVidia hardware to take advantage of those tools. CUDA has now taken root.
If nVidia were to start keeping their drivers updated with the latest version of OpenCL, i suspect CUDA's dominance would start to wane. Though the CUDA roots have gone deep, so it may take several years to reverse its dominance. However, its highly unlikely that nVidia will update their drives any time soon. The only thing that would probably force them is AMD gaining market dominance for a few years.
The most interesting thing about AMD is their ability to create GPUs with superior synthetic specs than their NVIDIA competitor and yet they lose across the board in anything but embarrassingly parallel applications. Not only do they lose, but they lose big.
64 of them train Alexnet in ~5 hours. A DGX-1 does it in 2. Don't have/Can't get a $129K DGX-1? Fine, buy a Big Sur from one of several vendors and throw in 8 GTX 1080s as soon as they're out, implement "One Weird Trick", and you'll do it in <3. That ought to run you about $30-$35K versus >$600K for those 64 Xeon servers.
FPGAs OTOH are fantastic at low memory bandwidth embarrassingly parallel computation like bitcoin mining. Deep Learning is not such a domain.
I don't see anywhere on that page any benchmark about 5000 images/sec, maybe you should update your link to a page showing that info or update your claim to match what you link to?
Even your previous post had links to two page that were not measuring the same thing... and thus could not support your claims?
Based on the text you apparently couldn't be bothered to read, for AlexNet, a forward pass on 128 images (Input 128x3x224x224) takes ~25 milliseconds (ignoring O(n) dropout and softmax where n is 128). I'll let you do the math for the rest of this...
> The GP104 Pascal GPU Features 2x the performance per watt of Maxwell.
And
> The GTX 1080 is 3x more power efficient than the Maxwell Architecture.
I think that someone get carried away by imagination.
I found that 980 has 4.6 TFLOPs (Single precision)
And assuming that 1080 performance (9 TFLOPs) is also for single precision and new card has the same TDP, this is 1.95x increase, so it is ~2x
EDIT: I found that 1080 will have 180W TDP, where 980 has 165W, so correction it will be 1.79x increase
Weird comment. Like a grandma driving a Ferrari with a broken gearbox: "I am perfectly fine driving at moderate speed, maybe I am not pushing the car hard enough. I am perfectly fine that the mechanic deactivated the malfunctioning higher gears".
You may not notice it nowadays because several games received patches and newer nVidia drivers limit the memory consumption to just below 3.5 GB. Otherwise you would experience a major slowdown. Also a major problem for CUDA.
If games truly don't need access to the last .5GB (and they seem to not in most cases if there's patches there) and all the parent does is game (which is what I do on my 970) then why would it even matter if the last .5GB is slow, or that the card wasn't being "pushed hard enough"? It's a gaming card, and using it for gaming seems to be working just fine, in my anecdotal experience at least. Gaming benchmarks have also been great for the card.
Just because not everyone writes CUDA applications that need access to all 4GB does not mean that people are using it wrong, or that they should mind that the last .5GB is slow. If it works for others' use then that's great for them.
I see you point, but I use a 970 and just finished Dark Souls 3 on max settings without a single hiccup.
No doubt that with future games the 970 will not perform as well as it should, but I will have a different card by then. What they did is a shame and shouldn't happen again, but I haven't noticed any real-world ramifications yet.
Nvidia didn't release as much information on the GTX 1070, but we know it will have 8GB GDDR5 (not GDDR5X) and offer 6.5 TFLOPs (vs. 9 TFLOPs for the GTX 1080). It will have a $379 MSRP and officially launch on June 10.
What really blew me away is that they went straight for Displayport 1.4 (https://en.wikipedia.org/wiki/DisplayPort#1.4) that was only announced on 1st of March 2016. DP 1.3 was approved on 15th of September 2014 and as of today there are no cards supporting it except this one (with backwards compatibility).
The bad news is that there's no news about new gen displays that could take advantage of this graphic card. I'm talking about 4K 120Hz HDR (https://en.wikipedia.org/wiki/High-dynamic-range_rendering) displays. This is a total WTF - we have a graphic card with DP 1.4 and we don't even have a single display with so much as DP 1.3...
Yes, but inside the computer you have the luxury of static, pre-calculated track layouts that optimise for latency, noise and general integrity. A general purpose, variable length, high bandwidth cable and connectors is a much harder problem, thus the kind of issues we see here sending raw video signals of extremely high fidelity between card and monitor.
USB3 was also supposed to have fiber components. Really hard to find references for this online since at CES 2008 they showed the connectors without any optical stuff, but I found this: http://www.theregister.co.uk/2007/09/19/idf_usb_3_announced/
Anyone know why they didn't end up going optical + copper for USB3?
There are lots of options for optical cabling (e.g. cheap plastic fibers vs. glass fibers) and all have different tradeoffs, e.g. available bandwidth vs. cost. However from what I've seen basically all options are more expensive then equivalent electrical connections (we use both optical and electrical networks in cars).
A major advantage of optical is that there are no EMV problems and that you are electrically decoupled (e.g. no hum loops). This might be one of the reasons why toslink is optical. For displays ground loops are typically no problem compared to amplifiers/speakers.
I don't have access to the spec for 1.4, but from what I can understand it's still a single cable that will do 4K 120hz HDR. Obviously it'll have to be a supported 1.4 capable cable, but that's the same as CAT 5 vs CAT 6
Dual link DVI is significantly old-hat compared to the newer DisplayPort and HDMI standards. A single dual link DVI cable can carry either half a 4K frame (1920x2160) at 8 bits per color, 60hz, OR a single 1080p frame at 10 bits per color, 60hz. To get a 4K 60hz display running you need to use two dual link DVI cables, which is insanely bulky. Plus you at best get 8 bit color out of it, and no audio.
The latest HDMI standard can do 4096x2160 60Hz in deep color (12 bits per color).
DisplayPort 1.4 has a higher maximum bandwidth than HDMI 2.0 (31.4 Gbps vs 18Gbps). This allows it to do 120Hz, deep color/HDR or carry multiple 4K streams (multiple monitors or 3D) on one cable. It's by far the most advanced standard out there for consumer video transmission.
Source: I'm an engineer working with video systems, and Wikipedia
I wouldn't call it an advantage, but DP1.4 can cheat on the bandwidth department by using lossy compression.
Their press statements it's visually transparent, but the technical overview[1] says this:
> All of the analyses showed that the DSC algorithm outperformed five other proprietary algorithms on these picture quality tests, and was either visually lossless or very nearly so for all tested images at 8 bits/pixel.
Well, that's understandable, however comments like this are really discouraging: "Proper HDR support for monitors is years away. Some budget trash with HDR will be released, but they won't come close to matching the HDR standard and will have tons of issues." (https://www.reddit.com/r/Monitors/comments/4hj4xv/what_are_s...)
Your link is now bit outdated in the regard that these days there are plenty of consumer HDR standards/names out there. HDR10 and Dolby Vision are the two main competing standards, and we have marketing terms like "Ultra HD Premium" or "HDR Compatible".
I think NCX is being bit overly negative (in his characteristic style) in his predictions. Sure, there will be cheap crap released, but considering that almost all major manufacturers are pushing this tech heavily on TV screens it can't be that long until it trickles down to monitors too. Of course you can plug your PC right now to one of those fancy new HDR TVs, I'm not sure though if you can get HDR actually working. I guess that would be mostly a software issue?
I didn't say it was. None of that explains what HDR as it relates to monitors means technically. Everything uses the term, but we already have numbers for bit depth of the signal and contrast ratios of the monitor.
So if it isn't just more contrast then what does it actually mean? In images it means the image has values stored above what can be displayed.
IMO still a preferable situation than what we had before - which was displays coming out with hacked up interfaces because they needed more bandwidth than there were video cards capable of driving it.
Displays get to play catchup for a while. This is great - it means we can avoid stupid hacks like MST.
I think it's actually a smart choice, as as much as I doubt there will be monitors with that spec, there will for sure be VR headsets that will be able to use it, and likely not that far in the future
It likely has to do with the timing at which the standards were finalized. DP1.3 / DP1.4 probably weren't ready when they needed to finalize TB3. Unfortunate but alas, standards.
DisplayPort 1.4 can support 8K UHD (7680×4320) at 60 Hz with 10-bit color and HDR, or 4K UHD (3840×2160) at 120 Hz with 10-bit color and HDR. 4K at 60 Hz with 10-bit color and HDR can be achieved without the need for DSC. On displays which do not support DSC, the maximum limits are unchanged from DisplayPort 1.3 (4K 120 Hz, 5K 60 Hz, 8K 30 Hz). https://en.wikipedia.org/wiki/DisplayPort#1.4
> There will be two versions of the card, the base/MSRP card at $599, and then a more expensive Founders Edition card at $699. At the base level this is a slight price increase over the GTX 980, which launched at $549. Information on the differences between these versions is limited, but based on NVIDIA’s press release it would appear that only the Founders Edition card will ship with NVIDIA’s full reference design, cooler and all. Meanwhile the base cards will feature custom designs from NVIDIA’s partners. NVIDIA’s press release was also very careful to only attach the May 27th launch date to the Founders Edition cards.
I'm somewhat dismayed that the GTX 1080 has only 8GB of VRAM considering that previous generation AMD products already had 8GB, and the previous generation Titan model had 12GB—the latter of course being outperformed by the GTX 1080.
Then again, rendering a demanding title at 8192x4320 on a single card and downsampling to 4k is probably wishful thinking anyways. However, it's definitely a legitimate concern for those with dual/triple/quad GPU setups rendering with non-pooled VRAM.
On the bright side, 8GB should hopefully be sufficient to pull off 8k/4k supersampling[0] with less demanding titles (e.g. Cities: Skylines). Lackluster driver or title support for such stratospheric resolutions may prove to be an issue, though.
It's possible Nvidia is saving the 12GB configuration for a 1080 Ti model down the road. If they release a new Titan model, I'm guessing it'll probably weigh in at 16GB. Perhaps less if those cards end up using HBM2 instead of GDDR5X.
(nitpick) actually 10 TFLOPs is a more appropriate target. Doesn't make sense to aim to cripple your performance by a factor of 822 for marketing reasons...
225 comments
[ 4.9 ms ] story [ 179 ms ] threadOne thing not mentioned in this gaming-oriented press release is that the Pascal GPUs have additional support for really fast 32 bit (and do I recall 16 bit?) processing; this is almost certainly more appealing for machine learning folks.
On the VR side, the 1080 won't be a minimum required target for some time is my guess; the enthusiast market is still quite small. That said, it can't come too quickly; better rendering combined with butter-smooth rates has a visceral impact in VR which is not like on-screen improvements.
There seems to be Nvidia's pascal, gtx, titan, etc. Something called geforce. And I believe these are just from Nvidia.
If I'm interested in building out a desktop with a gpu for: 1. Learning computation on GPU (matrix math such as speeing up linear regression, deep learning, cuda) using c++11 2. Trying out oculus rift
Is this the right card? Note that I'm not building production models. I'm learning to use gpus. I'm also not a gamer, but am intrigued by oculus. Which GPU should I look at?
If not, the 980 Ti or Titan X offer excellent deep learning performance, albeit only at FP32. And their scheduling/preemption is not entirely there, they may not be as capable of Dynamic Parallelism as Kepler was. The 780 Ti is actually a more capable card in some respects.
The new consumer Pascal cards will almost certainly not support FP64, NVIDIA has considered that a Quadro/Tesla feature since the OG Titan. If DP performance is a critical feature for you and you need more performance than an OG Titan will deliver, you want the new Tesla P100 compute cards, and you'll have to convince NVIDIA you're worthwhile and pay a 10x premium for it if you want it within the next 6 months. But they probably will support compute better, although you should wait for confirmation before spending a bunch of money...
For VR stuff or deep learning, the consumer Pascal cards sound ideal. Get a 1070 or 1080, definitely. The (purportedly) improved preemption performance alone justifies the premium over Maxwell, and the FP16 capability will significantly accelerate deep learning (FP16 vs FP32 is not a significant difference in overall net output in deep learning).
NVIDIA is totally dominant in the compute market. They have an enormous amount of software running on CUDA that you would be locking yourself out of with AMD, and since NVIDIA has such a dominant share of the compute hardware you would also be tuning for what amounts to niche hardware.
AMD has recently been working on getting a CUDA compatibility layer working, hopefully this will improve in the future.
I am purely after FP64 performance for scientific compute.
a) what does the "OG" stand for? b) what about the titan black model? seems to offer yet a bit more FP64 performance than the normal Titan?
The 1080 will probably be the best card available until next year, when HBM2 cards (~3x memory bandwidth) reach general availability. I'm hoping for a 1080 Ti or a new Titan then.
What are the characteristics of NVIDIA GPUs that make them superior for deep learning applications?
Phrased another way, if you're designing a card specifically to be good for training deep neural nets, how does it come out differently from cards designed to be good at other typical applications of GPGPU?
On the hardware end, Nvidia has slightly superior floating point performance (which is the only thing that matters for neural nets). Pascal also contains 16 bit floating point instructions, which will also be a big boost to neural net training performance.
I would be interested to hear about the difference in floating point performance. I would have guessed that, at this point, pretty much every chip designer in the world knows equally well how to make a floating-point multiplier. So it must be that nvidia vs amd have made different trade-offs in when it comes to caching memory near the FP registers or some such thing?
- Pascal: codename for Nvidia's latest microarchitecture, used in the new GeForce 10-series announced today. (AMD counterpart is "Polaris".)
- GeForce: Nvidia's consumer graphics card brand. (AMD equivalent is "Radeon".)
- GTX: label for GeForce graphics cards meant for gaming and other demanding tasks (see AMD Radeon "R9").
- Titan: special name for super high-end GeForce graphics cards. (For AMD, see "Fury".)
You might also encounter Nvidia Quadro graphics cards (comparable to AMD FirePro), which are meant for professional workstations, and Nvidia Tesla graphics cards, which target high performance computing.
It's also nice to have an all Intel machine for Linux. I'd use a lowend NV Pascal to breathe new life into an older desktop machine since NV seems to always have a bit better general desktop acceleration that really helps out old CPUs. If building a high end gaming rig I'd probably wait for AMD's next chip. I've liked them more on the high end for a few generations now. Async compute and fine-grained preemption, generally better hardware for Vulkan/DX12. AMD is also worth keeping an eye on for their newfound console dominance, subsequent impact if they push dual 'Crossfire' GPUs into XboxOne v2, the PS4K and Nintendo NX. That would be a master stroke to get games programmed at a low-level for their specific dual GPUs by default. Also, the removal of driver middleware mess with the return of low-level APIs to replace OGL/DX11 will remove the software monkey off AMD's back. That always plagued them and the former ATI a bit.
I'll probably buy the KabyLake 'Skull Canyon' NUC revision next year and if I end up missing the GPU power, hook up the highest end AMD Polaris over Thunderbolt. Combining the 256MB L4 cache that Kabylake-H will have with Polaris will truly be next-level. Kaby also has Intel Optane support in the SODIMM slots, it's possible we'll finally see the merge of RAM+SSDs into a single chip.
But more than anything, I want Kabylake because it's Intel's last 14nm product so here's to hoping they finally sort out their electromigration problems. Best to take a wait and see on these 16nm GPUs for the same reason. I'm moving to a 'last revision' purchase policy on these <=16nm processes.
Can you elaborate on this? Never heard of this.
Back in the CS < 1.3 days everyone used 640x480 but I never really understood why.
Also there are plenty of full HD displays with 120Hz+ now ...
https://docs.google.com/spreadsheets/d/1UaM765-S515ibLyPaBtM...
A little bit of everything but note how common 1024x768 is. It looks a little sharper, enough that it doesn't need AA (which I always felt introduced some strange input lag).
Guarantees good performance, makes the models slightly bigger. Gives you a slight edge. You can also run a much slower GPU, that's how I'm getting away with using stuff like Intel Iris Pro. I'd run 1280x1024 with all options on lowest even if I had a Fury X. Other people I know have GF780s and do the same. I only turn up graphics options in single player games, at which point I don't care about FPS dips. When playing competitively I want every edge I can get.
Jesus christ, I can't even imagine the leaps in skill I'd have to make before the difference between 50fps and 150fps had any impact on my performance. Watching gaming at that level feels like being a civilian in Dragonball Z, just seeing a bunch of blurs zipping in the sky while wondering if I'm about to become obsolete.
Keep in mind fps are normally given in averages. You can go up to a wall and have nothing on screen and get an insane FPS or you can back out and take in all of the area with max view distance and see a lower FPS. Add in players/objects/explosions etc and your FPS dips and dives.
Having an average of 50FPS means you likely ARE going to notice it at certain dips while at 150 average would be much less likely to get a perceivable fps dip
+ This is a "small" part of the cost, but doing 5m polygons at 60 fps can result in about 30 GFLOPS of compute for that single matrix operation (in reality, there are many vertex operations and often many more fragment operations).
edit: apparently Pascal still has a warp-size of 32.
Threads are executed an entire warp at a time (32 or 64 threads). All threads execute all paths though the code block - eg if ANY thread executes an if-statement, ALL threads execute an if-statement. The threads where the if-statement is false will execute NOP instructions until the combined control flow resumes. This formula is followed recursively if applicable (this is why "warp divergence/branch divergence" absolutely murders performance on GPUs).
When threads execute a memory load/store, they do so as a bank. The warp controller is designed to combine these requests if at all possible. If 32 threads all issue a request for 32 sequential blocks, it will combine them into 1 request for 32 blocks. However, it cannot do anything if the request isn't either contiguous (a sequential block of the warp-size) or strided (a block where thread N wants index X+0, N+1 wants thread X+0, X+2N, etc). In other words - it doesn't have to be contiguous, but it does have to be uniform. The resulting memory access will be broadcast to all units within the warp, and this is a huge factor in accelerating compute loads.
Having a warp-size of 64 hugely accelerates certain patterns of math, particularly wide linear algebra.
edit: apparently Pascal still has a warp-size of 32.
The NVIDIA memory model also goes through L1 cache - but that's obviously not very big on a GPU processor (also true on AMD IIRC). Like <128 bytes per thread. It's great if your threads hit it coalesced, otherwise it's pretty meaningless.
At glance there is a lot of legacy stuff so I'd look at anything related to GCN, Sea Islands and Southern Islands. Evergreen, R600-800 etc are legacy VLIW ISA as far as I know.
memory accesses in a warp do not necessarily have to be contiguous, but it does matter how many 32 byte global memory segments (and 128 byte l1 cache segments) they fall into. the memory controller can load 1, 2 or 4 of those 32 byte segments in a single transaction, but that's read through the cache in 128 byte cache lines.
thus, if every lane in a warp loads a random word in a 128 byte range, then there is no penalty; it's 1 transaction and the reading is at full efficiency. but, if every lane in a warp loads 4 bytes with a stride of 128 bytes, then this is very bad: 4096 bytes are loaded but only 128 are used, resulting in ~3% efficiency.
http://www.wihlidal.ca/Presentations/GDC_2016_Compute.pdf
look up "dual-issue":
http://on-demand.gputechconf.com/gtc/2013/presentations/S346...
I'm just realizing that GPUs could be a viable alternative to specialized (eg, expensive) encryption offload engines or very high end (eg, expensive) v4 Xeons with many, many cores. However, it is rather hard to find data on the bandwidth at which GPUs can process a given cipher. I'm just now starting to look into this, so I could very well be mis-understanding something,
With each step you not just cut down your bandwidth but also you increase your latency and that has a huge impact.
[1]: http://shader.kaist.edu/sslshader/
Also would data-dependencies be different on a GPU than with parallelization on a CPU? Because modes like GCM (i.e. CTR) both the encryption and decryption are parallelizable (with CBC it's just decryption).
I wonder if AMD GPU's would be better suited to the task than Nvidia's (at least their existing GPU's) provided you could actually saturate the GPU with enough work to make it worth it.
Issue rate is in the CUDA documentation for Maxwell (cc 5.2) at least, presumably they'll update with Pascal soon:
http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.h...
Where they would win is in memory bandwidth for intermediate results (e.g., in/out of registers or smem or even global memory), assuming that you aren't ultimately bound by I/O over the bus from the CPU.
Am I wrong to think this card could really shake bcrypt GPU cracking up?
I'm interested to know how quickly I can plug in a machine learning toolkit, it was bit finicky to get up and running on a box with a couple of 580GTs in it but that might just be because it was an older board.
built to give a 'desktop' cluster class performance based on GPUs. I wonder how it would fare against a 1080.
> The performance of NVIDIA’s GeForce GTX 1080 is just off the charts. NVIDIA’s CEO, Jen-Hsun, mentioned at the announcement that the GeForce GTX 1080 is not only faster than one GeForce GTX 980 but it crushes two 980s in SLI setup.
From the article, I got the impression that the difference was like night and day. I wonder when we will be getting some early real-world benchmarks.
I'm sure some of those expensive cards have extra features, but the pricing differences are still crazy.
Meanwhile we have no problems with an office running on 950Ms and 780/980 GTXes, except for some weird driver bugs that only occur on the single Quadro machine that we keep for reference.
AFAICT, PassMark's GPU benchmarks are most often used simply because it's usually the first Google result for "video card benchmark".
Source: http://www.anandtech.com/show/10304/nvidia-announces-the-gef...
Idle power consumption is an order of magnitude less. For the 980 Ti, you're looking at about 10 W of power consumption while running outside of a gaming application. Maybe 40 W if you're doing something intensive. [1]
[1]: http://www.tomshardware.com/reviews/nvidia-geforce-gtx-980-t...
Consumer cards are a good deal if you know your algo works well on GPUs in single (32-bit) floating point and you don't mind dicking around with hardware configurations to save money.
Generally, it makes sense to have a local GPU to try out models, and move to the cloud for large scale computing, once you have a better idea of the model.
E.g. if you play around for a little bit, you could end up spending $50 on the cloud. By contrast, a card will set you back $350+.
If you're doing lots of machine learning, the cloud will quickly rack up.
There's other factors. E.g. the cloud's performance can vary depending on the time of day (and whose using the application).
Don't buy a graphics card unless you're certain you're going to do lots of ML. Or buy one if you think that'll motivate you to do justify the purchase by doing lots of ML.
Going beyond that is not impossible but requires server grade hardware. For example, the Dell R920/R930 has 10 PCIe slots so does the Supermicro SuperServer 5086B. The barebone for the latter is above $8K. You need to buy Xeon E7 chips for these and those will cost you more than a pretty penny. I do not think $20K is unreasonable to target.
Not enough? A single SGI UV 300 chassis (5U) provides 4 Xeon E7 processors, up to 96 DIMMs, and 12 PCIe slots. You can stuff 8 Xeon E7 CPUs into the new HP Integrity MC990 X and have 20 (!) PCIe slots. How much these cost? An arm and two legs. I can't possibly imagine how such a single workstation would be worth it instead of a multitude of cheaper workstations with with just 4 GPUs (you'd likely end up with a magnitude more GPUs in this case -- E7 CPUs and their base systems are hideously expensive) but to each their own.
Setting aside this 1080, just on per CUDA core prices, cheaper workstations with pairs of 970s are much better deals, because everything gets cheaper, not just the GPUs.
Vega is also exiting because it should be released when AMD's Zen CPU's will be released, and there were rumors about a HPC Zen APU with Vega and HBM.
[1] http://imgur.com/rRjKZDk [2] http://i.imgur.com/lkjgayO.jpg?1
At first I thought it means that HBM2 would be available across their whole lineup, but I think if that was the case they would've just said so.
I've also read a bit about how Nvidia wanted to go with HMC (Hyper Memory Cube) instead of HBM, but it seems it's still at least twice as expensive/GB, and they needed to go with high-end GPUs that have at least 16GB. HMC is not even at 8GB yet. Intel also seems to have adopted HMC for some servers.
So is the "next-gen" memory HMC, or something else? AMD is supposed to come out with it in 2018, hopefully at 10nm.
Also, GDDR5 does not have higher latency than DDR3. The memory arrays themselves are the same and exactly as fast, and the bus doesn't add any extra latency. GDDR5 doesn't trade latency for bandwidth, it trades density for bandwidth.
Things built to use GDDDR5 often have much higher latencies than things built to use DDR3, but this has nothing to do with the memory and instead about how the memory controllers in GPUs delay accesses to merge ones close to each other to optimize for bandwidth.
Witness also PyOpenCL versus PyCUDA - both are essentially equally convenient to use, while raw OpenCL C is a nightmare compared to CUDA C.
Python has some good support for this already. I haven't done a project like this yet, but if setting up something fresh I'd immediately jump on NumPy interacting with my own kernels and some Python glue for the node level parallelism.
I don't understand why AMD doesn't just jump on the CUDA bandwagon and support the API?
2. It's not just the API. It's the entire SDK that has NV specific hooks that AMD can't tap into
OpenCL (using an ATI card) was much harder to program, since the abstraction level was much higher. Writing two separate kernels and have them each be faster than a generic version that ends up compromising for compatibility.
The OpenCL one ended up being faster, but I suspect that's due to ATI hardware being superior at the time.
The current versions only work with a couple of latest AMD chips though. And CUDA is just one part of it, they have nothing equivalent to CuDNN
nVidia released new features and capabilities on CUDA that later versions of OpenCL also supported, but were not available on nVidia cards due to lack of OpenCL driver support. Hence, lots of people developed libraries and tools for CUDA, with great encouragement and support from nVidia (which attracted developers). These tools were useful and powerful, encouraging others to also use nVidia hardware to take advantage of those tools. CUDA has now taken root.
If nVidia were to start keeping their drivers updated with the latest version of OpenCL, i suspect CUDA's dominance would start to wane. Though the CUDA roots have gone deep, so it may take several years to reverse its dominance. However, its highly unlikely that nVidia will update their drives any time soon. The only thing that would probably force them is AMD gaining market dominance for a few years.
If you read between the lines of how dismissive AMD were of NVidia's work on self driving cars you can see they don't care about compute.
They are chasing the VR market, and that is fine.
http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2012/...
https://github.com/GPUOpen-ProfessionalCompute-Tools/HIP
Here are two examples. First, the engine behind Folding@Home: https://simtk.org/plugins/moinmoin/openmm/BenchmarkOpenMMDHF...
See also AlexNet (Deep Learning's most prominent benchmark): https://github.com/amd/OpenCL-caffe versus GTX TitanX: https://github.com/soumith/convnet-benchmarks
TLDR: AMD losing by a factor of 2 or more...
So unless this changes dramatically, NVIDIA IMO will continue to dominate.
But hey, they both crush Xeon Phi and FPGAs so it doesn't suck to be #2 when #3 is Intel.
600 images/s inference (highest ever publically reported), projected to be as high as 900 images/s. ~$5000 per FPGA.
Compare and contrast with TitanX/GTX 980 TI now topping 5000 images/s. GTX 1080 will only be faster than this and $600.
https://github.com/soumith/convnet-benchmarks
And so far, no FPGA training numbers. But here are some distributed training numbers for $10Kish Xeon servers:
https://software.intel.com/en-us/articles/caffe-training-on-...
64 of them train Alexnet in ~5 hours. A DGX-1 does it in 2. Don't have/Can't get a $129K DGX-1? Fine, buy a Big Sur from one of several vendors and throw in 8 GTX 1080s as soon as they're out, implement "One Weird Trick", and you'll do it in <3. That ought to run you about $30-$35K versus >$600K for those 64 Xeon servers.
FPGAs OTOH are fantastic at low memory bandwidth embarrassingly parallel computation like bitcoin mining. Deep Learning is not such a domain.
Even your previous post had links to two page that were not measuring the same thing... and thus could not support your claims?
I think you need to clarify.
And
> The GTX 1080 is 3x more power efficient than the Maxwell Architecture.
I think that someone get carried away by imagination.
I found that 980 has 4.6 TFLOPs (Single precision) And assuming that 1080 performance (9 TFLOPs) is also for single precision and new card has the same TDP, this is 1.95x increase, so it is ~2x
EDIT: I found that 1080 will have 180W TDP, where 980 has 165W, so correction it will be 1.79x increase
GTX 970: It was revealed that the card was designed to access its memory as a 3.5 GB section, plus a 0.5 GB one, access to the latter being 7 times slower than the first one. -- https://en.wikipedia.org/wiki/GeForce_900_series#False_adver...
You may not notice it nowadays because several games received patches and newer nVidia drivers limit the memory consumption to just below 3.5 GB. Otherwise you would experience a major slowdown. Also a major problem for CUDA.
Just because not everyone writes CUDA applications that need access to all 4GB does not mean that people are using it wrong, or that they should mind that the last .5GB is slow. If it works for others' use then that's great for them.
No doubt that with future games the 970 will not perform as well as it should, but I will have a different card by then. What they did is a shame and shouldn't happen again, but I haven't noticed any real-world ramifications yet.
Source: http://www.anandtech.com/show/10304/nvidia-announces-the-gef...
Even with 3.5gb it's still a great cost/performance card.
The bad news is that there's no news about new gen displays that could take advantage of this graphic card. I'm talking about 4K 120Hz HDR (https://en.wikipedia.org/wiki/High-dynamic-range_rendering) displays. This is a total WTF - we have a graphic card with DP 1.4 and we don't even have a single display with so much as DP 1.3...
Advantages over other standards: https://en.wikipedia.org/wiki/DisplayPort#Advantages_over_DV...
Also, another major advantage - USB Type-C compatibility (http://www.displayport.org/what-is-displayport-over-usb-c/)
https://en.wikipedia.org/wiki/Thunderbolt_(interface)
Anyone know why they didn't end up going optical + copper for USB3?
A major advantage of optical is that there are no EMV problems and that you are electrically decoupled (e.g. no hum loops). This might be one of the reasons why toslink is optical. For displays ground loops are typically no problem compared to amplifiers/speakers.
Dual link DVI is significantly old-hat compared to the newer DisplayPort and HDMI standards. A single dual link DVI cable can carry either half a 4K frame (1920x2160) at 8 bits per color, 60hz, OR a single 1080p frame at 10 bits per color, 60hz. To get a 4K 60hz display running you need to use two dual link DVI cables, which is insanely bulky. Plus you at best get 8 bit color out of it, and no audio.
The latest HDMI standard can do 4096x2160 60Hz in deep color (12 bits per color).
DisplayPort 1.4 has a higher maximum bandwidth than HDMI 2.0 (31.4 Gbps vs 18Gbps). This allows it to do 120Hz, deep color/HDR or carry multiple 4K streams (multiple monitors or 3D) on one cable. It's by far the most advanced standard out there for consumer video transmission.
Source: I'm an engineer working with video systems, and Wikipedia
Their press statements it's visually transparent, but the technical overview[1] says this:
> All of the analyses showed that the DSC algorithm outperformed five other proprietary algorithms on these picture quality tests, and was either visually lossless or very nearly so for all tested images at 8 bits/pixel.
[1] http://www.vesa.org/wp-content/uploads/2014/04/VESA_DSC-ETP2...
It would also be useless to have the displays first and cards later
I guess some manufacturers have it lined up already
Your link is now bit outdated in the regard that these days there are plenty of consumer HDR standards/names out there. HDR10 and Dolby Vision are the two main competing standards, and we have marketing terms like "Ultra HD Premium" or "HDR Compatible".
https://www.cta.tech/News/News-Releases/Press-Releases/2015-...
http://www.whathifi.com/advice/ultra-hd-premium-what-are-spe...
I think NCX is being bit overly negative (in his characteristic style) in his predictions. Sure, there will be cheap crap released, but considering that almost all major manufacturers are pushing this tech heavily on TV screens it can't be that long until it trickles down to monitors too. Of course you can plug your PC right now to one of those fancy new HDR TVs, I'm not sure though if you can get HDR actually working. I guess that would be mostly a software issue?
So if it isn't just more contrast then what does it actually mean? In images it means the image has values stored above what can be displayed.
Displays get to play catchup for a while. This is great - it means we can avoid stupid hacks like MST.
> There will be two versions of the card, the base/MSRP card at $599, and then a more expensive Founders Edition card at $699. At the base level this is a slight price increase over the GTX 980, which launched at $549. Information on the differences between these versions is limited, but based on NVIDIA’s press release it would appear that only the Founders Edition card will ship with NVIDIA’s full reference design, cooler and all. Meanwhile the base cards will feature custom designs from NVIDIA’s partners. NVIDIA’s press release was also very careful to only attach the May 27th launch date to the Founders Edition cards.
But oh man am I excited!
Then again, rendering a demanding title at 8192x4320 on a single card and downsampling to 4k is probably wishful thinking anyways. However, it's definitely a legitimate concern for those with dual/triple/quad GPU setups rendering with non-pooled VRAM.
On the bright side, 8GB should hopefully be sufficient to pull off 8k/4k supersampling[0] with less demanding titles (e.g. Cities: Skylines). Lackluster driver or title support for such stratospheric resolutions may prove to be an issue, though.
It's possible Nvidia is saving the 12GB configuration for a 1080 Ti model down the road. If they release a new Titan model, I'm guessing it'll probably weigh in at 16GB. Perhaps less if those cards end up using HBM2 instead of GDDR5X.
[0] https://en.wikipedia.org/wiki/Supersampling
Accidentally multiplied DCI 4K, not UHD 4K.
"10 Gaming Perfected"
Seems like a missed marketing opportunity to make it a 10 GFLOPs card.