I don’t know specifics on what these coins are honestly but there are a few clients like Nicehash which basically group a lot of different mining tasks by algorithm for you. It has an idea as to which will be most profitable for your hardware at any given time, and mines that currency for you.
For mining ETH or Monero maybe, but they didn't add that many more INT cores, so I think a couple 1070s would still be a better investment on the Nvidia side of things.
Somewhat surprising they didn't release this card as a Quadro GPU, given its high price tag even by Titan standards. Still for single GPU machine learning, this is almost exactly a V100 unless you need the extra 4GB of memory.
You could also buy ~4 1080Ti cards, but unless you can't use the FP16 tensor cores, you'd still lose heavily to a single V100.
Ugh, this company desperately needs some competition to keep prices in check. AMD really has no one to blame but themselves for their lack of success in ML. They could have been a real competitor years ago with a relatively small investment in software but they didn't even start seriously trying to compete with CuDNN until this year.
Edit: I didn't originally see that the die size of this chip is nearly double that of the older Titan XP. Perhaps the price isn't as outrageous as I first thought! I'm actually glad that there's a big enough market now to support this kind of ultra high end product. It's certainly a beast and I want one.
Prices as well as the tech itself. $3000 and still just 12GB. I wonder why this isn't in the range of ~64/128GB yet... too expensive for the type of memory?
High bandwidth GDDR5 is more expensive than DDR4. HBM is more expensive again. On top of that the high bandwidth chips are lower density, and the complexity of higher bandwidth links gets more complicated as you get more chips involved.
Low end servers use normal dimm, but typically not many dimms per channel, typically 1-3. But to hit the higher memory configurations you end up putting buffer chips on the dimms which makes them more expensive and also slightly higher latency.
So 500GB/sec to 128GB ram using HBM2 would be difficult, expensive, and likely wouldn't fit in today's GPU form factors... yet.
GPUs generally seem to be mostly hungry for more bandwidth and not that many cases (percentage wise) is 16GB a major limitation. However in those cases check out the power9, the nvlink that x86's uses for GPU<->GPU communication can be used for CPU <-> GPU communication and avoid the high latency/low bandwidth pci-e links.
Larger (higher capacity) HBM2 stacks do not yet exist and the manufacturing process is pretty complicated/expensive. It's worth noting that all the stacks end up on the same piece of silicon as the GPU itself. Don't assume that they are holding back tech due to lack of competition.
Even if it just were DDR4 (which of course it is not) - have you looked at RAM prices recently? 128GB will cost you upwards of 1000$ alone. I don't even want to think how much 128gb DDR5 or HBM2 (if it did exist) would cost you... about the current price of the card, probably.
Based on what Google has shared a second generation TPU which includes four chips is capable of 180 Tflops which would be a decent amount more powerful than this new Nvidia option.
Speaking about buying TPUs, yesterday I vaguely remembered that the company that sells mining ASICs like the Antminer was planning to start selling AI chips ala Google's TPU. I checked their website and they have actually started selling them already https://www.sophon.ai/
I think it's pretty exciting seeing how we are getting increasingly more specific chips for AI tasks. I wonder what their performance is like compared to GPUs.
I would be quite interested in a comparison between Google's TPUs and this Sophon chip, but it seems it's quite new and they have barely started selling them, so we might have to wait.
> 25 ms/batch of 128 is 5120 images/second... Is that correct? Am I missing something?
I'm not particularly knowledgeable about this, so you will have to wait for someone experienced to chime in.
Wanted to note that Lambda Labs now offers Titan Vs on our Lambda Quad workstations. They're available here: https://lambdal.com/products/quad.
We're excited to see Volta come down below $5,000 / GPU. We're also looking forward to AMD, Intel, Graphcore, Rex, Bitmain, Vathys.ai, Wave Computing, Tesla or Cerebras providing competitive alternatives.
What's up with the hidden $857 "Labor/Shipping/Handling" charge? It seems a little out of place given that you guys already seem to be charging huge premiums on all the hardware.
This all seems really expensive compared to the local PC shops that seem to offer most of the components for less, while only charging ~100 euros for assembly and testing.
Do you guys have some really fancy cooling solution that you forgot to mention on the site?
Pretty much, yeah. Though I'm still curious if getting your computers "Built by Researchers" has some novel advantages over "Built by Lawyers" or "Built by the guys at the local computer shop"
Perhaps the presence of smart Researchers while the computer is being born will make the machine better at learning, improving tensorflow performance?
It has me wondering if people are actually buying these from them. If so, undercutting their pricing would be trivial - and maybe an easy idea for a side business.
Form a distributed team, package the orders one at a time saying shamanic ML chants over them while burning incense, ship to happy customers. Write bloviating blog posts about 'AI' that incorrectly reduces an entire field to a de jure technique. Grow business, cash out after first enterprise capture. Rinse. Repeat.
This is targeted at ML, which means this is targeted at Linux. Which means us system administrators have to deal with the piece of shit that is the Nvidia binary driver.
- it doesn't work with Wayland
- it doesn't work in UEFI mode with CSM enabled. There are motherboards available that won't boot in UEFI mode with CSM disabled without a monitor attached.
- it doesn't work with Secure Boot
- about half of their releases fail to work with at least one of the pieces of software we try to use it with. The current release 384.90-0ubuntu0.16.04.2 in particular crashes our software.
And for some reason Nvidia has the reputation for being good at drivers. FirePro was also really bad, but I haven't had to deal with that for years now...
Some of the software we use is linked against GL to display results when run on the desktop. It doesn't actually use GL when run on the server, but it's still linked against GL so we need it on the server.
What we do is we install the nVidia binary driver package inside the docker container, then delete some files that overlap with those that nvidia-docker injects.
AMD had released the most documentation and yet the community loves to pile on AMD. I think there is no company that could have done better than what they did in 2014 Linux Driver announcement https://www.phoronix.com/scan.php?page=article&item=amd_cata...
AMD also has the ROCm commitment going on. All the ROCm drivers are fully open source on GitHub. ROCm doesn't have display drivers, but the compute-drivers are more important for well... compute tasks. (You'll probably be doing development on another machine, transfer code to the ROCm server box and then execute it through SSH or something).
So forward looking, one can reasonably expect that AMD's Open Source drivers will be superior. But this only applies to RX 4xx series and newer, and not even to all those cards. (Some cards like RX 550 just don't work right now, despite claimed support on marketing pages).
There need to be more hardware companies putting any effort into ML. (Besides, say, Alphabet, which seems to be acting as a hardware company whose only customer is themselves.)
Intel has indicated that they're working on it, and that's who I'm hoping for right now. If Intel came up with on-chip ML operations that are fast enough and made them work with, say, TensorFlow and PyTorch, I think that researchers would abandon Nvidia pretty much instantly. I've never had to worry about my "Intel driver" breaking.
Interesting. I had assumed most people just did ML on Windows specifically because of graphics driver issues. Even if this list weren’t a problem, Windows drivers tend to outperform Mac and Linux drivers by a significant margin.
From my experience, the main pain point with Linux GPU drivers is for standard "desktop" tasks, like not breaking when Ubuntu switches to Wayland... A server designated for GPGPU applications just need the basic configuration to get the Nvidia toochain for CUDA up and running
It'd be nice if their CUDA libraries worked with the open source "nouveau" library. All the driver does is init the card and do some resource management; all the secret sauce is in the CUDA libraries.
The CUDA compiler would have to emit something loadable by Nouveau. The tools around CUDA have several hooks into the drive, especially for performance tracing.
I have way more graphics/cuda driver issues on Windows... I'm not seeing a performance delta that extreme either. If you are comparing AAA games, the driver does complete code rewrites on anything important, that's an apples/oranges comparison.
The NVIDIA driver is most certainly not a piece of shit.
NVIDIA is a bit behind on things like Wayland, but it has been way ahead of AMD for the last 10+ years on Linux for people running mission-critical software that requires good OpenGL drivers.
Try running with AMD hardware on RHEL/CentOS 6/7 (the OS many major VFX facilities use), you'll have a Bad Time (tm).
I am speaking as someone who has been maintaining software & hardware running Linux making heavy use of NVIDIA hardware for the last ten years.
The NVIDIA driver on macOS on the other hand, now there is a lamentable situation...
RHEL 6 was released in 2010. At that time Nvidia was way better than AMD. In the last 7 years AMD has improved dramatically, but nVidia has gotten worse.
Ubuntu and Red Hat will probably have new LTS versions using Wayland coming out in 2018. Which means that 2017 is the time to start testing for them.
And it's not really a glowing endorsement if Nvidia is only the second worst driver on Linux, not the absolute worst.
> "What AMD's seeking to do by changing up their high-performance Linux driver is having a completely free kernel driver that's relied upon by Catalyst and would live within the mainline Linux kernel in order to ease support for Linux consumers."
I think AMD has gone above and beyond, especially since 2014's above announcement. I think today's AMD are better than NVIDIA's and they are a much better company for Linux
My experience with consumer cards may not apply, but AMD drivers, especially their open source ones, have been very good for the past two years or so, especially after AMD's announcement of their intention to mainline much of the graphics driver. From Phoronix's test, they seem to have promising performance as well.
The main things that Mesa lacks for "pro" customers are support for OpenGL compatibility profiles, and ISV certification. You also generally will want to use AMD's proprietary OpenCL until ROCm stabilizes and the requisite patches are (of a quality suitable for) upstream. Though as a professional, and a user of a "professional" AMD card (Radeon WX Pro 7100, bought entirely for the amount of VRAM), I opt for Mesa.
What about the open-source drivers? The state of Nouveau (open source Nvidia drivers) is pretty terrible[1], right?
The proprietery AMD Catalyst™ (fglrx) drivers are known to be bad[2], but AMDGPU[3] is supposed to be the best, when it comes to open source GPU driver support on Linux, based on what I've read.
Better than Nouveau or Nvidia's proprietery blobs. Don't people sometimes buy AMD cards just for its excellent native/open-source driver support?
I'm not arguing that AMD's new open-source drivers appear to be pretty good, but you can't use them on current RHEL 7 since the RHEL kernel is frozen on the 3.10.x line, which locks out a lot of people from using AMD GPU hardware on Linux.
> People buy AMD cards just for its excellent native/open-source driver support.
I can confirm this being my main reason for going with AMD. They have discrete GPUs which meet my performance requirements, and the Mesa-based drivers perform very well, with very few issues (none on my own workloads). There has been a bit of a gap in display output features, specifically FreeSync support, but that's coming along too, and should be at least of "experimental" quality in Linux 4.15.
We have used previous Titan cards in lab workstations with Fedora Linux to get reasonably current versions of many different numerical libraries and apps.
A frustrating aspect of the NVIDIA drivers is the poorly thought out update protocol. For our academic lab environment, admins want to run nightly package updates as a general practice to get most network-facing security updates as soon as possible. However, this doesn't mean we expect to logout or reboot every machine every night. Lab users keep odd hours and run jobs at unpredictable crunch times.
In practice, nightly automated updates just work, since most lab apps lack dynamic library loader shenanigans which would get confused by new libraries on disk being swapped into place while an older process is still running older libraries in their memory space.
The update experience is that one day everything works, and then next all commands fail to get an OpenGL context because the NVIDIA-related package update replaced the user-space libraries with ones that are apparently not ABI compatible with the still running kernel module. It seems absurd to me that the kernel interface and user-space library are not designed to allow seamless updates. The new user-space driver should be willing to talk to the previous kernel shim as well as the new one which may have more features. It should not expect a flag-day update for every little periodic update.
The users have had to learn to reboot the workstation and hope that it will resolve the problem. Once in a while, the kernel shim apparently fails to recompile on the latest kernel update, and the reboot does not help. The admins then have to intervene to find some temporary solution such as downgrading the libraries or finding a previous kernel version where the latest kernel module successfully recompiles.
Your story sounds very familiar. Add to that list the regular regressions in new driver release that require rollback, but the binary blob drivers sometimes forget to clean up properly and you can end up in a library mixup hell... no fun.
On the other hand, in terms of robustness and reliability on the compute side of things, the NVIDIA drivers are decent. I've only experienced 1-2 hard kernel crashes the last year -- which used to be an at least weekly event 5+ years ago.
I still wish they'd split their driver and contributed upstream parts of it if for nothing else, to make compatibility layers and shims more robust, but knowing NVIDIA I doubt that's on their roadmap for the coming century. :)
I wish it was irrelevant, but CUDA requires the NVidia binary driver.
Why can't CUDA work with nouveau? All the special sauce is in the CUDA libraries, I'm sure all the driver does is init the card and do some resource management.
Nvidia 384 drivers on Ubuntu 17.10 have left me unable to reboot or shut down, I have to manually hard cycle off the power. And this is on Xorg, not Wayland.
And good luck getting Nvidia "Optimus" properly configured and running smoothly.
That's been the case for at least the last few years, and it's been quite common the last 5 years.
It's part of the 'gamer aesthetic'. If I could speculate, I'd suggest they might intentionally do this to push businesses away from consumer-level stuff.
Businesses shove them in opaque chassis. Businesses don't care. The primary means of pushing businesses away from consumer-level stuff is SKU-ing out nvlink and putting the power connector parallel to the PCI slot instead of perpendicular (makes it a 3.5U server instead of a 3U)
Nice clean design. I get that its largely subjective and comes down to personal preference but I remember being a teen and thinking a graphic card looked cool in their natural state (i.e seeing the PCB, chips and heatsinks/pipes)
Sadly this is no longer possible - the plastic shroud actually serves a cooling purpose by directing air out the back of the card rather than blower-style coolers that direct hot air into the rest of the case.
But I agree with you, "gamer" hardware aesthetic is grating. It's as if a F-22 and a Nerf Gun decided to have a child together.
The Titan is a blower-style card. Open-air coolers, on the other hand, do result in higher ambient case temperatures but typically keep the graphics card itself cooler blower-style (at lower fan speeds, therefore more quietly as well).
Bottom line is there are lots of factors, most significantly case design and cooling strategies for other components.
I’d flip this around and ask when did we start expecting kit to be so drab? This thing doesn’t hit the level of having, say, an SGI Onyx beside my desk, but did make me smile. I’ll remember that next time I need GPUs.
Also something I've been annoyed with in high end consumer routers for the last 10 years. Most of them have looked like either a cyberpunk spider or a batmobile.
Only recently can you avoid it with various manufacturers mesh routers or Ubiquiti units, albeit at low-end business prices.
It’s mostly a shroud around a few heat pipes, a fan, and a lot of heat sinks. It’s a necessary part since it directs airflow across all the heat sinks and pushes hot air out the back of the card and outside the case; how would you color it?
MSI's Aero lineup looks quite good, very similar to the Quadro cards.There are some simple white lights on the logo, although those can be disabled if they bother you.
A long time ago. I think these NVidia designs look good. 10 years ago, every card had some (usually) gamer-y character art on them. You'd put them in a case and almost never see them again.
I think you have some kind of selective amnesia. Graphics cards have become far more tasteful over the past decade. Consider these gaudy monstrosities, which are entirely representative of the gamer aesthetic circa 2007:
I do deep learning for a day job, this card is for me. Going on the specs it'll be worth it upgrading my 1080 TIs but I'll wait for a proper review before committing $12K.
That Radeon card has 2TB of onboard memory, and seems to be aimed at very heavy content creation workloads (8k VR). I think it's in a different (and ultra-niche) league to this Nvidia card. I would guess this is aimed at heavy ML workloads?
Very few people who do DL professionally or as casual hobbyists have the time to sort out ports of various frameworks to work with in-progress compute drivers/toolchain. Most engineers aren't cash strapped, they'll just buy what they need instead of waste time, especially when the field and the tools move so quickly. It's just not important enough when NVidia cards work well out of the box and can be acquired at both the hobbyist low end (GTX 1060) and professional high end (Tesla series), have proven track records, and every DL framework works more-or-less immediately.
They need the patches to be upstream and widely maintained by the developers (and be involved themselves) before people 'at large' will really consider using them. The code is there, but practically doesn't exist for anyone yet, outside of AMD developers and a minuscule amount of AMD users. This is also true of all of Intel's ports of various DL frameworks, too. One thing NVidia has learned is that sticking their engineers directly into the pipeline works -- from everything from game development studios, and now to deep learning frameworks (which have nvidia engineers).
That said, AMD cards do have some excellent advantages like unlocked FP16 that Nvidia does not offer outside of the Tesla series, and bringing card price down would rock. So I'm hopeful that they'll sort all this out and make themselves competitive. I'm hoping to see MIOpen support (and if we want ponies, an upstream ROCm driver) sometime in the next ~year for at least a few frameworks (Caffe2 or PyTorch would be good.) But until then, CUDA, CuDNN and Nvidia technology are really the only name in town for like +90% of people, for better or worse.
This seems like a shameless money grab. Even the historical "enterprise" GPUs were never this expensive. Nvidia seems like they are just taking their new GPU design which is legitimate (Volta) and trying to cash in on the ML mania.
GPUS should be general purpose and with Nvidia throwing all their eggs in the ML basket and charging a fortune it might risk alienating their core users (gamers).
I don't know much about the market for this type of GPU, but the article does note that the Tesla model (priced at about 10,000) is selling out as soon as they make them. It seems like the demand is there but the supply isn't.
It is true that Nvidia could be artificially reducing supply.
The Titan cards were not intended for the gaming market, though some gamers who wanted every last ounce of (severely diminished return) performance certainly tried nonetheless. I really don't see the issue.
Sometime early next year Nvidia will announce the Volta enthusiast cards (GTX 1170, 1180?), priced in line with the current-gen enthusiast cards (GTX 1070, 1080).
If you want the new shiny right now, you can, but as always you can get 85% of the same gaming performance for 25% of the price by waiting a few months.
This has 13 teraflops of single-precision performance, which is a small bump over the 12 of Titan Xp. But it also adds 110 teraflops of matrix-multiplication power with the dedicated tensor hardware. That means it can outperform 8 Titan Xp cards for certain increasingly-popular workloads, and still fit in the same physical and power budget as one. That's why they can charge $3k.
Not all GPU's need to be good at all things. There are plenty of general-purpose chipsets out there for those that want one, and I'd say there's room in the market for highly specialized GPU's and if those are priced incorrectly, the market will correct it. The irony in your statement is that this is designed to be a cheaper version of a very popular card.
Why is that exactly? This looks to be a great deal given its specs in comparison to the rest of the market, and us ML folks like our application specific GPUs because a specific card will always outperform a generic card and you're not paying for things you don't need. I can't even imagine what a generalist card which works for ML would look like. Do you actually work in this space?
I don't get the price complaints. This is targeted for deep learning and looks to be similar in performance to a card that currently costs 3x the amount. This is a huge price cut for the compute performance and should make cloud GPU time cheaper and/or more efficient.
> should make cloud GPU time cheaper and/or more efficient.
No way. These will be not put in data centers of cloud providers. I'm not even sure many regular data centers will have anything like this -- especially if NVIDIA keeps pushing and blackmailing server vendors to stop selling servers packed with GeForce cards if they don't want to loose big [1,2].
"With TITAN V, we are putting Volta into the hands of researchers and scientists all over the world"
Screw you NVIDIA. I've been vocal about this for a while but this is now getting utterly ridiculous, and frankly insuting to the research community.
I don't mind that NVIDIA is ripping of the big derplearning -- and more recently "data" -- companies (Goog, FB, Baidu, etc.), but this greedy bullshit attitude led to the Tesla cards all of a sudden going from expensive, with a 2-2.5x price jump, to the ridiculously priced. In the meantime, they've been "enabling" research by throwing peanuts at the science community with cancer stunts [1]. However, that was just the facade, the real story was that they'd found the cash-cow and the enthusiastic HPC/sci-comp crowd that helped NVIDIA grow out of their sexy-witch-on-the-front-panel-of-the-GTX company box and step into the "big compute boys' league" is no more than just another means to convert peanuts to marketing material.
After years of complaints that there are no sensibly prices _compute_ cards devs and independent researchers can afford, constant battles with NVIDIA to stop crippling their management tools on GTX cards, to allow cheap cards to be used productively for compute and development, etc. they step up their game and release a $3k "affordable" card for "researchers and scientists". This claim is beyond ridiculous, revolting and, as a researcher (working in HPC on a large FOSS simulation project) this is insulting. Which researchers are they talking about, the ones employed by the big ad, finance, companies? Definitely not me, not us!
Time for us to start showing NVIDIA the middle finger for good! Get porting people, support AMD, test ROCm, their openness and attitude receptive to public feedback and criticism has impressed me recently. At the same time, we need to push NVIDIA too: file bug reports against NVIDIA's shitty OpenCL support, against their consumer card crippling drivers and management tools, and resist their efforts to blackmail OEMs to stop selling servers with GeForce.
If you can't afford the $3K card, buy the $800 card. The $800 card that outperforms the fastest SIMD computer any amount of money could buy just a few years ago.
NVIDIA makes fast computers and sells them for market prices. I'm not sure we need to storm their castle with pitchforks.
Please show me where can you buy a $800 card that can be used to develop for the _compute_ architectures; i.e. the GV100 or the GP100 for that matter.
Also, please show me how can you use those $800 cards for serious computing/research without potentially running into silly issues and a lot of pain (from not being able to use the "supported" standard OpenCL to running into cooling, fan speed, clocking issues that makes reliable perf tuning a pain, etc.).
It is strange that you expect to be able to buy the server grade GPUs for GeForce series prices and are outraged when reality doesn't meet your expectations.
It strikes me as pretty awesome that the V100 is now available for only $3k in the Titan V, rather than ~$10k for the Tesla V100. I reckon you need to readjust your expectations.
> It is strange that you expect to be able to buy the server grade GPUs for GeForce series prices
Respectfully, you are clearly missing the point. You can buy <$500 USD Xeon Silver parts (and fairly affordable barebone workstation or servers to go with) and you'll get the Xeon-SP architecture to develop for. You can even buy the rebadged i9 parts too for ~$1000-$1200 with both FMA unit enabled, but admittedly those are also rather expensive.
> and are outraged when reality doesn't meet your expectations.
Untrue. I want an option, any option that allows researchers to develop for the fancy high-end chips -- especially when they claim that they want to and are providing such an option.
Computational scientist and HPC researchers who aim to write code for, or porting community codes to machines like DOE Sierra, Summit or future V100-based machines need something that is within the reach of a junior PI, a researcher in countries less well off than the rich western Universities and research institutes.
In contrast, jack up the price this high means that the Facebook et. al will still find it worthwhile, but the grand majority of actual researchers, those that they claim to be enabling and helping with this card will not be able to afford it.
It all too easy to forget how the "democratization" of supercomputing was NVIDIA's slogan very ambitious and cool [1] that has over time turned into a marketing claim [2] that offers little in terms of bottom-up enabling the community.
$3k is within reach for most labs. It’s lot of money for an individual. But a PhD researcher’s annual salary is probably more than 35 times that amount in a coastal city. An ML researcher could easily be 200 times that amount.
A new assistant professor at a medium sized school in North America can afford a few of these.
This is a cheap bit of equipment for a professional researcher.
It’s an expensive bit of equipment for an independent researcher or student, or someone in a poorer part of the world. But why would anyone expect the best kit for hobby prices?
I'm saddened by the arrogance you display. Does your definition of "world" really not go beyond the "coastal cities" and "North America"? Are you seriously assuming there are no ML researchers outside of the cozy little world you're referring to? People in the Middle-East, Balkans, South-Asia, etc. should not worry about "high-end kit", they should stick to "hobby" stuff?
(In case if this all leaves you baffled, e.g. assistant professors will often earn 2-4 TITAN V's worth of money per year in places like Eastern Europe -- and that's still on the map of the "Western world" by most definitions.)
> It’s an expensive bit of equipment for an independent researcher or student, or someone in a poorer part of the world.
Allow me to correct you: that is _much_ (if not most) of the world.
> But why would anyone expect the best kit for hobby prices?
Strawmen. Please read my point again (and see the concrete examples of affordable "kit" that can be used to target the high-end HPC iron).
132 comments
[ 3.2 ms ] story [ 185 ms ] threadYou could also buy ~4 1080Ti cards, but unless you can't use the FP16 tensor cores, you'd still lose heavily to a single V100.
A price premium that will really stack up when certain cloud providers want to buy some of these in bulk.
Edit: The *999 pricing confused me, it's actually $5000 more.
Edit: I didn't originally see that the die size of this chip is nearly double that of the older Titan XP. Perhaps the price isn't as outrageous as I first thought! I'm actually glad that there's a big enough market now to support this kind of ultra high end product. It's certainly a beast and I want one.
Prices as well as the tech itself. $3000 and still just 12GB. I wonder why this isn't in the range of ~64/128GB yet... too expensive for the type of memory?
Low end servers use normal dimm, but typically not many dimms per channel, typically 1-3. But to hit the higher memory configurations you end up putting buffer chips on the dimms which makes them more expensive and also slightly higher latency.
So 500GB/sec to 128GB ram using HBM2 would be difficult, expensive, and likely wouldn't fit in today's GPU form factors... yet.
GPUs generally seem to be mostly hungry for more bandwidth and not that many cases (percentage wise) is 16GB a major limitation. However in those cases check out the power9, the nvlink that x86's uses for GPU<->GPU communication can be used for CPU <-> GPU communication and avoid the high latency/low bandwidth pci-e links.
650 GB/s is pretty impressive.
I think it's pretty exciting seeing how we are getting increasingly more specific chips for AI tasks. I wonder what their performance is like compared to GPUs.
Is this because in China you're expected to ignore all certificate warnings?
> I wonder what their performance is like compared to GPUs.
Me too. I tried to find a similar table and found this: https://github.com/tobigithub/tensorflow-deep-learning/wiki/...
25 ms/batch of 128 is 5120 images/second... Is that correct? Am I missing something?
I can access it no problem with Chrome. The price seems to be 589$ per chip.
Google wrote an article some time ago about the their TPUs. There are a few charts at the end of the page: https://cloud.google.com/blog/big-data/2017/05/an-in-depth-l...
I would be quite interested in a comparison between Google's TPUs and this Sophon chip, but it seems it's quite new and they have barely started selling them, so we might have to wait.
> 25 ms/batch of 128 is 5120 images/second... Is that correct? Am I missing something?
I'm not particularly knowledgeable about this, so you will have to wait for someone experienced to chime in.
We're excited to see Volta come down below $5,000 / GPU. We're also looking forward to AMD, Intel, Graphcore, Rex, Bitmain, Vathys.ai, Wave Computing, Tesla or Cerebras providing competitive alternatives.
This all seems really expensive compared to the local PC shops that seem to offer most of the components for less, while only charging ~100 euros for assembly and testing.
Do you guys have some really fancy cooling solution that you forgot to mention on the site?
Perhaps the presence of smart Researchers while the computer is being born will make the machine better at learning, improving tensorflow performance?
JTFC!
- it doesn't work with Wayland
- it doesn't work in UEFI mode with CSM enabled. There are motherboards available that won't boot in UEFI mode with CSM disabled without a monitor attached.
- it doesn't work with Secure Boot
- about half of their releases fail to work with at least one of the pieces of software we try to use it with. The current release 384.90-0ubuntu0.16.04.2 in particular crashes our software.
And for some reason Nvidia has the reputation for being good at drivers. FirePro was also really bad, but I haven't had to deal with that for years now...
Some of the software we use is linked against GL to display results when run on the desktop. It doesn't actually use GL when run on the server, but it's still linked against GL so we need it on the server.
So forward looking, one can reasonably expect that AMD's Open Source drivers will be superior. But this only applies to RX 4xx series and newer, and not even to all those cards. (Some cards like RX 550 just don't work right now, despite claimed support on marketing pages).
Don't buy NVIDIA if you don't care about performance of OpenGL/etc.
Intel has indicated that they're working on it, and that's who I'm hoping for right now. If Intel came up with on-chip ML operations that are fast enough and made them work with, say, TensorFlow and PyTorch, I think that researchers would abandon Nvidia pretty much instantly. I've never had to worry about my "Intel driver" breaking.
- nVidia cards work excellently on Linux,
- they are the only choice for serious graphics, GPGPU and ML professionals
- the driver problems are overblown. In comparison to the world AMD has caused me over the years they are angelic.
NVIDIA is a bit behind on things like Wayland, but it has been way ahead of AMD for the last 10+ years on Linux for people running mission-critical software that requires good OpenGL drivers.
Try running with AMD hardware on RHEL/CentOS 6/7 (the OS many major VFX facilities use), you'll have a Bad Time (tm).
I am speaking as someone who has been maintaining software & hardware running Linux making heavy use of NVIDIA hardware for the last ten years.
The NVIDIA driver on macOS on the other hand, now there is a lamentable situation...
Ubuntu and Red Hat will probably have new LTS versions using Wayland coming out in 2018. Which means that 2017 is the time to start testing for them.
And it's not really a glowing endorsement if Nvidia is only the second worst driver on Linux, not the absolute worst.
https://www.phoronix.com/scan.php?page=article&item=amd_cata...
I think AMD has gone above and beyond, especially since 2014's above announcement. I think today's AMD are better than NVIDIA's and they are a much better company for Linux
Is enterprise hardware extremely different?
The proprietery AMD Catalyst™ (fglrx) drivers are known to be bad[2], but AMDGPU[3] is supposed to be the best, when it comes to open source GPU driver support on Linux, based on what I've read.
Better than Nouveau or Nvidia's proprietery blobs. Don't people sometimes buy AMD cards just for its excellent native/open-source driver support?
[1] https://www.phoronix.com/scan.php?page=article&item=nouveau-...
[2] https://www.phoronix.com/scan.php?page=article&item=openclos...
[3] https://help.ubuntu.com/community/AMDGPU-Driver | https://help.ubuntu.com/community/RadeonDriver
I can confirm this being my main reason for going with AMD. They have discrete GPUs which meet my performance requirements, and the Mesa-based drivers perform very well, with very few issues (none on my own workloads). There has been a bit of a gap in display output features, specifically FreeSync support, but that's coming along too, and should be at least of "experimental" quality in Linux 4.15.
A frustrating aspect of the NVIDIA drivers is the poorly thought out update protocol. For our academic lab environment, admins want to run nightly package updates as a general practice to get most network-facing security updates as soon as possible. However, this doesn't mean we expect to logout or reboot every machine every night. Lab users keep odd hours and run jobs at unpredictable crunch times.
In practice, nightly automated updates just work, since most lab apps lack dynamic library loader shenanigans which would get confused by new libraries on disk being swapped into place while an older process is still running older libraries in their memory space.
The update experience is that one day everything works, and then next all commands fail to get an OpenGL context because the NVIDIA-related package update replaced the user-space libraries with ones that are apparently not ABI compatible with the still running kernel module. It seems absurd to me that the kernel interface and user-space library are not designed to allow seamless updates. The new user-space driver should be willing to talk to the previous kernel shim as well as the new one which may have more features. It should not expect a flag-day update for every little periodic update.
The users have had to learn to reboot the workstation and hope that it will resolve the problem. Once in a while, the kernel shim apparently fails to recompile on the latest kernel update, and the reboot does not help. The admins then have to intervene to find some temporary solution such as downgrading the libraries or finding a previous kernel version where the latest kernel module successfully recompiles.
On the other hand, in terms of robustness and reliability on the compute side of things, the NVIDIA drivers are decent. I've only experienced 1-2 hard kernel crashes the last year -- which used to be an at least weekly event 5+ years ago.
I still wish they'd split their driver and contributed upstream parts of it if for nothing else, to make compatibility layers and shims more robust, but knowing NVIDIA I doubt that's on their roadmap for the coming century. :)
I'd bet on seeing open source drivers in the kernel at least within the next 5 years, probably sooner
Why can't CUDA work with nouveau? All the special sauce is in the CUDA libraries, I'm sure all the driver does is init the card and do some resource management.
And good luck getting Nvidia "Optimus" properly configured and running smoothly.
It's part of the 'gamer aesthetic'. If I could speculate, I'd suggest they might intentionally do this to push businesses away from consumer-level stuff.
http://images.bit-tech.net/content_images/2016/06/amd-radeon...
But I agree with you, "gamer" hardware aesthetic is grating. It's as if a F-22 and a Nerf Gun decided to have a child together.
Bottom line is there are lots of factors, most significantly case design and cooling strategies for other components.
Only recently can you avoid it with various manufacturers mesh routers or Ubiquiti units, albeit at low-end business prices.
Wish they'd ship the a version of the GTX line with the design language of the more boxy looking Quadro cards (M4000 etc).
https://www.newegg.com/Product/Product.aspx?Item=9SIA85V4R81...
http://images.dailytech.com/nimage/6446_large_8800GT-512DDR3... https://images.anandtech.com/reviews/video/ATI/2900XT/2900xt... https://images10.newegg.com/NeweggImage/ProductImage/14-102-...
The gold shroud on the Titan V is a bit bling-bling, but at least the shroud design is part of a consistent design language from NVIDIA.
https://twitter.com/BitsBeTrippin/status/939031211233525760
Just ML? Higher memory bandwidth?
https://pro.radeon.com/en/product/pro-series/radeon-pro-ssg/
https://rocm.github.io/dl.html
They need the patches to be upstream and widely maintained by the developers (and be involved themselves) before people 'at large' will really consider using them. The code is there, but practically doesn't exist for anyone yet, outside of AMD developers and a minuscule amount of AMD users. This is also true of all of Intel's ports of various DL frameworks, too. One thing NVidia has learned is that sticking their engineers directly into the pipeline works -- from everything from game development studios, and now to deep learning frameworks (which have nvidia engineers).
That said, AMD cards do have some excellent advantages like unlocked FP16 that Nvidia does not offer outside of the Tesla series, and bringing card price down would rock. So I'm hopeful that they'll sort all this out and make themselves competitive. I'm hoping to see MIOpen support (and if we want ponies, an upstream ROCm driver) sometime in the next ~year for at least a few frameworks (Caffe2 or PyTorch would be good.) But until then, CUDA, CuDNN and Nvidia technology are really the only name in town for like +90% of people, for better or worse.
Titan V has HMMA, so it most likely also has full hardware FP16. It seems to have a binned V100 chip with 3 functioning HBM2 stacks.
GPUS should be general purpose and with Nvidia throwing all their eggs in the ML basket and charging a fortune it might risk alienating their core users (gamers).
It is true that Nvidia could be artificially reducing supply.
Sometime early next year Nvidia will announce the Volta enthusiast cards (GTX 1170, 1180?), priced in line with the current-gen enthusiast cards (GTX 1070, 1080).
If you want the new shiny right now, you can, but as always you can get 85% of the same gaming performance for 25% of the price by waiting a few months.
Not all GPU's need to be good at all things. There are plenty of general-purpose chipsets out there for those that want one, and I'd say there's room in the market for highly specialized GPU's and if those are priced incorrectly, the market will correct it. The irony in your statement is that this is designed to be a cheaper version of a very popular card.
Why is that exactly? This looks to be a great deal given its specs in comparison to the rest of the market, and us ML folks like our application specific GPUs because a specific card will always outperform a generic card and you're not paying for things you don't need. I can't even imagine what a generalist card which works for ML would look like. Do you actually work in this space?
> should make cloud GPU time cheaper and/or more efficient.
No way. These will be not put in data centers of cloud providers. I'm not even sure many regular data centers will have anything like this -- especially if NVIDIA keeps pushing and blackmailing server vendors to stop selling servers packed with GeForce cards if they don't want to loose big [1,2].
[1] https://www.pcgamesn.com/nvidia-geforce-server [2] https://www.digitimes.com/news/a20171027PD200.html
Screw you NVIDIA. I've been vocal about this for a while but this is now getting utterly ridiculous, and frankly insuting to the research community.
I don't mind that NVIDIA is ripping of the big derplearning -- and more recently "data" -- companies (Goog, FB, Baidu, etc.), but this greedy bullshit attitude led to the Tesla cards all of a sudden going from expensive, with a 2-2.5x price jump, to the ridiculously priced. In the meantime, they've been "enabling" research by throwing peanuts at the science community with cancer stunts [1]. However, that was just the facade, the real story was that they'd found the cash-cow and the enthusiastic HPC/sci-comp crowd that helped NVIDIA grow out of their sexy-witch-on-the-front-panel-of-the-GTX company box and step into the "big compute boys' league" is no more than just another means to convert peanuts to marketing material.
After years of complaints that there are no sensibly prices _compute_ cards devs and independent researchers can afford, constant battles with NVIDIA to stop crippling their management tools on GTX cards, to allow cheap cards to be used productively for compute and development, etc. they step up their game and release a $3k "affordable" card for "researchers and scientists". This claim is beyond ridiculous, revolting and, as a researcher (working in HPC on a large FOSS simulation project) this is insulting. Which researchers are they talking about, the ones employed by the big ad, finance, companies? Definitely not me, not us!
Time for us to start showing NVIDIA the middle finger for good! Get porting people, support AMD, test ROCm, their openness and attitude receptive to public feedback and criticism has impressed me recently. At the same time, we need to push NVIDIA too: file bug reports against NVIDIA's shitty OpenCL support, against their consumer card crippling drivers and management tools, and resist their efforts to blackmail OEMs to stop selling servers with GeForce.
[1] https://www.nextplatform.com/2016/11/15/deep-learning-superc...
NVIDIA makes fast computers and sells them for market prices. I'm not sure we need to storm their castle with pitchforks.
Also, please show me how can you use those $800 cards for serious computing/research without potentially running into silly issues and a lot of pain (from not being able to use the "supported" standard OpenCL to running into cooling, fan speed, clocking issues that makes reliable perf tuning a pain, etc.).
It strikes me as pretty awesome that the V100 is now available for only $3k in the Titan V, rather than ~$10k for the Tesla V100. I reckon you need to readjust your expectations.
> and are outraged when reality doesn't meet your expectations.
Untrue. I want an option, any option that allows researchers to develop for the fancy high-end chips -- especially when they claim that they want to and are providing such an option. Computational scientist and HPC researchers who aim to write code for, or porting community codes to machines like DOE Sierra, Summit or future V100-based machines need something that is within the reach of a junior PI, a researcher in countries less well off than the rich western Universities and research institutes.
In contrast, jack up the price this high means that the Facebook et. al will still find it worthwhile, but the grand majority of actual researchers, those that they claim to be enabling and helping with this card will not be able to afford it.
It all too easy to forget how the "democratization" of supercomputing was NVIDIA's slogan very ambitious and cool [1] that has over time turned into a marketing claim [2] that offers little in terms of bottom-up enabling the community.
[1] http://gpgpu.org/static/asplos2008/ASPLOS08-1-intro-overview... [2] http://assets.nvidia.com/nv/tesla/pdf/NVIDIA_Accelerate%20Yo...
A new assistant professor at a medium sized school in North America can afford a few of these.
This is a cheap bit of equipment for a professional researcher.
It’s an expensive bit of equipment for an independent researcher or student, or someone in a poorer part of the world. But why would anyone expect the best kit for hobby prices?
(In case if this all leaves you baffled, e.g. assistant professors will often earn 2-4 TITAN V's worth of money per year in places like Eastern Europe -- and that's still on the map of the "Western world" by most definitions.)
> It’s an expensive bit of equipment for an independent researcher or student, or someone in a poorer part of the world.
Allow me to correct you: that is _much_ (if not most) of the world.
> But why would anyone expect the best kit for hobby prices?
Strawmen. Please read my point again (and see the concrete examples of affordable "kit" that can be used to target the high-end HPC iron).