If they get TensorFlow/Torch/MXNet running with no changes in the code, 32GB HBM2 for training models would be awesome! These days hitting memory limitations is biting more than general slowness of GPUs during Deep Learning training.
Not with mainline kernel (which supports up to Polaris), but it it already got merged into 4.18 that is going to be released in a couple of weeks. Until then it requires a bit extra effort https://github.com/RadeonOpenCompute/ROCm#dkms-driver-instal...
It depends on your dataset. For example, a single class (face, pedestrian or car) object detection model with ten of thousands of positive samples should take at least 18 hours to complete on a modern Intel CPU with a target FPR (false positive rate) set to 0.00001. The lower the fpr is, the more training time is required.
AMD is scrambling to build a CUDA-esque thing called ROCm and get various frameworks including TF ported over, but it’s not even in “early adopter” state yet, so i doubt anyone is using it. However they seem to be making progress.
Could it really be that hard? They’ve had at least a few years heads up and this feels like something that could be accomplished relatively quickly with 5-10 10x engineers. As a high profile project that shouldn’t have difficulty finding the talent.
I wonder if this will get the Crypto-mining people interested too? I am really looking forward to them having more specific hardware to waste their money on, so we can return to buying GPU's for gaming at a sensible price!
Ideally the increased demand from to crypto-mania would be channeled to fund R&D for Graphics & AI chips but AFAIK its mostly the middlemen (distributors and retailers) who pocket these premiums.
The more difficult the hash the stronger the given blockchain. Faster chips is not what users of crypto benefit from, we want larger networks of chips no more powerful than those imagined by the blockchain creators.
>> I am really looking forward to them having more specific hardware to waste their money on
That's not how it works. You won't make GPUs that suck up all the demand while you get good gaming GPUs. If it can process stuff fast - either in computation or in memory speeds - then cryptominers want it, and so do you. The card you wish to exist simply doesn't, and if the cards are good, then they are profitable, which means they buy more of everything.
Most intelligent GPU miners have a diversified farm of Nvidia and AMD cards in some proportion to cover the specific algos that are good on each platform.
basically 7nm+ means tweaks to the process with improved speed and power consumption, stays 7nm from [0]
""" TSMC plans to introduce a second improved process called 7nm+ a year later, which will introduce some layers processed with EUVL. This will improve yields and reduce fab cycle times. The 7nm+ process will deliver improved power consumption and between 15-20% area scaling over their first generation 7nm process. """
To add to other answers, at both GlobalFoundries and TSMC:
- 7nm has have very limited extreme ultraviolet patterning on larger features of the chip
- 7nm+ will have significantly more EUV patterning which will allow shrinking some stuff that wasn't for 1st process iteration because yields would be too low - should have slightly higher density, possibly higher clocks/lower power draw.
Here's a benchmark comparing the latest ROCm software with Vega cards from 2017 (with tensorflow 1.3).
TLDR; - Vega's are nearly as good as V100s for CIFAR-10!
That post doesn't specify whether the training was fine with single or half precision floating point. Given that it's on the website of an AMD GPU cloud provider, I tend to think they probably used fp32 since it will make vega look better.
43 comments
[ 1.6 ms ] story [ 93.6 ms ] threadhttps://github.com/ROCmSoftwarePlatform/tensorflow
https://github.com/ROCmSoftwarePlatform/cutorch_hip
https://github.com/ROCmSoftwarePlatform/mxnet
https://github.com/ROCmSoftwarePlatform/hipCaffe
Not with mainline kernel (which supports up to Polaris), but it it already got merged into 4.18 that is going to be released in a couple of weeks. Until then it requires a bit extra effort https://github.com/RadeonOpenCompute/ROCm#dkms-driver-instal...
> Keras?
over opencl with https://github.com/plaidml/plaidml
And even ignoring that, you still can’t use PyTorch.
https://sod.pixlab.io/c_api/sod_realnet_train_start.html
https://sod.pixlab.io
https://sod.pixlab.io/samples.html for some code samples.
The commenter asked how the performance compares to an NVidia 1080Ti GPU.
A precise benchmark may not be necessary but in general, it will take for example how many Skylake cores at 3.5 Ghz to approximate the GPU example?
https://github.com/pytorch/pytorch/blob/master/tools/amd_bui...
No small task really. ML models are not even resistant enough to same framework upgrade, let alone hardware switch.
That's not how it works. You won't make GPUs that suck up all the demand while you get good gaming GPUs. If it can process stuff fast - either in computation or in memory speeds - then cryptominers want it, and so do you. The card you wish to exist simply doesn't, and if the cards are good, then they are profitable, which means they buy more of everything.
Most intelligent GPU miners have a diversified farm of Nvidia and AMD cards in some proportion to cover the specific algos that are good on each platform.
Wouldn't hold your breath.
That looks like things will begin getting bigger again in a few years?
""" TSMC plans to introduce a second improved process called 7nm+ a year later, which will introduce some layers processed with EUVL. This will improve yields and reduce fab cycle times. The 7nm+ process will deliver improved power consumption and between 15-20% area scaling over their first generation 7nm process. """
[0] https://en.wikichip.org/wiki/7_nm_lithography_process
update: link
- 7nm has have very limited extreme ultraviolet patterning on larger features of the chip
- 7nm+ will have significantly more EUV patterning which will allow shrinking some stuff that wasn't for 1st process iteration because yields would be too low - should have slightly higher density, possibly higher clocks/lower power draw.
http://blog.gpueater.com/en/2018/04/23/00011_tech_cifar10_be...