4x Stacks of HBM2 means 1TBps memory bandwidth at 16GB. Only 60-compute units enabled (maybe 4-CUs are expected to break during manufacturing? Its a weird number for sure...).
Since it shares dies with the MI50, the Radeon VII will have 1/2 speed double-precision, making this the cheapest high-performance double-precision card in existance.
------
FP16 compute is supported at double speed, but there are no tensor cores. So FP16 matrix multiplication / tensor ops are still a major benefit to NVidia.
But the memory size and bandwidth is quite salivating. That's a lot of bandwidth, and a number of problems are known to be memory-bound. Deep learning enthusiasts probably will stick to NVidia cards, but other compute problems may want to start playing around with this thing.
------
Video gamers seem meh about the specs. But I think anyone looking at this card for its compute performance would be impressed.
> Deep learning enthusiasts probably will stick to NVIDIA cards
We've been building the wrong hardware for ML for a while now.
A TPU doesn't delivery supremacy for problems over GPU hardware. It's a Google senior engineering retention and PR project.
Exceedingly few problems resemble image recognition. You wouldn't be able to tell from the research, because due to the tools it's sort of the only affordable thing to do.
Playing Go barely looks like it. Or more accurately, Go doesn't look like most other games. Molecular physics and synthesis barely looks like it. Neural programs/neural Turing machines that matter (converting complex sim code like game rules and physics into neural networks via learning) may never achieve the accuracy or performance/watt necessary to really compete with just making a bunch of CPUs or straight up dedicated hardware.
Word embeddings are a disaster. People keep trying to do innovative stuff with it, and it's 2018 and we've just just recently got entity names from Wikipedia. I think Peter Norvig, having basically only machine translation to point to, is really going to eat crow when non-perceptual AI tasks sort of have nothing to do with stuff Google has developed. But what do I know.
I'm sure someone's going to trot out some obscure TPU or GPU-driven thing. It's been years! Awesome innovations in hardware really do lead to obvious, immediate crazy cool stuff. I'm really talking about the Kinect as an example here, in that it was really hyped and not much came of it. The TPU, and ML-specified hardware targeted to today's problems, is a slower-motion Kinect of our time.
Mining cryptocurrency, the most recent innovation, is the opposite of cool.
I'm confident these tools exist to capitalize on the subsidy from the gaming industry. AMD is wise to not chase ML features.
The Japanese PEZY machine looks incredibly interresting. Too bad it is only used in that one super-computer.
Its overall design sits somewhere between a GPU and CPU. Like a GPU, it supports 8-way hyperthreading (GPUs typically support 10. CPUs usually only support 2... maybe 4 for Larabee). Like a GPU, its programmed through an OpenCL-like language (I know you don't like that, but its how it works).
However, the overall layout is very CPU-like as well: a "City" is composed of "Villages", which is composed of multiple "Processing Elements". Each PE is scalar like a normal CPU, although 8-way SMT. So its not like a GPU's SIMD or SIMT architecture in this regards.
I'd assume because of the extra serial processing power of a CPU.
x86 isn't the right choice though. As the chips get smaller, all the x86 cruft starts to become a larger portion of each core.
RISCV with 40-ish instructions plus a SIMD extension would probably be ideal. They've already proven that those designs use less space per core than ARM.
I am interested in seeing benchmarks regarding this. I believe this is true for models with truly huge embeddings, but otherwise if your models are even a little compute dense then GPUs are faster.
My experience is that you need the additional caveat of a streaming fairly homogeneous and highly parallelizable approach or the memory transfer, branching and communication overheads will eat away nearly all the GPU gains.
I've also noticed that a lot of the time, people are comparing GPU to naive implementations (and sometimes, implementations written in dynamic languages) instead of to highly tuned BLAS or MKL implementations. For a large swathe of problem types, using the fastest math library will reduce the CPU/GPU gap to less than an order of magnitude or less.
> comparing GPU to naive implementations (and sometimes, implementations written in dynamic languages) instead of to highly tuned BLAS or MKL implementations
In your experience, are these comparisons using naive GPU implementations as well? If so, that still seems like a valuable comparison.
I don't dispute this. If you can remove memory transfer overhead and meet the requirements I mentioned, then GPUs will be much better. But there are many problems that do not fit that (SIMD per warp) regime. In short, GPUs are no panacea and have their own trade-offs and bottleneck sensitivities, just like everything else.
I agree that they are no panacea, but we also have an example of the Intel Xeon phi, which did have high speed hbm memory. That too did not compete with GPUs. I think the majority of it has to do with the GPU chip itself being extremely simple, with no Branch prediction, no branch difference, very simple caching behavior, etc. They are able to allocate a much larger portion of the die to just throw fma units at it.
Well, I'm speaking from personal experience, for many (but not all) text-classification IO-intensive models with large embedding tables Xeon on TF compiled with AVX2 or AVX514 and FMA is as fast as GPU. Obviously these models need to be on the lower spectrum of computational complexity.
> PrimeSense was never solely responsible for the technology in Microsoft’s Kinect — as evidenced by the huge improvements Microsoft made to Kinect 2.0 on its own — and it’s also obvious that Apple is doing plenty of new software and processing work on top of this hardware. But the basic idea of the Kinect is unchanged. And now it’s in a tiny notch on the front of a $999 iPhone.
Matrix multiplication is the bread-and-butter of ML (both training and inference), for fairly obvious and solid reasons. Luckily for us, it's also very important in all sorts of domains, including graphics, rendering and gaming. That's all there is to it. There's no real "subsidy from the gaming industry", only a cross-subsidy situation in which common problems that are shared across domains can take advantage of the increased scale that comes from sharing the GPU-compute platform.
I'm not sure what you mean by "ML features", but AMD/ATI's dismissive attitude towards the ML and GPU-compute communities is looking quite silly by now.
I'm not sure what you mean by "ML features", but AMD/ATI's dismissive attitude towards the ML and GPU-compute communities is looking quite silly by now.
I disagree, ML computing has tons of competition and its margins will be quickly diminish. I think they are right to focus on gaming.
I don't know dude, if there weren't such lucrative jobs for academics at places like Google and pharmaceutical research companies, the discourse wouldn't be so narrowly focused on doing blah blah neural networks on huge libraries of images and molecular hashes.
People with academic backgrounds would be less afraid to point out how much vaporware machine learning has produced. They'd take stock in that their tenure, or six year PhDs, give them an opportunity to not chase trends so aggressively. They'd shill online less, for some giant faceless corporation like Google.
I'm not going to shit on a bunch of great stuff people have made. But the scary thing is that Noam Chomsky is probably right, that AI science hasn't advanced much at all. That we're still in an AI winter where it matters, the science, despite all the money and attention.
> the discourse wouldn't be so narrowly focused on doing blah blah neural networks on huge libraries of images and molecular hashes.
Do you mind clarifying what exactly your issues are with these specific applications of machine learning? Is it that you think there isn't really any meaningful value that can be gained in working within these areas? Or is it just that you want people to spend more time researching a different topic that you think would be more interesting/compelling?
Care to expand? Prima facie this is a very reasonable stance. Further, image data has a lot of structure not present in other data and it's far from obvious what to do with ML techniques in other contexts.
This is coming from a physicist starting to investigate ML tools in my own work.
Some of the greatest advancements in research and application of machine learning have been within image recognition in the past few years. The inflection point and subsequent boom of deep learning circa 2012 is basically marked by the ImageNet competition of that year.
If it could somehow actually sell for the MSRP (which of course it won't), it is a better gaming card than the RTX 2080. The 2080 costs more, and it's flagship feature is something you'll almost certainly want to turn off for gaming since it kills your frame rate and doesn't provide anything most people will notice while playing a game. Screen space reflections and cubemaps are convincing enough when you're zipping around a game world.
Well, that's why I linked to PCPartPicker, to show that RTX 2080 models at MSRP do exist. They may have been scarce at launch, but they can be purchased today less than 4 months later.
Since AMD will sell the Radeon VII directly at first, I think early adopters will probably face limited quantities rather than price markups (not counting unauthorized resellers). After that, we'll have to see how the situation with retailers and later on non-reference models shakes out.
There's no guarantee this won't have gimped FP64, Hawaii for example had 1/2 rate FP64 but the consumer cards were 1/8 speed. But even 1/8 would leave the 1/32 NVIDIA cards in the dust.
If they don't gimp FP64 it won't leave a whole lot of reason to buy the MI50 for probably 10x the price.
Not at a 10x price difference it isn't, at least if you're at all price conscious and looking for workstations / small clusters. This makes it far cheaper to just re-run simulations as necessary to get the same statistical guarantees with regards to memory corruption errors.
What AMD needs right now is marketshare so more stuff gets written in something cross-platform instead of cuda. I'd imagine that 1/2 has at least been discussed at great length for this card.
It looks like one of the card’s headline features, 1TB/sec of memory bandwidth, was only achievable with a 16gb design (basically a doubling of channels/memory over the previous Vega 64 design, with which this card shares many similarities).
> They are presumably not going to be able to match NVIDIA’s energy efficiency, and they won’t have feature parity since AMD doesn’t (yet) have its own DirectX Raytracing (DXR) implementation.
I assume this means that it won't support raytracing at all? Or will it, but with some non-DXR implementation?
NVidia's raytracing is hardware accelerated. There's specific cores that can ONLY traverse a BVH tree built into the NVidia hardware. Furthermore, NVidia's raytracing features leverage FP16 matrix multiplication for denoising.
The Radeon VII will certainly be slower at raytracing (if AMD ever decides to support the feature). It has FP16 support and huge RAM bandwidth (which will help traverse a BVH Tree), but I would expect it to be many times slower than dedicated hardware units. Without tensor cores (VII only supports hardware-accelerated FP16 Dot-products), there's no way it'd keep up with Nvidia on denoising or BVH traversals.
Based on the pixel sparkling in the video I've seen if Battlefield, it looks like Nvidia's raytracing is being used for each pixel. I'm not sure if it has to be that way or if it's just underoptomozed at the moment.
I think you could get something 95% as good with far better performance by only tracing vertices and then raster rendering the resultant transform.
Would be a good approach for accurate shadows too where you don't really care about texture fragments.
> Based on the pixel sparkling in the video I've seen if Battlefield, it looks like Nvidia's raytracing is being used for each pixel. I'm not sure if it has to be that way or if it's just underoptomozed at the moment.
1-sample per pixel is already incredibly low for raytracing. A raw 1 spp raytraced image looks HORRIBLE.
The reason why it looks anywhere close to usable is because NVidia post-processes all of that noise through their tensor-cores FP16. It is some kind of spacial / temporal-filter effect (averaging pixels across time, and space... where "space" is mapped to geometry framebuffers to keep object boundaries correct).
If your physically accurate rendering is pushing the hardware limits so far that you need extensive post processing to maintain the framerate then it stops being so physically accurate.
At that point, it seems like there are more interesting things you can do with the hardware than a naive raytrace that would look and perform better while, admittedly, not being 100% physically accurate.
I'd be very surprised if most common usage of RTX would turn out to be anything even remotely physically accurate.
A more likely use in the immediate future is to simplify some parts of the rendering pipeline, and improve quality on a few others. Without looking too much into to it, improving and simplifying soft shadows and are lighting seems like a no-brainer. Both are really hard to implement efficiently using rasterization, and/but both needs a good denoiser.
Will be interesting to see thought. I believe it's a bit later than I predicted that we get to see real time ray tracing on commodity hardware, but what is a couple of years over a few decades?
If I understand you correctly, they mix rays coming from one spot over many frames instead of shooting more rays in a single frame? That sounds great and a bit inspired by natural vision. Optical flow sensors are cheap enough for computer mouses and the like. But who says it's really just one ray? The number of frames to integrate would be too high
Of course that'd work for fps as slow as battle of duty \s
It likely won’t without ray tracing specific hardware. Theoretically, they don’t need the hardware and can use the directx ray tracing api and do software acceleration. It will just run very very slowly.
In July 2015 AMD released a 28nm chip with more GCN compute units than this card at a lower price - the Radeon R9 Fury X had the full 64 CUs / 4096 shaders enabled for $649. And it was considered a bit of a dud as a product. This Radeon VII is 2.5 process nodes better, but launches with 60 CUs / 3840 shaders enabled. All still on essentially the GCN architecture. And it launches at $699, more than three and a half years after the former.
Yet due to sheer clock speed increases and HBM improvements (16GB, 1 TB/s, wow) they actually seem surpisingly competitive. I'm both incredibly impressed and incredibly underwhelmed.
Depends on how you define competitive, in gaming which this card at least partially is aimed at its about as fast as a 1080ti a card launched 2 years ago for $700.
They can’t do 8 GB there are no 2GB HBM2 modules apparently and they can’t cut the amount of modules without cutting the CUs in half.
It’s essentially just to keep up some production volume this is their HPC die sold to the consumer market its the Titan V equivalent die of AMD minus the Tensor Cores just for much much cheaper.
It’s likely going to be a flop gaming wise, 2x8 pin power connectors put it at 300W TDP with performance only matching that of a 1089ti/2080 in the best case scenarios AMD currently has.
The 1080ti can be had for under $600 if you can find it the 2080 is around $700 but you get RTX and DLSS which while isn’t that useful yet is going to be more useful for gaming than 16GB of memory.
It would all depend on next generation of consoles and their release year. I bought 3Gb card 4 years ago, when top cards from both manufacturers had 3 and 4 Gb and it was an overkill. Now it seems that 8 Gb is a normal thing (same as PS4) and at least 4 is mandatory for AAA games.
Vega II is 60CUs at 1.8 GHz. AMD Fury is 64CUs at 1GHz.
60 CUs at 1.8GHz is way faster than AMD Fury ever was. Not only do you get the GHz advantage, but more L2 cache, more HBM2 Bandwidth, and architectural advancements (various instruction set improvements like double-speed FP16 support)
EDIT: AMD Fury's main advantage was that 1GHz clock made all of the nanosecond-level timings super easy to understand. 10-clock cycles was 10-nano seconds. Hurrah!
I think it shows that GCN is reaching the end of its usable life. It's not been a bad microarchitecture - but hopefully Navi does something different so they aren't hamstrung in the same way.
NAVI is still GCN likely the same 15-25% uptick over Polaris as we’ve seen with this new Vega, 2020-2021 is when the next gen uArch is going to come.
GCN’s vector machine doesn’t scale as well as NVIDIA’s scalar architecture NVIDIA can add cores without caring about concurrency or ILP, since each core is an individually addressable scalar ALU, this is why “async” compute doesn’t benefit them as much as it does GCN cards that have a 4 wide SIMD array that is nearly as hard to feed as their old VILW arch and a massive external compute scheduler that sits idle when doing graphics.
Your comment is extremely confused. Let's unpack some of the confusion...
> NVIDIA’s scalar architecture
NVidia doesn't have a scalar architecture in any meaningful sense of the word. Individual work items execute in warps, meaning (up to) 32 items execute the same instruction, at the same PC. If you have divergence, i.e. fewer than 32 items at the same instruction, you lose ALU throughput. This is the same as in AMD (and every other GPU architecture out there, for that matter).
The difference to AMD's architecture since Volta is that there is hardware support for each SIMD lane having its own PC, which makes it easier to have independent forward progress and a bunch of other features. The separate lanes don't have separate instruction fetch, though -- they all execute the same instruction (and if they can't, then some lanes will be disabled).
> this is why “async” compute doesn’t benefit them as much as it does GCN cards
Nvidia literally cannot do async compute in the same way as AMD, because their micro-architecture cannot switch between graphics and compute on as finely grained a basis as GCN cards. That's why they don't benefit from async compute. (Though I'm not sure whether that's still true since Volta.)
> GCN cards that have a 4 wide SIMD array
GCN has 64-wide waves, though with a bit of a weird execution scheduling which means that 16 lanes out of the 64 are computing at a time. Nowhere in GCN are there any 4-wide SIMD arrays.
> a massive external compute scheduler that sits idle when doing graphics.
Also nonsense. Both AMD and Nvidia have the majority of die space allocated to the compute cores (whatever they're calling them). The pieces of fixed function logic that distribute work onto the cores (whether for graphics or compute) are comparatively tiny.
>NVidia doesn't have a scalar architecture in any meaningful sense of the word.
CUDA threads are scalar, each CUDA core is a single scalar ALU, which CUDA cores are assigned to a warp is flexible.
>Nvidia literally cannot do async compute in the same way as AMD, because their micro-architecture cannot switch between graphics and compute on as finely grained a basis as GCN cards.
Both AMD and NVIDIA GPUs have to do context switching. The difference is in scheduling GCN has a dedicate scheduler with 8 queues only for compute that is the ACE it sits utterly idle while doing graphics the graphics scheduler sits within each CU.
>Nowhere in GCN are there any 4-wide SIMD arrays.
Each CU contains an array of 4 SIMD units which each take a vec4 input, GCN is much closer to VILW4 than you think.
Each CU also contains a scalar unit it cannot excute them in parallel.
If the SIMD units cannot be fed they sit idle, idle CUDA cores can be assigned to a different warp.
>Also nonsense. Both AMD and Nvidia have the majority of die space allocated to the compute cores
ACE takes up about 15% of the die space that is big in my book.
Unfortunately, I agree with the other poster. There are a number of inaccuracies in your post.
> Each CU contains an array of 4 SIMD units which each take a vec4 input
AMD SIMDs are (poorly named) scalar units, which are programmed using scalar arithmetic. FP16 is the only exception, as FP16 is a true SIMD operation (taking place inside a AMD scalar SIMD 2-at-a-time).
Yeah, AMD really needed to name their stuff better. "vGPRs" ("vector General Purpose Registers") are in fact scalar units, as documented in the Vega Instruction Set.
The SIMDs replaced historical SIMD-units from the 6000-series roughly 10 years ago. I bet that the name is historical in nature. But AMD's cards are scalarly programmed, just like NVidia's. There is no need to use vec4 on modern (aka: anything in the last 10 years) cards from AMD. Half2 is the only thing that gets a benefit.
> Each CU also contains a scalar unit it cannot excute them in parallel.
The "Scalar" unit in AMD GPUs is... closer to a boolean-vector unit. AMD really chose bad names for these things... But anyway, it is a 64-bit unit, where each bit is used to calculate the execution masks to the vector units.
The "Scalar" unit runs once every clock-tick. "Vector" units repeat themselves over 4-clock ticks (and therefore execute 4x slower than the Scalar unit). Ultimately, 1-Scalar unit can perform roughly the same amount of work as the 4-vector units.
See page 31 for a great example on how the sALU of Vega is used to create loops and handle divergent cases. The "sALU" is effectively the unit to calculate "if" statements and "loops".
> The "Scalar" unit in AMD GPUs is... closer to a boolean-vector unit. AMD really chose bad names for these things... But anyway, it is a 64-bit unit, where each bit is used to calculate the execution masks to the vector units.
This is a really cool way to put it, though I'd point out that the scalar unit is actually also used for (uniform) arithmetic, especially on pointers / indices. For example, if you have memory accesses of the form uniform_base + stride * thread_id, you can use the scalar ALU to compute / manipulate the uniform_base part.
If you think of the GCN compute units as CPU cores with a really wide SIMD + powerful scatter/gather + texture sampling unit, the chosen terminology of scalar/vector ALUs makes sense.
The difficulty is in mapping the (work item-centric) programming model that developers see to the hardware, but that's conceptually fairly similar to what ISPC does.
> idle CUDA cores can be assigned to a different warp.
I have a feeling you're confusing lanes vs. cores here. Individual "threads" (work items) cannot migrate between warps, and every "core" only issues one instruction at a time (per execution port, in some of Nvidia's architectures). Each execution port is a SIMD execution port executing that instruction for multiple work items. If the work items of a warp diverge, some of the lanes of those SIMD execution units will be idle.
This is the exact same behavior as on AMD's GCN, though obviously the terminology is different and some of the other details may be wired differently.
> ACE takes up about 15% of the die space that is big in my book.
I have a feeling that you're just making up numbers here. Can you point to evidence?
> NVidia doesn't have a scalar architecture in any meaningful sense of the word.
While I agree with your post in general, I'm not totally in agreement here. If you look at the context in which G80 and CUDA were introduced, it has a significant scalar dimension: for a single "thread" writing an addition between two float3 ended up being executed as 3 separate additions on the scalar components of the vector. This was, if I remember correctly, a departure from previous architecture optimized for graphic processing.
In this sense the ISA is/was mostly scalar.
This also applies to AMD GCN, though. So if you compare GCN to the earlier TeraScale architecture, you could argue (like a sibling comment does) that AMD's GCN is a scalar architecture and so is Nvidia's.
If you look at the actual hardware, though, they're clearly both vector architectures: each "core" has N SIMD-style lanes executing the same instruction simultaneously; it's just that each of those lanes corresponds to a different work item of the compute dispatch.
This applies equally to AMD's and Nvidia's architectures.
I mean, maybe you think their choice to deploy as an ERC20 was a bad choice; fine. But they're not adding any additional computation to the environment.
I don't understand how it's possible to think that LivePeer is bad for the environment without thinking the same thing about, for example, Elastic Transcoder. Your comment reads like a non-sequitur to me.
Presumably hughes is just making a knee-jerk comment about how 99% of PoW systems are doing useless work. Personally I am very skeptical of any system that tries to do proof of useful work like transcoding; it seems vulnerable to cheating.
Hi wmf - I work on Livepeer, happy to clarify on some of the skepticism here.
Livepeer uses cryptographic signatures to prove authenticity of the video transcoding results, and that signature can be used as proof to punish cheaters (if you signed the result, you are responsible for the correctness - and since it's easy to validate whether the transcoding is done correctly, anyone can use that signature to punish you by taking away your stake). If no one cheats, no one gets punished. Since the cheater doesn't know how many people are monitoring the results, their best strategy is to be honest.
Hey ericxtang. I've met your boy Phillip a few times (that's how I found out about LivePeer) - solid guy. Very good ambassador for your project and all that it can mean for the world.
it pushed computing/cooling/energy performance forward and it's not bad for the environment.
>Iceland, which is another hot spot for Bitcoin miners, makes use of 100 percent renewable geothermal and hydropower energy sources. The clean energy researcher explains that in this case, the energy demand for miners is almost irrelevant to the health of the environment.
I'm interested in an answer to this also. If you find one, let me know.
In my experience, the "Build the Free Web" events are usually selected on the basis of actual engineering and not hype; the presenters there are typically from projects that are pretty good.
I work at NuCypher, and I think our project is very much based in sanity and solid programming. I feel confident that people will actually use our token to build interesting distributed things.
When we attend blockchain-oriented conferences, there are a few projects whose teams we usually gravitate towards and who seem (at least to me) to have similar principles (less hype, more logic).
These include LivePeer, Fluence, and Origin (I'm not currently invested in any of those, but actually I think I want to change that).
In addition to actual crypto assets, I also think that the Python tooling is really coming of age and full of a lot of very sober and solid programming - most of the projects that Piper Merriam and Jason Carver touch are worth looking at.
It is not that easy. Depends on what you mean by that. There are a lot of aggregators which track different metrics. There are a lot of great teams, great tech, but no use cases are poor token economics, weak marketing, etc., as well as many scams. But Livepeer, Origin and NuCypher are solid ones :)
It'd be super-nice if AMD also produced, at a minimum, fully supported versions of PyTorch and TensorFlow for these. NVIDIA would crap its pants if perf is comparable (and possibly drop the prices some).
90 comments
[ 3.2 ms ] story [ 156 ms ] thread4x Stacks of HBM2 means 1TBps memory bandwidth at 16GB. Only 60-compute units enabled (maybe 4-CUs are expected to break during manufacturing? Its a weird number for sure...).
Since it shares dies with the MI50, the Radeon VII will have 1/2 speed double-precision, making this the cheapest high-performance double-precision card in existance.
------
FP16 compute is supported at double speed, but there are no tensor cores. So FP16 matrix multiplication / tensor ops are still a major benefit to NVidia.
But the memory size and bandwidth is quite salivating. That's a lot of bandwidth, and a number of problems are known to be memory-bound. Deep learning enthusiasts probably will stick to NVidia cards, but other compute problems may want to start playing around with this thing.
------
Video gamers seem meh about the specs. But I think anyone looking at this card for its compute performance would be impressed.
We've been building the wrong hardware for ML for a while now.
A TPU doesn't delivery supremacy for problems over GPU hardware. It's a Google senior engineering retention and PR project.
Exceedingly few problems resemble image recognition. You wouldn't be able to tell from the research, because due to the tools it's sort of the only affordable thing to do.
Playing Go barely looks like it. Or more accurately, Go doesn't look like most other games. Molecular physics and synthesis barely looks like it. Neural programs/neural Turing machines that matter (converting complex sim code like game rules and physics into neural networks via learning) may never achieve the accuracy or performance/watt necessary to really compete with just making a bunch of CPUs or straight up dedicated hardware.
Word embeddings are a disaster. People keep trying to do innovative stuff with it, and it's 2018 and we've just just recently got entity names from Wikipedia. I think Peter Norvig, having basically only machine translation to point to, is really going to eat crow when non-perceptual AI tasks sort of have nothing to do with stuff Google has developed. But what do I know.
I'm sure someone's going to trot out some obscure TPU or GPU-driven thing. It's been years! Awesome innovations in hardware really do lead to obvious, immediate crazy cool stuff. I'm really talking about the Kinect as an example here, in that it was really hyped and not much came of it. The TPU, and ML-specified hardware targeted to today's problems, is a slower-motion Kinect of our time.
Mining cryptocurrency, the most recent innovation, is the opposite of cool.
I'm confident these tools exist to capitalize on the subsidy from the gaming industry. AMD is wise to not chase ML features.
Its overall design sits somewhere between a GPU and CPU. Like a GPU, it supports 8-way hyperthreading (GPUs typically support 10. CPUs usually only support 2... maybe 4 for Larabee). Like a GPU, its programmed through an OpenCL-like language (I know you don't like that, but its how it works).
However, the overall layout is very CPU-like as well: a "City" is composed of "Villages", which is composed of multiple "Processing Elements". Each PE is scalar like a normal CPU, although 8-way SMT. So its not like a GPU's SIMD or SIMT architecture in this regards.
--------
See information here:
https://en.wikichip.org/wiki/pezy/pezy-scx/pezy-sc
It really is an interesting design that toes the line between CPU and GPU.
/s
Real talk, though. Why do you think larabee is the "right" hardware?
x86 isn't the right choice though. As the chips get smaller, all the x86 cruft starts to become a larger portion of each core.
RISCV with 40-ish instructions plus a SIMD extension would probably be ideal. They've already proven that those designs use less space per core than ARM.
I've also noticed that a lot of the time, people are comparing GPU to naive implementations (and sometimes, implementations written in dynamic languages) instead of to highly tuned BLAS or MKL implementations. For a large swathe of problem types, using the fastest math library will reduce the CPU/GPU gap to less than an order of magnitude or less.
In your experience, are these comparisons using naive GPU implementations as well? If so, that still seems like a valuable comparison.
https://software.intel.com/en-us/mkl/features/benchmarks
In no case will you see it get close to 10TFLOPS. GPUs easily do this, and can approach 100TFLOPS with tensor cores.
The underlying technology was acquired by Apple and is used on iPhones for FaceID, https://www.theverge.com/circuitbreaker/2017/9/17/16315510/i...
> PrimeSense was never solely responsible for the technology in Microsoft’s Kinect — as evidenced by the huge improvements Microsoft made to Kinect 2.0 on its own — and it’s also obvious that Apple is doing plenty of new software and processing work on top of this hardware. But the basic idea of the Kinect is unchanged. And now it’s in a tiny notch on the front of a $999 iPhone.
I'm not sure what you mean by "ML features", but AMD/ATI's dismissive attitude towards the ML and GPU-compute communities is looking quite silly by now.
I disagree, ML computing has tons of competition and its margins will be quickly diminish. I think they are right to focus on gaming.
It makes it extremely difficult to take any of what you just said seriously knowing that you made this statement.
People with academic backgrounds would be less afraid to point out how much vaporware machine learning has produced. They'd take stock in that their tenure, or six year PhDs, give them an opportunity to not chase trends so aggressively. They'd shill online less, for some giant faceless corporation like Google.
I'm not going to shit on a bunch of great stuff people have made. But the scary thing is that Noam Chomsky is probably right, that AI science hasn't advanced much at all. That we're still in an AI winter where it matters, the science, despite all the money and attention.
Do you mind clarifying what exactly your issues are with these specific applications of machine learning? Is it that you think there isn't really any meaningful value that can be gained in working within these areas? Or is it just that you want people to spend more time researching a different topic that you think would be more interesting/compelling?
This is coming from a physicist starting to investigate ML tools in my own work.
Since AMD will sell the Radeon VII directly at first, I think early adopters will probably face limited quantities rather than price markups (not counting unauthorized resellers). After that, we'll have to see how the situation with retailers and later on non-reference models shakes out.
If they don't gimp FP64 it won't leave a whole lot of reason to buy the MI50 for probably 10x the price.
MI50 has ECC RAM support. That's absolutely necessary for high-end scientific compute.
It's not gimped according to Ryan Smith from AnandTech: https://twitter.com/RyanSmithAT/status/1083959608371175424
Which means it destroys it's only real competitor for consumer FP64 - the Titan V - which costs 3000 Dollar.
https://news.ycombinator.com/item?id=18868617
I assume this means that it won't support raytracing at all? Or will it, but with some non-DXR implementation?
The Radeon VII will certainly be slower at raytracing (if AMD ever decides to support the feature). It has FP16 support and huge RAM bandwidth (which will help traverse a BVH Tree), but I would expect it to be many times slower than dedicated hardware units. Without tensor cores (VII only supports hardware-accelerated FP16 Dot-products), there's no way it'd keep up with Nvidia on denoising or BVH traversals.
I think you could get something 95% as good with far better performance by only tracing vertices and then raster rendering the resultant transform.
Would be a good approach for accurate shadows too where you don't really care about texture fragments.
1-sample per pixel is already incredibly low for raytracing. A raw 1 spp raytraced image looks HORRIBLE.
The reason why it looks anywhere close to usable is because NVidia post-processes all of that noise through their tensor-cores FP16. It is some kind of spacial / temporal-filter effect (averaging pixels across time, and space... where "space" is mapped to geometry framebuffers to keep object boundaries correct).
If your physically accurate rendering is pushing the hardware limits so far that you need extensive post processing to maintain the framerate then it stops being so physically accurate.
At that point, it seems like there are more interesting things you can do with the hardware than a naive raytrace that would look and perform better while, admittedly, not being 100% physically accurate.
A more likely use in the immediate future is to simplify some parts of the rendering pipeline, and improve quality on a few others. Without looking too much into to it, improving and simplifying soft shadows and are lighting seems like a no-brainer. Both are really hard to implement efficiently using rasterization, and/but both needs a good denoiser.
Will be interesting to see thought. I believe it's a bit later than I predicted that we get to see real time ray tracing on commodity hardware, but what is a couple of years over a few decades?
Of course that'd work for fps as slow as battle of duty \s
https://www.anandtech.com/show/12547/expanding-directx-12-mi...
Yet due to sheer clock speed increases and HBM improvements (16GB, 1 TB/s, wow) they actually seem surpisingly competitive. I'm both incredibly impressed and incredibly underwhelmed.
16gb seems like overkill for the next couple years of game releases.
It’s essentially just to keep up some production volume this is their HPC die sold to the consumer market its the Titan V equivalent die of AMD minus the Tensor Cores just for much much cheaper.
It’s likely going to be a flop gaming wise, 2x8 pin power connectors put it at 300W TDP with performance only matching that of a 1089ti/2080 in the best case scenarios AMD currently has.
The 1080ti can be had for under $600 if you can find it the 2080 is around $700 but you get RTX and DLSS which while isn’t that useful yet is going to be more useful for gaming than 16GB of memory.
60 CUs at 1.8GHz is way faster than AMD Fury ever was. Not only do you get the GHz advantage, but more L2 cache, more HBM2 Bandwidth, and architectural advancements (various instruction set improvements like double-speed FP16 support)
EDIT: AMD Fury's main advantage was that 1GHz clock made all of the nanosecond-level timings super easy to understand. 10-clock cycles was 10-nano seconds. Hurrah!
GCN’s vector machine doesn’t scale as well as NVIDIA’s scalar architecture NVIDIA can add cores without caring about concurrency or ILP, since each core is an individually addressable scalar ALU, this is why “async” compute doesn’t benefit them as much as it does GCN cards that have a 4 wide SIMD array that is nearly as hard to feed as their old VILW arch and a massive external compute scheduler that sits idle when doing graphics.
> NVIDIA’s scalar architecture
NVidia doesn't have a scalar architecture in any meaningful sense of the word. Individual work items execute in warps, meaning (up to) 32 items execute the same instruction, at the same PC. If you have divergence, i.e. fewer than 32 items at the same instruction, you lose ALU throughput. This is the same as in AMD (and every other GPU architecture out there, for that matter).
The difference to AMD's architecture since Volta is that there is hardware support for each SIMD lane having its own PC, which makes it easier to have independent forward progress and a bunch of other features. The separate lanes don't have separate instruction fetch, though -- they all execute the same instruction (and if they can't, then some lanes will be disabled).
> this is why “async” compute doesn’t benefit them as much as it does GCN cards
Nvidia literally cannot do async compute in the same way as AMD, because their micro-architecture cannot switch between graphics and compute on as finely grained a basis as GCN cards. That's why they don't benefit from async compute. (Though I'm not sure whether that's still true since Volta.)
> GCN cards that have a 4 wide SIMD array
GCN has 64-wide waves, though with a bit of a weird execution scheduling which means that 16 lanes out of the 64 are computing at a time. Nowhere in GCN are there any 4-wide SIMD arrays.
> a massive external compute scheduler that sits idle when doing graphics.
Also nonsense. Both AMD and Nvidia have the majority of die space allocated to the compute cores (whatever they're calling them). The pieces of fixed function logic that distribute work onto the cores (whether for graphics or compute) are comparatively tiny.
CUDA threads are scalar, each CUDA core is a single scalar ALU, which CUDA cores are assigned to a warp is flexible.
>Nvidia literally cannot do async compute in the same way as AMD, because their micro-architecture cannot switch between graphics and compute on as finely grained a basis as GCN cards.
Both AMD and NVIDIA GPUs have to do context switching. The difference is in scheduling GCN has a dedicate scheduler with 8 queues only for compute that is the ACE it sits utterly idle while doing graphics the graphics scheduler sits within each CU.
>Nowhere in GCN are there any 4-wide SIMD arrays.
Each CU contains an array of 4 SIMD units which each take a vec4 input, GCN is much closer to VILW4 than you think. Each CU also contains a scalar unit it cannot excute them in parallel.
If the SIMD units cannot be fed they sit idle, idle CUDA cores can be assigned to a different warp.
>Also nonsense. Both AMD and Nvidia have the majority of die space allocated to the compute cores
ACE takes up about 15% of the die space that is big in my book.
> Each CU contains an array of 4 SIMD units which each take a vec4 input
AMD SIMDs are (poorly named) scalar units, which are programmed using scalar arithmetic. FP16 is the only exception, as FP16 is a true SIMD operation (taking place inside a AMD scalar SIMD 2-at-a-time).
Yeah, AMD really needed to name their stuff better. "vGPRs" ("vector General Purpose Registers") are in fact scalar units, as documented in the Vega Instruction Set.
See chapter 6 for proof: https://developer.amd.com/wp-content/resources/Vega_Shader_I...
The SIMDs replaced historical SIMD-units from the 6000-series roughly 10 years ago. I bet that the name is historical in nature. But AMD's cards are scalarly programmed, just like NVidia's. There is no need to use vec4 on modern (aka: anything in the last 10 years) cards from AMD. Half2 is the only thing that gets a benefit.
> Each CU also contains a scalar unit it cannot excute them in parallel.
The "Scalar" unit in AMD GPUs is... closer to a boolean-vector unit. AMD really chose bad names for these things... But anyway, it is a 64-bit unit, where each bit is used to calculate the execution masks to the vector units.
The "Scalar" unit runs once every clock-tick. "Vector" units repeat themselves over 4-clock ticks (and therefore execute 4x slower than the Scalar unit). Ultimately, 1-Scalar unit can perform roughly the same amount of work as the 4-vector units.
See page 31 for a great example on how the sALU of Vega is used to create loops and handle divergent cases. The "sALU" is effectively the unit to calculate "if" statements and "loops".
This is a really cool way to put it, though I'd point out that the scalar unit is actually also used for (uniform) arithmetic, especially on pointers / indices. For example, if you have memory accesses of the form uniform_base + stride * thread_id, you can use the scalar ALU to compute / manipulate the uniform_base part.
If you think of the GCN compute units as CPU cores with a really wide SIMD + powerful scatter/gather + texture sampling unit, the chosen terminology of scalar/vector ALUs makes sense.
The difficulty is in mapping the (work item-centric) programming model that developers see to the hardware, but that's conceptually fairly similar to what ISPC does.
I have a feeling you're confusing lanes vs. cores here. Individual "threads" (work items) cannot migrate between warps, and every "core" only issues one instruction at a time (per execution port, in some of Nvidia's architectures). Each execution port is a SIMD execution port executing that instruction for multiple work items. If the work items of a warp diverge, some of the lanes of those SIMD execution units will be idle.
This is the exact same behavior as on AMD's GCN, though obviously the terminology is different and some of the other details may be wired differently.
> ACE takes up about 15% of the die space that is big in my book.
I have a feeling that you're just making up numbers here. Can you point to evidence?
While I agree with your post in general, I'm not totally in agreement here. If you look at the context in which G80 and CUDA were introduced, it has a significant scalar dimension: for a single "thread" writing an addition between two float3 ended up being executed as 3 separate additions on the scalar components of the vector. This was, if I remember correctly, a departure from previous architecture optimized for graphic processing. In this sense the ISA is/was mostly scalar.
If you look at the actual hardware, though, they're clearly both vector architectures: each "core" has N SIMD-style lanes executing the same instruction simultaneously; it's just that each of those lanes corresponds to a different work item of the compute dispatch.
This applies equally to AMD's and Nvidia's architectures.
I mean, maybe you think their choice to deploy as an ERC20 was a bad choice; fine. But they're not adding any additional computation to the environment.
I don't understand how it's possible to think that LivePeer is bad for the environment without thinking the same thing about, for example, Elastic Transcoder. Your comment reads like a non-sequitur to me.
I'm not familiar with the nitty-gritty, but video transcoding seems to me to be something that can be verified much, much more cheaply than performed.
FWIW, I think they're already launched, so do feel free to break it.
Livepeer uses cryptographic signatures to prove authenticity of the video transcoding results, and that signature can be used as proof to punish cheaters (if you signed the result, you are responsible for the correctness - and since it's easy to validate whether the transcoding is done correctly, anyone can use that signature to punish you by taking away your stake). If no one cheats, no one gets punished. Since the cheater doesn't know how many people are monitoring the results, their best strategy is to be honest.
>Iceland, which is another hot spot for Bitcoin miners, makes use of 100 percent renewable geothermal and hydropower energy sources. The clean energy researcher explains that in this case, the energy demand for miners is almost irrelevant to the health of the environment.
Interesting.
Can anyone suggest a good curated list of solid blockchain projects?
In my experience, the "Build the Free Web" events are usually selected on the basis of actual engineering and not hype; the presenters there are typically from projects that are pretty good.
I work at NuCypher, and I think our project is very much based in sanity and solid programming. I feel confident that people will actually use our token to build interesting distributed things.
When we attend blockchain-oriented conferences, there are a few projects whose teams we usually gravitate towards and who seem (at least to me) to have similar principles (less hype, more logic).
These include LivePeer, Fluence, and Origin (I'm not currently invested in any of those, but actually I think I want to change that).
In addition to actual crypto assets, I also think that the Python tooling is really coming of age and full of a lot of very sober and solid programming - most of the projects that Piper Merriam and Jason Carver touch are worth looking at.
* https://rocm-documentation.readthedocs.io/en/latest/Deep_lea... * https://github.com/ROCmSoftwarePlatform/tensorflow-upstream
† and ARM one day: I'm talking desktop/laptop/server here…