>Machine learning, data scientist and real time professional simulations.
With photogrammetry (my field) we need to process anywhere between 1GB and 500GB of DSLR photos in one project and triangulate points between all these photos. Lots of RAM helps. Lots of VRAM helps JUST as much.
Also crypto mining takes advantage of large VRAM too.
Google recently released their Performer architecture paper where one of the major achievements was making it work within 16GB. More headroom is incredibly useful!
Need? Who knows. Do customers want it for various reasons? Absolutely.
4k and 8k gaming are definitely going to be a thing. Those textures are pretty big. (I think Final Fantasy XV is like 80GB larger if you get the 4k textures?)
The article says that it is unlikely for consumer cards to get 20gb. The same chips go to pro-level cards, and they will have it.
As far as I know from the latest gossip and tech details, 4k textures pretty much fit 8gb, and 4k gaming still struggles with 60fps, which 3840 cores will not change by a large margin. From that, 8k is simply a marketing move (apart from eye physics). Also, 4k/144 requires an active cooling of a monitor, and these still cost too much to become mainstream in 30xx lifetime. 8k is basically 4k x4, do the math.
YES! For rendering medium-large scenes in GPU based rendering engines such as Octane, Redshift, Arnold, etc.
Take a sportscar sample model with good resolution, say 250k polygons. If you want to create an animation with 100 cars crashing into each other, you'll need a lot of GPU RAM.
I imagine at least a part of that is because gamers want to see new hardware have bigger numbers than the old hardware and RAM is a pretty easy metric. RTX 2080s have 8-11 GB.
Well that is HBM, not GDDR. And price / value differs to each individuals. Example the GeForce RTX 2080 Ti is only $1099. A Chip that is nearly 800mm2 die size. Pretty damn good if you ask me.
They could have bump the Memory to 32GB and charge me $1500 which would give them even higher margin.
I hope that is something AMD would do, pushing more Cores in CPU, pushing more Graphics Memory in GPU, let the developer figure out how to best use it.
The slight problem is space on die. Higher density GDDR6 just don't exist, and attempting to connect more chips quickly runs into die size limitations and routing problems.
For Machine Learning (esp. computer vision) it is really important.
First, as others have pointed out, it lets you use bigger batches of data in a single forward pass. This has not only performance aspects, but also certain regularization aspects.
Second, if you want/need to use higher resolution inputs then you really need a lot of memory. For example, MaskRCNN - a popular instance segmentation network - can only do batch_size=1 at 960x960 pixel images (with some tweaks). 960x960 is not a particularity high resolution by contemporary standards and also a batch_size=1 can cause gradient fluctuations (this is usually not desired).
There is also some trend in NLP (Natural Language) to work with lots of data/bigger models that are memory hungry (e.g. GPT-2).
In the future, yes. As screen resolution increases a lot of textures increase as well. These mostly does not scale linearly as well so a large increase is to be expected from higher resolutions.
The cuckatoo_mean_cuda_rtx_32 solver for Grin's Cuckatoo32 proof-of-work needs that much memory. This PoW is actually intended for ASICs, that would need only 512MB of SRAM with a different approach, but those ASICs have yet to appear.
The big news to me is that the top card features a 320 bit memory bus, which should offer much better memory bandwidth than the previous generation (256 bits). Assuming a linear scaling, we should see a 1.25 improvement compared to the previous generation.
>The big news to me is that the top card features a 320 bit memory bus, which should offer much better memory bandwidth than the previous generation (256 bits).
RTX 2080 is 352 bits, and RTX Titan is 384. Nvidia has always had high bit rate memory bus. The one that had 256 bit was AMD Radeon GDDR. Arguably bit bandwidth is meaningless without looking at actual memory type, or the total bandwidth from memory.
Interestingly, Nvidia decided to use a bigger die (relative to other dies in the family) for the 3080 than for the 2080. TU104 (2080) vs GA103 (3080).
Probably to prepare for big Navi.
I think 500$ for 2080ti won't happen that quickly. There are many hobbyists now who would happily grab several of them for a deep learning project of choice. If they become that cheap I would buy 6 (or however many I could reasonably put in one mining rig like box) just to run Leela Chess Zero training and analysis. I know a few more people in the same boat.
The biggest hope for cheap GPUs is AMD actually delivering something. While I don't doubt they will get there in hardware department I am pessimistic about software.
You don't need 144fps for a 144hz monitor. You didn't say that that is your setup, so I am just assuming here, but the 144 sounds too specific for it to be coincidence.
The thing about refresh rates for screens, as I understand it, is not that you need to have a 1:1 relation between your fps and hz, but you don't want to 'lose' frames due to your hz being too low. The lower both are, the more likely you are to miss frames as the image might enter the monitor exactly at a moment where it can't render it. That is why running a game @ 40fps will still be smoother on a 144hz monitor than it is on a 60hz monitor. Higher hz, just means you have a better chance at seeing all the rendered frames.
Sure 144fps sounds like a great situation for a 144hz monitor, but I think in reality it just means that you are missing a higher percentage of your frames. So you'd either need some sort of syncing technology (which AMD and NVIDIA have) or an even higher hz monitor. Again, I think, that if you don't, you will just be rendering many of your frames for nothing, as your fps and hz are not aligned. So you might as well get a GPU that renders your games at 80fps for your 144hz monitor. The difference between 144fps and 80fps in terms of money is probably a lot higher than lets say going from 40 to 80.
This is all based on my limited knowledge that I acquired from my research I did when I bought my own 144hz monitor.
For the competitive games where higher framerate is most advantageous (i.e. shooters), I come to the opposite conclusion. Games like that tend to have long periods of low activity where you aren't being challenged, with short periods of chaotic fighting where framerates are significantly worse. It's only during those chaotic times where the performance (of both the hardware and the player) actually has an effect on the outcome of the match.
I think it's far more important to have a consistent framerate than it is to worry about wasting potential (fast) frames. Inconsistent smoothness and input lag will create a disadvantage compared to just playing at a capped (lower) framerate.
So I think a balanced system (both the hardware and the graphical settings) with a 144hz monitor should be able to run games at far beyond 144 fps most of the time, so that the framerate will only drop down to around 144 in the worst moments. I would even suggest setting a framerate limit near the worst framerate your system often drops down to in the game.
How much your hardware needs to be "overprovisioned" depends on how poorly optimized the game is. For many games this can as much as a factor of two difference. So you might only be putting 50% load on your GPU most of the time, but peak at 100% load at important times.
That depends mostly on the game. Look at it from the game developers perspective. If I launch a game at the same time as these cards, how much performance do I have available at the highest graphics settings?
I make the ultra setting play smoothly on the beefiest graphics card. But I don't release the game where the ultra setting is 144Hz on the beefiest graphics card, because so few users have that anyway.
Perhaps a good target to aim for is not dropping below 60Hz in Ultra on the top hardware. Some probably aim even lower, e.g. a higher percentile of frames under 60Hz.
"Consistent 144fps" is extreme, and isn't really useful unless you are a competitive gamer. If you use a 144Hz screen without adaptive sync (I thought that was pretty uncommon?) then frame drops might lead to stuttering or tearing, but with adaptive sync you'll likely never notice if frame rates drop temporarily because there is non tearing or stuttering. Hopefully card manufacturers assume that anyone who is even remotely interested in 100+Hz also has adaptive sync. Because if you have $800+ to spend on a graphics card you should could get an adaptive sync screen and a cheper card really.
I’m not informed on the PCI specs but I bet the bottleneck is simply bandwidth (or CPU), the GPU is capable enough.
I have a 2080ti as an Thunderbolt 3 eGPU, which has an even smaller bandwidth than PCI Express so the effect is accentuated. I lose about 10% perf on 4K compared to a desktop computer, and 40% at 1080p. I don’t recall the exact numbers but a benchmark would do 80fps on 4K and 120fps on 1080p, even though it’s 1/4th the amount of pixels. This is because most of the time is spent transferring data on the bus.
At this point all I care about is the price. The price of GPUs has ballooned in an insane way recently and it’s impossible to get a decent deal that’s significantly faster than what I have now (and I haven’t upgraded in years). The used market seems to have been decimated, with even last gen cards going for crazy high prices. I suspect the 30x0 line will be even more expensive than the current one, and I hate it.
Yeah, I really don't want to pay more than around $120 for a decent GPU, which gives me a used gtx 1060 or 780ti. Both of those play new games at high settings on 1440p, what more could you want? Shelling out $700 for a GPU to get good VR or RTX tech demos just isn't worth it.
It is if you use a high refresh rate. My 1080ti is barely adequate for 1440p165... and even at that I’m usually turning setting down to average 100–120fps.
You do not play new AAA games on 1440p high with a $120 card.
High end RTX are worth it on 4K screens. I can confirm that a £1200 RTX 2080 Ti can run new AAA games on 4K high at a stable 60 to 70 fps. Not much higher than that.
The later 1xxx models are expensive in the aftermarket because a handful of cryptocurrencies are still ROI positive mining on them.
To a lesser extent the 2xxx series is going to be dominated by people wanting to do password cracking. A 2080 Ti will do about 13 billion checks/sec (using NTLM benchmarks), which starts to make financial sense for cracking high complexity passwords from data breaches.
It would be nice if in the 3xxx series they could figure out a way to make it only fast for games and not ML/cracking/crypto.
Unfortunately, it seems they're more than happy with the current state of things and they don't particularly care who buys their cards. They can push up the price and the used market will move in lockstep because demand is so high.
FWIW, I think they do care, because it would be bad for them if gaming on the PC dies, since it drives a lot of innovations in graphics, and nvidia is still at heart a graphics company.
However, there is little you can do when the stock of gaming oriented graphic cards disappears on day 0, and re-appears on ebay on day 1 at 4x their recommended price.
This products are just very valuable to a lot of people nowadays. Nvidia could ramp up production, but production is not something that you can quickly ramp up or down.
There is the fear that crypto will die, and then you end up with a lot of stock that you have to eat at a loss.
>FWIW, I think they do care, because it would be bad for them if gaming on the PC dies, since it drives a lot of innovations in graphics, and nvidia is still at heart a graphics company.
More than volatile, at some point, using GPUs is just not worth it for cryptomining.
Honestly, I think that once they discovered the issue, and decided what to do about it, then the ROI on mining bitcoin was already fading and people were migrating to FPGAs, so maybe they thought "things will get back to normal on its own".
Turns out, the number of cryptocurrencies exploded, and many of them are worth mining. Also, every time the ROI on mining one disappears, a new cryptocurrency appears.
Maybe they are just still waiting for either this problem to fix itself, by slowly ramping up production, but nothing too steep.
Well Nivida just released a dev board for $99 that has the GPU chip from a Titan X in it. That is nearing Raspberry Pi levels of "democratized" computing.
I don't know much about ML, but my knowledgeable friends say they are saving tons of money moving from GPU farms to GCP TPUs for training.
The only thing it has in common with the (2015) GTX Titan X is the Maxwell architecture for the GPU but with only 128 cores and lower clocks. That would be some magic if Nvidia managed to get the GM200 GPU's power consumption down one order of magnitude from 250W.
The Jetson delivers ~2 TFLOPs of single precision performance vs. Titan X's ~6 TFLOPS. Great from such a small board but not quite "same GPU".
From what I can tell researching it, the 99$ Jetson Nano has around 1/2 teraflop of computing power for 7.5 watts of power. The Jetson TX2 has from 1.5-2 teraflops and costs around 500-600 but there was/is a student Dev kit for 300$, whereas the Titan X has 12 teraflops of computing power.
Indeed I posted the numbers for the bigger brother. The Nano is offically slated at 472GFLOPS. The 2015 GTX Titan X (GM200, ~6 TFLOPS, +/- 10% depending on source) is the only one that has something in common with the Jetson - the Maxwell architecture.
The one with 10-12TFLOPS is the 2016 Titan X (GP102-400) but that was already using Turing so the comparison would be even more stretched.
Pretty sure he means gaming, even my 2080 ti struggles with 4k gaming (well imagine wanting the highest setting of every game in 4k, most gpus can handle that in fullhd n 2k, but 4k every gpu is jsut to weak)
Can you expand on what you mean by 'crypto is pretty much over'? I thought miners had simply moved to better cards to stay competitive resulting in said price drop of older cards?
Miners didn't move to better cards, they moved to ASICs. Once the currency is too hard for a consumer GPU to mine, miners move to specialized hardware which is far better at mining. When mining gets even harder, the only way it can be done profitably is to set up entire farms of chips that mine away, and that's a serious amount of cash at this point, far beyond a 1200 dollar card. By this point, mining is pretty much over for the rest of us mortals.
Which just goes to prove that the emperor is wearing no clothes: The vision of democratized crypto mining has not been realized. Instead of millions of people collectively holding the network in check, the mining power is being centralized by big key players. It's only a matter of time until the mining power has centralized enough to where the network can be sabotaged and taken over by whichever party is mining more than 50% of the work.
There's an expectation that the price / performance ratio also improves as time goes on. Put another way, you should be spending the same (plus inflation) every few years, and getting significantly more bang for your buck. I haven't done the maths, but I guess a lot of people are not feeling this.
They are and they aren't. We're not seeing definitive levels of higher resolution performance, we see concrete performance boosts but in the real world and in terms of real use it's pretty marginal considering the costs involved. Hardware Raytracing is clearly an attempt at a value add bit the results have been somewhat underwhelming. For people content at 1080p60fps we're not that far behind if you ignore RTX/AMD Equivalent.
I mean I know there are some real gains, some amazing gains on paper or specific application but when playing a game the realizations are a bit less pronounced for everyone that is not hounding on RTX or pixel peepling.
Just my $0.02 of course and I can respect any differing view but this is just my thoughts from watching and waiting with a 1070 I use for 1080p60fps. It certainly taking more fiddling with newer titles to get that level of IQ and performance but I'm still happy in my niche. I would love to move up a level or two in resolution or FPS but it's hard to justify at the price levels we are seeing.
I worked in a life assurance company in 1996. The networked shared hard drive that the entire 15-ish person IT department worked from was 4GB. We considered that to be a decent size, and rarely ran into space issues. I keep wondering where this ends up - are we going to have 4TB graphics cards and 1PB hard drives in 2040?
1. You're only going to see so much details, so making bigger textures will not make much sense soon. We're already at the level where the art design / mapping / movement / realistic lighting could improve the situation much more than extra resolution. Games can already run at up to 4k and virtually nobody even has hardware for it.
2. There's only so much bandwidth on the storage - graphics memory and graphics memory - GPU lines. If you can't realistically process the textures/geometry you want to display, there's no point making it.
1PB drives will come though for long term storage. We already have petabytes of data being processed in some companies. It's just a question of $/B ratio, whether the R/W speed is reasonable for that size, and how reliable do you want that storage to be.
76 comments
[ 67.3 ms ] story [ 278 ms ] threadYou need to have your models in video ram to be able to "play" with them in real time so with 20Gigs they can access larger models.
With photogrammetry (my field) we need to process anywhere between 1GB and 500GB of DSLR photos in one project and triangulate points between all these photos. Lots of RAM helps. Lots of VRAM helps JUST as much.
Also crypto mining takes advantage of large VRAM too.
Another use case is the Cloud gaming market where a single GPU needs to serve multiple users.
4k and 8k gaming are definitely going to be a thing. Those textures are pretty big. (I think Final Fantasy XV is like 80GB larger if you get the 4k textures?)
As far as I know from the latest gossip and tech details, 4k textures pretty much fit 8gb, and 4k gaming still struggles with 60fps, which 3840 cores will not change by a large margin. From that, 8k is simply a marketing move (apart from eye physics). Also, 4k/144 requires an active cooling of a monitor, and these still cost too much to become mainstream in 30xx lifetime. 8k is basically 4k x4, do the math.
Take a sportscar sample model with good resolution, say 250k polygons. If you want to create an animation with 100 cars crashing into each other, you'll need a lot of GPU RAM.
I could start playing around with GPGPU Database.
They could have bump the Memory to 32GB and charge me $1500 which would give them even higher margin.
I hope that is something AMD would do, pushing more Cores in CPU, pushing more Graphics Memory in GPU, let the developer figure out how to best use it.
RTX 2080 is 352 bits, and RTX Titan is 384. Nvidia has always had high bit rate memory bus. The one that had 256 bit was AMD Radeon GDDR. Arguably bit bandwidth is meaningless without looking at actual memory type, or the total bandwidth from memory.
The 2080 ti has 352bit. https://www.techpowerup.com/gpu-specs/geforce-rtx-2080-ti.c3...
Interestingly, Nvidia decided to use a bigger die (relative to other dies in the family) for the 3080 than for the 2080. TU104 (2080) vs GA103 (3080). Probably to prepare for big Navi.
If yes, I'm getting one.
I doubt that the 3xxx series will be strong enough to support consistent 4k144fps.
I just hope they are so that I can get a 2080ti for around 500 USD.
The biggest hope for cheap GPUs is AMD actually delivering something. While I don't doubt they will get there in hardware department I am pessimistic about software.
The thing about refresh rates for screens, as I understand it, is not that you need to have a 1:1 relation between your fps and hz, but you don't want to 'lose' frames due to your hz being too low. The lower both are, the more likely you are to miss frames as the image might enter the monitor exactly at a moment where it can't render it. That is why running a game @ 40fps will still be smoother on a 144hz monitor than it is on a 60hz monitor. Higher hz, just means you have a better chance at seeing all the rendered frames.
Sure 144fps sounds like a great situation for a 144hz monitor, but I think in reality it just means that you are missing a higher percentage of your frames. So you'd either need some sort of syncing technology (which AMD and NVIDIA have) or an even higher hz monitor. Again, I think, that if you don't, you will just be rendering many of your frames for nothing, as your fps and hz are not aligned. So you might as well get a GPU that renders your games at 80fps for your 144hz monitor. The difference between 144fps and 80fps in terms of money is probably a lot higher than lets say going from 40 to 80.
This is all based on my limited knowledge that I acquired from my research I did when I bought my own 144hz monitor.
I think it's far more important to have a consistent framerate than it is to worry about wasting potential (fast) frames. Inconsistent smoothness and input lag will create a disadvantage compared to just playing at a capped (lower) framerate.
So I think a balanced system (both the hardware and the graphical settings) with a 144hz monitor should be able to run games at far beyond 144 fps most of the time, so that the framerate will only drop down to around 144 in the worst moments. I would even suggest setting a framerate limit near the worst framerate your system often drops down to in the game.
How much your hardware needs to be "overprovisioned" depends on how poorly optimized the game is. For many games this can as much as a factor of two difference. So you might only be putting 50% load on your GPU most of the time, but peak at 100% load at important times.
I make the ultra setting play smoothly on the beefiest graphics card. But I don't release the game where the ultra setting is 144Hz on the beefiest graphics card, because so few users have that anyway.
Perhaps a good target to aim for is not dropping below 60Hz in Ultra on the top hardware. Some probably aim even lower, e.g. a higher percentile of frames under 60Hz.
"Consistent 144fps" is extreme, and isn't really useful unless you are a competitive gamer. If you use a 144Hz screen without adaptive sync (I thought that was pretty uncommon?) then frame drops might lead to stuttering or tearing, but with adaptive sync you'll likely never notice if frame rates drop temporarily because there is non tearing or stuttering. Hopefully card manufacturers assume that anyone who is even remotely interested in 100+Hz also has adaptive sync. Because if you have $800+ to spend on a graphics card you should could get an adaptive sync screen and a cheper card really.
I cannot believe that 2080TI cannot run at fullHD@144fps. I have 2080TI and I get constant AAA 4k@60fps.
I have a 2080ti as an Thunderbolt 3 eGPU, which has an even smaller bandwidth than PCI Express so the effect is accentuated. I lose about 10% perf on 4K compared to a desktop computer, and 40% at 1080p. I don’t recall the exact numbers but a benchmark would do 80fps on 4K and 120fps on 1080p, even though it’s 1/4th the amount of pixels. This is because most of the time is spent transferring data on the bus.
High end RTX are worth it on 4K screens. I can confirm that a £1200 RTX 2080 Ti can run new AAA games on 4K high at a stable 60 to 70 fps. Not much higher than that.
To a lesser extent the 2xxx series is going to be dominated by people wanting to do password cracking. A 2080 Ti will do about 13 billion checks/sec (using NTLM benchmarks), which starts to make financial sense for cracking high complexity passwords from data breaches.
It would be nice if in the 3xxx series they could figure out a way to make it only fast for games and not ML/cracking/crypto.
However, there is little you can do when the stock of gaming oriented graphic cards disappears on day 0, and re-appears on ebay on day 1 at 4x their recommended price.
This products are just very valuable to a lot of people nowadays. Nvidia could ramp up production, but production is not something that you can quickly ramp up or down.
There is the fear that crypto will die, and then you end up with a lot of stock that you have to eat at a loss.
And because cryptomining is volatile as hell.
Honestly, I think that once they discovered the issue, and decided what to do about it, then the ROI on mining bitcoin was already fading and people were migrating to FPGAs, so maybe they thought "things will get back to normal on its own".
Turns out, the number of cryptocurrencies exploded, and many of them are worth mining. Also, every time the ROI on mining one disappears, a new cryptocurrency appears.
Maybe they are just still waiting for either this problem to fix itself, by slowly ramping up production, but nothing too steep.
I don't know much about ML, but my knowledgeable friends say they are saving tons of money moving from GPU farms to GCP TPUs for training.
The Jetson delivers ~2 TFLOPs of single precision performance vs. Titan X's ~6 TFLOPS. Great from such a small board but not quite "same GPU".
The one with 10-12TFLOPS is the 2016 Titan X (GP102-400) but that was already using Turing so the comparison would be even more stretched.
Most people doing ML either use Nvidia for Cuda or GCN based gpus for their compute.
Crypto is pretty much over by this point Look at all the cheap 4/580s on ebay.
I mean I know there are some real gains, some amazing gains on paper or specific application but when playing a game the realizations are a bit less pronounced for everyone that is not hounding on RTX or pixel peepling.
Just my $0.02 of course and I can respect any differing view but this is just my thoughts from watching and waiting with a 1070 I use for 1080p60fps. It certainly taking more fiddling with newer titles to get that level of IQ and performance but I'm still happy in my niche. I would love to move up a level or two in resolution or FPS but it's hard to justify at the price levels we are seeing.
Doesn't seem too crazy high. How much are you looking to pay and what do you have now?
1. You're only going to see so much details, so making bigger textures will not make much sense soon. We're already at the level where the art design / mapping / movement / realistic lighting could improve the situation much more than extra resolution. Games can already run at up to 4k and virtually nobody even has hardware for it.
2. There's only so much bandwidth on the storage - graphics memory and graphics memory - GPU lines. If you can't realistically process the textures/geometry you want to display, there's no point making it.
1PB drives will come though for long term storage. We already have petabytes of data being processed in some companies. It's just a question of $/B ratio, whether the R/W speed is reasonable for that size, and how reliable do you want that storage to be.
However for other applications it's not the same thing. I work in medical 3D imaging - we are constantly memory constrained.