This isn't unique to games, and it's not just "today". Go back a decade [0] find people making similar observations about one of the largest tech companies on the planet.
And that's consumer apps, having only glimpsed in the world of back-end / cloud shenanigans, there's heaps of data being generated and stored in datacenters. Useful data? Dunno, how useful are all access logs ever?
But it's stored because it's possible, easy, and cheap. Unlike older games, where developers would hide unused blocks of empty data for some last-minute emergency cramming if they needed it.
The idea is to duplicate assets so loading a "level" is just sequential reading from the file system. It's required on optical media and can be very useful on spinning disks too. On SSDs it's insane. The logic should've been the other way around. Do a speed test on start an offer to "optimise for spinning media" if the performance metrics look like it would help.
If the game was ~20GB instead of ~150GB almost no player with the required CPU+GPU+RAM combination would be forced to put it on a HDD instead of a SSD.
This idea of one continuous block per level dates back to the PS1 days.
Hard drives are much, much faster than optical media - on the order of 80 seeks per second and 300 MB/s sequential versus, like, 4 seeks per second and 60 MB/s sequential (for DVD-ROM).
You still want to load sequential blocks as much as possible, but you can afford to have a few. (Assuming a traditional engine design, no megatextures etc) you probably want to load each texture from a separate file, but you can certainly afford to load a block of grass textures, a block of snow textures, etc. Also throughput is 1000x higher than a PS1 (300 kB/s) so you can presumably afford to skip parts of your sequential runs.
I was curious if they optimized the download. Did it download the 'optimized' ~150 GB and wasting a lot of time there or did it download the ~20 GB unique data and duplicated as part of the installation.
I still don't know but found instead an interesting reddit post were users found and analyzed this "waste of space" three month ago.
My takeaway is that it seems like they did NO benchmarking of their own before choosing to do all that duplication. They only talk about performance tradeoff now that they are removing it. Wild
It's pretty standard to do that duplication for games on CD/DVD because seek times are so long. It probably just got carried over as the "obviously correct" way of doing things, since HDDs are like DVDs if you squint a bit
> our worst case projections did not come to pass. These loading time projections were based on industry data - comparing the loading times between SSD and HDD users where data duplication was and was not used. In the worst cases, a 5x difference was reported between instances that used duplication and those that did not. We were being very conservative and doubled that projection again to account for unknown unknowns.
Non-made up numbers from Vermintide 2 (same engine): On PS4 when an optimized build took around 1.5 minutes to boot to main menu, the unoptimized version would take 12-15 minutes [1]. A different benchmark than SSD vs HDD, but shows that the optimization was certainly needed at the time.
Though the PS4 was partially to blame as well, with it's meagre 5400 RPM spinny drive.
For their newer instalment, Fatshark went with a large rework of the engine's bundle system, and players on HDDs are complaining about long loading times expectedly. That game is still large at ~80GB, but not from duplication.
The good old "studios don't play their own games" strikes again :P
Games would be much better if all people making them were forced to spend a few days each month playing the game on middle-of-the-road hardware. That will quickly teach them the value of fixing stuff like this and optimising the game in general.
It's an valid issue, those of us who worked back in the day on GD/DVD,etc games really ran into bad loading walls if we didn't duplicate data for straight streaming.
Data-sizes has continued to grow and HDD-seek times haven't gotten better due to physics (even if streaming probably has kept up), the assumption isn't too bad considering history.
It's a good that they actually revisited it _when they had time_ because launching a game, especially a multiplayer one, will run into a lot of breaking bugs and this (while a big one, pun intended) is still by most classifications a lower priority issue.
You can't bench your finished game before it exists and you don't really want to rock the boat late in dev, either.
It was a fundamentally sound default that they revisited. Then they blogged about the relatively surprising difference it happen to make in their particular game. As it turns out the loading is CPU bound anyway, so while the setting is doing it's job, in the context of the final game, it happens to not be the bottle neck.
There's also the movement away from HDD and disc drives in the player base to make that the case as well.
I love Helldivers 2, but from what I can tell it's a bunch of enthusiasts using a relatively broken engine to try to do cool stuff. It almost reminds me of the first pokemon game. I'll bet there's all sorts of stuff they get wrong from a strictly technical standpoint. I love the game so much I see this more as a charming quirk than I do something which really deserves criticism. The team never really expected their game to be as popular as it's become, and I think we're still inheriting flaws from the surprise interest in the game. (some of this plays out in the tug of war between the dev team's hopes for a realistic grunt fantasy vs. and the player base's horde power fantasy.)
Pretty cool. I think it’s completely normal to be under a crunch and just go with some standard practices under normal conditions. Cool that they went back and sorted it out afterwards!
I’ve got to say. I do find it somewhat unusual that despite the fact that every HN engineer has John Carmack level focus on craftsmanship, about 1/100k here produce that kind of outcome.
I don’t get it. All of you guys are good at pointing out how to do good engineering. Why don’t you make good things?
The negativity towards this is wild. A company followed relatively widely accepted industry practice (lots and lots of other games also have huge sizes on disk for the exact same reason), then eventually they decided to do their own independent testing to check whether said common practice actually makes things better or not in their case, found that it didn't, so they reversed it. In addition, they wrote up some nice technical articles on the topic, helping to change the old accepted industry wisdom.
This seems great to me. Am I crazy? This feels like it should be Hacker News's bread and butter, articles about "we moved away from Kubernetes/microservices/node.js/serverless/React because we did our own investigation and found that the upsides aren't worth the downsides" tend to do really well here. How is this received so differently?
I'm glad they've been able to do this, looks like a huge improvement for HD2 on PC.
I've been on PS5 since launch and aside from Baldur's Gate 3, it's been the best game this gen IMO.
The negativity I see towards the game (especially on Youtube) is weird. Some of the critiques seem legit but a lot of feels like rage bait, which appears to be a lot of YT videos around gaming lately.
Anyway, a big improvement for a great game. Seems like less of an incentive now to uninstall if you only play now and then.
If this article was exciting for you, I also highly recommend this one. A random dude fixed a bug in GTA 5 that was the root cause of it loading insanely slowly since the game came out!
38 comments
[ 2.5 ms ] story [ 70.6 ms ] thread[0] https://news.ycombinator.com/item?id=10066338
This reminds me of the old days when I check who's using my PC memory every now and then.
But it's stored because it's possible, easy, and cheap. Unlike older games, where developers would hide unused blocks of empty data for some last-minute emergency cramming if they needed it.
If the game was ~20GB instead of ~150GB almost no player with the required CPU+GPU+RAM combination would be forced to put it on a HDD instead of a SSD.
Hard drives are much, much faster than optical media - on the order of 80 seeks per second and 300 MB/s sequential versus, like, 4 seeks per second and 60 MB/s sequential (for DVD-ROM).
You still want to load sequential blocks as much as possible, but you can afford to have a few. (Assuming a traditional engine design, no megatextures etc) you probably want to load each texture from a separate file, but you can certainly afford to load a block of grass textures, a block of snow textures, etc. Also throughput is 1000x higher than a PS1 (300 kB/s) so you can presumably afford to skip parts of your sequential runs.
I still don't know but found instead an interesting reddit post were users found and analyzed this "waste of space" three month ago.
https://www.reddit.com/r/Helldivers/comments/1mw3qcx/why_the...
PS: just found it. According to this Steam discussion it does not download the duplicate data and back then it only blew up to ~70 GB.
https://steamcommunity.com/app/553850/discussions/0/43725019...
[0] https://partner.steamgames.com/doc/sdk/uploading#AppStructur...
282 comments
>we looked at industry standard values and decided to double them just in case.
They basically just made the numbers up. Wild.
For their newer instalment, Fatshark went with a large rework of the engine's bundle system, and players on HDDs are complaining about long loading times expectedly. That game is still large at ~80GB, but not from duplication.
[1]: https://www.reddit.com/r/Vermintide/comments/hxkh0x/comment/...
Games would be much better if all people making them were forced to spend a few days each month playing the game on middle-of-the-road hardware. That will quickly teach them the value of fixing stuff like this and optimising the game in general.
Data-sizes has continued to grow and HDD-seek times haven't gotten better due to physics (even if streaming probably has kept up), the assumption isn't too bad considering history.
It's a good that they actually revisited it _when they had time_ because launching a game, especially a multiplayer one, will run into a lot of breaking bugs and this (while a big one, pun intended) is still by most classifications a lower priority issue.
I don’t know about the Xbox, but on PS4 the hard drive was definitely not fast at all
It was a fundamentally sound default that they revisited. Then they blogged about the relatively surprising difference it happen to make in their particular game. As it turns out the loading is CPU bound anyway, so while the setting is doing it's job, in the context of the final game, it happens to not be the bottle neck.
There's also the movement away from HDD and disc drives in the player base to make that the case as well.
I’ve got to say. I do find it somewhat unusual that despite the fact that every HN engineer has John Carmack level focus on craftsmanship, about 1/100k here produce that kind of outcome.
I don’t get it. All of you guys are good at pointing out how to do good engineering. Why don’t you make good things?
This seems great to me. Am I crazy? This feels like it should be Hacker News's bread and butter, articles about "we moved away from Kubernetes/microservices/node.js/serverless/React because we did our own investigation and found that the upsides aren't worth the downsides" tend to do really well here. How is this received so differently?
I've been on PS5 since launch and aside from Baldur's Gate 3, it's been the best game this gen IMO.
The negativity I see towards the game (especially on Youtube) is weird. Some of the critiques seem legit but a lot of feels like rage bait, which appears to be a lot of YT videos around gaming lately.
Anyway, a big improvement for a great game. Seems like less of an incentive now to uninstall if you only play now and then.
iO
https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times...
Just don’t get caught at the end!
I feel like writes would probably be quite painful, but with game assets are essentially write-once read-forever so not the end of the world?
As an aside, its messed up that people with expensive SSDs are unnecessarily paying this storage tax. Just feels lazy...