Did the duplicated files were even used on pc? Like, do you even have such low access to the file system that you can deduce which duplicated instance has a faster access time on a mechanical hard drive?
Well yes? Have you played the game? It's got lot of content and is very pretty.
Those high resolution textures will just take space. You could obviously decrease the graphical fidelity but I'd guess that most players (me very much included) would rather play a very pretty 23GB Helldivers II than a 5GB ugly Helldivers II.
150GB was very annoying, ironically forcing me to install it to a HDD. 23GB isn't even worth thinking about for me.
I've been really curious precisely what changed, and what sort of optimization might have been involved here.
Because offhand, I know you could do things like cute optimizations of redundant data to minimize seek time on optical media, but with HDDs, you get no promises about layout to optimize around...
The only thing I can think of is if it was literally something as inane as checking the "store deduplicated by hash" option in the build, on a tree with copies of assets scattered everywhere, and it was just nobody had ever checked if the fear around the option was based on outcomes.
(I know they said in the original blog post that it was based around fears of client performance impact, but the whole reason I'm staring at that is that if it's just a deduplication table at storage time, the client shouldn't...care? It's not writing to the game data archives, it's just looking stuff up either way...)
I did similar work on a game a long time ago and it took over a month to slim it down to 1/4 of the size but in this case 'at runtime' - the producer wasn't impressed. It looked exactly the same. I wonder if they had any pushback.
Possibly a similar process to when you go into an AWS account, and find dozens of orphaned VMs, a few thousand orphaned disk volumes, etc., saving like $10k/month just deleting unused resources.
> With their latest data measurements specific to the game, the developers have confirmed the small number of players (11% last week) using mechanical hard drives will witness mission load times increase by only a few seconds in worst cases. Additionally, the post reads, “the majority of the loading time in Helldivers 2 is due to level-generation rather than asset loading. This level generation happens in parallel with loading assets from the disk and so is the main determining factor of the loading time.”
It seems bizarre to me that they'd have accepted such a high cost (150GB+ installation size!) without entirely verifying that it was necessary!
I expect it's a story that'll never get told in enough detail to satisfy curiosity, but it certainly seems strange from the outside for this optimisation to be both possible and acceptable.
a one time cost of a big download is something customers have shown time and again that they're willing to bear. remember that software is optimized for ROI first and all other things second. Sometimes optimizing for ROI means "ship it today and let the first week of sales pay salaries while we fix it", sometimes ROI means picking between getting the file size down, getting that new feature out and fixing that game breaking edge case bug. Everything you do represents several things you choose not to do.
Back of the envelope, in the two years since the game was released, this single bug has wasted at least US$10,000,000 of hardware resources. That's a conservative estimate (20% of people who own the game keep it installed, the marginal cost of wasted SSD storage in a gaming PC is US$2.50 per TB per month, the install base grew linearly over time), so the true number is probably several times higher.
In other words, the game studio externalised an eight-figure hardware cost onto their users, to avoid a five-to-six-figure engineering cost on their side.
Data duplication can't just be banned by Steam, because it's a legitimate optimisation in some cases. The only safeguard against this sort of waste is a company culture which values software quality. I'm glad the developers fixed this bug, but it should never have been released to users in the first place.
What if.. the management made a request to make the game take more space than the previous release?
So everyone could see just how much content there is and how much better everything is.
I mean, the developers cannot be that incompetent while being able to ship a high quality product.
I recently downloaded Hunt showdown. I think it was around 70 gigs. About a month later, I had to update it. The download was the same size. I think they literally just overrode the entire game because they were too lazy to update it properly.
> "It's no surprise to see modern AAA games occupying hundreds of gigabytes of storage these days"
Is it not? I've genuinely never understood it!
I used to do a little bit of level building for IdTech3 games back in the day but it's been 20 years. I'm not totally ignorant of what's involved, just mostly ignorant. I really want to know though, what is all that data!? Textures?
In particular I find the massive disparity between decently similar games interesting. Indiana Jones and the Great Circle takes something like 130gb on my Xbox, whereas Robocop: Rogue City takes something like 8gb. They have similar visual fidelity, I would say Robocop might have a little bit of a lead, but Indiana Jones has fancier dynamic lighting.
At 130gb though, I almost could have streamed my entire playthrough of the game at 4k and came out on top.
It was legit faster to delete and redownload this game than update it since steam considered my SSD too full (WITH 200 GIGS FREE) to download the files to said SSD, instead opting to use my SLOWEST HDD as the cache drive for the download.
It would then proceed to download the update in 5 minutes and spend 8 HOURS UPDATING.
A full download of the game? 10 minutes.
Glad to see this update. I hope more games follow suit
“On a higher PC it wouldn’t be an issue. On a medium or moderate PC, it wouldn’t be an issue, it’s that on a two-core [machine] with where our min spec is, we couldn’t dedicate those resources to audio.”
It was BS considering countless other games having no problem with sound. Decoding something like Opus takes ~30MHz of a single CPU core[1], meaning even an unreasonable situation of decoding 16 simultaneous uninterrupted 128Kbit Stereo streams would only eat half of one core.
It's also bullshit. Half Life 2 didn't need uncompressed audio. MP3 decompression is damn near free.
Half life also did it with half the required minimum cores, and 1 ghz less clock speed on that CPU. It released a decade before Titanfall 1. Sure sure, it's got so much more going on, but uh, that much?
For a reference of how trivial it is for CPUs to decode MP3 files, software decoders take tens of MIPS. Remember that unit? Less than a percent of one of those minimum spec CPUs required.
You know what's funny? The source engine only supports MP3 compressed audio. Do you know what titanfall 1 downloads and decompresses to to create 30gb of audio data? Lossy compressed, 160kb/s OGG format audio.
40 comments
[ 3.4 ms ] story [ 61.0 ms ] threadThose high resolution textures will just take space. You could obviously decrease the graphical fidelity but I'd guess that most players (me very much included) would rather play a very pretty 23GB Helldivers II than a 5GB ugly Helldivers II.
150GB was very annoying, ironically forcing me to install it to a HDD. 23GB isn't even worth thinking about for me.
Because offhand, I know you could do things like cute optimizations of redundant data to minimize seek time on optical media, but with HDDs, you get no promises about layout to optimize around...
The only thing I can think of is if it was literally something as inane as checking the "store deduplicated by hash" option in the build, on a tree with copies of assets scattered everywhere, and it was just nobody had ever checked if the fear around the option was based on outcomes.
(I know they said in the original blog post that it was based around fears of client performance impact, but the whole reason I'm staring at that is that if it's just a deduplication table at storage time, the client shouldn't...care? It's not writing to the game data archives, it's just looking stuff up either way...)
It seems bizarre to me that they'd have accepted such a high cost (150GB+ installation size!) without entirely verifying that it was necessary!
I expect it's a story that'll never get told in enough detail to satisfy curiosity, but it certainly seems strange from the outside for this optimisation to be both possible and acceptable.
In other words, the game studio externalised an eight-figure hardware cost onto their users, to avoid a five-to-six-figure engineering cost on their side.
Data duplication can't just be banned by Steam, because it's a legitimate optimisation in some cases. The only safeguard against this sort of waste is a company culture which values software quality. I'm glad the developers fixed this bug, but it should never have been released to users in the first place.
It's also a title that shows you can have a really good game without the latest tech.
I mean, the developers cannot be that incompetent while being able to ship a high quality product.
https://www.pcgamer.com/call-of-duty-take-note-warframe-is-r...
Is it not? I've genuinely never understood it!
I used to do a little bit of level building for IdTech3 games back in the day but it's been 20 years. I'm not totally ignorant of what's involved, just mostly ignorant. I really want to know though, what is all that data!? Textures?
In particular I find the massive disparity between decently similar games interesting. Indiana Jones and the Great Circle takes something like 130gb on my Xbox, whereas Robocop: Rogue City takes something like 8gb. They have similar visual fidelity, I would say Robocop might have a little bit of a lead, but Indiana Jones has fancier dynamic lighting.
At 130gb though, I almost could have streamed my entire playthrough of the game at 4k and came out on top.
It would then proceed to download the update in 5 minutes and spend 8 HOURS UPDATING.
A full download of the game? 10 minutes.
Glad to see this update. I hope more games follow suit
Random IO can be 100 to 200 times slower than sequential reads on small file workloads.
from https://www.escapistmagazine.com/titanfall-dev-explains-the-...
[1] iPod Classic (1998 era ARM9) decodes 128 kbps stereo Opus at ~150% real time at stock cpu frequency. Opus is not the lightest choice either https://www.rockbox.org/wiki/CodecPerformanceComparison#ARM
Half life also did it with half the required minimum cores, and 1 ghz less clock speed on that CPU. It released a decade before Titanfall 1. Sure sure, it's got so much more going on, but uh, that much?
For a reference of how trivial it is for CPUs to decode MP3 files, software decoders take tens of MIPS. Remember that unit? Less than a percent of one of those minimum spec CPUs required.
You know what's funny? The source engine only supports MP3 compressed audio. Do you know what titanfall 1 downloads and decompresses to to create 30gb of audio data? Lossy compressed, 160kb/s OGG format audio.