Text _segment_ size. That's executable instructions; presumably it makes a bit more sense that Google would have a binary with 300MB worth of instructions laying around.
Google has very large applications, but also they are using static linking: imagine that you statically link all the possible dependencies in a single binary for each application.
300MB is not considered a large binary at Google actually, I don't know what is public but you may look at Table 2 in this publication for instance: https://www.researchgate.net/publication/314105281_ThinLTO_S...
I repro here some data, there is a bit more in the paper:
> I'm curious what kinds of tasks require a binary with a 300 MB text size…
C++ lends itself to bloated binaries. Template instantiations are duplicated even for types where the instantiations could have identical code. Compilers have traditionally favored overly aggressive inlining over whole-program optimization.
Rust didn't solve the problem; it just duplicated the C++ approach, and made it worse by effectively compiling with the equivalent of C++ exceptions enabled. The next great systems language should target binary size and CPU frontend overhead.
> Template instantiations are duplicated even for types where the instantiations could have identical code.
compilers and linkers had identical code folding optimizations for a looooong time (20+ years for msvc, a decade or so for GNU gold (--icf option) and GCC (-fipa-icf))
I was going to reply with a similar comment, but I'm unable to actually get this to happen when I tried it now: https://godbolt.org/z/4VkEQc. Perhaps the optimization needs more work?
> compilers and linkers had identical code folding optimizations for a looooong time (20+ years for msvc, a decade or so for GNU gold (--icf option) and GCC (-fipa-icf))
If you look at the resulting binaries, they don't work as well as you might think.
Not sure why you get downvoted. I've seen presentations by Ubisoft for example where they replaced templated vectors because of the code bloat which added up too much in their large code base.
Rust still has some room to improve over C++ here. Because it knows up-front which operations generic code can use on its type parameters, it can do things like "polymorphization": https://github.com/rust-lang/rust/pull/69749
This means the compiler will automatically detect how much duplication is actually needed ahead of time, eliminating the need for after-the-fact identical code folding. (This should also drastically reduce compile times for some programs!)
And it could go even further than that. For example, some code could be duplicated based only on the size+alignment of its parameters (C#-style generics without a JIT). The analysis could be finer-grained than per-function, sharing parts of a function across instantiations (kind of like outlining).
With the detection happening this early in the pipeline, the compiler could even tell the user about it and guide them toward techniques that reduce the amount of duplication. (A different language could go further, of course.)
This is profile-guided optimization: you run your program, collect data on what code runs, and use it to optimize the layout of your program. Dynamo runs optimizations at runtime.
27 comments
[ 3.0 ms ] story [ 75.3 ms ] threadI'm curious what kinds of tasks require a binary with a 300 MB text size…
I repro here some data, there is a bit more in the paper:
Clang 1.9k Files, 217MB (-g0), 3554MB (-g) Chromium 17.8k Files 706MB (-g0), 7544MB (-g) Ad Delivery 13.8k Files 1073MB (-g0), 7469MB (-g)
(this is the number of object file that are participating in the link).
Ah, that would probably be it.
C++ lends itself to bloated binaries. Template instantiations are duplicated even for types where the instantiations could have identical code. Compilers have traditionally favored overly aggressive inlining over whole-program optimization.
Rust didn't solve the problem; it just duplicated the C++ approach, and made it worse by effectively compiling with the equivalent of C++ exceptions enabled. The next great systems language should target binary size and CPU frontend overhead.
compilers and linkers had identical code folding optimizations for a looooong time (20+ years for msvc, a decade or so for GNU gold (--icf option) and GCC (-fipa-icf))
On my machine I get no unsigned instantiation, only the signed one :
and the main :If you look at the resulting binaries, they don't work as well as you might think.
This means the compiler will automatically detect how much duplication is actually needed ahead of time, eliminating the need for after-the-fact identical code folding. (This should also drastically reduce compile times for some programs!)
And it could go even further than that. For example, some code could be duplicated based only on the size+alignment of its parameters (C#-style generics without a JIT). The analysis could be finer-grained than per-function, sharing parts of a function across instantiations (kind of like outlining).
With the detection happening this early in the pipeline, the compiler could even tell the user about it and guide them toward techniques that reduce the amount of duplication. (A different language could go further, of course.)
https://cseweb.ucsd.edu/classes/sp00/cse231/dynamopldi.pdf ?
Interesting stuff!