Also, I wonder how many obscure bugs were introduced by using more aggressive compiler flags? The image has "lto" in the name, and at https://github.com/docker-library/python/issues/160 I see this comment:
> In the future the Python developers may decide to turn on further optimizations based on this argument, for example link-time optimizations (LTO), though they haven't worked out all the bugs for that one yet.
Yeah I thought it would be nice to release this at North Bay today. We also released our new website redesign (and new infrastructure setup) last night so there is probably some dust on a few things we'll clean up over the next few days.
I don't think we ran benchmarks with LTO and PGO in isolation, so honestly not sure exactly how much of a contribution each makes on each benchmark. PGO is definitely the bulk however.
So I see that profile-guided optimization are being used.
Profile-guided optimization, in layman terms, means that you run you code under a profiler for a while, see what parts (branches, functions, data structures etc) are being used the most and use this information to make a build of your code that considers the profiler's findings when doing optimization.
So what does it mean? It means, basically, that revsys is publishing a python build that is optimized for their use-case. Which may or may not be your use-case. This is not good nor bad.
Still, the claim "up to 19% faster" is false in general (but true in a particular case -- their use case).
Just keep this in mind, because this python build might perform worse than a regular python build.
Actually I thought that as well when I first saw it was an option, but these are numbers comparing the official images to these images we built using the standard Python benchmarks. Not compared to say some code we wrote or run in production.
We did 10 runs of each on a large AWS instance and averaged the results.
However, definitely agree that any optimization like this could have negative performance consequences for certain specific bits of code. But that’s likely true of any build to the next.
17 comments
[ 4.3 ms ] story [ 51.6 ms ] thread> In the future the Python developers may decide to turn on further optimizations based on this argument, for example link-time optimizations (LTO), though they haven't worked out all the bugs for that one yet.
(Thanks mastax for finding the issue!)
Here's the Dockerfiles and readme:
https://github.com/rcarmo/ubuntu-python
I have some trust concerns about basing my infrastructure on something opaque.
Another alternative for a fast speed up by changing your base image is trying out the pypy images.
Profile-guided optimization, in layman terms, means that you run you code under a profiler for a while, see what parts (branches, functions, data structures etc) are being used the most and use this information to make a build of your code that considers the profiler's findings when doing optimization.
So what does it mean? It means, basically, that revsys is publishing a python build that is optimized for their use-case. Which may or may not be your use-case. This is not good nor bad.
Still, the claim "up to 19% faster" is false in general (but true in a particular case -- their use case).
Just keep this in mind, because this python build might perform worse than a regular python build.
We did 10 runs of each on a large AWS instance and averaged the results.
The build info and benchmark results are up here https://github.com/revsys/optimized-python-docker if we’re wrong about something please let us know.
However, definitely agree that any optimization like this could have negative performance consequences for certain specific bits of code. But that’s likely true of any build to the next.