If anyone's looking for a great long-form article about calculating digits of Pi from the good ol' days, this 1992 New Yorker profile of the Chudnovsky brothers (creators of algorithm used by Google here!) is fantastic:
Reading that, it's hard not to see the parallels to Darren Aronofsky's movie Pi. Building a supercomputer in an apartment to find patterns in Pi that have some higher meaning, the computer constantly breaking, etc.
The economic impact is not negligible, even if it does come out of the GCP PR budget: about $360k. Yada yada idle servers, but what could all that idle compute be used for to benefit humanity?
Not quite sure how to calculate it, but a n2-highmem-128 instance running at near peak for 24 hours surely uses more electricity than the energy required to heat a dozen or two gallons of water per day.
64 cores and close to 1 TB of RAM will probably consume not more than 2.5 kW or so based on available TDP/power consumption figures.
In 24 hours it consumes about the same amount of energy (2.5kW * 24h = 60 kWh) as charging a Model 3 LR's battery from 0 to 75%. So definitely more than heating up two dozen gallons of water even to boiling temperature (8 kWh), but still within the realm of household consumption with EVs.
When I am king I will make it illegal to loudly worry about environmental impact (or anything else) without providing an order of magnitude estimate for the size of the problem. You don't even have to be right, just take five seconds to conjure a vaguely plausible number.
I'm hoping for something closer to the ending of Carl Sagan's novel, "Contact". It only mentioned pi on half a page, but it sticks with you.
I won't spoil it, but that book beautifully walked a skeptical / existential / transcendental line and makes you feel a special warmth despite being so immeasurably small.
The book was much better than the (quite adequate) film.
There is a dead post in this thread talking about what if "at 101T digits we find an infinite series of 000000000".
I just wanted to point out that this is impossible, because in that case pi would be a rational number. However, it's been proven to be a transcendental number, which is a stronger condition than just being irrational. So, it's not possible that pi ends with any infinitely repeating sequence.
But more weirdly, it's possible to find any string of digits within the digits of pi. So there are going to be quite a few zeros in there, followed by ever more non-zeros.
Fun fact: Mahler famously showed that if p/q is a rational number then |\pi - p/q| > 1/q^42. So we can't have a string of 4242T zeros after first 101T digits :-) The exponent 42 has since been improved by a series of works to 7.1032 - https://arxiv.org/abs/1912.06345
You can't have a string of 4242T zeroes immediately after the first 101T digits but presumably you could have them sometime later (say after another 10000T digits)
I seem to remember reading a sci-fi story quite a long time ago where the characters discover a message embedded in pi but only after many digits were calculated.
I think the implication was something about simulations or maybe instructions on how to build a device. Can't quite remember the details.
In Northern Exposure season 4 episode 3 ("Nothing's Perfect"), a grad student comes to Cicely to study Pi.
"I just have this feeling if I take pi, well past all this static, take pi to 10 million, 20 million digits, that I'll find something really incredible. Not just a pattern, not just an order, but a sign. A mathematical sign."
> Total storage size: 663 TB available, 515 TB used
I was interested to see how they're storing the result (100T digits), it's not clear if this if memory for the result or for the algorithm. I'd be interested to hear what the encoding is; a byte/char per digit, or a nibble?
I would imagine doing something like storing 12 decimal digits every 5 bytes (exploiting 10^12 <≈ 2^40) would be simple enough, great for random access, and good enough at compressing (within 10% of theoretical optimal treating pi as a random sequence of digits) that it's not worth spending a lot of time trying to improve upon that.
Of course, under a different set of constraints, one may find something better.
So we've already had the Bailey–Borwein–Plouffe (BBP) formula for computing an arbitrary digit of pi in base 16, which by itself is mind blowing, but I just found out that Plouffe has come up with a new version for base 10.
But the base 10 formula needs Bernoulli numbers, which themselves are defined recursively, meaning the required iterations are just hidden and not removed.
And how much a computer that could have done the computation all by itself would have cost (by putting enough storage in a single chassis without needing a SAN).
The most disappointing part: It's done with a closed-source program. Just like almost all pi records in the past decades.
The authors of the programs refuse to open source and instead opt to just release free binaries. Their reasons are that their algorithms are secret.
I suspect they don't want their program forked and don't want someone to take credit for a computation done with their program. Surprisingly, it has worked so far.
It should be noted that y-cruncher, that closed-source program, is responsible for every pi computation record since 2010 (back then the record was 2.7T digits). Traditionally these records were held by supercomputers, but it seems that pi computation is no longer a good way to benchmark or stress-test them [1], leaving only interested individuals in the scene.
If pi computation were useful by its own, or there were a lot of people working towards records, we can expect multiple competitive pi computation programs where some of them are hopefully F/OSS. (The algorithm itself is not exactly secret anyway and big ideas are already in public, what makes y-cruncher special is its meticulous optimization.) But I don't see anything like that. There seem no actual demands for pi computation software, not just F/OSS.
The main node has 64 physical cores and 1TB of RAM, and the 24 storage nodes have 8 cores and 16 GB of RAM (192 cores and 384GB of RAM total).
So we are using 256 cores and 1.5 TB of RAM, so probably about 7.5 kW or so. Over 5 months, the compute would have used 27 MWh, so about the energy content of ~750 gallons of gas.
The storage will obviously use more energy (I don't know if they are using spinning disks or SSDs), but I think ~50-100 MWh for the entire computation is a reasonable estimate.
50 comments
[ 3.4 ms ] story [ 108 ms ] threadhttps://www.newyorker.com/magazine/1992/03/02/the-mountains-...
https://cloud.google.com/blog/topics/sustainability/clean-en...
In 24 hours it consumes about the same amount of energy (2.5kW * 24h = 60 kWh) as charging a Model 3 LR's battery from 0 to 75%. So definitely more than heating up two dozen gallons of water even to boiling temperature (8 kWh), but still within the realm of household consumption with EVs.
And figure: oh shit? Did we solve pi?
I won't spoil it, but that book beautifully walked a skeptical / existential / transcendental line and makes you feel a special warmth despite being so immeasurably small.
The book was much better than the (quite adequate) film.
Sagan was an extraordinary communicator.
(Also Pi is irrational, so this won't happen.)
I just wanted to point out that this is impossible, because in that case pi would be a rational number. However, it's been proven to be a transcendental number, which is a stronger condition than just being irrational. So, it's not possible that pi ends with any infinitely repeating sequence.
https://en.wikipedia.org/wiki/Normal_number
He listed at least 21 states between "conjecture" and "proof".
Transcendental numbers are truly awesome.
I think the implication was something about simulations or maybe instructions on how to build a device. Can't quite remember the details.
"I just have this feeling if I take pi, well past all this static, take pi to 10 million, 20 million digits, that I'll find something really incredible. Not just a pattern, not just an order, but a sign. A mathematical sign."
https://moosechick.com/43.html
https://storage.googleapis.com/pi100t/index.html
I was interested to see how they're storing the result (100T digits), it's not clear if this if memory for the result or for the algorithm. I'd be interested to hear what the encoding is; a byte/char per digit, or a nibble?
Of course, under a different set of constraints, one may find something better.
For some reason the blog post mentions the 31.4T record and the 62.8T record, but not the 50T record [1]. Weird.
[1] https://blog.timothymullican.com/calculating-pi-my-attempt-b...
https://mathworld.wolfram.com/Digit-ExtractionAlgorithm.html
The authors of the programs refuse to open source and instead opt to just release free binaries. Their reasons are that their algorithms are secret.
I suspect they don't want their program forked and don't want someone to take credit for a computation done with their program. Surprisingly, it has worked so far.
If pi computation were useful by its own, or there were a lot of people working towards records, we can expect multiple competitive pi computation programs where some of them are hopefully F/OSS. (The algorithm itself is not exactly secret anyway and big ideas are already in public, what makes y-cruncher special is its meticulous optimization.) But I don't see anything like that. There seem no actual demands for pi computation software, not just F/OSS.
[1] http://numberworld.org/y-cruncher/faq.html#why_desktops
> End time: Mon Mar 21 04:16:52 2022 UTC
> Total elapsed time: 157 days, 23 hours, 31 minutes and 7.651 seconds
> Total storage size: 663 TB available, 515 TB used
> Total I/O: 43.5 PB read, 38.5 PB written, 82 PB total
Yea, but how much energy did you waste for 100T digits of crap, Google???
This was done thru a massive energy crisis, mind you.
So we are using 256 cores and 1.5 TB of RAM, so probably about 7.5 kW or so. Over 5 months, the compute would have used 27 MWh, so about the energy content of ~750 gallons of gas.
The storage will obviously use more energy (I don't know if they are using spinning disks or SSDs), but I think ~50-100 MWh for the entire computation is a reasonable estimate.
The energy crisis mainly affects oil and coal prices in the US so I don't believe Google's renewable-powered computations should have any effect.
If they'd run it in Germany where people are being asked to conserve electricity, things might be different.
[0]: https://github.com/GoogleCloudPlatform/pi-delivery/blob/main...
[1]: https://cloud.google.com/sustainability/region-carbon