156 comments

[ 4.3 ms ] story [ 174 ms ] thread
(comment deleted)
But is it really random? #conspiracy
It's open to the public‽ Seems like a bad idea to let a potential spy in there to set up a camera and de-randomize this source of random info. Also, the headline on HN declares it a fact that the lava lamps are assisting in the encryption, but the article is careful to say "maybe", "might", etc.

This seems wildly insecure and much more likely to represent a weak link than to actually aid in randomness.

Even if the data is not random it doesn't weaken your kernel prng by writing non-random data to urandom.

Don't overcomplicate your threat model with non-existent risks

Didn't DJB somewhere describe a PRNG model in which this is not true and you shouldn't let an adversary give you PRNG seeds?

(I don't have any problem with what CloudFlare is doing.)

Edit: While I was writing this comment, drewbug linked to the exact DJB post I was thinking of. Thanks!

Someone observing the lava lamps still can't control their appearance.
The nice thing about randomness is that given a random blob, you can xor a non random blob & it's just as random. So this is one input of entropy amongst a cocktail of sources
That’s true statically. Keeping that true with changes over time is HARD. For example, it matters that the non random blob never gets to see either the random input or the output.
Actually that's the property of xor, not randomness. Data xor uniform distribution -> results in uniform distribution. It doesn't have to be random.
You would also need to eavesdrop on a geiger counter in Singapore that's measuring radiation, and a chaotic pendulum in London: https://www.fastcodesign.com/90137157/the-hardest-working-of...

The effort involved in doing this would maybe be as taxing as brute-forcing the keys in the first place. It seems gimmicky but they're almost surely producing reliable random data that would be almost impossible to simulate otherwise.

How do they prevent the camera or its outgoing feed from being hacked or replaced by the NSA?

Edit: I see this is addressed partially by wkandek's link elsewhere in this discussion.

Hack the Planet = Hack a Camera?
Ah, apparently, based on one of the detailed links just posted, it just provides one feed of entropy to the mix -- not the sole source.
I might be prejudiced, but this looks like a big PR stunt/done for the cool factor kind of thing. Aren't there simpler/saner alternatives for getting good randomness?
What would be an easier way? A wall of lava lamps seems pretty easy to set up.
busy streets webcams?
That would be susceptible to traffic patterns.
Not meaningfully. The input doesn't need to be uniform, just unpredictable. Statistical patterns are fine, as long as no one can predict exactly what's going to happen (to the pixel level). Even if all the drivers on the road were collaborating, they couldn't coordinate well enough to feed the camera adversarially predictable inputs. At least, not human drivers...
I was thinking more along the lines of busy/non-busy times may affect how much random data you get at different times of the day/week.

Not specifically some sort of giant coordinated attack to modify the randomness generated to specific values.

Cars don't look exactly the same and are not in the exact same places. Lighting changes. People look very different and walk different paths. It's going through PRNG so even very small pixel differences (which you can find even on an empty street thanks to changing weather, litter, wind and so on) are good enough if they are unpredictable. In other words, when you compare pictures of the same lava lamp with pictures of some random cars, I think you will be able to find more patterns in the lava lamp movement. You would also likely be able to predict it better (well that's the same thing I guess).
My guess is there’s still enough noise it doesn’t matter.
Put a fountain in the lobby and have a microphone listening to the splashes.
There are an infinite sources of random audio out there that aren't susceptible to human interference.
The zillions of sources of EM/thermal/etc noise. It's so prevalent, most of the time the effort is spent in trying to get rid of it.
Very wideband Zener noise sources are a common piece of measurement equipment; and many folks built their own noise sources (also for RNGs) using Zener junction breakdown. While building my own I needed quite a bit of shielding to avoid interference issues by RF sources (mostly HF fluoerescent lights and LEDs), so these sources might be susceptible to bias by EMI...?

Other junction breakdowns have noise as well, resistive noise is very low power. Atmospheric noise and background noise is a thing too, but I'd guess they are too susceptible to interference by malicious actors...

Such sources are always post-processed to deal with bias and correlation. No sane implementation just takes the straight measured output, whether it's keyboard interrupt timing or radioactive decay.
the random noise from the camera sensor is plenty good randomness. It would work just as well without the lava lamps.
For sure. In another office they use a geiger counter (a much more standard way to get randomness, and probably easier to setup).

That said, this doubles as a public art installation in their lobby - something many companies spend thousands (or millions!) of dollars on. So it's not just PR, it's actually surprisingly practical.

I know I'm nitpicking here - it's a private art installation.
Anyone can visit it though, according to the article.
A private collection doesn’t mean the outside world isn’t allowed to visit. It just means it’s not owned by a public institution.
If it was a public art installation, wouldn't that leak their entropy?
You’d have to get a picture in the exact same angle, which is unlikely.
It leaks information at any angle.
Doesn’t really matter. With modern CSPRNGs, even if one input is compromised you’re still as strong as the remaining entropy (as long as those sources are statistically independent from the compromised one).

If this was the only source of randomness it might be a problem, but if they’re `cat`ting it into `/dev/random` as an external source, it can only really improve things. Even if someone were to compromise the feed from the office to the datacenter, it wouldn’t matter since they’d have to know the internal RNG state in order to “negate” its randomness with the lava lamp feed. If they have that, you’re already lost anyway.

You’d also have to be using very similar equipment and know precisely how they derive their random numbers from the image.
This is addressed in the last paragraph of the article.
Not really. They say that having random (ha) people in the picture improves the entropy, but they don't address the possibility of people using it to gain information about Cloudflare's entropy pool.
presumably if you set up camp with a camera in their lava lamp room, somebody is going to raise some eyebrows.
Not only that but your cameras are going to have to be precisely in the same place as theirs, with an identical view and be identical models with identical sensor variation, dust on lens etc etc etc ad nauseam.

As you say, that might raise a few eyebrows.

Wouldn't you just cover their camera with paper or some other known pattern, if you were going to try that?
Ooh! Can you shine a laser into the camera to blind it? Burn the sensor to a crisp and then let that seed the CSPRNG!
Great way to test how quickly you can get thrown out by security guards
Short of them actually publishing the image signature the camera takes as the random input, no. Imagine just how much sensor noise is sitting across the image, and figure on other elements others have mentioned: the angle, the aperture, etc. I imagine two near-identical frames without any movement between them taken on a single camera might still have a significant amount of entropy between them, especially if you're using the camera raw data, for those reasons.

They (CF) discuss this on their deep dive:

>>>The flow of the “lava” in a lava lamp is very unpredictable,6 and so the entropy in those lamps is incredibly high. Even if we conservatively assume that the camera has a resolution of 100x100 pixels (of course it’s actually much higher) and that an attacker can guess the value of any pixel of that image to within one bit of precision (e.g., they know that a particular pixel has a red value of either 123 or 124, but they aren’t sure which it is), then the total amount of entropy produced by the image is 100x100x3 = 30,000 bits (the x3 is because each pixel comprises three values - a red, a green, and a blue channel). This is orders of magnitude more entropy than we need. [1]

1: https://blog.cloudflare.com/lavarand-in-production-the-nitty...

If the information encoded in the lava lamps is not important for the entropy, why do they even need to be there?

If it just needs to be some image, and it doesn't actually matter if others know what it can see, why not just point the camera at a normal lamp, or an empty room, or the sky?

It is, the author is just saying that even without it they still have enough entropy in the system to be secure enough.

Any old image doesn't have pretty strong random number generators in it.

Geiger counter might happen to be very vulnerable to collocated attacks. Overwhelm counter with non-random radiation, compromise a batch of keys produced during the burst,
Are you suggesting a gamma ray beam? Or neutron beam? (You couldn't do an alpha particle beam - that's stopped by a sheet of paper... and the beta particle ray would have difficulty with the building construction).

This then goes to the question of "where are you getting a focused gamma ray beam that has sufficient rate that you're able to modulate the period between two detection events?"

This depends on what the counter in question is measuring. If it's "background" radiation, where background is outside the counter, it should be pretty easy to bring your own foreground.

If it's an internal source and it is properly shielded, this becomes much harder.

Even if you did either of those... the bits (at least the hotbits algorithm) works off of the detection of multiple events. Bringing in some Iodine 128 into a place (half-life of 25 minutes) into the area, that would change the rate of bit generation, but not actually influence the bits themselves.

Lets say you've got some cesium that's creating 800 bits per section. That one (now that I go back and read it closely) is based on four events: E1 <-(T1)-> E2, E3 <-(T2) -> E4. If the time between E1 and E2 is less than the time between T3 and T4 - its a 0. If its greater, its 1.

Bringing in some other source wouldn't change that algorithm unless one could control the "when" to some degree so that the gigercounter detects your events with greater frequency than the other source... and at that point, you might as well unplug the gigercounter from the com port and feed in your own data.

This could be done with background radiation too - but that's many fewer events and so a lower rate of germination. The time between two events is random and will remain random even with more radioactivity in the area.

Often times devices of secure randomness run the bell test to ensure that they aren’t being tampered with. I believe that blasting the counter with a gamma ray beam would indicate this.
(comment deleted)
> this looks like a big PR stunt/done for the cool factor kind of thing

Probably, but it is indeed pretty cool. I appreciate when people take something potentially mundane and bring style to it.

I suspect its a matter of getting a lot of random rapidly with easily replaceable and unregulated devices.

The classic HotBits ( https://www.fourmilab.ch/hotbits/ ) uses radioactive decay to get its random data. This generates 100 bytes per second of randomness. But that also depends on getting a radiation source. A single one costs about $80. A lava lamp costs $10.

Working off the list of isotopes that they provide, You've got things that have half-lives. The issue there is that the way that one generates random data from radioactivity is get an event (A), and then get another event (B) and another event (C). The time between A and B is compared to the time between B and C. If AB is shorter than CD you've got a 1. If its longer than you've got a 0. If its the same, then throw about AB. As the material decays, its rate of generating bits of data slows down.

Lava lamps are easily replaceable and don't have half-lives.

I suspect there's also issue with "here is our lead lined vault where we keep a bunch of radioactive disks to generate random data". Sure, the units are micro Curie (the most active one is a 10uCi Cesium 137). But... do you want to go into that room and change a burnt out detector? How about swapping out a lava lamp because of a burnt out bulb?

>As the material decays, its rate of generating bits of data slows down.

can't you use something with a really long half-life?

but then your initial rate is much slower.
So get more material!
(comment deleted)
A 10 μCi 137Cs source is not dangerous. Even here in California, you can legally throw these away in the regular trash (if solid) or wash them down a sink drain (if liquid). Compared to the kind of ordinary household chemicals people are using every day in large volumes, these things are harmless. You definitely don't need a lead-lined room.

Source: I worked in a lab

It is entirely, 100% a publicity stunt with no practical importance whatsoever.
Yes, the article and the form in which it is exhibited is a PR stunt. However, that doesn't mean it's not practical in any capacity....if you locked this away in a room and never told anyone, it would still hold a utility for the company. Are there alternatives? Sure, but again that doesn't make it impractical.

It's also a fairly novel way to explain to people that `random()` isn't truly random.

It is unlikely that it holds any utility, or that it is practical in any capacity.
I think you're confusing the medium with which this achieved versus it's utility.

Surely you're not suggesting PRNG/DRBG isn't completely safe from hacking?

I don't even understand this question. Can you reword it?
Sure. I'm struggling to understand what utility your questioning. Is is it the fact that they are using a hardware based random generator or the way in which they are using a hardware generator (e.g. lava lamps)?

If the former - my question would be - surely you, as a security expert, would agree that a hardware random generator is more secure than a software based one (PseudoRandomNumberGenerator or DeterministicRandomBitGenerator)?

If the latter - why does the way in which they use a hardware generator matter as long as it provides some utility and advantage over a software-based one which has a deterministic set of numbers?

Real world RNGs get randomness from two sources: (1) the timings of random events on the machine (primarily network traffic), or (2) a hardware device that runs several oscillators on different clocks and detects coincidence in the derived square waves. Even #1 alone is normally sufficient in the real world. While you're right that there are attacks against RNGs, it's never going to be because an attacker gets control of every little bit of stray RF in your data center such that he can control the exact timing of packet arrival, packet retransmits, etc.
tptacek is right. It's a cool art installation, but for practical purposes this is utterly useless. A modern hardware RNG (which is entirely silicon based) generates random bits at a vastly higher rate, and with better reliability, like built-in continuous statistical tests, than something rigged together with lava lamps, Geiger counters or the like.

I've built those kinds of science projects, and they are great for fun and for learning, but they are not practical or necessary in any production environment.

> but they are not practical or necessary in any production environment

But do they provide any greater advantage or greater utility to your "science projects" other than being cheaper to run? What do higher rates and higher reliability have to do with being able to create a non-deterministic and thus truly random bit? In other words, how on earth would a hacker even conceivably be able to hack the lava lamp setup?

The writing by Cloudflare on the subject (hyperlinked here several times already, so I won't) indicates that the utility that Cloudflare sees in it is in militating against the sort of malicious entropy source attacks described by Bernstein. The idea is that one has a set of disparately placed entropy sources, in Cloudflare's head office (of which the lava lamps are just one) and in Cloudflare's data centre, making it hard to supplant one and simultaneously observe all of the others.

I am not sure that Cloudflare is correct about this, however. It seems to me that at the point where the entropy sources are finally mixed, on the beacon machine in the datacentre, it does not matter that the lava lamps are far away, and this factor is just window dressing. The data that they generate has to arrive at the beacon machine on a serial port, Ethernet interface, or other input device, and that is the point where it can be observed/supplanted.

See cloudflare blog post conclusion - they're quite aware it's hopefully unnecessary and possibly just a flair factor (their pun :) but may help prevent an attack ever so slightly.

"Hopefully we’ll never need LavaRand. Hopefully, the primary entropy sources used by our production machines will remain secure, and LavaRand will serve little purpose beyond adding some flair to our office. But if it turns out that we’re wrong, and that our randomness sources in production are actually flawed, then hopefully LavaRand will be our hedge, making it just a little bit harder to hack Cloudflare."

You know, that, or actually fuzzing their proxy code.
The best kind of fun is good clean wholesome fun. I love when we take a "detour" at work to do something in a fun way.
It looks quite simple and sane to me.

A big plus over the alternatives I see mentioned is that you can easily see if it's working right now.

What bothers me is that at least 80% of the wall is unrandomized wall and lampfeet.

Doesn’t matter. As a simplistic example, if you send the image through SHA-256 it doesn’t matter how many bits are predictable — as long as there are at least 256 bits of entropy in the image, the output will have those 256 bits of entropy smeared across the output evenly (barring breaks in the hash function itself).
Unfortunately they're not cool, a single lava lamp puts off a lot of heat, I can't imagine the heat generated from a wall of them. Seems like you could get similar randomness from a wall of betta fish and have a lower electricity bill.
There are plenty of simpler sources of high quality randomness. The important bit is whitening. Virtually all measurements of anything are not as precise as the number of bits used to represent the sample.

Virtually any sensor measuring real world data will be imprecise in the last 1-4 bits. Take 256 raw samples from virtually any real world sensor, run the raw data through SHA-256, voila, you have 256 random bits that's secure against the overwhelming majority of hackers. Take 256,000 samples, run the raw data through SHA-256, now you have 256 random bits that's secure against the NSA.

Cameras are great for this, but first you have to ensure that you're getting the raw data and not an image processed with HDR or jpeg compression or whatever special sauce Google/Apple do these days, and you also have to test to ensure that the image isn't overexposed. If it's an 8 bit sensor and all pixels are pegged at 255 you don't have useful randomness. Although the opposite isn't true: it's impossible to peg all pixels at 0. You'll always get values of 1-10 or so with the same pixels fluctuating +/- 1 or so, even in a perfect darkroom with perfect insulation from cosmic rays and background radiation.

The average smartphone is packed full of gadgets that could be used to generate random data after running it through whitening. Accelerometers, magnetic compass, capacitance of each pixel of the touch sensor, all the radio antennas pulling atmospheric noise, (CDMA, GSM, GPS, bluetooth ...) microphone, camera...

The lava lamps themselves might make the process more efficient, because now some of the higher order bits are random as well. But honestly, from a realistic security standpoint... they're just as well off taking pictures of a blank, uniformly lit white wall. As long as they do whitening properly.

edit: It's still super cool though. Just not... in terms of temperature. It's probably really hot.

Do they have the distribution posted somewhere?
The issue with this is the nature of devices (cameras) and device drivers - which both have non random characteristics.
Assuming this is not a PR stunt: Wouldn't different lighting conditions throughout the day lead to patterns in the randomness?
Was my thought too, you're introducing a number of unknown variables that could skew the distribution.
Another pattern is the bases of the lamps rarely change. Patterns are OK; the important thing is that unpredictable components are present.
We did this exact thing at SGI 20 years ago. https://en.wikipedia.org/wiki/Lavarand

I wonder if Cloudflare was inspired by that.

Yes, they were; they mentioned it explicitly.
The article doesn’t actually mention SGI.
No, just the name of SGI's lava lamp random number generator, referenced as prior art.
It’s tptacek. He’s probably spoken with Cloudflare directly or consulted for them.
(1) No.

(2) No love lost there.

(3) That is a weird comment to write.

The article explains how it works, without any mention of the inspiration.
This article. Others make the association clear.
IRIX was nothing if not very pretty. Thanks for XFS :)
> We're not the first ones to do this. Our LavaRand system was inspired by a similar system first proposed and built by Silicon Graphics and patented in 1996 (the patent has since expired).

-- https://blog.cloudflare.com/randomness-101-lavarand-in-produ...

I know that people patent things for various reasons but would people truly expect the holder of that patent to litigate against another company using that method to ensure "more true" randomness in whatever application they were using? I, personally, imagine the patent holder would be glad to see others using their method to ensure true randomness so long as they were using the technology for anything other than evil.
I'm not familiar enough with the management of Silicon Graphics to say if they specifically would be likely to litigate, but I doubt CloudFlare would take the risk on an unexpired patent - management changes, and plenty of companies enforce patents that could do much more good than strengthening cryptography in the hunt for profit (see: the medical industry).
What's special about sources of randomness? Would you have the same intuition about a patented process for creating physical objects?
In today's world, yes, I would expect a patent holder to sue.
It’s a useful technology that they spent the time and money to patent. There’s no reason to assume anything about the intent of the patent holders.
Well, I’d expect the patent-holder to want to make money.
Before SGI, there was the Aquari-rand, but I can't find a link for it. Basically an Aquarium with a digital camera pointed at it..
This is pretty cool use of tech that goes back to SGI. It's definitely not the practical solution to TRNG's. There are analog solutions that use basic physics and EE techniques to generate noise fast, cheap, in tiny footprint if you want, and with a lot of potential diversity in supply chain. Here's an example of an open one:

https://github.com/waywardgeek/infnoise/blob/master/README.m...

If you can't afford that many lava lamps, NIST provides an alternative, free service:

https://beacon.nist.gov/home

WARNING: DO NOT USE BEACON GENERATED VALUES AS SECRET CRYPTOGRAPHIC KEYS.

So probably not a good idea.

Using the values directly is not the same thing as feeding the values into a RNG.
No, the reason you aren't supposed to use it to generate cryptographic keys is because it's public: so it effectively provides no (or nearly no) entropy. It's the same reason you shouldn't use the current time as a seed for a PRNG.

NIST Beacon is more intended for things like lottery drawings, where you want to prove that you're generating the random numbers in an unbiased manner.

For a few seconds I was thinking...Bacon.
My lesson from my personal experiments with Lavarand: you must have more than one lamp, not necessarily for more entropy, but for fail-over and uptime. At approximately ~30 hours, my vintage 70's lamp 'gives up' - the fluid temperature becomes pretty even between the bottom and the top. It's all essentially superheated as far as the wax is concerned and it simply stays in one place as a dome at the bottom, barely moving. This isn't good for creating random data. By using multiple lamps, it's possible to power cycle them. Ideally, every ten hours or so, remaining off for a couple hours.
(comment deleted)
Would attaching a heat sink to the top of the lamp solve that problem? So there's always a heat gradient.

It would probably work better in a colder room, too.

Put a TEC on the top with a heatsink if your room is warm.
Would a heat sink on top help, perhaps?
It's worth nothing that the lava lamps are visible street side through large glass windows, so it serves as an eye-catching artpiece for passerbys.
Seems like a waste of energy... I can imagine putting a weather station on the roof would be more useful (albeit less cool). Use multiple sensors for rainfall, UV, wind speed, wind direction, temperature, pressure and aggregate the signals from each... Surely the combination of localized weather readings would provide enough randomness.
It's ~100 lamps for entropy that covers ~10% of the internet, energy consumption is a ridiculous complaint.
This isn't how entropy works. They have more than enough "entropy" for "10% of the Internet" without the lava lamps.
So how many bits does it produce per second per lamp?
That's approximately 100 lava lamps each at 100W = 10,000W = 10kW/h * 20.4 cents = $2.04/h * 24 = $48.96/day $1,489.20/month.

Ignoring the fact that is very little money in Silicon Valley. Lava lamps consume a large amount of electricity in order to generate the heat they need. There are cheaper better ways to generate randomness, this is purely for spectacle clearly.

It makes me nervous more than anything. If that's the front they put up, inside is there a Rube Goldberg machine that triggers DDoS protection?

They should "power" the lamps with excess CPU/processor heat.
Not too much power though. They can explode.
IIRC a standard size lava lamp used much less than a 100W bulb. So your costs are maybe half that.
40W on mine fwiw
As someone else pointed out, they overheat. So if you want to run them all day you might go down to a 30 or 25 watts.

But there are probably a bunch of other Brownian motion machines you could use that take a lot less power. Like those glitter lamps. You’d need a higher res camera.

Infinite improbability is just around the corner, eh? (-:
I think the intent is for this to be a functional art installation more than anything. Of course there are less expensive (and more "practical" ways of accumulating entropy), but this is a very cool art project that provides a pretty core function. Yeah, maybe it's kind of expensive to run, but that's not that extreme. A lightbulb is like 100W too right?
100 watt is way too much, 25W/each is a more realistic number.
If you want to go with lamps or light as the source of entropy and still have a nice wall to show people, plasma globes are probably a better way of doing it.