Curious how exactly this generates truly random numbers (and didn't see an explanation in the link). Can anyone explain this in technically-inclined laymen's terms?
fwiw, I'm a programmer but don't know anything about random number generation (other than I'm not supposed to rely on the built-in random functions of most languages/libraries for security purposes because they're not truly random).
We think that cosmic background radiation and certain subatomic mechanics are truly random. If you attach extra hardware to a computer, you can observe these and send a stream of bits to the computer.
You don't usually want to use truly random numbers for cryptography though; /dev/random is more appropriate. I've never understood why, exactly, but that's generally the advice.
You can use "truly" random numbers for cryptography. /dev/random is not recommended (except for perhaps long-term keys if you're worried about the randomness algorithms having a vulnerability), use /dev/urandom.
There is no difference between /dev/random and /dev/urandom in the Linux kernel, other than /dev/random blocks when the entropy estimate is low or depleted. Both source them same CSPRNG, which is based on SHA-1. /dev/urandom is absolutely recommended for long-term keys.
On most systems, urandom and random are the exact same thing. Only Linux does this weird "blocking random" thing, based on some weird estimate of "running out of entropy" which is not an actual thing.
(u)random is powered by a CSPRNG which is seeded by "truly random numbers" gathered from multiple hardware sources.
Cosmic background radiation as observed by a sensor connected to the computer is not random. All I'd need to mess with the randomness is to put a focused radiation source near the device. Sampling randomness from the environment surrounding the physical installation of the device is not good. I hope no one is doing this.
There's no reason why you wouldn't want to use truly random numbers for cryptography. That's what /dev/random is supposed to provide.
They tend to generate electrical noise (often from a semiconductor), amplify that noise, sample it, de-skew the stream, and present that to a port for reading.
The key components are the two transistors connected back to back. The first (Q1, pins 1, 2 and 6) is effectively used as a zener diode which will exhibit breakdown noise at low currents [1]. The second transistor (also labelled Q1 but on pins 3, 5 and 4) is used as an amplifier of said noise.
These tokens, like the OneRNG, are quite interesting from an engineering point of view, but as a developer all you ever need to know or do about secure randomness is to use /dev/urandom and nothing else (unless it's a wrapper for /dev/urandom).
Not /dev/random, not rand(), not something you found in a library. Just bytes, read from /dev/urandom.
RtlGenRandom[1] is probably the most convenient way on Windows. Does require certain definitions when including the header file for it[2] but otherwise does not require a CSP context and can generate any number of CSPRNG bytes.
You're right. Calling RtlGenRandom() directly is probably the best option. I forgot that rand_s() calls RtlGenRandom(), not CryptGenRandom(). Plus both Firefox and Chrome ran into rare crashes when some bad third-party software (antivirus or malware) injects advapi32.dll hooks, causing rand_s() to crash when it tries to load advapi32.dll in order to call RtlGenRandom().
You can also specify the number of bytes you want with RtlGenRandom. rand_s only gives an unsigned int per call. Not a huge deal but you may be able to cut down on some function call overhead depending on how stuff gets inlined/optimized.
CryptGenRandom[1] or RtlGenRandom[2]. Although CryptGenRandom is the offical version they probably would rather you use I'm fairly certain that most people are more likely to use RtlGenRandom because it's accessing the same thing but doesn't require a CSP context in order to use it (you can just call it straight up).
You have to include ntsecapi.h with some fixing definitions[3] to use RtlGenRandom.
High-assurance security products (eg Type 1 NSA stuff) have always relied on verifiable or dedicated RNG's. Typically TRNG's. Here's some reasons:
1. Easier to trust one based on physics with decades of verification of its behavior vs mathematical constructions with vastly less certainty or verification of correctness.
2. Bit-flips or EMI on internal state might affect a software algorithm's randomness. Won't happen with certain types of analog noise with carefully tuned and measured circuits.
3. Can verify the circuits do exactly what I expect by eye and hand with a logic analyzer. No speculation due to No 1. Can then trust that circuit as a pluggable component in any other system without further evaluation.
4. Building on above, a compromise of the host PC requires me to recover the host but not dedicated chips like the TRNG or proper HSM. I can still trust them if they have no firmware (eg analog TRNG) or it's not write-able.
5. Using Nizza Architecture, I can put Linux in an untrusted partition with crypto/CRNG/secure-storage directly on microkernel in dedicated, isolated partition. TRNG HW or dedicated crypto SOC gives me this plus benefits above.
So, the difference between high-assurance setup with HW support for security-critical functions and software on an untrustworthy box... probably x86 with its vPro backdoor circuitry... is pretty astounding in capabilities. Seeing the differences & how cheap you can do that stuff should make anyone wonder why it's not standard in security-focused projects.
I've never worked on state-level appliances, so thank you for the information, but just for the sake of stopping urandom-fear, let me spell out why this all does not apply to the 99% of us developing for consumer processor architectures (which I'm sure you realize):
- the "uncertain mathematical constructions" are hashes. More precisely, in Linux case, SHA-1 preimage (not collision) security. No civilian cryptosystem would survive the fall of hashes.
- all other points are void as soon as you use that randomness from a regular application running on regular CPUs, as all the bit-flips, compromises and backdoors can simply happen there.
- having the keys on hardware DOES make compromise recovery much easier, but there we are speaking of HSM capable of doing the private key operations themselves, not simply providing randomness.
"the "uncertain mathematical constructions" are hashes. More precisely, in Linux case, SHA-1 preimage (not collision) security. No civilian cryptosystem would survive the fall of hashes."
The fall of a RNG by biasing its bits is more likely than a full, second-preimage attack on a hash. So, a failure of hash function in one app != failure in all apps. You're certainly right that it would have bigger implications but might still be useful as MD5 is if collisions are out of scope.
"all other points are void as soon as you use that randomness from a regular application running on regular CPUs, as all the bit-flips, compromises and backdoors can simply happen there."
There's a lot of CPU's to choose from to reduce that risk. There's systems like Gaisler's Leon3-FT that are open-source, have resistance to bitflips, support mods like TRNG's, and run Linux. At this point, they have to subvert your ASIC process, mask manufacturing, or supply chain. Harder than a purchasing a Linux 0-day if target has decent OPSEC and avoids spy-friendly jurisdictions. ;)
If it's on Intel/AMD/IBM, your comment is true in most cases.
"but there we are speaking of HSM capable of doing the private key operations themselves, not simply providing randomness."
I was speaking of both. One problem I have is bootstrapping randomness. I do it by hand with cards whose entropy feeds into a CRNG unless I have a trustworthy TRNG. A plug-in, analog source with dumb, interface chip isn't going to be compromise by hackers... probably. Likewise, I can go further with a simple, coprocessor w/ ROM, memory protection, dumb interface, and onboard TRNG. Many available from many vendors of varying trustworthiness. Still safer than base system w/ kernel RNG and can even monitor (or restore) host system if I choose.
"let me spell out why this all does not apply to the 99% of us developing for consumer processor architectures"
I still agree with that, though. Main reason is nobody in 99% is going to use anything high-security because it will cost more, be inconvenient, and lack feature/app X, Y, or Z. Always was the case. :)
Why don't you drive a different randomised route to and from work every day and keep a log of cars you keep recognising in the rear view mirror, to detect government/foreign intelligence spying on you?
Or: why doesn't your car have an FIA compliant roll cage, a fire extinguishing system and six point harnesses?
Same answer: for you (and 99.9% of us), the costs outweigh the benefits.
The cost was nearly free for the basic protections. High-assurance design or components applied in otherwise commercial. We call this medium assurance but it's quite effective in practice. Even high-assurance projects like LOCK program said A1-class requirements only added around 30-40% to the cost of otherwise solid, development process. That's not bad for software that should rarely fail, get hacked, or be unrecoverable. I'd pay $150 for Windows instead of $100 or so. :)
Meanwhile, methods like Cleanroom and Eiffel showed great reductions in defects and problems while around same cost and time to market due to reduced debugging. Sometimes faster or cheaper slightly. Sometimes slightly more cost or time. Not much of a difference and low technical debt meant it got cheaper to improve over time.
So, your argument might apply if we're talking keeping legacy software at full speed on cheapest hardware with zero modifications. That really puts high-assurance in a box. Yet, if you're talking dramatically improving assurance of isolation or recovery at the least, then you can do it for a significant, but not heavy, additional cost.
And remember much of those exorbitant costs came from straight-up inventing the stuff on the spot. Proven examples to draw from in the literature, including full designs, should drop that many fold. GEMSOS was $15 million over 2-5 years. seL4 was about $5 million over 2-4. A handful of academics put together Muen in SPARK pretty quickly so maybe a few hundred K to a million there. See how the costs drop as we do similar jobs with improved tools and prior examples?
That's hard to say. I need to re-research it and write another essay on it using info I've learned recently. Here's a few issues:
- High-assurance is a barely developed field due to neglect. This means each new problem is essentially a R&D problem then an engineering problem. That increases cost and risk while slowing things down a bit. Good news is there's examples of many kinds of things to work with out of both proprietary and CompSci (mainly) with almost nothing out of FOSS despite them being attracted to interesting, challenging problems. Weird but that's the spread.
- High-assurance applies rigorous techniques to systematically eliminate problems in requirements, design, code, deployment and maintenance. The extra care increases both upfront investment and time-to-market. The latter is often critical as whoever has more features at any given point gets more market share and profit. That high-assurance delays to get it right the first time is an unforgivable sin in capitalist market.
- A follow-up here, identified by a founder of INFOSEC (Schell), is that businesses make more money on broken software. They get market share by not charging too much plus adding features. So, that means they need to find minimal amount of features to add at minimal labor cost. They keep customers with lock-in, which neglects quality automatically. Common to charge for updates and fixes delivered regularly in a way that's good for financial reports. Schell reported that non-IT industry he met with was aware of the game early where they said IT industry knew about his and other methods for robust software, that they worked, and refused to apply them to continously charge for broken software. A true conspiracy that became the default as high-assurance was forgotten, everyone grew up with shit software, and everyone [wrongly] believes it's inevitable rather than intentional.
- Demand is a huge problem. Most people blame software makers... which admittedly conspire... yet users will rarely buy secure stuff or even higher quality stuff. Any HN reader knows how consumers do economics and most businesses do IT. If they don't give a shit, then the supply side shouldn't per capitalism. They want insecure OS's running with no safety checks in CPU supporting insecure peripherals/apps and nearly-backdoored wireless standards? Well, you better provide it or you make no money. FOSS has similar problem for popularity or uptake. There's a niche that does higher-security stuff, mostly in defense but not all. Yet, higher cost plus abysmally low volume = very high unit prices or OEM licenses. It's like a trap. Worst, another conspiracy Schell's industry people noticed early on was mainstream "security" vendors buy up high-assurance vendors, then eliminate or water down their offerings. Might be incidental or nefarious but it's a real effect.
- Walker's Computer Security Initiative, along with Schell's work, countered all of this to invent INFOSEC field, create standards, incentivize them, improve mainstream, and bring high-assurance market up. Bell, of Bell-LaPadula, describes in a paper that NSA killed it by competing with them directly w/ government solutions and reneging on all promises of longevity & pay. Post-Snowden, a dumb move or intentional sabotage? Still not sure but NSA is real obstacle. EAL6/7 products still considered munitions for export although maybe not enforced. Government keeps steady development of high assurance (Type 1 especially) for use by military and defense contractors but bans us from using it. They're steady obstacle, esp NSA and DOD but DARPA & NSF a helpful neutrals.
- Worst one is ignorance and apathy. You'll see me be harsh on security industry here for a reason: they don't know any of this shit or even what high-assurance is mostly. It's like CompSci, high-assurance, defense, mainstream INFOSEC, and common IT are all silo'd from each other with little knowledg...
A paranoid civilian could use a dedicated random number generator like this device to implement one time pad encryption, which doesn't depend on the unproven security of one way functions or a complicated codebase inside the linux kernel.
See Markus Ottela's Tinfoil Chat. He incorporated many suggestions from several of us in high-assurance security. Only guy doing FOSS to do that IIRC.
It's Python since he's not a system programmer. A port to a safe subset of C or SPARK Ada on stripped BSD or microkernel is necessary before I'd start trusting it.
Well, you do need to seed /dev/urandom with sufficient entropy. Hardware random number generators work here. After it's seeded, my only concern would be backtracking resistence, but haveged(8) works well here.
Yes, but no. The kernel and the OS will take care of seeding for you, and haveged can't create out of thin air any unpredictable events that are somehow invisible to the kernel.
Anyway, Linux urandom is backtracking safe as long as SHA-1 preimage security holds.
Well, not all Linux hardware provides sufficient entropy on first boot, such as solid-state embedded hardware or virtual machines. The CSPRNG will happily provide random bits regardless of initial seeding or not. Flip a coin 512 times, record the results, and "echo tththhhhthh... > /dev/random". Most people don't want to do this, so hardware RNGs simplify this.
If only the Linux kernel used as entropy everything that was available to it. That's the pitch of haveged(8) I think. There are significantly more timing events the kernel could use for input entropy that it's not. It would be great to see these timing events land in random.c, but they're not there.
Lastly, slightly off-topic, but the Linux CSPRNG is poorly designed. In reality, while preimage SHA-1 might still have a sufficient security margin, it suffers from poor performance due to ad-hoc design. Instead, using AES-128 CTR_DRBG with AES-NI would give the kernel CSPRNG a much needed performance boost, provide backtracking resistence, and remain secure. Current testing shows it could improve from 20 MiBps to 2 GiBps.
Or the getrandom() syscall, once it has trickled into all mainstream distributions' libcs. Which is basically /dev/random, but without the need to open() and close() the device first and after you've consumed your entropy.
If you're going to have a blackbox (the SOC), you might as well get a BitBabbler as they have better design for noise and the black box is dumbest USB chip they could find.
Needs more peer review and independent testing anyway. It's designed so you can do it yourself with a logic analyzer or whatever.
EDIT: I should add that, quite interesting to me, BitBabbler folks are building this for people who already know why a TRNG or analog source is important. They're not interested in evangelizing or winning anyone over: just people already wanting one buying it and preferably analyzing it to give them feedback on esoteric failure modes they haven't thought of. Only contact them if you're serious.
They were in the $100-200 range IIRC. Funny thing about your comment on design:
"If you want to know more about the company where this all started, the Voicetronix website can tell you all about that, in all its only slightly updated since the mid 90's glory. We're engineers who build great hardware and provide exceptional support for it ... Web 2.0 bling isn't quite our favourite thing to spend time on :) " (source: bottom of Buy page)
Haha. Anyway, I tried to find price but couldn't. If you want one, just email them at the address in BitBabbler Buy tab for a price quote. That's how they typically work. Here's the company if you're curious:
Their differentiator, per their lead engineer, was quality hardware with free as in speech software and custom service/support. I found that interesting esp given it's a 90's era business that predates the OSS HW fads. A model worth copying perhaps.
Note: I haven't hands on reviewed a BitBabbler or their other products. Not endorsing them. Just like what I saw and heard vs what I usually see and hear. Suggesting others review, test, or use the products to see if they live up.
Has anyone done any cryptographically analysis with these things?
I'd love to get my hands on one of these and do some statistics on a lot of data pulled off of these.
To be honest, I think that at certain controlled temperatures, I'd bet you could start to weight numbers for prediction if you are trying to break crypto keys generated by these.
If those kind of devices interest you, I'm quite intrigued by the IDQuantique ones that are based on quantum physics to generate randomness. They have a USB product that has a random stream of 4Mbits/sec and is used by lotteries and gambling sites.
Intel has a patent on a TRNG that has a throughput of 2 Gbps+. They published a paper on its operation in JSSC a while back [1]. It passes all NIST RNG tests and is PVT tolerant. Too bad it's patented...
Noise-source based random number generators can be vulnerable to interference. If there's a radio transmitter nearby, the RF may add an input to the noise. An RF source with a strong carrier will add a periodic value. Not good.
Radiation sources really are random. You get a nice strong signal from each decay sensed, and the time between those events is random.
Anything which depends on high amplification of an analog signal has potential interference problems. You need a physical phenomenon which produces discrete events, or good shielding. It's not hard to shield against RF and magnetic interference, but it takes some metal and size, which can be a problem for a USB connector sized device. You also have to filter the power going into the shielded area.
It's useful to take the output of the noise generator and run it into a spectrum analyzer while using electric motors and walkie-talkies close to the device. If the noise spectrum change shape when you do that, it's not random.
Why aren't 800 honest-to-goodness random bits per second enough for all practical purposes? If you need more than that, you should be playing in the big leagues, where using a little USB fob is not the answer!
Why aren't 800 honest-to-goodness random bits per second enough for all practical purposes?
For seeding the system CSPRNG, that's fine. For securely destroying data, such as wiping hard drives, or creating underlying noise for encrypted filesystems, 800 Bps is going to take you years to work through a 4 TB hard drive.
800 Bps is going to take you years to work through a 4 TB hard drive
But you don't need to do that. You simply take 256 bits and use them as an AES key. You then generate a stream of bits and write that to disk. Reading the disk afterward is equivalent to breaking AES. Or take Bernstein's advice and use an even better stream cipher. https://cr.yp.to/streamciphers/why.html
And even if you re-key every 1 GB (just for grins), you're still hardly using any random bits at all.
IDQuantique sells a product that purportedly generates millions of random bits per second. If I were gambling at a casino, I'd feel more comfortable with that than with randomness from an AES stream.
Yeah, I know I'm thinking irrationally. But I'm sure I'm not the only one.
But you don't need to do that. You simply take 256 bits and use them as an AES key. You then generate a stream of bits and write that to disk. Reading the disk afterward is equivalent to breaking AES.
Completely agreed. I've already mentioned in this post about only using the Linux CSPRNG to seed a userspace CSPRNG, because the Linux CSPRNG is horrendously slow, due to its ad-hoc design. However, if the Linux CSPRNG was AES-128 using the standard CTR_DRBG, with AES-NI, it could go north of 2 GiBps without breaking a sweat, keeping clean random.c code (it already ships aes.h), and it's backtracking resistant.
However, if you're going to spend good money for hardware producing random values, at least for me, I'm going to want the best bang for my buck. If you look at https://en.wikipedia.org/wiki/Comparison_of_hardware_random_..., you can spend $35 for the WaywardGeek "Infinite Noise RNG", which outputs 300 Kbps, or spend $24 for an RTL-SDR dongle and get about 2.8 MiBps. The value of bits per dollar is much higher with the RTL-SDR than the WaywardGeek. This is considerably cheaper than getting a radioactive sample, coupled with a Geiger counter.
IDQuantique sells a product that purportedly generates millions of random bits per second. If I were gambling at a casino, I'd feel more comfortable with that than with randomness from an AES stream. Yeah, I know I'm thinking irrationally.
Yeah, that is irrational. The problem with most TRNGs is that their output is biased. As a casino operator, I might be interested if the bias benefits my casino, but I would also need to prove to regulators that I'm not cheating the public. Personally, I'd err on the side of caution when it comes to the law.
So either I would whiten the output with a cryptographic primitive (John von Neumann debiasing is too costly, losing at least 1/2 the bits), or just write a userspace CSPRNG, and get the throughput you'll need for the demand. I could then constantly rekey/rseed the generator with the TRNG if I was tinfoil-hat-paranoid.
In either case, TRNG or CSPRNG, you'll want exceptional throughput for your casino. Which brings us back to another example of which 800 bps just isn't going to cut it. :)
Is there any reason we don't have hardware RNG in modern CPUs/MB? It sounds like an obvious feature to have as a standard, when we think of the efforts required to create a decent software pseudo RNG, and the fact that crypto is absolutely everywhere.
>> Relying solely on the hardware random number generator which is using an implementation sealed inside a chip which is impossible to audit is a BAD idea.
I don't think so. In the terms of "random that does not follow a known deterministic algorithm or formula", both chaotic sources like lightning strikes, and quantum states, like photon spin, can be so random, that for all definitions of "truly random", they fit the bill.
I noticed that this hasn't been mentioned yet, so I'll bring it up here. While RF-based noise generators can be influenced by outside sources, using an RTL-SDR with https://github.com/pwarren/rtl-entropy can be a great way to get environmental RF noise.
In actual reality, the generator won't be influenced, because it will require a strong signal, close by, likely transmitting illegally on the frequency you are listening to (of course, the attacker would need to know the frequency you're tuned to). Simple monitoring tools and scripts can be used to test the quality of the generator output. If the quality of the randomness begins to fail, as would happen with deliberate RF interference, then the script would stop outputting bits, and alert the administrator. Or, even better, switch to a different frequency, chosen at random by the kernel CSPRNG.
What's great about using the RTL-SDR as an RNG, is these devices are cheap (about $20) and can output 3 MiBps, which vastly out performs any USB HWRNG that I have personally tested for the value (bits per second per dollar).
FWIW, random.org uses RF noise as the basis for delivering random numbers to the Internet.
Would have been nice if they posted some of the output of the RNG's as well as the results from some tests on the randomness of the output like Die hard[1]:
well, it makes a lot of man in the middle : the HW USB susbsystem, the USB driver, the OS, the driver or the electronic doing the interface. Plus it makes a lot of electromagnetic vulnerabilities (reading/influencing with a magnetic field)
A purely paranoid person would really prefer something with an antenna (to get some noise by detection), some RLC, a defective/chaotic oscillator on the dye of an a SOC that makes it embedded and you can verify by looking with a microscope that you shield in a Faraday cage.
And no, deciphering logic of a microchip with a microscope is not that hard ; almost anyone can do it with a tad of training and some education.
74 comments
[ 2.9 ms ] story [ 136 ms ] threadfwiw, I'm a programmer but don't know anything about random number generation (other than I'm not supposed to rely on the built-in random functions of most languages/libraries for security purposes because they're not truly random).
You don't usually want to use truly random numbers for cryptography though; /dev/random is more appropriate. I've never understood why, exactly, but that's generally the advice.
On most systems, urandom and random are the exact same thing. Only Linux does this weird "blocking random" thing, based on some weird estimate of "running out of entropy" which is not an actual thing.
(u)random is powered by a CSPRNG which is seeded by "truly random numbers" gathered from multiple hardware sources.
There's no reason why you wouldn't want to use truly random numbers for cryptography. That's what /dev/random is supposed to provide.
http://www.2uo.de/myths-about-urandom/
Here's some information I had from a while ago. (Not all of this is about hardware RNGs) https://news.ycombinator.com/item?id=6060636
https://en.wikipedia.org/wiki/Noise_generator
http://holdenc.altervista.org/avalanche/
http://freenrg.info/Physics/Scalar_Vector_Pot_And_Rick_Ander...
A similar circuit is explained in more technical detail here: http://www.nutsvolts.com/magazine/article/bipolar_transistor... (look for the "WHITE NOISE GENERATORS" chapter title)
[1]: https://en.wikipedia.org/wiki/Noise_generator#Zener_diode
https://www.schneier.com/blog/archives/2013/10/insecurities_...
Not /dev/random, not rand(), not something you found in a library. Just bytes, read from /dev/urandom.
Here is my CCC talk about why that is, which includes everything from kernel sources to authoritative quotes: https://media.ccc.de/v/32c3-7441-the_plain_simple_reality_of...
EDIT: the talk is in English, but you have to select it from the video UI, in the bottom right.
[1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...
https://msdn.microsoft.com/en-us/library/sxtz2fa8.aspx
[1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...
[2] https://boringssl.googlesource.com/boringssl/+/master/crypto...
Firefox bugs: https://bugzil.la/1240589, https://bugzil.la/1167248, https://bugzil.la/694344
Chrome bug: https://crbug.com/348400
You have to include ntsecapi.h with some fixing definitions[3] to use RtlGenRandom.
[1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...
[2] https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...
[3] https://boringssl.googlesource.com/boringssl/+/master/crypto...
http://ubld.it/products/truerng-hardware-random-number-gener...
1. Easier to trust one based on physics with decades of verification of its behavior vs mathematical constructions with vastly less certainty or verification of correctness.
2. Bit-flips or EMI on internal state might affect a software algorithm's randomness. Won't happen with certain types of analog noise with carefully tuned and measured circuits.
3. Can verify the circuits do exactly what I expect by eye and hand with a logic analyzer. No speculation due to No 1. Can then trust that circuit as a pluggable component in any other system without further evaluation.
4. Building on above, a compromise of the host PC requires me to recover the host but not dedicated chips like the TRNG or proper HSM. I can still trust them if they have no firmware (eg analog TRNG) or it's not write-able.
5. Using Nizza Architecture, I can put Linux in an untrusted partition with crypto/CRNG/secure-storage directly on microkernel in dedicated, isolated partition. TRNG HW or dedicated crypto SOC gives me this plus benefits above.
So, the difference between high-assurance setup with HW support for security-critical functions and software on an untrustworthy box... probably x86 with its vPro backdoor circuitry... is pretty astounding in capabilities. Seeing the differences & how cheap you can do that stuff should make anyone wonder why it's not standard in security-focused projects.
- the "uncertain mathematical constructions" are hashes. More precisely, in Linux case, SHA-1 preimage (not collision) security. No civilian cryptosystem would survive the fall of hashes.
- all other points are void as soon as you use that randomness from a regular application running on regular CPUs, as all the bit-flips, compromises and backdoors can simply happen there.
- having the keys on hardware DOES make compromise recovery much easier, but there we are speaking of HSM capable of doing the private key operations themselves, not simply providing randomness.
The fall of a RNG by biasing its bits is more likely than a full, second-preimage attack on a hash. So, a failure of hash function in one app != failure in all apps. You're certainly right that it would have bigger implications but might still be useful as MD5 is if collisions are out of scope.
"all other points are void as soon as you use that randomness from a regular application running on regular CPUs, as all the bit-flips, compromises and backdoors can simply happen there."
There's a lot of CPU's to choose from to reduce that risk. There's systems like Gaisler's Leon3-FT that are open-source, have resistance to bitflips, support mods like TRNG's, and run Linux. At this point, they have to subvert your ASIC process, mask manufacturing, or supply chain. Harder than a purchasing a Linux 0-day if target has decent OPSEC and avoids spy-friendly jurisdictions. ;)
If it's on Intel/AMD/IBM, your comment is true in most cases.
"but there we are speaking of HSM capable of doing the private key operations themselves, not simply providing randomness."
I was speaking of both. One problem I have is bootstrapping randomness. I do it by hand with cards whose entropy feeds into a CRNG unless I have a trustworthy TRNG. A plug-in, analog source with dumb, interface chip isn't going to be compromise by hackers... probably. Likewise, I can go further with a simple, coprocessor w/ ROM, memory protection, dumb interface, and onboard TRNG. Many available from many vendors of varying trustworthiness. Still safer than base system w/ kernel RNG and can even monitor (or restore) host system if I choose.
"let me spell out why this all does not apply to the 99% of us developing for consumer processor architectures"
I still agree with that, though. Main reason is nobody in 99% is going to use anything high-security because it will cost more, be inconvenient, and lack feature/app X, Y, or Z. Always was the case. :)
Or: why doesn't your car have an FIA compliant roll cage, a fire extinguishing system and six point harnesses?
Same answer: for you (and 99.9% of us), the costs outweigh the benefits.
Meanwhile, methods like Cleanroom and Eiffel showed great reductions in defects and problems while around same cost and time to market due to reduced debugging. Sometimes faster or cheaper slightly. Sometimes slightly more cost or time. Not much of a difference and low technical debt meant it got cheaper to improve over time.
So, your argument might apply if we're talking keeping legacy software at full speed on cheapest hardware with zero modifications. That really puts high-assurance in a box. Yet, if you're talking dramatically improving assurance of isolation or recovery at the least, then you can do it for a significant, but not heavy, additional cost.
And remember much of those exorbitant costs came from straight-up inventing the stuff on the spot. Proven examples to draw from in the literature, including full designs, should drop that many fold. GEMSOS was $15 million over 2-5 years. seL4 was about $5 million over 2-4. A handful of academics put together Muen in SPARK pretty quickly so maybe a few hundred K to a million there. See how the costs drop as we do similar jobs with improved tools and prior examples?
- High-assurance is a barely developed field due to neglect. This means each new problem is essentially a R&D problem then an engineering problem. That increases cost and risk while slowing things down a bit. Good news is there's examples of many kinds of things to work with out of both proprietary and CompSci (mainly) with almost nothing out of FOSS despite them being attracted to interesting, challenging problems. Weird but that's the spread.
- High-assurance applies rigorous techniques to systematically eliminate problems in requirements, design, code, deployment and maintenance. The extra care increases both upfront investment and time-to-market. The latter is often critical as whoever has more features at any given point gets more market share and profit. That high-assurance delays to get it right the first time is an unforgivable sin in capitalist market.
- A follow-up here, identified by a founder of INFOSEC (Schell), is that businesses make more money on broken software. They get market share by not charging too much plus adding features. So, that means they need to find minimal amount of features to add at minimal labor cost. They keep customers with lock-in, which neglects quality automatically. Common to charge for updates and fixes delivered regularly in a way that's good for financial reports. Schell reported that non-IT industry he met with was aware of the game early where they said IT industry knew about his and other methods for robust software, that they worked, and refused to apply them to continously charge for broken software. A true conspiracy that became the default as high-assurance was forgotten, everyone grew up with shit software, and everyone [wrongly] believes it's inevitable rather than intentional.
- Demand is a huge problem. Most people blame software makers... which admittedly conspire... yet users will rarely buy secure stuff or even higher quality stuff. Any HN reader knows how consumers do economics and most businesses do IT. If they don't give a shit, then the supply side shouldn't per capitalism. They want insecure OS's running with no safety checks in CPU supporting insecure peripherals/apps and nearly-backdoored wireless standards? Well, you better provide it or you make no money. FOSS has similar problem for popularity or uptake. There's a niche that does higher-security stuff, mostly in defense but not all. Yet, higher cost plus abysmally low volume = very high unit prices or OEM licenses. It's like a trap. Worst, another conspiracy Schell's industry people noticed early on was mainstream "security" vendors buy up high-assurance vendors, then eliminate or water down their offerings. Might be incidental or nefarious but it's a real effect.
- Walker's Computer Security Initiative, along with Schell's work, countered all of this to invent INFOSEC field, create standards, incentivize them, improve mainstream, and bring high-assurance market up. Bell, of Bell-LaPadula, describes in a paper that NSA killed it by competing with them directly w/ government solutions and reneging on all promises of longevity & pay. Post-Snowden, a dumb move or intentional sabotage? Still not sure but NSA is real obstacle. EAL6/7 products still considered munitions for export although maybe not enforced. Government keeps steady development of high assurance (Type 1 especially) for use by military and defense contractors but bans us from using it. They're steady obstacle, esp NSA and DOD but DARPA & NSF a helpful neutrals.
- Worst one is ignorance and apathy. You'll see me be harsh on security industry here for a reason: they don't know any of this shit or even what high-assurance is mostly. It's like CompSci, high-assurance, defense, mainstream INFOSEC, and common IT are all silo'd from each other with little knowledg...
It's Python since he's not a system programmer. A port to a safe subset of C or SPARK Ada on stripped BSD or microkernel is necessary before I'd start trusting it.
Anyway, Linux urandom is backtracking safe as long as SHA-1 preimage security holds.
If only the Linux kernel used as entropy everything that was available to it. That's the pitch of haveged(8) I think. There are significantly more timing events the kernel could use for input entropy that it's not. It would be great to see these timing events land in random.c, but they're not there.
Lastly, slightly off-topic, but the Linux CSPRNG is poorly designed. In reality, while preimage SHA-1 might still have a sufficient security margin, it suffers from poor performance due to ad-hoc design. Instead, using AES-128 CTR_DRBG with AES-NI would give the kernel CSPRNG a much needed performance boost, provide backtracking resistence, and remain secure. Current testing shows it could improve from 20 MiBps to 2 GiBps.
Just for comparison this is what Windows does[1] for it's PRNG stuff (in Vista SP1 and newer at least). It uses AES256 in CTR mode as a DRNG[2].
[1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...
[2] NIST 800-90
Or the getrandom() syscall, once it has trickled into all mainstream distributions' libcs. Which is basically /dev/random, but without the need to open() and close() the device first and after you've consumed your entropy.
http://man7.org/linux/man-pages/man2/getrandom.2.html
A question: Is there value in cryopto-signing the random numbers this emits ?
http://www.bitbabbler.org/index.html
Needs more peer review and independent testing anyway. It's designed so you can do it yourself with a logic analyzer or whatever.
EDIT: I should add that, quite interesting to me, BitBabbler folks are building this for people who already know why a TRNG or analog source is important. They're not interested in evangelizing or winning anyone over: just people already wanting one buying it and preferably analyzing it to give them feedback on esoteric failure modes they haven't thought of. Only contact them if you're serious.
"If you want to know more about the company where this all started, the Voicetronix website can tell you all about that, in all its only slightly updated since the mid 90's glory. We're engineers who build great hardware and provide exceptional support for it ... Web 2.0 bling isn't quite our favourite thing to spend time on :) " (source: bottom of Buy page)
Haha. Anyway, I tried to find price but couldn't. If you want one, just email them at the address in BitBabbler Buy tab for a price quote. That's how they typically work. Here's the company if you're curious:
http://www.voicetronix.com/
Their differentiator, per their lead engineer, was quality hardware with free as in speech software and custom service/support. I found that interesting esp given it's a 90's era business that predates the OSS HW fads. A model worth copying perhaps.
Note: I haven't hands on reviewed a BitBabbler or their other products. Not endorsing them. Just like what I saw and heard vs what I usually see and hear. Suggesting others review, test, or use the products to see if they live up.
I'd love to get my hands on one of these and do some statistics on a lot of data pulled off of these.
To be honest, I think that at certain controlled temperatures, I'd bet you could start to weight numbers for prediction if you are trying to break crypto keys generated by these.
I see a few hardware RNGs, and they're usually uncased. I think I'd want it in a good quality shielded case.
I'd wonder what big data analysis of this data would show us (IE occurrences of certain samples of 1 TB of data polled).
http://www.idquantique.com/random-number-generation/quantis-...
[1]: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6...
Radiation sources really are random. You get a nice strong signal from each decay sensed, and the time between those events is random.
Further, not all noise is RF-based. Thermal noise, for example, is not influenced by RF intereferrence.
It's useful to take the output of the noise generator and run it into a spectrum analyzer while using electric motors and walkie-talkies close to the device. If the noise spectrum change shape when you do that, it's not random.
Here's a cheap way to generate 800 bits per second from radioactive decay. https://www.fourmilab.ch/hotbits/
Why aren't 800 honest-to-goodness random bits per second enough for all practical purposes? If you need more than that, you should be playing in the big leagues, where using a little USB fob is not the answer!
For seeding the system CSPRNG, that's fine. For securely destroying data, such as wiping hard drives, or creating underlying noise for encrypted filesystems, 800 Bps is going to take you years to work through a 4 TB hard drive.
But you don't need to do that. You simply take 256 bits and use them as an AES key. You then generate a stream of bits and write that to disk. Reading the disk afterward is equivalent to breaking AES. Or take Bernstein's advice and use an even better stream cipher. https://cr.yp.to/streamciphers/why.html
And even if you re-key every 1 GB (just for grins), you're still hardly using any random bits at all.
OTOH check out this comment elsewhere in this thread. https://news.ycombinator.com/item?id=11560417
IDQuantique sells a product that purportedly generates millions of random bits per second. If I were gambling at a casino, I'd feel more comfortable with that than with randomness from an AES stream.
Yeah, I know I'm thinking irrationally. But I'm sure I'm not the only one.
Completely agreed. I've already mentioned in this post about only using the Linux CSPRNG to seed a userspace CSPRNG, because the Linux CSPRNG is horrendously slow, due to its ad-hoc design. However, if the Linux CSPRNG was AES-128 using the standard CTR_DRBG, with AES-NI, it could go north of 2 GiBps without breaking a sweat, keeping clean random.c code (it already ships aes.h), and it's backtracking resistant.
However, if you're going to spend good money for hardware producing random values, at least for me, I'm going to want the best bang for my buck. If you look at https://en.wikipedia.org/wiki/Comparison_of_hardware_random_..., you can spend $35 for the WaywardGeek "Infinite Noise RNG", which outputs 300 Kbps, or spend $24 for an RTL-SDR dongle and get about 2.8 MiBps. The value of bits per dollar is much higher with the RTL-SDR than the WaywardGeek. This is considerably cheaper than getting a radioactive sample, coupled with a Geiger counter.
IDQuantique sells a product that purportedly generates millions of random bits per second. If I were gambling at a casino, I'd feel more comfortable with that than with randomness from an AES stream. Yeah, I know I'm thinking irrationally.
Yeah, that is irrational. The problem with most TRNGs is that their output is biased. As a casino operator, I might be interested if the bias benefits my casino, but I would also need to prove to regulators that I'm not cheating the public. Personally, I'd err on the side of caution when it comes to the law.
So either I would whiten the output with a cryptographic primitive (John von Neumann debiasing is too costly, losing at least 1/2 the bits), or just write a userspace CSPRNG, and get the throughput you'll need for the demand. I could then constantly rekey/rseed the generator with the TRNG if I was tinfoil-hat-paranoid.
In either case, TRNG or CSPRNG, you'll want exceptional throughput for your casino. Which brings us back to another example of which 800 bps just isn't going to cut it. :)
Choice bits:
>> RDRAND is available in Ivy Bridge processors
Theodore Ts'o:
>> Relying solely on the hardware random number generator which is using an implementation sealed inside a chip which is impossible to audit is a BAD idea.
In actual reality, the generator won't be influenced, because it will require a strong signal, close by, likely transmitting illegally on the frequency you are listening to (of course, the attacker would need to know the frequency you're tuned to). Simple monitoring tools and scripts can be used to test the quality of the generator output. If the quality of the randomness begins to fail, as would happen with deliberate RF interference, then the script would stop outputting bits, and alert the administrator. Or, even better, switch to a different frequency, chosen at random by the kernel CSPRNG.
What's great about using the RTL-SDR as an RNG, is these devices are cheap (about $20) and can output 3 MiBps, which vastly out performs any USB HWRNG that I have personally tested for the value (bits per second per dollar).
FWIW, random.org uses RF noise as the basis for delivering random numbers to the Internet.
Anyway, FYI.
https://en.wikipedia.org/wiki/Diehard_tests
A purely paranoid person would really prefer something with an antenna (to get some noise by detection), some RLC, a defective/chaotic oscillator on the dye of an a SOC that makes it embedded and you can verify by looking with a microscope that you shield in a Faraday cage.
And no, deciphering logic of a microchip with a microscope is not that hard ; almost anyone can do it with a tad of training and some education.