This point bears repeating every so often: cryptography is difficult, and those of us who haven't spent years in its mathematical bowels are not qualified to create usable algorithms. Meaning: the encryption code you wrote yourself is probably hackable. Easily hackable.
Excellent point; I wish I'd thought to mention that. I spent a few years as a CISO, and honestly, I know that I really want anything to do with the security industry again, outside of making sure that systems and apps are as secure as I can make them.
Why? Because security is a lot harder than most people recognize. Blowfish is an excellent symmetric encryption algorithm, but, as you say, it would be easy enough to implement it badly and end up with insecure, trusted code.
Blowfish is not an excellent symmetric encryption algorithm. For instance, it has a 64 bit block size, which makes it riskier to use in CTR mode and makes some ciphertext malleability problems easier to exploit.
Great link! The takeaway for me is that in an ideal world we would be able to use only strong peer-reviewed crypto written by programmers experienced in codebreaking and cryptanalysis. Phil Zimmerman admits in that article that this is his ideal standard and yet his own PGP software doesn't meet his ideal.
So we're in a situation where we have to make imperfect choices. Is there any reliable guide that discusses the current best practices in securing things like ssh, VPN, etc?
I get the feeling that the preference of many of the infosec professionals on HN is that a properly configured SSH tunnel is the only secure way to communicate between machines on the Internet. ( IPSec VPN connections I guess are weaker?)
So just use stunnel or autotunnel? Tunneling traffic imo seems like an inelegant hack, but are there any crypto libraries that are reliable?
Crypto is hard, but to be fair it's also not easy to figure out what crypto to use if you want to rely on external libraries or systems like SSH.
"Crypto can't be a «bit broken». It can't be «mostly working». Either it's 100% correct, or you shouldn't have bothered doing it at all. The weakest link breaks the whole chain."
I completely agree. Ever seen a password hash mega-post on paste-bin? Bought to you by bad crypto. Saying "I don't
know" is one of the hardest things to do, and also one of the most powerful. I applaud you for knowing your limits in this regard. I like to say crypto is like surgery, best left for the experts. You never want to "I kinda messed up that triple bypass," because there are major consequences for it. The same is true for virtually all aspects of cryptography. Thank you for this post!
There ought to be a good set of FOSS unit tests for those who dare implement their own crypto. For instance, you let it hook in to your PRNG, and it'll tell you if the output is random-looking enough.
It wouldn't be a panacea for bad crypto, and it does create a risk of people thinking "oh, it passed all of the tests, it must be secure," while still implementing it overall incorrectly. But I still think it would mitigate these "foolish/easy" errors and allows devs to focus on proper overall implementation.
The problem is that it's really hard to write unit tests to determine if you've gotten the crypto right. For one, the unit tests would have to have a standardized interface to your cryptographic protocol. But if you're using a standard cryptographic protocol, why would you be rolling your own implementation instead of using an existing, well tested and vetted one?
Second, it's quite difficult, in an automated way, to look for the kinds of security vulnerabilities you would need to audit. Timing attacks may show up in different places for different software. Poorly seeded entropy is quite hard to detect; Debian systems were generating keys with only about 16 bits of entropy for years, which is enough that it's hard to detect in an automated way unless you are really looking for that specific breakage but is a devastating security hole. You may have bugs that are not crypto related at all but simple buffer overflows or the like. And so on.
You could probably automate some of it. But on the whole, you need many good, experienced people regularly auditing the code. And it's a lot easier to get that by using an existing, widely used and widely audited library than by rolling your own.
However these are just statistical tests; they tell you nothing about how much actual entropy you're actually getting (cf. Debian). That said, it probably would have caught the Cryptocat bias.
Not only the number of bits in the seed, but also their unpredictability.
There are many things that can go wrong. Some examples:
- the seed is timer-based and has far less entropy (information) than its bit size
- the seed is OK, but the generator is somehow weak (e.g., only made with linear components, or throws away parts of the seed, or outputs too much of its internal state)
- Bugs, bugs everywhere (cute example: an RC4-based generator that uses the XOR trick to swap elements)
No standard statistical test would catch those (it would catch the latter, actually), even though the security of the generator would be gravely diminished.
You can't unit test crypto. There are tests you can run to check for PRNGs being obviously broken, but that doesn't help you: the point of the article is that crypto has to be 100% right or it's 100% broken. It doesn't matter much to an attacker whether a flaw is "foolish/easy" or not: he can exploit you either way.
Also, a lot of the time the vulnerabilities are not in primitives like PRNGs, but in how application programmers combine the primitives in their application, and there's no way to unit test the security of that. The way to mitigate this is by developing and using high level library interfaces that make it very difficult to screw up.
Unfortunately, it's really really difficult to test for randomness. We can test for particular patterns, but nowhere near what a determined human with access to the source code can find.
At the moment, the only "correct" test I know of is hilariously impractical: enumerate all possible programs up to a reasonable length, run those programs against output from the random number generator for a few minutes each, and see if they're better than chance at recovering the seed you used.
There exist test suites which look for some typical problems. The http://en.wikipedia.org/wiki/Diehard_tests were well-known long ago when I last thought seriously about randomness, and a few minutes with wikipedia told me about http://www.phy.duke.edu/~rgb/General/dieharder.php too. But your dream of software that can truly automatically "tell you if the output is random-looking enough [for crypto]" is deep unrealistic, akin to the deep confusedness sometimes exhibited by business majors dreaming up copy protection features. Merely defining the notion of "random enough for crypto" is a cornucopia of interesting technical problems, and the useful definitions naturally tend to involve the boundaries of complexity classes and thus a bunch of open conjectures, and it gets worse from there. (Things like: Even if you could prove the conjecture that this complexity class is good enough, how are you going to test for nonmembership in it faster than your adversary can break it? Most people are not very interested in crypto which can be broken by an adversary in a month of CPU time, and most people are not interested in tests that take a month of CPU time to run.)
It's not just a matter of perfection in the field of programming. It's an entirely different field that happens to involve some of the same things and ideas. Just because you're a driver, it doesn't mean you're a qualified mechanic, and if you're a mechanic, it doesn't mean you're a forensic crash investigator. They are related in terms of subject matter, but they are not equivalent skill sets.
As programmers, we are dealing with interfaces and movement/copying of data. Security deals with epiphenomenal leakage of access and data. It is not programming, though it involves programming.
The inexperience and hubris of youth. We all have this to varying degrees, but those entirely new to the field for whom it is their first profession have it in spades (that's both a gift and a curse; great things are built by "naïve" trailblazers).
SJCL would have been helpful but still isn't a complete solution to the (exceptionally difficult, even for cryptography) problems they chose to tackle.
These also seem like good reasons not to use proprietary/closed source solutions.
At least with cryptocat everyone know knows about the security issues now and take precautions as if their entire encrypted communications have been compromised. With proprietary solutions you have to rely on an honor system where you hope the vendor tells you there is an issue.
The thing about crypto is that nobody can get it right the first time. It requires two things that most programmers have in relatively short supply. Radical transparency and radical humility.
I mentioned on another crypto thread the frustrating fact that more than one professional crypto friend of mine is working on designs that could replace PGP, not to mention a lot of terrible ad hoc app crypto --- but because they're pro's, they're uncomfortable sharing designs until they're confident that they've fully validated them. That's what humility looks like: knowing you're an industry leading expert at cryptographic design and still waiting months or even years to publish so you can make sure you got things right.
This is not really the most insightful comment, but anyway.
I found the article to be needlessly defeatist.
I am not a crypto expert. I've read Practical Cryptography and have a lots of experience with software engineering in general.
This article (yet again) loosely says "crypto" without specifying whether he's talking about "crypto protocols" or "crypto primitives" (I use "protocol" in a theoretical sense: saving a file piped through gpg with passphrase rememembered in memory constitutes a protocol).
It's well understood that mere mortals shouldn't create "crypto primitives". But I would argue that we're soon going to reach the point where many software engineers will have to understand crypto protocol creation.
Just like many software engineers in the past 10 years or so have had to become aware of multicore (NoSQL, horizontal scaling, Go/Rust concurrency, probably even async callbacks in Javascript etc are all different aspect of multicore, imho).
I don't think we as a profession should abdicate our responsibility to store/transport data securely by just saying "crypto is hard; so don't do it".
I also take issue with the claim that crypto code is either 100% working, or 0% working. From a crypto theory point-of-view, yes, that is how cryptographers think.
But in practice, there is a VAST difference between an attack that requires 2^32 INTERACTIONS with a remote server and one that requires 2^32 COMPUTATIONS on the attackers machine. A cryptographer would say both attacks are equally easy (kinda like O(n) notation).
Just my two cents.
And finally, re: the RNG vulnerability in Cryptocat, that is very bad and just sloppy coding. But even that vulnerability required that the attacker compromise the private SSL key of cryptocat's server. Defense in depth FTW.
I don't think practical cryptographers ignore the difference between computations and interactions—there are different threat models and they are carefully studied. Part of the problem might be that a system designed to be safe against 2^32 interactions is deployed in a place where the system is vulnerable to attacks on the order of 2^32 computations.
The problem I think the author is highlighting is that the mistake risk distribution is hard to understand. Some kinds of off by one errors may cause relatively small reductions in security margins. Some kinds may cause complete loss of system integrity. It's hard to distinguish between the two without extensive testing and expertise. Furthermore, the kind of user who would never tell you about your error is exactly the kind who will find it.
Finally, loss of your security infrastructure is unlikely to cause just small visual discomfort to your users—it's likely to hurt them materially.
No, it's not just crypto primitives, it's crypto protocols, too. It's even designing other protocols that run on top of crypto protocols (see CRIME). Even implementing existing crypto primitives is hazardous: D.J. Bernstein's work on cache timing attacks against AES are proof of that.
The problem with crypto is that it's a specialist profession that generalists think they can do. In reality, crypto is more like law than software engineering: You can't just reason it from first principles, and you can't write any tests that will tell you that it's working. You have to know the specific attacks that people are capable of, and come up with a strategy that will avoid not only the current attacks that you know about, but future attacks that haven't been discovered yet. You're tasked with building a system of obstacles that nobody will be able to find any clever workarounds for, and your adversaries are smarter than you, more numerous than you, more well-funded than you, they're experts in breaking crypto, and they're all from the future.
That's not something a handful of engineers working in isolation can do. It takes the whole community years to even come close.
Saying that software engineers can design crypto protocols is like saying that software engineers can be their own lawyers. A few can, but almost all engineers who think they know the law are wrong. Even lawyers routinely lose cases. The best we can do is to try to give them better tools to handle the common cases (e.g. Creative Commons, SSH, better APIs), and remind them to talk to the specialists before getting too creative.
The whole purpose of Cryptocat is that people are supposed to be able to use it without trusting the integrity of the server. That's practically the project thesis of Cryptocat.
Assuming you're talking about implementing, and not using preexisting, vetted libraries:
Spend a long time in the community. Listen closely to the people who have a good reputation. Have your code publicly vetted before it's ever used. Expect your code to be torn apart.
All these "crypto.... scary!" things are right of course. But somehow tiresome. They're ripe for satire... something involving Chuck Norris being the only one perfect enough to encrypt stuff, just as he's able to compress random data. 100%. With his biceps.
I would agree, but for something as important as cryptography, I would prefer people error on the side of too cautious or repetitive if it means less people making the same mistake twice. It's the same reason why I don't mind repeated HN submissions if it's a good enough delay - the discussion was clearly relevant the first time, and new people have joined the community since then and deserve an equal chance to talk about the subject with new perspective.
Caution is a good attitude to have when working on critical code with non-obvious modes of failure.
However I agree with davidw; this "dont' touch it" or "we're not smart enough" attitude is tiresome. Couldn't we instead admit that this is a difficult problem which requires some engineering rigor to solve well and talk about how we can address such problems like professionals instead of hiding from them?
Most people don't actually want to do their own surgery. Even if you put a scalpel in someone's hands, and gave them all the anesthesia, they still wouldn't be tempted to remove someone's appendix after reading an eHow article about it.
The same isn't true of crypto. People know both intellectually and viscerally that they shouldn't be doing their own surgeries. But they only have an intellectual understanding that crypto is unsafe.
A big part of that is because the harm caused by crypto mistakes are often an externality to the developer.
Just pick up a $10 book at your local Home Depot and if you can read and follow the simple instructions and guidelines inside, you'll be able to do most simple electrical work.
Some electricians may say that it's foolish to do your own electrical work unless you're a trained/certified electrician.
Lots of people do most of their own electrical work without problems, and only hire expert electricians for the really risky or technical parts such as connecting to the utility companies or dealing with really high voltages.
"Simple electrical work" might equate to "wiring Nacl into an application". Designing a cryptosystem to replace Nacl in your application is more akin to climbing up the pole in your alley and working from there.
But I'm particularly interested in David's response, because he raises this concern pretty regularly.
Is there some kind of cryptography for idiots guide around?
Something that explains how to use a high level cryptography library safely. A guide that recognizes that the reader is probably stupid enough to create a buffer overflow if he even goes near C++. A guide that assumes the reader will use rand to generate keys if able to and not reminded multiple times not to. A guide for people who have never heard about side channels. A guide that seriously cautions not to hand over the plaintext to Eve even if she asks nicely.
If you use the right library --- Nacl or Keyczar, or call out to PGP --- you don't need to be an expert to build cryptography into your application.
In all the stories you read about some generalist developer blowing their users' feet off with bad crypto, what you're reading about is someone reinventing these libraries with vanity crypto.
Lot of people get shocked and die too. Ask any electrician or bench tech or radio tech how many times they have been "bit". I'm professionally trained to work on nasty stuff and I ended up in the hospital due to an untrained persons actions on a job site. And I've lost a few colleagues due to antenna raisings/maintenance gone wrong.
Home electrical work is a great example, because like crypto, there are many opportunities to screw up. The Home Depot book usually assumes that the underlying electrical system is reasonably modern and implemented in accordance with the NEC.
Not always true! Your simple change like adding a GFCI outlet or a socket somewhere may not behave as expected if some idiot 30 years ago did something like attach ground to the neutral wire. Or, unbeknownst to you, that old line that you make a pigtail with may be an aluminium wire, which requires specific hardware to safely splice. 5 years later, the trivial act of installing a ceiling fan could be a fire.
Crypto is similar, there are lots of icebergs lurking around. That doesn't mean that you need to fear it, but you do need to know what you are doing, or take measures to validate that what you're learning and applying is accurate.
I wouldn't mind doing most of my electrical work myself if it wasn't because I would potentially get in trouble with various insurances policies and/or laws doing it. I can string a wire and put up a new outlet as well as anybody (if, perhaps, not as fast). I could save quite a mint doing it.
It only bothers me in the same sense that I'd be bothered seeing the 3rd or 4th article about "why I don't conduct routine operations on my family to save some money".
More seriously, as someone else says elsewhere, it seems better to confront the problem head on, and help people figure things out in terms of what's a really bad idea, what you can reasonably hope to piece together yourself using off the shelf pieces, and what the heck: what to do to get more experience if you're interested in different areas of crypto and security. I mean, those "perfect" people who write stuff had to get their start somewhere, right? The author of this particular piece strikes me as correctly cautious - "I'm not smart enough" - or something like that. Should crypto writing be left to people who feel like they are pretty hot shit? Somehow that doesn't strike me as a good way to decide. What are some materials you should master in order to do X, Y, or Z with some degree of certainty?
I don't know much about crypto and security stuff, so I'm not the one to write it, but I think a more detailed article might be more beneficial, because "don't touch crypto stuff" is awfully vague. For instance, utilizing GPG or SSL seems like it makes sense for some situations, and that there are a fair number of people who can make them work ok. Designing your own crypto algorithm and implementing it in C probably is not something most people ought to do outside of experimenting for their own interest. But they why's and how's ought to be explained some too. I guess what bugs me about some of these articles is that they feel a bit cargo-culty in the sense of worshiping the security gods that you must make obeisance to rather than simply explaining some facts about various aspects of crypto and security.
One more thought on the subject: if people don't understand why some things are bad ideas, how can they explain it to their bosses asking them to do something silly?
I think at heart, the thing I'm trying to say is that a "just the facts, m'am" approach will work well with some people like myself. I am fully convinced that I am not competent to write crypto algorithms; nor am I enthusiastic about attempting to code them in C. But at the same time, there's something grating about the "ooooh, scary" approach. Maybe it works better with younger developers who aren't as cognizant of the risks and difficulties?
You listed stuff that only affect yourself when you screw it up. But when you build systems for others (read: paying customers) you have to make sure you do your very best. And when it comes to cryptography 99% of developers are better of using well established technologies instead of brewing their own.
That attitude to crypto is pervasive, annoying, and wrong. We don't tolerate the "you're too stupid to use that" attitude in any other part of software development, and we shouldn't tolerate it in cryptography.
Every developer needs to touch crypto. Encrypted communications needs to be our default. And yes, of course, we should prefer verified, standard algorithms (NSA Suite B, for example).
It's OK to get it wrong, it's OK to fail forward, even with cryptography. ROT13 will protect you very well, if your attack vector is someone glancing over your shoulder for 1 second. As long as the code is open, and you're honest about what it does, you've made people a little bit safer.
There's a fair amount of gloating around Cryptocat, but it protected people's communications from me, because I didn't know how to break it. So that's better than nothing.
>We don't tolerate the "you're too stupid to use that" attitude in any other part of software development, and we shouldn't tolerate it in cryptography.
We shouldn't, but we should provide tools that allow software engineers to securely design applications without having to be crypto experts, in much the same way I can write python code without being a kernel hacker. Two examples spring to mind: Authenticated https api calls and bcrypt. These both work securely without requiring deep knowledge and they are so easy to setup it is unlikely someone will roll their own.
You can PLAY with crypto, discuss it try things out and have fun with it. After you have been doing that and hanging out in the right circles for a few years, reading lots and probably breaking other people's ideas and implementations not just yours then you will be some sort of position to judge whether your work may be safe to unleash on the public as anything more than a low security experiment.
One of the problems is with security and crypto is that the people who really understand it make fairly weak promises such as that it is "Pretty Good Privacy" but the incompetent, greedy or malicious make strong marketing claims about the security that they are offering. Emphasis on incompetent in the Cryptocat case.
Crypto is an area where the Dunning Kruger effect[1] seems both especially strong and especially dangerous.
Everybody should be able to fly a plane, too. Think of how awesome that would be! A rebirth of the American general aviation industry; new airplane designs; solutions to congested airports.
It does not follow from that sentiment that anyone should be able to jump into the cockpit of a Cessna and just figure things out for themselves.
The part where this analogy breaks down is that some random person will probably die in a fire pretty quickly if they try to fly a Cessna. Crypto is like a Cessna that's easy to fly but if you don't fly it exactly right, a few months after your flight, the ground under your flight path spontaneously combusts.
There's a fair amount of gloating around Cryptocat, but it protected people's communications from me, because I didn't know how to break it. So that's better than nothing.
Not if 'nothing' is "don't send the message", rather than the "send the message in the clear" that you're assuming.
Bad crypto gives end users false confidence in the security of their messages. They then send messages they normally wouldn't, and suffer the consequences when those messages end up being read by others.
Amateurs can play with crypto all they like for fun, but they have no business releasing a product to end users.
Good thing Satoshi wasn't scared of cryptography or we wouldn't have bitcoin. Also good thing the Truecrypt devs weren't scared of it, or the Debian devs, or the OpenBSD devs and others who gave us OpenSSH.
Moxie Marlinspike doesn't seem to be scared of it either. According to him it's not impossible to learn simple, tried and true methods for crypto engineering that are all spelled out in Schneier's books and various white papers. These standards were completely ignored by the cryptocat guys (and ignored repeated warnings from auditors). So if you can read, comprehend and do bare minimum research of NIST standards you're pretty safe rolling your own crypto software if you know about attention to detail, the doom principal, PRNGs and their flaws, and know not to attempt to make your own primitives. If you guy's can push out incredibly complex and relatively secure financial trading software there's no reason you can't roll your own SHA-256 on your Android phone replacing SHA-1 FDE or even design Redphone like Moxie did. Would also help if you hung around hashcat forums, read Schneier's cryptogram newsletter, and watched some lectures about software crypto engineering.
"Good thing Satoshi wasn't scared of cryptography or we wouldn't have bitcoin."
...a system for which there is no security definition, no security proof, a known polynomial time attack (for some vague notion of "attack"), etc. I do not think this is the best example you can come up with.
One of the problems here is that cryptography cannot just be cobbled together from component pieces. Things do not always compose securely. Bitcoin is not a signature system, so even if the ECDSA implementation is perfect it does not guarantee that Bitcoin is secure. One of the things that makes cryptography, especially in the public-key setting, difficult is the very precise assumptions and very precise definitions of security; it is easy to make an assumption that is not implied by a security definition, and very easy to violate a key assumption without even realizing it. Worse, cryptosystems must be composed with other software, and such compositions can be insecure (which is basically what happened with Skype).
It's not that people should be scared away. Rather, we need to develop better tools that help programmers catch or avoid these kinds of bugs.
Satoshi is more than likely one or more individuals with significant crypto experience. Open{SSH/BSD/SSL} developers have significant crypto experience.
Don't get me wrong I am a huge Debian fanboy, but how much crypto code do Debian developers write?
"Satoshi is more than likely one or more individuals with significant crypto experience"
I doubt that very much, based on reading the original Bitcoin paper and his posts to the cryptography mailing list. More than likely Satoshi was an amateur/enthusiast.
"Luciano Bello discovered that the random number generator in Debian's openssl package is predictable. This is caused by an incorrect Debian-specific change to the openssl package (CVE-2008-0166). As a result, cryptographic key material may be guessable."
Certainly provides support for "don't touch crypto" guidance, but that was not your original comment. I thought your response unfairly characterized the entire DD team over one incident.
No, some "tried and true" methods of using crypto are in one of Schneier's books, _Practical Cryptography_. _Practical_ is great, but it's not complete, a fact mitigated by the effort Schneier and Furgusen go to in that book to convince people not to write casual crypto.
The methods described in _Applied Cryptography_ are unfortunately well-tried, but few of them are true. _Applied_ is an almanac of cryptographic concepts. Where _Practical_ tries hard to present best practices at every point in the book, _Applied_ instead strives for the broadest coverage; it's a survey, not an instruction guide. Unfortunately for all of us, ~20 years of _Applied_ readers have tried to put directly into practice the material in that book, much of which is (relevant to modern standards) half-baked.
It's also important to know that _Practical_ is also showing its age. There are very common, very serious vulnerabilities that _Practical_ does very little to prevent. For instance, "Design Rule 4" in _Practical_, "The Horton Principle", implies that message authentication should precede encryption. This construction is now disfavored; protocols that use mac-then-encrypt have been broken with side channels abetted by attacker chosen ciphertext.
Other concepts missing from _Practical_: elliptic curve --- particularly given endangered status of RSA, which would be a nit if not for the extremely detailed coverage Schneier gives to more theoretical threats to AES, (EC)DSA parameter tampering, the notion of minimizing randomness from cryptosystems, the modern AE modes, an in-depth treatment of side channels (where "in-depth" might mean "at least as much coverage as is given to the question of what block cipher mode to use") --- particularly error oracles, applications of hash collisions, RSA message formatting (it doesn't even cover OAEP), and secure key derivation.
I write this from a place of love; _Practical Cryptography_ is one of my all-time favorite software security books. It's one of those books you can "read backwards" to learn how to break systems in addition to learning how to build them. With age, though, _Practical_ is becoming more useful as a breaker's guide and less and a builder's guide.
There is no book anywhere that a generalist developer can read to learn how to build a secure cryptosystem from scratch, and generalist developers should be relying instead on high-level libraries like Keyczar and Nacl.
I don't know a think about Satoshi's competence (I couldn't, because nobody knows who he is), but Moxie Marlinspike has spent over 15 years building his competence. If you're Moxie, build whatever you want.
What a dumb comment; as if "security contractors" had anything to do with the quality of books, or didn't work in a field that overwhelmingly rewarded publishing vulnerability information.
That's why I wrote you should research all the new info like the Doom principal (encrypt then MAC) www.thoughtcrime.org/blog/the-cryptographic-doom-principle/
I dont think this is easy at all just lots of articles saying dont even try it, when I bet theres more than competent people here like you who could probably pull off impressive alpha software for long and thorough peer review instead of us all being stuck with decades old clunky software like Truecrypt or GnuPG Assistant.
Who knows maybe in a year cryptocat will be excellent software at least the devs listen to auditors now
In other words, you think that the people writing crypto software should simply do the work to become competent cryptographers. I strongly agree with you.
I find your sentiment about GnuPG a little galling. GnuPG is widely trusted and one of the most reviewed cryptosystems in the world; you advocate that it should be replaced --- by Cryptocat, no less --- because you find it "clunky". It's as if the thought hadn't occurred to you that a better exercise than designing one's own vanity cryptosystem might instead be to come up with a new UX for an existing trustworthy system.
I'm curious as to what web application you foresee supplanting Truecrypt, which is a full-disk encryption system whose entire UX is "enter the password to access this disk".
I wrote replace GnuPG Assistant not GnuPG. I have never had a problem using command line gpg or GPA but seems millions of ppl can't figure it out so they don't use it, thus NSA wins. I also dont think cryptocat should replace it (dont know where I inferred it should).
TC is clunky and has distro rights problems so a lot of releases like Tails are dumping it.
If I remember rightly, they actually say in Practical Cryptography that some of the worst crypto they've seen comes from people who've read Applied Cryptography and then decided to roll their own. Which is presumably one of the dangers in writing an "accessible" cryptography textbook is that then people who read it think they understand it... (I say this as a fan of Applied Cryptography, but more as an academic text -- I'd never dream of writing crypto code myself)
What is the deal with Practical Cryptography versus Cryptography Engineering? I have frequently seen people say they are the "same book." How similar are they? The publication date is seven years apart. Did they just add a new foreword and change the title?
The very few changes from Practical to Engineering, summarized:
* Added a co-author (Tadayoshi Kohno)
* Added handy exercises for self-study (presumably the work of Kohno)
* Buried the opening "Design Philosophy" chapter in the following "Context" chapter, in a more nuanced, mature form
* Drops a discussion of UMAC for one of GMAC which is similar in that they both eschew the use of normal hash functions for something called a "universal" hash
* Discussion of OCB and CCM modes is updated with info about CWC and GCM, since that took place in the publishing interim
* Index is only a page longer, but seems more complete
* Reduced from 410 pages to 353, with a font that appears slightly larger and much less LaTeX-ugly; perhaps the kerning is better too?
That's all I found in 15 minutes of skimming. Note that flaws which were understandable in Practical are more glaring in Engineering: recommending XML where JSON would be better, advocating MAC-then-encrypt rather than encrypt-then-MAC, etc. I probably would not have purchased Engineering if I had realized how minor its updates were. It would be better to purchase if you don't own either book, though.
Thanks for sharing your perspective and knowledge here.
Really important to know how to be safe and what to look out for. I wasn't getting the feeling that many infosec professionals on HN were able/willing to outline reasonable guidelines for experienced devs/admins to follow.
This too is an irritating statement given that me, Sean, and Marcin have been having 1:1 manual email correspondences with something like eight thousand strangers to teach them crypto attacks (the current standings are level 0 (8152), level 1 (608), level 2 (181), level 3 (61), level 4 (58), level 5 (42), level 6 (51) --- http://tinyurl.com/mtsocrypt).
What I perceive is that the problem isn't really that "infosec professionals on HN" are unwilling to teach developers, but rather that we're not teaching developers what they want to hear about crypto.
Thanks for your hard work and for setting that up. Sounds like an awesome program. If I had more time right now, I'd love to check it out more.
I think we're on the same page here, but coming at it from different perspectives. I often have to find ways to secure as best as possible insecure systems that are already built. And I'm trying to find ways to incrementally improve systems over time so they get more and more secure.
I can't wait to deploy code until its 100% secure, or the startups I've worked for would just find someone else to do the job who wouldn't be a 'blocker'.
I appreciate your appreciation, and note also that we're well compensated in other ways for doing that work. The challenges will be around for several more months.
I'm just rebutting the idea --- which maybe you didn't mean --- that elitist infosec people were trying to set themselves up as gatekeepers. The opposite is true. Sean and I are working hard on getting as much of what we know about breaking cryptosystems into the challenges (set 7 is coming out probably next week, and I'm working on ECC stuff for set 8 now). We want the info out.
But look at the numbers I just posted: 8100 people have started the challenges. Just 51 people have managed to finish. I like the challenges a lot, but candidly: on the scale of professional cryptographic research, it's pretty basic; being among the 0.5% of challenge finishers wouldn't qualify you to design cryptosystems.
"I can't wait to deploy code until its 100% secure, "
That's fine if your working on a social network for cats, or another rage face Internet meme generator, but please don't use that excuse if you're storing my credit card details or health records (or writing an encrypted chat client you're promoting to political activists who's lives might literally depend on it).
For Bitcoin what would the consequences have been of getting it wrong at the beginning? Essentially no money would have been lost, no lives lost etc. Also it appears that the person who did create it had a good understanding of crypto and the recent literature (and may even have written it).
It is possible to learn it but learn it first and ship it to the public later or if you must with great big experiment/insecure warnings all over it.
What I've been doing is just reading the code of already audited inplementations of cryptography to understand it better. For example Skein/Threefish. As a test to see what kind of speed SHA-3 would have on a mobile device I implemented my own Threefish SHA3 on Android for full disc encryption. First I looked at Pyskein, Java skein and all the other implementations released by pros. Then I looked at everything the authors who submitted other NIST3 finalists wrote, blogged and argued about on mailing lists. Then I read about the android SHA-1 design decisions. Then I watched lectures on crypto software engineering. Then I looked up everything about accelerometer side channels, PRNGs problems and gotchas, Moxies blog posts, and his yaffs source he used for whispercore, and went through tons of Dos and Dont comments both here and on Schneier's blogs about writing secure software. Then I watched BSDcon coding the openbsd secure way, and talked with maintainers who implement crypto on the openbsd mailing list.
Result is SHA3 full disk for Android. Its certainly not foolproof or nor do I claim its secure but a lot of people are digging through the source and forensics hackers are interested in seeing what they can do with it. I realize I broke cardinal rule of using something that hasn't been proven for decades first (like AES 256) but its been a good learning experience thats for sure.
You're right, but there's still a valid point I think.
I'm a reasonably competent home mechanic. I happily rebuild engines and gearboxes for cars and motorcycles at home. I can modify and tune engines. I easily do all my households vehicle maintenance. And I happily help friends out with their repairs and maintenance. One thing I won't do though, is fix a friends brakes. I'm happy working on my own brakes, and happy to accept the responsibility for my actions. I won't take hat responsibility on for friends though. If you're not happy/capable of fixing your own brakes, take it to someone qualified and insured.
I think crypto is the same. There's nothing wrong with learning about and writing your own crypto as a curious amateur. But deploying that crypto where it can get other people killed is a bigger deal. Don't be "the cryptocat guy", "Oh yeah, I just found out you can't use water for brake fluid, turns out it boils when it gets hot! Who knew? Anyway, I just put brake fluid in my brakes, you should probably upgrade yours too sometime…"
I think if you want to _deploy_ crypto to other people, you owe it to yourself to have at least as much "professional training" as an auto mechanic gets before they're allowed to repair your brakes. And to further stretch the analogy - I'll point out tha most mechanics these days don't "repair" much in brake systems, they'll just replace large components - its becoming more common to not be able to buy seals and pistons any more - you're expected to just replace entire calipers or master cylinders. In my head that's the same approach as saying "I'll choose to use GPG or OpenSSL, but I won't 'do my own crypto'" - for someone who's _not_ prepared to put in a few hundred hours of proper crypto training, I think "rolling your own crypto software" is something best left to personal/educational projects, and never deployed I a way that other people might rely upon it.
I have said this before, but what we really need is the equivalent of SQL for cryptography. We need a language that can be used to describe what kind of security is needed from a cryptosystem and a compiler to turn that into code. It is not just that cryptography is hard to implement; it is also easy to use it incorrectly, compose it poorly with other systems (see the Skype attack), etc.
I recently skimmed a giant manuscript about the creation of a language for defining cryptographic requirements and primitives (120ish pages). I seem to recall that the author was either from Stanford and/or had a name that begins with a "P". I wish I had saved the citation. Hopefully someone else will be able to recognize what I am referring to with my shoddy reference.
When you can't have bugs, you can have proofs. While unknown angles of attack can exists, it is still possible to prove that the known ones are closed.
The math behind OTP is pretty simple, but I may have made a mistake. I've posted the source code to crypto forums, HN, wilders security and emailed it to several prominent crypto developers/experts. No one seems to care nor want to look at it in-depth.
I've worked as a dev where we encrypted research data using standard, industry-accepted crypto (RSA and symmetric AES, etc) as well.
Having said that, I'm not an expert. And the real experts (Bruce Schneier) won't verify anything. They just say, "It's not been broken yet, which is a good indication."
Dive in and write some crypto code. Do it and make mistakes. That's how you learn. Not every program is life or death/mission critical. And if you never do it, you won't learn how. We learn by making mistakes.
Tarsnap is nice crypto code if you like C. It's easy to read too. I have no relation to Colin Percival or tarsnap. He's an expert, and even he makes mistakes:
No one seems to care nor want to look at it in-depth.
I'm not an expert, but I wouldn't look at an OTP implementation either. Key management is already hard enough. It's hard to imagine a useful system you could build on a one-time pad.
Anyone wanting to learn more about information theory, decrypting to multiple plaintext messages or unbreakable encryption would find it interesting. OTP is an edge case and it does not scale (I understand that), but is useful for small messages between two parties when privacy is paramount. Government states have used it.
Also, I can't find any other OTP source code that compiles and works.
Sure, learning is good. Any introductory crypto class would consider OTP for a bit. But you seem to be asking people to critique the system as if it would be used for a non-educational purpose. (If that's not what you mean I apologize for misunderstanding.) That just seems really unlikely.
You take a file an xor it with the pad. It's like five minutes work, tops. The user interface on top of that might be very interesting, but I honestly find it hard to imagine that the underlying algorithm could be messed up and that you'd still have something that could use the keyfile on something encoded to produce a readable plaintext.
Not wishing to be a drag here, but it's been done over and over again in every intro to cryptography class I've seen.
I don't like this attitude. It's really anti-hacker.
Of course, the world is littered with cryptosystems that some foo thought would be secure and wasn't. XOR isn't encryption. ;)
I would suggest that hackers interested in crypto take a sober and unafraid look at the history of crypto and then spend time breaking ciphers, "hacking" in all senses of the word. Read the crypto papers and work on the advanced math until you really and truly grasp "all the things". Pour those 10K hours in....
Crypto is hard. Doing it mostly right requires humility and hard study, and then you have a good chance of being wrong. Nothing to be afraid of if you understand the situation and understand the years and years it takes to really become good at it...
I feel kind of divided about using the Cryptocat as an example. On the one hand, yes, off-by-one errors (and similar simple mistakes) are really easy to make if you you're writing something with low feedback.
But the Cryptocat example is frustrating because the code is so bad. Not in a "poorly written" sense, but in a "going about things in a completely insane way" sense. The code generates a random number between 0 and 1 not by dividing a 53 bit random number by 2^53, but by generating 16 decimal digits, with the rationale that 2^53 ≈ 10^16. If the code hadn't been trying to do the wrong thing in the first place, there wouldn't have been an opportunity for that off-by-one error.
My point isn't that you'll be protected from simple classes of errors as long as you're coding sanely, but rather that that is the lesson you teach if you point to Cryptocat as your example. Crypto is hard, but you can't make that point very well by pointing to code written by someone who doesn't know the correct (and incredibly standard) solutions to common non-crypto problems.
He clearly doesn't touch unit tests, either, or he'd know that they would address ALL (as in, 100%) of his concerns here. Crypto is even easier to test than 99% of things... all you have is a couple of inputs, and an output. Boom, test cases.
I'd just like to point out that `cryptography' is _not_ just about testing that `decrypt(encrypt(msg)) == msg`. memfrob'ing a region of memory does this, but would we consider that a 'secure' encryption scheme? I don't think so . . .
Even if it was -- how do you unit-test a PRNG to ensure the output is uniformly random? How do you ensure you don't have collisions in a hash function? How can you ensure that there's no replay attack on your protocol? How can you use unit-testing to demonstrate that there is no hole in your protocol like, say, when the Germans repeated the key twice at the beginning of the message while using Enigma?
These are all very real and valid cryptography concerns. Sorry if this sounds snarky, but in all seriousness -- I'm very curious as to how you would unit-test these.
How do you ensure you don't have collisions in a hash function?
All hash functions have collisions, by definition, since they represent a unique checksum of a large amount of data in a smaller keyspace. What you are really asking is probably "how do you prove that you have difficult to predict collisions"? I don't know. (I assume that the mechanisms for doing that are based upon statistical analysis of the propagation breadth and speed of various single bit changes in the input stream over varying numbers of rounds?)
How can you ensure that there's no replay attack on your protocol?
A combination of session keys and message sequencing? (eg. Each message must include a hash of the last.)
How can you use unit-testing to demonstrate that there is no hole in your protocol like, say, when the Germans repeated the key twice at the beginning of the message while using Enigma?
There's an entire field of computer science around formal proofs; I don't know enough about it to really comment, however a partial answer may lay in there. Essentially that means adopting a design-time proof strategy in your testing rather than a post-facto unit-testing proof strategy, though.
Good points, but there is a domain of these claimed problems which are covered by automated testing, that's all I'm saying.
And no, you can't find security holes with tests (unless they are random-input style tests, I guess), but you can at least cover the hole afterwards with a test.
The article implies that the off-by-one error in the Cryptocat random number generator is catastrophic. Correct me if I am wrong, but surely that is hardly the case. The random numbers generated are a tiny bit less uniformly distributed than they would be without the error. That's an imperfection but does it have any practical consequences at all ? I know that flaws like this can introduce weaknesses that are vulnerable to attacks, but my intuition tells me this flaw is very close to the "doesn't matter" end of the catastrophic->doesn't matter spectrum. I am more than willing to be educated by someone with specialised domain knowledge.
"Writing a testcase for this would have required complicated thinking and coding which would be as likely to contain an error as it was likely for the code to be tested to contain an error."
i don't know the details, by why couldn't you just collect a million bits of supposedly random data from the generator and run it through the nist test suite? it's true that testing for randomness does require complicated thinking, but fortunately it's been done. standing on the shoulders of giants and whatnot.
But if few people bother themselves with encryption code then there are not many eyeballs on it and mistakes or backdoors become more likely. Probably encryption code should not be left entirely to the "experts" at NSA, Microsoft, etc.
128 comments
[ 1.9 ms ] story [ 171 ms ] threadI first encountered this in a classic essay by PGP's Phil Zimmerman: http://www.philzimmermann.com/EN/essays/SnakeOil.html
edit: fix incorrect link
Actually, most problems are at the security protocol level. Implementing those is the hardest.
Why? Because security is a lot harder than most people recognize. Blowfish is an excellent symmetric encryption algorithm, but, as you say, it would be easy enough to implement it badly and end up with insecure, trusted code.
I'm reminded of Yeats:
Anyone confident in their crypto is dangerous.So we're in a situation where we have to make imperfect choices. Is there any reliable guide that discusses the current best practices in securing things like ssh, VPN, etc?
I get the feeling that the preference of many of the infosec professionals on HN is that a properly configured SSH tunnel is the only secure way to communicate between machines on the Internet. ( IPSec VPN connections I guess are weaker?) So just use stunnel or autotunnel? Tunneling traffic imo seems like an inelegant hack, but are there any crypto libraries that are reliable?
Crypto is hard, but to be fair it's also not easy to figure out what crypto to use if you want to rely on external libraries or systems like SSH.
I completely agree. Ever seen a password hash mega-post on paste-bin? Bought to you by bad crypto. Saying "I don't know" is one of the hardest things to do, and also one of the most powerful. I applaud you for knowing your limits in this regard. I like to say crypto is like surgery, best left for the experts. You never want to "I kinda messed up that triple bypass," because there are major consequences for it. The same is true for virtually all aspects of cryptography. Thank you for this post!
Edit: Here a link to a blog post that was discussed recently on HN that is pertinent: http://www.daemonology.net/blog/2013-06-17-crypto-science-no...
It wouldn't be a panacea for bad crypto, and it does create a risk of people thinking "oh, it passed all of the tests, it must be secure," while still implementing it overall incorrectly. But I still think it would mitigate these "foolish/easy" errors and allows devs to focus on proper overall implementation.
Or does something like this already exist?
The problem is that it's really hard to write unit tests to determine if you've gotten the crypto right. For one, the unit tests would have to have a standardized interface to your cryptographic protocol. But if you're using a standard cryptographic protocol, why would you be rolling your own implementation instead of using an existing, well tested and vetted one?
Second, it's quite difficult, in an automated way, to look for the kinds of security vulnerabilities you would need to audit. Timing attacks may show up in different places for different software. Poorly seeded entropy is quite hard to detect; Debian systems were generating keys with only about 16 bits of entropy for years, which is enough that it's hard to detect in an automated way unless you are really looking for that specific breakage but is a devastating security hole. You may have bugs that are not crypto related at all but simple buffer overflows or the like. And so on.
You could probably automate some of it. But on the whole, you need many good, experienced people regularly auditing the code. And it's a lot easier to get that by using an existing, widely used and widely audited library than by rolling your own.
However these are just statistical tests; they tell you nothing about how much actual entropy you're actually getting (cf. Debian). That said, it probably would have caught the Cryptocat bias.
I'm not actually sure what "actual entropy" would mean in this context if not "the number of bits in the seed".
There are many things that can go wrong. Some examples:
- the seed is timer-based and has far less entropy (information) than its bit size
- the seed is OK, but the generator is somehow weak (e.g., only made with linear components, or throws away parts of the seed, or outputs too much of its internal state)
- Bugs, bugs everywhere (cute example: an RC4-based generator that uses the XOR trick to swap elements)
No standard statistical test would catch those (it would catch the latter, actually), even though the security of the generator would be gravely diminished.
Also, a lot of the time the vulnerabilities are not in primitives like PRNGs, but in how application programmers combine the primitives in their application, and there's no way to unit test the security of that. The way to mitigate this is by developing and using high level library interfaces that make it very difficult to screw up.
At the moment, the only "correct" test I know of is hilariously impractical: enumerate all possible programs up to a reasonable length, run those programs against output from the random number generator for a few minutes each, and see if they're better than chance at recovering the seed you used.
As programmers, we are dealing with interfaces and movement/copying of data. Security deals with epiphenomenal leakage of access and data. It is not programming, though it involves programming.
At least with cryptocat everyone know knows about the security issues now and take precautions as if their entire encrypted communications have been compromised. With proprietary solutions you have to rely on an honor system where you hope the vendor tells you there is an issue.
Compare and contrast.
I found the article to be needlessly defeatist.
I am not a crypto expert. I've read Practical Cryptography and have a lots of experience with software engineering in general.
This article (yet again) loosely says "crypto" without specifying whether he's talking about "crypto protocols" or "crypto primitives" (I use "protocol" in a theoretical sense: saving a file piped through gpg with passphrase rememembered in memory constitutes a protocol).
It's well understood that mere mortals shouldn't create "crypto primitives". But I would argue that we're soon going to reach the point where many software engineers will have to understand crypto protocol creation.
Just like many software engineers in the past 10 years or so have had to become aware of multicore (NoSQL, horizontal scaling, Go/Rust concurrency, probably even async callbacks in Javascript etc are all different aspect of multicore, imho).
I don't think we as a profession should abdicate our responsibility to store/transport data securely by just saying "crypto is hard; so don't do it".
I also take issue with the claim that crypto code is either 100% working, or 0% working. From a crypto theory point-of-view, yes, that is how cryptographers think.
But in practice, there is a VAST difference between an attack that requires 2^32 INTERACTIONS with a remote server and one that requires 2^32 COMPUTATIONS on the attackers machine. A cryptographer would say both attacks are equally easy (kinda like O(n) notation).
Just my two cents.
And finally, re: the RNG vulnerability in Cryptocat, that is very bad and just sloppy coding. But even that vulnerability required that the attacker compromise the private SSL key of cryptocat's server. Defense in depth FTW.
The problem I think the author is highlighting is that the mistake risk distribution is hard to understand. Some kinds of off by one errors may cause relatively small reductions in security margins. Some kinds may cause complete loss of system integrity. It's hard to distinguish between the two without extensive testing and expertise. Furthermore, the kind of user who would never tell you about your error is exactly the kind who will find it.
Finally, loss of your security infrastructure is unlikely to cause just small visual discomfort to your users—it's likely to hurt them materially.
The problem with crypto is that it's a specialist profession that generalists think they can do. In reality, crypto is more like law than software engineering: You can't just reason it from first principles, and you can't write any tests that will tell you that it's working. You have to know the specific attacks that people are capable of, and come up with a strategy that will avoid not only the current attacks that you know about, but future attacks that haven't been discovered yet. You're tasked with building a system of obstacles that nobody will be able to find any clever workarounds for, and your adversaries are smarter than you, more numerous than you, more well-funded than you, they're experts in breaking crypto, and they're all from the future.
That's not something a handful of engineers working in isolation can do. It takes the whole community years to even come close.
Saying that software engineers can design crypto protocols is like saying that software engineers can be their own lawyers. A few can, but almost all engineers who think they know the law are wrong. Even lawyers routinely lose cases. The best we can do is to try to give them better tools to handle the common cases (e.g. Creative Commons, SSH, better APIs), and remind them to talk to the specialists before getting too creative.
Spend a long time in the community. Listen closely to the people who have a good reputation. Have your code publicly vetted before it's ever used. Expect your code to be torn apart.
However I agree with davidw; this "dont' touch it" or "we're not smart enough" attitude is tiresome. Couldn't we instead admit that this is a difficult problem which requires some engineering rigor to solve well and talk about how we can address such problems like professionals instead of hiding from them?
The same isn't true of crypto. People know both intellectually and viscerally that they shouldn't be doing their own surgeries. But they only have an intellectual understanding that crypto is unsafe.
A big part of that is because the harm caused by crypto mistakes are often an externality to the developer.
http://www.schneier.com/book-applied.html
Good crypto needs engineering rigour, honest colective work, and time to mature.
If we were talking instead about people operating their own pharmacies, or doing their own home electrical work, nobody would bat an eyelash.
Just pick up a $10 book at your local Home Depot and if you can read and follow the simple instructions and guidelines inside, you'll be able to do most simple electrical work.
Some electricians may say that it's foolish to do your own electrical work unless you're a trained/certified electrician.
Lots of people do most of their own electrical work without problems, and only hire expert electricians for the really risky or technical parts such as connecting to the utility companies or dealing with really high voltages.
But I'm particularly interested in David's response, because he raises this concern pretty regularly.
Something that explains how to use a high level cryptography library safely. A guide that recognizes that the reader is probably stupid enough to create a buffer overflow if he even goes near C++. A guide that assumes the reader will use rand to generate keys if able to and not reminded multiple times not to. A guide for people who have never heard about side channels. A guide that seriously cautions not to hand over the plaintext to Eve even if she asks nicely.
In all the stories you read about some generalist developer blowing their users' feet off with bad crypto, what you're reading about is someone reinventing these libraries with vanity crypto.
(Just don't use compression with your encryption and avoid the issue entirely).
Spill a little dimethylmercury? Welp, you're dead.
Sign something with your DSA key using a dodgy PRNG? Welp, you're dead.
Not always true! Your simple change like adding a GFCI outlet or a socket somewhere may not behave as expected if some idiot 30 years ago did something like attach ground to the neutral wire. Or, unbeknownst to you, that old line that you make a pigtail with may be an aluminium wire, which requires specific hardware to safely splice. 5 years later, the trivial act of installing a ceiling fan could be a fire.
Crypto is similar, there are lots of icebergs lurking around. That doesn't mean that you need to fear it, but you do need to know what you are doing, or take measures to validate that what you're learning and applying is accurate.
More seriously, as someone else says elsewhere, it seems better to confront the problem head on, and help people figure things out in terms of what's a really bad idea, what you can reasonably hope to piece together yourself using off the shelf pieces, and what the heck: what to do to get more experience if you're interested in different areas of crypto and security. I mean, those "perfect" people who write stuff had to get their start somewhere, right? The author of this particular piece strikes me as correctly cautious - "I'm not smart enough" - or something like that. Should crypto writing be left to people who feel like they are pretty hot shit? Somehow that doesn't strike me as a good way to decide. What are some materials you should master in order to do X, Y, or Z with some degree of certainty?
I don't know much about crypto and security stuff, so I'm not the one to write it, but I think a more detailed article might be more beneficial, because "don't touch crypto stuff" is awfully vague. For instance, utilizing GPG or SSL seems like it makes sense for some situations, and that there are a fair number of people who can make them work ok. Designing your own crypto algorithm and implementing it in C probably is not something most people ought to do outside of experimenting for their own interest. But they why's and how's ought to be explained some too. I guess what bugs me about some of these articles is that they feel a bit cargo-culty in the sense of worshiping the security gods that you must make obeisance to rather than simply explaining some facts about various aspects of crypto and security.
One more thought on the subject: if people don't understand why some things are bad ideas, how can they explain it to their bosses asking them to do something silly?
Every developer needs to touch crypto. Encrypted communications needs to be our default. And yes, of course, we should prefer verified, standard algorithms (NSA Suite B, for example).
It's OK to get it wrong, it's OK to fail forward, even with cryptography. ROT13 will protect you very well, if your attack vector is someone glancing over your shoulder for 1 second. As long as the code is open, and you're honest about what it does, you've made people a little bit safer.
There's a fair amount of gloating around Cryptocat, but it protected people's communications from me, because I didn't know how to break it. So that's better than nothing.
We shouldn't, but we should provide tools that allow software engineers to securely design applications without having to be crypto experts, in much the same way I can write python code without being a kernel hacker. Two examples spring to mind: Authenticated https api calls and bcrypt. These both work securely without requiring deep knowledge and they are so easy to setup it is unlikely someone will roll their own.
One of the problems is with security and crypto is that the people who really understand it make fairly weak promises such as that it is "Pretty Good Privacy" but the incompetent, greedy or malicious make strong marketing claims about the security that they are offering. Emphasis on incompetent in the Cryptocat case.
Crypto is an area where the Dunning Kruger effect[1] seems both especially strong and especially dangerous.
[1] http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect
It does not follow from that sentiment that anyone should be able to jump into the cockpit of a Cessna and just figure things out for themselves.
Not if 'nothing' is "don't send the message", rather than the "send the message in the clear" that you're assuming.
Bad crypto gives end users false confidence in the security of their messages. They then send messages they normally wouldn't, and suffer the consequences when those messages end up being read by others.
Amateurs can play with crypto all they like for fun, but they have no business releasing a product to end users.
Moxie Marlinspike doesn't seem to be scared of it either. According to him it's not impossible to learn simple, tried and true methods for crypto engineering that are all spelled out in Schneier's books and various white papers. These standards were completely ignored by the cryptocat guys (and ignored repeated warnings from auditors). So if you can read, comprehend and do bare minimum research of NIST standards you're pretty safe rolling your own crypto software if you know about attention to detail, the doom principal, PRNGs and their flaws, and know not to attempt to make your own primitives. If you guy's can push out incredibly complex and relatively secure financial trading software there's no reason you can't roll your own SHA-256 on your Android phone replacing SHA-1 FDE or even design Redphone like Moxie did. Would also help if you hung around hashcat forums, read Schneier's cryptogram newsletter, and watched some lectures about software crypto engineering.
...a system for which there is no security definition, no security proof, a known polynomial time attack (for some vague notion of "attack"), etc. I do not think this is the best example you can come up with.
One of the problems here is that cryptography cannot just be cobbled together from component pieces. Things do not always compose securely. Bitcoin is not a signature system, so even if the ECDSA implementation is perfect it does not guarantee that Bitcoin is secure. One of the things that makes cryptography, especially in the public-key setting, difficult is the very precise assumptions and very precise definitions of security; it is easy to make an assumption that is not implied by a security definition, and very easy to violate a key assumption without even realizing it. Worse, cryptosystems must be composed with other software, and such compositions can be insecure (which is basically what happened with Skype).
It's not that people should be scared away. Rather, we need to develop better tools that help programmers catch or avoid these kinds of bugs.
Don't get me wrong I am a huge Debian fanboy, but how much crypto code do Debian developers write?
I doubt that very much, based on reading the original Bitcoin paper and his posts to the cryptography mailing list. More than likely Satoshi was an amateur/enthusiast.
The historical answer seems to have been "too much"...
"Luciano Bello discovered that the random number generator in Debian's openssl package is predictable. This is caused by an incorrect Debian-specific change to the openssl package (CVE-2008-0166). As a result, cryptographic key material may be guessable."
The methods described in _Applied Cryptography_ are unfortunately well-tried, but few of them are true. _Applied_ is an almanac of cryptographic concepts. Where _Practical_ tries hard to present best practices at every point in the book, _Applied_ instead strives for the broadest coverage; it's a survey, not an instruction guide. Unfortunately for all of us, ~20 years of _Applied_ readers have tried to put directly into practice the material in that book, much of which is (relevant to modern standards) half-baked.
It's also important to know that _Practical_ is also showing its age. There are very common, very serious vulnerabilities that _Practical_ does very little to prevent. For instance, "Design Rule 4" in _Practical_, "The Horton Principle", implies that message authentication should precede encryption. This construction is now disfavored; protocols that use mac-then-encrypt have been broken with side channels abetted by attacker chosen ciphertext.
Other concepts missing from _Practical_: elliptic curve --- particularly given endangered status of RSA, which would be a nit if not for the extremely detailed coverage Schneier gives to more theoretical threats to AES, (EC)DSA parameter tampering, the notion of minimizing randomness from cryptosystems, the modern AE modes, an in-depth treatment of side channels (where "in-depth" might mean "at least as much coverage as is given to the question of what block cipher mode to use") --- particularly error oracles, applications of hash collisions, RSA message formatting (it doesn't even cover OAEP), and secure key derivation.
I write this from a place of love; _Practical Cryptography_ is one of my all-time favorite software security books. It's one of those books you can "read backwards" to learn how to break systems in addition to learning how to build them. With age, though, _Practical_ is becoming more useful as a breaker's guide and less and a builder's guide.
There is no book anywhere that a generalist developer can read to learn how to build a secure cryptosystem from scratch, and generalist developers should be relying instead on high-level libraries like Keyczar and Nacl.
I don't know a think about Satoshi's competence (I couldn't, because nobody knows who he is), but Moxie Marlinspike has spent over 15 years building his competence. If you're Moxie, build whatever you want.
I dont think this is easy at all just lots of articles saying dont even try it, when I bet theres more than competent people here like you who could probably pull off impressive alpha software for long and thorough peer review instead of us all being stuck with decades old clunky software like Truecrypt or GnuPG Assistant.
Who knows maybe in a year cryptocat will be excellent software at least the devs listen to auditors now
I find your sentiment about GnuPG a little galling. GnuPG is widely trusted and one of the most reviewed cryptosystems in the world; you advocate that it should be replaced --- by Cryptocat, no less --- because you find it "clunky". It's as if the thought hadn't occurred to you that a better exercise than designing one's own vanity cryptosystem might instead be to come up with a new UX for an existing trustworthy system.
I'm curious as to what web application you foresee supplanting Truecrypt, which is a full-disk encryption system whose entire UX is "enter the password to access this disk".
TC is clunky and has distro rights problems so a lot of releases like Tails are dumping it.
http://cacr.uwaterloo.ca/hac/
https://news.ycombinator.com/item?id=6002694
* Added a co-author (Tadayoshi Kohno)
* Added handy exercises for self-study (presumably the work of Kohno)
* Buried the opening "Design Philosophy" chapter in the following "Context" chapter, in a more nuanced, mature form
* Drops a discussion of UMAC for one of GMAC which is similar in that they both eschew the use of normal hash functions for something called a "universal" hash
* Discussion of OCB and CCM modes is updated with info about CWC and GCM, since that took place in the publishing interim
* Index is only a page longer, but seems more complete
* Reduced from 410 pages to 353, with a font that appears slightly larger and much less LaTeX-ugly; perhaps the kerning is better too?
That's all I found in 15 minutes of skimming. Note that flaws which were understandable in Practical are more glaring in Engineering: recommending XML where JSON would be better, advocating MAC-then-encrypt rather than encrypt-then-MAC, etc. I probably would not have purchased Engineering if I had realized how minor its updates were. It would be better to purchase if you don't own either book, though.
Really important to know how to be safe and what to look out for. I wasn't getting the feeling that many infosec professionals on HN were able/willing to outline reasonable guidelines for experienced devs/admins to follow.
What I perceive is that the problem isn't really that "infosec professionals on HN" are unwilling to teach developers, but rather that we're not teaching developers what they want to hear about crypto.
I think we're on the same page here, but coming at it from different perspectives. I often have to find ways to secure as best as possible insecure systems that are already built. And I'm trying to find ways to incrementally improve systems over time so they get more and more secure.
I can't wait to deploy code until its 100% secure, or the startups I've worked for would just find someone else to do the job who wouldn't be a 'blocker'.
I'm just rebutting the idea --- which maybe you didn't mean --- that elitist infosec people were trying to set themselves up as gatekeepers. The opposite is true. Sean and I are working hard on getting as much of what we know about breaking cryptosystems into the challenges (set 7 is coming out probably next week, and I'm working on ECC stuff for set 8 now). We want the info out.
But look at the numbers I just posted: 8100 people have started the challenges. Just 51 people have managed to finish. I like the challenges a lot, but candidly: on the scale of professional cryptographic research, it's pretty basic; being among the 0.5% of challenge finishers wouldn't qualify you to design cryptosystems.
That's fine if your working on a social network for cats, or another rage face Internet meme generator, but please don't use that excuse if you're storing my credit card details or health records (or writing an encrypted chat client you're promoting to political activists who's lives might literally depend on it).
It is possible to learn it but learn it first and ship it to the public later or if you must with great big experiment/insecure warnings all over it.
Result is SHA3 full disk for Android. Its certainly not foolproof or nor do I claim its secure but a lot of people are digging through the source and forensics hackers are interested in seeing what they can do with it. I realize I broke cardinal rule of using something that hasn't been proven for decades first (like AES 256) but its been a good learning experience thats for sure.
I'm a reasonably competent home mechanic. I happily rebuild engines and gearboxes for cars and motorcycles at home. I can modify and tune engines. I easily do all my households vehicle maintenance. And I happily help friends out with their repairs and maintenance. One thing I won't do though, is fix a friends brakes. I'm happy working on my own brakes, and happy to accept the responsibility for my actions. I won't take hat responsibility on for friends though. If you're not happy/capable of fixing your own brakes, take it to someone qualified and insured.
I think crypto is the same. There's nothing wrong with learning about and writing your own crypto as a curious amateur. But deploying that crypto where it can get other people killed is a bigger deal. Don't be "the cryptocat guy", "Oh yeah, I just found out you can't use water for brake fluid, turns out it boils when it gets hot! Who knew? Anyway, I just put brake fluid in my brakes, you should probably upgrade yours too sometime…"
I think if you want to _deploy_ crypto to other people, you owe it to yourself to have at least as much "professional training" as an auto mechanic gets before they're allowed to repair your brakes. And to further stretch the analogy - I'll point out tha most mechanics these days don't "repair" much in brake systems, they'll just replace large components - its becoming more common to not be able to buy seals and pistons any more - you're expected to just replace entire calipers or master cylinders. In my head that's the same approach as saying "I'll choose to use GPG or OpenSSL, but I won't 'do my own crypto'" - for someone who's _not_ prepared to put in a few hundred hours of proper crypto training, I think "rolling your own crypto software" is something best left to personal/educational projects, and never deployed I a way that other people might rely upon it.
[1] https://eprint.iacr.org/2005/160
[2] http://www.cace-project.eu/
[3] http://www.cace-project.eu/downloads/deliverables-y2/CACE_D5...
[4] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.113...
This is going to be costly…
http://16s.us/FreeOTP/nsa/
The math behind OTP is pretty simple, but I may have made a mistake. I've posted the source code to crypto forums, HN, wilders security and emailed it to several prominent crypto developers/experts. No one seems to care nor want to look at it in-depth.
I've worked as a dev where we encrypted research data using standard, industry-accepted crypto (RSA and symmetric AES, etc) as well.
Having said that, I'm not an expert. And the real experts (Bruce Schneier) won't verify anything. They just say, "It's not been broken yet, which is a good indication."
Dive in and write some crypto code. Do it and make mistakes. That's how you learn. Not every program is life or death/mission critical. And if you never do it, you won't learn how. We learn by making mistakes.
Tarsnap is nice crypto code if you like C. It's easy to read too. I have no relation to Colin Percival or tarsnap. He's an expert, and even he makes mistakes:
http://www.daemonology.net/blog/2011-01-18-tarsnap-critical-...
I'm not an expert, but I wouldn't look at an OTP implementation either. Key management is already hard enough. It's hard to imagine a useful system you could build on a one-time pad.
Also, I can't find any other OTP source code that compiles and works.
Not wishing to be a drag here, but it's been done over and over again in every intro to cryptography class I've seen.
Sorry ^^;
Of course, the world is littered with cryptosystems that some foo thought would be secure and wasn't. XOR isn't encryption. ;)
I would suggest that hackers interested in crypto take a sober and unafraid look at the history of crypto and then spend time breaking ciphers, "hacking" in all senses of the word. Read the crypto papers and work on the advanced math until you really and truly grasp "all the things". Pour those 10K hours in....
Crypto is hard. Doing it mostly right requires humility and hard study, and then you have a good chance of being wrong. Nothing to be afraid of if you understand the situation and understand the years and years it takes to really become good at it...
But the Cryptocat example is frustrating because the code is so bad. Not in a "poorly written" sense, but in a "going about things in a completely insane way" sense. The code generates a random number between 0 and 1 not by dividing a 53 bit random number by 2^53, but by generating 16 decimal digits, with the rationale that 2^53 ≈ 10^16. If the code hadn't been trying to do the wrong thing in the first place, there wouldn't have been an opportunity for that off-by-one error.
My point isn't that you'll be protected from simple classes of errors as long as you're coding sanely, but rather that that is the lesson you teach if you point to Cryptocat as your example. Crypto is hard, but you can't make that point very well by pointing to code written by someone who doesn't know the correct (and incredibly standard) solutions to common non-crypto problems.
Even if it was -- how do you unit-test a PRNG to ensure the output is uniformly random? How do you ensure you don't have collisions in a hash function? How can you ensure that there's no replay attack on your protocol? How can you use unit-testing to demonstrate that there is no hole in your protocol like, say, when the Germans repeated the key twice at the beginning of the message while using Enigma?
These are all very real and valid cryptography concerns. Sorry if this sounds snarky, but in all seriousness -- I'm very curious as to how you would unit-test these.
NIST standard test suite @ http://csrc.nist.gov/groups/ST/toolkit/rng/documentation_sof...
How do you ensure you don't have collisions in a hash function?
All hash functions have collisions, by definition, since they represent a unique checksum of a large amount of data in a smaller keyspace. What you are really asking is probably "how do you prove that you have difficult to predict collisions"? I don't know. (I assume that the mechanisms for doing that are based upon statistical analysis of the propagation breadth and speed of various single bit changes in the input stream over varying numbers of rounds?)
How can you ensure that there's no replay attack on your protocol?
A combination of session keys and message sequencing? (eg. Each message must include a hash of the last.)
How can you use unit-testing to demonstrate that there is no hole in your protocol like, say, when the Germans repeated the key twice at the beginning of the message while using Enigma?
There's an entire field of computer science around formal proofs; I don't know enough about it to really comment, however a partial answer may lay in there. Essentially that means adopting a design-time proof strategy in your testing rather than a post-facto unit-testing proof strategy, though.
And no, you can't find security holes with tests (unless they are random-input style tests, I guess), but you can at least cover the hole afterwards with a test.
i don't know the details, by why couldn't you just collect a million bits of supposedly random data from the generator and run it through the nist test suite? it's true that testing for randomness does require complicated thinking, but fortunately it's been done. standing on the shoulders of giants and whatnot.