There ought to be a way for people to get 'writing some crypto code' out of their system without it being a part of a real project. A sort of play area where you can have a go but not harm anything and people can point out your errors.
Every time I write something that involves some crypto I get absolutely terrified of what I'm likely to mess up. For that reason I almost always use an existing system or protocol. For example, on one of the projects I'm working on at CloudFlare we needed a secure connection across the Internet so I went with TLS 1.2. For another I needed the same thing but with better authentication: TLS 1.2 with client and server certificates. If I need to exchange encrypted messages in some store and forward style I'd likely use PGP.
And then for another project I needed to actually get something someone else had encrypted with RSA and decrypt it. Nightmare. If you find yourself calling low-level crypto APIs start to worry.
The Matasano crypto challenges ( http://www.matasano.com/articles/crypto-challenges/ ) are a great way to do this. Not by implementing crypto, but by breaking crypto in ways that will scare you away from ever implementing crypto.
I recommend Coursera's Crypto 1 course - it walks you through the internals of a lot of the crypto stuff, hammers "Don't do it yourself!" into you and there are programming challenges that involve breaking crypto constructions, finding hash collisions etc etc.
I'm really hoping crypto 2 actually starts in about 10 days as it seems to have been put back a lot.
--edit-- Just to sell it further, it also goes into padding oracles, timing attacks and a variety of clever stuff to show you just how easy it is to screw up :)
Oh, man, that course contained so much of "Do you see any problems with this construction? No? It can't possibly be vulnerable to anything, you say? Boom, here's how to break it completely".
"Every time I write something that involves some crypto I get absolutely terrified of what I'm likely to mess up."
Good!
" For that reason I almost always use an existing system or protocol."
Excellent! If only every was like you we wouldn't have these problems.
The key to understanding this is to understand that cryptography is not 'writing some crypto code'. No part of cryptography has anything to do with the code. Cryptography is about math - feel free to do some cryptography math and work on a mathematical algorithm. Once you have a perfect algorithm that has been peer reviewed, then you can transcribe that mathematical algorithm into code.
Failing to understand the math first means you have already failed and simply should not have even begun.
Cryptography: math first, code later. Failure to follow this sequence means you will be ridiculed and any excuse is simply not good enough.
Thank you for being the voice of reason. It's so easy to get confused, too, because "crypto code" is indistinguishable from "other code", but the implications are much greater. You don't need to peer review a IRC library, but you _need_ to peer review cryptography.
That's just how it is, and if I had to highlight one sentence from your comment it would be "Failing to understand the math first means you have already failed and simply should not have even begun."
I wish people would understand this instead of thinking trial and error works for cryptography. I mean, heck, most of us learned to code through trial and error, and I can understand why people think they can learn cryptography the same way, but that is not true, and a very toxic mindset.
I agree in general, but you are painting with too broad strokes here:
> No part of cryptography has anything to do with the code.
There have been successful attacks against cryptography based on attacking the implementation. For example if two code paths take different time to execute, it is possible that this leaks information.
This is another reason why you should use a well honed library implementation of cryptography, for example OpenSSL.
This does of course not imply that the crypto–the maths of it–is broken. One just needs to be aware that it is not, in fact, sufficient to implement the maths correctly.
You're correct, and I think that means I'm painting my strokes not broad enough. After you get your math peer reviewed, you then need to get your implementation of the math peer reviewed as well - likely by a low-level hardware engineer and not just some web developers.
Generally, however, nearly all failures of security come from misunderstanding the math when creating the implementation. So I'd say that at least ensuring anybody writing crypto has a completely understanding of the crypto math before touching the code is a very good first step in getting somewhere.
> If I need to exchange encrypted messages in some store and forward style I'd likely use PGP.
Do you or others have any recommendations for cross-platform library code (at least OSX, Windows, Android, Linux, Windows 8) for this purpose?
I may have understood incorrectly, but I thought that using gpg from code required launching a sub-process (e.g. via gpgme). That makes me very nervous from a library point of view, perhaps I'm wrong to be nervous.
Is the answer still "use PGP/GPG" or is there somewhere else I could look to avoid home-made cryptosystem here?
Thanks for that. But afaics, libgcrypt seems to be on a similar level of abstraction as the openssl lib? i.e. implementations of ciphers etc. I then fall into the mistake of http://www.cs.berkeley.edu/~daw/teaching/cs261-f12/misc/if.h... I think? [linking to berkely.edu because the page seems to have gone from matasano.com?]
At the moment it looks like my choice is between implementing a cryptosystem on top of crypto primitives (from openssl or libcrypt) or seeing if I can get away with shipping a lib which spawns child processes via gpgme.
I think a major issue is the lack of understanding of best-practices with regards to security. So we tend to rely on the security of systems that aren't battle-proven. But even battle-proven solutions still have vulnerabilities, so we need to rely on multiple layers of security.
For me at least, this reinforces the lesson to put systems as much as possible behind multiple layers of security such as firewalls and VPN. And I should be careful to architect salt systems in a hub/spoke topology using salt-master and syndics. And encrypt any traffic from the salt-master to groups of servers using ssh tunneling. Looks like pyzmq now supports ssh tunneling, so maybe this would work:
There's an entry on the release notes [1], the changelog, etc; but unless I'm missing it, looks like there's not really proper management of security issues (same thing for the recent crypto.cat bug).
May be I'm old-school but when a project goes over certain size and there's no prominent "security" section (as important as downloads, IMHO), that's a red flag for me.
This shows crypto is much too hard. You can configure a bunch of things, but half the choices you can make (key length, exponent, etc...) will render your encryption worthless. I shouldn't have to know about the chinese remainder theorem to use crypto properly.
The usual ones to recommend here are KeyCzar and NaCl (apparently pronounced salt, not related to saltstack AFAICT).
NaCl in particular provides simple Public/Private and Symmetric modes. The Authenticated Encryption Symmetric mode here is good - http://nacl.cr.yp.to/secretbox.html
His complaint is that crypto is to hard to use. This is a valid complaint and "don't do it" isn't a valid solution. Too many "libraries" consist only of implementations of the crypto algorithms, but don't give any help with the proper use.
>> His complaint is that crypto is to hard to use.
Yes, and if you're going to use low level constructions he's right, and that will never change.
>> This is a valid complaint and "don't do it" isn't a valid solution.
Except that's not what I said, I said find a higher level library. When he asked I gave examples.
>> Too many "libraries" consist only of implementations of the crypto algorithms, but don't give any help with the proper use.
Then these may be the wrong libraries to use to secure your application. Great libs to poke around in the lower levels for a variety of other purposes of course.
Doing your own crypto is generally hard and risky, but with a little googling there are often reasonable defaults and suggestions available from trustworthy sources. The biggest danger seems to be that people pick some random set of values & constructions and assume that's good enough without further checking.
On the flip side this should have set off warning bells for them - it can be a little bit of a code smell to pick 1 for a configurable value in your encryption. Not always a bad choice, but 1 has a lot of properties that can simplify various mathematical operations and Basically setting a crypto value to the smallest positive integer (and a non-prime at that!) is like deciding that a set of numbers in your code should always be represented by two-byte signed ints. Sometimes that's _exactly_ what you want (say, various fast math libraries), but if you're doing general purpose code you should really stop and think about whether you might end up with a value outside that.
> This shows crypto is much too hard. You can configure a bunch of things, but half the choices you can make (key length, exponent, etc...) will render your encryption worthless. I shouldn't have to know about the chinese remainder theorem to use crypto properly.
This attitude scares me. Some things really are hard to do. Some things really are hard to understand. Some are even both. Remember what Euclid is supposed to have said to Ptolemy I of Egypt when the king complained that geometry was hard to learn: There is no royal road to mathematics!
There's no shame in not being able to do or understand every difficult thing under the sun, but don't complain that such things "shouldn't be so hard". They often are. I think it's part of what makes life interesting.
I expressed myself poorly. What I meant is that using cryptography shouldn't be that hard to use for the uninformed user. Introducing cryptography in an application is about security, and if we want more secure applications, cryptography should be easy as pie to use.
Of course cryptography is hard, and it is an interesting topic (one I know a few things about as it turns out).
I'm sort of split in the middle on the two sides here, but as an example of what I would hope for, in this case I'd say passing a 1 to this function in that location should have resulted in a run time error, not a happy useless implementation. Preferably with an error message that points to some helpful documentation on the topic. That's an example, that can't solve everything, and I'd despair of trying to fix every possible stupid error that way, but it would be helpful.
That's not even really a "security" concern, that's just good software engineering. Passing this function something that fails the prerequisites ought to be checked. (And yes, I'd check that it's prime, too. It's an expensive check once, but trivially memoized, and you can even prefill the memoization with the usual 65537 value to make it even cheaper. In this particular case, IIRC, the exponent is always or nearly always simply reused,)
In this case, I think, the author rolled his own RSA implementation. In general, I do not know why the developer of end software should even touch the public exponent. It can be reused, and is not a secret; the library should handle it. In fact, the library should have a function along the lines of (publicKey, secretKey)=RSA.keyGen(), and provide the simple documentation that you want to keep the secret key secret. If they have crypto option which they want to expose (such as key-length), then they can take an optional parameter that either takes a hard-coded constant (IE RSA.512), or (as you suggest) validates the sanity of the input.
You CAN fly a plane without knowing about the weather, because the pilot does the flying for you. All you want is to get from point A to point B via plane.
A crypto library is the pilot. A crypto library that allows you to use insecure settings is a pilot who crashes the plane into a mountain because you asked him to.
I think the analogous entity for "pilot" would be "application". The library is analogous to the airplane, and airplane will fly into a mountain when you ask it to.
Can someone comment on how broken it was to use 1 as exponent? Is it just a theoretical concern or does it mean the crypto could have been easily broken (as we saw with cryptocat yesterday)?
I'm asking because they seem to treat it like a minor issue:
It's of course questionable whether 1 (not a prime) was a good choice for the exponent to begin with, but it's hardly necessary to lose faith over this.
On a scale of 1-10, it was about a 15 on the broken scale. As explained in the comments on the patch, a public exponent of 1 means a private exponent of 1 (it's the inverse). This makes it completely, trivially broken.
So why don't the crypto libs at least throw a warning that "what you're doing is nonsensical" if not just reject an exponent of 1 outright? We hammer on people not to invent their own crypto, yet here someone is, using the RSA lib, and still doing it wrong.
Because crypto libs can't see the future. I'm struggling to think of a case where using cryptographic primitives should be bubblewrapped. Don't use the primitives if you don't know what you're doing.
In the RSA cryptosystem there are three really important numbers N (which is the product of two large primes N=pq), e (the public exponent) and d (the private exponent). The public key in an RSA scheme is <N,e> and the private key is <N,d>. The only bit of the private key that's private is d.
A large part of the security of RSA relies on you not being able to figure out d from N and e. Now the key element of RSA is that
de = 1 mod (p-1)(q-1)
So what happens if e = 1? You get
d = 1 mod (p-1)(q-1)
So you know the value of d. So you know the private key. So you can decrypt everything.
But you don't actually need to do that because what is RSA encryption? It's computing the following (c is the cipher text you'll transmit, m is the message being sent which is to be encrypted with the public key <N,e>)
c = m^e (mod N)
If e = 1 then that's
c = m (mod N)
but, oh wait, m is always < N so m (mod N) is just m, i.e.
c = m
i.e. the encryption does nothing.
Looking on the bright side an exponent of 1 does make RSA quite fast :-)
Also note that that quote from user madduck appears to be from a user of Salt in the issue tracker, not one of the project maintainers. It seems like they added a comment below and an edit to the first comment to clarify that.
It's amazing that this got through, you don't need to be a crypto expert for this to jump out as wrong. Anyone with undergrad number theory or crypto should easily be able to realize that using 1 is crazy.
I don't think you necessarily need special training. A value of 1 should immediately make you wonder why you are providing it as in input at all. Interestingly, even a small prime, like 3, has no known vulnerability (but it allows other vulnerabilities to be attacked faster)
Here's a ticket to track for following the state of crypto in Saltstack, https://github.com/saltstack/salt/issues/5913. It includes a lot of thoughts from the project founder on how to handle it.
55 comments
[ 2.5 ms ] story [ 117 ms ] threadEvery time I write something that involves some crypto I get absolutely terrified of what I'm likely to mess up. For that reason I almost always use an existing system or protocol. For example, on one of the projects I'm working on at CloudFlare we needed a secure connection across the Internet so I went with TLS 1.2. For another I needed the same thing but with better authentication: TLS 1.2 with client and server certificates. If I need to exchange encrypted messages in some store and forward style I'd likely use PGP.
And then for another project I needed to actually get something someone else had encrypted with RSA and decrypt it. Nightmare. If you find yourself calling low-level crypto APIs start to worry.
I'm really hoping crypto 2 actually starts in about 10 days as it seems to have been put back a lot.
--edit-- Just to sell it further, it also goes into padding oracles, timing attacks and a variety of clever stuff to show you just how easy it is to screw up :)
My mind was blown on every course video.
Good!
" For that reason I almost always use an existing system or protocol."
Excellent! If only every was like you we wouldn't have these problems.
The key to understanding this is to understand that cryptography is not 'writing some crypto code'. No part of cryptography has anything to do with the code. Cryptography is about math - feel free to do some cryptography math and work on a mathematical algorithm. Once you have a perfect algorithm that has been peer reviewed, then you can transcribe that mathematical algorithm into code.
Failing to understand the math first means you have already failed and simply should not have even begun.
Cryptography: math first, code later. Failure to follow this sequence means you will be ridiculed and any excuse is simply not good enough.
That's just how it is, and if I had to highlight one sentence from your comment it would be "Failing to understand the math first means you have already failed and simply should not have even begun."
I wish people would understand this instead of thinking trial and error works for cryptography. I mean, heck, most of us learned to code through trial and error, and I can understand why people think they can learn cryptography the same way, but that is not true, and a very toxic mindset.
> No part of cryptography has anything to do with the code.
There have been successful attacks against cryptography based on attacking the implementation. For example if two code paths take different time to execute, it is possible that this leaks information.
This is another reason why you should use a well honed library implementation of cryptography, for example OpenSSL.
This does of course not imply that the crypto–the maths of it–is broken. One just needs to be aware that it is not, in fact, sufficient to implement the maths correctly.
Generally, however, nearly all failures of security come from misunderstanding the math when creating the implementation. So I'd say that at least ensuring anybody writing crypto has a completely understanding of the crypto math before touching the code is a very good first step in getting somewhere.
# TODO add crypto
vs
# TODO get someone to review the crypto later
Do you or others have any recommendations for cross-platform library code (at least OSX, Windows, Android, Linux, Windows 8) for this purpose?
I may have understood incorrectly, but I thought that using gpg from code required launching a sub-process (e.g. via gpgme). That makes me very nervous from a library point of view, perhaps I'm wrong to be nervous.
Is the answer still "use PGP/GPG" or is there somewhere else I could look to avoid home-made cryptosystem here?
If you want the PGP-level stuff then there's GPGME: http://www.gnupg.org/related_software/gpgme/index.en.html
And I think GPGME is a lib which "under the hood" spawns a command-line gpg process? http://www.gnupg.org/faq/GnuPG-FAQ.html#cant-we-have-a-gpg-l...
At the moment it looks like my choice is between implementing a cryptosystem on top of crypto primitives (from openssl or libcrypt) or seeing if I can get away with shipping a lib which spawns child processes via gpgme.
I think a major issue is the lack of understanding of best-practices with regards to security. So we tend to rely on the security of systems that aren't battle-proven. But even battle-proven solutions still have vulnerabilities, so we need to rely on multiple layers of security.
For me at least, this reinforces the lesson to put systems as much as possible behind multiple layers of security such as firewalls and VPN. And I should be careful to architect salt systems in a hub/spoke topology using salt-master and syndics. And encrypt any traffic from the salt-master to groups of servers using ssh tunneling. Looks like pyzmq now supports ssh tunneling, so maybe this would work:
http://zeromq.github.io/pyzmq/ssh.html
So this bug was found and fixed by a proper professional audit. Neat.
May be I'm old-school but when a project goes over certain size and there's no prominent "security" section (as important as downloads, IMHO), that's a red flag for me.
This is the way you do it:
- http://httpd.apache.org/security_report.html
- http://openssh.org/security.html
- http://nginx.org/en/security_advisories.html
All projects doing sensible tasks have a security history. Don't hide it, make it public and accessible to your users.
[1]: http://docs.saltstack.com/topics/releases/0.15.1.html#rsa-ke...
https://blog.crypto.cat/2013/07/new-critical-vulnerability-i...
https://news.ycombinator.com/item?id=5989707
Crypto is hard, and you shouldn't really be making these choices.
>> I shouldn't have to know about the chinese remainder theorem to use crypto properly.
Then stay out of it. Choose a library/framework that is popular, well tested and high level enough that you don't have to make these choices.
NaCl in particular provides simple Public/Private and Symmetric modes. The Authenticated Encryption Symmetric mode here is good - http://nacl.cr.yp.to/secretbox.html
Yes, and if you're going to use low level constructions he's right, and that will never change.
>> This is a valid complaint and "don't do it" isn't a valid solution.
Except that's not what I said, I said find a higher level library. When he asked I gave examples.
>> Too many "libraries" consist only of implementations of the crypto algorithms, but don't give any help with the proper use.
Then these may be the wrong libraries to use to secure your application. Great libs to poke around in the lower levels for a variety of other purposes of course.
On the flip side this should have set off warning bells for them - it can be a little bit of a code smell to pick 1 for a configurable value in your encryption. Not always a bad choice, but 1 has a lot of properties that can simplify various mathematical operations and Basically setting a crypto value to the smallest positive integer (and a non-prime at that!) is like deciding that a set of numbers in your code should always be represented by two-byte signed ints. Sometimes that's _exactly_ what you want (say, various fast math libraries), but if you're doing general purpose code you should really stop and think about whether you might end up with a value outside that.
This attitude scares me. Some things really are hard to do. Some things really are hard to understand. Some are even both. Remember what Euclid is supposed to have said to Ptolemy I of Egypt when the king complained that geometry was hard to learn: There is no royal road to mathematics!
There's no shame in not being able to do or understand every difficult thing under the sun, but don't complain that such things "shouldn't be so hard". They often are. I think it's part of what makes life interesting.
Of course cryptography is hard, and it is an interesting topic (one I know a few things about as it turns out).
That's not even really a "security" concern, that's just good software engineering. Passing this function something that fails the prerequisites ought to be checked. (And yes, I'd check that it's prime, too. It's an expensive check once, but trivially memoized, and you can even prefill the memoization with the usual 65537 value to make it even cheaper. In this particular case, IIRC, the exponent is always or nearly always simply reused,)
I shouldn't have to know about gasoline to drive a car.
What's wrong with these statements?
The universe doesn't care that something "should" be easy.
A crypto library is the pilot. A crypto library that allows you to use insecure settings is a pilot who crashes the plane into a mountain because you asked him to.
I'm asking because they seem to treat it like a minor issue:
It's of course questionable whether 1 (not a prime) was a good choice for the exponent to begin with, but it's hardly necessary to lose faith over this.
A large part of the security of RSA relies on you not being able to figure out d from N and e. Now the key element of RSA is that
So what happens if e = 1? You get So you know the value of d. So you know the private key. So you can decrypt everything.But you don't actually need to do that because what is RSA encryption? It's computing the following (c is the cipher text you'll transmit, m is the message being sent which is to be encrypted with the public key <N,e>)
If e = 1 then that's but, oh wait, m is always < N so m (mod N) is just m, i.e. i.e. the encryption does nothing.Looking on the bright side an exponent of 1 does make RSA quite fast :-)
PS Test code for those interested showing the same N with exponents 1 and 65537 and effect of encryption: https://gist.github.com/jgrahamc/5933984
The 1 mod (p-1)(q-1) comes from Euler's theorem and (p-1)(q-1) is how you calculate the totient for numbers that are the product of 2 primes p and q
Item 4 here: https://en.wikipedia.org/wiki/RSA_(algorithm)#Key_generation