From the very top of the original disclosure webpage:
"What leaks in practice?
We have tested some of our own services from attacker's perspective. We attacked ourselves from outside, without leaving a trace. Without using any privileged information or credentials we were able steal from ourselves the secret keys used for our X.509 certificates, user names and passwords, instant messages, emails and business critical documents and communication."
So they explicitly confirmed they were able to get private key information. I've seen similar confirmations on twitter.
I think it would be extremely unwise to trust that your keys are safe based on assumptions about the malloc implementation. And no offense to the OP, but I think the standard for "don't worry" on this issue is far higher than this blog post.
On the other hand, it is likely that the discoverers of Heartbleed, during their testing, were changing their service configs and restarting the SSH/Apache/other daemons frequently in order to test as many configurations as possible.
It's entirely possible then that the OP's assertion is mostly correct, and that your key is more likely to be leaked if the attack is one of the first few requests to the daemon after it is started and the heap has been barely touched, and is significantly less likely to be leaked if the daemon has been up for some days and served many requests and its heap is more fragmented.
Though I'm skeptical of the final "not possible", and would personally downgrade that to - at most - "very unlikely". It'd be nice to see some kind of study into how frequently the private key data is recoverable from the attack, against how many "normal" requests preceded the attack request.
So, I've managed to obtain private keys from a test copy of Apache on Linux - very unreliably but it works sometimes - and I can't actually seem to get it to work at all with a freshly-started Apache. As far as I can tell, there actually needs to be a certain amount of prior heap activity before the attack stands a chance of working. Whilst I haven't analyzed exactly what happens, this particular attack is intended to look for keys in the format OpenSSL uses to store them permanently in RAM.
Wait, what? Didn't someone just post to Twitter this morning that they'd successfully recovered private keys from FreeBSD + Apache?
It's true, private keys are one of the harder things to pull out of OpenSSL due to the memory lifecycle of private keys. But this blog posts goes too far with that analysis.
I'll confess I don't see the purpose of this post -- it contains no actionable information (you certainly shouldn't choose not to update your keys based on this analysis that it's somewhat unlikely for a key to leak). And as jasonwatkinspdx points out, multiple people have confirmed that they've been able to extract keys.
Memory allocation analyses like these can be quite interesting, but this one seems rushed, potentially dangerous (if it encouraged anyone to not replace their key), and not particularly insightful.
Until there is final proof and production of sufficient evidence, this post is as valid as the claims that information was extracted.
Whilst I completely agree that if taken as a reason not to update this is unhelpful, the OP isn't enumerating reasons to not update, he's providing early assessment and commentary.
As an industry we've once again not done ourselves any favours with the way this has been handled in the media - it started out as the apocalypse of SSL and all we hold dear, and whilst the update exercise was valid, it appears that the percentage of affected systems was actually quite low and the extent of the apocalypse might well have been overstated.
That's not to say there isn't a problem here, by any means - I've spent all day scanning servers and quietly weeping as I watched usernames and passwords dance around my screen - but getting to the analysis of what actually happened should have started before the headlines ran away with themselves.
I think this misses the point. The unchecked copy won't just copy freed memory, but will rather snapshot up to 64k of the heap, by issuing multiple malicious heartbeat packets and varying their size an attacker can probably build a fairly comprehensive view of the heap.
it might be interesting to consider storing keys out of the heap
Stack is traditionally accessible too. So you mean global/static memory? What languages would let you edit that? Can you get a pointer to it in C and change things, or is it stored in protected memory blocks with the program text?
You seem to suggest that the snapshot of the heap gets not just freed sections of the heap but the whole heap? If so then my other idea is invalidated (allocate on the heap a chunk of memory large enough for the keys and never free it, just re-use it when needed).
Exactly, this fails to understand a fairly fundamental aspect of the attack: you can read past the end of the buffer. Looking at the OpenSSL code, it appears the size of the buffer you're reading from is 16KiB+a few hundred bytes. (I could be wrong - not 100% sure I've followed the code correctly - but it's not going to be larger than that for various reasons.) You can read up to 64KiB-1 bytes from it.
a) by writing down your private key secrets (openssl rsa -text -in ./private.key) on a paper or in a python keyhunt.py and heartbleeding a real server (initialized with a known private key) few times and then inspecting the output (requires a bit of patience) [X]
b) by running openssl in a debugger and understanding what happens with the crypto secrets in a real live run (requires a bit of reversing) [X]
c) by running the openssl under e.g. valgrind and checking how the allocation of different forms of the crypto secrets really works (requires tools) [X]
d) by modifying the openssl CRYPTO_malloc() code to initialize crypto buffers with a recognizable patterns and then heartbleeding your test server (requires a bit of coding) [X]
e) reading the code and seeing how all those intermediate forms of the private key are handled (and reading it again, and again until you are certain of your conclusion) (requires skill) [X]
f) by reading from the web accounts and results of people who have seen the private keys (requires trust) [X]
g..z) by something else ... [X]
Then post your how many X's you had to go through before you see the leak. Good luck! :D
I didn't get far before it was seeing the leak time for me, but then I cheated and saw most of those done anyway, was painful but fun. :)
14 comments
[ 4.7 ms ] story [ 19.6 ms ] thread"What leaks in practice?
We have tested some of our own services from attacker's perspective. We attacked ourselves from outside, without leaving a trace. Without using any privileged information or credentials we were able steal from ourselves the secret keys used for our X.509 certificates, user names and passwords, instant messages, emails and business critical documents and communication."
So they explicitly confirmed they were able to get private key information. I've seen similar confirmations on twitter.
I think it would be extremely unwise to trust that your keys are safe based on assumptions about the malloc implementation. And no offense to the OP, but I think the standard for "don't worry" on this issue is far higher than this blog post.
It's entirely possible then that the OP's assertion is mostly correct, and that your key is more likely to be leaked if the attack is one of the first few requests to the daemon after it is started and the heap has been barely touched, and is significantly less likely to be leaked if the daemon has been up for some days and served many requests and its heap is more fragmented.
Though I'm skeptical of the final "not possible", and would personally downgrade that to - at most - "very unlikely". It'd be nice to see some kind of study into how frequently the private key data is recoverable from the attack, against how many "normal" requests preceded the attack request.
It's true, private keys are one of the harder things to pull out of OpenSSL due to the memory lifecycle of private keys. But this blog posts goes too far with that analysis.
Memory allocation analyses like these can be quite interesting, but this one seems rushed, potentially dangerous (if it encouraged anyone to not replace their key), and not particularly insightful.
Whilst I completely agree that if taken as a reason not to update this is unhelpful, the OP isn't enumerating reasons to not update, he's providing early assessment and commentary.
As an industry we've once again not done ourselves any favours with the way this has been handled in the media - it started out as the apocalypse of SSL and all we hold dear, and whilst the update exercise was valid, it appears that the percentage of affected systems was actually quite low and the extent of the apocalypse might well have been overstated.
That's not to say there isn't a problem here, by any means - I've spent all day scanning servers and quietly weeping as I watched usernames and passwords dance around my screen - but getting to the analysis of what actually happened should have started before the headlines ran away with themselves.
it might be interesting to consider storing keys out of the heap
You seem to suggest that the snapshot of the heap gets not just freed sections of the heap but the whole heap? If so then my other idea is invalidated (allocate on the heap a chunk of memory large enough for the keys and never free it, just re-use it when needed).
How this bug actually ends up dumping information really depends a lot on luck.
https://news.ycombinator.com/item?id=7561901 Maybe CAs need to revoke their master keys?
This is not a competent analysis of the situation. Flagged.
Private key comes (as a surprise) in many forms, including:
1) base64 encoded version
2) decoded version
3) "rocket science" parts (n, p, d, ...): openssl rsa -text -in ./private.key | grep "^[a-z].*:$" -> modulus:, privateExponent:, prime1:, prime2:, exponent1:, exponent2:, coefficient:
You may assess the leak potential by yourself:
a) by writing down your private key secrets (openssl rsa -text -in ./private.key) on a paper or in a python keyhunt.py and heartbleeding a real server (initialized with a known private key) few times and then inspecting the output (requires a bit of patience) [X]
b) by running openssl in a debugger and understanding what happens with the crypto secrets in a real live run (requires a bit of reversing) [X]
c) by running the openssl under e.g. valgrind and checking how the allocation of different forms of the crypto secrets really works (requires tools) [X]
d) by modifying the openssl CRYPTO_malloc() code to initialize crypto buffers with a recognizable patterns and then heartbleeding your test server (requires a bit of coding) [X]
e) reading the code and seeing how all those intermediate forms of the private key are handled (and reading it again, and again until you are certain of your conclusion) (requires skill) [X]
f) by reading from the web accounts and results of people who have seen the private keys (requires trust) [X]
g..z) by something else ... [X]
Then post your how many X's you had to go through before you see the leak. Good luck! :D
I didn't get far before it was seeing the leak time for me, but then I cheated and saw most of those done anyway, was painful but fun. :)