Ask HN: how do you manage encrypted email?
1) Reading encrypted mail that I sent. If I need to remind myself what I said to someone, or recover an attachment, etc., I can't, because the only copy of my message is encrypted with the recipient's public key. I could work around this by Bcc'ing myself on every message, but that would have the mildly annoying effect of duplicating all my outgoing messages; every time I were to look for a message I sent to Mr. X, I'd get two results, and I'd have to figure out which one was encrypted with my key to read it.
2) Search. The more serious issue is that I can't search encrypted email, whether I sent it or received it. It is conceivably possible to search mail encrypted with my public key by decrypting it before running the search (though not encrypted mail that I sent, pending a good solution to problem 1). However, that seems like it would be extremely slow in practice, and I am not aware of any software that would make this simple or practical.
I am currently using Mutt as a client for a local Maildir, with offlineimap and notmuch to download and index my mail, but I am willing to switch to a different setup if there is one out there that solves these problems. What's frustrating is that both problems seem pretty obvious, and solutions are conceivable, but search results for practical solutions elude me.
I don't want to recommend to others that they use PGP if it means they cannot reasonably search their mail archive and read their sent messages. What should I tell them to do?
11 comments
[ 4.9 ms ] story [ 34.7 ms ] threadhttp://stackoverflow.com/questions/597188/encryption-with-mu...
The tools to do this simply don't currently exist, but GnuNet has developed an approach to convergent encryption of keywords which points to how one might implement this in an email system.
The first step to do this would be to run the email through an extractor. It might do things like extract keywords from the email text (perhaps all non-common words and their frequency), attachment file names, MIME types &c. This extractor would then encrypt or hash these keywords and add an entry into the index for each keyword/email ID pair.
As a simple example, let's take your first paragraph. It might reduce down to the following words/weights:
((convince 1) (couple 1) (decided 1) (effort 2) (encryption.html 1) (fairly 1) (http://www.ocf.berkeley.edu/ 1) (http://www.ocf.berkeley.edu/~rwl/ 1) (http://www.ocf.berkeley.edu/~rwl/encryption.html 1) (linked 1) (make 1) (others 1) (pgp 2) (problems 1) (quickly 1) (recently 1) (regularly 1) (run 1) (signature 1) (solve 1) (too 1) (use 3))
You might then hash each element with your keyword hashing key, to get the following:
(("bea916398ce5fafc4497d378fc4555d9" 1) ("c81dc4aa3b1ae95487f288b297f51a6b" 1) ("8b801b4290ce10372b400330c737858b" 1) ("80b3e9090ee572563e67ebdecdb9b936" 2) ("cdd4c4efda80451e4dd1ff340dd2ed17" 1) ("9d4bcdc88159f8d830384f1696c32c19" 1) ("cb7baeb2e45e5e9d69fcf37c4df25669" 1) ("707f687e23bbbce127970361fd6bda10" 1) ("e6656976f2c6944f7aefd16b5786d6cd" 1) ("df9c284344194e1887e56478ec655dda" 1) ("ec8e0c526c105fc8ebb1ebdf293cf717" 1) ("f2d031d514795444be32f18690f6961f" 1) ("270273a76f0d974c2a06f96d40e2430a" 2) ("46bd41b18d2de638bbd3c126cb4ed716" 1) ("d96c629e01788ed763529620f308bd0c" 1) ("2d5fe4b610334160f178a1977b7d1db1" 1) ("211d0c274e8fb59bbcfe6e74241c317a" 1) ("ae5360545c4e008ed683df468136a8b0" 1) ("d6a5babaa2da167ef4548e7c34b49f40" 1) ("adcf3cd322f7ce16ad015f4725ddbda8" 1) ("e7e89d9c445263f9835c483aabf4f453" 1) ("84f0d11faa98a86f9624df86bd71901d" 3))
And then you'd store your document ID (in this case, the paragraph ID) and the weight in the index under each keyword.
To search for "use PGP", you'd extract the keywords "use" and "pgp", then hash them with your hash key to produce "84f0d11faa98a86f9624df86bd71901d" and "270273a76f0d974c2a06f96d40e2430a", look those up in your index, see that paragraph ID and the weights 3 and 1, retrieve the paragraph, decrypt it with your decryption key and you're in business.
Interestingly, this same architecture works securely if the index lives on another system, so long as the keyword hashing is executed on your system and your hash key is itself secure. Thus, there's no good reason that Google couldn't offer keyword-based searching on messages that only you can decrypt—no reason except that they do not wish to.
> The tools to do this simply don't currently exist, but GnuNet has developed an approach to convergent encryption of keywords which points to how one might implement this in an email system.
What you describe sounds a lot like "fully homomorphic encryption" (given my pretty amateur understanding of that term). Is the technique you describe one that could be layered on top of PGP? Or does it need an entirely different crypto scheme?
Well, not really from a technical standpoint. Homomorphic encryption is about me encrypting 1 and 2, handing you #x8495 and #x7835, you executing (homomorphic-add #x8495 #x7835) to get #x1932 and handing it back to me, whereupon I'm able to decrypt it to get 3.
You can imagine why this could be cool: there's the potential for someone to be able to perform work on one's behalf without compromising one's privacy (imagine TurboTax working on encrypted data, with Intuit unable to know what one's finances are).
The technique I describe is different, but it does enable a similar end result: someone else can run an indexing/search engine on my behalf, and I can consult it to search my data without ever revealing that data to the engine-runner. The engine isn't really able to perform any calculations though.
> Is the technique you describe one that could be layered on top of PGP?
Well, it would work in parallel to PGP (or any other data-at-rest scheme). PGP would encrypt the plaintext, and this would be used to index the plaintext.
GNUnet uses this technique so that two different people can arrive at the same encryption for keyword searches (which of course has vulnerabilities, since one's adversary can perform those same searches); I use it so that the same person at two different times can index and search by keyword, and protect against an adversary through the use of a secret hash key.