Most systems never found a good way to encrypt data at rest.
You either have to let the user manage the keys / passphrase, giving up things like search and password recovery, or you have to manage the keys in a service, which doesn't provide privacy from the service that manages the keys.
A few years (almost two decades!!!) when the filters weren't as good I read a proposal to make an email system that charges $0.01 to the sender. Spammers would be bankrupt.
It was difficult to implement and up against an already entrenched tech.
With Bitcoin I've been thinking of that idea again. Could a payment system on top of existing email be created? The receiver of the email could get the senders penny and Bitcoin might get a killer app that doesn't involve drugs...
It could be even more convenient, if it only charged for unauthorized contacts. Emails back and forth between friends/coworkers? No charge. Email newsletter you signed up for? One-time cost to them to send you the opt-in message. Random spammers? Never get authorized; have to pay every time.
It would also put a slight amount of friction in place for contacting people out-of-the-blue (when e.g. sending fan-mail to famous people)—but the fact that some stranger paid $0.05 to say something to you could also make you just fractionally more interested in what they have to say.
On the other hand, for accepting bug reports, help tickets, etc., there'd have to be some kind of email equivalent to an "800 number", that is free to initiate contact with. Either that, or a lot of behind-the-scenes pairing magic to get your email address into the directory service of all the products and services you use.
The difference, I think, is that you can at least apply individual filtering to the problems of annoying aunts and mass-emailing friends, of the same sort as you see with Facebook's "remain friends but unfollow" option.
Spam is hard because there's no manual filter you can write that targets "everyone you don't know", without removing the ability to be reached-out-to by real people who have a genuine desire to tell you something you care to hear.
Or use Hashcash, which is a proof-of-work algorithm created for anti-DoS and anti-spam, and the inspiration of Bitcoin's mining algorithm.
It's possible the botnets are strong enough to create Hashcash or mine Bitcoin and still spam enough to annoy us... and it would probably take either Google, Yahoo, or Microsoft announcing they would stop accepting legacy emails (where "legacy" could just be ones without Hashcash or BTC), for anyone to make a change.
SMS messages are somewhere around $0.01 to $0.04 for sending in bulk, depending on region. Doesn't stop spammers.
And a fee will also never, ever fly for email, being not a matter of entrenched tech, but rather one of artificial scarcity, being the reason for why SMS is dying.
But why would you need the sender's key? They need to encrypt with your public key, which you only provide to a limited list, and change when it gets blown. What we need is an automated way for requesting someone's key, approving or disapproving that request, and then storing their key for future use....in gmail.
Most spam isn't encrypted. The overhead's too high.
Behavioral methods might still work, particularly whitelisting/greylisting. Whitelisted senders will be passed, greylisted ones filtered. Spam can still be filtered locally with a key, if necessary, though that misses the leverage of delivery-time notification.
A notify-and-fetch dynamic might also work. Rather than delivering messages at send time, a notification that there is a message is sent, an on its strength, the message is later requested. Spammers have a much larger mail storage problem. Whitelisted senders might have mail delivered automatically. And collaborative spam filtering could mark and downrate spammer reputations before many recipients get to requesting the message, sparing them the spam.
Greylisting / teergrubing / delayed receipt (even just 1-2 retry intervals) cuts down much on spam as well.
Play the costs.
There are also distributed anonymous reputation systems, designed mostly for Tor / Web, though similar concepts for email might work. FAUST and Fair Anonymity:
A Spamhaus DNSBL alone drops 99% of spam with almost no FPs. Add DROP, fullbogons, and greylisting to increase the percentage further. None of these methods read the message body, and there are plenty of other options as well. I self-host and receive about 5 spam messages per year, all without looking at the message body.
Do you have any cites for this? Everything I've seen suggests that Zen has a very low false positive rate. That's the point of Zan - block a decent amount of spam and no real email so that your slower better filters can work.
It's worth noting that spamhaus had no real reason to do this as they had already blocked the hosting company a2b was providing bandwidth to, so blocking all of a2b didn't actually stop any more spam.
Zen is far far from not blocking any real email, they regularly block large ISPs that refuse too indiscriminately drop customers on their request.
> Update: Spamhaus has now commented on the case in a blog post, explaining how Cyberbunker (also known as CB3ROB) has been involved in hosting malware and phishing sites. Ironically, this very week Spamhaus has been working with the Dutch hi-tech crime unit to investigate the criminal activity hosted by Cyberbunker and routed by A2B.
This is really your example of false positive blocking?
A2B was routing CB, CB was not using A2Bs IP space. Spam was not coming from A2B. A2Bs IP space was blocked by Spamhaus to force them to drop CB.
What exactly justified blocking A2Bs IP space here?
Spamhaus is a really shady organization with a history of unethical behavior, look up the nic.at fiasco as another example... Or just ask anyone who works in hosting.
> Per Spamhaus policy, on October 6th, after notifying A2B several times since June without results, an SBL listing which A2B had been ignoring was escalated to the SBL's "providing a spam support service" category and increased to include one of A2B's IP ranges. The escalated SBL record SBL112638 listed 178.249.152.0/21 for providing routing "knowingly and for profit" to a rogue host known as "CB3ROB" or "Cyberbunker", an outfit which Spamhaus has long seen involved in hosting cybercrime and spam outfits. SBL listings of CB3ROB had been mounting steadily during 2011 for hosting malware, phishing and websites selling fraudulent goods advertised via spam. CB3ROB had announced that it would not terminate customers due to spam listings - an announcement which sent a golden invitation to even more spam and crime customers to the point where all of CB3ROB was placed on the Spamhaus DROP ("Don't Route Or Peer") list at the beginning of October.
That's exactly what people want from something like the SBL, which is why it's used by so many people.
Your reference to nic.at is weird - again they were providing services to a bunch of criminals.
When complaining about false positives it'd be really helpful if you could point to people who aren't fucking scumbags who are getting blocked.
Make encrypting an email cost something. not necessarily money. kinda like minimg bitcoints but instead of getting bitcoins, you get to encrypt and send an email.
As in, each email requires a bruteforced hash that takes about half a second to calculate on a desktop CPU, before it will be accepted by the mail server?
That's actually the best idea for spam reduction I've heard so far (which is not really saying a lot). Half a second won't be noticed by a normal user, but it will be noticed by people sending thousands or millions of emails.
Sure Spammer will install large GPUs hashing rigs and still manage to send thousands of emails per second, but it would cost them a lot more than it does now.
One problem will be phones, a hash that takes about half a second on a desktop will take tens of seconds on lower end mobile phones, which is hardly unworkable but you could offload the calculations to a 3rd party server (with a large GPU based hashing rig) for say a fraction of a cent, especially if the email contents were encrypted by the phone before hashing.
And it could be introduced without breaking the email system. Initially the presence of a hash will just be used as an additional spam filtering signal, but as support grows over time you can make it harder and harder for emails without a hash to get through.
Like bitcoins, you would need a mechanism to increase the required work over time, though it could just be ad-hoc based on what mail server operators choose to accept.
You may want to take a look at hashcash[1]. It was a proof of work scheme for email invented in the late 90s, and they claim it was adopted by bitcoin[2]. (Sadly, it never caught on for its 'intended' purpose.)
The problem I can see there is with legitimate large mailing lists. Think security announcements, etc. Putting a burden like that on smaller teams, or teams without a lot of funding, could prove problematic.
You could deal with large regular mailing lists, you want to move to a system where users can subscribe to a signed mailing list and have the mail server remember that subscription.
But yeah, irregular mailing lists where you suddenly need to tell your million users that your database has been breached and they need to change their passwords will be an issue.
But we are talking about $0.0005 (0.05 cents) worth of hashing on CPU systems per email. With GPU based hashing it should be 50 times cheaper.
If your team has 1 million customers, and can't afford the $10 to pay a GPU based hashing service to hash 1 million emails for you, you might have a bigger problem.
I suspect the biggest issue for such a scheme is $10 for 1 million emails is potentially still profitable for spammers while being at the limit of how much cpu time you can expect to waste for regular (particularly mobile) users. And that's before you consider the possibility of ASIC hashing for emails.
monetary costs can never be imposed on spammers, if we can move money on that scale along with email we've solved bigger problems!
what other costs can we impose? the email lists that spammers harvest stay valid for a long long time, what if they were invalidated faster? to have a high hitrate on their lists they'd need Google scale crawling capabilities, which kinda then defeats their point of trying to make a living from spam.
why are we converting long term relationships with servers(email id and password to login) into relationships with every Tom on the Internet(no control over who can reach you, hence you are in a relationship whether you like it or not)? Do we still have to remember emails or phone numbers, aren't they names in an address book already? why don't we just move off email, use a new system that uses long, hidden alphanumeric ids to identify relationships between people that can be invalidated by either party?
As mentioned by other posters, it opposes convenience too much. I say it's unfortunate that SMTP/IMAP was not made easy/free enough to self-host. I understand cost, uptime, and admin challenges, I am just saying these communication mechanisms will continue to be centralized while there is financial incentive to make them better and appeal to the masses.
Self-hosting is not a solution, in particular because it simply pushes the "entity you have to trust" from Google to your VPS provider or residential ISP.
A reply disagreeing got deleted, but in case anyone else is wondering, you're trusting your ISP because SMTP's STARTTLS, the most standardized, widespread email "encryption", is basically worthless[1] if you're really paranoid. It's not required, so if you try to require it you're going to be unable to receive/deliver some email. It's negotiated over plaintext, so ISPs in between can just make the two servers think each other doesn't use it. I believe there's also some confusion in the standard, or at least widespread bad implementations, about how to verify the certificate, so an attacker can probably MITM the cert too.
You can of course encrypt the mail yourself with GPG etc, but that would make self-hosting, the topic of this sub-thread, less relevant, and nobody but your fellow cryptonerds will do it.
It's all about ease of use. https is probably the only user-friendly / good UX crypto solution in common/widespread use (unless you count transparent drive encryption), and it achieves this in part by being centrally controlled via registrars to eliminate the need for the user to understand PKI. This in turn makes it kind of a joke from a crypto-snob POV, though it's good enough for most routine stuff.
Even MacGPG plugins for Mac Mail.app are really hard to install, manage, and use, and require a level of expertise and knowledge of how PGP/GPG works that makes them off limits for non-technical users.
OTR is great if all endpoints have the same software clients (e.g. client-side IM). Doesn't necessarily work so great if your clients are all "in the cloud" though.
Recipents basically receive a password from you in another channel (email, IM, phone convo, face to face) and can use it to decrypt the email from TutaNota when they receive it. I tried it out myself and the only clunky thing is that they load up their whole web app just to read one message, but it's kind of a clever piece of marketing because the web app works pretty well so the recipient who doesn't use encryption gets exposed to an interface that's just as easy to use as what they're used to.
For everyone using TutaNota, the decryption is transparent which definitely is encouraging.
I've seen some people sending around credit card info or social security info and forcing the intended recipient to use a password when receiving that info is a small price to pay for piece of mind. It's almost criminal to send signatures on documents and social security/credit card info in plain-text!
Didn't know about them before, looks interesting. I've managed to find their site - https://tutanota.com - even though I was searching for Tutanova (should have just used copy & paste. The name is indeed not easy to remember...
A few questions remain:
* How do they 'encrypt emails to anybody'?
* Where do they encrypt - locally or on the server?
My incoming mail goes through SpamAssassin at the IMAP server, which does a decent job. All my mail addresses are public on web sites, so there's a lot of junk filtered out.
My phone also uses the IMAP server, so I don't need any Google services. It's an Android phone, but it's never been logged into Google, wasn't bought from a carrier, and many of the built-in Google apps have been removed. (Hint: when you start up a new Android phone, and it wants you to log into Google, click "Later". Then remove the "Google One Time Startup" app to shut up that message.)
Don't use standard e-mail (gmail or otherwise) for anything that needs guaranteed privacy. Once you send an e-mail, it is not within your control what happens to it or who sees it.
That's why, for example, in healthcare -- messages are commonly wrapped in secure message "containers". The e-mail simply has a link that takes you to a third party where the actual message is stored, encrypted, has an expiration date, and you must authenticate to view it.
Convenient? No. But privacy always comes at a cost of some sort.
Aka, the "second-hand smoke" effect of gmail (or any similar service). Even if you are fine with giving your data Goggle (or Facebook, etc), it's a problem if your forcing that decision on others.
The solution is probably using email in a manner closer to how it was originally designed: many local MTAs instead of a handful of centralized MTAs.
Here's a product idea, which could make someone a lot if they can find a way to implement it: a home server that is a minimal, secure, locally contained, turn-key home email MTA (with some localhost-only IMAP/webmail interface). You sell the server (and maybe a 1-time service to help people set it up).
This server would have to include a "choose your vendor, enter your credit card, click subscribe" super-easy interface to let people subscribe to vendors (multiple, if desired) that provide lower-MX-priority backup MTA services. (perhaps with an optional "hidden MTA" option, where your home server and lowest MX is only visible to the service, so the home router only connects to know hosts, IFF that is something that the customer wants.
People already understand local-only delivery because it emulates snail-mail. If you can let people receive messages and data unattended and asynchronous, so they can run the server when they want and receive their data directly (nobody else involved except for the backup services of their choice, then you have a product that will sell. It doesn't even need to have to have great uptime (that's what multiple MX records are for) or performance.
It's Aral Balkan's analogy, from his talk[1] that I strongly recommend showing during That Talk. His presentation is amazingly approachable for a broad audience. Even my not-interested-in-tech mother has started to understand these issues specifically because of Mr. Balkan's explanation.
Configuring and maintaining email servers is a mess. Most of the people are just outsouring it. There is a lack of simple to use free tools to provide self hosted email.
55 comments
[ 3.9 ms ] story [ 121 ms ] threadYou either have to let the user manage the keys / passphrase, giving up things like search and password recovery, or you have to manage the keys in a service, which doesn't provide privacy from the service that manages the keys.
It was difficult to implement and up against an already entrenched tech.
With Bitcoin I've been thinking of that idea again. Could a payment system on top of existing email be created? The receiver of the email could get the senders penny and Bitcoin might get a killer app that doesn't involve drugs...
It would also put a slight amount of friction in place for contacting people out-of-the-blue (when e.g. sending fan-mail to famous people)—but the fact that some stranger paid $0.05 to say something to you could also make you just fractionally more interested in what they have to say.
On the other hand, for accepting bug reports, help tickets, etc., there'd have to be some kind of email equivalent to an "800 number", that is free to initiate contact with. Either that, or a lot of behind-the-scenes pairing magic to get your email address into the directory service of all the products and services you use.
For me there's an annoying aunt who periodically send the latest get-mad-at-this email.
Or, how many times do we receive a mass email from friends who use CC instead of BCC and then we're flooded by everyone's replies?
If you're in my address book, then you should be charged more!
Spam is hard because there's no manual filter you can write that targets "everyone you don't know", without removing the ability to be reached-out-to by real people who have a genuine desire to tell you something you care to hear.
But seriously, does it still fail if I have a local filter were I strongly prioritize emails of ppl who show up on the BC ledger?
If ppl in high tech start doing this to lower their own email burden, I can envision normal business ppl copying the technique to lower their own.
It's possible the botnets are strong enough to create Hashcash or mine Bitcoin and still spam enough to annoy us... and it would probably take either Google, Yahoo, or Microsoft announcing they would stop accepting legacy emails (where "legacy" could just be ones without Hashcash or BTC), for anyone to make a change.
And a fee will also never, ever fly for email, being not a matter of entrenched tech, but rather one of artificial scarcity, being the reason for why SMS is dying.
I'm not proposing paying a third party to send me email. You'd be paying me for my scarse resource - my time and inbox space
Probably not be that hard to sort out.
Behavioral methods might still work, particularly whitelisting/greylisting. Whitelisted senders will be passed, greylisted ones filtered. Spam can still be filtered locally with a key, if necessary, though that misses the leverage of delivery-time notification.
A notify-and-fetch dynamic might also work. Rather than delivering messages at send time, a notification that there is a message is sent, an on its strength, the message is later requested. Spammers have a much larger mail storage problem. Whitelisted senders might have mail delivered automatically. And collaborative spam filtering could mark and downrate spammer reputations before many recipients get to requesting the message, sparing them the spam.
Greylisting / teergrubing / delayed receipt (even just 1-2 retry intervals) cuts down much on spam as well.
Play the costs.
There are also distributed anonymous reputation systems, designed mostly for Tor / Web, though similar concepts for email might work. FAUST and Fair Anonymity:
https://gnunet.org/node/1704
http://arxiv.org/pdf/1412.4707v1.pdf
They make it near impossible for bigger hosts not to get blacklisted unless they fully comply with spamhaus.
https://www.virusbtn.com/blog/2011/10_13.xml
It's worth noting that spamhaus had no real reason to do this as they had already blocked the hosting company a2b was providing bandwidth to, so blocking all of a2b didn't actually stop any more spam.
Zen is far far from not blocking any real email, they regularly block large ISPs that refuse too indiscriminately drop customers on their request.
> Update: Spamhaus has now commented on the case in a blog post, explaining how Cyberbunker (also known as CB3ROB) has been involved in hosting malware and phishing sites. Ironically, this very week Spamhaus has been working with the Dutch hi-tech crime unit to investigate the criminal activity hosted by Cyberbunker and routed by A2B.
This is really your example of false positive blocking?
What exactly justified blocking A2Bs IP space here?
Spamhaus is a really shady organization with a history of unethical behavior, look up the nic.at fiasco as another example... Or just ask anyone who works in hosting.
That's exactly what people want from something like the SBL, which is why it's used by so many people.
Your reference to nic.at is weird - again they were providing services to a bunch of criminals.
When complaining about false positives it'd be really helpful if you could point to people who aren't fucking scumbags who are getting blocked.
That's actually the best idea for spam reduction I've heard so far (which is not really saying a lot). Half a second won't be noticed by a normal user, but it will be noticed by people sending thousands or millions of emails.
Sure Spammer will install large GPUs hashing rigs and still manage to send thousands of emails per second, but it would cost them a lot more than it does now.
One problem will be phones, a hash that takes about half a second on a desktop will take tens of seconds on lower end mobile phones, which is hardly unworkable but you could offload the calculations to a 3rd party server (with a large GPU based hashing rig) for say a fraction of a cent, especially if the email contents were encrypted by the phone before hashing.
And it could be introduced without breaking the email system. Initially the presence of a hash will just be used as an additional spam filtering signal, but as support grows over time you can make it harder and harder for emails without a hash to get through.
Like bitcoins, you would need a mechanism to increase the required work over time, though it could just be ad-hoc based on what mail server operators choose to accept.
[1]: http://www.hashcash.org/ [2]: http://www.hashcash.org/bitcoin/
But we are talking about $0.0005 (0.05 cents) worth of hashing on CPU systems per email. With GPU based hashing it should be 50 times cheaper. If your team has 1 million customers, and can't afford the $10 to pay a GPU based hashing service to hash 1 million emails for you, you might have a bigger problem.
I suspect the biggest issue for such a scheme is $10 for 1 million emails is potentially still profitable for spammers while being at the limit of how much cpu time you can expect to waste for regular (particularly mobile) users. And that's before you consider the possibility of ASIC hashing for emails.
what other costs can we impose? the email lists that spammers harvest stay valid for a long long time, what if they were invalidated faster? to have a high hitrate on their lists they'd need Google scale crawling capabilities, which kinda then defeats their point of trying to make a living from spam.
why are we converting long term relationships with servers(email id and password to login) into relationships with every Tom on the Internet(no control over who can reach you, hence you are in a relationship whether you like it or not)? Do we still have to remember emails or phone numbers, aren't they names in an address book already? why don't we just move off email, use a new system that uses long, hidden alphanumeric ids to identify relationships between people that can be invalidated by either party?
invalidate unnecessary relationships faster = invalidate spammer lists faster :)
You can of course encrypt the mail yourself with GPG etc, but that would make self-hosting, the topic of this sub-thread, less relevant, and nobody but your fellow cryptonerds will do it.
[1] https://blog.filippo.io/the-sad-state-of-smtp-encryption/
Even MacGPG plugins for Mac Mail.app are really hard to install, manage, and use, and require a level of expertise and knowledge of how PGP/GPG works that makes them off limits for non-technical users.
Recipents basically receive a password from you in another channel (email, IM, phone convo, face to face) and can use it to decrypt the email from TutaNota when they receive it. I tried it out myself and the only clunky thing is that they load up their whole web app just to read one message, but it's kind of a clever piece of marketing because the web app works pretty well so the recipient who doesn't use encryption gets exposed to an interface that's just as easy to use as what they're used to.
For everyone using TutaNota, the decryption is transparent which definitely is encouraging.
I've seen some people sending around credit card info or social security info and forcing the intended recipient to use a password when receiving that info is a small price to pay for piece of mind. It's almost criminal to send signatures on documents and social security/credit card info in plain-text!
A few questions remain:
* How do they 'encrypt emails to anybody'?
* Where do they encrypt - locally or on the server?
(thanks for the new clickable site-history link; it makes resurfacing the duplicate posts much easier)
Also: (2014)
Since it has been over a year since that thread, though, we won't bury the repost as a dupe.
https://www.youtube.com/watch?v=jh8supIUj6c
My phone also uses the IMAP server, so I don't need any Google services. It's an Android phone, but it's never been logged into Google, wasn't bought from a carrier, and many of the built-in Google apps have been removed. (Hint: when you start up a new Android phone, and it wants you to log into Google, click "Later". Then remove the "Google One Time Startup" app to shut up that message.)
This isn't a security thing; it's to block ads.
Don't use standard e-mail (gmail or otherwise) for anything that needs guaranteed privacy. Once you send an e-mail, it is not within your control what happens to it or who sees it.
That's why, for example, in healthcare -- messages are commonly wrapped in secure message "containers". The e-mail simply has a link that takes you to a third party where the actual message is stored, encrypted, has an expiration date, and you must authenticate to view it.
Convenient? No. But privacy always comes at a cost of some sort.
The solution is probably using email in a manner closer to how it was originally designed: many local MTAs instead of a handful of centralized MTAs.
Here's a product idea, which could make someone a lot if they can find a way to implement it: a home server that is a minimal, secure, locally contained, turn-key home email MTA (with some localhost-only IMAP/webmail interface). You sell the server (and maybe a 1-time service to help people set it up).
This server would have to include a "choose your vendor, enter your credit card, click subscribe" super-easy interface to let people subscribe to vendors (multiple, if desired) that provide lower-MX-priority backup MTA services. (perhaps with an optional "hidden MTA" option, where your home server and lowest MX is only visible to the service, so the home router only connects to know hosts, IFF that is something that the customer wants.
People already understand local-only delivery because it emulates snail-mail. If you can let people receive messages and data unattended and asynchronous, so they can run the server when they want and receive their data directly (nobody else involved except for the backup services of their choice, then you have a product that will sell. It doesn't even need to have to have great uptime (that's what multiple MX records are for) or performance.
[1] https://projectbullrun.org/surveillance/2015/video-2015.html...