Ask HN: Why isn't email spam solved using Proof of Work yet?
Why not solve spam emails by letting senders work on some cryptographic puzzle (e.g. Proof of Work) that recipients can verify. This would prohibit spammers from sending massive amounts of email, since the required proof of work would scale linearly with the number of recipients.
For example let the sender search for a hash that must satisfy following condition: hash(recipient-email-address+message-content+nonce) % 42 == 0
If the hash does not match this condition, proof of work is missing and the email likely spam.
Another parameter could be the amount of work done by choosing different hash functions or hash functions that have parameters that tweak their run time.
54 comments
[ 2.3 ms ] story [ 113 ms ] threadHow will doing the work reduce spam? There is already a cost of sending spam in terms of infra.
I think spam is more a trust issue.
In short, if people won't use the simple mechanisms that already exist, they probably won't add a new mechanism or make use of it. One argument against the existing method is friction or false positives, but really it just forces people to update DNS correctly.
To add a math puzzle to email servers, you would also need to update every MTA and email server to understand this concept. All the major providers, all MTA's, etc... All the smtp libraries in all the programming languages would also need to understand this concept. java, golang, python, php, perl, C, C++, C#, etc...
[1] - http://www.gabacho-net.jp/en/anti-spam/anti-spam-system.html
[2] - http://www.uceprotect.net/en/index.php
Ill give you an example of how this played out in the past. Up until about a decade ago, if I enforced TLS on my mail servers, about 80% of the malware bots could not connect as their libraries did not support TLS. As more smtp libraries added support for TLS, the malware was updated and now enforcing TLS only limits a smalll subset of spammers. There was no added cost to the spammer. So yes for a while, PoW will create a window of time where there is less spam for those that adopt this. The spammers will eventually catch up. Nowadays malware frameworks are much easier to update.
Even if you focus on transactional emails only (e.g. signup/order verification, password resets, billing notifications, etc) where users are implicitly or explicitly opted in... the amount of mail volume involved is massive.
Given that this is a pure overhead charge, you can be sure that email providers are going to pass the cost on to the senders (e.g. whoever owns the relationship with the end-user). The larger that organization is the more likely they'll build that into the cost of the product and pass it down to the user.
SPF/DKIM/etc are a huge help. Even when spammers use it correctly it provides reliable attribution for establishing (or rather, destroying) reputation for the domain involved. I would love to see something more and think your intuition is good: an increase to the cost of send email is more likely to weed out illegitimate/unwanted messages.
This in turn allows small organizations / private mail servers that are not on common anti-spam white lists to proof their good intentions in a decentralized fashion.
PoW makes mass mailing very expensive and thus directly hurts spammers that are not on the common anti-spam white lists. Organizations that need to mass mail users for legitimate reasons are most likely already on anti-spam white lists and thus need not to do a very hard PoW hash as they are already proven to not be spamming.
Sending emails is hard because there is no central authority. Instead, each provider checks if the sender does some “best practices”. And the exact details vary - so if you got it working once with gmail, then there is no guarantee it will work next time with live.com. So you got to do them all.
For example, I have heard an unconfirmed rumor that one of the major provider started penalizing email sender machines without IPv6 address. If I had to maintain a mail sender, this’d mean have to upgrade my architecture to use IPv6 ASAP, which is non trivial at all. This is why most people outsource email sending.
Do you know how to read email headers? Are you subscribed to mailing lists? If yes, go to your inbox and check a few messages for the sender. You would be surprised at how many senders are there.
For transactional emails / regular mailing lists, I estimate that you just need like 0.25 FTE working on email, maybe even less. While this is not something everyone can do, there are tens of thousands email senders out there.
For example, I am subscribed to my town’s unofficial mailing list, which has tens of thousands of people, dozens of messages per day, and no budget. The mailing lists are also very popular with open source - LKML being most famous example.
And the worst thing, the spammers will not be affected as much. They’d just rent infected windows machines and do the calculations there.
Because:
(a) that would then have to be applied to every email (not just spam), which means every email infrastructure that needs to be aware of it (e.g. not transparent proxies etc), would need to be updated to be made aware of it and enforce it
(b) if applied to every email, it would increase global energy consumption/waste
(c) if there are "whitelisted" (which can bypass "proof of work") emails, then who would serve as the authority for those?
Abstract: ”A frequently proposed method of reducing unsolicited bulk email (“spam”) is for senders to pay for each email they send. Proof-of-work schemes avoid charging real money by requiring senders to demonstrate that they have expended processing time in solving a cryptographic puzzle. We attempt to determine how difficult that puzzle should be so as to be effective in preventing spam. We analyse this both from an economic perspective, “how can we stop it being cost-effective to send spam”, and from a security perspective, “spammers can access insecure end-user machines and will steal processing cycles to solve puzzles”. Both analyses lead to similar values of puzzle difficulty. Unfortunately, real-world data from a large ISP shows that these difficulty levels would mean that significant numbers of senders of legitimate email would be unable to continue their current levels of activity. We conclude that proof-of-work will not be a solution to the problem of spam.”
Yes, in theory PoW makes mass mailing very expensive and thus directly hurts spammers that are not on the common anti-spam white lists. Organizations that need to mass mail users for legitimate reasons are however most likely already on anti-spam white lists and thus need not to do a very hard PoW hash or none at all as they are already proven to not be spamming.
So this paper's conclusion is not the real deal.
But having such list will make email effectively centrally controlled. I would hope we never end up in this situation. The current system is bad, but making it centralized is even worse - just read what happens to businesses when Facebook/google bans them.
Of course some of the larger services may offer a default white list for common domains.
I suppose you can have a special system where you have a browser plugin which automatically generates some sort of subscription token and pastes it alongside email into subscription form.. but this will need cooperation from every mail-sending app, and will not handle non-web uses cases at all...
[1] https://hey.com/features/the-screener/
This is my take: Your post advocates a
In this case, proof-of-work is proof-of-waste.
While the fees were outstanding prior to receipt (or rejected by the recipient), they could participate in interest-earning liquidity pools to fund global efforts, like planting trees.
Spam will exist as long as it is profitable. If inboxes only show email which has paid a very small cost (like .001 cents), it would quickly make spam unaffordable.
> Anecdotal reports place the retail price of spam delivery at a bit under $80 per million [22]. This cost is an order of magnitude less than what legitimate commercial mailers charge, but is still a significant overhead; sending 350M e-mails would cost more than $25,000. Indeed, given the net revenues we estimate, retail spam delivery would only make sense if it were 20 times cheaper still.
https://www.zdnet.com/article/how-email-spammers-really-make...
I can't tell if Google is just not telling me about the vast ocean that the filter considers obvious, or if it's just gotten so effective that the spammers gave up. Whatever it is, it's working. I dunno if other email providers are similarly effective, but the tech exists if they want it.
Now, there's still web forms, which don't have the vast spraying power of SMTP, and also don't have the same kind of access to data to drive email spam filters. They usually try to de-automate the process with CAPTCHAs, which are also kind of a proof-of-work system (a "work" that's supposed to be cheap for humans and expensive for computers).
Maybe you could install a proof-of-work based system there? Not being email, it sidesteps some of the issues on the form reply that SI_Rob reposted.
It's never been adequately explained why no one implemented it.
It might be the obvious, incremental gains from filtering spam have kept users happy enough. No one company wants to take the first hit of educating users and the incompatibility with other systems.
It might complex processes like the engineers asked to implement it think users give a shit about mailing lists (maybe they did in 2004 when Gates told Davos he'd kill spam). Maybe marketing drones on about big business in all the meetings, crushing a good idea. Or maybe Microsoft went for micro payments over PoW, not realising micro payments was in itself an impossible dream.
Ask Bill next AMA on Reddit.