Ask HN: Why is spam email still a thing?

110 points by kisamoto ↗ HN
If you've had an email account for a while you probably know what I'm talking about. Open your spam folder and undoubtably it's filled with poorly written and worded emails still advertising "free sex", "you've won", "open for a gift card" or otherwise.

These emails are so bad and there is almost no chance of them finding their way through a spam filter, why are people still sending them?

193 comments

[ 2.7 ms ] story [ 215 ms ] thread
> These emails are so bad and there is almost no chance of them finding their way through a spam filter, why are people still sending them?

Most likely because the cost to send per email is so low as to be essentially free.

As well, there may be some amount of 'Nigerian email' issue here. I've heard it said that the "Nigerian prince with 65M needs help moving money to your country" emails are so poorly worded on purpose to specifically filter out individuals who are not good marks for exploitation. I.e., if the recipient fails to notice the poor wording and grammar then they may also be easier to exploit. It may be the same with the spam. If an individual responds to the spam, then the spammer knows they have possibly found a very gullible individual that can be easily exploited.

The cost for spammers can be actually free because they often use botnets and compromised accounts to send spam.
I reckon it’s just a numbers game.

Gmail will every now and then let absolutely ridiculous emails through the spam filter.

Badly spelled, bogus URLs, with $$$$ all over the place and All CAPS and just so clearly to any human a phishing attempt or just general scumminess and gmail takes a look at it and says “yup everything seems legit here. On to the inbox!”

My Gmail inbox from 2004 is 99% spam. I've tried everything I can imagine, nothing works. It is absolute chaos.
I've got the opposite problem - I get almost no spam to my inbox but regularly get legitimate emails routed to my spam folder, so I have to treat my spam folder like a second inbox anyway.
Google has to intentionally inbox a tiny fraction of spam, otherwise they'll never get corrective feedback on false positives, because basically nobody looks in their spam folder. Also, they have to permit a teensy fraction of stuff that would normally be temp-failed at SMTP time, for the same reason: otherwise it would be impossible for address and network reputation to improve. So sometimes if you see something completely ridiculous get inboxed, it might be that they are just trying to get data.
Ah, that explains some of what I see. I had noticed that my GMail spam folder went from thousands to dozens. I assumed they were zapping most of it but wasn't sure.
The overwhelming majority of suspected spam is rejected with temporary failure codes in SMTP and never makes it to the spam folder. The stuff in your spam folder is the cream of the crop!
There is a whole economy around spam at some scales: sometimes the spammer (the one actually sending the mail) has convinced a scammer (or slightly less illegitimate seller) to pay them to send mail because they claim to be good at getting through - in those cases the spammer actually gets paid a small amount (depending on their situation, a small amount to us could be a huge amount to them) no matter what.
botnets are not free. It takes time to develop and maintain. But it does make the cost of email using them very, very low.
sending a mailshot through a botnet has zero marginal cost. Maintaining the botnet has overhead, but that's not a cost of sending the email, it's an overhead.
I doubt this. Because even bot nodes will get blocked eventually so each email is degrading that value of that node. Again, very, very low, but not exactly zero.
> If an individual responds to the spam, then the spammer knows they have possibly found a very gullible individual that can be easily exploited.

This is absolutely key. The marginal cost of sending the spam is nil, but the marks self-select.

And you don’t need a lot if them to make a good income, especially if you’re in a developing country. In most of subsaharan africa, median income is under $1000. In some countries it’s under $500.

Due to conversion, minimum wage in Brazil is now $217,18
Correct - because the cost of the spam itself is negligible, but the cost of developing a prospective mark into cash-in-pocket is quite high in terms of time & resources.

The last thing the scammer wants is for the mark to back out after the scammer has invested resources into them, and before they have realised any gains.

Effectively, insufficiently gullible marks reduce their profit margins.

I would like an email service where, when someone wants to send me an email, they have to first get some hash, e.g., "{hash}.myaccount@myprovider.com".

Then, in order to get the hash, they have to paypal me 20 cents.

If my attention is not worth 20 cents to them, then they should not be contacting me in the first place.

Further replies to the same thread should be free for some period of time (e.g. a month or so).

I'd pay for a service that responds to every spam email with a GPT-generated conversation, with the goal of wasting more of their resources than yours.

Maybe you could train it on a central database that keeps track of which conversations get the most consecutive replies from scammers.

You might not even need GPT. Maybe just the act of responding with a reasonable amount of text would create a large filtering burden for scammers.

I have to believe someone has tried this.

Does not even need to be that smart . "Lenny" and others are existing anti-telemarketer scripts and recordings. Reportedly 10-15 different answers from a "confused old man" are enough to waste their time.

So something similar with a bit of variation thrown in should turn that spammer advantage on it's head with the 65million spam emails getting 1million replies instead of 100.

I built it for a hackathon project.

https://devpost.com/software/test-7gzse1

It hooked into your email client, identified scams in a very mediocre way, and then sent back a autogenerated (not GPT-3 response).

The big problem is privacy. I am not sure how to effectively solve that one. Why I did not win, lol.

I see something similar as as a risk for the general internet. We are already seeing GPT-based content farms masquerading as product review sites. It's easy to monetize traffic via affiliate links.

It will get harder and harder to separate genuine (human-generated) content from AI-generated. Eventually I think we will have browser extensions designed to detect AI-generated content and it will result in another arms race.

I think this might be a good thing in the long run. It may force us back to smaller social communities, where we get recommendations by asking our friends and their friends, rather than trusting search engines to curate the internet for us.

There are risks. You don't want people to get into echo chambers. But that's happening anyway.

I don't think it'll waste much time, they'd just run their own conversation generating bot for the first few emails before passing to a human. It'll just be two NN models talking to each other.
I wished there would be some service like 'Lenny for phone spam' that will automatically keep on replying to spam emails and waste their time and resources.
This is an awesome idea. With something like GPT3 you could probably keep them going for a really long time. I have to imagine that if something like that became (somewhat) common, it would really cause some changes in how spammy sales works.
And then the spammers get AI to operate their side too...
This was I think the basis for Dan Bernstein's "Internet Mail 2000"[0] idea - stop spam by making the sender responsible for mail storage.

[0]: https://cr.yp.to/im2000.html

Mail storage cost is negligible.
Per email?

Or per address/domain?

What's the best "altruistism wins" way?

Paid addresses, like ETH network?

Total speculation: there are billions of people with access to the Internet. Every once in a while one of them says, "Maybe I'll try the Nigerian prince thing." Maybe it works, but maybe it never does. It is just that people can try it, basically for free, so some do.
One thing that https://hey.com/ made me realize (even though I'm not a client) is that my inbox is open for everyone. Being able to screen emails is a cool way to prevent a lot of clutter.
Simply, it works some % of the time greater than 0 and it costs close to nothing to send it.
It still works. You don't need a lot of little old men buying knockoff Viagra or little old ladies looking for love to make a decent wage in a lot of countries. Also, you take for granted that everyone is benefiting from Google and Microsoft's SPAM filters attached to GMail and O365/Outlook. There are still a LOT of email addresses running on SMTP or even POP3. And a LOT of those email addresses are legacy addressses still in use by people that are rich targets for SPAM: old peopl using the addresses set for them in the 90's. A non-zero number of the messages that make it to your SPAM folder actually wind up in inboxes. Which inboxes? The ones that don't have good SPAM detection and, more importantly, are used by people that don't get a lot of email and may well be thirsting for connection. My parents practically dive over furniture during dinner to answer the phone just to see who it is. This behavior extends to inboxes for a lot of people.

Sadly, better SPAM filter technology won't be the end of SPAM. SPAM will lose its efficacy when old, unfiltered inboxes stop being used. And they will stop being used when the (largely) older demographic stops using them as a result of the passing of time.

As someone who snagged a FirstInitialExtremlyCommonLastname@outlook.com address (think asmith@,bpatel, etc), I can say that their spam filter is absolutely worthless. Never bothered to use that account because it was so horrendous.
Of course such an obvious account would be attacked. It is very easily scripted. first.last@, last.first@, etc are all going to get pounded. Corp email policies are just ripe for scripting attacks like this.
Why does Gmail even show them in the spam filter? Shouldn't it be so obvious spam that I don't have to deal with it? And then only keep the dubious stuff and show it to me.
If suspected spam e-mails are simply discarded, the users will not find that any false positive rate other than zero is acceptable.

There is a way to reject suspected spam, which is to do it at the SMTP level (refuse to deliver) so that the sender gets a non-delivery notice.

Accepting everything (pretending to deliver it) and then silently dropping some of it is generally a nonstarter.

In general, what you see in the spam filter is the stuff that isn't obviously spam--the latter category is entirely black-holed.

False positives in spam detection are really bad, so you want to tune your algorithm to have really low false positive rates--which means you get high false negative rates in turn. Having a spam folder to filter non-black-holed spam into allows you to introduce another level where false positive rates can be higher without being catastrophic.

It shouldn't ever be black holed, it should rejected in the SMTP transaction.

I'd love to never have to ask "did you get my email message?"

The stuff he's talking about getting black-holed is conclusively spam; it's not stuff they're guessing about.
The problem is by the time the message is deemed to be spam it's too late to reject the mail.

Ie. The first step verifies the from/to addresses, then the message body is sent. So, if the message body is clearly spam and you reject the message body all you've done is confirmed to the spammer that they are sending to a legit email address and they need to change their message so it no longer is rejected as spam.

As bad as the current system is it leaves spammers with uncertainty over whether their message made it through the spam filter.

What's stopping spammers from just having a GMail account they can use to test if the message made it through?

I mean; I have to assume the engineers in charge of this @ Google are making the right call, but the cynic in me can't help but worry they are making the call in a way that benefits them at the cost of a healthier SMTP ecosystem. (ie, "Lets make email just unreliable enough that folks switch to Google Email 100%, or use some other Google Product for messaging ...")

You can reject the message anytime after TO/FROM/DATA w/o signaling which of those 3 caused it the message to be rejected.

> The problem is by the time the message is deemed to be spam it's too late to reject the mail.

That is incorrect. After the last DATA command from the spammer, your mail server can look at the mail and, if it’s spam, just say

  558 5.7.1 That mail is ugly and you are ugly
This rejects the mail and makes it bounce to the sender.
Yes, but unless that error is used frequently for other situations then it now becomes an obvious signal to spammers that their message has been flagged as spam.
Then use 550 instead.
I don't agree. There are servers and domains that only send spam. There's no point in passing those through a computationally-expensive filter system; we already know they are spam.

> I'd love to never have to ask

Yeah, me too. But email is a best-efforts delivery system; email delivery can fail for reasons other than spam-filters.

You don't have to send it through the filter. You can decide @ accept() time that the message will rejected. If you intended to blackhole it; you were planning on burning bandwidth and letting the sender send their bytes entirely, and then responding "2xx OK, whatever"

Instead, just simply say "500 FOAD." after DATA and be done with it on the non-zero chance the sender actually has something important to say, and needs to decide if they need to use another channel.

(comment deleted)
They do only show you things they are not sure about. They filter out a lot more spam than what you see in your spam folder. I for one have found a lot of legit emails in my spam folder especially from smaller retailers etc.
I remember hearing a rumour that nearly half of all mail that hits Google's SMTP servers ever land in an inbox. While there are reasons such as wrong addresses the vast majority of that is spam.
I would not be surprised if it was more than half given the trivial cost of sending email.
I think of spam like a lot of direct physical mail. There’s two effects going on:

1. People will continue to use an easy tool even if it has become ineffective, because it’s available. It doesn’t take many spammers to send 100,000,000,000 emails. Direct mail isn’t what it used to be, but it’s baked into the systems for every car dealership and so it continues. Spam is probably 1000x less effective than it was in 2002, but most spammers aren’t running A/B tests either.

2. Just like direct mail, it’s easy to think that no one looks at spam because I don’t. In fact, there really are a lot of people who look at every coupon in the Captain D’s flyer. Same thing with spam…even if the open rate is 0.1%, that could still be 1 in 1000 people.

Apparently enough people ultimately click the links to make it worth it for the spammers, otherwise it would have stopped.

It's probably something like less than 1 in 10,000 emails getting a click, which is depressing when you consider all the computing resources wasted by receiving email servers and then by all the recipients which need to filter out the noise (for example I still at least scan subject lines of items in my Gmail Spam folder).

So with that considered, spammers clearly completely lack empathy for their fellow human beings, they have zero care on the cost of their practice, as long as it makes them a few bucks. Sure, there are people who do far worse things, but that fact in no way redeems spammers.

> So with that considered, spammers clearly completely lack empathy for their fellow human beings, they have zero care on the cost of their practice, as long as it makes them a few bucks.

I'll start by saying I in no way excuse like or can stand spammers, I think action should be taken against them as they've destroyed every communication medium in existence.

But I have some friends that come from some third world countries in Africa and similar places. From their perspective the few bucks to us makes them some of the richest and well to do people where they are at, and all they are doing is taking it from fat rich wealthy Americans and Europeans that already have too much money, even if the person is close to destitute by our standers from the perspective of many people in the third world if they have a roof over their head and know where their next meal is coming from, to mention nothing of ac and a car they are living an unimaginable lifestyle.

My point isn't to defend scammers it's just to point out that many of them arent malicious simply trying to survive in a difficult world, I would argue many of us here on Hm would do the same if faced with the choice after all how many people here already go to work on ad or surveillance tech of dubious ethical vlaue because the pay is insane.

It sounds like these people are small scale spammers, doing things similar to 419 scams. A bit annoying, but really low volume.

The kind I'm referring to are single organizations which aim to send out 10s or possibly 100s of a millions of spam emails _every day_. I would expect there to be at least dozens of organizations at this scale. They're the ones who use botnets, open relays, and weird characters in their emails to get around filters.

It is true that it most likely works well enough to get some returns. However on this point:

> Apparently enough people ultimately click the links to make it worth it for the spammers, otherwise it would have stopped.

Each generation has to learn new. Every day, there is somebody new to try out spamming. Every day somebody got their first mail address and their first spam.

Most of those spammers will quit soon again, some will improve and have some small win.

There are a lot of e-mail systems out there with not very good spam filters. And spammers do change up their methods to get around filters. You might not notice them, but people with less effective spam filters do get them. (And some people are dumb enough to read e-mails in their spam folder)

It's always been an economic numbers game. You might send 1 million spams that costs you $100, but you might get $400 in return from whoever paid for the spam campaign. If you're running your own spam operations for your own "products", you get more profit, but the risk and difficulty is higher.

I also have a theory that spies use spam as a form of steganography. If spam naturally contains a lot of variable information, and it comes from random places and is sent literally everywhere, it's not hard for a spy to receive an encoded message dropped into their mailbox without anyone even knowing what their e-mail address is.

Filtering is also not simple problem. One user might want to filter the "newsletter" subscriptions they have been automatically singed up for. And per best advise should not try to unsubscribe from as this tells the sender that it is good address to sell... Other one might actually want to see those.

Not to even mention those crap sites that allow you to signup with email without verification, looking at Instagram...

These days spam is less about selling you viagra or other crap, but probably more trying to phish or infect you in some way. They ransomeware you via some drive-by infection vector from email links, and then they get paid when you realize you're screwed.
Are drive-by attacks still actually possible if you're using a modern browser?
Because email is essentially one of the oldest internet applications and also essentially has never changed.

That means, as has been stated in this thread a bunch of times, that sending spam is essentially "free", especially since they like to use exploited email accounts to do the sending if possible.

I am all for a re-envisioning of email from the ground up, tbh.

I'm all for reimagining the email system, but considering how wide spread email is right now I think the replacement might need to be at least partially compatible to allow interoperation during transition
The tech industry’s ‘reenvisioning’ of a free/cheap older, but universal, standard is usually to replace it with something expensive, proprietary, and limited in scope.
My hope is that it would be an open and free standard instead of some of the usual bullshit vendor lock in
When has that ever happened?
That's an extremely broad question. Are you asking specifically about email replacements, communication standards, or just proprietary software in general?
When has ever a “tech industry’s ‘reenvisioning’ of a free/cheap older, but universal, standard” resulted in “an open and free standard” and not “the usual bullshit vendor lock in”?
Never that I can think of, which is why I stated my hope that it wouldn't be like that. But maybe that just means we can't necessarily depend on a profit motivated corporation to do the 'reenvisioning'
Bingo. See also the rumored “definition of insanity”; i.e. doing the same thing over and over again, but expecting a different result.
Basically anything? When was the last time something like email happened? And it wasn't a giant tech company that created that either.

Maybe something like Webrtc?

Can you elaborate on what you would like to see from a reimagined e-mail?
It's taken us something like 30 years to start seeing wide IPv6 adoption. Even then most a running dual stacks to still support IPv4. How long do you think it would take to have a secure email protocol running by most users?

It could probably happen quickly if a few big players came together and made the push. Unfortunately the only innovation in email these days is occurring in ways that help user lock-in. E.g. IMAP is stagnant, Gmail added 'dynamic emails' that only work in their ecosystem.

Not just email. I got 3 texts this morning trying to pitch scam jobs, not to mention the endless robo calls
Also snail mail. And on the radio. And billboards in town or at the least stapled to telephone poles. I've seen scams on banners dragged by airplanes.
“But not in dreams.”
Ironically, those podcasts you are supposed to listen to while sleeping play ads throughout.
Personally, I believe a spam will exist while it makes a reasonable profit. In other words, while people are still clicking links in those emails, and sending a thousand emails is dirty cheap - it will exist. I also believe that spam is unbeatable as a matter as it's now everywhere not only in "dirty cheap emails": we all faced robo-calls, sms, messages in social networks, etc. I'm even still receiving a spam into my analog postbox right at front of my house. It's something that will pursue us while it makes a sense for businesses.
Dude, spam on Usenet is still a thing. alt.books.iain-banks, for example, is utterly full of garbage, and that's for a discussion forum technology that's been obsolete for nearly twenty years.

I get spam from Google Apps customers, Office 365 customers, Mailgun customers, and more, despite these providers' terms of service. I get spam on my LinkedIn spamtrap address, my FreeBSD ports spamtrap address, and more. I'm seriously considering a switch to whitelisting, which is what I had to do on my phone to deal with all the robocallers. It's insane with motivated evil people will do for a little money.

Because email is a system designed for ARPAnet not Internet, and it was designed to be used be people who could be trusted to not spam. No protections were built in, and email hasn't fundamentally changed since the late 70's. In mid-late 90's the masses came, and since then we've been adding band-aids to it to keep it alive but the truth of the matter is that it's still an open door just waiting to be walked into. You can DKIM, SPF, and do all the SMTP authentication you want, but the spammers still get through.

Sure, it might end up in the spam box. But so do real emails. So, spammers still get their emails viewed. As do phishers.

https://www.bankinfosecurity.com/tricked-rsa-worker-opened-b...

    A well-crafted e-mail with the subject line "2011 Recruitment Plan" tricked an RSA employee to retrieve from a junk-mail folder and open a message containing a virus that led to a sophisticated attack on the company's information systems,
https://www.wired.com/story/the-full-story-of-the-stunning-r...

TL;DR: Email is fundamentally broken because it was designed in a time when you could leave your doors unlocked at night.

Any system that runs off anonymous messages is going to have a spam problem. There isn't anything special about how email is designed that enables spam past that. If you were to design a new system you would have to have signed messages. But we could just do that now with email.

It isn't a design choice, it is a choice that we make every time we send an unsigned email.

I think this is the essential fact, thanks for posting.

Lots of people complain about the “insecurity” of email, but as long as you want this feature (sending anyone a message knowing nothing more than just their email address), there will always be spam.

And if this is a feature you can do without, then there are lots of alternative communication systems, or you can always use signatures in email.

Yes, it was a design choice made in the early 1970's before public key cryptography was even invented. And that's why it's broken and always will be. This is why email needs to go away completely.

But then again, there are still laws on the books saying the only way to securely transmit a document is by fax machine. So even if somebody reinvents a backwards compatible email system tomorrow that solves all the problems, people will be to afraid to use it.

HTTP was invented before SSL/TLS. Does that mean that the web needs to go away completely?
Outright spam is one thing. What’s more prevalent these days and vastly more annoying is corporate spam. Every purchase now is apparently free license to send you pure garbage emails multiple times per day.
Please report spam on these. The email providers won't know they were unsolicited and unwanted if you just unsubscribe.
Marketing email from an account you chose to sign up for may well be unwanted, but it isn’t entirely unsolicited. And it’s not at all the same as spam from companies that you haven’t interacted with. There are problems with reporting any and all email you don’t want as spam, especially when you’ve done business with that company. That devalues the meaning of spam and it will cause email providers and spam filter providers to become less stringent on identifying spam, not more. It’s already happening to gmail.
The Report Spam button informs both the ISP (i.e. Gmail) and the ESP (sender, usually someone like Mailchip or Mailgun) that I do not want this message in a process known as a Feedback Loop [0]. This allows the ISP to ding the ESP a bit on the reputation of their IP address (assuming enough spam complaints). It also allows the ESP to tell their customer who is sending on their platform that they are sending badly, and potentially dial back their service or shut them off entirely.

If you are sending good mail to a double-opt-in, highly intentful marketing list, then you will receive minimal spam complaints. If you are sending to people who don't want it (they didn't check the opt-in button), it doesn't muddy the water because it is spam.

That being said, there are legal requirements (CANSPAM) [1] for mail senders around the unsubscribe link, but there are no legal requirements around the report spam button, so either kind of works.

[0] https://support.google.com/mail/answer/6254652?hl=en [1] https://www.ftc.gov/tips-advice/business-center/guidance/can...

We are our own ESP at work and I work on the email team. When we get a (few) FBL from you, we block you from receiving any of our email whatsoever. Forgot your password or want to contact support? Oh well, we won’t spam you again. There are unsubscribe links, use those unless it really is unsolicited spam.

What I find most interesting about email is how some people will mark email as spam literally months or years after we sent the original email. We send over a billion emails a month (not spam) and people’s behavior around email is fascinating.

> mark email as spam literally months or years after we sent the original email When a persistently spammy company compels me to open the Rules page to filter their trash, you bet I’m also going to Select-All and Report As Junk in hopes of maximally dinging their reputation. No idea if it works, but there’s one explanation :)
While this is a reasonably accurate picture of what happens when you hit the report spam button, you’ve left open a somewhat false dichotomy that is quite central to the mis-categorizing of marketing email as spam. Checking in the opt-in button takes on several forms, an extremely common one of which is signing up for a service where the opt-in button is in the terms or other fine print, it is not always (or even usually) a default-off explicit and separate button click somewhere. There is often, and with most good companies, an explicit opt-out button somewhere. That is what should be used before reporting spam, if we want spam filtering to remain reasonably good.

You’ve described what happens when you report spam, but not what happens in the future when more people get upset over email and reflexively report all marketing email for accounts they chose to sign up for, and opted-in to marketing email for (by agreeing to the terms), and potentially still need transactional emails for. The average person doesn’t know the difference between transactional email and marketing email, and if you follow @Old_Thrashbarg’s advice to report any email you didn’t expect as spam before adjusting your settings, then eventually we might lose those settings as companies and providers all come to the conclusion that people can’t be bothered.

You’re also suggesting that there’s some broad segment of good marketing email that people don’t consider unsolicited or even spammy, which is by and large not true. There is practically no such thing as highly intentful ads that most people want, aside from the occasional short-lived viral campaign. By definition, marketing is a push initiated by the company to sell their wares, and most people would prefer not to watch ads given the choice.

Don’t forget that Mailchimp, Mailgun, Gmail, Hotmail, and almost every other service you can name here is actively making their income from email marketing. As much as we want to, it’s going to be difficult to block all marketing email, and they all have a vested interest in delivering email, especially from the people who are paying for the service.

Whenever I order something online, I always uncheck the box that says "I want to receive your newsletter" or never check it the first place. Inevitably, in the next few days, I start to receive their product offerings
Guess what happens when you click "unsubscribe"?
Your email gets added to known good addresses list and sold around?
I at least haven't seem to have had that experience with legitimate companies. Spam, yeah, I don't even load images to those emails
I don't receive any emails for about 6-12 months, and then I start receiving emails again. At least that's what has happened with NextDoor, twice - once each for different emails months/years apart.

NextDoor also used to violate the CanSpam Act by requring you to log in to unsubscribe, but they fixed that a couple of years ago.

My nearest neighbor is more than a quarter mile away. In a two-mile radius, I have less than a dozen "neighbors", and yet somehow, I have a NextDoor account that I receive spam emails about.
Do the content previews in the emails look like they're for your area, or could it be someone with a similar name registered somewhere else with the wrong email address and NextDoor never validated the email address?

I hate it when companies allow people to signup with email addresses that aren't theirs, never validate the email address, and then start spamming me. I mark all of those as spam. The latest offender is Chime.com (I have a common-ish FirstLast@gmail that I'm phasing away from.)

It's definitely the correct geographical location.
Despite me unsubscribing two days ago, I just received another email from NextDoor. *marks as spam*
I have a lastfirst@gmail and the fellow owning firstlast@gmail has occasionally used my email address by accident.
I just learned how bad NextDoor's dark patterns are yesterday. I realized it was about the 4th time I'd tried to unsubscribe, navigated to the notifications page, and see that they have about 40 categories of emails they send you, split across several parent categories. You could turn off parent categories, but not silence everything at once.
Fun fact about CAN-SPAM: your recourse is supposed to be reporting them to your ISP.

Except that Comcast/Xfinity itself is violating the CAN-SPAM act. I constantly get product offers and other junk from them, despite being clearly marked as "unsubscribed from all" when I click the unsubscribe link.

They are also a local monopoly and I have no affordable alternative for "basic broadband".

There does not seem to be any facility to report this to any regulatory agency of the US government or my state government.

You get a ridiculous page that makes it as confusing as possible to actually unsubscribe from everything. Once you succeed at that, you stop getting emails from them until they roll out their next marketing campaign, which you couldn't unsubscribe from because it didn't exist yet.
A lot of the time it goes via a link that gets blocked by ABP, or my pi-hole - so I just give up and route it to junk
This is why I have a catch-all and give each company a different address. I can just forward that address to junk¹ instead of bothering to unsubscribe. It isn't just when they ignore their own consent options for sending their own stuff, often mailing lists are hacked out of companies and become junk targets that way. Or they are actively sold: the amount of crap I get to the addresses I have registered with kickstarter²³, paypal, etc., seems to confirm that is very common.

[1] Or set it to bounce, but I prefer /dev/null as bouncing increases the chance I'm becoming part of a back-scatter issue.

[2] At some point soon I'll get around to cycling those addresses to something else again, I haven't for a while, then the current ones will be redirected to spam training.

[3] Including some newsletters specifically targetting people who have interacted with crowd-funding so some project's team has given my address out against my wishes⁴. I wish we could give them specific addresses when supporting, so I could tell who (I don't cycle the main address I use for such things often enough to tell).

[4] Though those have saved me money: if I think of supporting a project I search the junk file first and if it has been mentioned by “backer-update”, “grant @ t.k.p.”, “kickstartech”, etc, I'll walk away - but only one has lost my help thus far that way and doubt anyone else is doing it so this petty little victory probably means nothing in the grand scheme of things.

> bouncing increases the chance I'm becoming part of a back-scatter issue

Bouncing doesn't cause backscatter, provided it's a real bounce; that means a bounce that is a rejection by your mailserver, performed during the SMTP transaction. The message commonly known as a "bounce message" is technically known as a delivery status notification. The actual bounce message is composed by the sender's mailserver, and can't cause backscatter.

Basically, a bounce sent after the SMTP server has said "Message accepted for delivery" is a bogus bounce, and causes backscatter. Sending bogus bounces is a reprehensible practice (and quite useless).

Aye.

It sometimes happens because mail gets accepted then put into a local queue for processing (anti-spam checks and other scans) and the fake bounce sent after that. A badly designed system (and such do exist) will do this even for messages they could immediately reject based on the invalid incoming address.

It can also happen because there is an intermediate SMTP server between the sender and receiver, the intermediate has dropped the connection to the sender before the final destination rejects, so if you want to bounce at all it has to be a fake bounce.

I don't know without checking (and I've been too lazy to test!) if my mail services properly bounce based on target address or fake bounce, so I err on the side of not causing disruption if it is the bad one. I'll be replacing Zimbra with something else (probably self-built from standard parts as my needs are much more limited than when I picked Zimbra many years ago, just mail, nothing else) for my main mail service soon, this will be one of the things to make sure I get right.

I bought a pair of car sidelight bulbs from a small ecommerce outfit and received multiple emails about special offers and such. Can't remember what the cost was but I do remember the postage charge was the biggest part of the transaction.

I'm sure the business owner really thought he was promoting his business well but who buys bulbs in such quantities to make this worthwhile?

90% of the things I order from Amazon result in a "customer service" check in email from some random vendor.
> These emails are so bad

From my understanding, that is a feature. It acts as a filter for the top % of more gullible targets.

I wonder if spam filters could use a combination of spell checking and NLP to detect these and block them.
It's a numbers game. The cost is almost zero to send, and one success can make a few thousand dollars or more. That's a ton of money in some of the world, especially parts of Africa and South America.
It's a real conspiracy. Simple as that.

Google proved they could beat spam around year 2000, but now suddenly they are now letting messages through with headers that a toddler could tell you are fake.

They know it' spam, we all do. But they let it in anyway because there's some secret economic or political dynamic we aren't privy to.

Spam exists because sending as others here have mentioned is free. The knee-jerk reaction might be to make it not free and technically that would be rather simple. Beyond adding pre-approved DKIM signed domains to an approve-list, one could modify existing grey-list daemons to require either a one-time code or click a URL to enter authentication to approve the sender and add rate limiting for know abusers, but that isn't the problem. People are accustom to how email currently works and adding any barriers or changing communication flows or adding cost adds friction. Businesses and people will oppose friction.

I would be curious to see if anyone here has added such barriers and what their results were. What methods did you use to make spam expensive for spammers and how did it affect your legit customers and prospects?

I would pay 1 cent per email/recipient if it meant receiving no spam, and that my emails would never get filed into a spam folder. (I'm just a small time regular user, but with a personal domain name.)

Unfortunately, there are probably businesses where even with that cost, it would still be profitable to spam - just look at physical mail or any other type of physical advertisement, or heck... even online advertisements still have to pay money for impressions.

Because it works. Still relevant, from Microsoft Research:

"Why do Nigerian Scammers Say They are from Nigeria?"

This approach suggests an answer to the question in the title. Far-fetched tales of West African riches strike most as comical. Our analysis suggests that is an advantage to the attacker, not a disadvantage. Since his attack has a low density of victims the Nigerian scammer has an over-riding need to reduce false positives. By sending an email that repels all but the most gullible the scammer gets the most promising marks to self-select, and tilts the true to false positive ratio in his favor.

https://www.microsoft.com/en-us/research/wp-content/uploads/...

Also, a lot of people I suspect still have old email software (Outlook Express anyone?) which might not even have spam filters set up.

It is sad because I think it isn't just gullible people generally, but those who did not grow up with computers and won't understand that "This is your bank informing you of a security breach" is bogus are unfairly hit.

Other thing is that the cost of sending massive numbers of emails is basically nothing so why not? If only 100 people out of an email list of 100M click through, that might pay you for this month.

Who uses spam filtering in their email client anymore? That's done at the server level.
I do, on my own domain, because it's easier to train the filter within the client, but mainly because that client is the only place I access that email account.
FWIW, every server-level spam filter I'm used has had so many false positives it's caused me significant problems.

Also FWIW, people to whom I send email regularly tell me that their email systems mis-classify my emails as spam. I've spoken with several networking specialists, and they all shrug their shoulders and say: "It's the server level spam filtering ... it's not very good." GMail in particular is known to be aggressively negative towards email originating from anywhere other than another GMail account.

So in answer to your no doubt intended to be rhetorical question, I use spam filtering in my email system, not on the server.

Just as a counterpoint, my company uses Google Workspace for email and we receive a good deal of spam. Been using it a couple years now and it's been very good at identifying spam, with very few false positives.
(I've up-voted you ... our contexts are different, our experiences are different, the results are different, and it's important people get all points of view).

I have/work with three different companies ... two of them use Google based email systems, one (my own) does the filtering "in house". One of the G-based systems performs well, the other has multiple false positives every day.

I'm sure it performs well for the majority of people ... it's likely that the emails I send and receive are atypical, and Google performs badly because they are not "normal".

Everyone has to decide where that trade-off lies. In my context, I simply cannot trust Google, or any other server-side filtering, for important email. As others have said, it simply means the spam folder is a second inbox.

As I say to my clients about so many things: take advice, understand the trade-offs, make a decision, deal with the consequences.

> GMail in particular is known to be aggressively negative towards email originating from anywhere other than another GMail account.

This just isn't correct. Gmail does indeed have a HUGE market share at around 20-30% depending on source but with every business using mail for notification and business communication you will note that an even smaller minority of mail comes from other gmail accounts. If Gmail actually worked like that it would have ceased to work entirely and lost most of its users. If I understand correctly the actual issue is all the major players vs very small players like people running their own server.

In that scenario the users whether it be on yahoo, google, msn what have you get 99.999% of their intended email while the person running their own email server will have a much larger miss rate a bigger problem and one that can't be solved on the client side at all.

From the perspective of the end user who is neither aware nor concerned about small servers issues server side filtering works very well. Looking at my spam folder it seems to be filled not with nigerian scams which don't even make it to my spam folder but with emails from people who have send me emails which have some degree of legitimacy but which I have repeatedly deleted unread like political emails requesting contributions. From my perspective it works great.

As someone who up until recently managed the mail system and antispam for a business processing ~250k mails a das, false positives are an outlier. We had a ticket because of this perhabs once a week and most of the time it was because of poorly setup Mailservers in the other end. Server side filtering works quite well, most of the spam comes from hijacked leggid accounts that are burned quite quickly.
Great, I'm genuinely pleased to hear that your experience of server-side filters is a positive one. Mine is negative. The fact that you have a positive experience does not refute that mine is negative.

Please: I've experimented with this for over two decades, and continue to do so. I know that lots of people say that server-side filtering is fine. My direct personal experience is that for me it is not fine, including the spam filtering provided by Google's email service. I do not deny that for other people it's fine, I'm just saying ... don't deny my experience.

Never had much of a problem with Gmail's filter. An odd newsletter end up in spam once every while. How does your issue manifest?
It's his Nigerian Prince Scam How-To newsletter, goes missing every week!
> How does your issue manifest?

I regularly have emails to my GMail from colleagues end up in the spam folder. Perhaps as many as 5 or 10 per week, enough that I have to trawl through the spam folder every few days to make sure I've not missed something important.

Similarly, reasonably often when people reply to me, often after being prompted with a second email, or even a phone call, people tell me that emails from my "other" ISP to their GMail provided mailboxes have ended up in the spam folder.

It happens in waves ... it can be fine for months, and I start to think it's all good and I can trust it, and then there's a spate of emails going to spam when they really shouldn't. It's simply unreliable.

So for incoming email I do my own filtering. I can do nothing about my outgoing emails ending up in someone else's GMail spam folder except to be aware that it happens, and to keep track of when I'm expecting a reply. With some colleagues I've gone to a "positive acknowledge" system to preempt what have been expensive problems in the past.

A couple things I can think of:

* What's your DKIM/SPF/DMARC setup like? That all needs to be on point to get delivery to gmail to be reliable.

* Are you using a provider like Sendgrid, Mailchimp, etc. to send mail? Are you on a plan that gives you a dedicated IP address or is it part of their shared pool? The "it happens in waves" sounds a lot like a common problem with the shared pools where spammers send a bunch of mail through a service, the IP gets blacklisted, and it takes them a couple days to notice and remove it from the pool.

* do you have any "noisy" alerts or similar going out to people? Not necessarily spam, but eg, sometimes users getting frequent notifications, daily summaries, etc. will sometimes just flag them as "spam" rather bother to turn them off or manage them properly. At some point, that trains Gmail's filters to increase the spam likelihood on your entire domain.

* I am assured by my ISP provider that the settings are as squeaky clean as they can be.

* No, I'm sending email from my domain on my domain host. That has a single IP address shared among all their customers, but it's a very small ISP, and there are no spammers, and the IP address is not on any blocklists.

* No, I send standard email "by hand", and I know that people are not marking them as spam.

It's nice to see you go through these things because these are exactly the things I've gone through several times, so it serves to confirm that I've addressed the right things.

You should be able to look at the SPF/DKIM/DMARC related DNS entries yourself with dig and verify that they are correct. I wouldn't trust an ISP's word, especially if you are experiencing problems. I would also recommend setting up weekly DMARC digests via Postmark's free service: https://dmarc.postmarkapp.com/ which should allow you to actually see whether emails are getting rejected and why and give you a lot more data about your actual SPF and DKIM alignment.
I'm not a computer person, certainly not a networks person, and I know nothing about any of these things, so everything you say might very well be true, but it amounts to days even to work out what you're saying. It's very likely you could talk me through it all, or I could embark on a course of finding out about these things, but I really just don't have time. I'm already over committed.

You say:

> ... look at the SPF/DKIM/DMARC related DNS entries with dig ...

I don't know how to do that.

> ... verify that they are correct.

I wouldn't know what it means for them to be correct.

As a further example, in trying to use postmark's service I get this:

    In order for us to send you email digests,
    you must set up a DMARC policy associated
    with your domain. This allows us to collect
    reports from the major ISPs about your DMARC
    alignment.
OK, that's fine. Then:

    Add this TXT record to your domain’s DNS at
    _dmarc.solipsys.co.uk. with the following
    data:

    ...
I have no idea how to do that, and suddenly I'm down the rabbit hole of things that other people think are obvious and easy, but leave me with no idea of what to do.

I really appreciate your feedback, and I'm passing it on to the people I pay to do this for me. The thing is, that's my ISP, and you tell me I shouldn't trust them.

Yeah, the thing is that if you have your own domain and want to send mail from it, that's really no longer the sort of thing where you can just connect to port 25, send mail and expect it to work. Spammers ruined that a long time ago and now you need to understand a lot about DNS and those mail related standards and do a significant amount of work to stay on top of the configuration to have a snowball's chance in hell of your mail getting through. It's not a Gmail thing, it's the entire landscape of email delivery. If you don't have the knowledge and time for that, you're much better off delegating it to Google, AWS SES, Sendgrid, Mailchimp, or some other service that takes care of it properly. If you're paying your ISP to handle sending mail for you and that mail's not getting through, you need to take it up with them.
I have passed this conversation to them, and they say that all the records and settings are correct. They say that the problem is the recipients' servers are marking the emails as spam, even though all the settings are correct, and the emails are clearly legitimate.

> ... now you need to understand a lot about DNS and those mail related standards and do a significant amount of work to stay on top of the configuration to have a snowball's chance in hell of your mail getting through.

That's exactly what I pay someone else to do, and they are doing it correctly (as best I can tell).

So ... I just have to live with it. As far as I can tell, short of simply giving up entirely and using Google, I've done everything I can.

You seem pretty set on staying with your ISP and I'm not going to try to convince you otherwise, but "we're doing things right and Gmail is wrong about how to process email" wouldn't carry much weight for me. I'm 100% sure that the Gmail devs know more about email delivery to Gmail addresses than I do and 99.99% sure that they know more about it than your ISP. If you want to keep believing that your ISP knows better than them, then I guess you're going to.

Is solipsys.co.uk your domain (from an earlier comment)? If so, I can see right away that there's no DMARC entry for it. That means that it's vulnerable to spoofing (I or any attacker could easily send an email to someone making it look like it came from your domain, eg, as part of a phishing attack). Gmail definitely penalizes domains without DMARC, making emails more likely to be classified as spam.

> You seem pretty set on staying with your ISP ...

I am, for many reasons, so thank you for understanding that.

> If you want to keep believing that your ISP knows better than them, then I guess you're going to.

It's not that I believe my ISP knows better than Google, it's that I believe they are doing what they have been told is necessary.

> ... solipsys.co.uk ... no DMARC entry for it.

I will raise a ticket and get it dealt with, and as a part of that I can get the entire issue looked at more closely.

Thank you.

Ah, so you are self hosting your email! That's where the trouble lies then. I wouldn't do that myself, too much headache. Just use Gandi's webmail.
I suspect you are using terms that to you are obvious, but to me are technical terms that don't mean what they seem to mean based on the words involved.

Based on what you've said here I don't know what you mean by "self-hosting". I have a package that I purchase from a company that provides me with my email and web hosting. I pay them, I configure my web client to connect using the details they provided, they deal with the configuration.

They are qualified in this field, they have certifications.

Everyone here seems to be using terms that obviously mean something to them than is different to what the terms imply to me. I have machines that connect to my email, hosted by the ISP with whom I have the contract. To me, that doesn't mean "self-hosted" ... I don't do any configuration of DKIM/SPF/DMARC/etc. That's all done by the people I pay.

So ... I really don't know what you're talking about. I suspect you know too much to be able to communicate effectively with me, who knows nothing. It's plausible that the field of discourse is simply too technical.

I have to admit, I never paid this much thought, but your comment made me laugh out loud. Hahaha!

Thank you!

I’ve discovered that paper a couple years back and indeed, once you hear it it just makes total sense, in a funny way
Not so funny for some. One person I know, (80+) got scammed out of a reasonably large sum of money (very large for them, probably pocket change for HN). They could not imagine that people would be so mean as to feign an emergency to scam others. These assholes really hurt people.
Yup, they're robbing old people of their life savings. It seems nobody seems to be doing anything to prosecute or even track these people down.
It's an international crime most of the time, with operations that can be opened in a day. It doesn't help that local law enforcement in some countries actively covering these types of crimes.
Yep, often praying on good will or desperation of older generations.

A lot of people think it's laughable that anyone would fall for "simple obvious scams" but I've met a good handful of older family and their more elderly friends who either fell to a scam or were only stopped once someone at a store questioned them as to why they were buying lots of gift cards all at once. In some cases scammers will work on people over a long period of time to gain their trust.

NB: “Preying”, not “praying”.
You may have misunderstood my comment. The way the introduction of the article is phrased is funny, what scammers do is horrifying.

I've had people in my close family fall into debts to those folks, they target the weakest and literally go heaps and bounds to extract as much as they can from their 'mugus'.

Not fun at all :S

Isn't the gullibility scale a spectrum though? Perhaps if the emails sent purportedly originated in nations of higher repute, he would scam all of the most gullible people and in addition, some less gullible people, resulting in a greater profit.

After all the marginal cost of sending out extra emails is basically free. It's not as if he needs to only select the most gullible people to save on costs.

I think it's not just that you need somebody really gullible - it's that you need somebody both gullible, and dead-set on not feeling like they've been fooled. Extremely gullible, desperate, mentally ill people are more likely to stick, even when they get hints or straight assessments that it's a scam.

At least, that was the impression I got from an old lady who lived near me, who managed to shout down a stream of people telling her that her 'secret agent boyfriend from plymouth' she was sending all her money to wasn't real.

I think there's a bit of a parallel to cults like scientology - the more incrementally ridiculous you make the story, the higher the feeling of shame is when you have to own up to being fooled, making some people go in deeper rather than back out.

To get this approach to work you need to mass email to get over the low hit rate. The next question that follows, is why don't providers crack down on these scammers who need to send mass emails for their numbers to even pencil out? I'm struck this all hasn't happened 25 years ago at this point.
Because newsletters, special interest groups, internet forums, product updates also a valid uses of mass email.
If you can filter that from spam you can also filter these users from those that abuse mass email.
I feel like this has some parallel in the crypto space :)
Greed can make people do stupid things in any space
Oh good, looks like I get to be the one to repost this classic [1]:

"Your post advocates a:

( ) technical ( ) legislative ( ) market-based ( ) vigilante

approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

( ) Spammers can easily use it to harvest email addresses

( ) Mailing lists and other legitimate email uses would be affected

( ) No one will be able to find the guy or collect the money

( ) It is defenseless against brute force attacks

( ) It will stop spam for two weeks and then we'll be stuck with it

( ) Users of email will not put up with it

( ) Microsoft will not put up with it

( ) The police will not put up with it

( ) Requires too much cooperation from spammers

( ) Requires immediate total cooperation from everybody at once

( ) Many email users cannot afford to lose business or alienate potential employers

( ) Spammers don't care about invalid addresses in their lists

( ) Anyone could anonymously destroy anyone else's career or business

*Specifically, your plan fails to account for:*

( ) Laws expressly prohibiting it

( ) Lack of centrally controlling authority for email

( ) Open relays in foreign countries

( ) Ease of searching tiny alphanumeric address space of all email addresses

( ) Asshats

( ) Jurisdictional problems

( ) Unpopularity of weird new taxes

( ) Public reluctance to accept weird new forms of money

( ) Huge existing software investment in SMTP

( ) Susceptibility of protocols other than SMTP to attack

( ) Willingness of users to install OS patches received by email

( ) Armies of worm riddled broadband-connected Windows boxes

( ) Eternal arms race involved in all filtering approaches

( ) Extreme profitability of spam

( ) Joe jobs and/or identity theft

( ) Technically illiterate politicians

( ) Extreme stupidity on the part of people who do business with spammers

( ) Dishonesty on the part of spammers themselves

( ) Bandwidth costs that are unaffected by client filtering

( ) Outlook

*and the following philosophical objections may also apply:*

( ) Ideas similar to yours are easy to come up with, yet none have ever been shown practical

( ) Any scheme based on opt-out is unacceptable

( ) SMTP headers should not be the subject of legislation

( ) Blacklists suck

( ) Whitelists suck

( ) We should be able to talk about Viagra without being censored

( ) Countermeasures should not involve wire fraud or credit card fraud

( ) Countermeasures should not involve sabotage of public networks

( ) Countermeasures must work if phased in gradually

( ) Sending email should be free

( ) Why should we have to trust you and your servers?

( ) Incompatiblity with open source or open source licenses

( ) Feel-good measures do nothing to solve the problem

( ) Temporary/one-time email addresses are cumbersome

( ) I don't want the government reading my email

( ) Killing them that way is not slow and painful enough

*Furthermore, this is what I think about you:*

( ) Sorry dude, but I don't think it would work.

( ) This is a stupid idea, and you're a stupid person for suggesting it.

( ) Nice try, assh0le! I'm going to find out where you live and burn your house down!"

[1] https://craphound.com/spamsolutions.txt

I switched away from Gmail a couple of years ago – I have gotten the impression that spam is only still a thing for Gmail users.

Gmail's spam filters would break horribly for me every few years, either flooding my inbox with spam, or filing obviously-not-spam emails in the spam folder.

With Fastmail, my experience is that all spam goes to the spam folder, and only a few questionable newsletters get put in the spam folder by "mistake".

My personal email address has been gmail since it launched. I find occasionally a spam email gets into my inbox. It's very rare, maybe 4 or 5 per year. I don't remember the last time a legitimate email got marked as spam, I'm not sure it has ever happened to me.
Gmail does suck on both get spam out of your inbox and get real email into your inbox but it's not the worst one around.

And well, that could answer the OP's question, people didn't give up because there are a few providers that will receive their email and show it to users. Take any Exchange service as an example, if your email is technically formed enough to be received (many real senders will be rejected here), it will be received, and it's up to the user to forbid that one instance of spam, like if it's the 90's.

I've been using gmail (Google Apps for Business or what ever they call it now days) for like 10 years and the spam functionality has always worked pretty well for me.
I agree, though over the past 2 years more and more keeps sneaking through their spam filters.
I get a lot more spam with Gmail than Fastmail, but it all goes directly to the spam folder so the experience is largely the same for me.

Hard to say how much of it is that my Gmail account has been around a lot longer and part of multiple email leaks.

I don't have that issue. In general, spam goes into the spam folder, which I can conveniently ignore. This wasn't the case some years ago. I might occasionally mark something as spam. I occasionally check the spam folder. I tell myself this is to make sure I didn't miss anything but realistically, it is so I can laugh at the ridiculousness. Only rarely does something I want get put in the spam folder.
Check "scambaiting" generally, or for example "Kitboga" or "Scammer Payback" specifically on YouTube, for some funny-sad look on how many people can become gullible and vulnerable to various deception tactics. I only hope I'll never fall for one, but never am completely 100% sure; and we are going to get older at some point...