HN Explorer

Tell HN: Gmail's spam filters have gone bonkers

129 points by thyrox ↗ HN
I don't know what is happening with Gmail but lately Gmail's aggressive spam filtering created a huge problem for me.

It was a reply I sent to an email yet Gmail somehow figured out it was spam and the other person never got it (which I learned after 1 week)

Two weeks before they sent the property tax notice (from the govt) in spam and had I missed that email there would have been a penalty.

This is a very old Gmail address almost 10 years or more so I don't know what's happening but I've stopped trusting Gmail altogether after this.

This spam thing is such a black box, so don't be like me and check your spam folder just like you check your inbox if you're on Gmail.

P.S. Also looking for a good alternative to gmail if anybody has some suggestions. Right now I've just created a filter to send all email to inbox.

100 comments

[ 2.3 ms ] story [ 192 ms ] thread
lgl
A perfect anti-spam solution doesn't exist.

Even if you switch e-mail provider, as long as they're also using some sort of anti-spam system, false positives will always happen occasionally.

The obvious recommendation is to always add any trusted senders to your contacts or, as you've already done, create some filters.

Alternatively, switch to a provider that doesn't use any spam filtering at all. But that will open up a whole different can of worms.

intc
We use server side spam filtering at c1 (see: https://c1.fi/about/). So far things are working pretty well but certainly false positives may occur. On the positive side client can contact yours truly and we can usually do something to sort these out.
itintheory
You have a few typos on that page:

Minimzed client tracking on our websites (no 3rd party trakcers)

I believe that should be "Minimized" instead of "Minimzed", "trackers" instead of "trakcers".

I didn't read the whole thing carefully - you might want to spellcheck the entire page.

mindslight
I've never run a spam filter. Basic greylisting stopped the sheer majority of spam, with no chance of false positives (by definition). My greylisting setup fell by the wayside but I haven't bothered to fix it. It's been easy enough to just manually delete spam, most by looking at the sender/subject.
neilsimp1
I have the opposite issue - a LOT of spam makes it's way into my Inbox.
h2odragon
They marked one of my two "Google Calendar" notifications as spam, this weekend.

I think that "providing service to customers" was never very high on the list of priorities there, and has fallen off completely in the past few years.

Anybody working there who still has that as a priority is probably facing an uphill battle against management process and social issues to even open the conversation.

nlitened
Hell, they mark as spam my colleagues’ Google Docs share notifications sent within our own paid Google Apps (Google Suite? Google Workspace?) domain.
coffeedan
They mark all of my Google Calendar invitations as spam. Frustrating…
prepend
I’ve noticed gmail getting worse lately too. Hotmail as well, and I think even much more.

I’m not sure what’s happening and wonder if it’s just spammers being smarter.

I’ve had gmail since a few months after launch and spam seems worst this last year. Today I got two spam messages in my inbox offering to eliminate my energy bill.

And I’ve had OTP codes sent to spam.

This is particularly risky as my banks and pretty much everything are pushing paperless. And I had a mortgage company that wouldn’t ever send paper and was completely digital.

Avamander
> I’m not sure what’s happening and wonder if it’s just spammers being smarter.

The issue is more that spammers are moving forward and finding creative ways to abuse systems but most regular users aren't spending time on improving their posture.

I found the statistics collected by BIMI Radar really illuminating. The state is really bad and it's certainly not helping with the spam: https://www.bimiradar.com/no-top-plc

Ignoring the BIMI part, only 2.3% of domains do the actual minimum they should.

kioleanu
We've had our problems, but Fastmail does a pretty good job. Not perfect, sure, but I don't remember having to send an e-mail to spam in the last 2 years.

Or not receiving e-mails. Or recipients not receiving my emails. I had my doubts, because of the custom domain, but it's been ok until now.

I have it set up to load my gmail e-mails also and it filters spam messages that gmail missed. Meaning, I log in to gmail and see spam messages in the inbox, that never reacahed my Fastmail inbox. In the past months, with about 100% success rate.

wpietri
Yeah, I've been using Fastmail a couple of years and its very good. At least as good as what I was running on my own, probably better. I'm glad I switched to a paid provider; free email isn't worth the price.
rgavuliak
At the same time for me Spam gets through that pretends to be tied to loyalty programs.
Demonsult
The "user as the product" model works well for the user for a while, when business is booming. I now pay Microsoft for email simply because they have someone answering the phone over there, competent or not. I get Office as part of the deal.
[deleted]
(comment deleted)
frogperson
I suspect it has something to do with Gmail's policy change on nov 2022 when they started requiring spf, dkim, and dmarc.

There are probably a ton of misconfigured domains and non-google email servers that interpret the new configurations slightly differently.

paulgb
Maybe that's part of it, but to add some datapoints, I've had a bunch of legit email from GMail addresses flagged as spam, and had people tell me that my emails (from a GMail account I've had for ~18 years) landed in spam.

(This HN post caused me to check and I found another important GMail-originated message in spam)

kccqzy
That's just because spammers have wised up and started using Gmail itself to send spam, including old established accounts via weak passwords. Those spam would pass SPF, DKIM, DMARC.

It would be both inappropriate and probably illegal for Gmail to automatically mark emails sent by Gmail as non-spam.

gandalfian
Still better than the filters that just invisibly destroy spam without ever telling you. (I'm looking at you namecheap and numerous others. And my aunt's isp that vaporised any incoming email containing the word chicken and had no one who could help. Most British isps have given up having email at all, the expertise has gone).
[deleted]
(comment deleted)
quags
The silent discard is the worse. I see it more frequently on hotmail/outlook but it happens on gmail too. Name cheap last I recall used CPAnel which is a basic email set up and a support tech with a bit of clue should be able to identify the issue from the exim logs easily.
warrenm
>This is a very old Gmail address almost 10 years or more so I don't know what's happening but I've stopped trusting Gmail altogether after this.

I haven't "trusted" Gmail (except to stay online) for a lot longer than 10 years (and my 'main' Gmail address goes back to when it was invite-only almost 19 years ago)

MandieD
My home state’s (Texas) online ballot printing site for small counties got its mails to my GMail sent to spam in 2018 and 2020, and not even that far in 2022.

That domain doesn’t have SPF or DKIM configured, and the link to click for generating the ballot is HTTP and an IP address (not even a proper DNS name), so no wonder.

However, I’d expect those mails to be delivered anyway since I marked the address as safe each time, but here we are.

gameshot911
>However, I’d expect those mails to be delivered anyway since I marked the address as safe each time, but here we are.

Exactly - this is the biggest thing for me. I understand that some mail may erroneously be identified as spam, but when I explicitly mark it as Not Spam, I would expect my mail provider to respect that decision (for my inbox personally, at least).

I had a re-occuring issue where the payment receipt from my small gym kept going to spam. I think after marking it Not Spam for like 6 months it eventually started appearing in the Inbox sportatically.

its-summertime
but there is no ownership of the mail sender, so how would it know if its the same mail sender?
causi
"Safe" and "Spam" buttons in Gmail are like the button on a crosswalk. Sometimes they do something, sometimes they're not even hooked up.
ultrarunner
And just like even the buttons that work are prioritizing cars potentially five miles away, they were never intended for your benefit in the first place.
nicoburns
> However, I’d expect those mails to be delivered anyway since I marked the address as safe each time, but here we are.

Presumably if there's no SPF or DKIM, then it's impossible for Gmail to verify that the mail has actually come from the address you whitelisted. It could just as easily be a phishing scam. IMO it's quite reasonable to reject such mail by default.

ipaddr
The same ip would have been used in both cases and the person receiving is logged in and can connect the mark safe event and the incoming email.
readonthegoapp
my gmail filters started blowing up some time ago -- a year or two?

i variously deactivated some/all/etc., re-activated, re-created, broke-out, aggregated, etc.

none of it really seems to matter.

i don't actually get a lot of mail, i just stopped checking all day every day, so i started missing a lot of stuff b/c i would just never catch up.

rwmj
Gmail spam filtering has never worked. Even after "training" the spam filter for months on end it still filtered obviously good messages to spam. Now I download everything, do the spam filtering locally, and read it through mutt.
epaga
I have had Google's own Google Drive notifications land in my Gmail spam. Crazy to think Gmail used to be best-of-class when it came to spam detection.
Avamander
There is quite a bit of Drive, Calendar and Forms abuse, so not surprised a few legit ones end up in spam as well eventually.
PurpleRamen
This can happen even with the best filter, as long as there is some dynamic outside your control. For me, it always flags the newsletter from the Elite Dangerous-makers (Frontier Games?). And even after setting a filter with the flag to not handle it as spam, it still gets a warning of being a potential flag in the mail-view. That's just the world of our new AI-Overlords. Just cope with it, always check your spam-folder and be nice to the bots.
snapplebobapple
This is a radical idea, but this thread is full of continued abuse from what seems like an uncaring partner that is only getting worse so that seems to justify it: you guys and google need a divorce. I suggest fastmail but there are lots of decent options.
1shooner
I switched to fastmail about a year ago, and I still have my gmail piping into it as an external mailbox. Fastmail's spam filter consistently catches gmail's false negatives.
2000UltraDeluxe
This is a well-known problem in the industry. Gmail's spam filtering is a black box with no apparent way to prevent emails from randomly ending up in spam.
a9h74j
> randomly ending up in spam

If only we had some technology that could reliably execute if-then instructions.

(Recalling a time when people would say: "If only we had something [Google] that could look up an answer for that.")

schlauerfox
The incentives have really fallen apart in technology. You have a world of people relying on services they don't pay for because 'ads', equipment they don't own because of 'subscriptions', and middlemen between every interaction we have with each other taking a dollar and policing the discourse. Unhealthy in all respects.
2000UltraDeluxe
I get that it's a black box. After all, nobody wants to tell the spammers what the ingredients of their secret anti-spam sauce are.

The problem is that it's obviously not working; the amount of false positives are staggering and from what I've heard they're still letting a lot of actual spam through.

But, as the sibling comment mentioned, there's little to no incentive for the freemail giants to improve the situation when they're themselves offering paid services with better spam filtering.

Euphorbium
I constantly get spam in my inbox, I think that is because they classified me as an unproffitable client and want to boot me off the service.
SamBam
For me the most egregious way that Gmail is failing recently is that its search 100% sucks. It sucks in a way that's almost unimaginable. It uses fuzzy matching and single words when you use multiple terms.

I search "stock transfer" to find an email in my main mail from last week. It returns dozens of junk emails from my "updates" and "promotions" folders that just contain the words "share" ("share your recipes!") or "security" ("update your home security") or "inventory" (??).

Whatever happened to the promise of Gmail (I remember 2004) that folders and tagging were a thing of the past, that you could use the awesome power of Google to "just search" and the email you wanted would pop up? What happened to the Google that claimed to know whether your search for "bass" was for the fish or the guitar? Now it can't even tell that a search for "stock transfers" is not for "share your recipes?"

And yes, I know I can use quotes. Searching email shouldn't be something only the power users know how to do.

lexapro
Using quotes in search is hardly power user knowledge.
BitwiseFool
I yearn to return to the days where search engines worked on keywords. When you could explicitly tell it what terms you were looking for. None of this "trying to guess what you really meant" stuff. I've found that all of the old 'google-fu' techniques only work incidentally. Perhaps it is because they want people to ask questions using natural language? I understand that an "answer engine" has its use cases, but that shouldn't be the end-all-be-all of what it means to query something online.

I have no idea what kind of metrics lead Google towards fixating on the most generic terms of a search, but it is infuriating that Bing is moving in this direction too. It seems like even Google's competitors cannot help but follow its lead.

placatedmayhem
> I have no idea what kind of metrics lead Google towards fixating on the most generic terms of a search

Total shot in the dark, but I'm guessing that interpreting a query string more generically allows Google to serve up a broader collection of ads.

There used to be an operator in search engines of yore (including Google, iirc) that specified, basically, "expand beyond exact matches" to include synonyms and such, maybe "~". Google has basically inverted that, so exact matching requires explicit input. I also find it frustrating.

lokar
Matching query terms against content vs ads has always been separate. On the ads side there a pretty complex systems to expand the query to match more.
calr
I’ve had the same experience. Search is unusable far too often for general keywords; I end up making a failed keyword search and then fall back to making a search based on sender address. Searching on sender address works fine but requires me to correctly recall who sent an email I’m looking for.
theossuary
You can also use category:primary, I add that to all my searches, gmail search is useless with out it.
bondarchuk
It's funny that search quality is declining in mail too, because it runs counter to the claim that the decline in quality of www search is strictly/mostly due to spam sites/adversarial SEO getting better.
mattwad
I don't know why ,but the search results in the suggestions in the search input dropdown are way more accurate than what shows up after you actually submit it.
SamBam
Yup, absolutely. It shows that they can provide good results if they want to.
[deleted]
(comment deleted)
mattgperry
I have the opposite problem recently where a very common class of spam (usually "you have won") just slips straight into my inbox no matter how many times I flag it. It looks so obviously like spam it's quite shocking it doesn't get picked up.
dhruvarora013
I have one of these in my inbox every single day and it is so incredibly frustrating! I've banned the entire phrase using a filter but definitely not ideal
mh-
I've been getting multiple of these per day. Nearly identical to one another, super spammy text and from obvious low quality email addresses. Reporting them as spam diligently doesn't seem to do improve anything.
mynameisvlad
I gave up on my Gmail over a year ago due to these emails. Flagging does nothing, they still come in.

Ended up switching to my own domain and while there’s still a few places which don’t let me change my email, most things weren’t very difficult to switch over. It did take a long time since I did it organically for the most part.

leohart
I am considering doing the same thing. Do you end up hosting it via Gmail or is there a provider that you go with? I am looking for something that doesn't have too crazy of a limit, have iOS/Android support and allows *@domain.name to go to the same inbox.
mynameisvlad
Used Hey for Domains for a year and change. Ended up switching to Fastmail fairly recently and creating flows similar to Hey using filters and labels.

Fastmail has been absolutely great so far, no complaints. Powerful filter/rule engine, the ability to choose folder or label systems for your mail, a great app but also IMAP/DAV access for inter-compatibility, and tons more. I haven't played around too much with a bunch of the other things it offers (calendar, files, notes, etc) but for mail and contacts, at least, it's been solid.

I have a wildcard setup for vlad.gg so all emails go to my inbox, with specific aliases set up that get auto-labeled using filters. I also have wildcards set up for various other domains and then on a family domain, I have aliases that redirect emails to their own personal inboxes (on Outlook/Hotmail FWIW). All super easy to set up.

0xBDB
Protonmail will let you point a custom domain at them and deliver to the same mailbox as your protonmail e-mail, starting with their lowest subscription tier (which I personally would have signed up for anyway to support them). Product experience is pretty good. I don't disclose that address to many people, so I can't vouch for the spam filtering.
andrewflnr
My guess is that the problem OP is having is a reaction to this older problem. I did notice that one of the really obvious, persistent categories of spam started actually going to spam pretty recently.
WastingMyTime89
I have both some extremely obvious false negative (garbage content, title which is obviously spam, phishing related address) and a lot of false positive including some which hard extremely hard to understand (daily email with always the same title and from the same address - one randomly ends up in the spam folder).

At this point, I think Google is just intentionally sabotaging Gmail.

extasia
The spam filtered out an urgent email about my Doctorate application.
justusthane
> P.S. Also looking for a good alternative to gmail if anybody has some suggestions.

100% Fastmail. I've been on them for eight years now, after switching from a decade of Gmail. I've had zero complaints.

Switching from Gmail isn't even that bad - you can keep your Google account using your new email address as the ID, so that you can still use Docs and Drive and such. I wrote a short guide here: https://www.justus.ws/tech/how-to-ditch-gmail/

jpmattia
> P.S. Also looking for a good alternative to gmail if anybody has some suggestions.

We've been self-hosting. The first few weeks cost some extra time as you add items to the spam filters, but it works pretty well. Our system kills off about 1500 spam emails a day, probably 2-3 get through but most of those are obvious ("Your funding is approved for up to $700,000!!!" or "Did you get my previous email?")

edit: I should add that we're US based, so disallowing all domains besides .com, .edu, .org is easy and useful.

mynameisvlad
> I should add that we're US based, so disallowing all domains besides .com, .edu, .org is easy and useful.

A bunch of sites nowadays use TLDs outside of those. io and tv are notable TLDs for countries but used by entities all over the world. Then there’s the new TLDs like app, etc which are increasingly being used.

Have you accounted for these? I feel like their usage is only going to go up since there’s more names available.

jpmattia
My edit above is a lie, after checking the config. (Re-editing that comment has timed out.) I'm in the TV business, and yes there is a decent amount of valid traffic from .tv

After checking, there wasn't a whitelist, but it was a longish blacklist: .icu, .xyz, .pro, .date, .science, .top, .today, .download, .work, .click, .cloud, .link, .diet, .review, .party, .zip, .stream, .bid, .me, .website, .buzz, .cyou, .monster, .casa, .bar, .shop, .sbs, .quest, .cfd, .live, .cam, .rest, .life, .art, .site, .digital, .beauty, .online, .club, .email, .best, .ru.com, .sa.com, .za.com, .in.co, .co.in, .rch002.net

I'm sure there are others that will be blacklisted in the future too.

amelius
Search in Gmail is bonkers too.

I can't even find mail back using a literal search for text that appears in a recent e-mail.

I thought Google was a search company ...

DamnInteresting
> I thought Google was a search company ...

They are an ad company that once used a nice search engine to serve ads.

rurp
I've been having this issue for years, with some really awful misses on the part of Google. For example emails from relatives whose emails I almost always respond to sometimes get spam binned. Why?! God know how many valid emails have been lost over the years because I forgot to check the spam folder for a while.