Google sends me reports on the spam we get to one of our mailing lists.
I hadn't seem them in a while and found that they themselves were being sent to the spam folder.
edit - I'd forgotten about this actually, and checked, it happened again today.
It's apparently happening because people send me spam, so I mark it as spam. Because I've marked things sent to that email address as spam, they're saying their own report is now spam. The description of the problem is sounding more python-esque as I continue so I'll stop there.
You say that like Google cares, over the years they've amassed a large swatch of consumer email users (probably due to Android) - they have basically gotten to a position where they set the rules and if it hurts you tough luck.
Email deliverability is a hard problem these days, spam has caused us so much grief that large email providers are doing really annoying things in their spam filters to combat the problem but I've noticed more and more legitimate email is getting caught in the crossfire. Unfortunately, I don't see this being fixed anytime soon - and it's personally making my life increasingly difficult as someone who had to give up self-hosting his email and switching to a hosted Zimbra service just to stop getting flagged by Google as spam (even though I passed DMARC/DKIM and SPF validation and had a non-blacklisted IP).
> as someone who had to give up self-hosting his email and switching to a hosted Zimbra service just to stop getting flagged by Google as spam (even though I passed DMARC/DKIM and SPF validation and had a non-blacklisted IP).
There is a trick how you can get out of the spamfilter: Send mail, regularly (like a few mails per week) to friends/family that use Googlemail, and tell them to mark your mail as "not spam". After roughly a month, you’ll have gathered enough points that Googlemail won’t mark you as spam anymore.
The people sending these newsletters aren't even their customers, for the most part. They're customers of AWeber, Mailchimp, etc.
And for the people receiving these emails? This is a nuisance at best. Although if 99% of the population stopped receiving emails from their friends, etc, THEN this would be a bigger mainstream issue. Right now? Nobody's worrying (especially in Google right now) except a few newsletter senders.
When I saw the friend was using Mailchimp, it made a lot more sense.
I get lots of spam from Mailchimp (ie. people subscribing me to their newsletter, without my consent), and complaining to Mailchimp about it seems to have no effect.
So since Mailchimp allows bad actors to send mail with their service, it’s not entirely unfair that Mailchimp mail gets flagged as spam.
The author uses Aweber, not Mailchimp. Besides, Mailchimp is quite strict with their opt-in policy and they don't hesitate to suspend accounts that ignore it.
Sorry, I wasn’t aware that was a mail service, my bad. His friend (from the iMessage convo) was using Mailchimp, though.
> Mailchimp is quite strict with their opt-in policy and they don't hesitate to suspend accounts that ignore it.
They might say they’re strict, but almost any time I give my e-mail address some where online, I’m automatically signed up for some Mailchimp newsletter, and Mailchimp can’t even be bothered to reply if you report it.
Google could get really evil in this regard. But then again Facebook has been doing this for years. "We didn't think the content was relevant to this audience. Pay us some money and we will allow your email to reach more people..."
This is already the case today. I get so much irrelevant mail in my main inbox that I ignore a lot of legitimate email in the process. And you can buy Gmail ads which appear in the top of your inbox.
I've had emails from family members go straight to gmail's spam folder on two different occasions, and a third occasion in which the mail appeared to have sent according to the sender's mailserver, but never showed up anywhere in my gmail account. From what I could tell glancing around some forum posts, mine was hardly a unique experience and is reliably solved (well, in the first two cases) only by explicitly whitelisting every single address from which you expect to receive correspondence. I'm glad to see more attention to this issue of overzealous filtering.
I only skimmed this so forgive me if I missed something. But could this simply be what happens to you when a sufficient number of email recipients classify you as spam, and you then reach some critical threshold?
>> I changed my sender email address to see if would help
But if the pattern of content got flagged as spam by Google users, the sender address might not be the only determinant.
>> I was relieved to discover that others with newsletters were having similar troubles very recently.
But if this results from crossing some sort of user-classified spam threshold, one would probably always be able to find some recent "victims".
This is not intended as a criticism -- I'm sensitive to Google's seemingly arbitrary power -- just an idea to consider or rule out.
That seems an unlikely thing to have happen to multiple high-profile newsletters at once in a short span of time. Particularly, if these people aren't seeing big changes in their users, why would the spam threshold be hit now? Did a large chunk of email readers suddenly decide enough is enough?
Then you'd expect this article to appear on the FP every week, surely? It seems more likely to me that whenever people write about something out of the ordinary, it would not happen ordinarily - otherwise there wouldn't be anything to write about.
No you'd expect people to be complaining about it somewhere at some frequencly...and it hasn't been established that isn't the case. However, if you search for "Google newsletter suddenly spam folder", you can easily find lots posts that are superficially similar-looking, from various time periods.
Maybe Google changed the threshold? Lets say 20% of recipients have historically classified the list as spam, and Google just adjusted the spam threshold from 25% to 15%.
Another possibility is that the spammers have adapted and rolled out the next phase of their plans. After creating a million or three Gmail accounts (aged, so they're all months old, and staggered, so they're not all created at the same time, and with some amount of fake "organic" traffic such that they seem they're owned by real people), the obvious first phase, as a spammer, is to sign up for your own newsletters, and mark those as "not spam" from your three million fake accounts.
As a spammer, you've got to assume Google's going to catch onto your first three million fake accounts and mark both those accounts, the email servers' IPs you use to send, and the email list itself as spam. Consider your first three million gmail account burned by that point.
The less obvious second phase then, using your next three million fake, aged, but-with-fake-email-traffic-to-make-them-seem-real Gmail accounts, is to sign up for not-spam newsletters, but mark those as spam. The wishful-thinking hope is that with enough of those marked as spam, the wider public will see more articles like the linked article, and ever so barely push back on Google, who then have to figure out another way to distinguish signal from noise in a vastly complicated and ML-enchanted algorithm.
I suspect it's a consequence of Google not caring whether or not you're actively subscribed to the newsletter, and treating it as spam based on all other signs.
Checkboxing "[ ] I want your newsletter" on some website doesn't inform Gmail to whitelist that site.
Similarly, it could have been 3-4 consecutive newsletters with particularly low engagement that put him across some threshold. Not people marking as spam, just not opening or opening and quickly deleting.
Personally, I can't remember the last time gmail marked anything important spam. Maybe a newsletter whose absence I hadn't even noticed, but nothing actually important to me.
I just wish I could get gmail spam filtering on my cell phone. Robocalls are out of control -- 4-5 / day. (I've looked at some of the apps / subscriptions for this, but I'm not ready to give a relatively unknown 3rd party access to my contacts or to adding contacts, which they generally require.)
Try making google voice your main number? They had solid robo filtering if I remember right. I'm on Fi now and I can't remember the last time I received a robo call.
> a sufficient number of email recipients classify you as spam
From Google's end, I can understand this decision. But from the consumer end, I only want that logic used for de novo email sources.
Even if many people think a newsletter is spam, I'd want my personal patterns to override that. If I've opened letters from the same sender repeatedly and never marked one as spam, it shouldn't much matter what other users have to say. If I've ever explicitly taken a sender out of spam and not put them in spam, that should basically settle the matter.
I regularly lose emails I've requested to Gmail spam, and even when I've un-spammed the same sender multiple times. I appreciate that Google achieves pre-emptive blocking via a large userbase, but I don't appreciate when the wisdom of the crowd overrules my own behavior.
The spam problem has never been solved, no matter how good spam filters may have become in the recent years. It's still an arms race, with a front that is moving one way or another every few months. Even today, a spam filter is a piece of software that you trust to throw away e-mails without letting you read them, based on its own decision that you don't want to read them.
It seems insane to me that people would willingly let a program decide to throw away inbound mail based on unspecified rules. And I nearly start frothing at the mouth when someone suggests that, since their software decided to throw away my e-mail, it is somehow my responsibility to fix that. It isn't.
If there were thousands of different anti-spam solutions, and one of them behaved badly, then senders wouldn't care, and users would vote with their feet (or wallet, or app store) until the behaviour was fixed. But since Google has a near-monopoly on e-mail, it's now up to the senders to do the work and jump through the badly-documented ever-changing hoops.
In the younger, private mail account demographic, I’ve seen as much as 80% of some e-mail lists being Gmail accounts. For business, Exchange is still king, but Gmail is gaining on them.
It's scarey stupid how much the "big 3" own email now. Google, Yahoo, Hotmail. When any of them change up their spam fighting techniques it causes ripple effects across the entire email ecosystem now.
Have Yahoo mail subscribers on a Mailman mailing list? Not if you're not running the latest Mailman which has a new workaround just for them. Yahoo started bouncing legitimate mail a couple years ago due to the normal way mailing lists have ALWAYS handled sender addresses, since before Yahoo existed.
I get it, sometimes desparate times call for desparate measures (for fighting spam). But jeeze, it's super annoying to have to break your package manager to get an upstream mail server to support one user who uses one certain mail hosting service.
I send newsletters (that people double opt in to and want to receive) to about 360k developers each week and a good 90% are on Gmail or have their domain on Gmail. This is very specifically developers, however.
> It seems insane to me that people would willingly let a program decide to throw away inbound mail based on unspecified rules.
See, you say that, and until a few days ago I'd have felt the same. But that was before I registered the first .com domain I've ever owned, and oh boy has that brought them out of the woodwork - I've gotten more spam in the last week than in any of the past ten years prior! Not just email, but phone calls, too; I even heard from one of those tech support scams, and it's been so long since the last time that happened that I couldn't even tell you when it was.
It seems to be settling down in the last day or so, which is nice. But if it picks up again, I'll absolutely be installing SpamAssassin on my MTA host, for the first time in the going on two decades I've been running my own mail.
To be clear, I totally agree with your point about Gmail - it's far too centralized and unaccountable for my own comfort, too, which is part of why I don't use it for any mail I care about. But a lot of my correspondents do, which means I'm nonetheless in the position of either taking assiduous care to maximize my deliverability, or risking my mail landing in spam folders that nobody ever really looks through. Not a good situation, but at least until someone comes up with a plausible means of addressing it from the outside, here we are.
I don't know where you registered your domain, but I have like 100 domains, and I've never been contacted about them (except by the registrar of course). Most of them have valid and router MX.
GoDaddy, and maybe that's my problem. On the other hand, I've been doing business with them for many years, and never had anything remotely resembling this happen before. I also didn't use whois privacy, but, again, I've made that choice on many domains before, and likewise never seen anything like this result from it.
That squares with what I'm seeing, yeah - a lot of cheaply automated nonsense trolling for a few suckers.
Is there anywhere I can look for more information on how this model works in detail, and how to avoid the same problem in future? It's a real pain in the neck. Thanks!
Except it's not new at all. Since I first got my email on my primary domain in 2008 or so I've had admin@, postmaster@ etc., Along with the address in whois all being spammed.
Some domains have been used as shit magnets and you can inherit that mess if you pick it up quickly after it's been dropped.
I wouldn't believe it myself, until it happened to me. I've run small mail servers for various domains for years with only "average" amounts of spam. Then one time I picked up a personal domain of my handle (not this one) which I'd wanted for years and I discovered it finally was free. The moment I setup an mx for that domain I got a firehose of spam coming in to it. In fact it was so bad I had to unreg the mx and leave it for another year before it was worth doing anything email-wise with the domain. Just the bounce processing was driving cpu and bandwidth more than I cared to deal with, even if it wasn't receiving any local mail.
I checked the IP and the domain before I went with either of them, and their reputations are both clean, so I don't think that's it. I've definitely had that kind of trouble in the past, though.
It'll take some doing for anyone to beat the long conversation I had to have, years back, with a representative of a very large film studio, who were concerned that a domain I'd registered might verge uncomfortably closely on their mark.
Worked out in the end - it only took them a few minutes actually looking at the content to realize, on the one hand, that it was clearly not being used for any commercial purpose, and on the other, that there was no meaningful risk of confusion between my domain name and their mark. They were happy to wait to buy it until I was done with it. But it was a surprising call to receive at seven o'clock on a Saturday evening!
At least personally as far as I can tell the spam problem has been solved by gmail. I can't even remember the last time I saw a spam message in my inbox and can count on one hand the number of times I've had to go into the spam folder to find a legitimate email.
> Even today, a spam filter is a piece of software that you trust to throw away e-mails without letting you read them, based on its own decision that you don't want to read them.
The way I set up rules on my mail server, they all just go into a spam folder that I browse every once in a while. If anyone is going to delete my shit, it's me.
The hilarious part about Google's spam filtering is that I have crons that send logs in emails to my own address using Google's own smtp servers (that require authentication with my own credentials) and they end up getting placed in spam.
That's hilarious: A newsletter about how-to-spam is treated as spammy and the author is upset. In the end, everyone thinks they're one of the good guys.
> The aim of this blog is to help anyone with a website find proven content ideas that they can use to drive more traffic and links to their own sites. The concept is already working, with one recent article helping a reader get on the front page of Product Hunt.
This is instantly what I thought too. Essentially this guy is teaching how to get rich by using SEO, catchy gimmicks/ideas, and the likes. The spam teacher themselves get filtered, how ironic.
Yes, welcome to the world of Viperchill - the shadow king of get rich by teaching others how to get rich schemes. I wouldn't be surprised if Google personally targeted him, I would do :)
Heh - I wouldn't be surprised if the language used sets off the filter. Machine learning isn't great at abstractions like "writing spam" versus "writing samples of spam someone else might write", so I imagine this sort of topic is particularly prone to filtering.
With a title of "Launching my new daily blog (and podcast)",
and sending in bulk, I'd expect a good spam filter to send it to spam.
"I’m Starting a Brand New Live Case Study. Over one hour of free training in today's video..." was also flagged as spam. On keywords alone, those titles just scream "spammer".
As much as I'd like to reduce my reliance on "cloud" services, Gmail is one of those that I just can't quit, since they've pretty much solved the spam problem for me. I'd rather a few false positives get filtered than the deluge that comes with inadequate (self-hosted) E-mail.
Sorry if this guy's "audience building" business is affected, but as an end user not worried about "my open rate", I appreciate Google's aggressive filtering and urge them not to let up.
That’s fine for you, but believe it or not, there are actually people who sign up for newsletters voluntarily because they want to stay informed about something. Google shouldn’t try to override the user’s preference about receiving newsletters.
Mark a few of the messages as important and you won't miss future ones. Given how pervasive getting signed up for email lists is even when I don't really want to be on them, the opt-in model makes more sense for me than the opt-out model.
Mailing lists that aren't double opt-in are way more likely to be managed by Google on behalf of the user than those that are. The spammy mailing lists are way more likely to get spam-filtered by users who were "opted-in" to them by the mailer.
Google's solution is necessary because bad actors in email abound.
More or less any business that sells domains also offers POP3 or IMAP mailboxes with spam filters and some web mail interface (which I'm not using).
That's what I've been using for my two primary domains (vanity personal and business one) since 2004 and 2006. Those filters work pretty well. I can't compare with Google's one but they're enough. I prefer to get some extra spam than losing a business email and maybe a new customer.
Maybe there is a way to keep using Gmail, read from an external POP3/IMAP mailbox and disable Google's antispam on messages fetched from there.
A reasonable sentiment, for sure. I'm curious though, have you tried self-hosting email recently? Spam-assassin with razor2, pyzor, and Bayesian learning is really working pretty well for me. I train every 6-12 months based on my most recent 10k spams. I get a few false negatives but I can tune my spam threshold so that swings to a few false positives with one quick vim session.
Dovecot + Thunderbird even gives you client editing of server-side filtering based on all sorts of criteria, and self-hosted webmail (e.g. MailBox) is pretty good these days as well for when you're on the go.
It's been on my list of things to do "some time later," as I understand the tools have gotten much better. I suppose my belief that self-hosted E-mail necessarily means inadequate spam filtering may be a relic of a time long past.
Spam-assassin soaked up too much memory for my cheap, tiny virtual server. I recently set up rspamd, which seems to work as well and, while it is still the largest process on my machine, hasn't forced a reboot yet.
The thing is, I run my own personal email server. It's not used for spam ever. It's only personal email to family members, friends, and the odd reach out to support. The volume of email is so small it's ridiculous. I might send up to 10 mail (1 receiver each) per month. I set it up with SPF and DKIM properly, triple checked.
And yet, for whatever reason, sometimes my email end in spam. Generally when I send attachment (like pictures). But sometimes it's just completely arbitrary.
How about this : for you it might be a "few false positive", because so much of the wanted mail traffic originates from google in the first place. But for the few of us that use mail as was intended, this becomes a huge problem.
Having a mailing list doesn't necessarely mean the content is unwanted. I'm subscribed to HN replies for instance. I want those email. If it started going to spam, I might consider "hey it's a few false positive". But for the HN replies guys, it would be maddening to be unable to fix the problem because Google can't be arsed to have a support channel.
From my POV, Google is way past "aggressive filtering" territory and way into "monopolistic behavior".
Yeah, I’ve seen the same, to the point that I’ve actually considered getting a Gmail account for my business, because customers not receiving important e-mail is bad for business.
Ok so as much as I promote self-hosting email, you're right. I've also had trouble with mail originating on my server to gmail and only gmail accounts. I did recently discover that I had some configuration issues via mail-tester.com though (my rDNS didn't match the HELO of my server) so now that I'm getting a 10/10 on that, things seem a bit better.
I definitely felt that gmail was being too aggressive on this too. I sent a bunch of detailed replies to middle/high schoolers working on science projects when they reached out through my public-education webpage and when they went to gmail I never got a reply but if they went to some other server, the conversations usually continued nicely. That was pretty frustrating.
> I set it up with SPF and DKIM properly, triple checked.
And yet, for whatever reason, sometimes my email end in spam.
It's probably your IP address.
Sure, SPF and DKIM say you are allowed to send these emails, you are who you say you are, etc, but you also aren't a Sendgrid IP address (or someone similar) so you're treated as less trustworthy.
I could be wrong, but I believe many email servers take these things into consideration (please let me know if I'm mistaken!)
This is fairly common - if you want a method that gets you around this at the cost of some privacy, you can use Mailgun or a similar service for free as your relay, which should pass most spam filters without issue.
Mailgun has similar issues though. We advise those signing up for accounts on our open source community site not to use Hotmail since mailgun emails tend to get binned by Hotmail. Not sorted to spam or bounced, straight up deleted.
I run my email through google for my domain, and the IP assigned to my mail server is actively used to send spam (like, daily active reports on SORBS), so I can't even fill out their form to get it unblocked.
As a result, I need to email my bank from a different email address. So stupid.
If you're literally running an e-mail server from your cable/DSL connection, you're much more likely to be marked as spam, regardless of doing everything else correctly (SPF/DKIM/reverse DNS), because there are lists like SORBS DUHL [1] and RBL-DNS DUL [2] that index these. Some servers block entirely, others just use it as an indicator that your mail is more likely to be spam.
Many spam filtering systems work on a 'points' system, so being on the dynamic IP list adds a lot of points, and if you also have other things that add points (being a mailing list, having 'unsubscribe' links, attaching images, having links) it might be enough to put it over the edge and get marked as spam.
You can do a lookup [3] to see if you're on any of these lists.
Although what you say is true, I've been running my own mail server out of my house for years with proper spf/dkim/dns and have never had problems of my mail being sent to spam.
I had my personal domain hosted with Google. One of the reasons I moved away is that emails that I was sending to other Google Apps domains were bouncing as spam when the destination address was an alias. The bounce came from Google Groups, which is what Google used for handling aliases within Apps domains.
That particular example was sent via SMTP, but I had the same problem when sending via the Gmail web interface, and it occurred sending to at least three different Google Apps domains.
After I moved my domain off Google Apps (I switched to Fastmail for a variety of reasons, but that issue was the kicker), I was able to send to those same addresses without issue. In fairness to Google, I was on the Google Apps free tier at the time, so there was nowhere to go for support.
ARC was designed to fix that kind of edge case, and Gmail was one of the first to start signing and validating ARC within the last couple months - so this may no longer be a problem.
What's the edge case? That the destination address is an alias? That I was using a personal domain? That Google implements aliases using Google Groups?
I never understood what was causing these bounces. The entire email chain occurred within Google's control, passed SPF and DKIM checks, my domain isn't on any blacklists nor used for spam that I know of, and isn't used for anything but personal emails. And I've had it registered for years. Nor was it the content of the message. I had a friend with another Google Apps domain setup an alias for testing and when I mailed that address it bounced the same way, regardless of the message body. Emails to other addresses in his domain were not marked as spam... it was only Groups aliases that bounced.
Whether by monopoly or sheer blindness, Google leans really heavily towards only letting massive institutions through.
I know a few people whose professional work involves a lot of delivering explicitly-requested, non-spam emails. With most email providers, they don't go to spam because they aren't spam. If they get flagged as spam servers (usually right after spinning up an instance, when the confidence is low), they can contact someone and get it sorted out.
With Gmail? They have extensive lists of tips and tricks to "sneak through" emails that their users specifically asked for. They 'prime' servers with extra-safe traffic to avoid getting frozen in the low-confidence early emails. And they still regularly run into problems. They just don't have the size to be trusted by Google, or to get business contacts to solve the problems that do arise.
It's a pretty concerning state of affairs - Gmail will deliver anything sometimes, but it seems to be a very high bar (vastly higher than "not spam") to ensure you don't regularly drop messages.
This is exactly my experience sending legit commercial emails to gmail. We jump through every hoop, we implement every suggestion, and they drop our email on the floor every time. I have no problem with deliveries anywhere else. They DGAF.
The idea that their spam filter is particularly good is ridiculous too. I get more spam to my gmail accounts than to my self hosted mail accounts with greylisting and spamassassin.
Nothing in the RCFs says emails are supposed to be instant. Reliable, yes. Instant, no.
First, greylisting is HIGHLY effective. Something like 95% of email connections are spam. Almost all of them fail without ever hitting disk, via greylisting. If they actually retry, then the chance that the sender IP or email content is in an RBL is MUCH higher, so spam that actually retries will be more likely to be rejected at HELO by an RBL or filtered after it is accepted. The incidence of false positives with grey listing is VERY VERY low. Using greylisting means that very few emails need to be accepted and then filtered. Those are the false positives that really hurt. The sender receives no bounce, and the recipient probably doesn't look at his spam folder every day, especially when it is full of hundreds of junk emails. Greylisting makes email more reliable.
In addition, better greylisting systems note senders that reliably retry and automatically whitelist them. My mail server actually delays less than 2% of legitimate emails due to greylisting.
I'm completely confused by Gmail's spam filter. We've already discussed its extreme tendency to false positives, but it's quite good about false negatives - no spam hits my inbox.
Except... Google claims >90% of spam never even reaches your Spam box, only the ambiguous messages. So why do I get 10x the spam at Gmail as at any other account? Are they not actually good at detection? Are they just a high value target that gets hit more often?
Or (my guess) are they trying to get clever? Google Doc's spellcheck is infamously bad because they're trying to use machine learning approaches without having a simple English dictionary as a fallback. Is Gmail trying to detect solely on email sender/content and totally abandoning approaches like greylisting?
Fully agree. I am in the same situation as you, and I even relay my emails through my ISP SMTP to avoid the issues such as 'residential IP marked as spammer'.
And yet, Gmail will randomly classify simple personal emails. It is the only mail provider to act like this, I never had any issue with any other provider (ISPs, CSPs...)
And to add insult to injury, they have this page: https://support.google.com/mail/answer/6227174?hl=en&ref_top... but as I send too few emails, it is completely useless.
> Having a mailing list doesn't necessarely mean the content is unwanted. I'm subscribed to HN replies for instance. I want those email. If it started going to spam, I might consider "hey it's a few false positive". But for the HN replies guys, it would be maddening to be unable to fix the problem because Google can't be arsed to have a support channel.
I'm not sure this argument is related to the rest of your post, but FWIW, GMail/Inbox puts emails from mail lists into their own "Forums" category, not in the spam folder.
SPF & DKIM are only the start of the story. Use a non consumer IPv6 address, TLS, and DMARC. Have perfect DNS (including correct reverse lookups). Sign your zones. Monitor your IP and domain status with every reputation list on earth. Sign up for the postmaster tools at Google and Outlook.
Running your own email ain't a hobbyist thing anymore, you gotta play like you're in the NBA. I'm doing all the above, my deliverability is solid. And pay attention to the changing landscape - for example, I will be implementing ARC real soon.
I think this is a matter of perspective. My response was to someone who, in my assessment of what they'd listed, had more of that work to put in. Note they were also talking of using email as a file transfer protocol, which you may not be doing. Also your years of operation may (as with my 17 years of spamless service) have built a very positive reputation for domains and addresses.
I mean, DMARC, ARC and TLS make perfect sense (though I have none setup because DMARC is nuclear, I didn't know about ARC, and TLS is just up for the submission port for me). But signed DNS ? What does that protect against, malicious registrar ?
And the postmaster tools don't work for me on google because the volume of email I send is too low. So there's that.
On a sidenote : It's about time something like ARC comes out. If I understand it correctly, it should fix many setups that involve mail forwarding. That'd be a godsend for me, I'll probably set it up soon enough.
Eh, you have the wrong approach. Just do everything and assume they all contribute to reputation. Maximise, don't minimise, your appearance of being a well-run ESP that takes internet citizenship seriously. For example, zone signing: I don't know if it makes a difference to domain reputation, but it might, so I do it.
DMARC is not a nuclear option, no-one starts out with the reject policy, in fact most of my domains are still p=none and I monitor the reports.
You absolutely can and should - irrespective of how much information it gives you in return - verify your domains in Google's Postmaster Tools, monitor your IP addresses and junk mail tagging at Hotmail via Microsoft SNDS/JMRP, and setup DMARC reporting e.g. using Postmark's weekly digest.
I strongly suspect your lack of TLS is hurting you in particular. For this and other reasons besides I advise you to enable TLS for all inbound and outbound. And use a real cert.
I've been on Fastmail going on a year now, and spam was my biggest worry coming from gmail. But honestly I don't think I've received any spam in my inbox at all since doing the switch, and only occasionally had to check the spam folder for a missing email I was expecting. It's been surprisingly solid I think. I'm very glad to be off gmail.
"I'd rather a few false positives get filtered than the deluge that comes with inadequate (self-hosted) E-mail."
The main reason you need a spam-filter is because your address ends with @gmail.com. Man I've lost important mails due to gmail, I'd never use it for anything remotely serious. Works fine for interactions within google-services though.
But apart from that there is no decent client and their notification support is not reliable either, nor can you rely on their spam-filter. Why even bother with email if you are going to use gmail?
> As much as I'd like to reduce my reliance on "cloud" services, Gmail is one of those that I just can't quit, since they've pretty much solved the spam problem for me.
It's overrated. And like mentioned in this article: it makes regular email delivery unreliable.
For me FastMail has largely replaced gmail. I'm much more happy with them than I ever was with gmail.
And they have an actual support channel.
Edit: the "gmail is the only real option" meme is getting pretty tired. They were clearly the best option 10 years ago, but times have changed. No need to repeat old truisms.
Back before I switched to fastmail I'd sometimes stop receiving newsletters. Because Google randomly decided the newsletter I've been receiving for ages and never marked as spam is suddenly spam.
Fastmail on the other hand only classifies spam as spam (with way less than one spam mail per month getting through) which for me is far superior to Google's "I have to check the spam folder regularly because anything can end up there"
I would love if I could set the spam filter to include "audience building"/"growth hacking" emails. I have TENS OF THOUSANDS of emails in my inbox because every startup wants to "boost their engagement". Startups have given me Pavlovian email anxiety.
Literally just started to switch back today from Fastmail
> since they've pretty much solved the spam problem for me
#1 reason. I tried Fastmail for a month or two but I've only just hit the 200 spam messages to train my "personal" spam filter, all of them "greetings of the day" website/SEO related. Realised I've been fighting to get to this personal filter level and then when it enabled.. I received another spam message.
I'm out. Love the idea of not relying on Google for everything but I guess for now spam filtering is just one of those things Google does good.
You just barely trained your spam filter and are surprised that you got another spam? Why are you giving up so early? The spam filter only gets better over time, and you're quitting without giving it a chance.
Also note that you can tweak your spam preferences. For example, I have mine set to mark things at spam at a score of 4.0 instead of the default score of 5.0.
Gmail user here: I like the fact that Gmail's spam filter just works. The false positive rate is incredibly low for me.. things rarely end up in my spam folder that don't belong there. And I didnt have to configure anything. Why should I want to struggle for so long to get the same level of functionality?
Back before I switched to FastMail, I did actually have to keep an eye on my Gmail spam folder, because stuff I wanted would end up there every now and then.
And the very subject of this article is emails that people want to receive getting filed into spam by Gmail (but not by other mail hosts).
It was the #1 reason. Other issues were I had about 480 emails in my inbox. I'm not sure where the difference is but I've always managed an "Inbox Zero" with Googles' Inbox
There are other things I missed from Inbox too such as reminders "hit t - set reminder", snoozing emails till another time and I have a few people that contact me via Hangouts so I ended up loading Inbox occasionally anyway to see those messages.
No fault on Fastmail really, if I'd never discovered Inbox I'd still be using it. In regards to spam I just can't be dealing with training a spam filter, I was reminded it was one of the reasons I switched from self hosted to Gmail in the first place.
Edit: For what it's worth the SEO and website spam emails weren't getting a spam rating at all, most were <2.0
Edit 2: Also while looking at the spambox for that last message I now realise a lot of my server backup status messages were going into the spam box with a 7.1 spam rating.
Edit 3: I'm going to stop editing after this one. Edit 2 was on me, the backup messages were sending direct rather than via mailgun.
FWIW there are other mail clients that add snoozing behavior to arbitrary mail providers. For example, the Spark app will snooze emails on my FastMail account. I don't personally use that functionality, but I know it exists.
The core of the issue seems to be an anti-phishing feature that Google seems to have dialed too high in some cases
But yeah, communication channels are getting more and more clogged with irrelevant communications (not even span alone) so it's getting tough to get across the barrier
Even if you did sign up for it it's probably spam. That's the real issue here. Given how aggressive businesses are with signing up people for unwanted newsletters it seems completely natural for Google to be suspicious of them.
Yep. I waste an hour (probably) every week unsubscribing from various newsletters and marketing campaigns that I don't care about. Most corporations have largely taken "this person has given me an email address" -- even if one is necessary to use their service! -- as free license to hit them with all sorts of marketing messages. Really, these are spam -- unsolicited, unwanted email. It's just that "spam" has taken a stronger, more negative meaning.
I wonder if they've started handing SPF or DMARC misalignment differently. I've noticed a lot of those kinds of problems with newsletter-type mail in the past.
Gmail is not magically going to white-list recipients for receiving some spam that they opted into on some random website it knows nothing about.
People sometimes opt-in in good faith, and then when they get the e-mail, they realize it is garbage. They tell their e-mail client that it is junk, rather than going through the ritual of unsubscribing. Probably, in may cases, they probably don't even make the mental connection between what they received and the site they visited where they checkboxed something; they don't recognize that what they're receiving is something they signed up for. Their reaction is "WTF is this crap?" and report it as spam 500 milliseconds later without giving it another thought.
If enough users flag the posting as spam, Gmail will probably learn that and help the remaining users.
By using an ESP (like Aweber, Mailchimp, etc) means that you are sharing part of your "reputation" with thousands (or even millions) other senders. So even thou most of the ESP are quite efficient in cleaning-up their customer base from spammers, the more clients they have, the more it's difficult.
That said, you may switch ESP by choosing one that provides a you custom "identity" such as dedicated IP, DKIM, PTR, SPF records (which usually means quite a lot of setup and costs) or choose an ESP that doesn't have tons and tons of clients, eventually with a free version.
After reading this I checked my spam folder. Found an email from one of my best friends from highschool, who I've sent 100s of emails back and forth with, letting me know he would be in town in my spam folder... Well guess I didn't want to see him over the holiday weekend. GAAHAHAHHAH.
See as how a lot of my friends use their own domains / hosting for email I guess I'll be spending a lot of time in my spam folder.
Gmail is not a good provider if you care about seeing all of your emails all of the time.
It does a good job getting rid of spam but if you need to check the spam folder for missed emails then it makes sense to leave the spam in the main inbox and save a few clicks
Perhaps businesses are becoming too agressive at requesting email addresses, people are providing fake email addresses, and the recipients are responding by flagging it as spam?
I bring this up because I started receiving suspiciously legitimate spam a few years back. No red flags were raised by the email itself (e.g. no indications of phishing), the companies exist, the recipients exist, everything was geographically consistent. It was as though certain individuals were providing my email address as their own to businesses yet not to friends or colleagues.
When you consider how agressive some of these websites are with newsletters, I would not be surprised if people were providing fake email addresses just to get rid of the prompts. I would not be surprised if Google started flagging those newsletters as spam, even if the odd user genuinely wanted them.
Some years ago I got a reimbursement from google regarding expenses they were paying me. I though they had totally forgotten about it and contacted them.
Even the google representative knew their own financial related emails were systematically being tagged as SPAM on gmail.
158 comments
[ 2.5 ms ] story [ 101 ms ] threadI hadn't seem them in a while and found that they themselves were being sent to the spam folder.
edit - I'd forgotten about this actually, and checked, it happened again today.
It's apparently happening because people send me spam, so I mark it as spam. Because I've marked things sent to that email address as spam, they're saying their own report is now spam. The description of the problem is sounding more python-esque as I continue so I'll stop there.
They can't hurt their customers this way.
Email deliverability is a hard problem these days, spam has caused us so much grief that large email providers are doing really annoying things in their spam filters to combat the problem but I've noticed more and more legitimate email is getting caught in the crossfire. Unfortunately, I don't see this being fixed anytime soon - and it's personally making my life increasingly difficult as someone who had to give up self-hosting his email and switching to a hosted Zimbra service just to stop getting flagged by Google as spam (even though I passed DMARC/DKIM and SPF validation and had a non-blacklisted IP).
There is a trick how you can get out of the spamfilter: Send mail, regularly (like a few mails per week) to friends/family that use Googlemail, and tell them to mark your mail as "not spam". After roughly a month, you’ll have gathered enough points that Googlemail won’t mark you as spam anymore.
And for the people receiving these emails? This is a nuisance at best. Although if 99% of the population stopped receiving emails from their friends, etc, THEN this would be a bigger mainstream issue. Right now? Nobody's worrying (especially in Google right now) except a few newsletter senders.
If you can't figure out what is being sold in the situation to keep the business afloat, what is being sold is you.
I get lots of spam from Mailchimp (ie. people subscribing me to their newsletter, without my consent), and complaining to Mailchimp about it seems to have no effect.
So since Mailchimp allows bad actors to send mail with their service, it’s not entirely unfair that Mailchimp mail gets flagged as spam.
"Did you read the post"...that always shuts down discussion.
Sorry, I wasn’t aware that was a mail service, my bad. His friend (from the iMessage convo) was using Mailchimp, though.
> Mailchimp is quite strict with their opt-in policy and they don't hesitate to suspend accounts that ignore it.
They might say they’re strict, but almost any time I give my e-mail address some where online, I’m automatically signed up for some Mailchimp newsletter, and Mailchimp can’t even be bothered to reply if you report it.
And still I get quite a bit of Mailchimp-sourced spam I never opted-in to.
>> I changed my sender email address to see if would help
But if the pattern of content got flagged as spam by Google users, the sender address might not be the only determinant.
>> I was relieved to discover that others with newsletters were having similar troubles very recently.
But if this results from crossing some sort of user-classified spam threshold, one would probably always be able to find some recent "victims".
This is not intended as a criticism -- I'm sensitive to Google's seemingly arbitrary power -- just an idea to consider or rule out.
As a spammer, you've got to assume Google's going to catch onto your first three million fake accounts and mark both those accounts, the email servers' IPs you use to send, and the email list itself as spam. Consider your first three million gmail account burned by that point.
The less obvious second phase then, using your next three million fake, aged, but-with-fake-email-traffic-to-make-them-seem-real Gmail accounts, is to sign up for not-spam newsletters, but mark those as spam. The wishful-thinking hope is that with enough of those marked as spam, the wider public will see more articles like the linked article, and ever so barely push back on Google, who then have to figure out another way to distinguish signal from noise in a vastly complicated and ML-enchanted algorithm.
Checkboxing "[ ] I want your newsletter" on some website doesn't inform Gmail to whitelist that site.
Personally, I can't remember the last time gmail marked anything important spam. Maybe a newsletter whose absence I hadn't even noticed, but nothing actually important to me.
I just wish I could get gmail spam filtering on my cell phone. Robocalls are out of control -- 4-5 / day. (I've looked at some of the apps / subscriptions for this, but I'm not ready to give a relatively unknown 3rd party access to my contacts or to adding contacts, which they generally require.)
From Google's end, I can understand this decision. But from the consumer end, I only want that logic used for de novo email sources.
Even if many people think a newsletter is spam, I'd want my personal patterns to override that. If I've opened letters from the same sender repeatedly and never marked one as spam, it shouldn't much matter what other users have to say. If I've ever explicitly taken a sender out of spam and not put them in spam, that should basically settle the matter.
I regularly lose emails I've requested to Gmail spam, and even when I've un-spammed the same sender multiple times. I appreciate that Google achieves pre-emptive blocking via a large userbase, but I don't appreciate when the wisdom of the crowd overrules my own behavior.
It seems insane to me that people would willingly let a program decide to throw away inbound mail based on unspecified rules. And I nearly start frothing at the mouth when someone suggests that, since their software decided to throw away my e-mail, it is somehow my responsibility to fix that. It isn't.
If there were thousands of different anti-spam solutions, and one of them behaved badly, then senders wouldn't care, and users would vote with their feet (or wallet, or app store) until the behaviour was fixed. But since Google has a near-monopoly on e-mail, it's now up to the senders to do the work and jump through the badly-documented ever-changing hoops.
Have Yahoo mail subscribers on a Mailman mailing list? Not if you're not running the latest Mailman which has a new workaround just for them. Yahoo started bouncing legitimate mail a couple years ago due to the normal way mailing lists have ALWAYS handled sender addresses, since before Yahoo existed.
I get it, sometimes desparate times call for desparate measures (for fighting spam). But jeeze, it's super annoying to have to break your package manager to get an upstream mail server to support one user who uses one certain mail hosting service.
See, you say that, and until a few days ago I'd have felt the same. But that was before I registered the first .com domain I've ever owned, and oh boy has that brought them out of the woodwork - I've gotten more spam in the last week than in any of the past ten years prior! Not just email, but phone calls, too; I even heard from one of those tech support scams, and it's been so long since the last time that happened that I couldn't even tell you when it was.
It seems to be settling down in the last day or so, which is nice. But if it picks up again, I'll absolutely be installing SpamAssassin on my MTA host, for the first time in the going on two decades I've been running my own mail.
To be clear, I totally agree with your point about Gmail - it's far too centralized and unaccountable for my own comfort, too, which is part of why I don't use it for any mail I care about. But a lot of my correspondents do, which means I'm nonetheless in the position of either taking assiduous care to maximize my deliverability, or risking my mail landing in spam folders that nobody ever really looks through. Not a good situation, but at least until someone comes up with a plausible means of addressing it from the outside, here we are.
Is there anywhere I can look for more information on how this model works in detail, and how to avoid the same problem in future? It's a real pain in the neck. Thanks!
Hello,
My Name is Prk Aa
We are providing whois data in very low price
Daily more than 100,000 domains are registered
Reply Me with your contact number & skype id.
Warm Regards Prk Aa
I wouldn't believe it myself, until it happened to me. I've run small mail servers for various domains for years with only "average" amounts of spam. Then one time I picked up a personal domain of my handle (not this one) which I'd wanted for years and I discovered it finally was free. The moment I setup an mx for that domain I got a firehose of spam coming in to it. In fact it was so bad I had to unreg the mx and leave it for another year before it was worth doing anything email-wise with the domain. Just the bounce processing was driving cpu and bandwidth more than I cared to deal with, even if it wasn't receiving any local mail.
Worked out in the end - it only took them a few minutes actually looking at the content to realize, on the one hand, that it was clearly not being used for any commercial purpose, and on the other, that there was no meaningful risk of confusion between my domain name and their mark. They were happy to wait to buy it until I was done with it. But it was a surprising call to receive at seven o'clock on a Saturday evening!
The way I set up rules on my mail server, they all just go into a spam folder that I browse every once in a while. If anyone is going to delete my shit, it's me.
> The aim of this blog is to help anyone with a website find proven content ideas that they can use to drive more traffic and links to their own sites. The concept is already working, with one recent article helping a reader get on the front page of Product Hunt.
Lol, that's quite the jump you made there.
If you read the article, you'll find there are many other industries which (suddenly) have the same issue.
With a title of "Launching my new daily blog (and podcast)", and sending in bulk, I'd expect a good spam filter to send it to spam. "I’m Starting a Brand New Live Case Study. Over one hour of free training in today's video..." was also flagged as spam. On keywords alone, those titles just scream "spammer".
Sorry if this guy's "audience building" business is affected, but as an end user not worried about "my open rate", I appreciate Google's aggressive filtering and urge them not to let up.
Google's solution is necessary because bad actors in email abound.
That's what I've been using for my two primary domains (vanity personal and business one) since 2004 and 2006. Those filters work pretty well. I can't compare with Google's one but they're enough. I prefer to get some extra spam than losing a business email and maybe a new customer.
Maybe there is a way to keep using Gmail, read from an external POP3/IMAP mailbox and disable Google's antispam on messages fetched from there.
Dovecot + Thunderbird even gives you client editing of server-side filtering based on all sorts of criteria, and self-hosted webmail (e.g. MailBox) is pretty good these days as well for when you're on the go.
The thing is, I run my own personal email server. It's not used for spam ever. It's only personal email to family members, friends, and the odd reach out to support. The volume of email is so small it's ridiculous. I might send up to 10 mail (1 receiver each) per month. I set it up with SPF and DKIM properly, triple checked.
And yet, for whatever reason, sometimes my email end in spam. Generally when I send attachment (like pictures). But sometimes it's just completely arbitrary.
How about this : for you it might be a "few false positive", because so much of the wanted mail traffic originates from google in the first place. But for the few of us that use mail as was intended, this becomes a huge problem.
Having a mailing list doesn't necessarely mean the content is unwanted. I'm subscribed to HN replies for instance. I want those email. If it started going to spam, I might consider "hey it's a few false positive". But for the HN replies guys, it would be maddening to be unable to fix the problem because Google can't be arsed to have a support channel.
From my POV, Google is way past "aggressive filtering" territory and way into "monopolistic behavior".
I definitely felt that gmail was being too aggressive on this too. I sent a bunch of detailed replies to middle/high schoolers working on science projects when they reached out through my public-education webpage and when they went to gmail I never got a reply but if they went to some other server, the conversations usually continued nicely. That was pretty frustrating.
It's probably your IP address.
Sure, SPF and DKIM say you are allowed to send these emails, you are who you say you are, etc, but you also aren't a Sendgrid IP address (or someone similar) so you're treated as less trustworthy.
I could be wrong, but I believe many email servers take these things into consideration (please let me know if I'm mistaken!)
I run my email through google for my domain, and the IP assigned to my mail server is actively used to send spam (like, daily active reports on SORBS), so I can't even fill out their form to get it unblocked.
As a result, I need to email my bank from a different email address. So stupid.
Well, here's your problem. Have you thought about self hosting?
Many spam filtering systems work on a 'points' system, so being on the dynamic IP list adds a lot of points, and if you also have other things that add points (being a mailing list, having 'unsubscribe' links, attaching images, having links) it might be enough to put it over the edge and get marked as spam.
You can do a lookup [3] to see if you're on any of these lists.
[1] http://www.sorbs.net/faq/dul.shtml
[2] http://www.rbl-dns.com/dul.shtml
[3] http://multirbl.valli.org/lookup/
e.g. https://pastebin.com/u48DAaLP
That particular example was sent via SMTP, but I had the same problem when sending via the Gmail web interface, and it occurred sending to at least three different Google Apps domains.
After I moved my domain off Google Apps (I switched to Fastmail for a variety of reasons, but that issue was the kicker), I was able to send to those same addresses without issue. In fairness to Google, I was on the Google Apps free tier at the time, so there was nowhere to go for support.
http://arc-spec.org/
I never understood what was causing these bounces. The entire email chain occurred within Google's control, passed SPF and DKIM checks, my domain isn't on any blacklists nor used for spam that I know of, and isn't used for anything but personal emails. And I've had it registered for years. Nor was it the content of the message. I had a friend with another Google Apps domain setup an alias for testing and when I mailed that address it bounced the same way, regardless of the message body. Emails to other addresses in his domain were not marked as spam... it was only Groups aliases that bounced.
Check your reverse DNS as well. It helps if it matches the HELO advertised by your mail server.
I know a few people whose professional work involves a lot of delivering explicitly-requested, non-spam emails. With most email providers, they don't go to spam because they aren't spam. If they get flagged as spam servers (usually right after spinning up an instance, when the confidence is low), they can contact someone and get it sorted out.
With Gmail? They have extensive lists of tips and tricks to "sneak through" emails that their users specifically asked for. They 'prime' servers with extra-safe traffic to avoid getting frozen in the low-confidence early emails. And they still regularly run into problems. They just don't have the size to be trusted by Google, or to get business contacts to solve the problems that do arise.
It's a pretty concerning state of affairs - Gmail will deliver anything sometimes, but it seems to be a very high bar (vastly higher than "not spam") to ensure you don't regularly drop messages.
The idea that their spam filter is particularly good is ridiculous too. I get more spam to my gmail accounts than to my self hosted mail accounts with greylisting and spamassassin.
Don't do that! Emails should be instant, not with stupid greylisting delays.
Nothing in the RCFs says emails are supposed to be instant. Reliable, yes. Instant, no.
First, greylisting is HIGHLY effective. Something like 95% of email connections are spam. Almost all of them fail without ever hitting disk, via greylisting. If they actually retry, then the chance that the sender IP or email content is in an RBL is MUCH higher, so spam that actually retries will be more likely to be rejected at HELO by an RBL or filtered after it is accepted. The incidence of false positives with grey listing is VERY VERY low. Using greylisting means that very few emails need to be accepted and then filtered. Those are the false positives that really hurt. The sender receives no bounce, and the recipient probably doesn't look at his spam folder every day, especially when it is full of hundreds of junk emails. Greylisting makes email more reliable.
In addition, better greylisting systems note senders that reliably retry and automatically whitelist them. My mail server actually delays less than 2% of legitimate emails due to greylisting.
Except... Google claims >90% of spam never even reaches your Spam box, only the ambiguous messages. So why do I get 10x the spam at Gmail as at any other account? Are they not actually good at detection? Are they just a high value target that gets hit more often?
Or (my guess) are they trying to get clever? Google Doc's spellcheck is infamously bad because they're trying to use machine learning approaches without having a simple English dictionary as a fallback. Is Gmail trying to detect solely on email sender/content and totally abandoning approaches like greylisting?
I'm not sure this argument is related to the rest of your post, but FWIW, GMail/Inbox puts emails from mail lists into their own "Forums" category, not in the spam folder.
Running your own email ain't a hobbyist thing anymore, you gotta play like you're in the NBA. I'm doing all the above, my deliverability is solid. And pay attention to the changing landscape - for example, I will be implementing ARC real soon.
I'd disagree. I've been running my own just fine for years. Yeah, you have to put some work into it, but spf/dkim/dns have been suiting me just fine.
And the postmaster tools don't work for me on google because the volume of email I send is too low. So there's that.
On a sidenote : It's about time something like ARC comes out. If I understand it correctly, it should fix many setups that involve mail forwarding. That'd be a godsend for me, I'll probably set it up soon enough.
DMARC is not a nuclear option, no-one starts out with the reject policy, in fact most of my domains are still p=none and I monitor the reports.
You absolutely can and should - irrespective of how much information it gives you in return - verify your domains in Google's Postmaster Tools, monitor your IP addresses and junk mail tagging at Hotmail via Microsoft SNDS/JMRP, and setup DMARC reporting e.g. using Postmark's weekly digest.
I strongly suspect your lack of TLS is hurting you in particular. For this and other reasons besides I advise you to enable TLS for all inbound and outbound. And use a real cert.
The main reason you need a spam-filter is because your address ends with @gmail.com. Man I've lost important mails due to gmail, I'd never use it for anything remotely serious. Works fine for interactions within google-services though.
But apart from that there is no decent client and their notification support is not reliable either, nor can you rely on their spam-filter. Why even bother with email if you are going to use gmail?
It's overrated. And like mentioned in this article: it makes regular email delivery unreliable.
For me FastMail has largely replaced gmail. I'm much more happy with them than I ever was with gmail.
And they have an actual support channel.
Edit: the "gmail is the only real option" meme is getting pretty tired. They were clearly the best option 10 years ago, but times have changed. No need to repeat old truisms.
Fastmail on the other hand only classifies spam as spam (with way less than one spam mail per month getting through) which for me is far superior to Google's "I have to check the spam folder regularly because anything can end up there"
Because is about those ones the article is about.
Literally just started to switch back today from Fastmail
> since they've pretty much solved the spam problem for me
#1 reason. I tried Fastmail for a month or two but I've only just hit the 200 spam messages to train my "personal" spam filter, all of them "greetings of the day" website/SEO related. Realised I've been fighting to get to this personal filter level and then when it enabled.. I received another spam message.
I'm out. Love the idea of not relying on Google for everything but I guess for now spam filtering is just one of those things Google does good.
Also note that you can tweak your spam preferences. For example, I have mine set to mark things at spam at a score of 4.0 instead of the default score of 5.0.
And the very subject of this article is emails that people want to receive getting filed into spam by Gmail (but not by other mail hosts).
There are other things I missed from Inbox too such as reminders "hit t - set reminder", snoozing emails till another time and I have a few people that contact me via Hangouts so I ended up loading Inbox occasionally anyway to see those messages.
No fault on Fastmail really, if I'd never discovered Inbox I'd still be using it. In regards to spam I just can't be dealing with training a spam filter, I was reminded it was one of the reasons I switched from self hosted to Gmail in the first place.
Edit: For what it's worth the SEO and website spam emails weren't getting a spam rating at all, most were <2.0
Edit 2: Also while looking at the spambox for that last message I now realise a lot of my server backup status messages were going into the spam box with a 7.1 spam rating.
Edit 3: I'm going to stop editing after this one. Edit 2 was on me, the backup messages were sending direct rather than via mailgun.
But yeah, communication channels are getting more and more clogged with irrelevant communications (not even span alone) so it's getting tough to get across the barrier
People sometimes opt-in in good faith, and then when they get the e-mail, they realize it is garbage. They tell their e-mail client that it is junk, rather than going through the ritual of unsubscribing. Probably, in may cases, they probably don't even make the mental connection between what they received and the site they visited where they checkboxed something; they don't recognize that what they're receiving is something they signed up for. Their reaction is "WTF is this crap?" and report it as spam 500 milliseconds later without giving it another thought.
If enough users flag the posting as spam, Gmail will probably learn that and help the remaining users.
By using an ESP (like Aweber, Mailchimp, etc) means that you are sharing part of your "reputation" with thousands (or even millions) other senders. So even thou most of the ESP are quite efficient in cleaning-up their customer base from spammers, the more clients they have, the more it's difficult.
Here you can find an pretty accurate and updated analysis of ESP's clients "quality": http://mainsleaze.spambouncer.org/may-2017-in-spamtraps-esps... Aweber is #6, which is not a good signal.
That said, you may switch ESP by choosing one that provides a you custom "identity" such as dedicated IP, DKIM, PTR, SPF records (which usually means quite a lot of setup and costs) or choose an ESP that doesn't have tons and tons of clients, eventually with a free version.
http://www.emailvendorselection.com/ is an updated ESP directory. You are of course also welcomed to try mine: http://www.mailup.com
DKIM, PTR and SPF really ought to be enough, right?
See as how a lot of my friends use their own domains / hosting for email I guess I'll be spending a lot of time in my spam folder.
It does a good job getting rid of spam but if you need to check the spam folder for missed emails then it makes sense to leave the spam in the main inbox and save a few clicks
I bring this up because I started receiving suspiciously legitimate spam a few years back. No red flags were raised by the email itself (e.g. no indications of phishing), the companies exist, the recipients exist, everything was geographically consistent. It was as though certain individuals were providing my email address as their own to businesses yet not to friends or colleagues.
When you consider how agressive some of these websites are with newsletters, I would not be surprised if people were providing fake email addresses just to get rid of the prompts. I would not be surprised if Google started flagging those newsletters as spam, even if the odd user genuinely wanted them.
Even the google representative knew their own financial related emails were systematically being tagged as SPAM on gmail.
That's how aggressive it was.
I couldn't stop laughing for minutes.