Finally, Betteridge's law of newspaper headlines is letting us down :)
I've had a bit of a email server for my family in 1999-2002 era, as a high school and university student. I've looked into it several times recently and it seems like the barrier to (effective, practical, reliable) entry is so much HIGHER than it used to be, unlike with almost all other technology.
Out of curiosity, are you sending from a node you own and manage, or do you have your email hosted via a third-party (either with your branding skin on it or with the third-party's branding skin, i.e. are you "me@corp.com" or "me-corp@gmail.com")?
So my employer, for instance, "has our own email" but it's just Gmail and we never have problems sending or receiving because we're piggybacking on Gmail's "This is a corporate account with several years of good behavior under its belt" trust signal.
More directly: spammers are killing independent email. Email's peer-node-trust story is so "version 1.0 Internet" that webmasters are left basically using heuristsics, shared models, and tea-leaves to determine whether arbitrary incoming messages should be trustworthy or not, and "they should not" is a good first-pass guess!
So Google (as the thousand-pound gorilla) is serving as a lightning-rod for a larger network-effect problem, which is "Users generally consider themselves better served if most unsolicited email they receive with no strong trust priors drops into a black hole." But that makes it very hard to be a newcomer who wants to establish trust priors.
Spam is killing tons of services beyond just Email.
YouTube, Twitter, Facebook, Craigslist, ...
Nearly anything public is bombarded with spam. The big players are far better suited to deal with it than any newcomers, and even they can barely manage it on their own platforms.
They also have no real incentive to make it easier to combat, because that's part of their competitive advantage over self-hosting. As long as it works well enough that people realize the benefit they are providing (note that this does not mean that it's best for it to work perfectly, as that would be invisible), doing more risks worsening their position.
One that probably leads very quickly to WW3, as the protagonists will find out, that most of the scammers are in countries not on friendly terms with the west (and therefore they tolerate cybercrime against the west).
There are also plenty of scammer operating from the west, though, so that anti spam foundation would not run out of work even if they just limit their activities to the west.
There are middle grounds. I have a myriad of servers that accept email for any domain as fast as the spam bots can send it, meaning the bots will detect them as open relays. Some bots use tracking codes back to themselves to confirm the relay is indeed open but many do not. The SMTP prompt even says not to use it. Some spammers eventually catch on and start trying poorly to attack my nodes which leads me to believe most of the spammers are not very technical.
And it's not even limited to online or free. My snail mail is 90% spam even though people have to pay for it to be printed and delivered physically to my door. I wish I had spam filtering for USPS.
You would think so, but the Republicans have been trying to kill USPS for decades, and have managed to pass laws which both mandate that it pay for itself without taxpayer dollars, and hamstring its ability to be profitable. Some of these hamstringing measures were temporarily removed in 2022 as an emergency measure, but the USPS remains without a guarantee of stable funding in the future.
Really shows they don't really care about infrastructure. The markets could collapse and FedEx, UPS, and DHL could go under, but because the USPS is still there, you can still send mail from one coast to another.
The only thing more inefficient than a purely governmental organization is a private company providing services to the government. There seems to be a persistent belief amonst citizens that governmental services would make them more efficient, more responsive to customers, etc, but following the money suggests different motivations.
The reason it is inefficient is because it provides services that a private company would not, or would charge a ton of money for. The USPS has to deliver mail to everyone with a postal address -- they do a whole lot of 'last mile' deliveries for FedEx and UPS, etc. Getting rid of the USPS means that a lot of people would either not get mail or pay out the nose for it.
Some things just aren't profitable, and frankly shouldn't be. Let's not forget that.
Seal Team Six Bake Sale! GoFundMe for Black Hawk nav system retrofitting! Reservists doing Speedo/Bikini carwashes! I'm sure it would work out great...
> but the Republicans have been trying to kill USPS for decades
A little disingenuous. Republicans attempted to force lower government spending by causing a deficit decades ago. The strategy (for those that were good willed) was that a large deficit would politically force spending cuts across the board, obviously including USPS.
No spending cuts have ever been made and the deficit has grown exponentially instead. Congress discovered debt doesn't matter, and now neither side cares.
Welcome to the uniparty. You can continue to pretend DC Democrats and Republicans are different, but they're playing you with fake controversies that never are resolved by Congress
> > but the Republicans have been trying to kill USPS for decades
> A little disingenuous.
No, it's 100% true. I'm not a Democrat by any means and I'm in fact highly critical of the Democrats. But if you look at the history of laws proposed/passed with regards to the USPS, there's absolutely no ambiguity that the Republicans are trying to kill the USPS. This has nothing to do with spending, either: the USPS has not been funded by taxpayer dollars since 2006, with the exception of the 2022 bailout. After 2006, the USPS operated profitably for a few years, but Republicans have passed regulations which prevent the USPS from operating profitably, leading for the need for the 2022 bailout.
This isn't about debt or spending, and it isn't a matter where both parties are the same. I'm not a Democrat, and I don't even like Democrats, but the USPS' funding issues are unambiguously and intentionally caused by Republicans.
I'm not sure what total junk-mail volume is, but the USPS moved 127.3 billion "units" (pieces of mail) in 2022, down from a peak of 213 billion in 2006.
First-Class mail volume was 48.9 billion in 2022, single-piece (which I presume means non-bulk) 12.9 billion.
Assuming an average weight of 100g per item, that's 6.7 million tonnes of marketing (junk) mail, give or take an order of magnitude.
A tractor-trailer rig can carry a maximum of about 45,000 lb (20 tonnes).
The junk-mail delivered in a year is roughly 400,000 such truckloads of mail. Or assuming 5-day/week delivery, about 2,000 trucks operating daily for a year.
(I'm using round numbers as the initial estimate is rough, just trying to give rough scope to the scale.)
Yes, I'm against federal subsidies in many industries. If it's a public messaging system such as USPS, why does it deliver so much spam? How about we just pay for it and stop that part? Everyone loves that with Fastmail vs Gmail - how about we do it for our national message passing?
> capitalism
How did you arrive at capitalism when I criticized federal funding? You think the idea of a federal govt and paying for stuff with public taxes is a core pillar of capitalism?
Works very well in my experience. I never get unwanted ads. The only time this is mildly annoying is during election time. Parties usually distribute flyers/pamphlets with their ideas (which are otherwise known, but still interesting to compare to each other).
> I see minimal unsolicited mail because it is expensive to deliver.
This is where the US gets it wrong: over here, it's much cheaper for a company to send bulk unsolicited mail than it is for a regular person to mail a letter.
> I do miss the old phone books - nice thin and absorbent paper
Oof, if it's anything like the paper -- and, critically, ink -- used in our old phone books, well... yeah.
Yes and no. If the caller has an existing relationship with the customer they can always call you. So your bank can call you. There is a register "Reservasjon mot telefonsalg og adressert reklame" that you can put you mobile number, etc., on and everyone who might call is obliged to update their own records from it monthly so that they avoid calling anyone on the register.
I very occasionally, perhaps once every year or two, get cold called in Norway but now they are always from offshore. A few years ago I got a couple of cold calls from Norwegian companies but when I pointed out that I had registered all the family's mobile numbers with Brønnøysund [1] they apologized and that was the last I heard from them.
Speaking of which, wasn't there supposed to be a crackdown in the US? Something about the SHAKEN/STIR protocols? I'm still getting loads of spam calls, does anyone know where things got held up?
My understanding is that SHAKEN/STIR wasn't supposed to stop spam calls, but to stop caller ID spoofing. That way you could have "verified callers". Some calls I get on my phone (Android, T-Mobile) do have little checkmarks next to them, and I assume that means they had valid SHAKEN/STIR information.
Possibly at some point we'll be able to just blackhole calls that don't pass verification. But that won't really stop spam calls; it just means that they'll come from verified entities. I guess in that case you can go after them with complaints and get them fined in the cases where their calls violate the law, but that doesn't sound like a great solution... I have better things to do than chase down misbehaving companies.
The question I have is, why does the USPS provide a discount for bulk mail at all? Sending bulk mail is less than half the price if a regular letter. It’s basically a subsidized door-to-door garbage delivery. My recycling bin is right next to my mailbox, and my daily routine is removing the garbage from my mailbox and putting it in the bin. Such a ridiculous waste. And it’s my responsibility to opt out of this nonsense? Make it more expensive to send and maybe that will reduce the crap being sent.
They could just as easily make a rule that bulk mail costs twice as much as normal mail, and still require senders to presort them.
Sure, some senders would then try to send their bulk mail as normal mail. Then there could be another rule that defines what "bulk mail" is, and that anything falling under that definition must be sent bulk, with the consequence being fines and refusing to deliver that mail.
Yes, this is more complicated, and I'm sure some bad actors would still be able to get through, but I expect this would greatly reduce spam.
I imagine part of the issue with something like this is that the USPS is chronically underfunded, and policing and enforcing it would be to much extra work.
One example might be an energy company (they tend to be big) that will send bills to a significant percentage of any residential area in its catchment area, probably four times a year at around the same time for everyone. Now repeat for water companies, phone companies, local government et cetera.
Bulk, for a mail delivery company, refers exclusively to volume. You get discount on volume. Creating massive volumes of customized mails has been trivial for years.
For the USPS, "bulk mail" is a term of art, more properly bulk presorted mail.
There are several categories: 500 pieces of first-class mail, 200 pieces or 50 lb as "USPS Marketing Mail", Parcel Select Ground (50-piece minimum per mailing), presorted and carrier route sourted bound matter (300 pieces), library mail (300 pieces), media mail (300 pieces).
The tradition of discounting certain mailings, especially publications (e.g., newspapers) and books dates to the very beginning of the US postal service and was established by Benjamin Franklin.
Just because there are legitimate reasons for bulk mail to exist doesn’t mean it should be discounted. Most bills are electronic nowadays, and for the smaller number of people that live in rural areas and maybe still rely on snail mailed bills, I’m sure another 30 cents isn’t going to break the bank, when the bank is making 20% interest on the credit card bill.
There's no reason why it has to be defined this way, though. Create a new category for "marketing mail" (or whatever), define what belongs to it, and require senders to continue to presort it, but make it cost 10x as much (or whatever).
This is never going to be perfect, of course, but anything that reduces the amount of crap I get in my mailbox every day isn't just great for my levels of annoyance, but also great for the environment. 95% of what shows up in my mailbox ends up in the recycling bin. Creating, printing, shipping, and recycling that material is a huge waste.
It's certainly possible to do that, but keep in mind that if the government (or an organ of the government) is in the business of deciding what category of speech something is, that is eventually going to cause a problem.
> The question I have is, why does the USPS provide a discount for bulk mail at all?
Because those are their customers, and they treat their customers well. The hundreds of millions of USPS mail recipients, on the other hand, are the product. USPS works for their customers, not for the rest of us.
>‘You mentioned making the service better for our customers; but the American citizens aren’t our customers—about 400 junk mailers are our customers. Your service hurts our ability to serve those customers.”’
-- Patrick Donahoe, US Postmaster General
This is of course an evil fascist talking point though, because the USPS has a large union that consistently votes Democrat, and how could we do wonderful mail-in ballots without the wonderful US Postal Service? Don't be a fascist, mindlessly cheer for USPS.
And this is why privatizing essential government services (ok, the USPS is a weird public-private hybrid, taking on some of the worse attributes of both) is a bad idea. The USPS should exist to serve regular-Joe recipients of mail, not to serve the bulk mailers that pay the USPS to put literal garbage in our mailboxes every day.
But once they have to worry about revenue and income statements and all that, an organization is going to care more about the interests of the people who pay them the most money, rather than the people they're actually there to serve.
The privatization is the bad part? Never a piece of junk mail until...
Oh, wow. Look at this, never actually privatized.
> The USPS should exist to serve regular-Joe recipients of mail,
Can we get a federal telegraph agency too, just in case I need to receive some telegrams? You have no idea how horrible it is having to pay exorbitant sums of cash on private telegraph service.
The trouble is that the USPS has a union which is a reliable voting bloc for the Democrats. That makes it sacred. Even if it performs a service no one has given a shit about since the 1970s, everyone will gush on about how "what if I need to receive some mail" and pretend that mail service belongs somewhere other than a museum exhibit. I predict that every 10 years or so, it will be necessary to invent a new reason why we couldn't ever get rid of it.
Chiming in with something I've posted in the past that I've found reduced about 80% of this spam mail and only takes 10 minutes to set up: https://news.ycombinator.com/item?id=31070730
Bots have also made using nearly any dating app a chore. All apps are ruined by it, what we need is some regulations and jail (or some equivalent) to end this.
The death throes of a doomed industry. Be it steel, horseshoes or advertisements, as profit margins drop production rates will increase to compensate. Then as the machine is running as fast an as efficiently as it ever has, suddenly the margin becomes zero and there isn't any more room to optimize. The entire industry suddenly stops overnight. I await that day.
That is a slightly different issue. This would be like counterfeit steel being sold as of it were the real thing. Or direct-to-consumer horseshoes that will never arrive.
Much of this spam is actively out to trick you, as opposed to legitimate players who have no margin left to give.
Thanks. Yet another thing I hate about my work machine: window edge spellcheck. It autocorrects without asking me. So I don't notice that I have misspelled something. It just gets corrected to some other word.
For automatically dealing with the type of spam that is frequently seen in YouTube comments, I can't help but wonder if perhaps a continuously trained LLM would be well suited.
Youtube spam comments tend to have highly atypical patterns, using things like unicode characters to avoid triggering keyword and URL filters. Something along the lines of GPT should pick up on these things pretty easily, and could similarly pick up on the actions these comments are requesting of users "message me on telegram", etc. It could also probably detect when spammers are trying to impersonate youtubers.
It's not really the kind of thing that spammers can use LLMs themselves to work around, either. Any attempt to get past the anti-spam LLM is going to look quite unusual compared to the typical comment which would tip it off.
Strangely Google seems reticent to try something in this vein though…
I'd be curious to hear what gives you this impression. I've read that shadowbanned users on Reddit get banished to a sort of "heaven" echo chamber where they can see only bots replying to and agreeing with them, but I've not seen anything suggesting that happens for unbanned users.
There are loads of comment-copier/remixer karma farmer bots but reddit users are becoming better at recognizing and flagging those.
Would a service provider like YouTube have an incentive to shut down every single spam post? It's plausible that spam can drive up engagement in some cases.
I actually tested this at one point with GPT-3.5 just by finding spam and non spam comments on a series of Mr Beast videos and, yeah, it was pretty great at it. Even ones that I wasn't 100% sure about it echoed that but would lean one way or another. I asked for outputs like Confidence score|Spam/Not Spam|Explanation and never saw it mark a comment that I'd consider genuine as spam and vice versa.
Obviously this has a selection bias because I had to choose the inputs but there were some that said things like "I ate a ghost pepper on my channel" and stuff that were clearly spam but, to someone not aware that kind of thing is trying to bait you into looking at their channel, it'd appear as possibly just genuine. Heck it may have been typed by a human who owns the channel but is still spam. GPT got it.
I tested this after the video came out a while back from one of the larger channels pleading for Google/YouTube to do something about all the spam comments and the general consensus seemed to be there was "just nothing they could do". Testing this lead me to believe they just don't want to do anything because if it's simple enough that some rando in his house can craft a prompt and get some examples to test in an hour or 2 then a multi-billion dollar company should be able to do SOMETHING.
I can imagine a situation where someone creates a walled garden that people will willingly pay to be part of. Costs money, involves actual proof of identity (but you could still be broadly anonymous within the garden), with the value proposition being an elimination of all advertising. Web ads, spam, all of it.
There is probably some critical mass where this would work. Some people would pay to be able to just not have to do combat with the whole world simply to enjoy the Internet.
I would like to see this done using domain validated identities. It might not work for everything, but, if I can prove I own a domain, it's a globally unique handle that can be used to build online reputation and trust.
It would also make is possible for larger companies to attest to purchases made and / or the quality of participation within their community. Imagine a scenario where I donate $50 to an open source project using GitHub Sponsors with my domain as the identity and GitHub attests to me spending that $50.
Over time, it would be possible to demonstrate a significant "investment" in your domain validated identity and it would be done by spending money online like you normally would without any additional cost. The attestation that you spent the money is simply a side effect of something you're already doing, but it's a really good indicator (over time) that you're a normal participant.
At the very least, I think having a domain as a globally usable handle would help to reduce impersonation which is a serious, difficult to solve problem right now.
Figure out how to keep sites hosted off the platform available without advertising, say by a monetary agreement with either them or their ad platform, and I'll be the first in line.
Well, the monetary system itself ticks most of those boxes, it's just very limited in what you can communicate. And it is not perfect either I suppose.
In fact there might be significant overlap between something that can support a monetary system and something that supports trustworthy communication. The best you could hope for by solely focussing on communication is that it doesn't collapse under the pressure of having to keep financial transactions secure. Except any secure form of communication is probably going to be used for transactions of some kind at some point.
The way Keybase handles identity was a good way forward, in my opinion, that could be extended upon but since they were bought up I’m unsure about its future and no one else is building off of it or copying it, that I know of.
I can imagine using levels of its trust model to handle the levels of email I receive e.g. this domain has X validations and my friends haven’t blocked it.
That almost exists with facebook except your idea is more strict with identity verification for all accounts and instead of anonymity facebook instead lets you use your real name or a pseudonym.
I notice that as operations like Twitter and Facebook attempt to get leaner and meaner the spam problem gets worse. I've never seen so many spambots on Twitter advertising porn and guns, and every third comment I get on Facebook gets replies from a supposed scantily clad woman who wants me to DM her.
I run a business Instagram account and my routine is: Make a post, add hashtags, refresh page, delete the spam comments. Its always similar accounts with similar messages and they don't seem to follow the post after it is initially shared.
Tighter integration between browsers, phones, and email could help with this quite a bit, I think. Default-allow every domain you give an email to, default-allow every specific address in your contact book (and maybe everyone you know on social networks?), default-deny everything else.
A decent first-pass solution to part of this might be to just have email allow every domain in my password manager.
I think the data's there to make this work a lot better, it's just that all the parts aren't talking to one another.
The thing is that humans might actually use Email to talk to humans that they have never met before and that should be a legitimate use case of email, but because spam has made it impossible to keep up with the volumes we are bombarded with that is no longer the case.
You're correct, but this is one of the reasons the megacorps dominate this space; they have the resources to do that integration. In contrast, what would it look like for an independent operator to roll out or maintain such an ecosystem?
It used to have, but I would say it's been getting worse. The number of false positives is definetly going up and I would argue that that already gives us an indication of how important Google sees email. If they would consider email a channel that carries imporrtant information they would optimise to reduce false positives not minimise false negatives.
Are they the same target email address though? If not, surely that's not a fair test if the gmail account has been around for 10+ years or so?
(I have the same situation, an 18-year old gmail account and a 6 year old fastmail account, but the reason I don't get ANY spam at all in the fastmail account is I only use it for certain things and it's much newer, so I'd argue at least in my case, that's not a fair comparison).
Gmail is subject to specific targeting by spammers in a way that fastmail is not. The returns for spending weeks or months finding a niche way through Gmail's filters are justified by the number of gmail addresses that can be targeted, which is probably 3 orders of magnitude larger than the total number of fastmail subscribers.
I was on fastmail for about a year. It didn't filter nearly as much, and Fastmail as a service was constantly experiencing outages. Pretty much weekly. I would say it's an almost unanimously inferior service.
I did use it during that period (and still do). I've never seen an outage. Are they regional outages? Or do I just go to bed at the right time to consistently avoid them?
But the spam filter isn't as good as Google's, 100% agree with that.
Sounds like some super weird, extremely local issue. And yes, I’ve been using it as main mail account for (after checking) almost 9 years now, so that includes that period.
Not an entire year. It was 2-3 day outages, weekly, for quite a while. I've been trying to find the tweets, but as Elon took away tweet searching it's been a challenge to go that far back
edit I remember they were open about experiencing a denial of service attack over several weeks.
The same tech works in both directions. Spammer creates 1000 email variants using a LLM, spam filter collapses those 1000 variants back into easily classifiable embeddings
In the end it's content that matters, not the form that it's send to - I don't care if it's my grandma sending me offers for viagra pills, or a spammer, and I don't care about the language either - I just don't want to get such offers.
On the other hand, if there is an e-mail that may be interesting to me, I don't care if it was sent by a human, or by a machine - I want to see it in my inbox.
In other words - it's not about distinguishing human/machine written text. It's about distinguishing content that is worthwhile for me from the one that isn't.
I use some publicly available spam IP blocklists and my server started rejecting mail from gmail.com because they ended up on one of the lists for sending spam. I thought it was funny.
When this happens, gmail informs the sender that the mail wasn’t delivered and they try a few more times before telling the user that no more attempts will be made.
> When this happens, gmail informs the sender that the mail wasn’t delivered and they try a few more times before telling the user that no more attempts will be made.
That depends entirely on how the receiving server rejects the mail. If it just drops it into the spam folder or /dev/null Gmail won't be able to know. If it responds with a non-transient error code then Gmail will (correctly) NOT try again and inform the user of that.
>Users generally consider themselves better served if most unsolicited email they receive with no strong trust priors drops into a black hole.
If users really want that then they can just dump all email from addresses not in their address book. Some already do that. It turns out that most people actually want to be able to get email from entities they do not yet know.
Extrapolated from "Nearly 85% of all email is spam." At that volume, if your spam filter started with a random coin-toss you're more likely going to serve the user's interest than not.
Rejecting email takes more than zero bytes back over the wire. When the other party in the transaction is assumed hostile, the principle is to provide them as little information as possible.
The fact that they don't acknowledge it doesn't mean they don't drop it.
I've had to set up someone's mail server last year. All mails sent to gmail were silently dropped until we set up all the current buzzwords for the domain/email server. Then they magically started to show up.
Possibly we were lucky that "just" setting up SPF DKIM etc fixed it.
> I've had to set up someone's mail server last year. All mails sent to gmail were silently dropped until we set up all the current buzzwords for the domain/email server. (...) Possibly we were lucky that "just" setting up SPF DKIM etc fixed it.
SPF and DKIM are nowadays the very basic methods used to verify if your emails are spoofed or not. Not having them in place is as good as setting up a spam farm.
> They do: the test emails were never rejected, but never arrived in the test gmail account either. Not in spam, not in all mail, not in the inbox.
And that's fine because that's exactly what SPF/DKIM were designed to do. The spam folder is not a dump of true positives. It's the bucket where you train the filter by evaluating somewhat likely false positives.
You have to understand that telling the spammers each and every detail is simply not viable. So you're gonna end up with situations where lazy/uninformed postmasters get caught in-between, sucks but unavoidable.
You have to understand a rejection is not telling every detail. And diligent postmasters have observed Google and Microsoft drop email from low volume senders with faultless configuration and reputation.
> Sometimes it's too much info to give out. You want to accept phish/malware and silently drop it so that the bad actors don't know you've detected it.
Users' interests are served by being informed when this happens. And plain text correspondence is not easily mistaken for malware.
> A perfect technical configuration doesn't make you inherently trustworthy or provide you with faultless reputation.
> The spam folder is not a dump of true positives.
That's exactly what it should be. If somebody sends me an email, it should arrive somewhere that I can see. If that means needing tiers of spam folders, each one spammier than the last, then so be it.
This used to be a cardinal sin for mailserver operators; mail should always either be delivered, or result in a non-delivery notification ("bounce"). Dropping mail on the floor was a no-no.
Bounces didn't become haram because of gmail but because of backscatter spam. You can still reject the mails at submission time but that changes where you have to run your spam filter.
Gmail's spam filtering has lots of monopoly hardening convenient limitations. Like the fact that sending out to or not-spamming a DKIM authenticated non-gmail sender once isn't sufficient to prevent their future DKIM authenticated messages from going to spam.
Of course, spammers have significantly moved to using gmail and it just streams right through.
> I don't buy it. If a potential customer contacts a business first, and the business sends unsolicited messages to the customer, that reply shouldn't get dropped as spam. And yet it does.
You mean a scenario where the customer contacts the business over e-mail, and the business then starts spamming from the exact same address? Does this actually happen to a meaningful extent?
Google is Directly killing e-mail: my friends can no longer receive emails from our server even if they have me whitelisted, in their contacts and repeatedly marked as not spam.
Nor is any mail server that interacts with other mail servers.
Using an email delivery service to ensure mail gets delivered is probably the easiest fix for singularity2001's issue. Otherwise they're looking at changing their hosting, changing their IP, or fighting the nearly impossible battle with Google to get their mail to be accepted by its system.
Spam email would likely be reduced 99% if it cost the sender a penny to send each email.
To implement, establish an email header field for a token. The tokens are one-time use, and can be purchased from some token selling authority. The recipient can check with the selling authority if it is valid. If it is missing or invalid, it goes to the junk folder.
Domain names and IP addresses aren't free. If we just immediately trash all email which doesn't have valid SPF + DKIM + reverse DNS, we place a small upfront payment requirement on sending email. Domains or IPs can be placed on a blocklist if they still send spam, thus making the domain/IP wasted.
The problem is that waaaay too many legit senders still don't know how to configure their email servers, so you end up with a boatload of false positives.
> The problem is that waaaay too many legit senders still don't know how to configure their email servers, so you end up with a boatload of false positives.
Ironically Gmail has started demanding DKIM or SPF, possibly slowly saving email from this mess of unidentifiable mail.
I don't know .. we do have ads on the Internet and these do cost money to run.
Can't you pay less than a penny a user to run ads in Gmail today ?
The idea is fair though I don't think it's far from where we landed with how the large email broker systems operate with the large email system providers.
Would it? Somehow it makes financial sense to mail spam out physically, which I assume costs about that much. Is the conversion rate that much better for physical spam?
I had read some years ago that putting this "fee" on sending email wouldn't really deter the spammers, when they find one sucker (and they do find them, and there's usually more than one) they can extract so much money out of them that it would still be worthwhile to spend the pennies sending those emails.
Your post advocates a
( ) technical ( ) legislative (X) market-based ( ) vigilante
approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)
( ) Spammers can easily use it to harvest email addresses
(X) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
(X) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
(X) Requires immediate total cooperation from everybody at once
(X) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business
Specifically, your plan fails to account for
( ) Laws expressly prohibiting it
(X) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
(X) Asshats
(X) Jurisdictional problems
(X) Unpopularity of weird new taxes
(X) Public reluctance to accept weird new forms of money
(X) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
(X) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
(X) Extreme profitability of spam
(X) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
( ) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook
and the following philosophical objections may also apply:
(X) Ideas similar to yours are easy to come up with, yet none have ever
been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
(X) Countermeasures should not involve wire fraud or credit card fraud
(X) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
(X) Sending email should be free
(X) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
(X) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough
Furthermore, this is what I think about you:
( ) Sorry dude, but I don't think it would work.
(X) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your
house down!
I run my own email server in part so I could implement a spam filtering strategy that had access to my outgoing mail. Anything received from someone I sent mail to is assumed non-spam, and that is the anchor for the rest of the filtering.
I was planning to build a pretty sophisticated bayesian filter on top of that but it turned out to be unnecessary. The strategy I ended up with, which has served me well for many years now, is:
1. Block a small number of extremely spammy TLDs. There are about a dozen of these, including .biz, .casa, etc.
2. Anything received from someone I've sent mail to, or a sender I've previously marked as good, is assumed good.
3. Anything received from an address whose name contains an English word on a relatively short list of spammy words (discount, offer, etc.) is assumed spam.
4. Anything received from an address that I have received email from before and not marked as good is assumed spam.
That leaves emails from addresses that are sending me email for the first time. These are overwhelmingly spam, but after the four filters above there are few enough of these that I just scan them manually once a day or so.
The real key here is treating the first email from any given address as essentially a "contact request" with the default action being "deny in the future". That just turns out empirically to work incredibly well. More than 90% of my spam is from repeat offenders.
>1. Block a small number of extremely spammy TLDs. There are about a dozen of these, including .biz, .casa, etc.
Honest question: Is there any reason to not just blacklist all emails from TLDs that aren't the Big Four (.com, .net, .org, .gov) and country TLDs (eg: .co.uk, .jp, etc.)?
I can't recall ever needing, let alone wanting, email from the later TLDs like .biz or any of the bloody stupid new TLDs that comprise entire words.
as someone that’s never needed SEO perks of a .com since none of my customers would use a search engine to find me (they’ll click thru from posts and ads on social media and group chats)
I typically register both a new TLD and a .com, and I use the .com for the email campaigns and reply-to
if it gets flagged that doesnt affect my more official one-to-one emails from the actual domain
so I would say, maybe? because I’m not allergic to “bloody stupid” TLD’s and have already adapted to what you’re wondering about
There are lots of businesses using `.io` `.ee` `.co` etc that depending on your email use-case would be detrimental to block especially if you're doing any kinds of business or outreach.
> Honest question: Is there any reason to not just blacklist all emails from TLDs that aren't the Big Four (.com, .net, .org, .gov) and country TLDs (eg: .co.uk, .jp, etc.)?
And I get a fair bit of ham from other weird TLDs. So I think a whitelist is a bad idea. But that's because I prefer to have some spam get through than have ham get blocked by mistake. Even with my very sloppy algorithm, the only time spam ends up in my inbox is if a spammer spoofs one of my contacts, and that is extremely rare.
.co.uk isn't a country TLD, .uk is. If you block .ac.uk you'll block every UK university, .gov.uk the UK government and if you block .me.uk you'll block, well, me.
Anything that reaches addresses that you don't use gets marked as spam.
Common prefixes will work for cases where they're guessing your address, and hidden mailto links will cover situations where they're scraping your webpage.
Correct me if I'm wrong, but I think .tv domains are pretty pricey. They're loved by people who need a bulletproof domain for their sketchy business but I don't see how they would be useful as a disposable spam source that's going to get burned quickly anyways.
Oh, right, I actually forgot that .tv was actually a country code and not a TLD for television.
So... my apologies to the 12,000 residents of Tuvalu. I'm sorry to have to block you all, but for some reason your country code attracts spammers like flies to honey.
Keep in mind we're talking about a single person here. It's possible that everyone's individual list of tricks would be different given where they live and who they communicate with. If you were to use this blunt instrument on a few million subscribers you would undoubtably experience more false positives than OP experiences.
> I have no idea how blocking TLDs of entire countries with hundreds of millions of citizens can be considered a valid anti-spam advice.
It is very different if you offer it as a service to other people, or if you do it for yourself.
If you are doing it for yourself you know if you expect emails from country X or not. And if things change and suddenly you have a pen pal from there you can change your filters easily. So you can live with such simplifying assumptions.
If you are providing email services to masses of other people then you can’t make these assumptions of course. You have to use more sophisticated means then.
> If you are providing email services to masses of other people then you can’t make these assumptions of course. You have to use more sophisticated means then.
These things just get thrown in the large pot of heuristics.
I used to do this at one time. I had no users who corresponded with addresses in Russia, Romania or China, and those represented a very high proportion of inbound spam.
Nowadays (irony of ironies!) most inbound spam is from gmail.
Not bigotry at all. If my server gets a lot of chinese spam, and none of my users reads chinese, then it's just silly to waste bandwidth and storage on messages that aren't going to be read.
But you aren't blocking the Big5 character set, you've determined that 1 billion people can't possibly learn English and have no reason to contact your users. Because reasons.
When Chinese and Russian ISPs start taking abuse complaints seriously then they can be given the benefit of the doubt. But until that happens they're going to remain on most RBLs for a good reason.
So you've determined an entire population has a certain behavior because of past interactions with other entities from that continent-sized part of the world.
Do tell me more about this wonderful new method of spam filtering with absolutely no bigotry.
Judging someone by their current behaviour is not bigotry. As I said, they have the option of taking abuse complaints seriously, but they haven't for a really long time now.
there's no such thing as a "woke" spam filter... i mean, it'd hafta be inclusive or whatever to make sure none of the email feels marginalized or sad... so to be "PC" it's gotta let everything through... which means it isn't filtering spam...
This isn't about being "woke", whatever that means. It's just about not being a bigot and classify broad people in a certain way because of their geographic position and - by extension, ethnic and national origin.
Just imagine the racket if Gmail would start de-prioritizing email from Alaska or Hawaii because, in their experience, that's a large source of spam. If might even be true, so what? That's the exact thing racists say, racism works: we found the guy suspicious because no black people live around these parts, so we knew he was up to no good.
let me make sure I understand... filtering spam from geographical sources known to produce a disproportionate amount of spam, that someone never gets legit email from as they have no interaction with anyone there, means they're a bigoted racist... so instead we should just not filter spam, it's not fair to the spam for us to microaggressively generalize it into some group that gets marginalized, every email deserves equal treatment regardless of where it originated from (since that's all PC and woke, etc.), which ultimately makes sunshine and rainbows go everywhere and world peace and stuff?
you're making comparisons here that are so disjointed it's not even apples and oranges... it's more like apples and rocks...
this thread grew outta the spam filter setup of someone hosting their own email server. bringing gmail into this and talking about how they'd have a PR nightmare if they screw with hawaii doesn't work. yeah, they both send/receive email, that doesn't mean they're comparable in the way you're tryna say... i mean, apples and rocks are both round sometimes, but you wouldn't try comparing how they taste, right?
This thread grew out of someone claiming blocking entire countries is a good anti-spam technique because they just "know" those people have nothing to say to their users. This is bigotry pure and simple, no need to go any further, they hold strong opinions that can't be objectively verified about large parts of the human population based on national origin.
As someone who was on the receiving end of the "Romanian users have nothing to say to my users and postmastars there can't even RFC" blackholes, trying to get legitimate email delivered while respecting all the rules of the internet, I can only say that whomever thinks like that is an idiot and a bigot. Unfortunately, there are many such idiots on the internet at large, and the aggregate effect is that only large sites like Gmail can maintain good deliverability, which was the subject all along.
If I ran my own, personal email server, I could probably block the TLDs of every country in Asia, Africa, and South America and never actually block a single email. I don’t receive emails from those TLDs and I quite possibly never will. If I expected to, I wouldn’t block it. I don’t travel the world, making friends in every country I go to. That sounds nice, but that’s not my life.
The poster you’re replying to is talking about their filtering method for their own email. Presumably they know if they expect to ever receive an email from a given country.
Is there a particular reason to go ahead and preemptively blacklist every country's TLD you doubt you'd ever receive an email from? Would it not be better like if spam isn't expected to be an issue you might as well leave it open, then if you do one day receive a spam from that TLD, block it then?
Otherwise, let's hypothetically say you have a friend named "jacob peter" who really likes his initials (and doesn't understand the topic of this discussion), so he registers somewhere in Japan that lets him have dot-JP as his TLD. The only hypothetical way he knows to contact you is via your email server (let's also assume he doesn't do stuff like social media, phones, etc., so he can't find any other way and you have no way to contact him). If you have hypothetically preemptively blacklisted all of Asia's TLDs, then you'd never talk to him again for as long as you live, right?
If this entirely fictional friend only ever emailed me from his .jp email address and never contacted me any other way, I guess so, I would never talk to him again.
Since he’s not real I don’t know if that’s a good thing or a bad thing. Maybe he’s kind of a dick.
In my experience, it changes all the time. A strong factor is NameCheap doing dollar sales on some TLD; causing a surge in DKIM-signed spam using that TLD for a few months.
Honestly, just block all everything except the traditional TLDs like .com, .net, etc. and country-code TLDs for countries you expect to interact with. It's a trivially easy way to greatly reduce the amount of junkmail you receive, and saves on compute resources too (since you can reject mail before it even hits your AV/anti-spam/etc. stack).
In many years of doing this at work I think we've had to manually whitelist three domains (our corporate travel provider started sending from a .travel domain instead of their .com without telling us they would make that change...). I don't have numbers handy but it significantly reduced our junkmail intake.
Isn't that a bit like trying to predict who will want to talk with you? Maybe you are not in that sort of business but I constantly give out my email to people I meet at conferences and through that get Ham from pretty much any existing TLD, how do you cater for this case?
We mainly deal with governments and other businesses in the same fairly small industry, so it's somewhat easier for us in that respect - we do know most of the entities who walk to talk to us in advance. Also, we have a good relationship with our staff and they're usually proactive about telling us when they're expecting mail coming in / we add a new supplier / etc. so we can keep an eye on mail delivery.
There's always some mail that gets blocked, of course, but it's a relatively rare occurrence.
Maybe in your case something that allows users to view and release their own quarantined mail [1] coupled with a healthy dose of training on spotting malicious emails and so on?
I run my own mail server too. Rspamd does a reasonable job of cutting back the amount of down that gets though, but I feel like I have no control over it or visibility into what it's doing.
What tools are you using to implement the controls you describe above?
I rolled my own. It consists of Postfix, Dovecot, a custom milter written in Python, some small sieve scripts, and the spam filter written in Clozure Common Lisp using SQLite3 as the database.
I think spammers are just excuse that Google hides behind. I had static IP address for a decade, never used it to send spam (I filered SMTP ports from NATed clients), made sure I'm not blacklisted on any RBLs, configured SPF and DKIM.
Yet suddenly my emails go to spam in gmail.
Now, there's no way for me to do anything, because I'm nobody.
As for spam, my bayesian filter works just fine, so if I can successfully filter spammers out, I think Google can as well.
I feel like Google is doing this purposefully to make people like me to simply give up, and observing responses, it is successful.
>so if I can successfully filter spammers out, I think Google can as well.
This is absurd. The problems Google face are orders of magnitude more complex than your individual mail server. Your server benefits from the fact that no one cares about it. No one will develop a custom spam technique to send you spam. They will for Google. There will be whole offices in India dedicated to working out exactly what slips through Gmail.
You benefit from a kind of herd immunity where no one even sends the kind of spam that would reach you because it's blocked by Gmail so not worth sending.
The spammers don't attack mail servers, but individual mailboxes.
Your personal gmail account has the same risk of receiving spam as mine, but Google has much more tools available to detect spam. For example they can easily see somebody is spamming, when large gmail accounts are receiving the same message. They can easily see which hosts are compromised proxies as they receive tons of mail, and seeing large number of e-mails sent from questionable hosts should be easy to spot.
They didn't really improve spam detection much for for a decade, because blocking small hobbyist mail servers is more likely increase their user base.
Those special farms you're mentioning, wouldn't work, if filtering would be individual per user, as it should be, because each person's mailbox is different and each person has different definition what spam is.
> More directly: spammers are killing independent email.
Disagree.
In the early 00s, spam was killing email. I was getting thousands of spams per minute to my personal email domain (same one I still have and have had since the mid 90s). It was real bad back then.
Doing aggressive sanity checking on incoming email back then was problematic since nearly all legitimate senders also has badly misconfigured email servers.
Those days are gone. Legitimate senders have well-configured email now, one can do a lot of sanity checking on the SMTP connection and that alone cuts out most of the would-be spam before it is ever sent and none of the real email is impacted.
Sprinkle a bit of bayesian filtering on what remains and spam is mostly a non-issue anymore.
So what it the problem these days? Part of it is a legacy cultural problem. A lot of email admins wear so many scars from the early 00s that they still operate on a mental model that they are willing to throw away a lot of legitimate email just to prevent even one in a million spam to go through. That's highly counterproductive and causes a lot of harm to interoperability.
The other big factor of course is that the very deep pocket email providers (gmail, microsoft) have a strong vested interest in strenghtening their monopoly even further so they are happy to go along anything that decreases interoperability.
That all said, I strongly encourage everyone who can to run their own email. Interoperability can only be saved by exercising it as much as possible. And it still works fine despite the naysayers. I've been running my email infrastructure for a long time and have no deliverability problems anywhere that I've run into.
I assume badly misconfigured servers meant something like open relay setup. Not DMARC and dependent technologies, which are not protection against spam at all, anyway. They are protection against various kinds of spoofing.
And I concur with the original commenter. I do zero filtering on my SMTP server (no reverse DNS lookups, no blacklists, no nothing), in order to be sure I receive every single email, and SPAM looks like it would not be an issue even if I disabled the bayes classification client side, these days.
I mostly rely on providing unique random addresses to each service. My oldest 18 year old address gets quite a bit of spam so I blocked it and switched to different one, but newer addresses that I use for the last 5 years didn't accumulate much spam, despite being public and crawlable on the web.
Well then they for sure can't use gmail. If you still do use gmail, please consider moving to something else. And until you actually do, make sure to keep a close eye on the spam folder because you've most probably lost important mails.
I couldn't disagree more. I have several email accounts and the gmail one by FAR lets through the most spam, a lot more than my Fastmail emails which despite being more published have far less spam than Google.
But do you know for a fact that it lets through a larger proportion of the spam sent to it, or are you just noticing that it gets a larger total volume of spam than the others? Because if it's the latter, that could be an effect of more spam being sent to your gmail than to your other accounts.
IMO email is dead and spam killed it. Who uses email anymore? It’s basically business communication (but all the biz communication I’ve done in the past years has been through telegram / signal / whatsapp / etc.) and registering to websites.
Once we figure out a way to handle website registration that’s not tied to email (whatever that means) I’m not sure email will be useful for anything.
Most useful conversations have moved to messaging apps.
Nome of the service you mention have the async nature of email.
If I want you to be aware of a long article I'll send it to you in an email because if I message it, most people will open it, see it takes more than 2 minutes to read and close it and never go back to it.
Really? WhatsApp is end-to-end encrypted unlike something like email or slack. If your company doesn't understand that they should hire a security team.
> Most useful conversations have moved to messaging apps.
That's not a future I'd like to ever live in.
Why? Because all of those are proprietary solutions that don't interoperate with anything and their behavior and existence depend on the whims of a single corporation. No thanks.
Email is standard, interoperable, owned by nobody, accessible to everyone. It's the optimal way to communicate.
While one could heavily overgeneralize and say that "everyone uses google for mail" (iirc they own like 20% of the email market) they certainly don't own the protocol or its development.
I think it is exactly these sentiments that do most harm.
I mean, that is completely false at every level (technical and operational).
Sure, gmail is a big player but there are many others. Most importantly, there are thousands of smaller players. All of which interoperate.
I can't tell if you say that sarcastically or from a position on being uninformed about how email works. If the latter, do take some time to understand email infrastructure at a high level because it is a case study in beautiful interoperability of open standards. Something we must all keep in mind for the future of the Internet.
Email has been resilient and an open communication mechanism from everyone since the 70s precisely because it cannot be owned by anyone, it is an open interoperable standard.
There will never be anything equally long-lived and feature rich from a single corporation. Corporate interest don't align with that. Proprietary solutions will always be about locking you in and limiting interoperability.
So many people completely miss this point. If whatever is supposed to replace email it will have to be very similar in features. That is
1. Permissionless (open for anyone to send messages or operate a server)
2. Asynchronous
3. Resilient
4. Searchable
No IM solutions I know can do that at all. Matrix would come closest but asynchronicity and resilience are still troublesome on that part and all clients are more geared towards IM instead of async longform discourse
Email is incredibly heavily relied upon and with great success. People get their pay receipts, their rental applications, notifications of mentions on social media, speeding tickets, music and gig notifications, reminders to get their pet to the vet, X-ray reports, prescription reminders and I could go on.
Not to mention the huge boom in email digests and blogs over the past 5+ years.
I haven't seen - _any_ businesses communication conducted over signal or WhatsApp - ever. I'm not saying it doesn't happen somewhere but it's certainly not common place. And telegram? I didn't even realise that still existed!
> That all said, I strongly encourage everyone who can to run their own email. Interoperability can only be saved by exercising it as much as possible.
While I have no doubt that doing this is a lot easier and more hands-off than the last time I did this (for a few years during the mid '00s), I just don't have the time or energy for it. Also, email is a critical service, something where I don't want to have to deal with downtime, especially if I'm away traveling or something like that.
I think as a decent alternative for people like myself, it's at least helpful to use an alternative hosted email provider (other than GMail or Microsoft) to help keep the ecosystem open.
> Also, email is a critical service, something where I don't want to have to deal with downtime, especially if I'm away traveling or something like that.
I have to say that as the most critical service of all, that's one of the reasons I do host email myself!
I can't and won't take the risk of being locked out by the whims of some third party misconfiguration (we've all surely seen the endless stream of misfortunes by those getting locked out of gmail/etc).
People put up with millions of ads everywhere but a handful of spam mails a day and everyone looses their minds. Meh, I'd rather have the spam then be beholden to centralized gatekeepers.
Spam is killing independent email.
gmail is fairly good at blocking it, is easy enough to get into, so it's been winning for years.
I used to run independent email. It took constant work to get close to gmail's level of spam blocking. So I switched. Found most alternatives weren't anywhere near good enough, and I don't have enough hours in the day even for my own email.
This has been getting consistently worse over the last few years, in my experience. At least on the user end, i.e. far more false negatives getting through.
True in my experience. Doubly so when dealing with foreign languages. I run a pretty international business with a lot of different email distributions. It's actually the spam filter in Google Groups that seems to constantly get tripped up with non-English, and in particular non-Latin, characters in emails. It puts far more of them into spam than it should.
I noticed a correlation between the volume of spam I'm receiving to my spam folder, and the increase in false negative arriving in my inbox. Spam volume is increasing in my experience.
I think it's been true for a long time that most of the spam doesn't hit your spam folder either, so this is not the signal you suggest. It could mean more false positives at a lower layer though.
I also monitor email at a network infrastructural level via CloudFlare for the domains, and the volume has definitely increased overall, quite significantly, including on domains belonging to clients. But yes, the amount that actually ends up in the spam folder itself is far lower than what actually gets sent to us.
There's definitely an uptick in forgeries, which is also probably why Google has started strictly enforcing the existence of SPF or DKIM even without DMARC.
plus it doesn't seem to tune. no matter how many times i mark steam or fedex emails as not-spam, i just have to check there every now and again these days.
> Spam is killing independent email. gmail is fairly good at blocking it, is easy enough to get into, so it's been winning for years.
> I used to run independent email. It took constant work to get close to gmail's level of spam blocking.
This is very contrary to my experience on both fronts.
gmail spam filtering is mediocre. Spam gets through and good emails are flagged as spam. It's not very good.
For my own domain where I self-host my email, my spam filtering is quite a bit better. Approximately nothing is ever mis-flagged as spam (has not happened in years) and the spam that gets through is a tiny fraction of what my gmail address gets. So for me my own spam filtering is much better than gmail.
And no, it doesn't take constant work. In fact doesn't take any work at all. I set it up years ago, the bayesian filtering goes through all the mail but it takes zero effort from me.
I wish this myth that gmail has some incredible filtering technology that nobody can replicate would just die already. Gmail isn't terrible at spam/ham filtering, but it's not that good either, just mediocre. You can do much better with minimal resources on your own.
Spam (and the resultant filtering) is killing independent email, and it's an ongoing problem on Big Company Hosted Email too.
I barely use email anymore. Everything at work is mediated by Slack or Atlassian. With friends and family it's almost all text messaging. My kids' schools and sports teams use a bunch of different proprietary web and mobile apps to communicate with parents.
If you were out on a walk and 9 out of 10 people where trying to mug you, you'd very quickly adjust your behavior to only walk in very safe places and let as few people as possible access that area.
There is a significant cost in spam protection by tracking reputation and content for the unending ocean of bullshit flooding the SMTP lines. Most providers want to cut communication with the spam source as quickly as possible to reduce costs.
Is spam also killing every social media platform, messaging app and even Google itself? Is spam not an indicator that people still use it if it's still lucrative spamming people, thus proving it isn't dying and is actually a sign it is still used?
Consider that email accounts are the go-to account recovery method for most services, and it's ubiquitous in biz. Also consider that you can prioritize specific domains or filter x domains to never go to spam, e.g., your own company's domain.
Any "death" is that people struggle with their own mailserver as a general rule of thumb. Does that thus mean email is dying? No. As the article says, perhaps independent email is, but it hasn't been in a good place for over a decade at this point.
Oh my god, yes. To all appearances they declared defeat in the Great Webspam War some time around '08 or '09 and their results have been markedly worse ever since.
In terms of end-user quality of result, yes, I agree, but they're still wildly profitable, ergo they are not dying. They're a business. Their pulse is measured in dollars, not quality or user sentiment.
I've been on Telegram since 2016 and I probably receive one spam message per 4 months. And I'm super active in a lot of groups. Possibly could be related to the types of groups you join though. I've noticed that 100% of the spam I receive on Telegram is about blockchain scams so perhaps being in tech or finance related groups (which I am not) could increase the risk.
Like with Snapchat, almost all of mine are camshow and crypto spam. It must definitely be some kind of demographic thing with what you're active in; lonely nerds for the camshow spam? Lmao. It also wouldn't surprise me if there's lists for sale of people of xyz demographics.
Would Spam stop if people stopped responding to it? There has to be a non zero amount of stupid people that react to junk mail and make a purchase or fall for some scam. This number is only increasing with more people coming online.
There is a wide spectrum of unsolicited mail, not all of it is stupid people responding to scams. I suspect the quality and response curves are inversely related.
>Would Spam stop if people stopped responding to it?
I don't ever intend to respond to spam, and have become extremely adept at spotting the patterns and swatting it away. However, it becomes a game of chance, when a service like Outlook puts it right at the top of the app (both iOS and Android) where you would reflexively jab at it, unless of course, you pay the premium to remove it.
For now, I have found a way to stop this nuisance. However, MS are playing fast and loose with their policies and now very legitimate looking spam is leaking into the inbox, escaping any filters. Since last year it is appearing along with the glaringly obvious Unicode riddled ones, with increasing regularity. It seems like a matter of time and co-incidence, where you would end up interacting with a piece of disguised mail you were expecting e.g. an order from Amazon or a service which you use regularly, and possibly respond without checking the header.
This recent episode was probably the worst experience, albeit not the first time it has happened.
I kind of wished the 1 penny an email idea would have taken hold. As much email as I send $5 would last me years. It would have probably deterred the people sending out millions of spam mails.
From my experience (admittedly with independent email services that were around before Gmail was even a gleam in Larry Page's eye), Gmail is only a modest fraction of the problem. Other big players - especially Microsoft - are generally worse.
Flip-side, there seem to be more spammy messages sent from @gmail.com addresses than from any of the other email A-listers.
Agreed. I self-host for my and my friends' personal and project domains, and delivery to gmail works. Granted, nothing is commercial and the volume is so low that rate limiting is not an issue, but if you set things up properly, they'll take your mail, and if you don't, they're pretty good about telling you what's wrong.
On the other hand, it's simply impossible to satisfy Microsoft. We're irrevocably tainted by being in a netblock of a well-known provider, despite having held the same IPv4 address clean for over a decade.
As a user, O365's default spam filtering was just terrible about 2 years ago. I got so many false positives that I had to check my spam folder multiple times per day. I ended up adding very aggressive domain whitelists because I was so tired of it.
For what it's worth, I managed to get whitelisted by Microsoft a few months ago after... 15 years of undeliverability or so.
I followed the process, and then kept insisting a bit by answering the emails saying they were not going to do anything and I had to check if I was complying with their rules etc. After two emails I had a real person answer me, and a few more emails later (basically insisting I was already enrolled in their various bullshit spam reduction programs and there was zero spam problem with my domain) I got told that I had been whitelisted.
FWIW, a client of ours got their office mail server off MS's blacklist a few years back. In less than a week. But that seemed to require their ISP (a mid-sized firm in the Midwest, with awesome customer service) going to bat for them with MS.
I'm guessing some of the issues are just from Google being random about what it considers spam no matter who it comes from. I remember a comment on a somewhat recent thread from someone who had to move their business mail away from Gmail because Gmail would classify mail from one paid account at their organization to another under the same organization as spam.
I was just going to say that for the past year I (my server for the whole family) sent maybe 10 messages to google and I personally received at least 15 spam messages from gmail with SEO offers and other kind of spam.
"It goes without saying that our messages are not spam" haha no. I don't know what it is with these people who think their god gave them the inalienable right to send messages without rate limits just because they are signed with DKIM. The most likely explanation for why that site "School Interviews" got rate-limited is people marked their junk as spam and their sending IPs got bumped down into the bozo quota. And the most likely reason for people to have marked them as spam is they failed to do verified double-opt-in and just started spamming away at whatever address their customers mistakenly typed.
Microsoft ban itself (azure) to spam, they use ml and while they cand add exceptions in general the systems are so complex and have so much hardening that is not posible our "messages are not spam "
I mentioned it before here on an old mail related thread: my Google powered inbox managed to flag as spam an email from Google Domains about an upcoming renewal.
Some things are just baffling. How can they manage to flag themselves as spam it’s beyond me.
There's also the possibility that their emails are full of links, which is a decent indicator of a low quality spammy email.
There are exceptions, e.g., I sent an email full of research with sources to a family member I've been emailing with for fifteen years and it went to spam, despite it being @gmail.com to @gmail.com. In retrospect, I was misusing email versus sending a document or a link to one with it in.
Is it? If I'd sent a link to a document, they'd have the latest version and I could continue appending to it, and they'd have a version history and could even contribute to it if I permitted them.
The times have changed. It's not unreasonable to expect how people use email to have changed, ergo people sending emails full of URLs are statistically more often than not spammers.
Is it a broad stroke? Yup. But I'm willing to bet that I'm a fringe case and it prevents a ton of spam.
I deal with email at global scale, and yes, you're a fringe case. There are many billions of messages sent every day which have lots of links, and which recipients in general are interested in, and want delivered to the inbox (or promotions), rather than spam.
I agree. I also believe that message contents are much less important for abuse classification than nerds generally believe. Spam is about behavior more than it is about messages.
Speaking from quite a bit of experience, content can matter at Gmail if it's very obviously malicious/spammy, but not usually otherwise. Metrics matter more than content at gmail.
For other filters/ISPs, content can matter a bit more, but as a general rule for consumer mailbox providers, metrics are the primary thing that affects filter outcomes.
The real question is why obvious spam enablers like GetResponse and other email services don't get the same /dev/null treatment by Google. None of the email gatekeepers you now need to use to have your mails arrive do any verification (CSV import hello), yet they are obviously not met with the same bans.
Because those companies maintain enough "good" customers that it impacts real businesses if they get entirely shitcanned, and those businesses complain to Google from both sides.
You're correct that a lot of senders have no idea when they're sending unwanted email, and that unwanted email is well within the realm of possibility here. But don't assume DOI is a panacea; you can use DOI and still send unwanted email. It can improve quality metrics (fewer bounces), but engagement metrics are a much stronger signal, especially for gmail.
Can you think of a scenario where a well-intentioned organization doesn't realize they're sending some unwanted mail, and by looking at the right metrics they realize they have a problem and take steps to fix it?
Same. I don’t wanna subscribe to any newsletter. If some website tricked me into accepting it, or just added me to the list anyway, it’s going to spam. Hope they enjoy the Gmail reputation penalty.
This almost makes me want to go through the experiment of setting up a legit email server with all the various authentication mechanisms and see if I could even get one email through to google or protonmail
I just set up a brand new server with a never-before-used IP address. I set up SPF, DMARC, and DKIM, created a brand new account with a fake name, and sent an email to my Gmail account. Within 3 seconds of being in transit, the email landed straight in my inbox with 0% doubt of it being spam.
[Edit]: Forgot to mention that the IP is home IP, not cloud provider/hosting or something.
And as as sidee note:
the problem lies with the users and not with the email sender. For instance, if I send an important email to someone about an item they ordered from my site, and they tell me to send it to their gmail account, I connect to gmail to send the email, but if Gmail rejects it, I have fulfilled my part of the deal. If an email is marked as spam or rejected, it's on the recipient's end, not the sender's. The sender did not flag the message as spam before it is sent, the recipient did it after receiving it.
You absolutely can. Yahoo and O365/Outlook would be actually difficult, but you can build up reputation with those as well. Making it through small providers' filters will be your endless battle though.
There's a problem that's just as bad, it's the reverse POV of this article.
Basically, you can't block the big providers - Gmail, Microsoft, AWS SES, Mailchimp, Mailgun and friends - because everyone and their dog is using them. But their reaction to abuse reports is spotty at best... you're stuck between a rock and a hard place.
The root cause obviously is spammers and scammers, but governments don't care about putting a final stop to bad actors.
>> you can't block the big providers - Gmail, Microsoft, AWS SES, Mailchimp, Mailgun
Why? Respectable businesses send from their own domains. Friends and family never send emails nowadays, there are messengers for that. Anything from google goes straight to Junk folder.
I think there’s a related problem here: spammers (often doing cold outreach) are very happy to use gmail to send their wares. Gmail provides no
mechanism to independent mail servers to report those people.
Gmail might have a monopoly over consumer email but Microsoft has one over enterprise email.
B2C email is quite competitive with dozens of services.
Overall I’d say the email ecosystem is relatively healthy. It’s more competitive and interoperable that instant messaging with greater security than SMS.
regex is too specific, LLMs seem like they could help us make more general types if we can avoid hallucinations, maybe we could teach one to generate spam in order to teach another how to recognize it
Maybe not even going to fully ML/AI but definitely some NLP to understand the intent behind the spam. It's sort of like the approach taken with https://spampatrol.io for forms. Probably can use it for email too.
The other thing that bugs me about these articles is the brazen two-faced argument where email is at once an open, distributed protocol between independent peer operators, and also there's exactly one way to do it, the way German privacy zealots insists on doing everything, and there's not another way!
If Google wants to receive your traffic at a later date thats their business and not yours. It's an open system where sites set their own policies. Access to a site's eyeballs is not the right of an outside sender!
So many comments here saying that spam is killing e-mail, but even with very light spam filtering on a non-gmail account maybe 1 spam message a day gets through? I get more physical junk-mail than I do spam.
Running SA, I get maybe half that per day and false-positives are extremely rare. Get about 1 false negative per day as well (which is fine; I could live with 5-6 really).
It probably depends on how long you've had your email address. I've had mine for 20+ years would be borderline unusable without spam filtering.
It's been much better since I took the time to set things up so marking and email spam automatically fed it into sa-learn. I still have to have a handful of rules to filter out senders who are "legit" enough to make it through, but ignore unsubscribe requests.
I'm about to venture into setting up a new email server. Mainly in that I'd been wanting to play with WildDuck.. and second in that I'd like to stop paying to relay though SendGrid, which I've been doing the past several years.
I setup a dedicated server not on a major cloud host, and am not looking forward to all the details involved in the lack of trust starting out. Let alone the dark art of spam detection. But I want to get back into it if only because I don't like how the major parties are cornering things up. I also want to be able to actually handle mail for several domains and not have it nickel and dime me to death. It costs way more for a single email account these days than it does to run a few dozen minor websites.
While it's nice that Google Domains (when you use their DNS) and Cloudflare both have included email forwarding, sometimes you want an actual box to send from too. And with the partitioning that GMail now does, I can't find anything anymore without hunting for it... the only benefit is two of the subtabs, I'm able to just delete all once in a while.
I wish that email were much more reliable and able to actually setup 2-way relationships similar to IM clients. And of course, limit/remove third party info sales/spam in those relationships.
If you're only a small time sender lack of trust is permanent. I've been self hosting for three years now, never sent spam, never had the server breached and GMail and Microsoft both still send mail to spam.
At least Microsoft no longer outright 550 refuse my mail so I have that going for me which is nice.
It's not permanent. Polish up your sending practices, slowly build up sending and you can deliver to all big providers from DO or OVH for all you care. It's the small providers that'll give you the most trouble.
Last time I tried, Google and even O365 delivered fine... Hotmail and Outlook.com, ironically considering they're also MS managed were the biggest problems from my testing. At this point, I'm willing to let some of it go to spam, as I think most people wind up having to check there regularly anyway. There's a lot of false positives, but so much actual spam and uce I don't really blame the carriers.
DKIM, SPF, etc only go so far anymore, and even graylisting doesn't seem nearly as effective as it once was. Again, not really looking forward to parts of this, but I do want to at least try self-hosting as much as I can. I like the cloud offerings from MS/Google, but don't like the companies or their actions in and of themselves. Only one real way to push back, and that's to actually try.
And on the small providers, yeah I can get that. I'm a bit more than a decade removed from self-hosting much of anything, and even then balancing DNS RBL with other factors was at least interesting if not frustrating.
Many comments here say spam is the culprit. But spam has been a solved problem for two decades.* Ironically, Gmail doesn't even implement the solution: private, individually-trainable stastitical filters.
* In fact, I'm still using the filter I installed on my machine in 2003.
The true problem, I think, is: do I have the _right_ to send a legitimate (nonspam) email? And does an email provider have a _duty_ to deliver legitimate emails?
508 comments
[ 2.6 ms ] story [ 387 ms ] threadFinally, Betteridge's law of newspaper headlines is letting us down :)
I've had a bit of a email server for my family in 1999-2002 era, as a high school and university student. I've looked into it several times recently and it seems like the barrier to (effective, practical, reliable) entry is so much HIGHER than it used to be, unlike with almost all other technology.
> Betteridge's law of headlines is an adage that states: "Any headline that ends in a question mark can be answered by the word no."
Duelling anecdotes!
So my employer, for instance, "has our own email" but it's just Gmail and we never have problems sending or receiving because we're piggybacking on Gmail's "This is a corporate account with several years of good behavior under its belt" trust signal.
More directly: spammers are killing independent email. Email's peer-node-trust story is so "version 1.0 Internet" that webmasters are left basically using heuristsics, shared models, and tea-leaves to determine whether arbitrary incoming messages should be trustworthy or not, and "they should not" is a good first-pass guess!
So Google (as the thousand-pound gorilla) is serving as a lightning-rod for a larger network-effect problem, which is "Users generally consider themselves better served if most unsolicited email they receive with no strong trust priors drops into a black hole." But that makes it very hard to be a newcomer who wants to establish trust priors.
YouTube, Twitter, Facebook, Craigslist, ...
Nearly anything public is bombarded with spam. The big players are far better suited to deal with it than any newcomers, and even they can barely manage it on their own platforms.
Spoilers, but the basic gist is that an advanced anti-spam AI decides to tackle the problem at the source, so to speak. Recommended reading.
One that probably leads very quickly to WW3, as the protagonists will find out, that most of the scammers are in countries not on friendly terms with the west (and therefore they tolerate cybercrime against the west).
There are also plenty of scammer operating from the west, though, so that anti spam foundation would not run out of work even if they just limit their activities to the west.
Some things just aren't profitable, and frankly shouldn't be. Let's not forget that.
A little disingenuous. Republicans attempted to force lower government spending by causing a deficit decades ago. The strategy (for those that were good willed) was that a large deficit would politically force spending cuts across the board, obviously including USPS.
No spending cuts have ever been made and the deficit has grown exponentially instead. Congress discovered debt doesn't matter, and now neither side cares.
Welcome to the uniparty. You can continue to pretend DC Democrats and Republicans are different, but they're playing you with fake controversies that never are resolved by Congress
> A little disingenuous.
No, it's 100% true. I'm not a Democrat by any means and I'm in fact highly critical of the Democrats. But if you look at the history of laws proposed/passed with regards to the USPS, there's absolutely no ambiguity that the Republicans are trying to kill the USPS. This has nothing to do with spending, either: the USPS has not been funded by taxpayer dollars since 2006, with the exception of the 2022 bailout. After 2006, the USPS operated profitably for a few years, but Republicans have passed regulations which prevent the USPS from operating profitably, leading for the need for the 2022 bailout.
This isn't about debt or spending, and it isn't a matter where both parties are the same. I'm not a Democrat, and I don't even like Democrats, but the USPS' funding issues are unambiguously and intentionally caused by Republicans.
https://about.usps.com/newsroom/national-releases/2022/1110-...
- Taxpayer support for the service were increased.
- Junk mail delivery were reduced.
?
I'm not sure what total junk-mail volume is, but the USPS moved 127.3 billion "units" (pieces of mail) in 2022, down from a peak of 213 billion in 2006.
First-Class mail volume was 48.9 billion in 2022, single-piece (which I presume means non-bulk) 12.9 billion.
"Marketing mail" volume, 67.1 billion.
<https://facts.usps.com/table-facts/>
First-class letter max weight is 3.5 oz (100 g).
Assuming an average weight of 100g per item, that's 6.7 million tonnes of marketing (junk) mail, give or take an order of magnitude.
A tractor-trailer rig can carry a maximum of about 45,000 lb (20 tonnes).
The junk-mail delivered in a year is roughly 400,000 such truckloads of mail. Or assuming 5-day/week delivery, about 2,000 trucks operating daily for a year.
(I'm using round numbers as the initial estimate is rough, just trying to give rough scope to the scale.)
That's basically every large industry.
Your complaint is about capitalism itself. USPS is not the cause of this.
Yes, I'm against federal subsidies in many industries. If it's a public messaging system such as USPS, why does it deliver so much spam? How about we just pay for it and stop that part? Everyone loves that with Fastmail vs Gmail - how about we do it for our national message passing?
> capitalism
How did you arrive at capitalism when I criticized federal funding? You think the idea of a federal govt and paying for stuff with public taxes is a core pillar of capitalism?
I do miss the old phone books - nice thin and absorbent paper. The old (UK) large format Yellow Pages could see you right for ages if money was tight.
This is where the US gets it wrong: over here, it's much cheaper for a company to send bulk unsolicited mail than it is for a regular person to mail a letter.
> I do miss the old phone books - nice thin and absorbent paper
Oof, if it's anything like the paper -- and, critically, ink -- used in our old phone books, well... yeah.
I very occasionally, perhaps once every year or two, get cold called in Norway but now they are always from offshore. A few years ago I got a couple of cold calls from Norwegian companies but when I pointed out that I had registered all the family's mobile numbers with Brønnøysund [1] they apologized and that was the last I heard from them.
[1] https://www.altinn.no/skjemaoversikt/bronnoysundregistrene/r...
Possibly at some point we'll be able to just blackhole calls that don't pass verification. But that won't really stop spam calls; it just means that they'll come from verified entities. I guess in that case you can go after them with complaints and get them fined in the cases where their calls violate the law, but that doesn't sound like a great solution... I have better things to do than chase down misbehaving companies.
With e-mail identifying the responsible sender is a lot harder.
<https://faq.usps.com/s/article/Refuse-unwanted-mail-and-remo...>
From the old Junkbusters site, there is a Form 1500 which can be filed against any sender:
<https://web.archive.org/web/19970713104642/http://www.junkbu...>
Form 1500, "Application for Listing and/or Prohibitory Order":
<https://about.usps.com/forms/ps1500.pdf>
Though specified as against "sexually-oriented" material, that has been deemed by courts to be at the sole judgement of the mail recipient.
You can also directly contact bulk senders through the DMA mail preference service, Valpack, RedPlum, and others. See one listing of these here:
<https://www.huffpost.com/entry/how-to-stop-junk-mail_n_5b27b...>
The Form 1500 is the take-off-and-nuke-it-from-orbit option, however.
Because as a condition of the discount, it is presorted by ZIP code, eliminating a significant part of the labor cost of processing it.
Sure, some senders would then try to send their bulk mail as normal mail. Then there could be another rule that defines what "bulk mail" is, and that anything falling under that definition must be sent bulk, with the consequence being fines and refusing to deliver that mail.
Yes, this is more complicated, and I'm sure some bad actors would still be able to get through, but I expect this would greatly reduce spam.
I imagine part of the issue with something like this is that the USPS is chronically underfunded, and policing and enforcing it would be to much extra work.
Bulk email is discounted because it saves the mail company a lot of money: it comes presorted, and there is no need to mess around with stamps.
There are several categories: 500 pieces of first-class mail, 200 pieces or 50 lb as "USPS Marketing Mail", Parcel Select Ground (50-piece minimum per mailing), presorted and carrier route sourted bound matter (300 pieces), library mail (300 pieces), media mail (300 pieces).
<https://www.usps.com/business/business-shipping.htm>
The tradition of discounting certain mailings, especially publications (e.g., newspapers) and books dates to the very beginning of the US postal service and was established by Benjamin Franklin.
<https://www.history.com/news/us-post-office-benjamin-frankli...>
I've nearly done the same several times in this thread myself. "Mail" really doesn't roll off my fingertips so readily these days.
This is never going to be perfect, of course, but anything that reduces the amount of crap I get in my mailbox every day isn't just great for my levels of annoyance, but also great for the environment. 95% of what shows up in my mailbox ends up in the recycling bin. Creating, printing, shipping, and recycling that material is a huge waste.
Because those are their customers, and they treat their customers well. The hundreds of millions of USPS mail recipients, on the other hand, are the product. USPS works for their customers, not for the rest of us.
https://insidesources.com/outbox-vs-usps-how-the-post-office...
>‘You mentioned making the service better for our customers; but the American citizens aren’t our customers—about 400 junk mailers are our customers. Your service hurts our ability to serve those customers.”’
-- Patrick Donahoe, US Postmaster General
This is of course an evil fascist talking point though, because the USPS has a large union that consistently votes Democrat, and how could we do wonderful mail-in ballots without the wonderful US Postal Service? Don't be a fascist, mindlessly cheer for USPS.
But once they have to worry about revenue and income statements and all that, an organization is going to care more about the interests of the people who pay them the most money, rather than the people they're actually there to serve.
Oh, wow. Look at this, never actually privatized.
> The USPS should exist to serve regular-Joe recipients of mail,
Can we get a federal telegraph agency too, just in case I need to receive some telegrams? You have no idea how horrible it is having to pay exorbitant sums of cash on private telegraph service.
The trouble is that the USPS has a union which is a reliable voting bloc for the Democrats. That makes it sacred. Even if it performs a service no one has given a shit about since the 1970s, everyone will gush on about how "what if I need to receive some mail" and pretend that mail service belongs somewhere other than a museum exhibit. I predict that every 10 years or so, it will be necessary to invent a new reason why we couldn't ever get rid of it.
https://www.cbsnews.com/amp/miami/news/change-coming-only-pa...
Following someone on Twitter is now the email equivalent of "double opt-in" for mailing lists.
The death throes of a doomed industry. Be it steel, horseshoes or advertisements, as profit margins drop production rates will increase to compensate. Then as the machine is running as fast an as efficiently as it ever has, suddenly the margin becomes zero and there isn't any more room to optimize. The entire industry suddenly stops overnight. I await that day.
Much of this spam is actively out to trick you, as opposed to legitimate players who have no margin left to give.
Youtube spam comments tend to have highly atypical patterns, using things like unicode characters to avoid triggering keyword and URL filters. Something along the lines of GPT should pick up on these things pretty easily, and could similarly pick up on the actions these comments are requesting of users "message me on telegram", etc. It could also probably detect when spammers are trying to impersonate youtubers.
It's not really the kind of thing that spammers can use LLMs themselves to work around, either. Any attempt to get past the anti-spam LLM is going to look quite unusual compared to the typical comment which would tip it off.
Strangely Google seems reticent to try something in this vein though…
There are loads of comment-copier/remixer karma farmer bots but reddit users are becoming better at recognizing and flagging those.
Obviously this has a selection bias because I had to choose the inputs but there were some that said things like "I ate a ghost pepper on my channel" and stuff that were clearly spam but, to someone not aware that kind of thing is trying to bait you into looking at their channel, it'd appear as possibly just genuine. Heck it may have been typed by a human who owns the channel but is still spam. GPT got it.
I tested this after the video came out a while back from one of the larger channels pleading for Google/YouTube to do something about all the spam comments and the general consensus seemed to be there was "just nothing they could do". Testing this lead me to believe they just don't want to do anything because if it's simple enough that some rando in his house can craft a prompt and get some examples to test in an hour or 2 then a multi-billion dollar company should be able to do SOMETHING.
There is probably some critical mass where this would work. Some people would pay to be able to just not have to do combat with the whole world simply to enjoy the Internet.
It would also make is possible for larger companies to attest to purchases made and / or the quality of participation within their community. Imagine a scenario where I donate $50 to an open source project using GitHub Sponsors with my domain as the identity and GitHub attests to me spending that $50.
Over time, it would be possible to demonstrate a significant "investment" in your domain validated identity and it would be done by spending money online like you normally would without any additional cost. The attestation that you spent the money is simply a side effect of something you're already doing, but it's a really good indicator (over time) that you're a normal participant.
At the very least, I think having a domain as a globally usable handle would help to reduce impersonation which is a serious, difficult to solve problem right now.
In fact, it would give yet another reason to hijack a domain.
In fact there might be significant overlap between something that can support a monetary system and something that supports trustworthy communication. The best you could hope for by solely focussing on communication is that it doesn't collapse under the pressure of having to keep financial transactions secure. Except any secure form of communication is probably going to be used for transactions of some kind at some point.
I can imagine using levels of its trust model to handle the levels of email I receive e.g. this domain has X validations and my friends haven’t blocked it.
Google Drive is infamous for its spam problem.
Meh. I've watched in the hundreds of hours of YouTube videos without ever seeing or receiving any spam.
I guess you must be referring to the comments? If so, perhaps I'm missing out but I've never gone to YouTube for the comments.
A decent first-pass solution to part of this might be to just have email allow every domain in my password manager.
I think the data's there to make this work a lot better, it's just that all the parts aren't talking to one another.
I wonder though, with the advent of new LLM models, it should now be trivially possible to build a zero-shot spam-filtering bot that is self hosted.
(I have the same situation, an 18-year old gmail account and a 6 year old fastmail account, but the reason I don't get ANY spam at all in the fastmail account is I only use it for certain things and it's much newer, so I'd argue at least in my case, that's not a fair comparison).
But the spam filter isn't as good as Google's, 100% agree with that.
I would guess so? It was days. Many days of "No mail was lost but access is down" messages.
I don't think so. They posted on twitter about it as a genera purpose outage.
Anyway. I'm glad they exist and I'm glad you've found value in them. I just couldn't continue paying for an email provider I could never access.
edit I remember they were open about experiencing a denial of service attack over several weeks.
edit again: https://www.fastmail.com/blog/fastmail-fights-off-ransom-cyb...
a reddit thread about it: https://www.reddit.com/r/fastmail/comments/qkk8qh/ddos_attac...
In the end it's content that matters, not the form that it's send to - I don't care if it's my grandma sending me offers for viagra pills, or a spammer, and I don't care about the language either - I just don't want to get such offers.
On the other hand, if there is an e-mail that may be interesting to me, I don't care if it was sent by a human, or by a machine - I want to see it in my inbox.
In other words - it's not about distinguishing human/machine written text. It's about distinguishing content that is worthwhile for me from the one that isn't.
When this happens, gmail informs the sender that the mail wasn’t delivered and they try a few more times before telling the user that no more attempts will be made.
That depends entirely on how the receiving server rejects the mail. If it just drops it into the spam folder or /dev/null Gmail won't be able to know. If it responds with a non-transient error code then Gmail will (correctly) NOT try again and inform the user of that.
If users really want that then they can just dump all email from addresses not in their address book. Some already do that. It turns out that most people actually want to be able to get email from entities they do not yet know.
citation needed for "most". "some" people want emails from unknown entities. I am not sure about how big that fraction is.
I email a car dealer. I expect a response. But I don’t know if it’ll be from Bob or Larry.
And I definitely don’t want to be added to thirty two different email lists they manage.
What is the evidence for this claim? Not Microsoft or Google acknowledge they drop email.
https://dataprot.net/statistics/spam-statistics/
I've had to set up someone's mail server last year. All mails sent to gmail were silently dropped until we set up all the current buzzwords for the domain/email server. Then they magically started to show up.
Possibly we were lucky that "just" setting up SPF DKIM etc fixed it.
SPF and DKIM are nowadays the very basic methods used to verify if your emails are spoofed or not. Not having them in place is as good as setting up a spam farm.
They do: the test emails were never rejected, but never arrived in the test gmail account either. Not in spam, not in all mail, not in the inbox.
And that's fine because that's exactly what SPF/DKIM were designed to do. The spam folder is not a dump of true positives. It's the bucket where you train the filter by evaluating somewhat likely false positives.
A perfect technical configuration doesn't make you inherently trustworthy or provide you with faultless reputation.
Users' interests are served by being informed when this happens. And plain text correspondence is not easily mistaken for malware.
> A perfect technical configuration doesn't make you inherently trustworthy or provide you with faultless reputation.
No one claimed such.
That's exactly what it should be. If somebody sends me an email, it should arrive somewhere that I can see. If that means needing tiers of spam folders, each one spammier than the last, then so be it.
This used to be a cardinal sin for mailserver operators; mail should always either be delivered, or result in a non-delivery notification ("bounce"). Dropping mail on the floor was a no-no.
Then along came gmail...
“Webmaster” isn’t a title I’ve heard in about 20-years.
Of course, spammers have significantly moved to using gmail and it just streams right through.
Nor is any mail server that interacts with other mail servers.
Using an email delivery service to ensure mail gets delivered is probably the easiest fix for singularity2001's issue. Otherwise they're looking at changing their hosting, changing their IP, or fighting the nearly impossible battle with Google to get their mail to be accepted by its system.
To implement, establish an email header field for a token. The tokens are one-time use, and can be purchased from some token selling authority. The recipient can check with the selling authority if it is valid. If it is missing or invalid, it goes to the junk folder.
https://en.wikipedia.org/wiki/Hashcash
Domain names and IP addresses aren't free. If we just immediately trash all email which doesn't have valid SPF + DKIM + reverse DNS, we place a small upfront payment requirement on sending email. Domains or IPs can be placed on a blocklist if they still send spam, thus making the domain/IP wasted.
The problem is that waaaay too many legit senders still don't know how to configure their email servers, so you end up with a boatload of false positives.
Ironically Gmail has started demanding DKIM or SPF, possibly slowly saving email from this mess of unidentifiable mail.
Can't you pay less than a penny a user to run ads in Gmail today ?
The idea is fair though I don't think it's far from where we landed with how the large email broker systems operate with the large email system providers.
I was planning to build a pretty sophisticated bayesian filter on top of that but it turned out to be unnecessary. The strategy I ended up with, which has served me well for many years now, is:
1. Block a small number of extremely spammy TLDs. There are about a dozen of these, including .biz, .casa, etc.
2. Anything received from someone I've sent mail to, or a sender I've previously marked as good, is assumed good.
3. Anything received from an address whose name contains an English word on a relatively short list of spammy words (discount, offer, etc.) is assumed spam.
4. Anything received from an address that I have received email from before and not marked as good is assumed spam.
That leaves emails from addresses that are sending me email for the first time. These are overwhelmingly spam, but after the four filters above there are few enough of these that I just scan them manually once a day or so.
The real key here is treating the first email from any given address as essentially a "contact request" with the default action being "deny in the future". That just turns out empirically to work incredibly well. More than 90% of my spam is from repeat offenders.
Honest question: Is there any reason to not just blacklist all emails from TLDs that aren't the Big Four (.com, .net, .org, .gov) and country TLDs (eg: .co.uk, .jp, etc.)?
I can't recall ever needing, let alone wanting, email from the later TLDs like .biz or any of the bloody stupid new TLDs that comprise entire words.
I typically register both a new TLD and a .com, and I use the .com for the email campaigns and reply-to
if it gets flagged that doesnt affect my more official one-to-one emails from the actual domain
so I would say, maybe? because I’m not allergic to “bloody stupid” TLD’s and have already adapted to what you’re wondering about
> Honest question: Is there any reason to not just blacklist all emails from TLDs that aren't the Big Four (.com, .net, .org, .gov) and country TLDs (eg: .co.uk, .jp, etc.)?
And you left out dot-edu.
And I get a fair bit of ham from other weird TLDs. So I think a whitelist is a bad idea. But that's because I prefer to have some spam get through than have ham get blocked by mistake. Even with my very sloppy algorithm, the only time spam ends up in my inbox is if a spammer spoofs one of my contacts, and that is extremely rare.
https://en.wikipedia.org/wiki/Country_code_top_level_domain
Anything that reaches addresses that you don't use gets marked as spam.
Common prefixes will work for cases where they're guessing your address, and hidden mailto links will cover situations where they're scraping your webpage.
'top','tv','biz','rocks','ru','science','bid','date','casa','buzz','work'
I have no idea how blocking TLDs of entire countries with hundreds of millions of citizens can be considered a valid anti-spam advice.
Just like the parent comment explained, you are actively pushing all of those people into the hands of Gmail (or Yandex in this case).
$35/year. That's not too bad. Compare to e.g. https://tld-list.com/tld/protection
So... my apologies to the 12,000 residents of Tuvalu. I'm sorry to have to block you all, but for some reason your country code attracts spammers like flies to honey.
It is very different if you offer it as a service to other people, or if you do it for yourself.
If you are doing it for yourself you know if you expect emails from country X or not. And if things change and suddenly you have a pen pal from there you can change your filters easily. So you can live with such simplifying assumptions.
If you are providing email services to masses of other people then you can’t make these assumptions of course. You have to use more sophisticated means then.
These things just get thrown in the large pot of heuristics.
Nowadays (irony of ironies!) most inbound spam is from gmail.
Do tell me more about this wonderful new method of spam filtering with absolutely no bigotry.
Just imagine the racket if Gmail would start de-prioritizing email from Alaska or Hawaii because, in their experience, that's a large source of spam. If might even be true, so what? That's the exact thing racists say, racism works: we found the guy suspicious because no black people live around these parts, so we knew he was up to no good.
you're making comparisons here that are so disjointed it's not even apples and oranges... it's more like apples and rocks...
this thread grew outta the spam filter setup of someone hosting their own email server. bringing gmail into this and talking about how they'd have a PR nightmare if they screw with hawaii doesn't work. yeah, they both send/receive email, that doesn't mean they're comparable in the way you're tryna say... i mean, apples and rocks are both round sometimes, but you wouldn't try comparing how they taste, right?
As someone who was on the receiving end of the "Romanian users have nothing to say to my users and postmastars there can't even RFC" blackholes, trying to get legitimate email delivered while respecting all the rules of the internet, I can only say that whomever thinks like that is an idiot and a bigot. Unfortunately, there are many such idiots on the internet at large, and the aggregate effect is that only large sites like Gmail can maintain good deliverability, which was the subject all along.
The poster you’re replying to is talking about their filtering method for their own email. Presumably they know if they expect to ever receive an email from a given country.
Otherwise, let's hypothetically say you have a friend named "jacob peter" who really likes his initials (and doesn't understand the topic of this discussion), so he registers somewhere in Japan that lets him have dot-JP as his TLD. The only hypothetical way he knows to contact you is via your email server (let's also assume he doesn't do stuff like social media, phones, etc., so he can't find any other way and you have no way to contact him). If you have hypothetically preemptively blacklisted all of Asia's TLDs, then you'd never talk to him again for as long as you live, right?
Since he’s not real I don’t know if that’s a good thing or a bad thing. Maybe he’s kind of a dick.
In many years of doing this at work I think we've had to manually whitelist three domains (our corporate travel provider started sending from a .travel domain instead of their .com without telling us they would make that change...). I don't have numbers handy but it significantly reduced our junkmail intake.
There's always some mail that gets blocked, of course, but it's a relatively rare occurrence.
Maybe in your case something that allows users to view and release their own quarantined mail [1] coupled with a healthy dose of training on spotting malicious emails and so on?
[1] e.g. this sort of thing https://community.spiceworks.com/topic/2474067-office-365-by... - not endorsing M365, just as an example.
What tools are you using to implement the controls you describe above?
Yet suddenly my emails go to spam in gmail.
Now, there's no way for me to do anything, because I'm nobody.
As for spam, my bayesian filter works just fine, so if I can successfully filter spammers out, I think Google can as well.
I feel like Google is doing this purposefully to make people like me to simply give up, and observing responses, it is successful.
This is absurd. The problems Google face are orders of magnitude more complex than your individual mail server. Your server benefits from the fact that no one cares about it. No one will develop a custom spam technique to send you spam. They will for Google. There will be whole offices in India dedicated to working out exactly what slips through Gmail.
You benefit from a kind of herd immunity where no one even sends the kind of spam that would reach you because it's blocked by Gmail so not worth sending.
Your personal gmail account has the same risk of receiving spam as mine, but Google has much more tools available to detect spam. For example they can easily see somebody is spamming, when large gmail accounts are receiving the same message. They can easily see which hosts are compromised proxies as they receive tons of mail, and seeing large number of e-mails sent from questionable hosts should be easy to spot.
They didn't really improve spam detection much for for a decade, because blocking small hobbyist mail servers is more likely increase their user base.
Those special farms you're mentioning, wouldn't work, if filtering would be individual per user, as it should be, because each person's mailbox is different and each person has different definition what spam is.
We need to stop buying the narrative that they are doing this to fight spam and are doing it as a gate keeping exercise.
Disagree.
In the early 00s, spam was killing email. I was getting thousands of spams per minute to my personal email domain (same one I still have and have had since the mid 90s). It was real bad back then.
Doing aggressive sanity checking on incoming email back then was problematic since nearly all legitimate senders also has badly misconfigured email servers.
Those days are gone. Legitimate senders have well-configured email now, one can do a lot of sanity checking on the SMTP connection and that alone cuts out most of the would-be spam before it is ever sent and none of the real email is impacted.
Sprinkle a bit of bayesian filtering on what remains and spam is mostly a non-issue anymore.
So what it the problem these days? Part of it is a legacy cultural problem. A lot of email admins wear so many scars from the early 00s that they still operate on a mental model that they are willing to throw away a lot of legitimate email just to prevent even one in a million spam to go through. That's highly counterproductive and causes a lot of harm to interoperability.
The other big factor of course is that the very deep pocket email providers (gmail, microsoft) have a strong vested interest in strenghtening their monopoly even further so they are happy to go along anything that decreases interoperability.
That all said, I strongly encourage everyone who can to run their own email. Interoperability can only be saved by exercising it as much as possible. And it still works fine despite the naysayers. I've been running my email infrastructure for a long time and have no deliverability problems anywhere that I've run into.
And I concur with the original commenter. I do zero filtering on my SMTP server (no reverse DNS lookups, no blacklists, no nothing), in order to be sure I receive every single email, and SPAM looks like it would not be an issue even if I disabled the bayes classification client side, these days.
I mostly rely on providing unique random addresses to each service. My oldest 18 year old address gets quite a bit of spam so I blocked it and switched to different one, but newer addresses that I use for the last 5 years didn't accumulate much spam, despite being public and crawlable on the web.
The article said people said so on HN.
Or they work at a company that demands it...literally demanding IT remove all spam... and also demanding that nothing important ever gets blocked.
Once we figure out a way to handle website registration that’s not tied to email (whatever that means) I’m not sure email will be useful for anything.
Most useful conversations have moved to messaging apps.
If I want you to be aware of a long article I'll send it to you in an email because if I message it, most people will open it, see it takes more than 2 minutes to read and close it and never go back to it.
EDIT: agree though on your point that for some the right bookmarking tool hasn't been built yet
Do you have access to WhatsApp‘s source code then?
That's not a future I'd like to ever live in.
Why? Because all of those are proprietary solutions that don't interoperate with anything and their behavior and existence depend on the whims of a single corporation. No thanks.
Email is standard, interoperable, owned by nobody, accessible to everyone. It's the optimal way to communicate.
I think it is exactly these sentiments that do most harm.
I mean, that is completely false at every level (technical and operational).
Sure, gmail is a big player but there are many others. Most importantly, there are thousands of smaller players. All of which interoperate.
I can't tell if you say that sarcastically or from a position on being uninformed about how email works. If the latter, do take some time to understand email infrastructure at a high level because it is a case study in beautiful interoperability of open standards. Something we must all keep in mind for the future of the Internet.
Email has been resilient and an open communication mechanism from everyone since the 70s precisely because it cannot be owned by anyone, it is an open interoperable standard.
There will never be anything equally long-lived and feature rich from a single corporation. Corporate interest don't align with that. Proprietary solutions will always be about locking you in and limiting interoperability.
So many people completely miss this point. If whatever is supposed to replace email it will have to be very similar in features. That is
1. Permissionless (open for anyone to send messages or operate a server) 2. Asynchronous 3. Resilient 4. Searchable
No IM solutions I know can do that at all. Matrix would come closest but asynchronicity and resilience are still troublesome on that part and all clients are more geared towards IM instead of async longform discourse
Email is incredibly heavily relied upon and with great success. People get their pay receipts, their rental applications, notifications of mentions on social media, speeding tickets, music and gig notifications, reminders to get their pet to the vet, X-ray reports, prescription reminders and I could go on.
Not to mention the huge boom in email digests and blogs over the past 5+ years.
I haven't seen - _any_ businesses communication conducted over signal or WhatsApp - ever. I'm not saying it doesn't happen somewhere but it's certainly not common place. And telegram? I didn't even realise that still existed!
While I have no doubt that doing this is a lot easier and more hands-off than the last time I did this (for a few years during the mid '00s), I just don't have the time or energy for it. Also, email is a critical service, something where I don't want to have to deal with downtime, especially if I'm away traveling or something like that.
I think as a decent alternative for people like myself, it's at least helpful to use an alternative hosted email provider (other than GMail or Microsoft) to help keep the ecosystem open.
I have to say that as the most critical service of all, that's one of the reasons I do host email myself!
I can't and won't take the risk of being locked out by the whims of some third party misconfiguration (we've all surely seen the endless stream of misfortunes by those getting locked out of gmail/etc).
I used to run independent email. It took constant work to get close to gmail's level of spam blocking. So I switched. Found most alternatives weren't anywhere near good enough, and I don't have enough hours in the day even for my own email.
This has been getting consistently worse over the last few years, in my experience. At least on the user end, i.e. far more false negatives getting through.
> I used to run independent email. It took constant work to get close to gmail's level of spam blocking.
This is very contrary to my experience on both fronts.
gmail spam filtering is mediocre. Spam gets through and good emails are flagged as spam. It's not very good.
For my own domain where I self-host my email, my spam filtering is quite a bit better. Approximately nothing is ever mis-flagged as spam (has not happened in years) and the spam that gets through is a tiny fraction of what my gmail address gets. So for me my own spam filtering is much better than gmail.
And no, it doesn't take constant work. In fact doesn't take any work at all. I set it up years ago, the bayesian filtering goes through all the mail but it takes zero effort from me.
I wish this myth that gmail has some incredible filtering technology that nobody can replicate would just die already. Gmail isn't terrible at spam/ham filtering, but it's not that good either, just mediocre. You can do much better with minimal resources on your own.
I barely use email anymore. Everything at work is mediated by Slack or Atlassian. With friends and family it's almost all text messaging. My kids' schools and sports teams use a bunch of different proprietary web and mobile apps to communicate with parents.
If you were out on a walk and 9 out of 10 people where trying to mug you, you'd very quickly adjust your behavior to only walk in very safe places and let as few people as possible access that area.
There is a significant cost in spam protection by tracking reputation and content for the unending ocean of bullshit flooding the SMTP lines. Most providers want to cut communication with the spam source as quickly as possible to reduce costs.
Consider that email accounts are the go-to account recovery method for most services, and it's ubiquitous in biz. Also consider that you can prioritize specific domains or filter x domains to never go to spam, e.g., your own company's domain.
Any "death" is that people struggle with their own mailserver as a general rule of thumb. Does that thus mean email is dying? No. As the article says, perhaps independent email is, but it hasn't been in a good place for over a decade at this point.
Oh my god, yes. To all appearances they declared defeat in the Great Webspam War some time around '08 or '09 and their results have been markedly worse ever since.
I get dozens of FB Messages weekly trying to scam me out of a verified Facebook Page.
I get dozens of WhatsApp messages per day spamming me.
I get tons of Twitter and Instagram spam to the point that my DMs are useless.
I get Telegram spam weekly.
I've even started getting spam on bloody Matrix protocol.
Just because your experiences are x doesn't mean that it doesn't happen.
I don't ever intend to respond to spam, and have become extremely adept at spotting the patterns and swatting it away. However, it becomes a game of chance, when a service like Outlook puts it right at the top of the app (both iOS and Android) where you would reflexively jab at it, unless of course, you pay the premium to remove it.
For now, I have found a way to stop this nuisance. However, MS are playing fast and loose with their policies and now very legitimate looking spam is leaking into the inbox, escaping any filters. Since last year it is appearing along with the glaringly obvious Unicode riddled ones, with increasing regularity. It seems like a matter of time and co-incidence, where you would end up interacting with a piece of disguised mail you were expecting e.g. an order from Amazon or a service which you use regularly, and possibly respond without checking the header.
This recent episode was probably the worst experience, albeit not the first time it has happened.
https://www.theverge.com/2023/2/20/23607056/microsoft-outloo...
Flip-side, there seem to be more spammy messages sent from @gmail.com addresses than from any of the other email A-listers.
On the other hand, it's simply impossible to satisfy Microsoft. We're irrevocably tainted by being in a netblock of a well-known provider, despite having held the same IPv4 address clean for over a decade.
I followed the process, and then kept insisting a bit by answering the emails saying they were not going to do anything and I had to check if I was complying with their rules etc. After two emails I had a real person answer me, and a few more emails later (basically insisting I was already enrolled in their various bullshit spam reduction programs and there was zero spam problem with my domain) I got told that I had been whitelisted.
It's being sent to an email address I know is not registered with Google.
I'm far from the only one with a similar issue. See https://support.google.com/groups/thread/68075070/i-get-goog... .
https://news.purelymail.com/posts/blog/2019-06-21-deliverabi...
I'm guessing some of the issues are just from Google being random about what it considers spam no matter who it comes from. I remember a comment on a somewhat recent thread from someone who had to move their business mail away from Gmail because Gmail would classify mail from one paid account at their organization to another under the same organization as spam.
Some things are just baffling. How can they manage to flag themselves as spam it’s beyond me.
There are exceptions, e.g., I sent an email full of research with sources to a family member I've been emailing with for fifteen years and it went to spam, despite it being @gmail.com to @gmail.com. In retrospect, I was misusing email versus sending a document or a link to one with it in.
The times have changed. It's not unreasonable to expect how people use email to have changed, ergo people sending emails full of URLs are statistically more often than not spammers.
Is it a broad stroke? Yup. But I'm willing to bet that I'm a fringe case and it prevents a ton of spam.
You can even test it yourself without fancy tools by sending emails full of links and mentioning keywords like Viagra et al.
For other filters/ISPs, content can matter a bit more, but as a general rule for consumer mailbox providers, metrics are the primary thing that affects filter outcomes.
[Edit]: Forgot to mention that the IP is home IP, not cloud provider/hosting or something.
And as as sidee note: the problem lies with the users and not with the email sender. For instance, if I send an important email to someone about an item they ordered from my site, and they tell me to send it to their gmail account, I connect to gmail to send the email, but if Gmail rejects it, I have fulfilled my part of the deal. If an email is marked as spam or rejected, it's on the recipient's end, not the sender's. The sender did not flag the message as spam before it is sent, the recipient did it after receiving it.
So 100% certainty of it being spam? Was that the correct judgment, in your opinion?
It should go without saying that confirmation emails of a user initiated booking is not spam, but let's just say it anyway
Basically, you can't block the big providers - Gmail, Microsoft, AWS SES, Mailchimp, Mailgun and friends - because everyone and their dog is using them. But their reaction to abuse reports is spotty at best... you're stuck between a rock and a hard place.
The root cause obviously is spammers and scammers, but governments don't care about putting a final stop to bad actors.
Why? Respectable businesses send from their own domains. Friends and family never send emails nowadays, there are messengers for that. Anything from google goes straight to Junk folder.
B2C email is quite competitive with dozens of services.
Overall I’d say the email ecosystem is relatively healthy. It’s more competitive and interoperable that instant messaging with greater security than SMS.
If Google wants to receive your traffic at a later date thats their business and not yours. It's an open system where sites set their own policies. Access to a site's eyeballs is not the right of an outside sender!
Is my experience unusual?
I have an account since the beginning of gmail, and I get around 120 spam messages a day (roughly one every 10 minutes)
Also, with self-hosted spam filters, I had issue with false positives - that is my filters flagging proper e-mails as spam.
I hope that the new LLM systems should finally fix all that.
Another issue is that if you self-host, your e-mails are more likely to land in spam in your recipients inboxes. Big providers don't mind that :/
It's been much better since I took the time to set things up so marking and email spam automatically fed it into sa-learn. I still have to have a handful of rules to filter out senders who are "legit" enough to make it through, but ignore unsubscribe requests.
I setup a dedicated server not on a major cloud host, and am not looking forward to all the details involved in the lack of trust starting out. Let alone the dark art of spam detection. But I want to get back into it if only because I don't like how the major parties are cornering things up. I also want to be able to actually handle mail for several domains and not have it nickel and dime me to death. It costs way more for a single email account these days than it does to run a few dozen minor websites.
While it's nice that Google Domains (when you use their DNS) and Cloudflare both have included email forwarding, sometimes you want an actual box to send from too. And with the partitioning that GMail now does, I can't find anything anymore without hunting for it... the only benefit is two of the subtabs, I'm able to just delete all once in a while.
I wish that email were much more reliable and able to actually setup 2-way relationships similar to IM clients. And of course, limit/remove third party info sales/spam in those relationships.
If you're only a small time sender lack of trust is permanent. I've been self hosting for three years now, never sent spam, never had the server breached and GMail and Microsoft both still send mail to spam.
At least Microsoft no longer outright 550 refuse my mail so I have that going for me which is nice.
DKIM, SPF, etc only go so far anymore, and even graylisting doesn't seem nearly as effective as it once was. Again, not really looking forward to parts of this, but I do want to at least try self-hosting as much as I can. I like the cloud offerings from MS/Google, but don't like the companies or their actions in and of themselves. Only one real way to push back, and that's to actually try.
And on the small providers, yeah I can get that. I'm a bit more than a decade removed from self-hosting much of anything, and even then balancing DNS RBL with other factors was at least interesting if not frustrating.
* In fact, I'm still using the filter I installed on my machine in 2003.