I believe this volatility with email notifications is one reason why some small companies choose to direct people to social media platforms like the-platform-formally-known-as-Twitter, because it won't surprise them with a fee when they need to get the word out. It's pretty sad to hear when some indie creator can't use the opt-in email list he built up when release day comes around.
It should be "the platform FORMERLY known as Twitter", but it's pretty funny this way since I imagine it's still technically Twitter in most formal ways.
Gmail support on the back end has solved issues like this within a day or two. I think it's perfectly possible to run your own server but it does take a lot of active administration
> Gmail support on the back end has solved issues like this within a day or two
I mean, has it? Even for plebes who might not know someone on the inside who can file an expedited request?
The problem isn't running your own server (that part has been a solved problem for over 25 years), it's keeping it off of these arbitrary "naughty lists". If Google decides to blacklist your IP, you're practically SOL. It becomes a Red Queen problem where you're forced to constantly fight these arbitrary battles.
The blacklist companies are kind of annoying. I’ve been hit by a range blacklist. It’s generally temporary and annoying to opt out of. I get why they exist though. There are spammers that will register hundreds, thousands of domains behind anonymous registrars from the Caymans and try to run a phishing scam by spoofing their domains, etc. You can report abuse to the registrar but… it’s a lot of work to get through each of 50917.com, 643801.com, etc for each offence. Much easier to setup spamassassin to report to a blacklist.
It causes splash damage to legit, indie operators… but aside from engineering a completely new protocol and migrating everyone in the world over to it, blacklisting is not great but it’s easier than all that.
I’d love to find a way to make it better and get back at these scammers much the same way folks are figuring out how to report and deter phone scammers.
I don't know how much sense it makes to be "sovereign" in a system designed around interaction with others but if it helps, my maildir is held on a disk I control and no one else is scanning it for marketing insights.
You're not "running your own mail server" if you're using Amazon SES (or some other service) to send your email. Which is the point the post on Twitter is making.
I am, literally, running my own mail server. telnet port 25 on mail.calpaterson.com if you don't believe me.
Relaying is a central feature of email infrastructure and it isn't somehow cheating to relay outgoing mail to someone else rather than send it directly.
I run my own servers. They’re on a VPS so I can empathize with folks when their mails get blacklisted because their server is on the CIDR range as some offender.
When I get repeat, bad spam that manages to sneak past my DKIM filters I generally reach out to the ISP that originated the messages and report the abuse before I report to the blacklists. Admins are generally quick to respond.
The problem is the spammers, scammers, blackmailers, etc. And maybe the domain registrars who enable them and anonymize them a little bit.
AOL Feedback loop... Hey, someone from your domain sent someone on our domain an email they classified as spam. Who? We won't tell you. Don't do it again, or we will refuse all mail from your domain. k thx bye - AOL
Blame SPAMmers who abuse the system for their profit.
Blame our lawmakers who appear to be very lax in regulating SPAM. (In all forms, not just email.)
Blame the design of email that allows anyone to email anyone at any time for anything.
Blame the people who think that AI is capable of sorting out the good emails from the bad ones.
Gmail's incentives are to keep customers, and a big part of that is blocking SPAM. Unfortunately, running your own email server is always going to be a cat-and-mouse game with regard to making sure that you're following whatever best practices are needed to make sure that you're not a SPAMmer.
I think you can blame Google in this case. They are marking email as spam from someone who has gone through all necessary verification steps to confirm he is not a scammer.
I own my own domain and rely on the hosting company's servers for sending email. (But I send most email from my @gmail.com address.)
I understand the desire to operate your own email servers; but keep in mind that the cat-and-mouse game is going to continue to put the squeeze on you and require more and more of your attention. It's "not a big deal" if you're a large company, or operating email servers is your full-time job. Otherwise, if this is a hobby, the $100 / year to have someone else run your email server is probably a lot cheaper than your time.
The fact that Google thinks that they don't need to federate with email merely means that delivery to GMail is unreliable.
There is a market-based solution: stop using GMail, and tell clients and partners that you don't expect email to GMail to work reliably.
There is a regulatory solution: require all commercial email providers (i.e. organizations that provide email to non-employees in exchange for money) to interoperate by providing a working grievance procedure for rejection of non-bulk email.
Nobody really wants the regulatory solution, so it's time to widely implement the market solution.
Gmail will mark my outgoing mail as spam if I include a simple link to my personal website. I literally can't email people links to my site if I want to email them from my GMail account. It's insane.
The market based solution will work about as well as telling prospective employers, friends, family, your kids friends etc that you don't accept calls from Verizon. It won't work and nobody will do it.
Your regulatory solution on the other hand appears well thought out and workable.
Can you even tell if a call is coming from verizon? AIUI, in the US, mobile numbers look like any other phone number. (here, each mobile carrier got an 08X prefix where the X was exclusive to them. But you can port numbers instantly, so that only tells you where they started.)
A few years ago I bought my own easily-pronounceable domain, pointed its MX records at Fastmail's servers, and transitioned away from Gmail. In the 3+ years I've been a Fastmail customer, I have never had a problem with my email being sent or received.
Is it full sovereignty? Depending on your definition, probably not. But it does give me a few things:
1. I'm not contributing to the Gmail monoculture
2. If Fastmail stops being awesome, I can point my MX records somewhere else
Even higher levels of control would be nice, and Gmail is trying to shut down independent operators, but the choices aren't "run your own server" or "give in and use Gmail". There's a spectrum of email sovereignty, and the kind of nearly-complete sovereignty that comes with a personal domain is well within reach of most people on HN.
In my experience the amount of spam is directly related to the domain (and what it's digital footprint is).
That being said, for my hobby/personal domains, the spam has been as close to zero as you could reasonably expect (maybe a few per week, and they are correctly flagged as spam).
I see less spam in my inbox (effectively 0) than I did with Gmail, but that could be because I've been more careful about what I sign up for than I was with my old email (which I got as a kid).
I've been toying with the idea of setting up a parallel email system using the same protocol and server software, but as a separate space entirely. Perhaps with a gateway (via a commercial provider) to/from the larger email system, but perhaps not. I'm not convinced there's much value to doing that.
The idea would be to bring the value of email back to at least my social circle.
I think there's a lot of value. IMO we should add one of two (or some combo) things:
- Hashcash
- Clients can send an allow list to servers; anything from an address not on the allow list goes to the spam folder
Email is a foundational idea; I think if we give up on it we'll keep reinventing it (we already are: RSS, ActivityPub, etc). We can get it back if we move off of centralized machine learning as our spam solution.
There is something about having to wave goodbye to corporate Internet infra though. I think we gave them a shot and we discovered they're not good stewards (web, email, chat, etc). We need to rebuild it for ourselves, and if that means people using the corporate gateways miss out on things, OK then. That was true if you were using AOL and it's true again; we know how this goes.
I've been running my own mailservers for about two decades.
I rarely have deliverability problems and never to gmail.
I keep reading about this problem, bit at least for me it's just not there.
38 comments
[ 3.8 ms ] story [ 111 ms ] threadI believe this volatility with email notifications is one reason why some small companies choose to direct people to social media platforms like the-platform-formally-known-as-Twitter, because it won't surprise them with a fee when they need to get the word out. It's pretty sad to hear when some indie creator can't use the opt-in email list he built up when release day comes around.
I mean, has it? Even for plebes who might not know someone on the inside who can file an expedited request?
The problem isn't running your own server (that part has been a solved problem for over 25 years), it's keeping it off of these arbitrary "naughty lists". If Google decides to blacklist your IP, you're practically SOL. It becomes a Red Queen problem where you're forced to constantly fight these arbitrary battles.
It causes splash damage to legit, indie operators… but aside from engineering a completely new protocol and migrating everyone in the world over to it, blacklisting is not great but it’s easier than all that.
I’d love to find a way to make it better and get back at these scammers much the same way folks are figuring out how to report and deter phone scammers.
I had to get approved but they charge nothing for my usage level and it solved some of my (minor) delivery issues.
I recommend anyone running their own mail server look at it.
Relaying is a central feature of email infrastructure and it isn't somehow cheating to relay outgoing mail to someone else rather than send it directly.
When I get repeat, bad spam that manages to sneak past my DKIM filters I generally reach out to the ISP that originated the messages and report the abuse before I report to the blacklists. Admins are generally quick to respond.
The problem is the spammers, scammers, blackmailers, etc. And maybe the domain registrars who enable them and anonymize them a little bit.
Blame SPAMmers who abuse the system for their profit.
Blame our lawmakers who appear to be very lax in regulating SPAM. (In all forms, not just email.)
Blame the design of email that allows anyone to email anyone at any time for anything.
Blame the people who think that AI is capable of sorting out the good emails from the bad ones.
Gmail's incentives are to keep customers, and a big part of that is blocking SPAM. Unfortunately, running your own email server is always going to be a cat-and-mouse game with regard to making sure that you're following whatever best practices are needed to make sure that you're not a SPAMmer.
I understand the desire to operate your own email servers; but keep in mind that the cat-and-mouse game is going to continue to put the squeeze on you and require more and more of your attention. It's "not a big deal" if you're a large company, or operating email servers is your full-time job. Otherwise, if this is a hobby, the $100 / year to have someone else run your email server is probably a lot cheaper than your time.
"The death of decentralized email: A historical review of the multi-decade centralization and capture of the email protocol."
https://blog.lopp.net/death-of-decentralized-email/
There is a market-based solution: stop using GMail, and tell clients and partners that you don't expect email to GMail to work reliably.
There is a regulatory solution: require all commercial email providers (i.e. organizations that provide email to non-employees in exchange for money) to interoperate by providing a working grievance procedure for rejection of non-bulk email.
Nobody really wants the regulatory solution, so it's time to widely implement the market solution.
Your regulatory solution on the other hand appears well thought out and workable.
I think you will find the market will consider it your problem.
> Nobody really wants the regulatory solution
Speak for yourself.
Is it full sovereignty? Depending on your definition, probably not. But it does give me a few things:
1. I'm not contributing to the Gmail monoculture
2. If Fastmail stops being awesome, I can point my MX records somewhere else
Even higher levels of control would be nice, and Gmail is trying to shut down independent operators, but the choices aren't "run your own server" or "give in and use Gmail". There's a spectrum of email sovereignty, and the kind of nearly-complete sovereignty that comes with a personal domain is well within reach of most people on HN.
That being said, for my hobby/personal domains, the spam has been as close to zero as you could reasonably expect (maybe a few per week, and they are correctly flagged as spam).
I've been toying with the idea of setting up a parallel email system using the same protocol and server software, but as a separate space entirely. Perhaps with a gateway (via a commercial provider) to/from the larger email system, but perhaps not. I'm not convinced there's much value to doing that.
The idea would be to bring the value of email back to at least my social circle.
- Hashcash
- Clients can send an allow list to servers; anything from an address not on the allow list goes to the spam folder
Email is a foundational idea; I think if we give up on it we'll keep reinventing it (we already are: RSS, ActivityPub, etc). We can get it back if we move off of centralized machine learning as our spam solution.
There is something about having to wave goodbye to corporate Internet infra though. I think we gave them a shot and we discovered they're not good stewards (web, email, chat, etc). We need to rebuild it for ourselves, and if that means people using the corporate gateways miss out on things, OK then. That was true if you were using AOL and it's true again; we know how this goes.