Ask HN: I have tens of thousands of subscribers – how do I send a newsletter?

24 points by TekMol ↗ HN
A few months ago, I put a "Sign up for my newsletter" form on my website. When someone entered their email, I simply stored the mail, sent a "Thanks for signing up" email via PHP's built-in mailer and did nothing else.

Now there are tens of thousands of subscribers. I got some opt-outs which I manually deleted. And a few hundred bounces, which are still in my inbox.

What is the best way to send out a newsletter now? I think I will need some service that I can:

1: Import the mails

2: Import the bounces so they get analyzed and removed from the list.

What is a good service for that? Or should I use some open source software that I host myself?

46 comments

[ 5.0 ms ] story [ 112 ms ] thread
There are 2 options that are very popular these days.

1. Substack.com. This one would take care of everything for you. Import your subscribers and start writing.

2. Ghost.org. This is if you want to set up yourself and have full control.

I did a bit of googling like "substack import bounces" and "ghost import bounces" but nothing came up.

So I am not sure those would do the job.

Why can't you just remove bounced addresses manually (with a one-off script, etc) from your list before importing it?
Bounces probably have all kinds of funky formats and edge cases.

A software that analyzes the bounce mails probably needs to know about all the different formats in which mail servers tell you "A mail from you bounced: Here is the recipient".

unfortunately, substack does not have an API to continually import new subscribers.

MailChimp might be the best option here.

Do you have a list of 10.000 email addresses or do you have 10.000 emails in a dedicated inbox?
A list of more than 10k email addresses.

The addresses are not the hard part. The hard part are the bounces. Those are just emails in my inbox. I could write the software to parse those. Most seem to be in the form of

    Blablabla ... The following address failed:[some whitespace]joe@example.com...bla bla bla
But there are tons of other formats also. Like:

    Bla bla bla Your message couldn't be delivered to joe@example.com because the remote server is misconfigured
Some are even in foreign languages.

So I would prefer if I don't have to write that myself.

If you need it as a one-off I would think that a simple regex to search for anything looking like an email address would get you 80% of the way there, and the rest can be done by hand.
Why do you need to handle the bounces specially?

I presume that whatever service you pick to send the mails will have its own bounce handling. You can give it the full list of emails and the bad addresses will be removed after the first newsletter you send out.

I would expect sending to bounced mails will lower your karma with Google and Co.
Do they all include the words “postmaster” or “server at”?

You also have to handle subscribers who want to unsubscribe. You need a footer with an unsubscribe link and a web server and software at your end that handles that. Or, use a mail list service that handles that.

SES: Cheap & DIY & somewhat time intensive & Ok but not great delivery. Add dedicated IP for like $60/mo and you’ll get into pretty good to great delivery

Sendgrid / Mailgun: DIY & somewhat costly & pretty good delivery

ConvertKit: $100-150/mo for you current list. Focused on creators. Some marketing automation. They use a third party like Sendgrid but they manage everything. Fantastic to mediocre results. Fairly responsive team. Least labor requirement.

Keys to success DMARC. CFL participation. Problem with CFL participation, some of the big guys (like Comcast) won’t let small ships into the system. CFL essential to catching “Spam” button events to unsubscribe and not destroy your sender reputation.

Then there’s the whole spam control monoculture/cartel of BrightWorks / Symantec et.Al. Misbehave with one of their customers and you’re fucked for all their customers.

What benefit do I get from using those services, over putting an open source solution on a $5/month machine and using that?

This one has 8k stars on Github:

https://github.com/knadh/listmonk

This one also seems to be popular:

https://github.com/phpList/phplist3

Deliverability.

If you DIY it and mess one thing up (out of like dozens of configs) you're going to get blacklisted by huge vendors who don't care about your tiny business and it's really hard to get out of that.

Email is one thing you really really really really don't want to be cheap on.

Why would the "huge vendors" care about my configs?

Let's take Google for example. Wouldn't they just watch the bahaviour of their users? How they interact with the mails, if they flag them as spam etc.

No, there are a few other domain level settings and shared 3rd party reputational blacklists and maybe throttling concerns and such
The biggest cost you'd have is getting a "clean" IP in the first place. Forget about any kind of cheap server or cloud provider - you'd want something that's unlikely to have been used by a spammer recently. I've had good results with leased lines but those are expensive and not worth it if all you need is email deliverability.
I use the mail server of my hosting provider. It is a cheap mass hosting provider that offers mails for free or almost for free when you have a domain with them.

Do you mean their IP?

When I send one of my confirmation mails to mail-tester.com, I get 9/10:

    + SpamAssassin likes you
    - No DKIM
    - Your message does not contain a List-Unsubscribe header
    + You're not blacklisted
I mean the IP of whatever will end up talking to the mail servers of your recipients - it may be, though is not always, the IP of the SMTP server your sending code is talking to.

In general when it comes to hosting providers, if they're doing shared hosting you're guaranteed to have some compromised Wordpress/etc crap that will be spamming and will tank the deliverability of everyone using this hosting provider's email server (and their IP ranges in general).

In addition, beyond public blacklists that Mail-tester/etc can query, big email providers maintain their own private ones which you can't query and don't know the inner workings of it. Microsoft is a major pain in this regard.

Why would Google need to base their spam detection on IP alone?

If they see low "mark as spam" ratios for my mails, why would they care if they come through the same IP as spam from compromised wordpress instances?

In general, even DKIM and a clean domain isn't a safe indicator of non-spamminess. It doesn't take much effort nor cost for a spammer to get a new domain, set up DKIM/etc correctly and then start spamming, and those emails would appear as "clean" as yours do. Therefore they use extra heuristics including the source IP ranges.

Plus in your case your current mail tester result suggests you don't even have DKIM set up, so as of right now there's literally no way for a recipient server to tell your email apart from a spammer impersonating you.

(to be clear I'm not defending the practice, just explaining my understanding of the current situation)

    you don't even have DKIM set up, so as of right now there's literally no
    way for a recipient server to tell your email apart from a spammer
    impersonating you.
Coudln't they do that via SPF?
SPF is less of a guarantee - if your SPF policy whitelists a shared email server then anyone using that can still impersonate you. DKIM is a stronger guarantee as it requires having access to a signing key.
If your time is worth Zero then your math is correct.

Here’s a hint: I started running my own email servers in 1997 and I eventually ended up using all of the above I suggested.

Sending email is not the same thing any more. My best bud is a Chief Engineer at a large ISP and he and his teammates code and config their custom Carrier Grade email sever Farm.

They have a 60PB mail store. He says 93% of all email that arrives is SPAM that is summarily discarded after being scored by their third party service similar to BrightWorks. We’re talking emails that NEVER hit your Junk Folder, just drained into the bit bucket. So if they did not stop these emails, their mail store would be ~1,000PB.

If your email doesn’t walk the line carefully, you have a 93% chance of it being vectored into space. If you want to reach @gmail @icloud @outlook etc you don’t have the skills to do it at scale. Delivering 100 emails is not the same thing as delivering 5,000.

So far, I used the rawest approach possible, by sending tens of thousands of mails via PHP's internal mail function.

When I send one to mail-tester.com, I get 9/10. Seems pretty good?

I don't see why Google should care for technicalities. They can see the behavior of their users. The behavior should tell them that the mails are not spam.

(comment deleted)
Klaviyo or Mailchimp? They are both easy to work with. But a little bit pricey. With that amount of emails, I think it would be around 150$/month.

You can save a lot of time with email delivery issues.

Mailchimp is the big one here because of its crm-lite features, automations, drips, etc. -- if you want to be able to take actions based on behaviors over time (it doesn't have to be nefarious, like you can send more detailed follow-ups to a subset of your readers who are especially interested in a niche).

If your newsletters each standalone and you don't really need marketing and CRM features, then there is a generic class of service called "transactional emails". There's sendgrid and a few other big players, still very cheap but with good delivery. The difference is that they're much less feature packed, but if you just want to get a bunch of emails send to a bunch of people with minimal fuss, that's a good way to do it. (they're called transactional because they allow customizations per message too. Like an order receipt. But you can also use them to mass mail newsletters)

Mailchimp seems crazy expensive. For 30k contacts, they want $300/month. Jesus!

I'm not selling anything. I will just let people know from time to time about my latest ideas, experiments and new cool stuff I'm working on.

Also, I can't find how I can trigger "Process bounces" in Mailchimp, so it reads the bounces from my inbox and marks them as bounced in my list.

The more I think about it, the more I tend to use some open source solution and host it myself.

Unless you've successfully administered large outgoing email pools before, I'd advise against doing it yourself. Unlike much of tech, it's hard to undo email mistakes because most deliverability is controlled by a few big players who are very quick to blacklist domains and senders that look like spam, and very slow to process appeals. If you screw something up, your domain will keep a bad reputation for a long time and that would affect not just your newsletters but all emails.

You can try it, but just fully understand the risks before you do. Otherwise choose a bigger cheap provider. It doesn't have to be Mailchimp.

Aren't domains a dime a dozen?

If I was afraid about my-main-domain.com (Not sure yet if I am) - I could just send from my-main-domain-mails.com, right?

I get the feeling this "UUUhhh! Black magic! Don't DIY with mails!" is fueled by marketing consultants who want to sell their services. I don't see why Google should care for technicalities. They can see the behavior of their users. The behavior should tell them that the mails are not spam.

Yeah, if you don't care about your domain name, it's less risky. Your readers may or may not want to open an email with a random domain, but if that's not a concern to you, go for it!

> I get the feeling this "UUUhhh! Black magic! Don't DIY with mails!" is fueled by marketing consultants who want to sell their services.

I think that's true, but in fact it's quite a bit worse than that... the email deliverability issues exist essentially because email has become a cartel of big providers who can collude on anti-spam schemes that make it very difficult for small players to break through successfully. They come up with both standards-based domain configurations (which in and of themselves are hard enough to get right) and out-of-band reputational databases that they are largely opaque, yet frequently shared, and with very limited & slow appeals. And depending on the recipients, if too many customers are on the same hosts, that intermediary host may throttle or block you if you send too many emails to them at once, regardless of the content. If you're a small fry ISP or random VM on the internet, you're totally at the mercy of these big players with no real recourse. They just don't care if their spam filters screw up. (And I dunno, maybe there's a pro-user side of this in that probably many people would rather their inboxes stay cleaner, risking the occasional false positive?

> I don't see why Google should care for technicalities. They can see the behavior of their users. The behavior should tell them that the mails are not spam.

It's not just Google (and even then not just personal Gmail), like depending on the makeup of your customer list, it could also include Google Workspace (business accounts that can set their own spam rules), Microsoft, Yahoo, Zoho, etc., and then tens of thousands of ISPs who make their own rules.

There's a lot of "it would be nice if" in email, but it's an arms race that got this way after decades of uncontrolled spam. I have no vested interest in any email service, but I often had to fix deliverability problems for small to medium businesses who mistakenly thought email newsletters should be simple... but they aren't today, because of the deliverability cartel. After doing that a few times and spending many days & weeks fighting these systems, I swore to myself I'd never make that mistake again. The micro-cents per email just weren't worth losing my sanity over.

If they decide your IP address is a source of spam you will have a problem. And if you are using the cloud, you may end up being assigned an IP address previously identified as belonging to a spammer.
>PHP's built-in mailer We don't need any external service for this?
I used the mail server provided by my domain registrar / hoster. It is basically free.
If you like selfhosting, then self-host the CRM and data-management side, but outsource the mail transmission.

You can selfhost MailWizz ($59, as I recall, on Code Canyon) or use Mautic (free) or Listmonk (ditto). Transmit via Amazon Simple Email Service, which will run you $1 per 10k emails. Of the three, I found that MailWizz hit the sweet spot for me (~30k subscribers, emailing them 2-5x/mo.), but your mileage may vary.

Why not just send them via the mail servers of my domain registrar / hoster, just like I did with the "Thanks for signup up" mails?
Most (all?) of the major hosting platforms will throttle you on a per-hour or per-24-hour basis. If you can trickle each newsletter out over several days, you could go that route.
Because, to pick just one example from many, Gmail will automatically route your email to the spam folder of your @gmail.com recipient.

There are many hoops you have to jump through, and you don’t yet know what they are. So, read up on it and drive in. You’ll learn soon enough. It’s not impossible, and your newsletters aren’t business critical, if they get lost.

BTW, “I wish” and “I would think” don’t work with email in the 21st century, as others have explained.

P.S. you might just get a $50 Fastmail account and send via their servers. But, be sure to give your readers a way to unsubscribe, and be diligent about keeping the unsubscribe actions current. FM has generous limits on amount and frequency.

I highly recommend https://sendy.co/. It's not open source, but you can host it yourself. In case you need it, you can have multiple brands under the same installation. It's like a self hosted mailchimp.
It sounds like you don't want to use a service. Take a look at GNU Mailman, it's mailing list software, which should do all the basics for you.

Importing your list may not be super straight forward, and you'll need to manually process your current inbox of bounces before you continue, but if you do it right future adds would be easy, and future bounce processing should be automatic.

You could try our platform stck.me (Samir.stck.me).

What is the best way to send out a newsletter now? I think I will need some service that I can: 1: Import the mails

Our import is smart and capable of - importing from almost any format. No matter which service they are coming from - handling duplicates and invalid addresses in the import

2: Import the bounces so they get analyzed and removed from the list.

We automagically cleanup the hard bounces. So the customer don't have to keep paying for such contacts.

The pricing is 10 percent of revenue.