HN Explorer

Reliable Script Generated Email Delivery Anyone?

35 points by chrisconley ↗ HN
I'm ripping my hair out trying to reliably get emails through to my users for new invites, new messages, etc on my php app. For example, hotmail and yahoo aren't even letting the emails through to junk folders; the emails just disappear.

Has anyone had any experience with a trusted email delivery service that would handle making sure my valid emails get through to my users?

17 comments

[ 4.0 ms ] story [ 86.7 ms ] thread
ivan
Check the content of your e-mails. Are there links? Are there images? What is your domain name or sender name? What is the subject of your messages?
chrisconley
i've stripped html, images, etc, it's just text

i'm running php through suphp(i think?) so that emails are not sent from "nobody"

i've set up an spf record

i've had servint(where site is hosted) properly set up the A records(though i can't be totally positive that they know what they were doing)

the only thing i haven't done(that i know of) is gone through the domainkey process through yahoo

thanks again for any help!

dazzawazza
Are you sending the mail directly from PHP or are you sending to a mail relay you control (localhost)? I'd suspect your mail header are not correctly filled first. If you're not using a local relay your mail may be rejected?

Is there anything in any logs anywhere?

SwellJoe
Get right with RFC Jesus on your server.

Be sure DNS is correct. You need reverse resolution--it doesn't have to resolve to the name of your host, it just needs to resolve to something. A sane SPF record is necessary to deliver to some hosts. A DomainKey may help deliver to some, though its adoption is still quite low. There are tools on the web to check your DNS zone for you, to be sure things are sane (though some of them are overly critical of unimportant details...but if you get them all right, then you don't have to worry about which ones are important).

Be sure you are sending through a real MTA, rather than using something hacked together in PHP (or whatever language). Every variation from standards is another point or two in the spamminess of your message...Postfix, Sendmail, exim, and qmail can all be configured to strictly adhere to standards. Most scripted mail send tools cannot.

Look at your maillog when testing. You'll know immediately if something didn't make it to the server, and you'll probably know why. The post-processing that leads to "it disappeared" situations usually come from your message looking so spammy that it wasn't even worth saving. If you have a real mail server that can receive as well as send, then you will probably get a bounce. You need to be able to receive those bounces. This isn't optional.

dyu
How do you send a mail using a real MTA? If I set up Sendmail and use the mail function in python or something, is that the right way to do it?
SwellJoe
Do you see the message show up in the maillog/mail.log when you send that way? And, is it coming from a user that has a working mailbox (for bounces)? If both of those are true, then you're on the right track. Get DNS right, and your mail will probably work fine.
ojbyrne
If you have an email account somewhere with spamassassin, send some mails there, then look at the headers to see how SA scores you. Fix what's indicated and repeat. Not a 100% solution but it helps.
dzohrob
We ran into this, too, and have been dealing with it on some level for the past 6 months. It's probably not the content of your mail; this year, many major providers have gotten very aggressive with their anti-spam tactics, especially Microsoft.

1) You'll probably need to get a dedicated IP for sending mail if you're going to send at any volume; pretty much all the major operators calculate some sort of "score" based on your IP's record of delivery. The "score" includes: how many mails you send, how many are to "real" recipients (not deactivated users/nonexistent accounts), how many people complain about your mail, whether you not you hit any spam "trap" accounts, etc.

2) SwellJoe is right about a reverse DNS record that points to your mail server; it's basically required these days. A valid SPF record (http://openspf.org) is also basically required. You can also use DomainKeys or DKIM (http://dkim.org/) to improve your deliverability to Yahoo! accounts. If you're sending invites or other user-generated mail, read this in particular: http://www.openspf.org/Best_Practices/Webgenerated

3) Process your bounces. If you keep sending mail to a deactivated account, this looks very bad. Microsoft in particular has been super-aggressive about this. Set up a script that listens to incoming mail to your postmaster account.

4) Enroll in Microsoft's JMRP and SNDS programs (http://postmaster.msn.com/) -- there's a bunch of other stuff about Microsoft's practices there, too. SNDS lets you know how your mail is treated by the Hotmail servers, but you need to send at least 100 mails on any given day to get the data. JMRP forwards complaints from Hotmail users to you so that you can unsubscribe them.

5) This is the only guaranteed way to get into Hotmail --If you're serious about your site and you can cough up $400 to apply (and $1000 a year), Microsoft maintains a whitelist through a company called ReturnPath (http://returnpath.net) and a program called Sender Score Certified, which has some pretty intense requirements. The process can take a really long time, but we just went through it, and it's not too bad once you get the process going. (http://www.senderscorecertified.com/)

And finally, a bit of speculation -- I was doing some testing a few months back, and it seemed to help to have both an unsubscribe link and a postal address at the bottom of the mail, particularly for Yahoo!. Not sure if this is still true, but it might have something to do with CAN-SPAM filtering.

Hope that helps.

chrisconley
thanks for all of the help everyone; this is great stuff
bokonist
The easiest thing would be to sign up with a third party email service that has an API. They can handle all the deliverability issues, so you don't have to worry about it. I hear that Bronto is pretty good.
initself
How can a third party service handle mail sent addressed from a specific domain? I thought that was the issue this thread was trying to solve.
bokonist
Oh, and the other thing you can do, if you aren't sending tens of thousands of messages, is use Gmail to send your emails, and script it through a POP code library.
initself
Gmail has a 500 message limit as far as I know from tests.
Funky_
I've had this problem also.

What you need to do is sign up for sender id here:

https://support.msn.com/eform.aspx?productKey=senderid&p...

Make sure you have SPF installed on your smtp server. And also publish an SPF record on your domain. (TXT record)

If this dosen't work you need to fill out a form here:

https://support.msn.com/eform.aspx?productKey=edfsmsbl&c...

Microsoft will e-mail you back with a generic response. If they haven't helped you, you need to reply to this e-mail with more information about your problem. It might take a few e-mails back and forth with them but eventually I received this:

"I am pleased to inform you that we have taken steps to implement a temporary mitigation to your mail delivery problem. The mitigation will take 24-48 hours to fully reflect in our system."

"During the mitigation period your emails should not experience any issues arriving in Hotmail customer's inboxes. This period will also give our filters sufficient time to learn enough about your mailing practices that after the mitigation expires your mails shouldn't any issues with arriving in our Hotmail customer's inboxes."

So Microsofts spam filters basicly mess up and block all you e-mails. This is a microsoft issue and they need to fix their "SmartScreen" technology.

This worked for me and now I can send e-mail from my domain directly into the inboxes of hotmail users.

This is a BIG problem with hotmail and their spam technology and caused me a week long headache. I hope you have better results.

Funky_
Oh and one more thing. The funny things was I think what tipped MS off about my problem was that I was able to send e-mail though my smtp server on my domain to hotmail inboxes if and only if i used OUTLOOK from MS. The only difference was the e-mail headers!!

When I used outlook. 100% perfect. If I sent e-mail via thunderbird OR a php script my mail would never even reach the junk mail!! It would just black hole.

I compared the headers from outlook and thunderbird and my php script and the body was identical. Only the outlook e-mail had extra header information that MS refuses to identify or explain.