I really don't understand how people are afraid of email. It's no rocket science and smtp is a well documented and used standard.
Why would I lock myself into some API of a third party service (code I will have to change if I change services) instead of using practically any distributions default configuration of any SMTP server and then just talk SMTP.
If you own a static ip address not previously used for spam and if you have a valid PTR record, your mail will get delivered.
If you are in an ip block with a bad reputation (amazon ec2), you might need to sign up with an external provider but there are many more (and cheaper) smtp providers than there are json2mail services.
Of course if you need to track statistics, browsers, responses and bounces, it's a whole different problem which is solved by value-added services like Campaign Monitor or MadMimi and whatnot. But just sending mail?
Maybe I'm getting old. Back in my days, talking SMTP and setting up a mail server was nothing special at all.
(edit: just to reiterate: Even if you don't want to deal with the administrative issues, there is still the problem of the lock-in. Why not find a provider who does the work for you but speaks SMTP? That way, you can replace them if you need a different service level or if they go away. Using a proprietary API means that you can't easily change providers any more)
Have you ever tried configuring your own mail server? It's very close to black magic at times. Installing a mail server is easy, but configuring it to actually get mail into inboxes and not spam filtered or even blacklisted is complex. You could certainly do it yourself and supplement it with a galaxy of addons like you mentioned, but why bother when Sendgrid does all of it, through a simple smtp interface, and for pennies?
I used to configure my own mail server for scribophile.com. It took weeks to get it in a state I felt was acceptable, and even then it was through copy and paste without me truly understanding what was going on. When I had to move servers I said fuck it and went with Sendgrid, and I'm so glad I did! Instead of messing with DNS and domain keys and bounces and spam reports and user deliverability preferences I just dump a message to their smtp server and forget about it.
In addition to this, once I get a mail server running, I'm terrified to make changes lest incoming mail end up in a black hole. One time when I made an incorrect change, the backup mail server that godaddy "helpfully" put in my DNS settings started bouncing everything coming in. Even in the absence of a broken backup server, my experience has been that errors in the process are too frequent and take too long (on the order of minutes) to discover.
I have indeed configured my mail server. Maybe it's my bad choice in server software (Exim), but setting it up (also for receiving messages and exposing them using IMAP) was really not hard[1]
The Exim that comes per default in Ubuntu server has a default configuration that works as an SMTP server for a web application - no tweaks needed.
As I said: Your SMTP server needs two things:
1) You need an IP address in good standing. If you don't have that (if you are on EC2), you might need to sign up for an SMTP service (or sendgrid, but my concerns of using a nonstandard JSON API over the SMTP standard still stand).
2) If you have an IP address in good standing, you need a valid PTR record. You probably can't do that for yourself (unless you have an octet on your own or your provider goes really great lenghts), but all providers I have dealt with set this up for you no questions asked and without charging extra.
Then, getting the mail through through spam filters closer to the users is another problem, but Sendgrid wouldn't help you there either as this is largely dependent on the content of the email (and maybe a header or two like Message-ID).
1) http://www.pilif.ch/mail.txt is documenting the configuration. It's based on a newsgroup posting in 2001 - and it's in German.
This configuration plus the PTR record have lead to a configuration that is sending out 10000s of messages per day and we never had a blacklisting or spam filtering incident.
> 1) You need an IP address in good standing. If you don't have that (if you are on EC2), you might need to sign up for an SMTP service (or sendgrid, but my concerns of using a nonstandard JSON API over the SMTP standard still stand).
You only need the JSON API header if you want to use their "apps". If all you want is to send simple emails you can just use their SMTP server as a real SMTP server.
How true. Very recently I tried to setup SMTP for intranet and gave up 2 days after. There is just no simple enough guides. Most of them make assumptions of my knowledge, when I had 0 of DNS/SMTP servers.
I've spent many hours screwing with Exim on ded server I was using to do shared hosting. Every few weeks I had to worry about open relays and spam blacklists. I know if I spent more time I could learn how to stop those, but I would rather spend my time coding instead of admin'ing a mail server.
Of course there are people that can handle setting up and maintaining high volume mail servers with analytics, etc themselves, and have the time and resources to do it. If a capable sysadmin is outsourcing sysadmin duties... well that just doesn't make sense.
I could probably figure out how to rebuild a transmission, and I've even got the tools and an engine hoist to do it, but it's not exactly high on my list of things to do and I'd rather take it to my friend that does it for a living so I can focus on driving the car.
Managing your own email server can be quite a task. If you're sending millions of emails per day you probably need a dedicated team just for this.
Some companies will prefer to outsource the sending of commercial emails and to keep the transactional emails in house. This is exactly what my employer is doing.
We're currently evaluating SendGrid as provider and I'm quite impressed with their services. If you don't want all the vendor locking then you can just use the SMTP server. You don't need to worry about any configuration, they will manage your server and all you have to do is to send plain emails.
If you want more services and are not afraid of vendor lockin then you can simple enable their "apps" and you get link tracking, event notifications, google analytics support, open tracking, etc.
While I like SendGrid, there are a few things I'd like to note:
* It's extremely expensive compared to handling your own email. If you count dollars, what they deliver isn't really worth the cost (expontentially so if you already have your own dedicated sysadmin, which you will want to have anyway). Our app currently sends out around 800,000 notification emails per month, or about $600/mo.
* It's been a rocky road. It's been quite buggy for us. I think I had two support tickets within an hour of starting to use it, then another 3-4 tickets open within a week. Things like their email activity and graphs page not working (badly optimized and overloaded, basically); JSON mangling in their API output; and thousands of emails being stuck in their mail server for several hours before being delivered. Everything has been resolved, however, so right now it's fine.
* There are some stupid weaknesses. For example, it's not possible to turn off their built-in spam and bounce handling. If a recipient accidentally marks an email as spam (it happens more than you think), SendGrid will simply stop sending email to that user. Similarly, if a recipient's mailbox goes full or has a temporary failure, SendGrid will often hard bounce, and stop sending emails. You can set the expiry times for bounces, but you can't turn them off.
Pretty much the only feature that is worth paying for is their stats; they have nice APIs to get stats about deliveries, reads and clicks. So while we are ditching SendGrid for our notification emails, we will continue to use them for newsletters.
The spam reporting thing is indeed annoying, but it's likely a contractual obligation. Yahoo, Hotmail, etc agree to share data on who has clicked the spam button in exchange for a promise that that person will be immediately removed from all future messages.
Hm. I never bothered to check, but I just assumed/hoped that there was RFC somewhere that allowed providers to ping a URL (specified in a domain record or similar) when the user marked something as spam. Can't find anything, so you're probably right, SendGrid gets that information "under the table" from email providers.
You might be thinking of the List-Unsubscribe header which supposedly provides a machine-readable method of unsubscribing. But, no spam reports are all proprietary. And my guess is that they don't want actual spammers to get their hands on them lest they figure out how to tune their messages to avoid getting reported.
We have to use SendGrid at work and more often than not email gets delayed way beyond acceptable (about 1 hour, for example). It does have great features but if I could choose, I'd chose not to use it... at least not for everything.
17 comments
[ 3.1 ms ] story [ 44.9 ms ] threadHow do you send email from your webapp?
Choose One: mail server on my web server http://wedg.me/qdGcZg or Sendgrid http://wedg.me/rg65fD
Why would I lock myself into some API of a third party service (code I will have to change if I change services) instead of using practically any distributions default configuration of any SMTP server and then just talk SMTP.
If you own a static ip address not previously used for spam and if you have a valid PTR record, your mail will get delivered.
If you are in an ip block with a bad reputation (amazon ec2), you might need to sign up with an external provider but there are many more (and cheaper) smtp providers than there are json2mail services.
Of course if you need to track statistics, browsers, responses and bounces, it's a whole different problem which is solved by value-added services like Campaign Monitor or MadMimi and whatnot. But just sending mail?
Maybe I'm getting old. Back in my days, talking SMTP and setting up a mail server was nothing special at all.
(edit: just to reiterate: Even if you don't want to deal with the administrative issues, there is still the problem of the lock-in. Why not find a provider who does the work for you but speaks SMTP? That way, you can replace them if you need a different service level or if they go away. Using a proprietary API means that you can't easily change providers any more)
I used to configure my own mail server for scribophile.com. It took weeks to get it in a state I felt was acceptable, and even then it was through copy and paste without me truly understanding what was going on. When I had to move servers I said fuck it and went with Sendgrid, and I'm so glad I did! Instead of messing with DNS and domain keys and bounces and spam reports and user deliverability preferences I just dump a message to their smtp server and forget about it.
The Exim that comes per default in Ubuntu server has a default configuration that works as an SMTP server for a web application - no tweaks needed.
As I said: Your SMTP server needs two things:
1) You need an IP address in good standing. If you don't have that (if you are on EC2), you might need to sign up for an SMTP service (or sendgrid, but my concerns of using a nonstandard JSON API over the SMTP standard still stand).
2) If you have an IP address in good standing, you need a valid PTR record. You probably can't do that for yourself (unless you have an octet on your own or your provider goes really great lenghts), but all providers I have dealt with set this up for you no questions asked and without charging extra.
Then, getting the mail through through spam filters closer to the users is another problem, but Sendgrid wouldn't help you there either as this is largely dependent on the content of the email (and maybe a header or two like Message-ID).
1) http://www.pilif.ch/mail.txt is documenting the configuration. It's based on a newsgroup posting in 2001 - and it's in German.
Later on, Oliver Siegmar took my configuration and built http://www.xams.org/pages/home/home.php
This configuration plus the PTR record have lead to a configuration that is sending out 10000s of messages per day and we never had a blacklisting or spam filtering incident.
You only need the JSON API header if you want to use their "apps". If all you want is to send simple emails you can just use their SMTP server as a real SMTP server.
I could probably figure out how to rebuild a transmission, and I've even got the tools and an engine hoist to do it, but it's not exactly high on my list of things to do and I'd rather take it to my friend that does it for a living so I can focus on driving the car.
Some companies will prefer to outsource the sending of commercial emails and to keep the transactional emails in house. This is exactly what my employer is doing.
We're currently evaluating SendGrid as provider and I'm quite impressed with their services. If you don't want all the vendor locking then you can just use the SMTP server. You don't need to worry about any configuration, they will manage your server and all you have to do is to send plain emails.
If you want more services and are not afraid of vendor lockin then you can simple enable their "apps" and you get link tracking, event notifications, google analytics support, open tracking, etc.
* It's extremely expensive compared to handling your own email. If you count dollars, what they deliver isn't really worth the cost (expontentially so if you already have your own dedicated sysadmin, which you will want to have anyway). Our app currently sends out around 800,000 notification emails per month, or about $600/mo.
* It's been a rocky road. It's been quite buggy for us. I think I had two support tickets within an hour of starting to use it, then another 3-4 tickets open within a week. Things like their email activity and graphs page not working (badly optimized and overloaded, basically); JSON mangling in their API output; and thousands of emails being stuck in their mail server for several hours before being delivered. Everything has been resolved, however, so right now it's fine.
* There are some stupid weaknesses. For example, it's not possible to turn off their built-in spam and bounce handling. If a recipient accidentally marks an email as spam (it happens more than you think), SendGrid will simply stop sending email to that user. Similarly, if a recipient's mailbox goes full or has a temporary failure, SendGrid will often hard bounce, and stop sending emails. You can set the expiry times for bounces, but you can't turn them off.
Pretty much the only feature that is worth paying for is their stats; they have nice APIs to get stats about deliveries, reads and clicks. So while we are ditching SendGrid for our notification emails, we will continue to use them for newsletters.