Neat. I'd always filed the "CNAME screws with MX" away in my mind but I never knew the underlying reason beyond "because the RFC says so and it doesn't work IRL".
Yeah, neither did I, I just assumed it was just a rule no one followed.... As you can see I was wrong. I'm just glad I finally tracked down the issue before it drove me crazy.
Using AWS' A record ALIAS concept does away with this problem. It does mean you have to use Route 53 though.
Alternatively, use an ALIAS record for your apex (domain.com) that is an S3 bucket that redirects to www.domain.com (which you'd proxy through Cloudflare).
EDIT: The ALIASing that Amazon does are internal health checks that are abstracted away. In the event of a failure, Amazon will update the A record with a new functional IP route extremely quickly (or rather, that's how its supposed to work). Also, you aren't charged for ALIAS queries, while you are for CNAME queries (which can add up depending on your traffic).
Yes, I don't go into detail in the blog but I link to the AWS Blog where they explain how to do just that. I didn't know about the health checks that's quite nice!
It's not really a limitation, per se. It's intentional behavior. Most people just use CNAMEs to as a sort of "A record redirect", but really they're for all records. You're saying "the canonical name for foo.com is blah.bar.com", and that means for everything: A, MX, NS, SOA, etc.
Yes, Heroku was another "culprit" of this kind of behavior that I would guess many people fall prey to. The worst part of it is how you will get some/most emails and some will just fall through the cracks. Unless you have a case like mine where you know an email should be coming and it's not then you can easily not know this is happening and be dropping emails.
I'd love for CloudFlare to add the equivalent of an ALIAS record to their DNS service. To the best of my knowledge one cannot use Route 53 and also take advantage of CloudFlare's services.
The correct way todo this is setup a load balancer, then use route 53 to manage the dns, set the root domain to your load balancer I'D then map the www cname to the root domain, this solves all problems and takes under 5 minutes to configure.
Except if you add the MX records with a hostname of "www", they'll be used for emails to "your_name@www.example.org", which is generally not the desired behavior.
And that still doesn't solve the issue with SOA/NS records, which need to exist at the root and thus will still conflict with the CNAME.
Just verified that this is not the case with dns
records done through the domain manager at mediatemple.net
May be the case elsewhere (just no time right now to test this).
I would have started this reply by saying "not true" but
don't know if the particular dns manager at mediatemple.net is
setup in a way to be not compliant with the RFC.
Update: Upon further testing it does return a MX record but can't resolve the host name for the MX record.
It's really not a good idea. The issue isn't just whether mediatemple lets you do it... it's whether every DNS server that needs to talk to your domain understands it (and they don't).
The article points to an AWS article on how to set up the whole thing correctly - but once you've done that, there's still another problem: You really shouldn't host public stuff with S3 at all, unless you use CloudFront in front of it. Try benchmarking it with Apache Bench: S3 is extremely slow, whereas CloudFront is fast and can take a large number of hits.
As other people have noted, the solution to this is quite simple. Several DNS hosts support using an ALIAS record. The ALIAS record is then replaced with an A record when actually queried.
I'm surprised that so many of the comments in this thread are about dns hosting services and not dns server software support for these so-called "alias records".
I was making a list of "why on Earth doesn't this simple software exist?" items the other day and "simple server-side CNAME support" was on the list. Does any common DNS server package provide this support easily out of the box? It seems like one of the most common uses cases possible.
Small warning: CNAME at apex is commonly used for pointing traffic to a CDN or load balancing kind of service. These services tend to use DNS for part of their traffic orchestration (e.g. different DNS response based on user region) which gets less than optimal or even plain wrong results if your DNS server software resolves the CNAME instead of the end-user.
If they weren't using DNS as part of their strategy they could've given you a simple IP instead of a CNAME, saving your clients some lookup time.
www.domain.com and domain.com should not just silently duplicate the same content. If you want one to be "canonical", then the other should respond 301 Moved Permanently with the correct address. Duplicate content across domains is a mess for SEO.
I'm fairly certain that is fine, it's not "on the root" because it's *.root.com not root.com, you have an A record on the root which is expected and correct.
In my head I just think of 301'ing to the naked domain but that is only about web. What is the end goal of redirecting the domain with dns records like this? (Newbie)
I'm a little confused by why this was even an issue.
Why couldn't he just setup an A record for both domain.com and www.domain.com, and in his apache/nginx/pick your server config file 301 redirect www.domain.com to domain.com.
Yes it is and has been for a long time. What this article really boils down to is a terribly poor understanding of how DNS works and why certain records are use for certain serivces. A CNAME doesn't "redirect" anything at any level outside of a resolver. In the case of www.example.com and example.com you use a web server to achieve the desired result. Even then it's not a redirect, it's a replacement the resolver determines the record is a CNAME, inserts the value the CNAME points to, then resolves the A record. In the case of the MX record for a domain a CNAME is a problem because in the absence of a MX record, the A RR record is used, in this case a CNAME. Would protocol specific SRV records be better? Yes, but since that change isn't happening quickly do what's right now and change in the future.
Because he's not self-hosting his website. He's using a S3 bucket, and the "normal" way of doing that is to CNAME to a S3-owned hostname, so Amazon can then mess with the IP address behind the scenes without telling you to update your A record.
I've always used only one "A" record for main domain like "domain.com" then everything else goes throught "CNAME", like "www.domain.com" or any oher sub-domain like "blog.domain.com" then have sepeate "MX" record for email. In Nginx I redirect useless (www) to (non-www) using permanent redirect (301) Never had any problems both with CloudFlare and NameCheap ^_^
We should not be using A records for websites. I'm hoping HTTP/2.0 will drag itself into this century and commit to using SRV records or something very like them instead, so that problems like this can go away with a general solution.
Before Amazon introduced the ALIAS record the workaround I had in place was a cronjob which queried for the A records of our Amazon ELB and replicated those to the root of our domain, which at the time was hosted with DNS Made Easy. It was an easy decision to move to Route 53 once the ALIAS record was introduced and retire my hack.
Hi:
I'm the co-founder and CEO of CloudFlare. This can be a problem with some mail servers (especially Exchange, in our experience). Good news is we have a solution. We have a feature called CNAME Flattening which dynamically turns a CNAME at the root into an IP address. This solves the issue. We've beta tested this for the last few months and it works great. We haven't exposed in our UI yet but if you write to our support team we'll enable it. We're offering it to all our customers, even as the free tier. We plan to roll it out in our UI soon. Functionally, it's very similar to services that have an "Alias" record without requiring users to learn a new, non-standard DNS record type.
Matthew Prince (@eastdakota)
Co-founder & CEO, CloudFlare
The problem doesn't have much to do with particular mail servers, but rather with the order in which queries may be made. Let's say the following two records are allowed to co-exist:
example.com 60 IN MX mail.example.com
example.com 60 IN CNAME foo.example.org
and that the MX record takes precedence over the CNAME when queried. Imagine a client, C, behind resolver R, and it makes the following queries;
1. C looks up qname=example.com qtype=MX, R returns the MX record
2. C Looks up qname=example.com qtype=A, R follows the CNAME
3. C looks up qname=example.com qtype=MX, R follows the CNAME
The resolver will first resolve the MX record, and cache that correctly, passing the result back to the client. The second query though will result in a CNAME, which will likely invalidate the cached MX record, and the resolver will then follow the CNAME and chase that. The third query may now also follow the CNAME, even though there may not be an MX record at the CNAME target (foo.example.org).
Any non-MX query behind resolver R effectively poisons R's cache, and the MX becomes unreachable. If R gets no queries other than MX queries, or if the MX and CNAME target answers are cacheable for identical periods of time, then it can sometimes work - but it's a tenuous and non-deterministic stroke of luck. If there's even one lookup behind R - from any of its clients for a non-MX, things break again. Some mail servers do perform an A lookup before an MX lookup (which is valid RFC822) and so things never work for those mail servers, as they always poison the cache, but that correlation isn't really the cause of the problem. The configuration is unreliable even in the best of circumstances.
I don't understand what you mean by cache poisoning. If a resolver was queried for a single record type, that doesn't mean that's the only record that resolver will return... That would break all kinds of things. Usually DNS servers return referrals to other record types if they don't find the one they're looking for, but in your example the correct record type always exists, so it should always be returned and cached properly. The mail server may decide to error out if it finds a CNAME where it expected only A and MX records.
edit Oh, I see, this is a DNS spec issue, not an MTA or resolver issue. According to DNS specification clarification RFC2128, if a CNAME exists, no other record should exist with that name - so in theory a DNS server or resolver could simply drop all other records if it found a CNAME. (That seems like very bad behavior, though!)
10.1. CNAME resource records
The DNS CNAME ("canonical name") record exists to provide the
canonical name associated with an alias name. There may be only one
such canonical name for any one alias. That name should generally be
a name that exists elsewhere in the DNS, though there are some rare
applications for aliases with the accompanying canonical name
undefined in the DNS. An alias name (label of a CNAME record) may,
if DNSSEC is in use, have SIG, NXT, and KEY RRs, but may have no
other data. That is, for any label in the DNS (any domain name)
exactly one of the following is true:
+ one CNAME record exists, optionally accompanied by SIG, NXT, and
KEY RRs,
+ one or more records exist, none being CNAME records,
+ the name exists, but has no associated RRs of any type,
+ the name does not exist at all.
It is confusing, but the difference is that a CNAME is cacheable for queries all types. For example, let's say I query the resolver for qname=example.com qtype=AAAA. The resolver finds the CNAME from example.com to foo.example.org and caches that internally. It then descends a level in its recursion, and finds the answer for qname=foo.example.org qtype=AAAA.
But if the next query is say "qname=example.com qtype=MX", the CNAME is still cached and this knowledge can be used to skip straight to the qname=foo.example.org qtype=MX stage; there's no need to ask the nameservers for example.com anything, since R knows that that name is a CNAME to foo.example.org. But of course foo.example.org doesn't have an MX record, and so things break.
This is the why the order matters, and what I meant by cache poisoning.
So what you're saying is, resolvers will only look up and cache the one type they were asked to query, instead of caching all types associated with the label? While at the same time later dropping all record types if at some point it finally looks up and caches a CNAME? Sounds like a race condition to me!
In general resolvers look up one type and cache the result for that one type. CNAMEs are special; firstly clients never lookup CNAMEs, it's never the qtype. A CNAME is a special kind of answer that says "This name should be considered equivalent to this other name, go find out what that other name has" and as-such the CNAME is cacheable for queries of all types.
It is a race condition, but this race condition only exists if you ignore the DNS RFCs and try to get behavior that DNS just plain doesn't support. I don't mean to point fingers, but in this case that's what happened; CloudFlare's behavior ignores the RFCs [1] and leads to nondeterministic results.
Also it seems that some DNS resolvers with DNSSEC enabled will not resolve APEX on CNAME (returning a SERVFAIL). I've disabled DNSSEC on my home resolver (unbound) because of this (damn heroku).
>I started to look for solutions and was expecting to have to start using my domain with the "www" until I stumbled across this AWS Blog on how to do exactly what I was wanted. The only "problem" was I would have to use Route 53 instead of CloudFlare for it to work.
56 comments
[ 3.1 ms ] story [ 112 ms ] threadMuch less intuitive is the fact that a MX record cannot refer to a record that is a CNAME itself. http://www.exchangepedia.com/blog/2006/12/should-mx-record-p...
Many soho routers performing DNS forwarding give very weird and unpredictive results if you do this.
Alternatively, use an ALIAS record for your apex (domain.com) that is an S3 bucket that redirects to www.domain.com (which you'd proxy through Cloudflare).
EDIT: The ALIASing that Amazon does are internal health checks that are abstracted away. In the event of a failure, Amazon will update the A record with a new functional IP route extremely quickly (or rather, that's how its supposed to work). Also, you aren't charged for ALIAS queries, while you are for CNAME queries (which can add up depending on your traffic).
http://www.dnsmadeeasy.com/services/aname-records/
And that still doesn't solve the issue with SOA/NS records, which need to exist at the root and thus will still conflict with the CNAME.
May be the case elsewhere (just no time right now to test this).
I would have started this reply by saying "not true" but don't know if the particular dns manager at mediatemple.net is setup in a way to be not compliant with the RFC.
Update: Upon further testing it does return a MX record but can't resolve the host name for the MX record.
Not sure which other companies support it, but http://dnsimple.com does.
I was making a list of "why on Earth doesn't this simple software exist?" items the other day and "simple server-side CNAME support" was on the list. Does any common DNS server package provide this support easily out of the box? It seems like one of the most common uses cases possible.
If they weren't using DNS as part of their strategy they could've given you a simple IP instead of a CNAME, saving your clients some lookup time.
https://gist.github.com/dav-/eaa487032588efc0b4a3
Why couldn't he just setup an A record for both domain.com and www.domain.com, and in his apache/nginx/pick your server config file 301 redirect www.domain.com to domain.com.
Is that not the best practice?
See: https://github.com/http2/http2-spec/issues/381
Matthew Prince (@eastdakota) Co-founder & CEO, CloudFlare
root and WWW for shiftyplatypus.com are both CNAMEs (pointing at Heroku).
Any non-MX query behind resolver R effectively poisons R's cache, and the MX becomes unreachable. If R gets no queries other than MX queries, or if the MX and CNAME target answers are cacheable for identical periods of time, then it can sometimes work - but it's a tenuous and non-deterministic stroke of luck. If there's even one lookup behind R - from any of its clients for a non-MX, things break again. Some mail servers do perform an A lookup before an MX lookup (which is valid RFC822) and so things never work for those mail servers, as they always poison the cache, but that correlation isn't really the cause of the problem. The configuration is unreliable even in the best of circumstances.
Full disclosure: I work on Route 53.
edit Oh, I see, this is a DNS spec issue, not an MTA or resolver issue. According to DNS specification clarification RFC2128, if a CNAME exists, no other record should exist with that name - so in theory a DNS server or resolver could simply drop all other records if it found a CNAME. (That seems like very bad behavior, though!)
But if the next query is say "qname=example.com qtype=MX", the CNAME is still cached and this knowledge can be used to skip straight to the qname=foo.example.org qtype=MX stage; there's no need to ask the nameservers for example.com anything, since R knows that that name is a CNAME to foo.example.org. But of course foo.example.org doesn't have an MX record, and so things break.
This is the why the order matters, and what I meant by cache poisoning.
It is a race condition, but this race condition only exists if you ignore the DNS RFCs and try to get behavior that DNS just plain doesn't support. I don't mean to point fingers, but in this case that's what happened; CloudFlare's behavior ignores the RFCs [1] and leads to nondeterministic results.
[1] http://blog.cloudflare.com/zone-apex-naked-domain-root-domai...
This is why rewriting exists.