> and also our feelings about Cloudflare attempting to build support in this manner, especially now, during the Corona Virus situation.
Weird angle. Unless the RPKI standard is somehow actively encouraging people to violate social distancing policies, I don't see any connection with Covid-19..
To me this whole article just reads like a network operator complaining that someone else is trying to hold them accountable.
It’s a small business. Their staff could be infected or furloughed, or worse.
In terms of our day to day lives it might feel like the proverbial month of Sundays right now, but for operations teams it’s more like an unending stream of Friday afternoons in terms of sensitivity to making big infrastructure changes.
Yeah, that was how I read it - the impact of getting this wrong is that you break the internet for your customers (and staff, if they're all or mostly WFH) at a time when they're potentially depending on it to eat (e.g. if you're in a vulnerable group and need to order food for delivery) or work.
We've known BGP's been vulnerable in this way for years, so it's a bit of a weird time to actively encourage people to publicly shame their ISPs for being "unsafe".
isbgpsafeyet.com only appeared at 4 p.m BST yesterday, a Friday [1]. It's the timing of that which I took the OP to be commenting on. The GGP mentioned that we're in a month of Friday afternoons, this page dropped literally towards the end of the working day on a Friday afternoon!
As you say, Cloudflare have been promoting RPKI for a couple of years now and it's disappointing that more of the big players haven't implemented it yet but is now the time?
For some, it's never the time they should do something. ISPs are notorious for dragging their feet and they'd just find new excuses if CF had delayed the publish.
I mean, the bigger ISPs will just ignore it like they've ignored IPv6 ¯\_(ツ)_/¯
On the other hand, AAISP started automatically assigning IPv6 addresses ~9 years ago, so you can hardly accuse them of dragging their feet. The OP was published on a Saturday, after all.
Maybe bad optics to do it right now but it needed to be released at some point. If they delayed it until we were at the tail end of the curve of Covid-19 infections, this blog could still rely on "we're still recovering from the pandemic" to support the "bad timing" argument.
While I am not a fan of some of Cloudflares actions over the years, they have been positive in the RPKI space for the last several years. They've hosted multiple meetings in their offices with some of the largest networks in the world to discuss RPKI strategy and deployment. They've opened sourced software to lower the bar for entry. Their staff was accomdating to other network operators when they rolled out Origin Validation to not black hole parts of the Internet and reached out to networks to let them know of the error to get it fixed. They, like the network I support have been impacted by some of the same hijacks and I share their frustration when major carriers are not only slow to deploy RPKI or have no plan at all (or even a plan to properly filter their customers: see Verizon). They've been a part of the fight along with other folks who are silent (but those who know, know them).
RPKI is no surprise. People have been beating on their upstreams for it for well over a year. Almost all Internet Exchanges have enabled BGP Origin Validation on their route servers (thanks to the efforts of folks like Job from NTT). It's about time we have a site like this that highlights the overall status of it. That said, there's more we can be doing here to provide metrics on RPKI adoption on the Internet.
My home ISP (RCN) also hasn't turned IPV6 on yet either. However, they turned on RPKI between when I tested IsBGPSafeYet.com in the morning and evening.
It doesn't only read like a asinine complaint, it actually is. "How dare someone highlight what we've left undone"
Saying things like "it's scaring our users", "others are not using it", "it's bad timing", "transit providers should be filtering", no actual non-emotional arguments why they aren't doing it and only shifting the responsibility to secure the internet. I'm too done with companies like that.
> To me this whole article just reads like a network operator complaining that someone else is trying to hold them accountable.
Not really though, they do agree in the post that something needs to be done, they just don't agree that RPKI is quite the right answer and that Cloudflare's fearmongering scaretactic is the right move to push for RPKI.
IMO it's easy to have an opinion on either side of the fence - based on what you've done (or not done). Cloudflare, for example, committed to RPKI very publicly in 2018 [0]. This article, by ThousandEyes, does a nice job of visualizing the problem [1], published in July of 2019. As I read the parent article to this thread it strikes me as a bit defensive - which smells of a lie of omission (not exactly the whole truth, but conveniently cherry picked). They do very little in the article to state two missing arguments: 1) their timeline to implement RPKI (they only state: "At this stage we are looking in to this. We want to be sure we take the right approach, some of which will involved asking our transit providers what they are doing about it.") and 2) the rationale for not being further along of protecting customers with regard to the topic of RPKI.
They also grab Coronovirus as a rationale for doing nothing right now:
"Since this has now happened a few times, we felt it worth giving some more information that may be useful to customers and others who've seen these tweets (either directed at us, or at other ISPs), explaining a bit about what BGP is and how RPKI can extend it, and also our feelings about Cloudflare attempting to build support in this manner, especially now, during the Corona Virus situation."
If you look at this NANOG thread [2] nobody is complaining about ATT announcing they have implemented RPKI. So is there a negative downside? No. Has CloudFlare pushed some carriers into an awkward position given they are showcasing the true state of carriers as it pertains to route security in BGP? Yes. Andrews & Arnold are trying to tell their customers that their safety is paramount. Yet, they don't have a timeline to address the problem that other carriers have spent considerable time implementing over the last couple years. So, while Andrews & Arnold may be a great ISP - are they above public disclosure of an area they need to improve? No.
I applaud CloudFlare for showing end users which carriers are not spending time and resources on doing their due diligence to protect their customers. Especially business customers who rely on their parent AS to operate their business safely. Andrews & Arnold's response is suspect at best given their subjective response to the "why" behind why they've chosen to do nothing.
Finally - beyond CloudFlare NIST has been publishing these statistics for much longer. Just because CloudFlare has shown light on the topic - does not mean they are the bad actor. There are plenty of other outlets that have been highly supportive of these deployments - NIST [3] and RIPE [4], among very vocal proponents.
So, after parsing the reality of the values of RPKI for a small amount of time - the question around why Andrews & Arnold have chosen to do nothing feels different and, in my opinion, even more appropriate. Beyond that their response feels very hollow and weak on the technicalities which have put them in a spotlight they'd rather not deal with right now.
Not really - at the moment all businesses are having to readjust efforts and work with less resources available.
They don’t want to jump into rash decisions with minimal staff or staff dispersed across home locations and not able to work as effectively as normal - which could lead to broken BGP routes.
Andrews & Arnold are the best UK ISP I've used in 20 years of being online.
As well as going above and beyond when it comes to sorting problems with BT, I appreciate that like with this post they are not afraid to stick their neck out and say what they think.
I have recently (within the last year) joined AAISP, and I am in agreement with you on this.
One thing it does show to me is that, even though the site was made available by Cloudflare only recently, my ISP has already acknowledged and responded to the post. Given that they have explained the situation, what it means, and that they are actively looking into it. The trust they built up with me since I started with them, means that I do trust that they are looking into this.
BGP as a whole does need a solution, and this may be one of a few ways to achieve this.
I am glad they are not rushing their solution so that my connection stays online while I have to stay isolated.
On the flip side, I do agree that some ISPs will drag their feet, and try to ignore everything for the sake of "money" or "can't be bothered" types of excuses. These are the ones that really should be pushed hard against to make the changes that are required. - I am just glad AAISP isn't one of them.
I agree. I've used them since ~2007 and though they may be a bit more expensive they provide a solid service and their tech support is damned fine. Prior to A&A I used Nildram but their service turned to crap after they were bought out by Tiscali.
Do not trust any centralised authority. But if PKI Involved, what are those and what is meant by court in the article? We have to be aware certain countries (China, Russia) may not be of best interest of humanity in mind.
I'd like to see what the potential issues with RPKI are (if there are any), this seems like a blog post that would address those but anything about the protocol isn't mentioned, outside of "it takes time to implement and we'd rather our upstream do it".
https://isbgpsafeyet.com/ is a bit misleading. They mention both IRR filtering and RPKI. Both of them are needed. RPKI currently only is used to validate the origin. When a peer only relies on RPKI, it is trivial to announce a route you are not authorized to announce: just append the legit ASN at the end of the AS path. That's what IRR filtering prevents.
Both are therefore useful. RPKI protects from mistake and mitigate a bit the ability to spoof a route (adding the legit ASN makes the AS path longer and less preferred).
This is snark but it's snarking up an interesting architectural issue, that being: "where do we draw the line?" Say we protect DNS (we could, with a better protocol design). But other redirection protocols aren't protected. Do we similarly authenticate OSPF? IP options? ARP?
The easy response to your snark is "no, people are just going to use DoH".
We've had decent authentication in OSPF for ... decades.
Most of the egregious problems with IP options have been worked around well, again for decades.
ARP is problematic, but only within a segment. Your attacker has to be close. For IPv6, we have SEND, but it is not without problems.
> The easy response to your snark is "no, people are just going to use DoH".
DoH doesn't prevent someone from poisoning well-trafficked DoH resolvers to redirect individual domains. DNSSEC does. I run it on my zones, and it's not so bad... About 40% of resolver traffic to my domains is verifying, too, so we're getting there...
DNSSEC does nothing to protect queries on the wire on their way to resolving servers (it collapses down to a single bit in the header), unlike DoH, which does protect those queries. Those queries are the most likely attack vector against DNS. Meanwhile, however many resolvers are validating queries to your own domains, virtually none of the most important (ie, well-trafficked) domains on the Internet are signed, or likely ever to be signed. At the same time, browsers have removed DNSSEC code. DNSSEC is moribund. DoH has already protected more people in the brief time that it has existed.
> DNSSEC does nothing to protect queries on the wire on their way to resolving servers (it collapses down to a single bit in the header), unlike DoH, which does protect those queries.
Yes, and DoH does nothing to protect queries between resolving servers and authoritative servers, unlike DNSSEC, which does protect those queries. Together, you end up with end to end protection.
That's true: DOH doesn't protect outbound communication from caches, until authority servers expose DOT or DOH endpoints for them to talk to --- which should happen, and is likely to happen long before DNSSEC. But DNS attacks almost always target endpoints (DNS attacks are rare, and heavily mitigated by the rest of the networking stack, which is why practically nobody deployed DNSSEC).
DOH resolvers don't recurse via HTTPS, and there are few/no authoritative DOH authorities. DOH just exposes much larger, more dangerous targets for poisoning at this point.
Something like 30% of domains in .COM have DNSSEC, in part because of things like Cloudflare making it so dang easy. Yes, the top domains / services tend not to, and this is unfortunate. But we're well on our way.
And the possibility for poisoning worsens other attacks (it's still trivial in many cases to steal email with DNS cache poisoning... 25 years after I first played with the idea).
There's no reason not to deploy DNSSEC today. The DOH resolvers you're enamored with are almost all validating, so it's a way to get end to end validation right now.
Nothing resembling 30% of the domains in .COM are DNSSEC-signed. The number is much closer to 1% (like, it may by now be 2%; it is not 4%). This is an easy stat to look up, but it's often presented misleadingly; you may need to look at whatever chart you're looking at knowing in advance how many domains are in .COM.
And, of course, that 1-2% is almost entirely a bunch of random tiny zones nobody cares about. You can quickly discover that for yourself by feeding any list of the Internet's most popular zones (the Moz 500 is an easy one to download) through a bash for-loop against "host -t ds".
There is no evidence of any cache-poisoning attacks on DoH servers that I'm aware of. DNS attacks are in general far rarer than you'd think given the attention they've received in the past (again: there isn't much you can do with a DNS cache-poisoning attack other than trying to get a certificate mis-issued, and CAs have multiple levels of defense against that attack).
It's interesting that you played with DNS cache poisoning 25 years ago. So did I! Small world. I wrote a QID-brute-forcing cache poisoner in late 1995 (to get onto IRC from funny domains with). Towards the end of 1996, I wrote the DNS cache poisoning checks for Secure Networks Ballista scanner, which did QID stuff a la Johannes Ulrich, and the DNS authority record caching tricks Kashpureff used, plus some others.
It's fun stuff! There's not much to do with it anymore, though. In 1995, reliable domain forgery would get you past rutils filters and NFS exports; it was a very big deal, so much so that people took the time to do crazy things like elaborate source-routing attacks. But just a year or two later, everything was SSH; a few years after that, and most sensitive things on the web were SSL (it wasn't good SSL, but DNS wasn't the low-hanging fruit).
It's been hard times for people who wanted to spend their careers protecting the DNS.
There are, of course, lots of good reasons not to want to deploy DNSSEC. A good starting point would be that it drastically harms the reliability of all your sites, because when DNSSEC fails, every site in your zone vanishes as if it was never there. That's what happened to HBO the week they rolled out HBO NOW.
But there are strong architectural arguments against it, too.
I end up saying a version of a bunch of this stuff any time people complain about DNSSEC not being available on this or that online service. I appreciate the opportunity to write a new and hopefully more interesting version of the comment. Thanks!
My Logs, for the last 30 days, domains validated with DNSSEC was 5.34%.
In general, it feels that ISPs like to be slow adopting standards, e.g. IPv6 it is still rolling.
Just look it up on dnssec-tools. As of March of this year, there were ~1.7MM signed domains in .COM. As of Q3 of last year, there were 144MM domains in .COM. Wherever you got that "30%" stat from, it was wildly off. Even I was overcounting.
And (I'm ninja-editing this in, so sorry if I caught you while replying), as you noted, much of the growth in .COM signings is attributable to Cloudflare auto-signing --- which is the same thing that is happening in places like .NL. But registrars auto-signing domains is security theater. Ironically: attacks against registrars are far more common than attacks on the DNS protocol itself; the recent spate of "DNS attacks" from the end of last year turned out to have been phishing attacks.
Yes, I slipped a zero. Still, your argument is: It's not adopted much, so we shouldn't adopt it--- plus it doesn't matter if naming isn't trustworthy? :P I don't find that a very good argument.
Your same argument seems to imply that since we have good TLS on many protocols, we shouldn't be too heartbroken about people stealing traffic with BGP, either.
So, that's actually not my argument. But, though you're trying to dunk on it, it is a solid argument. Here's why: the cryptography in DNSSEC is obviously outmoded (it's PKCS1v15 RSA signatures, most of them 1024-bit, with an outlandish hack that combines spoofing and a KDF to prevent zone enumeration). The protocol "supports" P-curve ECDSA signatures, and, of course, both the P-curves and the NIST DSA signing algorithm are themselves outmoded --- and that's the frontier of crypto in DNSSEC; Geoff Huston did studies of this and found a substantial fraction of deployed DNSSEC verifiers can't even handle that. The point being: it will take decades, literal decades, to get the baseline level of cryptographic competence we expect from secure messengers (to say nothing of modern signing crypto) into DNSSEC.
And every new deployment of 1990s-grade DNSSEC makes that problem harder.
If we started from scratch, the very first thing that would change would be the notion that cryptography is too expensive to support online signing, which is one of the two original sins of DNSSEC. The entire protocol would be drastically cleaner, easier to deploy, and, of course, much more secure.
This is not the core argument I have against DNSSEC (for that, just Google [against DNSSEC]). But it's a good one. What they say about PHP holds true for DNSSEC: it's a fractal of bad design.
I don't know what BGP has to do with any of this except to say that DNSSEC obviously doesn't do anything to prevent BGP hijacking attacks; attackers who control BGP control IP addresses, the things DNS maps in the first place.
I'm doing my best to keep these responses varied and interesting, because an HN search will show this is an argument I've made here a lot. Hopefully I'm doing an OK job of that.
> So, that's actually not my argument. But, though you're trying to dunk on it, it is a solid argument. Here's why: the cryptography in DNSSEC is obviously outmoded (it's PKCS1v15 RSA signatures, most of them 1024-bit,
Root zone has been signed with 2048 bit keys since 2016. Large numbers of domains are signed with 2048 bit keys, too.
> If we started from scratch, the very first thing that would change would be the notion that cryptography is too expensive to support online signing, which is one of the two original sins of DNSSEC.
Online signing still looks really expensive. Asking an authoritative server to do a 2048 bit signature per answer is costly, given the other conversation you've followed me into and argued basically the other side of. Especially since work you do to make DNS authoritative servers verify things is easily used for DOS, unlike work done for prefix verification in BGP).
> don't know what BGP has to do with any of this
Because we're in a thread talking about preventing BGP hijacking of prefixes with crypto, and the tangential discussion of preventing DNS hijacking of names with crypto (DNSSEC) came up. Some arguments you presented against caring about DNS security work equally well wrt: caring about BGP security--- who cares if someone intercepts our traffic, we're probably using TLS!
No. All modern DNSSEC implementations are online signers anyways, because the BCP mitigation for zone enumeration, "whitelies", depends on that. So what we in fact have is the worst of both worlds: a protocol that makes painful compromises to work in an offline-signer model, and all the operational concerns of online signers. A DNSSEC that was designed for online signers from the jump would be cleaner and more secure.
That the root zone was signed with 1024-bit RSA in 2016 is its own dunk, and I think you for making it for me, but of course the root zone isn't the only zone; the 1024-bit root just meant every TLD was capped at that level of security. Now they're not, but the rest of the DNS is still littered with them. See: every DNSSEC survey.
And, of course, you've addressed only a small part of that previous argument --- just the key sizes.
I don't know what point you're trying to make about BGP. I think you think I'm opposed to RPKI. I'm not.
We don't exactly need signatures that will stand forever to greatly mitigate DNS interception.
I agree DNSSEC isn't wonderful, but perfect is the enemy of good. I'd take 768 bit RSA over "doing nothing"-- it raises the difficulty level of a successful DNS attack enormously.
Some of us don't care about enumeration. If we do, there's whitelies, as you say, and eventually NSEC5 will be around.
> I don't know what point you're trying to make about BGP. I think you think I'm opposed to RPKI. I'm not.
You argued earlier that worrying so much about redirection isn't justified: "Say we protect DNS (we could, with a better protocol design). But other redirection protocols aren't protected. Do we similarly authenticate OSPF? IP options? ARP?"
In which case, maybe we should just ignore RPKI, too. :P And unfix source routing, not bother with SEND, etc.
If DNSSEC were free, rather than a many-tens-of-millions-of-dollars proposition, and if it were safe, rather than a demonstrated reliability nightmare, and if it was a well-contained solution to a specific problem, rather than a transparent attempt to bootstrap a government-controlled PKI, and if complicated new protocol deployments weren't a giant source of new bugs to exploit, sure. Sure, it would make sense to deploy a little bit of 768-bit RSA to fix some marginal problem the rest of the stack already heavily mitigates.
I don't think it's a many tens of millions of dollars proposition. It was about 45 minutes of my time to get it deployed on my zones, tested, automated, and then installed by my registrar.. and it's been reliable for 3 years since. I understand the tooling is better now, too, so some of the crap I fiddled with is probably not necessary anymore. And it's even mechanized entirely for you if you use Cloudflare DNS, etc.
No. TLS is a many-tens-of-millions proposition, demonstrably so, and TLS is easier than DNSSEC. You're intuiting global cost from a single extremely simple test case; I think you're mistaken both about the multiplicand and the multiplier.
Further evidence on this is easily obtained both from survey papers that collect DNSSEC misconfiguration statistics (they're hilarious) and from the IANIX collection; many of the organizations IANIX notes are extraordinarily clueful operators.
I'm right about this. It's feasible to get DNSSEC deployed globally (it would be a catastrophe, but we indeed have the tools to make that catastrophe happen). But it would be very, very expensive.
This has been a fun thread, but we are way into the right margin now and are probably the only two people reading it, so I suggest we tie it off at whatever your next response is.
If you're stating a global cost, make it explicit. Because most organizations can deploy DNSSEC cheaply, and who knows the global cost for total deployment of anything.
I have been an A&A customer for years and so this is disappointing because it's exactly the sort of crappy response I'd criticise other ISPs for. Hopefully in the not too distant future we can talk some sense into RevK and co.
The choice to block one route to make isbgpsafeyet apparently pass is bogus, it's like silencing a fire alarm (which is also something I see far too many businesses choosing to do, oh Zone A keeps alarming, we'll just disable it even though now that means more danger, engineer calls are too expensive)
Those familiar with my work will know that I read a lot of accident investigation reports, and a recurring theme in maritime accidents involving large vessels is that the BNWAS (a technology which alerts sleeping crew elsewhere on a vessel that for some reason nobody on the bridge is paying attention, probably because they're asleep) is "somehow" muted, with its logging disabled or even entirely switched off during an incident. That might not seem like a big deal but big ships are slow and so these incidents often take place over many minutes or hours, thus there is plenty of time for somebody to wake up, realise there's a problem, wake someone else up to help fix it, and fix the problem before you smash the ship into an island clearly marked on your chart.
If Cloudflare has any sense they'll change the bogus route up every week or two so that outfits like A&A keep getting these "annoying" alarms and concerns from customers never cease.
> This is marginally better than routing to somewhere else (some attacker) but it still means a black hole in the Internet.
No. It's the difference between an inconvenience (routing problem which can be reported and then corrected by a human) and a grave security problem (data is being sent to completely the wrong place because you didn't bother with the mechanism to prevent that).
Also, this glosses over the _very common_ scenario where the bogus routes are more specific than legitimate ones. If AA doesn't have a route to one specific /24 containing secure.bankofamerica.com that has "somehow" been advertised by a Taiwanese ISP for the past hour, that data will be bundled along with the rest of the /16 or whatever else (BoA's assignment is old and a mess) to the Akamai boxes (I think) which actually serve it anyway. Rather than a "black hole" there was no problem except for the bogus advertisement, which RPKI would have helped block.
I am not a fan of RPKI since it's centralized. I guess better than nothing, but what I find annoying is that IP protocol has the ability for source routing. So instead blinding trusting routes in BGP, clients/nodes who knows the route could avoid BGP entirely. It also allows the internet to be more resistant to censorship. Yet network operators seem to hate the idea of source routing because someone can spoof an ip address. So they just disable it so they don't have deal with it. Also smaller network operators seem to disable it as well to make firewall rules simpler
IPv6 implements source routing too, but again is hardly used. However it does include an other standard that also works like source routing for mobile networks, but is no where as powerful.
49 comments
[ 3.5 ms ] story [ 93.2 ms ] threadWeird angle. Unless the RPKI standard is somehow actively encouraging people to violate social distancing policies, I don't see any connection with Covid-19..
To me this whole article just reads like a network operator complaining that someone else is trying to hold them accountable.
In terms of our day to day lives it might feel like the proverbial month of Sundays right now, but for operations teams it’s more like an unending stream of Friday afternoons in terms of sensitivity to making big infrastructure changes.
We've known BGP's been vulnerable in this way for years, so it's a bit of a weird time to actively encourage people to publicly shame their ISPs for being "unsafe".
As you say, Cloudflare have been promoting RPKI for a couple of years now and it's disappointing that more of the big players haven't implemented it yet but is now the time?
1: https://blog.cloudflare.com/is-bgp-safe-yet-rpki-routing-sec...
> 17/04/2020, 4:00:00 pm BST
> Today, we are releasing isBGPSafeYet.com, a website to track deployments and filtering of invalid routes by the major networks.
On the other hand, AAISP started automatically assigning IPv6 addresses ~9 years ago, so you can hardly accuse them of dragging their feet. The OP was published on a Saturday, after all.
RPKI is no surprise. People have been beating on their upstreams for it for well over a year. Almost all Internet Exchanges have enabled BGP Origin Validation on their route servers (thanks to the efforts of folks like Job from NTT). It's about time we have a site like this that highlights the overall status of it. That said, there's more we can be doing here to provide metrics on RPKI adoption on the Internet.
My home ISP hasn't deployed IPv6 yet. Though, if they cited COVID-19 as a contributing factor when asked about it, I wouldn't be stunned...
This will allow them to pass Cloudflare's test page, without actually validating and rejecting invalid routes or even necessarily signing their own.
Saying things like "it's scaring our users", "others are not using it", "it's bad timing", "transit providers should be filtering", no actual non-emotional arguments why they aren't doing it and only shifting the responsibility to secure the internet. I'm too done with companies like that.
Not really though, they do agree in the post that something needs to be done, they just don't agree that RPKI is quite the right answer and that Cloudflare's fearmongering scaretactic is the right move to push for RPKI.
They also grab Coronovirus as a rationale for doing nothing right now:
"Since this has now happened a few times, we felt it worth giving some more information that may be useful to customers and others who've seen these tweets (either directed at us, or at other ISPs), explaining a bit about what BGP is and how RPKI can extend it, and also our feelings about Cloudflare attempting to build support in this manner, especially now, during the Corona Virus situation."
If you look at this NANOG thread [2] nobody is complaining about ATT announcing they have implemented RPKI. So is there a negative downside? No. Has CloudFlare pushed some carriers into an awkward position given they are showcasing the true state of carriers as it pertains to route security in BGP? Yes. Andrews & Arnold are trying to tell their customers that their safety is paramount. Yet, they don't have a timeline to address the problem that other carriers have spent considerable time implementing over the last couple years. So, while Andrews & Arnold may be a great ISP - are they above public disclosure of an area they need to improve? No.
I applaud CloudFlare for showing end users which carriers are not spending time and resources on doing their due diligence to protect their customers. Especially business customers who rely on their parent AS to operate their business safely. Andrews & Arnold's response is suspect at best given their subjective response to the "why" behind why they've chosen to do nothing.
Finally - beyond CloudFlare NIST has been publishing these statistics for much longer. Just because CloudFlare has shown light on the topic - does not mean they are the bad actor. There are plenty of other outlets that have been highly supportive of these deployments - NIST [3] and RIPE [4], among very vocal proponents.
So, after parsing the reality of the values of RPKI for a small amount of time - the question around why Andrews & Arnold have chosen to do nothing feels different and, in my opinion, even more appropriate. Beyond that their response feels very hollow and weak on the technicalities which have put them in a spotlight they'd rather not deal with right now.
[0] https://blog.cloudflare.com/rpki/ [1] https://blog.thousandeyes.com/visualizing-the-benefits-of-rp... [2] https://mailman.nanog.org/pipermail/nanog/2019-February/thre... [3] https://rpki-monitor.antd.nist.gov/#rpki_adopters [4] https://labs.ripe.net/Membe...
They don’t want to jump into rash decisions with minimal staff or staff dispersed across home locations and not able to work as effectively as normal - which could lead to broken BGP routes.
As well as going above and beyond when it comes to sorting problems with BT, I appreciate that like with this post they are not afraid to stick their neck out and say what they think.
One thing it does show to me is that, even though the site was made available by Cloudflare only recently, my ISP has already acknowledged and responded to the post. Given that they have explained the situation, what it means, and that they are actively looking into it. The trust they built up with me since I started with them, means that I do trust that they are looking into this.
BGP as a whole does need a solution, and this may be one of a few ways to achieve this.
I am glad they are not rushing their solution so that my connection stays online while I have to stay isolated.
On the flip side, I do agree that some ISPs will drag their feet, and try to ignore everything for the sake of "money" or "can't be bothered" types of excuses. These are the ones that really should be pushed hard against to make the changes that are required. - I am just glad AAISP isn't one of them.
Both are therefore useful. RPKI protects from mistake and mitigate a bit the ability to spoof a route (adding the legit ASN makes the AS path longer and less preferred).
https://coinmarket01.blogspot.com/2020/04/what-is-iq-option-...
The easy response to your snark is "no, people are just going to use DoH".
Most of the egregious problems with IP options have been worked around well, again for decades.
ARP is problematic, but only within a segment. Your attacker has to be close. For IPv6, we have SEND, but it is not without problems.
> The easy response to your snark is "no, people are just going to use DoH".
DoH doesn't prevent someone from poisoning well-trafficked DoH resolvers to redirect individual domains. DNSSEC does. I run it on my zones, and it's not so bad... About 40% of resolver traffic to my domains is verifying, too, so we're getting there...
Yes, and DoH does nothing to protect queries between resolving servers and authoritative servers, unlike DNSSEC, which does protect those queries. Together, you end up with end to end protection.
Something like 30% of domains in .COM have DNSSEC, in part because of things like Cloudflare making it so dang easy. Yes, the top domains / services tend not to, and this is unfortunate. But we're well on our way.
And the possibility for poisoning worsens other attacks (it's still trivial in many cases to steal email with DNS cache poisoning... 25 years after I first played with the idea).
There's no reason not to deploy DNSSEC today. The DOH resolvers you're enamored with are almost all validating, so it's a way to get end to end validation right now.
And, of course, that 1-2% is almost entirely a bunch of random tiny zones nobody cares about. You can quickly discover that for yourself by feeding any list of the Internet's most popular zones (the Moz 500 is an easy one to download) through a bash for-loop against "host -t ds".
There is no evidence of any cache-poisoning attacks on DoH servers that I'm aware of. DNS attacks are in general far rarer than you'd think given the attention they've received in the past (again: there isn't much you can do with a DNS cache-poisoning attack other than trying to get a certificate mis-issued, and CAs have multiple levels of defense against that attack).
It's interesting that you played with DNS cache poisoning 25 years ago. So did I! Small world. I wrote a QID-brute-forcing cache poisoner in late 1995 (to get onto IRC from funny domains with). Towards the end of 1996, I wrote the DNS cache poisoning checks for Secure Networks Ballista scanner, which did QID stuff a la Johannes Ulrich, and the DNS authority record caching tricks Kashpureff used, plus some others.
It's fun stuff! There's not much to do with it anymore, though. In 1995, reliable domain forgery would get you past rutils filters and NFS exports; it was a very big deal, so much so that people took the time to do crazy things like elaborate source-routing attacks. But just a year or two later, everything was SSH; a few years after that, and most sensitive things on the web were SSL (it wasn't good SSL, but DNS wasn't the low-hanging fruit).
It's been hard times for people who wanted to spend their careers protecting the DNS.
There are, of course, lots of good reasons not to want to deploy DNSSEC. A good starting point would be that it drastically harms the reliability of all your sites, because when DNSSEC fails, every site in your zone vanishes as if it was never there. That's what happened to HBO the week they rolled out HBO NOW.
But there are strong architectural arguments against it, too.
I end up saying a version of a bunch of this stuff any time people complain about DNSSEC not being available on this or that online service. I appreciate the opportunity to write a new and hopefully more interesting version of the comment. Thanks!
And (I'm ninja-editing this in, so sorry if I caught you while replying), as you noted, much of the growth in .COM signings is attributable to Cloudflare auto-signing --- which is the same thing that is happening in places like .NL. But registrars auto-signing domains is security theater. Ironically: attacks against registrars are far more common than attacks on the DNS protocol itself; the recent spate of "DNS attacks" from the end of last year turned out to have been phishing attacks.
Your same argument seems to imply that since we have good TLS on many protocols, we shouldn't be too heartbroken about people stealing traffic with BGP, either.
And every new deployment of 1990s-grade DNSSEC makes that problem harder.
If we started from scratch, the very first thing that would change would be the notion that cryptography is too expensive to support online signing, which is one of the two original sins of DNSSEC. The entire protocol would be drastically cleaner, easier to deploy, and, of course, much more secure.
This is not the core argument I have against DNSSEC (for that, just Google [against DNSSEC]). But it's a good one. What they say about PHP holds true for DNSSEC: it's a fractal of bad design.
I don't know what BGP has to do with any of this except to say that DNSSEC obviously doesn't do anything to prevent BGP hijacking attacks; attackers who control BGP control IP addresses, the things DNS maps in the first place.
I'm doing my best to keep these responses varied and interesting, because an HN search will show this is an argument I've made here a lot. Hopefully I'm doing an OK job of that.
Root zone has been signed with 2048 bit keys since 2016. Large numbers of domains are signed with 2048 bit keys, too.
> If we started from scratch, the very first thing that would change would be the notion that cryptography is too expensive to support online signing, which is one of the two original sins of DNSSEC.
Online signing still looks really expensive. Asking an authoritative server to do a 2048 bit signature per answer is costly, given the other conversation you've followed me into and argued basically the other side of. Especially since work you do to make DNS authoritative servers verify things is easily used for DOS, unlike work done for prefix verification in BGP).
> don't know what BGP has to do with any of this
Because we're in a thread talking about preventing BGP hijacking of prefixes with crypto, and the tangential discussion of preventing DNS hijacking of names with crypto (DNSSEC) came up. Some arguments you presented against caring about DNS security work equally well wrt: caring about BGP security--- who cares if someone intercepts our traffic, we're probably using TLS!
That the root zone was signed with 1024-bit RSA in 2016 is its own dunk, and I think you for making it for me, but of course the root zone isn't the only zone; the 1024-bit root just meant every TLD was capped at that level of security. Now they're not, but the rest of the DNS is still littered with them. See: every DNSSEC survey.
And, of course, you've addressed only a small part of that previous argument --- just the key sizes.
I don't know what point you're trying to make about BGP. I think you think I'm opposed to RPKI. I'm not.
I agree DNSSEC isn't wonderful, but perfect is the enemy of good. I'd take 768 bit RSA over "doing nothing"-- it raises the difficulty level of a successful DNS attack enormously.
Some of us don't care about enumeration. If we do, there's whitelies, as you say, and eventually NSEC5 will be around.
> I don't know what point you're trying to make about BGP. I think you think I'm opposed to RPKI. I'm not.
You argued earlier that worrying so much about redirection isn't justified: "Say we protect DNS (we could, with a better protocol design). But other redirection protocols aren't protected. Do we similarly authenticate OSPF? IP options? ARP?"
In which case, maybe we should just ignore RPKI, too. :P And unfix source routing, not bother with SEND, etc.
Further evidence on this is easily obtained both from survey papers that collect DNSSEC misconfiguration statistics (they're hilarious) and from the IANIX collection; many of the organizations IANIX notes are extraordinarily clueful operators.
I'm right about this. It's feasible to get DNSSEC deployed globally (it would be a catastrophe, but we indeed have the tools to make that catastrophe happen). But it would be very, very expensive.
This has been a fun thread, but we are way into the right margin now and are probably the only two people reading it, so I suggest we tie it off at whatever your next response is.
The choice to block one route to make isbgpsafeyet apparently pass is bogus, it's like silencing a fire alarm (which is also something I see far too many businesses choosing to do, oh Zone A keeps alarming, we'll just disable it even though now that means more danger, engineer calls are too expensive)
Those familiar with my work will know that I read a lot of accident investigation reports, and a recurring theme in maritime accidents involving large vessels is that the BNWAS (a technology which alerts sleeping crew elsewhere on a vessel that for some reason nobody on the bridge is paying attention, probably because they're asleep) is "somehow" muted, with its logging disabled or even entirely switched off during an incident. That might not seem like a big deal but big ships are slow and so these incidents often take place over many minutes or hours, thus there is plenty of time for somebody to wake up, realise there's a problem, wake someone else up to help fix it, and fix the problem before you smash the ship into an island clearly marked on your chart.
If Cloudflare has any sense they'll change the bogus route up every week or two so that outfits like A&A keep getting these "annoying" alarms and concerns from customers never cease.
> This is marginally better than routing to somewhere else (some attacker) but it still means a black hole in the Internet.
No. It's the difference between an inconvenience (routing problem which can be reported and then corrected by a human) and a grave security problem (data is being sent to completely the wrong place because you didn't bother with the mechanism to prevent that).
Also, this glosses over the _very common_ scenario where the bogus routes are more specific than legitimate ones. If AA doesn't have a route to one specific /24 containing secure.bankofamerica.com that has "somehow" been advertised by a Taiwanese ISP for the past hour, that data will be bundled along with the rest of the /16 or whatever else (BoA's assignment is old and a mess) to the Akamai boxes (I think) which actually serve it anyway. Rather than a "black hole" there was no problem except for the bogus advertisement, which RPKI would have helped block.
IPv6 implements source routing too, but again is hardly used. However it does include an other standard that also works like source routing for mobile networks, but is no where as powerful.