The linked page "CIDR Report" shows a rrdtool graph that seems to show we had just as many routes 3 days ago. The page doesn't say anything about widespread issues, and I don't find any source for that when googling.
So, where is your source about these widespread issues?
(Link title at the time of this post: The Internet hit 512K BGP routes today, causing widespread issues)
NANOG[0] warned about it in May, and there is a blog post [1] that expands on the risk. Cisco 6500/7600 (if they are set to hold the complete BGP table) are known to be affected.
As I understand it we came close a few days ago but finally rolled over 512K today at which point some routers hit their hardware limits and either stop learning new routes or start forwarding in software.
There is a lot of chatter about it on Twitter and some network mailing lists:
Sure. Lots of old/older routers can't hold more than 512K IPv4 routes in their "really fast routing table" memory (TCAM). This means that they'll have to either drop some routes - meaning some places on the internet will not be accessible from those routers - or that they'll have to use slower memory to store those routes - which means routing in general will be slow across the board.
Many, many ISPs, hosting and content providers use routers with this limitation. So their customers are going to be impacted, in some indeterminable way.
(Network geeks - I've glossed over a point or two, I know. Caveat lector.)
Is this limitation specific to IPv4? Would those same routers be able to handle a larger quantity of IPv6 addresses (if any), or does this highlight the need to upgrade those routers?
Assuming a Cisco 6500/7200 model router - the one most commonly affected by this issue - there's a certain amount of TCAM available by default for IPv4 routing, IPv6 routes, and multicast. By default, IPv4 only gets 512K routes. You can re-allocate more space to IPv4, but there's a potential cost... given the current rate of IPv6 adoption though, you could say that cost would be negligible for most people.
(The question I have is, why are people with these routers taking a full BGP feed of 512K routes? I'd assume that 90% of the time a default route will do, or even just default to your transit + their direct peer routes...)
As a person of similar layness but some parts bit-wise...
I think the answer is no, plain and simple because IPv6 addresses are literally four times as large.
If we were using IPv6 for years leading up to this, would it still be an issue? Obviously not the same question, but IPv6 having much larger potential address space and more opportunities for abbreviation in expressions of "what is a netblock range" means for one thing that you don't always need as many network blocks, you can have larger network blocks where a smaller one would be used in IPv4 because of limits and efficiency with the overall available and provisioned blocks...
Disclaimer: I have never been in charge of a large router, let alone a large network.
IPv6 addresses are larger, but routers only store prefixes, not full addresses. And yes, one of the design goals of IPv6 is to reduce address fragmentation, allowing the global routing table to contain fewer and shorter prefixes.
That's only achievable in the long term, though - the currently assigned IPv4 prefixes will have to coexist with the IPv6 ones for a long time.
I was trying to say something like this; I don't think it's true that you don't ever store full addresses in a router (You don't always store the address, sometimes just a prefix), but when you are describing a destination for a block, you would need a full address to finger that gateway, wouldn't you?
Unless the gateway is always at the front of the address block, which I'm sure "they never taught us" in small network design school, at least I believe I've seen routes where the gateway was inside the netblock range or at the end rather than beginning, but again... big networks maybe.
What I was trying to say was that when you have the freedom to write bigger ranges by an exponent factor of four like you have if you're the governing bodies of the internet with IPv6 vs IPv4, the expressive part of your network number can be made shorter, at least until your new network begins to approach the size of your old network.
You don't avoid this problem at all, even if you are not storing 128 bits of address there are probably never scenarios where two internets of the same size last longer on the same hardware because IPv6. You're also right that you don't really ever have a scenario in the real world where we are on IPv6 and there is no need for IPv4, at least not for a very long time.
> I was trying to say something like this; I don't think it's true that you don't ever store full addresses in a router (You don't always store the address, sometimes just a prefix), but when you are describing a destination for a block, you would need a full address to finger that gateway, wouldn't you?
Oh, I see your point now. You mean the IPv6 addresses assigned to the nodes that the router forwards the packets through (i.e., the egress nodes), not the destination addresses for those packets.
> Unless the gateway is always at the front of the address block, which I'm sure "they never taught us" in small network design school, at least I believe I've seen routes where the gateway was inside the netblock range or at the end rather than beginning, but again... big networks maybe.
Sure, the gateway address almost never belongs to the address block you are forwarding to. You could be forwarding to a very distant network which isn't directly connected to, nor belongs to the same address block, as your gateway.
I can't tell for sure how routers implement this, but it's safe to say that the total number of egress nodes for a router is much, much smaller than the total number of routes (prefixes). So the obvious optimization is to store the egress node addresses in a lookup table, and only store an index into that table for each route. That way, the fact that you are now forwarding through IPv6- instead of IPv4-addressed gateways does not increase the routing table size (though it does increase the size of the much smaller lookup table).
> What I was trying to say was that when you have the freedom to write bigger ranges by an exponent factor of four like you have if you're the governing bodies of the internet with IPv6 vs IPv4, the expressive part of your network number can be made shorter, at least until your new network begins to approach the size of your old network.
I think you probably meant "at least until the used fraction of your new address space begins to approach the used fraction of your old address space".
> You don't avoid this problem at all, even if you are not storing 128 bits of address there are probably never scenarios where two internets of the same size last longer on the same hardware because IPv6.
I'm not sure I understand the last sentence. The sense in which the IPv6 network will "last longer" in the same hardware is that, for the foreseeable future, you will be able to satisfy most new allocations by choosing an address block that is adjacent to an already existing allocation for the requesting entity. Because the two blocks can then be aggregated into a single prefix, the routing table stays the same size.
12 comments
[ 0.88 ms ] story [ 21.1 ms ] threadSo, where is your source about these widespread issues?
(Link title at the time of this post: The Internet hit 512K BGP routes today, causing widespread issues)
[0] http://mailman.nanog.org/pipermail/nanog/2014-May/066880.htm...
[1] http://packetlife.net/blog/2014/may/6/global-ipv4-routing-ta...
There is a lot of chatter about it on Twitter and some network mailing lists:
https://puck.nether.net/pipermail/outages/2014-August/007091...
It looks like some providers like Level3 have been having a bad day.
Many, many ISPs, hosting and content providers use routers with this limitation. So their customers are going to be impacted, in some indeterminable way.
(Network geeks - I've glossed over a point or two, I know. Caveat lector.)
Assuming a Cisco 6500/7200 model router - the one most commonly affected by this issue - there's a certain amount of TCAM available by default for IPv4 routing, IPv6 routes, and multicast. By default, IPv4 only gets 512K routes. You can re-allocate more space to IPv4, but there's a potential cost... given the current rate of IPv6 adoption though, you could say that cost would be negligible for most people.
More details here: http://www.cisco.com/c/en/us/support/docs/switches/catalyst-...
(The question I have is, why are people with these routers taking a full BGP feed of 512K routes? I'd assume that 90% of the time a default route will do, or even just default to your transit + their direct peer routes...)
I think the answer is no, plain and simple because IPv6 addresses are literally four times as large.
If we were using IPv6 for years leading up to this, would it still be an issue? Obviously not the same question, but IPv6 having much larger potential address space and more opportunities for abbreviation in expressions of "what is a netblock range" means for one thing that you don't always need as many network blocks, you can have larger network blocks where a smaller one would be used in IPv4 because of limits and efficiency with the overall available and provisioned blocks...
Disclaimer: I have never been in charge of a large router, let alone a large network.
That's only achievable in the long term, though - the currently assigned IPv4 prefixes will have to coexist with the IPv6 ones for a long time.
Unless the gateway is always at the front of the address block, which I'm sure "they never taught us" in small network design school, at least I believe I've seen routes where the gateway was inside the netblock range or at the end rather than beginning, but again... big networks maybe.
What I was trying to say was that when you have the freedom to write bigger ranges by an exponent factor of four like you have if you're the governing bodies of the internet with IPv6 vs IPv4, the expressive part of your network number can be made shorter, at least until your new network begins to approach the size of your old network.
You don't avoid this problem at all, even if you are not storing 128 bits of address there are probably never scenarios where two internets of the same size last longer on the same hardware because IPv6. You're also right that you don't really ever have a scenario in the real world where we are on IPv6 and there is no need for IPv4, at least not for a very long time.
Oh, I see your point now. You mean the IPv6 addresses assigned to the nodes that the router forwards the packets through (i.e., the egress nodes), not the destination addresses for those packets.
> Unless the gateway is always at the front of the address block, which I'm sure "they never taught us" in small network design school, at least I believe I've seen routes where the gateway was inside the netblock range or at the end rather than beginning, but again... big networks maybe.
Sure, the gateway address almost never belongs to the address block you are forwarding to. You could be forwarding to a very distant network which isn't directly connected to, nor belongs to the same address block, as your gateway.
I can't tell for sure how routers implement this, but it's safe to say that the total number of egress nodes for a router is much, much smaller than the total number of routes (prefixes). So the obvious optimization is to store the egress node addresses in a lookup table, and only store an index into that table for each route. That way, the fact that you are now forwarding through IPv6- instead of IPv4-addressed gateways does not increase the routing table size (though it does increase the size of the much smaller lookup table).
> What I was trying to say was that when you have the freedom to write bigger ranges by an exponent factor of four like you have if you're the governing bodies of the internet with IPv6 vs IPv4, the expressive part of your network number can be made shorter, at least until your new network begins to approach the size of your old network.
I think you probably meant "at least until the used fraction of your new address space begins to approach the used fraction of your old address space".
> You don't avoid this problem at all, even if you are not storing 128 bits of address there are probably never scenarios where two internets of the same size last longer on the same hardware because IPv6.
I'm not sure I understand the last sentence. The sense in which the IPv6 network will "last longer" in the same hardware is that, for the foreseeable future, you will be able to satisfy most new allocations by choosing an address block that is adjacent to an already existing allocation for the requesting entity. Because the two blocks can then be aggregated into a single prefix, the routing table stays the same size.