Experienced uncharacteristic connection issues in the last ~15 minutes, was primarily SSHing into AWS EC2 at the time though so can’t comment on the true scale.
That cloudflare status is very strange: it says (said) Angola, Dubai, Philadelphia (US), some Australian locations and some in South America were affected. Anyone have a good explanation of such a random distribution?
Telia, the ISP that cause this issue, is very very large -- one of few Tier 1 ISPs. That said, when Tier 1 ISPs experience issues they're incredibly wide spread and behave oddly.
The POPs you mention are the Cloudflare POPs that experienced degraded or dropped throughput due to Telia being (one of?) Cloudflare's preferred routes for that POP.
I don't believe that would be possible... (Caveat: I am not the World's Smartest & Bestest Network Engineer Ever™, would be great if other's chimed in!)
1. Each Cloudflare POP has multiple bandwidth providers.
2. Each Cloudflare POP can and does change it's preferred BGP routes.
3. Telia's outage wasn't 100% of their network, so there are plenty of Telia links -- including ones to Cloudflare POPs -- that were not impacted.
No, I have no public sources I can share. Just relaying what our CDN's have shared with me. I also can't provide anything specific. Sorry, I know it's unreliable and rumor but it's all I can provide.
I've had Pingdom for about three months now. They have failed to alert on every outage we've had since I started INCLUDING the first "test outage" I created on day 1 to verify the monitoring was working. I am not impressed at all by their service and will probably be asking for a full refund depending on the outcome of my current trouble ticket with them. Also they've been deleting history of our outages after the fact; i.e. an outage occurs, they fail to alert but they do log it, and then several days later the outage is removed from their history. Pingdom is run by absolute clowns.
A BGP problem again? Or something else this time? It amazes me that there has not been an alternative to handle routing except full trust for everyone.
We dropped Telia as a carrier about 2 months ago. I've never seen a network with such stability issues; outages/slowdowns occurred on a weekly basis for us and tech support was useless. The only good thing about Telia is they did a great job with service credits. According to VP's there they have been going through major transitions in the last year but given their Tier-1 status, you would expect more stability.
29 comments
[ 2.5 ms ] story [ 67.3 ms ] threadOr at least they showed me error message that their CDNs were down.
Haven't seen that kind of error before in Reddit
> Network Performance Issues in multiple locations
> The issue is related to a specific transit provider and we are working on temporarily disabling this provider to route around the issue
[1] https://www.cloudflarestatus.com/?
[2] https://news.ycombinator.com/item?id=14246888
The POPs you mention are the Cloudflare POPs that experienced degraded or dropped throughput due to Telia being (one of?) Cloudflare's preferred routes for that POP.
1. Each Cloudflare POP has multiple bandwidth providers. 2. Each Cloudflare POP can and does change it's preferred BGP routes. 3. Telia's outage wasn't 100% of their network, so there are plenty of Telia links -- including ones to Cloudflare POPs -- that were not impacted.
Our CDN's are having problems all over the place. No indications of what shit the bed, but this is more then Cloudflare or Level3.
https://www.cloudflarestatus.com/incidents/51q3xhq8w7t8