Network Outage in EU affecting AMS1 and AMS2 (digitaloceanstatus.com)

16 points by ibarrajo ↗ HN
From Digital Ocean:

At this time we're experience a network outage in our AMS1 and AMS2 datacenters. With secondary impact affecting customers in Europe. As a result, you may experience latency, connectivity issues, or slow pings.

Preliminary investigation indicates that Telias Transatlantic cable are down. We are working to resolve the issue and apologize for any interruption this causes for you. Mon May 19 20:29:32 2014

16 comments

[ 3.9 ms ] story [ 44.9 ms ] thread
As a side effect, anybody else getting intermittent false alarms from Pingdom?
You're not the only one.
I had this issue on the new beeper system, I switched back to the old system and have had no issues since.
yeah, got a bunch - guess they're not strictly speaking false, it's just not something most of us can do anything about...
It's not Digital Ocean's fault. It seems like CloudFlare, Twitter and Facebook's EU servers are down too..
I am in Europe, and I can reach Facebook's EU servers just all right :-).
(comment deleted)
(comment deleted)
A bit of tracerouting and some twitter searching[0] suggests that the issue likes with Telia.net. There are some unconfirmed suggestions that a cable was severed[1], which I'm not sure I believe yet but certainly any packet going via telia.net is having a bad time.

Edit: Even CloudFlare are reporting the cable cut[2] so... gosh.

[0] https://twitter.com/search?q=telia&src=typd

[1] https://twitter.com/CurseNetworkSys/status/46849347754249420...

[2] https://twitter.com/CloudFlare/status/468493714134822912

Transatlantic cable has been cut, lots of sites affected.
(comment deleted)
In Germany we felt it for about 15mins, now the routing is adjusted and we are back to normal.

This is a nice reminder that the Internet was designed to resist to such disturbances.

At PagerDuty we saw our alerting volume double between 19:50 and 20:00 UTC (about 90 minutes ago).
It looks like from our end that there was a large trans-atlantic issue which affected a lot of the connectivity between the US and EU and other providers were affected.

In our case we had to route around Telia which seemed to be the link that was having the most issues and we're getting more info from them to see what happened and when it will be safe to add them back into the mix.

Thanks, Moisey Cofounder DigitalOcean