17 comments

[ 4.8 ms ] story [ 56.1 ms ] thread
This sucks. Was on my way to the airport when my support team called me saying the app was down. But Pingdom didn’t alert me! Lots of users still having issues.

Trace route indicates a Telia interchange in ATL is causing the issue, at least for me.

I went home during lunch and am working from home this afternoon after the office went down, but it's not only Comcast having issues.

Level3: http://downdetector.com/status/level3/map/

Verizon: http://downdetector.com/status/verizon/map/

Spectrum: http://downdetector.com/status/spectrum/map/

ATT: http://downdetector.com/status/att/map/

My tinfoil hat says this is a state sponsored cyberattack, but this will probably come down to a scrub fat-fingering a config somewhere.

I feel like a state sponsored attack would coincide with some major event in the US. Is anyone aware of anything going on right now that fits that description?
Practice for the midterms?
Wonder how taking down network would affect the outcome of the midterms -- inducing chaos? sure; favoring one party than the other, not so sure.
Reducing overall turnout is very specific-party-favoring in the US.
never attribute to malice that which can be explained by one drunk idiot with a backhoe.

It's possible that several underlying carriers who actually own the dark (such as Zayo) have had the unfortunate conjunction of two cuts in two places, severing a great deal of diversity at the same time.

Best current info is as follows:

There is a fiber cut somewhere between NYC and Chicago.

At the same time there is also a fiber cut between Ashburn, VA, and locations in South Carolina. The cut is somewhere in NC.

Copy and paste from Level3 notification system:

"* CASCADED EXTERNAL NOTES 29-Jun-2018 15:13:37 GMT From CASE: 14815828 - Event Field Operations dispatched to the estimated failure location and upon arrival was advised by local law enforcement there was a truck that struck a utility pole, which then fell across the street and struck another pole, which then tore down the aerial fiber lines. In addition to the fiber lines being down, live power lines are down as well. OSP will be dispatching a fiber repair and construction crew to the site; however, until the local utility company has completed their repairs and deemed the area safe, fiber repairs cannot begin.

* CASCADED EXTERNAL NOTES 29-Jun-2018 19:21:05 GMT From CASE: 14815828 - Event The OSP has contacted the personnel transporting the replacement fiber, and they advised they remain en route, and on schedule for their ETA of 20:15 GMT. The next update will be provided upon their arrival, or as new information becomes available.

Experiencing it here. Seems like a DNS routing issue, as some website work while others don't.

Mobile hotspot is working just fine for all websites.

I happened to have a SSH session open all morning to my basement server, and I was getting things through SOCKS just fine, while everything else was reporting connectivity issues.