HN Explorer

24 comments

[ 2.8 ms ] story [ 65.8 ms ] thread
geerlingguy
Status page [1] not showing anything wrong currently, but I had my own VPS go offline a couple minutes ago, and now the login page for their Cloud UI is also offline with the message:

    {"error":"Looks like something went wrong. If this continues, please open a support ticket so we can help you out"}
Edit: And now all my NYC3 VPSes are going offline at 12:01 US Eastern (just got a bunch of outage emails from my monitoring service).

Edit 2: Cloud UI Login page briefly started working at 12:12 US Eastern, but 2FA login step is not completing. One of my servers is reachable, but about 5 others still aren't. Hopefully it's a sign things are coming back...

Edit 3: As of 12:16 US Eastern, all my NYC3 VPSes are back online, but Cloud UI login still failing at 2FA step.

Edit 4: As of 12:24 US Eastern, it seems everything is back up (for me), but they finally updated their status page with "Multiple Products Down".

[1] https://status.digitalocean.com

tech5000
Down for me as well. All sites completely offline and showing our CDN provider's Error 522 page on all customer-facing sites.

Same error as yours when I tried to log in and see what's up.

nnf
I'm experiencing the same thing. Can't reach my droplets, and can't sign in to see what's going on.

Edit [12:16pm EST]: my Droplets are reachable now, and my web apps are back, but I still can't access the control panel, and the status page hasn't been updated to show any problems.

geerlingguy
> the status page hasn't been updated to show any problems.

The only thing is 'Spaces' (which I'd never heard of until today), but I'm not sure how Spaces would affect the entire NYC3 data center (plus the cloud.digitalocean.com backend).

nnf
The Spaces (Digital Ocean's S3-like object storage service) outage was going on yesterday as well, so that particular part may be unrelated and ongoing.
bithavoc
Droplets stopped responding.

Edit: Droplets are back in NYC3

aosaigh
Just noticed the same. I'm still connected to my VPS over SSH, but HTTP traffic is timing out. Think it's US East only (my European sites are still OK).
james33
Looks like a widespread outage. We have servers in multiple regions that are all down and we can't even get into the DO dashboard.
soulchild37
Yes, I got noticed from uptime monitoring service, my SaaS are running solely on DigitalOcean ughhhh
marpstar
Same. Can't access cloud admin panel. I've got two sites, one on NYC1 and one on NYC3. One is up and running, the other is not.
blhack
Well that was a shitty 5 minutes!
nickjj
All of my NYC hosted sites went down too but they're back up now.

I'm not really upset over it and my whole business depends on my sites being up. DO's uptime has been really really good in general. I can't remember the last time everything went down like this in NYC3 over the last ~6 years.

adventured
Saw the same thing out of NYC. It seemed to only last a few minutes, I didn't bother to investigate further due to the short duration.
NicoJuicy
My droplets aren't down, my own status page didn't notice anything.

DC Netherlands

( Didn't login yet)

Ps. Status page+ checks have no dependency on DO

sillycube
Oh, I thought that it's my fault. There were a couple minutes outage for my app.
bovermyer
That was a really, really brief downtime. My two sites that were affected were offline for less than five minutes.
montroser
I would not call five minutes "really, really brief". Could have been worse, but for some applications, five minutes can be an eternity.
dsr_
If your application can't cope with five minutes of downtime, I'm sure you're not relying on a single cloud vendor.
username91
My London-hosted site stayed up just fine.

DO's uptime has been extremely impressive to me.

tech5000
Here is a data point: In the last ~90 minutes the external ping tool we use has shown the site changing between accessible then inaccessible 25 times.
tomxor
I know this is not usually a welcome question but... Why is this flagged?

It seems like legitimately concerning news many HN readers would want to know about, and I only barely caught it before it was banished from the front page. I really can't see a reason.

ruben81adelaide
Yes, I monitor about 1000 servers in Digital Ocean in New York, London and Singapore. Only New York went down for about 1 hour and a couple of hours later the global DNS system in DO went down as well for about 15 min. They initially didn't made any update in their Status page