Yes, this is the longest downtime I've had on App Engine for years. I've been up all night and the app engine team were up working on it as well and sending updates.
It seems that the default authentication method broke for some apps. They have different APIs to reach your data, and I tried another approach and it worked, but it's different enough, and we use Cloud Storage in too many places, that it would take a day or more to change our code to use it. I started on it anyway, in case the outage lasts longer.
We're also affected, our app runs in EU datacenters. We had auth errors on both Cloud Storage and BigQuery.
BigQuery seems to work now, but not Cloud Storage.
Our last outage was 2013, but this has been going on for over 12 hours..
At least Google admits they were down. When AWS goes down, the best you can hope for is a little "i" next to one of the green checks 6 hours after the outage is over.
15 comments
[ 3.3 ms ] story [ 44.1 ms ] threadEdit: It's back up now.
But wow... I thought 15 minutes would be a huge downtime for a service provided by Google.
It seems that the default authentication method broke for some apps. They have different APIs to reach your data, and I tried another approach and it worked, but it's different enough, and we use Cloud Storage in too many places, that it would take a day or more to change our code to use it. I started on it anyway, in case the outage lasts longer.