I wondered the same (selfishly.. we do too).. I thought only their Watch-Instantly streaming infra was on AWS though, and that main site is still self-hosted.
Nope, the main site is also hosted on EC2. I saw Adrian Cockroft give an abbreviated overview of their architecture recently. There's very little that isn't already on EC2 and they're actively working to move those things.
We are aware that some customers may have trouble logging in to the website or streaming to their Television. We're working on getting this fixed as quickly as we can. Thank you everyone for your patience!
I am sure whatever capacity or technical problems they're experiencing are being exacerbated by millions and millions of emails from angry assholes wondering why the hell their $8 a month isn't buying them immediate and perpetual access to a catalog of human creativity and knowledge that would have brought even the most advanced intellectual from 1980 to his knees, weeping tears of joy at the wond'rous advances man has made.
"Surely," he would say between sobs, "surely if man can accomplish this with something as relatively trivial as movies, a hundred years hence will see us spreading our race to the stars."
Unfortunately all most people know or give a damn about is that they shove $8 into the magic box on their desk and it shits American Pie movies. Pan-galactic empire will have to wait, I guess. TV's busted!
Sure, but on the other hand they have been terrible communicating what is going on. Not really a good example for a service with millions of users. Most people are usually forgiving when you give them some reasons.
I am saying that I am sure the level of vitriol being directed at Netflix right now via email or other anonymous/private communications channels is disproportionate relative to the otherwise high value that is derived from an $8/mo luxury purchase.
With tongue firmly in cheek I am urging calm in this moment of national crisis. In other words, it was a joke.
Anybody interested in tracking an outage in 2011 needs to know that twitter is accepted standard for communicating with your userbase.
It has the advantages of:
A) Not being your own infrastructure - You do not want to
use your own infrastructure to communicate a problem
with your own infrastructure.
B) Is inherently many-many.
C) Maintains an archive of all messages.
D) Has both a Web and Thick Client access - making it accessible to everyone.
With that said - Netflix could be doing a better job of giving people ETAs, and updates as to when service is, or is not, expected to be back up.
Transparency, in the face of adversity, is respected.
Even something as simple as "We're triaging, and will have an update in 60 minutes" would be sufficient.
I recently had to do a short scheduled downtime of my startup's site and cut the web server over to a "we'll be back soon" page with a simple embedded twitter stream widget on the page where I posted updates every few minutes.
The whole viewing world has been conditioned that way. I briefly worked on a set-top box for a major media distributor and you'd be shocked by the support calls, when TV stops working, people freak the ef out. TV stops working and it's within a couple days of a major cultural type event (final episode of some show, the start or end of dancing with the stars or whatever is hot at the moment, the superbowl, etc...) and they really freak out. Grandmas using profanity, issuing threats, etc..
I wonder if Netflix folks will follow up with a new blog post explaining what happened. Was the problem with a new code released to production or was it an AWS/infrastructure outside their control.
Considering this seems to be working at the moment (2 hours later) does this really qualify as news? Sure, significant downtime might be of interest, but are we really going to link the URL of every web-service that is down for an hour or two here or there?
A post about the causes or effects of said outage, or general related social commentary might be worthy of a HN submission - but to simply throw the URL of any service showing a 404 for some small amount of time seems a bit pointless (exceptions perhaps being Google/Facebook being down worldwide for a couple of hours).
I guess my point is, a post highlighting this relevance as your comment did makes it newsworthy, whereas "look some site is down" is mostly context-less for a lot of us.
It would be great if someone from Netflix could comment on what happened. Seen nothing on their blog or facebook page. When websites go down, the idea is not to point and make fun but more to learn what and why because you ll find yourself in those shoes sooner than you think.
38 comments
[ 3.5 ms ] story [ 86.9 ms ] threadhttp://status.aws.amazon.com/
http://www.readwriteweb.com/cloud/2010/12/chaos-monkey-how-n...
It's evolving, adapting and spawning other chaos monkeys.
[Edit]... course, not super complete ones..
We are aware that some customers may have trouble logging in to the website or streaming to their Television. We're working on getting this fixed as quickly as we can. Thank you everyone for your patience!
"Surely," he would say between sobs, "surely if man can accomplish this with something as relatively trivial as movies, a hundred years hence will see us spreading our race to the stars."
Unfortunately all most people know or give a damn about is that they shove $8 into the magic box on their desk and it shits American Pie movies. Pan-galactic empire will have to wait, I guess. TV's busted!
I'm sure there are a lot of Netflix customers that fall somewhere between the range of not caring right now and the huge extreme you describe.
Not sure what your point here is.
With tongue firmly in cheek I am urging calm in this moment of national crisis. In other words, it was a joke.
Anybody interested in tracking an outage in 2011 needs to know that twitter is accepted standard for communicating with your userbase.
It has the advantages of:
With that said - Netflix could be doing a better job of giving people ETAs, and updates as to when service is, or is not, expected to be back up.Transparency, in the face of adversity, is respected.
Even something as simple as "We're triaging, and will have an update in 60 minutes" would be sufficient.
I can no longer imagine doing it any other way.
The whole viewing world has been conditioned that way. I briefly worked on a set-top box for a major media distributor and you'd be shocked by the support calls, when TV stops working, people freak the ef out. TV stops working and it's within a couple days of a major cultural type event (final episode of some show, the start or end of dancing with the stars or whatever is hot at the moment, the superbowl, etc...) and they really freak out. Grandmas using profanity, issuing threats, etc..
I quit watching TV after seeing that stuff.
A post about the causes or effects of said outage, or general related social commentary might be worthy of a HN submission - but to simply throw the URL of any service showing a 404 for some small amount of time seems a bit pointless (exceptions perhaps being Google/Facebook being down worldwide for a couple of hours).