What i never see is how much servers/hardware/resources they are spending.
It's _always_ like "we process a quadrillion messages per second" but it never mentions that you'll need a quadrillion expensive servers to do so.
Same for many many database benchmarks, sadly...
they are the dominant vendor of hardware/hosting to the largest and most complex web applications on the internet. AWS is the pillar that some of the most high volume sites on the internet (like Twitter and Netflix) rely on. Its safe to say that without AWS (or some other similar vendor) the internet as we know it wouldn't exist.
That's ridiculous. It's like you're saying: without Honda, the car industry, as we know it, wouldn't exist.
The hosting industry is huge. There's a ton options, many of them better, larger and older than AWS. There are so many sites that don't use AWS. For example, I believe that, pre-Google, youtube used ServerBeach.
For every site that's reliant on AWS, there's probably more than 10 000 that aren't. The giants, for example: Google. Facebook. Wikipedia. Microsoft.
It's hard to get actual numbers, but Equinix might be the largest. I believe their Singapore data center is actually what AWS, Digital Ocean and Azure use. (there's a lot more of these, such as Internap and Level 3 (which provided (and might still) services for Netflix and iTunes.))
AT&T is quite large. World of Warcraft used to be hosted on AT&T's infrastructure (might still be).
The above tend to be for large setups. Going smaller, you have the Rackspaces and Softlayers (now IBM). And, to a lesser extent, providers like OVH and Hetzner.
And, you can't forget the likes of Digital Ocean and Linode.
It's hard to come up with examples because more things are examples than aren't. AWS' market share is growing, but that's to be expected, in a lot of ways, despite EC2 being ~10 years old , it's still a new player.
AFAIC, what's more interesting about AWS is:
1 - How dominating it is over Microsoft and Google
2 - How popular the less invasive offerings are. A lot of people are sticking to their colocated, dedicated or VPSs, while using some part of AWS for DNS, backups or email. I hope they keep this up.
I had a hunch I'd see the data structures du jour in there: Bloom Filters and HyperLogLog. Since they are pretty accurate by themselves already but run much faster than the brute force approach, why keep the latter? The error margin, if used properly, should be small enough so people shouldn't really care.
They are not for accuracy. One usage for instance is to recompute stats if there are bugs in the stats processing code. Or if you want to add more metrics. (I work at Twitter but not on Answers).
22 comments
[ 3.4 ms ] story [ 50.3 ms ] thread[1] http://nickcraver.com/blog/2013/11/22/what-it-takes-to-run-s...
http://highscalability.com/blog/2014/7/21/stackoverflow-upda...
The StackExchange performance site: https://stackexchange.com/performance
Per server or in total..? Either one is believable.
Image: http://i.imgur.com/q1oDamy.png
Yes, they own some of the computers that make up the Internet, but by far not all of them.
The hosting industry is huge. There's a ton options, many of them better, larger and older than AWS. There are so many sites that don't use AWS. For example, I believe that, pre-Google, youtube used ServerBeach.
Some examples would be more interesting than a general "you're wrong".
It's hard to get actual numbers, but Equinix might be the largest. I believe their Singapore data center is actually what AWS, Digital Ocean and Azure use. (there's a lot more of these, such as Internap and Level 3 (which provided (and might still) services for Netflix and iTunes.))
AT&T is quite large. World of Warcraft used to be hosted on AT&T's infrastructure (might still be).
The above tend to be for large setups. Going smaller, you have the Rackspaces and Softlayers (now IBM). And, to a lesser extent, providers like OVH and Hetzner.
And, you can't forget the likes of Digital Ocean and Linode.
It's hard to come up with examples because more things are examples than aren't. AWS' market share is growing, but that's to be expected, in a lot of ways, despite EC2 being ~10 years old , it's still a new player.
AFAIC, what's more interesting about AWS is:
1 - How dominating it is over Microsoft and Google
2 - How popular the less invasive offerings are. A lot of people are sticking to their colocated, dedicated or VPSs, while using some part of AWS for DNS, backups or email. I hope they keep this up.
WhatsApp (Pre-Facebook) - Softlayer
Youtube (Pre-Google) - ServerBeach
Vevo - Rackspace
Zendesk - Rackspace
GoCardless - Softlayer
Snapchat - Softlayer
Innovative!
But for real, that's some architecture, and very insightful to see what they've got. Totally nothing I'd have guessed.