I agree. If putting [video] on video links is a thing here, so should [paywall] or similar since it's even more annoying than videos.
However, I'm a subscriber to The Information so here's a share link: https://go.theinformation.com/f2b133baedc57a8c - I think it requres an email, alas, but it's something if people are desperate to read the story.
A lot of this article isn't that surprising -- especially the part where they're keeping "Business Critical" data on GCP. Of course they are! Every company needs a disaster recovery plan and in general shouldn't put their eggs in one basket.
I'd be interested to see where the AI/ML frameworks differ though, and what additional value one can extract using GCP versus AWS and at what cost/presumptions.
If your 500 servers are for storage, S3/GCS will beat you in cost. For compute, that might be a different story, but then you still have to consider regionality, networking, etc.
Let's be honest, even in the public cloud there are very few orgs that can survive regional failures and most people solve locality through CDN's for content delivery which means a private datacenter isn't really that different from public cloud for most orgs. Also, when when orgs move to the cloud they don't fire their networking team, they still have deskopts/laptops/servers/security devices/printers and large lans/wans to connect / peer already and these teams could staff up for private datacenter without really adding additional costs that aren't already incurred as part of doing business anyway. Our private datacenter is a few thousand servers leased space in a colo and we have one "datacenter dude" who handles hardware issues and our network team spends 90% of their time still on corporate infra vs actual supports with colo space. One can take some up front engineering/planning costs to scope out a private datacenter for about the same cost as it is to scope/plan VPC's and peering/connectivity/vpn services.
I completely agree. I jokingly tell everyone I know using AWS to "not use us-east1" and that they need to use another region not just AZs...and they always laugh and say "you were right" with us-east goes down.
At least from my view, there's a difference in backbone versus corporate space networking, but you're right, most everyone won't ever realize any savings from that side of things.
How often does us-east-1 go down? I know of about 4 big outages in the last 7 years but you're making it sound like it happens much more frequently. Is there a list of outages somewhere?
it's not just about it going down.. if you rely only on us-east they have had multiple times when you couldn't autoscale because there was no more capacity... if you rely on running lean and scale/up/scale down events us-east would have bitten your ass a lot the past year... Just for comparison sake even our private colo has had 0 issues the past decade that caused any interruptions to service
I do run such a service, one that must scale daily in response to traffic at multiples of off peak traffic and I can state unequivocally that this is false.
I can vaguely recall over the past several years single availability zones having short periods of time where single instance types were unavailable, but never the entire region... The idea that this happens "a lot in the past year" for the entire region isn't my experience at all.
You really have to factor in the labor costs of infrastructure and operations into that as well. Running your own private "cloud" isn't cheap. And no matter what you do, you aren't going to get even close to feature parity with the public cloud offerings (or even speed, if you are using virtual-machines). There are a lot of trade offs that you have to weigh in that don't always come down to bare dollars and cents.
Everything you'd have to pay for, Amazon or Google has to pay for for you... plus profit margin. At Netflix scale, it's almost impossible for someone else's cloud to be cheaper than running your own, and hiring the necessary staff and acquiring the necessary space and hardware to do so.
I honestly have to wonder if the level of Amazon's discount for Netflix is absurd to the tune of "we lose money doing this" just to retain them. AWS may need Netflix, to make serving it's smaller clients economically viable, more than Netflix needs AWS.
If you're a margin business then certainly. If you're not then there's a lot to be said for keeping your core competency lean and offloading the heavy lifting at a negotiated price.
This is assuming you can do everything at the same efficiency level as Amazon or Google. It's likely that you wouldn't be able to outperform on these metrics to a degree that it would make the profit margin difference a wash.
The exact point I am making is that at a certain point, you are able to do things as efficiently as they are. Netflix certainly qualifies at that scale.
I wouldn't do my own datacenter as in build out a building, but I would certainly lease a cage and build out my own infrastructure at probably half that count.
As far as I know, Netflix has POP's for content delivery in many carrier facilities so it's not a pure cloud player by any means.
For the cost of 1 year for a single cloud server you can buy a 52 core dell with 256gb ram and a few SSD's and have price/performance ratios that heavily scale in your favor and with a scheduler like marathon, k8s, nomad you can easily achieve elasticity in a private colo to a large degree without also marrying yourself to cloud specifics and nothing would stop you from peering with major cloud providers for seasonal elastic needs.
That's a good distinction. CoLo providers are pretty reasonably priced and most of the larger ones are good at what they do.
There's a HUGE scale difference between a CoLo and your own -- as soon as you start having to negotiate diesel supply contracts yourself things get crazy fast.
I think this is a crucial distinction. Netflix has a relatively unique architecture for their content delivery system compared to other online video providers, which is worth considering in context of this article.
With Netflix's Open Connect (https://openconnect.netflix.com/en/) system, most residential Internet Service Providers have appliances at the regional point of interconnection. These appliances are inside of or directly connected to the ISP's network in the region. Back in 2016 they had appliances deployed to over 1,000 locations. Content is pre-positioned during off-peak hours to the appliances.
Previously Netflix has been open about how they use AWS for "generic, scalable computing" including "all of the logic of the application interface, the content discovery and selection experience, recommendation algorithms, transcoding, etc."
At the time, they claimed that one of the advantages of doing this was the ease of use and growing commoditization of the “cloud” market, so it makes sense that they are evaluating alternative cloud providers for these commodity services.
I've used most of Hashicorp's tooling - Consul, Nomad, Vault, etc. - and I consider myself to be a Hashicorp fanboy. But if you're spinning up a bunch of VMs "in the cloud" and that's all you're doing, you're not taking advantage of the platform. The idea behind using AWS is not just to avoid your own capital infrastructure, it's also to allow someone else to do the "undifferentiated heavy lifting". In our case:
"not marrying yourself to a one cloud provider" is equivalent to "coding using a repository pattern to access the database" in both cases people claim to not wanting to tie themselves to one vendor and then they realize that they aren't taking advantage of what the vendor has to offer.
In my experience, no one wants to go through the pain of moving to another vendor just to save a little money.
This isn’t such an obvious decision. Running your own datacenter is difficult. Besides racking and stacking, you need to deal with power and cooling. Most likely you will outsource all of this to a collocations management services initially, which becomes your overhead just like AWS/Google. But they have economies of scale, in addition, they have expertise in managing all of this equipment with very high reliability. They also have tons of spare capacity (elastic) that are at your disposal, you’ll lose this in your own datacenter.
On top of that you need to hire specialists, and 24/7 noc teams, for making sure your site is always up. And it’s not just one, you need a minimum of two, geographically separated so that they are not effected by the same fire, earthquake, power outage from ice/wind/electrical storms.
At the end of the day, You need to beat AWS’ profit margin, and you still would have a huge amount of difficulty being able to match the uptime that you get when you’re so easily able to distribute services across regions and availability zones that are resilient to all the real world problems listed above. And all of those do happen.
Even for the largest of companies, I’m not sure it makes sense to compete with these services except in the most niche of markets (like dropbox and storage).
The cloud doesn't really have economies of scale unless your workloads scale opposite of pricing - which they very rarely do. For small shops, sure, but for the example of 500 instances running - you could lease space and grow with physical servers at a better economic scale. If you only need 500 servers for 1 busy season and can drop back down to 100 then stay in the cloud. If you need 500 always running instances and your growth is 20% year over year, i can almost guarantee a good colo will be cheaper and faster if you use the same automation and configuration as code philosophies in private dc as you would in public cloud. Puppet/foreman/cobbler/ansible/chef all work great in private DC
In general, I think it’s a wash. The margins you save on end up costing you in other ways, such as development velocity and having access to easily deployed high-scale solutions like dynamo and managed instances like RDS. it takes time and organizational know-how to get to a point where similar things can be managed in the same way as AWS.
Again, there are niche segments where this might not be true, 500 constant machines churning through massive datasets all day-long, etc, might fall into that category, but it’s not slam dunk.
The margins you're paying for include functionality you don't use and your own cost of integration, observation, and management of their stuff. The principal advantage of using a cloud provider is avoiding capital costs and any useful apis they have that you'd otherwise have to develop/maintain in-house.
There are lots of open source products that offer programmatic/desired state/idempotent config & apis for provisioning and hardware lifecycle management in private dc.
The most fun thing about startup world is that it pretends that normal rules of economics and accounting do not apply to it (until a startup runs out of money)
$X in CapEx makes balance sheet look better than $X in OpEx.
As generously as I can put it: It's a form of denial that people use when overwhelmed with everything else. There's typically too little real data on what the money's going to look like until a bunch of things are tried out, each essentially a random variable. So instead of freaking out about that, just bandage over it with some lies to yourself until you can actually start addressing it.
Businesses in well-known markets can start estimating their costs + potential revenue early, and optimize early. But for a business that might "pivot" a few times before finding "fit", how do you do it? You just look at time (runway: burn rate * remaining capital) you have to try stuff.
It's not great, but I can understand why some people do it.
It's a good question, but depends on your workload.
Netflix is all about content delivery -- which is both distributed and highly variable over the day/year.
Using Cloud means Netflix can deploy closer to the customer. Plus scale up and down as required. If they did this themselves it would be quite expensive and involve a lot of redundancy.
It’s not either/or. You want to be in a position where your own racks and public cloud providers are all running instances of the same distributed scheduler. Then you can buy capacity wherever it makes the most economic sense at the moment. Just proving that you have the capability to run your own hardware cost-effectively can negotiate Amazon/Google down to that price or lower.
This is probably the nicest answer. With things like kubernetes, hardware is quite abstracted. If it makes sense to use your own hardware, your software should be good enough to effortlessly shift some load to your own hardware and back incase own hardware can't handle a spike. Hybrids are probably the future where you rent when it makes sense, and buy when it makes sense and move compute and storage by dragging a slider.
And the standing problem now is that their secondary supplier they're considering is also a competitor in the same space. It's a wonder Netflix isn't testing the waters against Azure, since Microsoft just closed their video sales offerings down.
Microsoft pulled out of the music space, they're still selling & renting movies/shows and content services are a core feature of azure. Netflix uses its own POPS for of its content delivery.
Ah yes, I mixed that up. :/ Though who rents movies from Microsoft? And they don't have a subscription-based service for it, like both Amazon and Google do.
Original YouTube content is really not a huge part of YouTube's offerings. Even if it were a larger component, Google wouldn't jeopardize a large hosting account just to score points against a competitor.
Amazon, on the other hand, will partner and make nice with competitors until they're ready to abandon them. Amazon is planning on supplanting FedEx and UPS and already has its own fleet of cargo planes. Amazon is great for business until they decide they don't need you anymore.
Yeah. It might only be a matter of time until, to avoid "customer confusion", Amazon decides to no longer have Amazon services provide things to Netflix customers?
Cobra Kai is a pretty good example of the future of YouTube Red: actual Hollywood talent and directors, instead of YouTube stars with higher production values.
I'm as much a fan of Karate Kid as the next guy, but calling both Johnny and Daniel-san 'hollywood talent' is questionable. I'd not seen either of them in anything from Hollywood since the 1989 Karate Kid part 3
Perhaps long term, that would be an issue. But short term and even medium term, I don't see Netflix, Prime Video and YouTube Red as competition. Their real competition is cable/satellite and as long as Netflix helps round out a cord-cutting bundle that convinces people to ditch cable, it's a net benefit to Google/Amazon. And as cable dies, especially given the recent FCC direction, you can expect them to start to play more games with the internet connections they also provide to customers. I think Google and Amazon realize that Netflix will be much more sympathetic in that fight than either of them will be. So it makes sense for Amazon/Google to support Netflix as much as possible so they can use them as a proxy in the upcoming fight against the cable companies.
Honest question - is Amazon really seen as a competitor here? Every time I've attempted to use Prime video, I'd scurried away terrified of their awful ui. I guess it's hard to gauge subscribers since it's 'thrown in', but I'd be curious to know how many people really use the service.
Netflix has been "testing the waters" at GCP for years. And as their infrastructure grows, relying on a single cloud provider would be a silly limitation to put on themselves. Disaster-recovery/failover is a very natural use case for multi-cloud deployments. Not to mention that for certain workloads and patterns, GCP is cheaper and better. Netflix operates at a scale well beyond what most AWS managed services can really address, and broad cloud patterns like object storage, function-as-a-service, VMs, and networks, can be abstracted away given enough resources, which Netflix surely has. Not to mention that at Netflix's scale they have real leverage over cloud providers and what services they offer. They would be stupid _not_ to be playing AWS and GCP against each other for pricing (on which they already get a huge discount) and features.
I'm guessing there's some project getting underway to move/duplicate some subset of resources hosted in AWS on GCP. It's likely a huge project that dwarfs most corporations entire cloud presence, but I'm guessing for Netflix it's a small piece of a giant puzzle, and that the vast majority of their operations are still centered at AWS and will be for years and years to come.
You don't have to be Netflix to get AWS, GCP and MS to play against each on pricing. I've seen a number of my clients build a prototype on each platforms, and then use it to make final decision. Part of that final decision is having the cloud vendors compete on price.
I personally haven't seen any massive price breaks, but there definitely is room the margins.
For years Dell would let it "leak" out to the press that they were looking at supporting AMD chips. It would happen every year like clockwork. Then after a month you'd hear that Dell and Intel had come up with a new deal to remain Intel exclusive.
Before then – in the 90s both Intel and Microsoft used a lot of backroom deals to keep big vendors exclusive. If memory serves it took the Opteron’s crushing performance advantage before we started seeing that break for Intel servers; Linux users for years were stuck paying for OEM Windows installs which were never used.
Yes, but I've been around the industry a while. I specifically remember this happening during the late 90s and just in to the 2000s.
It was occurring at a time when AMD was fairly firmly beating Intel on a straight $/performance basis (~K6 era). If Dell had started shipping AMD based systems back then it could have been a big game changer.
The general narrative around the tech press at the time was that Intel knew this and were desperate to keep Dell as Intel exclusive.
I've love to see some open source tools that manage this automagically for me. :-)
It would also require an architecture that supports multiple interfaces. Totally doable (the facade pattern comes to mind) but you'd want to calculate the TCO to see it works out financially.
Shameless plug: we have an open source tool you can build stuff like this on. It'll run any logic you can implement in JavaScript, we have people doing things like "send traffic to backend with lowest latency" already, it'd be pretty easy to pull in some API data to pick based on cost.
I work for a large CDN. Lots of big customers do this, they shift their traffic around between different CDNs based on a lot of metrics, including price and performance.
Did this a lot with payment processors. Literally had application router settings that could route based on % or switch off once a certain monthly or yearly amount was hit. Configs could be updated in real time.
They'll play a few games with contracts to spread the costs around (e.g. giving n% discounts for m years of reserved capacity). You probably won't get much just by saying "I can just go to GCP" and asking for the world. The more you spend, the more distinct services you buy, and the longer you want to commit the steeper the discount. I've seen up to 10% but I'm sure they do more for bigger customers.
Margins are pretty high, in part due to scale. If AWS were to hypothetically sell 80% of capacity at cost, that makes the last 20% more profitable than it would be to just sell the 20% alone.
"We have $500M available this year that we can spend to lock in $1B in revenue. If you give us $1B in revenue, we would spend $500M on helping you to achieve your objectives. If you give us $100M in revenue, we give spend $50M in helping you to achieve your objectives"
So say I need to spend $10M on some project and I am looking at $20M of spend on hosting. You are offering me 10M CAF. I spend $20M on hosting, and 0 on some other project - instead you spend your 10M CAF on helping me with my project, which of course looks really good on my books.
I agree with most of what you said, however, Netflix most definitely does not operate at a scale well beyond what most AWS managed services can address.
Literally has been used as an intensifier for a figurative expression since the 1800s and has been defined in the dictionary that way for quite a while.
It was used in this manner by Charles Dickens, James Joyce and American authors as well.
Although, it pisses me off when I hear it used that way.
And I suppose no one will ever be able to describe the act of walking quick in work out clothes again because of all the people “running” to the store.
Firstly: not very new. Hasn't Netflix been at this for a long time?
Secondly, everyone should be checking. Not only is Google doing a lot of subsidization, but their services are really really good. My current employer ended up being unable to use them but it's I acutely regret every day I'm forced to build out things Google already has or deal with Amazon's frustratingly antiquated ELB infrastructure.
I see such intense benefits from HTTP/2 end to end, I'm quite happy to be called a shiny-chaser when my 50th and 99th drops so sharply.
Also, prefetch is absolutely incredible for webapps and I'm trying to get my peers at my current employer to embrace it even if it's not well supported in AWS.
To answer your question directly, AWS's poor spike handling has really made me distrustful of them. It's not that AWS's support staff is bad (actually, they're phenomenal given the difficulty of that job) but... the fact that I can be taken down by a big scale event because I can't pre-scale my ELBs without support help that takes hours is kinda annoying.
Now, I've been TOLD by ___________________________________ this is actually because it's a pain in the butt and they're working to fix it, but AWS's budget seems to favor new shinies over iterating on the existing environment. Look how long it took us to get a new and less buggy AWS dashboard and that still hasn't propagated to all the internal services.
I could go on. For example, my experience is that GCE's and even Azure's I/O and networking is way faster, more consistent and more reliable than AWSs. I host all my dev boxes on GCE because using an attached drive (so I can de/re-provision to save on costs) is much less agonizing. Also: google has some AMAZING database products that category kill whole infrastructure tasks. It's a very good kind of vendor lock-in when they can say, "You can't move away because no one else has found a way to provide our feature set."
I'm surprised they have thought to move some of their infrastructure in house. The clearly have the scale where this would make sense, but it may be that AWS and GCP provide them with such low special prices that it doesn't make sense...
The problem might just be that it's genuinely incredibly difficult to get infrastructure right at this level, and the cost to move in-house is still too high, even accounting for the savings over the long term.
This could also be titled "Netflix Ensures its AWS Discounts Continue"
AWS sales teams are quite responsive to threats of moving in terms of continuing discounts. I've drawn up migration plans solely to use as leverage against AWS for deep discounts in the past.
Just curious: Can you state a range of possible discounts against list price? The stuff I deal with is small fry but I always wonder how much cheaper the big guys get their cloud services compared to us.
Third hand knowledge - since I was not a part of the negotiations - fair warning. The company I worked for was getting discounts in the 40-50% range. It was the only way they could afford to use AWS, frankly, since it was all static provisioning (i.e. no elasticity in the workload). They were also approaching a petabyte of data stored in S3.
No way, if I was CTO of a big company (or, say, whoever at Pentagon is in charge of IT), I would love to speak to the guy that shifted Netflix away from in-house datacenters to "The Cloud".
You're ignoring the fact that he spent a few years at Battery Ventures in between Netflix and AWS. Adrian is a well respected professional in his field, has been around for decades, and nobody holds any negative perceptions of him.
He wasn't hired by AWS because he migrated datacenters to the cloud. He was hired by AWS because he created this: https://netflix.github.io/ - Netflix's OSS, which has allowed them to attract so much great talent, was a direct result of Adrian's passionate open source advocacy. He's doing the same thing at AWS now, and has already accomplished great things like open sourcing all of the AWS documentation.
My prediction is that by 2020 Netflix will move a significant portion of their IT to Google Cloud.
Why? Most of their architecture is microservices / linux containers / serverless, and Google is simply way better positioned there (with Kubernetes as their crown jewel). They will eventually win.
Source: no source, just my "expert" opinion (6 years at AWS, 2 at VMware's cloud, currently CEO at a startup).
AWS is catching up as far as Kubernetes support. I might agree that GCP is better positioned for some of that architecture, but they are still comparable. There are also some advantages to AWS. For example, I think that their RDS product is still more mature than Google Cloud SQL.
+1 regarding RDS. A company I spoke to is moving from GCP to AWS, with their biggest pain point being Cloud SQL is WAY less flexible than RDS. While GCP's k8s offering is better/easier to use, they still have some catching up to do.
To drive the second point home, GCP recently released VPC Flow Logs, while AWS has had it for a good while now.
How big doesn’t netflix have to get before it’s in their best interests to build data centers of their own?
Toolchains are maturing. They’ve figured out ways to get multiple uses from the same hardware. They shouldn’t be beholden to the rollout or geographical plans of two cloud providers.
I don’t even mean build out their entire infrastructure. I think even running your base load on your own hardware makes you understand your cloud situation much more profoundly.
They have data centers of their own and distribute all of their own content. AWS powers the UI requests and their back office number crunching stuff. Everything after you push play is done via Netflix's own hardware.
(Maybe there's an even bigger size where it no longer makes sense for them to outsource this, but bidding GCP and AWS against each other seems like a reasonable strategy for their use-case).
Another reason is the rumor that Google is going to make their own chips for their Cloud, possibly ARM based. This could have big cost savings with some applications. Video streaming and encoding are one of them.
While that might be the case, I wouldn’t be surprised if they let teams play with k8s as well. Besides, one should always look at these choices objectively; if Google + k8s is a better deal overall than AWS & Mesos, then you shouldnt let your earlier investments cloud your judgement.
I know both Kubernetes and Mesos quite well. To be frank, they're on two different levels - and I assume the smart(er than me) guys at Netflix know that as well.
If more people start using GCP, then in the near future, GCP's prices will go up. In contrast, AWS prices may come down to compete with GCP. And when they reach close parity, things will begin to commoditize and get even cheaper across the board. I see this being a good thing overall as an industry.
To whom? One would imagine that the viewing data they collect would be one of their advantages over their competitors, making it detrimental to their business to sell it?
I can't speak for Netflix's privacy policy, but, if your concern is with Google, we never ever ever ever mix/merge Google customer data with customer data. Like ever ever ever.
This is one reason you don't want to go with specialized databases or queues. If you used a managed Queue or Database, make sure you can get that same exact Queue or Database on another cloud provider.
This is natural commodification of compute, especially with such large deployments where there will be a lot of in-house tooling and apps that just need raw compute and storage to run, no matter where it comes from.
The rise of containers and Kubernetes makes this even easier now. It'll be interesting to see how the clouds differentiate beyond this as most managed services are still rather underwhelming compared to the hype and are also often beat by Kubernetes deployments if available.
lol, This a "press release" written by a PR firm employed by google
etflix, one of Amazon Web Services’ biggest customers, is expanding its use of Google Cloud, AWS’s biggest rival, according to two people with knowledge of the matter. The move, which hasn’t previously been reported, could portend a shift in the balance of power in the cloud computing market
145 comments
[ 2.6 ms ] story [ 210 ms ] threadEdit: Since this is being downvoted, here is the proof: https://screenshots.firefox.com/bgD6FxWvQS4e0sB2/www.theinfo...
I have no issue with paywalled-content, it should just be noted that this is, at least for some readers.
However, I'm a subscriber to The Information so here's a share link: https://go.theinformation.com/f2b133baedc57a8c - I think it requres an email, alas, but it's something if people are desperate to read the story.
I'd be interested to see where the AI/ML frameworks differ though, and what additional value one can extract using GCP versus AWS and at what cost/presumptions.
If your 500 servers are for storage, S3/GCS will beat you in cost. For compute, that might be a different story, but then you still have to consider regionality, networking, etc.
At least from my view, there's a difference in backbone versus corporate space networking, but you're right, most everyone won't ever realize any savings from that side of things.
I can vaguely recall over the past several years single availability zones having short periods of time where single instance types were unavailable, but never the entire region... The idea that this happens "a lot in the past year" for the entire region isn't my experience at all.
I honestly have to wonder if the level of Amazon's discount for Netflix is absurd to the tune of "we lose money doing this" just to retain them. AWS may need Netflix, to make serving it's smaller clients economically viable, more than Netflix needs AWS.
As far as I know, Netflix has POP's for content delivery in many carrier facilities so it's not a pure cloud player by any means.
For the cost of 1 year for a single cloud server you can buy a 52 core dell with 256gb ram and a few SSD's and have price/performance ratios that heavily scale in your favor and with a scheduler like marathon, k8s, nomad you can easily achieve elasticity in a private colo to a large degree without also marrying yourself to cloud specifics and nothing would stop you from peering with major cloud providers for seasonal elastic needs.
There's a HUGE scale difference between a CoLo and your own -- as soon as you start having to negotiate diesel supply contracts yourself things get crazy fast.
With Netflix's Open Connect (https://openconnect.netflix.com/en/) system, most residential Internet Service Providers have appliances at the regional point of interconnection. These appliances are inside of or directly connected to the ISP's network in the region. Back in 2016 they had appliances deployed to over 1,000 locations. Content is pre-positioned during off-peak hours to the appliances.
Previously Netflix has been open about how they use AWS for "generic, scalable computing" including "all of the logic of the application interface, the content discovery and selection experience, recommendation algorithms, transcoding, etc."
At the time, they claimed that one of the advantages of doing this was the ease of use and growing commoditization of the “cloud” market, so it makes sense that they are evaluating alternative cloud providers for these commodity services.
Consul -> ParameterStore
Vault -> KMS/Cognito/IAM
Nomad -> Elastic Container Services/Fargate (serverless Docker)/Lambda
SQL Server -> RDS -> Aurora
Memcached -> ElastiCache (Memcached protocol)
Consul+Fabio (service discovery/load balancing) -> Elastic Load Balancers/Route 53/Autoscaling
VSTS (MS's hosted CI/CD platform) -> AWS Code Build/Code Deploy/Code Pipeline
"not marrying yourself to a one cloud provider" is equivalent to "coding using a repository pattern to access the database" in both cases people claim to not wanting to tie themselves to one vendor and then they realize that they aren't taking advantage of what the vendor has to offer.
In my experience, no one wants to go through the pain of moving to another vendor just to save a little money.
On top of that you need to hire specialists, and 24/7 noc teams, for making sure your site is always up. And it’s not just one, you need a minimum of two, geographically separated so that they are not effected by the same fire, earthquake, power outage from ice/wind/electrical storms.
At the end of the day, You need to beat AWS’ profit margin, and you still would have a huge amount of difficulty being able to match the uptime that you get when you’re so easily able to distribute services across regions and availability zones that are resilient to all the real world problems listed above. And all of those do happen.
Even for the largest of companies, I’m not sure it makes sense to compete with these services except in the most niche of markets (like dropbox and storage).
Again, there are niche segments where this might not be true, 500 constant machines churning through massive datasets all day-long, etc, might fall into that category, but it’s not slam dunk.
$X in CapEx makes balance sheet look better than $X in OpEx.
Businesses in well-known markets can start estimating their costs + potential revenue early, and optimize early. But for a business that might "pivot" a few times before finding "fit", how do you do it? You just look at time (runway: burn rate * remaining capital) you have to try stuff.
It's not great, but I can understand why some people do it.
I keep being amazed that total business failures that result in pivots are considered to be successes in the startup world.
Netflix is all about content delivery -- which is both distributed and highly variable over the day/year.
Using Cloud means Netflix can deploy closer to the customer. Plus scale up and down as required. If they did this themselves it would be quite expensive and involve a lot of redundancy.
Amazon, on the other hand, will partner and make nice with competitors until they're ready to abandon them. Amazon is planning on supplanting FedEx and UPS and already has its own fleet of cargo planes. Amazon is great for business until they decide they don't need you anymore.
I'm guessing there's some project getting underway to move/duplicate some subset of resources hosted in AWS on GCP. It's likely a huge project that dwarfs most corporations entire cloud presence, but I'm guessing for Netflix it's a small piece of a giant puzzle, and that the vast majority of their operations are still centered at AWS and will be for years and years to come.
I personally haven't seen any massive price breaks, but there definitely is room the margins.
For example, this Ryzen laptop: http://www.dell.com/en-us/shop/dell-laptops/new-inspiron-15-...
It was occurring at a time when AMD was fairly firmly beating Intel on a straight $/performance basis (~K6 era). If Dell had started shipping AMD based systems back then it could have been a big game changer.
The general narrative around the tech press at the time was that Intel knew this and were desperate to keep Dell as Intel exclusive.
It would also require an architecture that supports multiple interfaces. Totally doable (the facade pattern comes to mind) but you'd want to calculate the TCO to see it works out financially.
https://github.com/superfly/fly
Some of our stuff dynamically utilizes other CDNs. A customer might go try and test deploy something and we might use (AWS or whatever) to do it.
It's CDNs all the way down.
We should rename our product "Infinite Regress" ;-)
[1] https://gravitational.com/telekube/
We the whole stack for most people, but also run our software on "private cdns" for some. We primarily target devs: https://fly.io
I've seen too many companies turn extractive (including, of course, that same Microsoft) to ever use the words "final" next to "platform decision".
just wondering, how huge (as a percentage) that discount is?
"We have $500M available this year that we can spend to lock in $1B in revenue. If you give us $1B in revenue, we would spend $500M on helping you to achieve your objectives. If you give us $100M in revenue, we give spend $50M in helping you to achieve your objectives"
So say I need to spend $10M on some project and I am looking at $20M of spend on hosting. You are offering me 10M CAF. I spend $20M on hosting, and 0 on some other project - instead you spend your 10M CAF on helping me with my project, which of course looks really good on my books.
AWS managed services literally sit behind billions of devices, a scale that makes Netflix look small in comparison: https://techcrunch.com/2018/02/27/apple-now-relies-on-google...
It was used in this manner by Charles Dickens, James Joyce and American authors as well.
Although, it pisses me off when I hear it used that way.
Secondly, everyone should be checking. Not only is Google doing a lot of subsidization, but their services are really really good. My current employer ended up being unable to use them but it's I acutely regret every day I'm forced to build out things Google already has or deal with Amazon's frustratingly antiquated ELB infrastructure.
Also, prefetch is absolutely incredible for webapps and I'm trying to get my peers at my current employer to embrace it even if it's not well supported in AWS.
Also as mentioned elsewhere, ALB or NLB might do what you want, either working the same way as regular ELB or via the same workarounds.
Now, I've been TOLD by ___________________________________ this is actually because it's a pain in the butt and they're working to fix it, but AWS's budget seems to favor new shinies over iterating on the existing environment. Look how long it took us to get a new and less buggy AWS dashboard and that still hasn't propagated to all the internal services.
I could go on. For example, my experience is that GCE's and even Azure's I/O and networking is way faster, more consistent and more reliable than AWSs. I host all my dev boxes on GCE because using an attached drive (so I can de/re-provision to save on costs) is much less agonizing. Also: google has some AMAZING database products that category kill whole infrastructure tasks. It's a very good kind of vendor lock-in when they can say, "You can't move away because no one else has found a way to provide our feature set."
[1] https://media.netflix.com/en/company-blog/how-netflix-works-...
AWS sales teams are quite responsive to threats of moving in terms of continuing discounts. I've drawn up migration plans solely to use as leverage against AWS for deep discounts in the past.
Adrian Cockcraft helped lead Netflix's effort to move to AWS and now is a VP at AWS
https://www.linkedin.com/in/adriancockcroft/
He did an interview with Software Engineering Radio back in 2014 explaining their motivation.
http://www.se-radio.net/2014/12/episode-216-adrian-cockcroft...
I always get a bad taste in my mouth when I read things like that. Was it really an objective choice, or was this AWS’ way of saying “thank you” ?
He wasn't hired by AWS because he migrated datacenters to the cloud. He was hired by AWS because he created this: https://netflix.github.io/ - Netflix's OSS, which has allowed them to attract so much great talent, was a direct result of Adrian's passionate open source advocacy. He's doing the same thing at AWS now, and has already accomplished great things like open sourcing all of the AWS documentation.
Disclaimer: work at AWS.
Why? Most of their architecture is microservices / linux containers / serverless, and Google is simply way better positioned there (with Kubernetes as their crown jewel). They will eventually win.
Source: no source, just my "expert" opinion (6 years at AWS, 2 at VMware's cloud, currently CEO at a startup).
It's all trade-offs.
To drive the second point home, GCP recently released VPC Flow Logs, while AWS has had it for a good while now.
Toolchains are maturing. They’ve figured out ways to get multiple uses from the same hardware. They shouldn’t be beholden to the rollout or geographical plans of two cloud providers.
I don’t even mean build out their entire infrastructure. I think even running your base load on your own hardware makes you understand your cloud situation much more profoundly.
https://media.netflix.com/en/company-blog/completing-the-net...
This was a big deal in 2008 :)
(Maybe there's an even bigger size where it no longer makes sense for them to outsource this, but bidding GCP and AWS against each other seems like a reasonable strategy for their use-case).
I read their privacy policy but it was so mealy mouthed it's hard to make heads or tails of to a non-lawyer.
I can't speak for Netflix's privacy policy, but, if your concern is with Google, we never ever ever ever mix/merge Google customer data with customer data. Like ever ever ever.
FWIW, we STRONGLY suggest (just to be doubly sure) that you encrypt at rest and/or offer the option to use Customer Supplied Keys. Encrypt all the things! https://cloud.google.com/security/encryption-at-rest/
The rise of containers and Kubernetes makes this even easier now. It'll be interesting to see how the clouds differentiate beyond this as most managed services are still rather underwhelming compared to the hype and are also often beat by Kubernetes deployments if available.
etflix, one of Amazon Web Services’ biggest customers, is expanding its use of Google Cloud, AWS’s biggest rival, according to two people with knowledge of the matter. The move, which hasn’t previously been reported, could portend a shift in the balance of power in the cloud computing market