Does Amazon S3 really save money?

Being able to smoothly scale from 1TB to 2TB (or down to 500GB) is nothing to sneeze at. Nor is having a metered 250mbps connection (shop around -- hard to get less than $30/mbps at low volume), or having someone else handle the pager 24x7x365, or paying $150 at the end of the month instead of $5,000 up front.

There are systems and scales for which S3 is actually too expensive, and of course Amazon is making a profit off of all of this.

But there are a lot of hidden costs to DIY. Ask anyone who's tried to get 6 more servers flown in on the weekend, or had to cut short a holiday to drive to the damned colo at 3am, or overbought capacity, etc.

(edit) As for the "unlimited" option -- SM et al know to the byte what their average user uses so they price the unlimited option to make a profit on the average case.

For that service, the cost is very reasonable.

Once you get into truly large data amounts, other things start to break (RAID 5, RAID 6, tape backup, disk backup, synchronization, the ability to replace storage systems without massive outages). The good news is that they're almost all solved problems, but you're usually stuck with buying overpriced crap from EMC, Hitachi, NetApp, 3PAR and IBM (storage is a protection racket). All of this combines to explain why a good storage admin pulls down 6 figures a year.

I may be a bit myopic, but I see a world coming where technology startups trade capital costs for operating costs. S3 is pricey if you're dealing with small quantities of data, but once the step increase in your per-GB storage costs goes over 30%, you might want to reconsider. The steps only get bigger.

However, spraying files everywhere is a pain! MogileFS makes it a lot better, but you're still in charge of monitoring it and making sure it's healthy. With only two boxes, you have to be always on call so that you can order another box from your provider fast.

Plus, there's the issue of multiple data centers. S3 doesn't just make redundant copies of your data. It makes copies across data centers. So, you're paying $0.10/GB for data in, but you don't have to pay for when it replicates copies into several data centers.

You also have to realize that you have to pay for excess capacity anytime you're doing your own storage system. If you like to keep a 50% buffer (a reasonable size), you're going to be paying 1.5x the base cost of $0.10/GB that you've come up with.

And then there's the issue of having to make sure you're monitoring it and that if you see a spike in storage usage you can add drives fast enough. . .

You pay for a bit of convenience with S3. I'm not going to argue that it's cheaper, but it's definitely a lot less headache. Are you going to colo several boxes in different data centers, constantly monitor the storage, make sure that they serve the files properly, making sure that more copies get replicated if one server dies, replace drives as they fail, adding more servers as needed. . .

If you're on a large scale, I'd say you should do your own storage because you can justify making that someone's job (or a large enough portion of their job). I'm not sure I agree with SmugMug using S3, but I'm not sure I disagree either - it allows them to concentrate on what they want to do. Remember, for every tech person on HN, there's 100 that will say they're doing backups and aren't (ok, maybe not true, but you have to find an employee to manage your storage who you trust as much as Amazon).

However, most people don't have that much to store. If you're storing 100GB of data, you'd then be paying for multiple servers all with RAID and managing MogileFS or the like for what? 20% savings? $150/year? I'm as cheap as the next person, but I also like sleep. I don't want a pager calling me telling me that one of my two file stores is down and that I need to provision and configure a new box at 2am. And do you want to focus your time on creating a compelling product that your customers think is awesome or do you want to spend your time creating an awesome file store that works really well? Life has tradeoffs. You're not wrong, but I don't see Amazon as ripping people off with their pricing and I don't mind someone profiting from giving me a hassle-free, no-lock-in solution.

EDIT: I personally think your estimate of buying boxes and colo'ing them is a tad low so my 20% might be your 50% and so it might make sense by your numbers more. Maybe I've just seen crappy colo offers. Link if you know good ones! I love being proved wrong.

Also, server warranties, UPSs and the sheer hassle of specifying, ordering configuring servers and taking them to somewhere and fiddling with them and regularly patching them and so on.

And increasing the storage capacity on each machine can be done at the price of $0.1/GB

No it can't, you need to pay a competent admin to go to the servers, shut them down, install new drives, and start them up again and expand the RAID onto them. 1hr minimum. Assuming the RAID is expandable and there is physical space in the server - if you need to add more mounted disks the app must be adjusted to support that. If there is no space, you may need to replace some drives and handle moving the data onto them (several hours?), or worse buy a new extra server.

Assuming your backup system can just take another TB of data without any changes.

And what if something does go wrong? You're talking of at least half a day from the time you find out until you get someone to go, wait for the travel time, diagnose and repair and restart, then dealing with problem reports and complaints.

I hope Amazon has better monitoring on S3 than anything I have set up as well.

You want to host a liveblog for the Apple keynote at Macworld. No matter if you are a small site or Engadget, your traffic for that 90 minutes would be 2x to 100x bigger than your daily average.

So do you:

a) buy or rent extra servers for the whole month, spend a few man-weeks (or pay somebody for) setting them up, working on their synchronization, etc.

b) write a small script that regularly uploads the static liveblog HTML (or JSON) to an S3 dubomain and rely on Amazon's thousands of servers and flexible scaling (Dynamo, which powers S3, will allocate as many servers as needed to handle your load) to do the work?

Granted, option b)'s per-GB cost would be a bit higher, but your fixed cost for labor and hardware in a) would be even bigger.

http://perspectives.mvdirona.com/2008/12/22/TheCostOfBulkCol...

His is a more datacenter-level view (considering $/GB, from the perspective of arbitrary GB), but builds a convincing case why S3 @ $0.18/GB is fair).

Also, you have to read the terms of some services closely. IIRC, some of the online backup services are limited to "personal use only". They have separate pricing tiers for business backup (because they know those customers will use more).

S3's most expensive tier is $0.15gb/mo

So, not counting bandwidth, a user would have to store roughly 22GB of images to eat up that cost.

Being conservative, a high-res JPEG out of camera (12mp) is probably 3mb on average. So, we're talking about over 7,000 images.

Amazon makes sense for companies that are too small to dedicate an admin to a server and/or to load that server and bandwidth to 60%+ capacity. At some point, when you have a need for a small server farm and the uptime of those servers is core to your business you will likely find it is more cost effective to do the operations in-house.

Think about companies like Salesforce.com, for their datacenter needs, I don't think you could ever justify Amazon (or a similar) service.

As a rule of thumb, I charge customers in my colo facility $60/U/mo as a starting price. You get a shared pipe and relatively unmetered/unlimited bandwidth (for "business" use, no torrent hosting or stupid shit). For a little more you can get dedicated bandwidth, 95% billing, etc. I've got someone who is looking at bringing in a 3U backup server, they'll pay $200/mo to have 3mb dedicated bandwidth and the colo space. That server will have a few TB of data replicated from their main site, I don't think S3 would make sense for them.

- Amazon keeps at least 3 copies of your data (which is what you need for high reliability) in at least 2 different geographical locations. That's what we'd do ourselves, too, if we continued to use our own storage internally. So your math is off both on the storage costs and then the costs of maintaing two or more datacenters and the networks between them.

- When Amazon reduces their prices, you instantly get all your storage cheaper. This isn't something you get with your capital expenditure of disks - your costs are always fixed. This has upsides and downsides, but you certainly don't get instant price breaks to your OpEx costs. When they added cheaper, tiered storage, our bill with Amazon dropped hugely.

- There's built-in price pressure with Amazon, too. The cost of one month's rent is roughly the same as the cost of leaving. So if it gets too expensive (or unreliable or slow or whatever your metrics are), you can easily leave. And Amazon has incentive to keep lowering prices and improving speed & reliability to ensure you don't leave.

- CapEx sucks. It's hard on your cashflow, it's hard on your debt position if you need to lease or finance (we don't, but that just means it's even harder on our cashflow), it's hard on taxes (amortization sucks), etc etc. I vastly prefer reasonable OpEx costs, with no debt load, which is what Amazon gets us.

- Free data transfer in/out of EC2 can be a big win, too. It is for us, anyway.

- Our biggest win is simply that it's easy. We have a simpler architecture, a lot less people, and a lot less worry. We get to focus on our product (sharing photos) rather than the necessary evils of doing so (managing storage). We have two ops guys for a Top 500 website with over a petabyte of storage. That's pretty awesome.

Hope that helps!

You can argue all you want about how reliable and convenient it is. But it's not cheaper. In fact, it can be very expensive especially as you consume more bandwidth and storage.