Compared to the cost of building and maintaining your own system I think it is cheaper. Add the cost of servers, colocation, bandwidth, people.
Also, on S3 you pay exactly for what you use, if you use your own server you are paying for them up front. Another bonus is the using their CDN if you have frequently downloaded files.
This has already been discussed in some detail at http://news.ycombinator.com/item?id=422225. I suggest anyone interested in talking price/value reads up on that thread before commenting here.
I run a fairly small site. We're upto around 7 million hits a month, still pretty tiny. Our costs right now are fairly low, but recently I made a post inquiring how to cluster efficiently, because we needed more bandwidth available (And we were going to use a few 10mbit servers).
Someone suggested that we use s3. This would've tripled our bandwidth costs.
Amazon S3 is actually relatively expensive. It's far cheaper in the long run to host it on your own server. However, if you don't have experience setting up scalable systems, tuning your configuration, or do not want to spend time doing that type of sysadmin stuff, it can be a good way to save time. It's like buying Coke at a vending machine instead of at a wholesale store. You're paying for the convenience more than anything else. Just be aware of how much you're being marked up.
Yep. Someone needs to build a S3-like solution and open source it, down to the cluster setup and harddrive specs and brand names. I have set up a storage cluster once with mogilefs and perlbal but finding documentation was a nightmare. There is none, you have to dig through the mailing-list archives.
But then again, that is the kind of stuff that people get paid ridiculous amounts of money to build, heck, I know I did. So I wouldn't bet on it. But if I had the time I would probably do it.
What is the actual site doing? Is it just a file sharing site where you have a 150mb limit, meaning the average size will be a few megs...or is this something special where people upload some special 150mb files?
Good question. It is a site where people can upload 1 special 150MB file (like a zip file) and people can download the entire thing. So the average file size will not be a few megs.
You need to bound the problem before you can figure out whether S3 makes sense.
* How many new files do you expect to have uploaded every month?
* How many files do you expect to be deleted every month?
* What's the average file size?
* How many times, on average, do you expect each file to be downloaded per-month?
Use that + Amazon's pricing to figure out how much you'll have to spend on S3 every month. Then compare that to your alternatives.
In my mind, the biggest reason to go with S3 is if you think you are going to end up with a lot more files than you could easily store and mirror on a few leased servers. S3 is a pretty good price for scalable, highly-available storage with a nice fat pipe to the Internet, but the cost per GB transferred isn't great. You can take advantage of the lost cost of storage by keeping track of how often files are downloaded and keeping a cache of locally hosted files that you serve directly off a leased server or VPS.
You need to bound the problem before you can figure out whether S3 makes sense.
* How many new files do you expect to have uploaded every month?
* How many files do you expect to be deleted every month?
* What's the average file size?
* How many times, on average, do you expect each file to be downloaded per-month?
Use that + Amazon's pricing to figure out how much you'll have to spend on S3 every month. Then compare that to your alternatives.
In my mind, the biggest reason to go with S3 is if you think you are going to end up with a lot more files than you could easily store and mirror on a few leased servers. S3 is a pretty good price for scalable, highly-available storage with a nice fat pipe to the Internet, but the cost per GB transferred isn't great. You can take advantage of the lost cost of storage by keeping track of how often files are downloaded and keeping a cache of locally hosted files that you serve directly off a leased server or VPS.
You need to bound the problem before you can figure out whether S3 makes sense.
* How many new files do you expect to have uploaded every month?
* How many files do you expect to be deleted every month?
* What's the average file size?
* How many times, on average, do you expect each file to be downloaded per-month?
Use that + Amazon's pricing to figure out how much you'll have to spend on S3 every month. Then compare that to your alternatives.
In my mind, the biggest reason to go with S3 is if you think you are going to end up with a lot more files than you could easily store and mirror on a few leased servers. S3 is a pretty good price for scalable, highly-available storage with a nice fat pipe to the Internet, but the cost per GB transferred isn't great. You can take advantage of the lost cost of storage by keeping track of how often files are downloaded and keeping a cache of locally hosted files that you serve directly off a leased server or VPS.
I presume this file doesn't change very often and doesn't require any security in front of it? To be cheap, you could sign up for a few crappy (like GoDaddy) hosting accounts with absurdly high bandwidth limits. Then, provide users with links to them sorta like mirrors. It's not the greatest user experience, but it might be the easiest cheap solution.
If you care a lot about user experience, go with S3.
Using Amazon services your initial cost is low and it's easy and fast to set it up. Using wisely AWS is a great service, especially bootstrapping a new service.
There are use cases where AWS is not recommended, having too many machines up and running 24x7 or transferring huge amount of data out of the cloud. For a site with heavy uploads it's even possible to store your data on S3 but serve your clients from a machine hosted outside where data transfer is cheaper.
S3 does look pretty expensive for this case. In general, I think S3 is best for storing files that aren't retrieved often. For hosting a download, there are better options.
I have an BingoDisk account that supposedly provides unlimited transfer. If you are just uploading and downloading stuff, it sounds like BingoDisk is ideal. (http://www.bingodisk.com/) I use it for hosting big files, like a 600M SQLite database with information about CPAN modules. It gets quite a few downloads, and nobody has ever told me I'm abusing the service.
(FWIW, I used to use BingoDisk for backups, but Linux's WebDAV support does not work well for backups -- way too much caching going on. S3 turned out to be infinitely more reliable. I don't think this has anything to do with BingoDisk, though, but rather how davfs2 is implemented. This may or may not be a problem for your use case.)
If you don't mind offloading the cost to your users, though, S3 is probably the way to go. I think Amazon will even bill your users for you now.
Because I'm tempted to say you should just try one of these cheapie web hosts and see if that fits your needs. For example, and I have no affiliation with this company other than using them a few times, webhost4life has an account with unlimited* bandwidth and unlimited* storage for $6.95 a month (http://www.webhost4life.com/linux.asp). That's a pretty standard price for linux based hosting.
The disadvantage to these types of deals is that they stuff a bunch of websites all on one server which makes performance slow. But it sounds like you'd only have a 1 or 2 page site so that might not be an issue for you.
* when these providers say unlimited there is often a clause in the fine print that sets a limit. You should check before signing up for one of these services.
Ah yes, a 150MB file site. Whenever I need to get my 150MB files, I'll go to your site!
In seriousness, I don't know what the price for amazon ec2 works out to be. However, during the development phase, it is too expensive. I believe it will cost you $72 to run an instance for a month, base price. I would suggest something like slicehost if all you need is a low power server instance for development, and then look at ec2.
S3 isn't about cheap, S3 is about reliable and scalable. Yes, it's inexpensive, compared to how complicated and expensive it would be to setup multi-site failover and support nearly unlimited scaling.
If you are on Windows try CloudBerry Explorer for Amazon S3. It makes managing files in S3 EASY http://cloudberrylab.com/ It supports most of the Amazon S3
35 comments
[ 9.3 ms ] story [ 90.7 ms ] threadIf a 150MB is downloaded 100 times it would cost $2.59. If it was downloaded 1000 times it would cost $25.55. Is that cheap?
Also, on S3 you pay exactly for what you use, if you use your own server you are paying for them up front. Another bonus is the using their CDN if you have frequently downloaded files.
Check this comment by Don MacAskill, CEO of SmugMug which stores a huge amount of data on S3: http://news.ycombinator.com/item?id=422574. Check his blog too: http://blogs.smugmug.com/don/tag/s3.
Well, I'll try to make up for it by pointing the original asker to http://simplecdn.com
From their frontpage:
A software developer with 100MB worth of downloads would pay $24.90 for lifetime storage, and would then receive a transfer rate of 3.9 cents per GB.
So, pay $24.90 once and serve your 1000 downloads for $5.80 then.
This has already been discussed in some detail at http://news.ycombinator.com/item?id=422225. I suggest anyone interested in talking price/value reads up on that thread before commenting here.
Someone suggested that we use s3. This would've tripled our bandwidth costs.
This is why I believe it's expensive.
I'm not saying you made the wrong decision, or the right one, but is the comparison apples-to-apples?
He seemed to be concerned about bandwidth also, for which S3, frankly, is not cheap.
But then again, that is the kind of stuff that people get paid ridiculous amounts of money to build, heck, I know I did. So I wouldn't bet on it. But if I had the time I would probably do it.
* How many new files do you expect to have uploaded every month? * How many files do you expect to be deleted every month? * What's the average file size? * How many times, on average, do you expect each file to be downloaded per-month?
Use that + Amazon's pricing to figure out how much you'll have to spend on S3 every month. Then compare that to your alternatives.
In my mind, the biggest reason to go with S3 is if you think you are going to end up with a lot more files than you could easily store and mirror on a few leased servers. S3 is a pretty good price for scalable, highly-available storage with a nice fat pipe to the Internet, but the cost per GB transferred isn't great. You can take advantage of the lost cost of storage by keeping track of how often files are downloaded and keeping a cache of locally hosted files that you serve directly off a leased server or VPS.
* How many new files do you expect to have uploaded every month? * How many files do you expect to be deleted every month? * What's the average file size? * How many times, on average, do you expect each file to be downloaded per-month?
Use that + Amazon's pricing to figure out how much you'll have to spend on S3 every month. Then compare that to your alternatives.
In my mind, the biggest reason to go with S3 is if you think you are going to end up with a lot more files than you could easily store and mirror on a few leased servers. S3 is a pretty good price for scalable, highly-available storage with a nice fat pipe to the Internet, but the cost per GB transferred isn't great. You can take advantage of the lost cost of storage by keeping track of how often files are downloaded and keeping a cache of locally hosted files that you serve directly off a leased server or VPS.
* How many new files do you expect to have uploaded every month? * How many files do you expect to be deleted every month? * What's the average file size? * How many times, on average, do you expect each file to be downloaded per-month?
Use that + Amazon's pricing to figure out how much you'll have to spend on S3 every month. Then compare that to your alternatives.
In my mind, the biggest reason to go with S3 is if you think you are going to end up with a lot more files than you could easily store and mirror on a few leased servers. S3 is a pretty good price for scalable, highly-available storage with a nice fat pipe to the Internet, but the cost per GB transferred isn't great. You can take advantage of the lost cost of storage by keeping track of how often files are downloaded and keeping a cache of locally hosted files that you serve directly off a leased server or VPS.
If you care a lot about user experience, go with S3.
There are use cases where AWS is not recommended, having too many machines up and running 24x7 or transferring huge amount of data out of the cloud. For a site with heavy uploads it's even possible to store your data on S3 but serve your clients from a machine hosted outside where data transfer is cheaper.
I have an BingoDisk account that supposedly provides unlimited transfer. If you are just uploading and downloading stuff, it sounds like BingoDisk is ideal. (http://www.bingodisk.com/) I use it for hosting big files, like a 600M SQLite database with information about CPAN modules. It gets quite a few downloads, and nobody has ever told me I'm abusing the service.
(FWIW, I used to use BingoDisk for backups, but Linux's WebDAV support does not work well for backups -- way too much caching going on. S3 turned out to be infinitely more reliable. I don't think this has anything to do with BingoDisk, though, but rather how davfs2 is implemented. This may or may not be a problem for your use case.)
If you don't mind offloading the cost to your users, though, S3 is probably the way to go. I think Amazon will even bill your users for you now.
Because I'm tempted to say you should just try one of these cheapie web hosts and see if that fits your needs. For example, and I have no affiliation with this company other than using them a few times, webhost4life has an account with unlimited* bandwidth and unlimited* storage for $6.95 a month (http://www.webhost4life.com/linux.asp). That's a pretty standard price for linux based hosting.
The disadvantage to these types of deals is that they stuff a bunch of websites all on one server which makes performance slow. But it sounds like you'd only have a 1 or 2 page site so that might not be an issue for you.
* when these providers say unlimited there is often a clause in the fine print that sets a limit. You should check before signing up for one of these services.
In seriousness, I don't know what the price for amazon ec2 works out to be. However, during the development phase, it is too expensive. I believe it will cost you $72 to run an instance for a month, base price. I would suggest something like slicehost if all you need is a low power server instance for development, and then look at ec2.
I pay $29/yr for 15GB space and unmetered bandwidth.
and CloudFront features and It is a FREEWARE