Ask HN: Using S3 for personal data backup?
Hi HN,
I would like to ask whether anyone has experience using S3 for personal data backup (documents, photos, video) and work.
Background: Just experienced an external hard drive failing on me. Traumatic and irritating =)
Hope that you can share how you've set it up (rsync?), some examples of the data you've stored and the costs that you are paying each month.
Luckily for me, Amazon just opened a data centre in Singapore =p
Thank you! Ryan
29 comments
[ 1.4 ms ] story [ 119 ms ] threadDon't backup your data to Singapore even if you are local, it will cost you more (compare costs across S3 DC's) and you don't need backup data to be close to you in the network.
S3 can work out expensive if you want to host a lot of data. Sometimes it can be cheaper to buy one of those 'unlimited' backup accounts with Mozy/etc as after a while the cost will be cheaper than Amazon.
They probably don't have the same amount of redundancy as S3 but then redundancy is a subjective thing.
A backup service might claim that your data is only stored in one place (ie no redundancy) but that it's very stable.
Amazon S3 might be more diverse but it's also volatile - Amazon claim that they cannot guarantee not to loose data.
Now, amazon is pretty expensive if you access the data often... but beating their prices for archive storage (with six copies!) is damn hard. I've been looking at competing with them for a while, and it looks like what I might end up doing is selling space with a significantly lower level of redundancy (2-3 copies) at a lower rate, with a significantly lower network transfer charge.
Even so, the amazon system is really nice.
How I set it up: http://www.tarsnap.com/
Ryan
http://code.google.com/p/brackup/
I don't actually use this, I just followed its development. For my backups, I have a script that spins up an Amazon EC2 instance, mounts an EBS disk and does an old-fashioned rsync of my entire machine.
This does mean I pay for the entire disk all the time every month, plus the EC2 time, and it's probably slower and more expensive than an equivalent brackup, but I also didn't have to think about it.
Right now, the EBS disk isn't bootable, but I could make it so and in theory just boot up an EC2 instance and immediately have my entire machine at the state it was last backed-up as.
What I really need is simplicity that _just works_. I found that in Dropbox and haven't gone back since. I use it to back up ~100Gb of data. On my other machines I just choose not to sync the larger directories and everything works perfectly.
But th best thing is that you can try it for free: it gives you 2GB initially, and an additional 250MB (up to 10GB) for each new user you refer. (E.g. if you apply via this link, I'll get an additional 250MB: http://www.dropbox.com/referrals/NTIxMzc2ODk _hint hint ;)_ )
And I would -not- recommend DropBox for critical backup files. First, they can read all content. Second, they got some problems with large folders or system directories.
That said, I'm only storing my parent's photos and videos of their grandchildren (I'll lose all my media if my apartment is truly wiped out by a tornado, then again I'm unlikely to survive that). If you need to backup a lot of stuff S3 will be quite a bit cheaper (for me either is pocket change as of now).
Linux backups are really easy: Wuala's client uses fuse to mount itself as a file system, so you can simply do periodic tar, or you can let their own backup mechanism back up your stuff to a special folder they maintain perfectly. You can get to your backed up files anywhere in the world via Wuala.com. This feature has saved me gobs of effort a bunch of times already.
The problem with paid services is that if you have even a temporary credit card/billing problem, they may nuke your data. Also, they're probably less reliable than gmail.