Ask HN: What is your startup's backup policy?

I do daily backups, automatically, using my own script which is a paid solution; my data is encrypted (and signed), and I back up complete filesystems minus easily re-downloadable stuff (e.g., the FreeBSD ports tree), to one location (unless you mean geographic location, in which case tarsnap counts as 2 locations).

For customer data, we do daily automated encrypted backups to S3.

We also do more frequent backups to an internal server during the day.

Our code is in SVN, which is also backed up to S3.

We have scripts that take a vanilla etch build and prepare it for production. That way we don't have to worry about backing up the OS.

I rely partially on backups and partially on wide distribution of key data.

There are at least 5 accessible copies of my source code at any moment, for example: my laptop, my server (source control), my server's backup from yesterday, my server's backup from a week ago, and my per-release In Case Shit Happens tarball chilling on a Google server farm somewhere.

Which means if my apartment burned down on the same day that Slicehost's colo facility got hit by a meteor, I would still have a copy left.

Customer records? Important but not as important as source code. I have five redundant copies of every transaction: one at the payment processor, one at e-junkie.com, and 3 in my database and its backups.

Email? Much less important to me. I pay Google to worry about it and assume they will be competent.

Analytics data? Nice to have, wouldn't cry too much if I lost it all tomorrow. Mine gets backed up as a side effect of backing up the other stuff. Google Analytics and Clicky also keep historical data for me, and I rely on them to make sure it doesn't vanish.

None of this is encrypted (except to the extent my service providers do it -- Paypal certainly does, for example). I don't store customer billing information, so the most sensitive data I have is a list of names. (Source code? Pfft. Very little of my business value is in the source code.)

We backup all data to S3 every 30 minutes. We are a bit paranoid about data loss as we host business critical data.

I use time machine to back up data on my mac os x box. I haven't launched yet but if I did I would create a mirror on S3 since I am going to use EC2.

Anecdotally, it seems like S3 is pretty popular here for backup purposes. Does anyone care to enumerate any pros/cons they've experienced?

i use mercurial for everything (code, images, generated html/jpg, etc) ... maybe i should put some in .hgignore especially the generated bits

so there are at least 2 copies (colo and desktop) ... and dvd

i do still tar on structural disruptions (jquery update, openbsd upgrade, data structure change, etc)

that's per site, every morning, semi-automatic (i still prefer to ssh and then manually hg update shrug)

rsnapshot (http://rsnapshot.org/) against all of our openbsd servers over the internet (via ssh) to an off-site machine. runs every few hours and keeps 6 hourly, 7 daily, 4 weekly, and 3 monthly backups. once a server's initial full rsync is done, the incrementals finish very quickly even on servers with lots of changing data (mail server messages, web server logs, etc.)

regular mysql dumps are taken from all databases on all servers every so often in case the rsync'd mysql binary files won't restore.

For the S3 users here, if you don't mind sharing: how do you actually do it? Do you use duplicity, Jungledisk, s3rsync.com, the s3sync utilities, tarsnap, something homegrown? Do your backups require incremental updates, snapshots, encryption?

  Manifest-Version: 1.0
  Created-By: IntelliJ IDEA
  Main-Class: com.threebanana.S3Backup

Server: iSCSI cross-country replication, snapshots (automated)

Desktop: Time Machine (hourly; automated)

All code is hosted at GitHub; I just sort of assume the EngineYard people know how to back stuff up. Google for e-mail, though it is locally mirrored via IMAP.

I backup to S3 using bacula and an s3sync wrapper which I made called s3-backup.py

Oh, and for my purposes I do daily incrementals and monthly fulls.

MySQL: replication across VPN to server in a different state. Week's worth of daily full backups (innobackup), compressed, encrypted, periodically sent to S3. As much binlog history as we have disk to keep.

MogileFS: real-time encrypted backup to S3 (in addition to multiple local copies). This covers all of our blob-like data, keeps MySQL relatively small.

Systems: logs nightly synced to S3, otherwise no data to back up. One puppet script away from reconstruction.

Email: local server rsync in addition to procmail failsafe copies.

Code: svn master encrypted and sent to S3 nightly.

We use boto to access Amazon services.

I do an automatic daily dump of all databases and an automatic rsync of the dump and all code, git repository + other files to a disk in a different location. Surely this could be improved upon, but I think it's a good start.

Nightly backups to S3 and to our in-office dev machine. We also all have copies of the code base that we are constantly pulling from Git, so the only thing to really worry about is DBs.

for in-progress code and data, we use Dropbox for everything, with two geographically segmented local caches. recovery takes only a few minutes.

we keep local (on our web box), networked (NFS) and S3 backups of a) our entire server build, b) our code base, c) our database dumps, d) our email & configs.

we backup nightly (we backup at the "quietest" period of last nights activity on the subsequent night)

on our local machine we can keep: yesterdays backup, day before yesterday, end of last week, and start of month backups.

it's entirely automated, except for the bi-monthly backup check (I CANNOT STRESS THIS ENOUGH - CHECK BACKUPS!) where we fully restore our data and code on a spare box.

by code i mean live code - we also backup our git working code base.

I put everything in git repositories and push changes to a clone on my slicehost slice.

when i first read the title three meanings came to mind:

1. how do you back up data?

2. how do you back up 'progress' (eg, code commits, releases)?

3. what is your plan B (eg, this startup failed, what do i do next)?

maybe it's just me, but verbal communication is funny.

Our policy is outsource the IT to WebFaction. They RAID our data and back it up daily. Thanks WebFaction!

If you are a start-up that doesn't have serious SysAdmin talent on-board, the only logical call is to outsource every machine possible. Even if you are ludicrously good hackers, you need to have a true Linux money, who isn't needed elsewhere coding, to consider not outsourcing web and email hosting. And you should back-up anything important on your personal computers to a managed box.

I really can't say how deeply I believe this. I've worked with multiple start-ups, and the minute you start trying to host your own website, do your own backups, serve your own emails - you're asking for trouble. Even if you set up everything right, just the time dealing with the computers could be better spent changing the world

Whatever you do, don't host actual email boxes in house. A telephone pole will get hit by a truck, and you will lose email for two days, and no amount of tech smarts will get you out of that. Trust me, it happens.

You can get a slice of a computer for $10 bucks a month these days from Webfaction - which comes with tech support that extends from installing PostGIS, to an optimized PostGres install, an optimized Apache static content server, and answers to tech support within the hour, even on the weekends. As soon as a machine you manage costs you an hour, you're financial decision has become a poor one.

Also, they have better security than you.

GitHub + Dropbox

We backup a complete snapshot of our customer data every hour to S3, via a Ruby cronjob.

We should be able to keep up this strategy because it's "just" a time tracking app (http://letsfreckle.com/) with fairly lightweight data.

I was originally worried that once we got into production, this'd cause a performance drop every time it ran, but so far it's all good.

As for our codebase, we're on github and our own 4 laptops. But thinking about it, we should have another system in place for that. looks around nervously

Daily + hourly + monthly backups to S3, my local machine, and a USB flash drive that I keep in my wallet at all times:

http://www.amazon.com/KingMax-Microsoft-Certified-Drive-Wash...

Source code is on a mix of Github and Subversion servers

Customer data (basically, SQL data) is backed up daily, archived, and put up encrypted on Amazon S3 with the date/time appended to it. The files are tiny so we have no issue with keeping year-old ones there as the costs are minuscule.

All other things like documents, staff related items, pictures of products are stored on a single server, that uses rdiff-backup and sends it to a second drive on the same pc. rdiff-backup is very nice as we've had a few instances where we needed to fetch a file from 2 months ago that was heavily modified since. The version history helps a lot

I do automated backups of the dev environment nightly to an external hard drive and to an ftp location off site.