Most definitely, I for one don't try many new programs without some sort of screenshot (if you do a screencast also put screenshots up as not everyone can always watch a video or wants to watch a video).
It doesn't work without an S3 account. The total cost is $29 (one time) plus monthly S3 charges. For example, 33GB backed up at S3 would cost $4.95/month in storage charges, plus some (relatively small) transmission and transaction charges. Upload transmission to S3 is free through June 2010.
Good thing somebody brought up the problem of encryption with other backup providers (like Backblaze).
Still, problem is that if you want to back up a lot of data, S3 quickly becomes rather expensive. In my case, storage costs alone for around 550GB that I currently store on Backblaze would run me $82.50 a month. Not exactly small change.
+1 on this. Any decent NAS software (FreeNAS, OpenSolaris, etc) with an iscsi target will work great with the (free) GlobalSAN initiator and Time Machine.
True, but one big "gotcha" to this is that there is no way to do a complete restore from a time machine volume. There is no iscsi initiator on the OS X system DVD (and therefore no way to select your TM backup for a complete system restore).
I was thinking of adding Pogoplug support. That way storage would be much cheaper, although backing up to a Pogoplug at a friend's house may be much less reliable than backing up to S3.
I second the motion. I have a ton of storage on Dreamhost that I could totally be using as backup. (And have tried a bunch of times, but it's a pain getting cron and rsync to do the right thing, and then remembering to set it up again when my computer gets changed around...)
Keep in mind that they don't allow unlimited private backups (they allow 50GB of backups store in a special backup user account), the data you store there is supposed to be web accessible.
The price scaling issue worried me at first, and then I noticed the monthly budgeting — great feature. In the long run, you’ll always sacrifice price (vs. an external hard drive) for off-site data persistence anyway.
In any case, this looks really awesome for documents/projects backup to supplement my TimeMachine drive.
Perhaps he is saying that the per-gig cost is not clear. It says buy now for $29, but the extra $0.15/GB/month to use S3 is hidden under "Why use S3?" (you might not expect costs to be there)
We both know what S3 is, and know that they charge per gig. But the market of people who want online backups for their macs don't necessarily know this.
Lots. Roughly US$112.50 per month. CrashPlan and SpiderOak are better price value, although I might look at using this to have a secondary backup of critical items. Even that, for me, though, is about 200GB (photos & music) or about $30/mo.
I'm a bit biased, but I don't feel S3 is the best possible fit for a frugal consumer's backup data. S3 is a high performance, general purpose solution capable of very low latency.
Having that performance makes it more expensive than an archival specific solution, which tolerates latency just fine as long as high throughput is maintained.
There's a reason why Mozy, Carbonite, SpiderOak, etc. all run our own data centers. I can sell a 100GB account for usable across unlimited computers for $50 per year and be profitable.
That said, the market certainly has plenty of room for more backup choices, and any option to help people avoid data loss is a win. An exclusively Mac app seems like a good idea.
I agree. For me though, backup comes down to trust. I trust Amazon to provide the S3 service reliably. I find it harder to trust Mozy, Carbonite and the rest, partly because they don't even bother to restore my Mac files' metadata correctly.
I don't know how other people do it, but certainly for me, I have things that are at different levels of importance to me. And a lot of the most important are not really all that big, nor are they changing particularly frequently. I wouldn't pay a premium of $100 a month just for the convenience of having everything backed up in this super-redundant, universally-accessible way.
I would happily use a product like Arq (or Time Machine) for my documents and current photos, to make sure the backup is always current, and then whenever the backup hits 50gb, move it over to an off-site drive (at work, a relative's house, even in a safety deposit box).
I have. I chose my own encryption password, but in order to restore files, I had to enter the password into the Backblaze web site. Backblaze used it to decrypt my files on their server and assemble them into a zip file on their server. They then sent me email saying I could download the zip file.
I had Mozy and I used their built-in encryption to backup ~250GB. I went to restore some wedding photos while at my parents' house, so I downloaded their decryption utility, and started the decryption of a 10gb subset of my backup.
Only about 1/3rd of the files decrypted correctly. The rest of the files were totally corrupted -- not cool. I contacted support and we investigated the problem for ~3 months. Luckily I didn't have dataloss, but what exactly was I paying for here?
We never got the problem resolved and I ended up getting a full refund for 1.5 years of service. Make sure to test your backups no matter where you store them.
You were paying for not worrying about whether you would lose your data. And that's exactly what Mozy gave you... at least until you discovered the corruption. :-)
Well, JungleDisk is a monthly service from $2-5 month (plus S3 charges) depending on the plan you choose. Arq is a 1-time $29 cost (plus S3 charges). The other big difference is that JungleDisk is a Java app and doesn't have a native Mac look and feel.
Also Arq is a lot simpler to use. I really need to put up a screencast to show that.
One of backup-bouncer's tests is called "combo-tests" where it tests several
different file metadata types on the same file. Carbonite didn't restore this
file at all.
Instead, Carbonite left a file called "Carbonite_Restore_F161_G1.tmp". It's a
1-line text file that reads "gotta boogie".
As the author of Tarsnap, I'm a bit biased, so I'm going to refrain from comparing the two; but I have one question for the author: Aren't S3 per-request fees a problem? One of the reasons I have all Tarsnap data go through the Tarsnap server is to allow me to "bundle" blocks together and amortize per-PUT costs.
The per-request fees ($.01 per 1000 PUTs and $.01 per 10,000 GETs) aren't a problem for 2 reasons:
1) Arq packs small files (under 64k) together into "packs", a bit like git does. So instead of thousands of PUTs of tiny files (which would also be very slow) it does fewer larger PUTs.
2) Arq caches indexes of those pack files locally, and caches the list of larger S3 objects as well. So it doesn't have to do a GET to figure out whether an object with a given SHA1 is already in the S3 account. (Arq stores every object by its SHA1 hash, like git does).
Arq packs small files (under 64k) together into "packs"...
Ok, so if you have lots of 64 kB files they're all PUT separately, using 15625 PUTs/GB (i.e., $0.15/GB)? I guess that's not too bad.
The file format is open
Nice! Good to see someone who believes in openness. (How about the source code, so that people can confirm that you're doing what the spec says?)
One thing caught my eye glancing through that document, though: "The name of each blob is its SHA1 hash." That's the SHA1 hash of the encrypted data, right? (If not, it definitely should be!)
Ok, so if you have lots of 64 kB files they're all PUT separately, using 15625 PUTs/GB (i.e., $0.15/GB)? I guess that's not too bad.
I guess I could ask you to choose the threshold instead of picking 64k, but generally I've tried to choose reasonable defaults so that the user doesn't have to make those kinds of decisions.
Good to see someone who believes in openness. (How about the source code, so that people can confirm that you're doing what the spec says?)
Well, you could actually write your own code to read the files, since the format is open.
That's the SHA1 hash of the encrypted data, right?
It's the SHA1 of the contents of the object, and the object is encrypted data, so yes.
>Well, you could actually write your own code to read the files, since the format is open.
I think the parent's point was not vendor lock-in but rather, that publishing the source code to your particular implementation would lend more credibility to it since anyone can inspect it and verify that it does what it says it does.
Right, but with Arq, unlike most online backup offerings, you have direct access to the files being stored online (they're in your S3 account). So you can verify that Arq says what it does because you can look at the results.
I have been using Arq for the last 3 weeks. It's very basic software at the moment, but it does the job. It lets me see file revisions and restore files as necessary. Unfortunately these files can't be seen directly in S3 due to the packing nature of how they're stored.. so if your mac crashes and you want a file, you'll need to find another mac and install Arq to retrieve your files.
Right. The files are named using the SHA1 hash of their contents (to enable de-duplication) and small files are packed together (to dramatically improve performance). I thought that was the best solution given the limits of the S3 API. JungleDisk does something similar I think.
What sort of alternative for restoring would you be interested in? Maybe an open-source command-line utility?
Check out Quillen for snapshot backups directly to S3. Simple command-line interface, easy for scripting. Also open source. http://quillen.googlecode.com/
Yes, you can set a maximum transfer rate. You can also choose "automatic transfer rate" which slows backups when you're using the Internet interactively.
52 comments
[ 2.9 ms ] story [ 102 ms ] threadI don't want to commit to downloading this app without seeing how it functions. I understand it lets me back up to S3, but how?
I dare say if you add these images, your conversion rates will skyrocket.
Assuming this is your site, anyway.
It worked for me :)
Still, problem is that if you want to back up a lot of data, S3 quickly becomes rather expensive. In my case, storage costs alone for around 550GB that I currently store on Backblaze would run me $82.50 a month. Not exactly small change.
Would anyone be interested in this?
http://www.dreamhoststatus.com/2007/10/17/policy-clarificati...
In any case, this looks really awesome for documents/projects backup to supplement my TimeMachine drive.
Maybe I should call out that use case on the landing page.
We both know what S3 is, and know that they charge per gig. But the market of people who want online backups for their macs don't necessarily know this.
Having that performance makes it more expensive than an archival specific solution, which tolerates latency just fine as long as high throughput is maintained.
There's a reason why Mozy, Carbonite, SpiderOak, etc. all run our own data centers. I can sell a 100GB account for usable across unlimited computers for $50 per year and be profitable.
That said, the market certainly has plenty of room for more backup choices, and any option to help people avoid data loss is a win. An exclusively Mac app seems like a good idea.
I would happily use a product like Arq (or Time Machine) for my documents and current photos, to make sure the backup is always current, and then whenever the backup hits 50gb, move it over to an off-site drive (at work, a relative's house, even in a safety deposit box).
Also the Mac metadata weren't restored properly: http://www.haystacksoftware.com/arq/backblaze-backup-bouncer...
Only about 1/3rd of the files decrypted correctly. The rest of the files were totally corrupted -- not cool. I contacted support and we investigated the problem for ~3 months. Luckily I didn't have dataloss, but what exactly was I paying for here?
We never got the problem resolved and I ended up getting a full refund for 1.5 years of service. Make sure to test your backups no matter where you store them.
You were paying for not worrying about whether you would lose your data. And that's exactly what Mozy gave you... at least until you discovered the corruption. :-)
Also Arq is a lot simpler to use. I really need to put up a screencast to show that.
One of backup-bouncer's tests is called "combo-tests" where it tests several different file metadata types on the same file. Carbonite didn't restore this file at all.
Instead, Carbonite left a file called "Carbonite_Restore_F161_G1.tmp". It's a 1-line text file that reads "gotta boogie".
1) Arq packs small files (under 64k) together into "packs", a bit like git does. So instead of thousands of PUTs of tiny files (which would also be very slow) it does fewer larger PUTs.
2) Arq caches indexes of those pack files locally, and caches the list of larger S3 objects as well. So it doesn't have to do a GET to figure out whether an object with a given SHA1 is already in the S3 account. (Arq stores every object by its SHA1 hash, like git does).
The file format is open (it's your data after all): http://www.haystacksoftware.com/arq/s3_data_format.txt
Ok, so if you have lots of 64 kB files they're all PUT separately, using 15625 PUTs/GB (i.e., $0.15/GB)? I guess that's not too bad.
The file format is open
Nice! Good to see someone who believes in openness. (How about the source code, so that people can confirm that you're doing what the spec says?)
One thing caught my eye glancing through that document, though: "The name of each blob is its SHA1 hash." That's the SHA1 hash of the encrypted data, right? (If not, it definitely should be!)
I guess I could ask you to choose the threshold instead of picking 64k, but generally I've tried to choose reasonable defaults so that the user doesn't have to make those kinds of decisions.
Good to see someone who believes in openness. (How about the source code, so that people can confirm that you're doing what the spec says?)
Well, you could actually write your own code to read the files, since the format is open.
That's the SHA1 hash of the encrypted data, right?
It's the SHA1 of the contents of the object, and the object is encrypted data, so yes.
I think the parent's point was not vendor lock-in but rather, that publishing the source code to your particular implementation would lend more credibility to it since anyone can inspect it and verify that it does what it says it does.
No matter how many tests you do, you can never catch all of these.
What sort of alternative for restoring would you be interested in? Maybe an open-source command-line utility?
http://sreitshamer.github.com/arq_restore/