I've been using tarsnap for years and am in the process of migrating away from it.
Things that are not cozy:
1) There's no way to monitor your monthly spend per host/credit left on the account/etc. apart of logging into your account in a browser and manually keeping a spreadsheet. There's no web API to do it. You get an email warning when you have about 7 days of credit left. That's it.
2) Nothing is "a precious few megabytes" anymore. What seems like a negligible monthly spend at first can quickly grow up on you and soon you're spending highly non-trivial amounts. Which you might not notice due to 1) unless you are diligent in your accounting.
3) tarsnap restores are slow. Really really slow. A full restore can take days if you have non-trivial amounts of data (and make sure you have enough credit in your account to pay for that server-to-client bandwidth!) My understanding is that throughput is directly related to your latency to the AWS datacenter where tarsnap is hosted. Outside of north America you can be looking at nearly dial-up speeds even on a gigabit link.
Again, a problem that can surprise you at the most inconvenient time. Incremental backups in a daily cronjob tend to transfer very small amounts of data, so you won't notice the slowness until you try to do a full restore. And you generally don't test that very often because you pay for server-to-client transfers.
There are some workarounds for 3) and there's a FAQ about it, but look at the mailing list and you'll see that it's something that surprises people again and again.
I used tarsnap for years, but as my data got bigger and I really wanted to have multipe offsite backups with different providers, I moved to restic.
I loved tarsnap - it's a great product. But restic feels very similar but you can backup to your local HD, a remote HD, or "the cloud" and everything is the same CLI commands.
I also switched away from Tarsnap because I needed to restore my personal PDF collection of like 20GB once and my throughput was like 100Kb/s, maybe less. It has been a problem for at least a decade, with no fix in sight.
I'm carefully monitoring plakar in this space, wondering if anyone has experience with it and could share?
I really wanted to like Tarsnap and gave it a good hard look for my backup needs. Ultimately my problem was that there's no way for me to gauge how much the service will cost me. Going just by the amount of data in my home dir, it would be cost prohibitive to upload to Tarsnap. The site does assure me that thanks to compression and deduplication, the actual cost will be far less than I might estimate, which is great! But also, as far as I can tell there's no way to have the client give me an estimate of "here's how much data you actually have once the secret sauce is applied". So while the dedup and compression might make the costs far more reasonable, I won't actually know until I pay to store some data. Which means I might find that suddenly I owe Colin a lot of money if the size savings aren't very big due to my data not being very amenable to those measures. That's not a risk I'm willing to take, so ultimately I pursued other options.
will tell you the compressed size of your deduplicated data, which gives you the upload cost and first-month cost. 4GB of files usually works out to 3GB of dedup/compressed archive data for most people, less for people with many similar files.
OP's cost estimator tells me it would cost a cool $250 per month to keep a terabyte of data backed up in Tarsnap. The same amount costs me $8.25 per month with Backblaze. That's not very cozy!
This sounds cool, but the other comments here are concerning. I've been considring Hetzner's Storage Box, as it's cheap and I could use just about anything to backup my stuff – although I prefer restic.
I'm really surprised to hear that the slow restore times from tarsnap are still as big a problem now as they were a decade ago when I last used it. I absolutely loved the interface and the security model, and I was willing to pay at the (very) premium price point, but it was just too impractical trying to restore anything from it at the speeds I could achieve. (If I remember right, there was some problem with the design which meant normal latency between the client and the server tanked throughput to crazily low levels.)
The interface is nice but it is excruciatingly slow at restores. I had to switch to borg, which has an extremely similar feature set but performs much better (and you can bring your own hardware).
I've been using Tarsnap for almost a decade for a small, but very important, personal data subset.
Tarsnap is very resilient; it doesn't do a lot, but what it does is solid. The mailing list is helpful, and you can reach out to its creators directly for prompt, useful responses if that is something you don't want on the mailing list (where names and email addresses are in the clear; use marc.info to search in it).
But if you are trying to start with Tarsnap, you should note a few things from the beginning:
- If you are looking for a completely (or even almost) frictionless backup experience - this is not it. Also, it doesn't have tons of features - which might be a good thing, but you should know and accept it.
- If you're used to tools like Backblaze, CrashPlan, Restic, or Borg, the limited feature set might frustrate you.
- Knowing this in advance will help you set expectations within its feature set. The doc/man pages are great resources once you actually read it.
- It has some quirks (may or may not be bugs) that require tinkering with your settings, env etc. Getting your hands dirty with sample data first is a great way to know Tarsnap.
- Set up your logs and scripts such that you can know/debug things later.
- Naming of your archives is important.
- You'll need at least two keys: a master key with read, write, and delete access on your archives/Tarnsap storage, and a un-passphrased regular key with only "write" permission for backups. Keep both safe, especially the master key. There's "nuke" as well…
- I used its GUI for the longest time but would absolutely not recommend it. It hides a lot, which might come back to bite you, and is not the most polished tool of all. Its last release was 7 years ago.
OP says:
> … If you use it solely to back up the few megabytes of “crown jewels” data we all have lying around"
and I actually use Tarsnap exclusively for my "crown jewels," which are in the early three digit MBs.
- So, unlike what many say, I do believe it is costly for today's storage/bw prices, especially if your data isn't very compressible. Tarsnap's compression is great, but not magic. However, i doesn't cost an exponential bomb either. Killer de-dupe though.
- You must have a plan for what and how much you want to back up, and the expected growth of that data.
- It is definitely not a "fire and forget" tool (and you should never forget your backups anyway).
I was frustrated with it until I gave up on the GUI, embraced the CLI/cron, reduced the amount of data being backed up and excluded (using copy and delete) some data being stored, and accepted what it can't do. Which is not really great but that's what it is.
Glaring omissions, IMHO: very few maintenance features (the scripts listed are not easy to work with), (almost?) no way of knowing what file changed in a certain archive, slow restores (may matter for a bigger data set), and the lack of an updated, polished GUI tool which I think is very important for personal data backup.
My request to cperciva would be: please consider this - while it's inspired by tar and stays close to it, it's also a cloud backup tool. Treating it a bit more like a modern cloud backup tool could be useful. Just my two cents.
22 comments
[ 3.6 ms ] story [ 44.7 ms ] threadThings that are not cozy:
1) There's no way to monitor your monthly spend per host/credit left on the account/etc. apart of logging into your account in a browser and manually keeping a spreadsheet. There's no web API to do it. You get an email warning when you have about 7 days of credit left. That's it.
2) Nothing is "a precious few megabytes" anymore. What seems like a negligible monthly spend at first can quickly grow up on you and soon you're spending highly non-trivial amounts. Which you might not notice due to 1) unless you are diligent in your accounting.
3) tarsnap restores are slow. Really really slow. A full restore can take days if you have non-trivial amounts of data (and make sure you have enough credit in your account to pay for that server-to-client bandwidth!) My understanding is that throughput is directly related to your latency to the AWS datacenter where tarsnap is hosted. Outside of north America you can be looking at nearly dial-up speeds even on a gigabit link.
Again, a problem that can surprise you at the most inconvenient time. Incremental backups in a daily cronjob tend to transfer very small amounts of data, so you won't notice the slowness until you try to do a full restore. And you generally don't test that very often because you pay for server-to-client transfers.
There are some workarounds for 3) and there's a FAQ about it, but look at the mailing list and you'll see that it's something that surprises people again and again.
I'm carefully monitoring plakar in this space, wondering if anyone has experience with it and could share?
will tell you the compressed size of your deduplicated data, which gives you the upload cost and first-month cost. 4GB of files usually works out to 3GB of dedup/compressed archive data for most people, less for people with many similar files.
Also cozy if your data fits. No monthly fee, just the cost of new/recycled thumbies
https://www.hetzner.com/storage/storage-box/
I'm backing up about 8TiB of data nightly using BorgBackup[0] + InterServer[1] and pay $240/yr.
This gives me differential encrypted rotating backups that are 100% mine and do not lock me into any specific storage vendor.
[0] https://borgbackup.readthedocs.io/en/stable/
[1] https://www.interserver.net/storage/
And Restic is good quality software.
Tarsnap is very resilient; it doesn't do a lot, but what it does is solid. The mailing list is helpful, and you can reach out to its creators directly for prompt, useful responses if that is something you don't want on the mailing list (where names and email addresses are in the clear; use marc.info to search in it).
But if you are trying to start with Tarsnap, you should note a few things from the beginning:
- If you are looking for a completely (or even almost) frictionless backup experience - this is not it. Also, it doesn't have tons of features - which might be a good thing, but you should know and accept it.
- If you're used to tools like Backblaze, CrashPlan, Restic, or Borg, the limited feature set might frustrate you.
- Knowing this in advance will help you set expectations within its feature set. The doc/man pages are great resources once you actually read it.
- It has some quirks (may or may not be bugs) that require tinkering with your settings, env etc. Getting your hands dirty with sample data first is a great way to know Tarsnap.
- Set up your logs and scripts such that you can know/debug things later.
- Naming of your archives is important.
- You'll need at least two keys: a master key with read, write, and delete access on your archives/Tarnsap storage, and a un-passphrased regular key with only "write" permission for backups. Keep both safe, especially the master key. There's "nuke" as well…
- I used its GUI for the longest time but would absolutely not recommend it. It hides a lot, which might come back to bite you, and is not the most polished tool of all. Its last release was 7 years ago.
OP says:
> … If you use it solely to back up the few megabytes of “crown jewels” data we all have lying around"
and I actually use Tarsnap exclusively for my "crown jewels," which are in the early three digit MBs.
- So, unlike what many say, I do believe it is costly for today's storage/bw prices, especially if your data isn't very compressible. Tarsnap's compression is great, but not magic. However, i doesn't cost an exponential bomb either. Killer de-dupe though.
- You must have a plan for what and how much you want to back up, and the expected growth of that data.
- It is definitely not a "fire and forget" tool (and you should never forget your backups anyway).
I was frustrated with it until I gave up on the GUI, embraced the CLI/cron, reduced the amount of data being backed up and excluded (using copy and delete) some data being stored, and accepted what it can't do. Which is not really great but that's what it is.
Glaring omissions, IMHO: very few maintenance features (the scripts listed are not easy to work with), (almost?) no way of knowing what file changed in a certain archive, slow restores (may matter for a bigger data set), and the lack of an updated, polished GUI tool which I think is very important for personal data backup.
My request to cperciva would be: please consider this - while it's inspired by tar and stays close to it, it's also a cloud backup tool. Treating it a bit more like a modern cloud backup tool could be useful. Just my two cents.