Last time I checked the deduplication only works per host when backups are encrypted, which makes sense. Anyway, borg is one of the three backup systems I use, it's alright.
I once met the Borg author at a conference, pretty chill guy. He said that when people file bugs because of data corruption, it's because his tool found the underlying disk to be broken. Sounds quite reliable although I'm mostly fine with tar...
Restic is far better both in terms of usability and packaging (borgmatic pretty much is a requirement for usability). Have used both extensively, you can argue that borg can just be scripted instead and is a lot more versitile, but I had a much better experience with restic in terms of setup and forget. I am not scared that restic will break, with borg I did.
Also not sure why this was posted, did a new version release or something?
I’ve been looking at this project occasionally for more than four years. The development of version 2.0 started sometime in April 2022 (IIRC) and there’s still no release candidate yet. I’m guessing that it’ll be finished in a year from now.
What are the current recommendations here to do periodic backups of a NAS with lower (not lowest) costs for about 1 TB of data (mostly personal photos and videos), ease of use and robustness that one can depend on (I know this sounds like a “pick two” situation)? I also want the backup to be completely private.
Borg 1.x will still be fine for your needs, especially if you only have a single machine that you want to back up. The main advantage for Borg 2.x will bethat multiple machines can backup to the same repository and de-duplication will work across all of them.
There are plenty of storage server providers where you can get ssh access and 1-2TB for a few dollars per TB per month. You can run multiple repositories from a single server.
As the data is encrypted, even if the storage server is compromised, your data can't be read by others without the key.
I'll die on this hill... If may files that are named like this:
DSC009847.JPG
were actually named like this:
DSC009847-b3-73ea2364d158.JPG
where "-b3-" means "what's coming before the extension are the first x bits (choose as many hexdigits as you want) of the Blake3 cryptographic hash of the file...
We'd be living in a better world.
I do that for many of my files. Notably family pictures and family movies, but also .iso files, tar/gzip'ed files, etc.
This makes detecting bitflips trivial.
I've create little shellscripts for verification, backups, etc. that work with files having such a naming scheme.
It's bliss.
My world is a better place now. I moved to such a scheme after I had a series of 20 pictures from vacation with old friends that were corrupted (thankfully I had backups, but the concept of "determining which one is the correct file" programmatically is not that easy).
And, yes, it detected one bitflip since I'm using it.
I don't always verify all the checksums: but I've got a script that does random sampling... It picks x% of the files with such a naming scheme and verifies the checksum of these x% of files picked randomly.
It's not incompatible with ZFS: I still run ZFS on my Proxmox server. It's not incompatible with restic/borg/etc. either.
This solves so many issues, including the "How do you know your data is correct?" (answer is: "Because I've already looked that family movie after the cryptographic hash was added to its name").
Not a panacea but doesn't hurt and it's really not much work.
"Baqpaq takes snapshots of files and folders on your system, and syncs them to another machine, or uploads it to your Google Drive or Dropbox account. Set up any schedule you prefer and Baqpaq will create, prune, sync, and upload snapshots at the scheduled time.
"Baqpaq is a tool for personal data backups on Linux systems. Powered by BorgBackup, RSync, and RClone it is designed to run on Linux distributions based on Debian, Ubuntu, Fedora, and Arch Linux."
I've been using the Vorta GUI [0] and Hetzner's Storage Box service for ages and it works great. Has saved me from some headaches.
I switched over from Duplicati a long while back when my laptop's sole HDD failed and Duplicati was giving me 143 year estimates for the restore to complete. This was true whether I aimed to restore the whole drive or just a single file.
20 comments
[ 11.4 ms ] story [ 49.7 ms ] threadAlso not sure why this was posted, did a new version release or something?
What are the current recommendations here to do periodic backups of a NAS with lower (not lowest) costs for about 1 TB of data (mostly personal photos and videos), ease of use and robustness that one can depend on (I know this sounds like a “pick two” situation)? I also want the backup to be completely private.
There are plenty of storage server providers where you can get ssh access and 1-2TB for a few dollars per TB per month. You can run multiple repositories from a single server.
As the data is encrypted, even if the storage server is compromised, your data can't be read by others without the key.
We'd be living in a better world.
I do that for many of my files. Notably family pictures and family movies, but also .iso files, tar/gzip'ed files, etc.
This makes detecting bitflips trivial.
I've create little shellscripts for verification, backups, etc. that work with files having such a naming scheme.
It's bliss.
My world is a better place now. I moved to such a scheme after I had a series of 20 pictures from vacation with old friends that were corrupted (thankfully I had backups, but the concept of "determining which one is the correct file" programmatically is not that easy).
And, yes, it detected one bitflip since I'm using it.
I don't always verify all the checksums: but I've got a script that does random sampling... It picks x% of the files with such a naming scheme and verifies the checksum of these x% of files picked randomly.
It's not incompatible with ZFS: I still run ZFS on my Proxmox server. It's not incompatible with restic/borg/etc. either.
This solves so many issues, including the "How do you know your data is correct?" (answer is: "Because I've already looked that family movie after the cryptographic hash was added to its name").
Not a panacea but doesn't hurt and it's really not much work.
https://man.openbsd.org/mtree
"Baqpaq takes snapshots of files and folders on your system, and syncs them to another machine, or uploads it to your Google Drive or Dropbox account. Set up any schedule you prefer and Baqpaq will create, prune, sync, and upload snapshots at the scheduled time.
"Baqpaq is a tool for personal data backups on Linux systems. Powered by BorgBackup, RSync, and RClone it is designed to run on Linux distributions based on Debian, Ubuntu, Fedora, and Arch Linux."
At: https://store.teejeetech.com/product/baqpaq/
Though personally I use Borg, Rsync, and some scripts I wrote based on Tar.
Not affiliated, just a happy user.
I switched over from Duplicati a long while back when my laptop's sole HDD failed and Duplicati was giving me 143 year estimates for the restore to complete. This was true whether I aimed to restore the whole drive or just a single file.
https://vorta.borgbase.com/
https://plakar.io/
We are doing our best to complete existing solutions :)