Ask HN: Theory of Backups
There seems to exist, buried underneath the superficial & the common sense, theory on how to do backups well.
I've found two elements upon which better theory concerning rotatioms & other details (EG hash verification scheduling, amount of different devices) can be built.
The first is the Tower of Hanoi scheduling scheme, which we will abbreviate TOH.
The second is the Incremental-Differential-Full backups concept, which we will abbreviate IDF.
The best available resource seems to be the Acronis websites' illustrated docs: http://acronis-backup-recovery.helpmax.net/en/understanding-acronis-backup-recovery-10/tower-of-hanoi-backup-scheme/. I request that you in good faith ignore Acronis is a company selling commercial Windows software; you are free to post better links in comments were you to find better info elsewhere.
We end up with a scheme we can call IDF-TOH. In it we have three types of backups:
- Incremental, at L_0 ("Level A" in the linked resource), the most frequent level, capturing only changes made since the last backup.
- Differential, at each level that belongs to the closed interval L_0-L_n, capturing changes made since the last full backup.
- Full, at L_n, the least frequent level, capturing the whole system to be backed up.
So now, what can we do? At least the following directions could be taken in further developing a Theory of Backups:
0 (backup scheduling): The frequencies can be chosen in many ways, and I am not sure which one is most optimal. Tower of Hanoi is for every level L_a, where a belongs to closed interval 0 ... n, 2^a. Frame-Steward may or may not be of any use in this.
1 (rotations): IDF-TOH does not address the problems of rotations. IE: if you make a backup that corrupts your previous data, and then repeat the mistake, you get in trouble quick. It's ALSO noteworthy that certain mediums may better fit certain layers in IDF-TOH & the future schemes. At least, for example, adrian_b three days before this wrote:
"... Of all the optical discs that have been available commercially, those with the longest archival time were the pressed CD-ROM with gold mirrors, where the only degradation mechanism is the depolymerization of the polycarbonate, which could make them fragile, but when kept at reasonable temperatures and humidities that should require many centuries..."
Consequently these would be the best for the Full backups, while Solid State Drives may work for the Incremental ones.
2 (perfecting IDF): The IDF scheme may not be perfect either & can probably be refined more or less.
3 (hashing): Verifying the backups matters & should be a part of a complete scheme.
---
This may not be valuable for all businesses but most invididuals already using rsync or borg would probably prefer to use the best available scheme if reduces probability of incidental data loss at minimal effort. The task of translating the best possible scheme to a config program with humane interface is an undertaking of its own.
21 comments
[ 5.6 ms ] story [ 62.6 ms ] threadThe other normal backups are usually managed by someone else, he just does the hardware, most of the time.
His backups are tested by experience.
I realized today that the mechanism, given multiple devices, where in an IDF one does not always backup from "target" to "drive" but occasionnally from "drive" to "drive" could unlock new possibilities.
Regardless, I would mention IDF to your friend, with the note that he'd have to use backup software that encrypts for it because encrypted LUKS images don't tell their contents and thus can't be used in IDF. This actually leads to the last point about interfaces and tools in my post: it seems that to gain the efficiency of IDF a tool must at least offer encryptation.
Consequently, we get yet another theoretical question: How should the cryptography in IDF-TOH be implemented?
I mean: making F 16 words, D 8 subwords of F, I 4 subwords of F, would hugely improve security!
…but I never delete because the more copies of the same thing there are, the more likely it will survive. If in fact I need it, time spent searching is far shorter than tedious backup procedure.
In addition, if I have to recreate something version 2 will be better because I keep getting better at the things I do.
But that is me not you. Good luck.
1. How long should you keep backups for - is the content of your backup covered by privacy laws that require you to not have copies of it after a certain period of time? is there a point where the content of your back up is so old that it's the logical equivalent of not having made a back up in the first place?
2. How much does your backup process cost - if it costs more to back up a system than it would cost you if you lost it, then you've got the backup process wrong (interestingly this can be affected by economies of scale)
3. What do you need to restore a backup - does your system requires bespoke hardware that might have been lost in whatever disaster you're trying to recover from?
Regarding the cost, I think that a backup solution is always much cheaper than the cost of data loss. So it's better to have a backup than not.
And, of course, restore and recovery checks are also important. Moreover, a backup solution should foresee different disaster scenarios. for example, disaster scenarios like those described here in the article: https://gitprotect.io/blog/github-restore-and-github-disaste...
Regarding backup scheduler - sometimes companies need to have frequent backups due to their RPOs and RTOs, for example, if they operate in a highly regulated industry. If someone can tolerate the loss of data of two hours, then, they need to have backup performed every 2 hours, if we speak here about 8 hours (working day), so why not to have backups on a daily basis?
Regarding rotations - everything depends on a backup solution, if it provides with immutable backups, so the entire data won't be corrupted. Thus, the faster someone notices the mistake, the faster they can restore their copy. IDF helps more to decide the issue with storage - not to overload it (here also worth mentioning deduplication and compression).
IDF-TOH will improve security if implemented in this manner: https://news.ycombinator.com/item?id=41179989
I realized writing this that it absolutely makes sense having different mediums do different jobs, since the I and D drives will be consumed at different rate too. This begs for further analysis because though we can obviously say that SSDs will be better for something that's accessed constantly the problem of which computer hardware to use given these parameters on use is ~solved.
I personally use the following backup strategy:
- Setup an encrypted ZFS Storage in the network (e.g. TrueNAS - in my case it is Proxmox)
- Enable zfs-auto-snapshot for 15 min snapshots auto rotation (keep 24 daily, etc.)
- NEVER (!) type in the passwords of ZFS Storage permitted users on any client, that could be affected by ransomware
- Provide a user authenticated samba share to store all important data - try to prevent local storage of data
- Sync the ZFS snapshots to an external USB drive every night (I use a tasmota shelly plug and an external usb case to power off the devices if they are not needed)
- On Windows and macOS, backup the OS on an external drive- Use restic to keep an additional copy of the local files and folders somewhere else
- Use a bluray burner to backup the most important stuff as a restic repository or encrypted archive (like very important documents, the best photo collections of you family, Keepass database, etc.) and put it to another location
- If cloud storage is affordable for the amount of data you have, consider using restic to store your stuff in the cloud
- From time to time try to restore a specific file from the backup and check if it worked and try to restore a full system (on an additional harddisk).
This may sound overkill, but ransomware is a pretty bad thing these days, even if you think you are not one of its targets.
I believe that any ransomware either (1) exists inside a backup and is thus removable (2) would be accounted bh IDF-TOH.
Actually, your comment also confuses me. By definition of a non-cloud backup, it follows that everything is not always online or accessable by a CPU. Whatever may the case be, your approach would be "practical" and not "theoretical" in that it does not seem to make any fruitful contribution to the problem I proposed.
Theoretical approach would be to create a general semi-formal model of the (1) problem; EG scope, variables, failures of previous solutions (2) foundation, IE more or less what I call IDF-TOH as the first iteration of a solution (3) separate discussion for preparing for the future practical implementations of the "final" technical solution, EG integrations with what exists out there, possibly gathering ideas for how to make the interface human-friendly.
Would you think an IRC, mailing list etc. would be good place – kn change of tactics – to stimulate discussion & research on this?
However, Ransomware these days can:
So even if you do a "cloud" backup and have the credentials on your system to do "convenient" backups, it may just delete them.A way to overcome this is to have an "append only" backup, so that the older backups are secure. I'm not sure you covered this, but as I already stated: I'm not a researcher.
This would use Bayesian heuristics, openly available data, user's personal requirements.
Non-technical granny can press the buttons, it gives them a printable PDF-sheet that says keep three copies of this; these are your best possible backup rules.
For a good design, the problem of changing requirements must be addressed. By mapping every scenario imaginable and implementing support for the transitions, usability would increase significantly.
This'd answer your problem – I'd call it paranoia – of ransomware. By the way, qubesos can make at arbitrary times exact or near exact VM image backups, and and move files from one host OS to another, without shutting down, mitigating every non-targeted piece of ransom malware & majority of non-state-backed targeted ransom malware.