HN Explorer

Ask HN: What is the most cost effective way to do backups?

37 points by xrd ↗ HN
I'm interesting in backing up some large VM images, say about 500 GB.

I want to optimize for lowest cost.

I would prefer to use a tool like rsync (and not an inflexible closed source GUI). So, things like Box don't appeal to me.

I'm doing this in case of catastrophic failure to my HD. It is a risk, of course, but I would prefer to pay less for a service that might "only be 90%" as reliable. I don't need 5 9s.

I see a variety of services, the best of which seem to be rsync.net which is priced at minimum of 700 GB for 1.5 cents a GB, so about $15/mo (https://www.rsync.net/pricing.html)

I could run my own VM on Vultr and can't do better than that AFAICT.

I see services like pCloud which have a *lifetime* cost of $199 for 500B (https://www.pcloud.com/cloud-storage-pricing-plans.html?peri...). That seems interesting, and I expect in five years they will say "Sorry, we are out of business!" I'm ok with that risk and will likely plan for that.

But, are there other options to consider? Is there a sweet spot for storage if I almost never need to put more files, can tolerate slow retrieval, and am OK with a discount because the company will probably go out of business?

I could just buy a used desktop off Craigslist with a 1TB drive for $200 and pay for the power myself. That's another option, but I wonder if there is some reason I can find a cheaper option.

62 comments

[ 3.6 ms ] story [ 123 ms ] thread
RainbowFriends
https://www.tarsnap.com/ or if you really want to DIY move AWS S3 to Glacier Deep Archive at $0.00099 per GB/month
xrd
My original comment was going to read: "I really like tarsnap, and the pricing seems much worse than rsync. $0.25 / GB-month. For 500GB that would mean $125/mo, right?"

Looks like I am totally wrong!

https://www.tarsnap.com/deduplication-examples.html

The de-duplication says you can store terabytes for $10/mo. I'm glad you forced me to respond to your comment!

danparsonson
Heavily dependent on the amount of duplication you have in your data though, yes? Maybe you can work that out ahead of time?
DirectorKrennic
How often do you plan to do backups? Continuously or periodically?
xrd
Very, very, very infrequently. These are VMs that I will upload once and then not think about them for years in case my local machines die.
sgc
In which case a hdd + bank deposit box might be cheaper than other solutions.
Dalewyn
Buy a HDD, backup to it, then put the HDD in a safe deposit box at your bank.
activiation
A few years old VM might break when you update it if you ever have to restore it.
electroly
You know your requirements better than we do, but at first glance this appears to be a mistake. You can see what's going to happen: when you go to restore the years-old VM backup, everything is so out of date that it doesn't work and the updates fail. "Do it once and forget about it for years" is essentially always a mistake with backups.
alexk307
Why not get a RAID array and store them in your basement?
warent
I think you're supposed to have a backup off prem in case of some disaster so you're not just losing everything at once
avgDev
I paid $89 for 15 months of Microsoft Office 365. It gives you 1TB per user for up to 5 users.

I use onedrive app on Iphone to backup images.

Next step is to sync my NAS but I have not gotten around to that yet.

hansc
Maybe not what you are looking for, but I do it the manual way. Low-end, but reliable once in a while: USB3.2 external SSD (example 60usd on NewEgg for SanDisk 1TB)
obituary_latte
You may already have some free storage from a service like Google or Apple (iCloud). You could use something like that and rsync. Otherwise, a cheap desktop +/ NAS in basement with rsync might be the cheapest option.
sokoloff
What threats are you trying to mitigate against? Only primary hard drive failures? Ransomware? User/application error? Fire/Flood/Earthquake?

If you only want 90% and don't care about local concentration of risk, it's going to be really hard to beat a $50 used computer with a SATA hard drive turned into a Linux box in your basement.

taneq
If you put it in someone else's basement and combine it with rsnapshot or rsync then you're pretty much 100% of the way there. My setup is:

1) Server runs rsnapshot to keep a local version history of all important files

2) Nightly snapshot of the server VM is saved to an offsite Proxmox backup server onto a ZFS volume which is mirrored between two physical drives.

So lost files due to user error can be pulled back off hourly, daily or monthly snapshots, total loss of the server for any reason just means restoring from the backup server (which I have tested, in an emergency with a little fiddling the backup server can take over as main server for a while), and no single drive failure on the backup server will lose old backups. If we get hit by ransomware and all our files get encrypted then our nightly backups would start timing out. :P

I also sync the whole tree to my local drive since I regularly need to work offline on remote sites, so I have totally separate fairly recent snapshots on two different laptops, only one of which is generally used at any given time.

If anything takes out both premises and both laptops simultaneously, I'm probably in no condition to care about our project folders anyway.

Did I miss anything?

quartz
Cloud storage is a massive ripoff. Just go buy a 2TB drive at Best Buy for $120 and stick it in an external usb-c enclosure from amazon for $15.
stavros
And then make sure to scrub it weekly to make sure the bits haven't rotted, and move it to another location every night to be sure it doesn't die in a house fire.
s1mon
While it doesn't support Linux, Backblaze is cloud storage and it's $70/yr for unlimited storage. To me, that's not a rip off for very secure off site back up. I backup my Mac locally to an external drive with Time Machine, many things (Photos, Mail, etc) are also in iCloud, and I use Backblaze.

If you only use local external drives, get more than one, and rotate them off site. All it takes is a fire or break-in and your computer and external backup drive could be gone. If you at least have a drive off site, there's a much higher chance that you'll be ok.

ASalazarMX
Cloud backups do help with disaster recovery. If your house burns or floods, you can recover your data anywhere without sifting ashes/fishing for your external drive.
omniglottal
Restic is the tool and Glacier is the service if you want it cheap.
ArtWomb
Ere by Fate or Happy Accident. You have stumbled upon the major pain point all of us "cloud architects" are having today. Dumping VMs to cloud store. We end up paying a 100x premium on what disc actually costs today for the privilege of "infinite" persistence. There's gotta be a better way ;)

Forget data centers, DNA is the Future of Data Storage

https://interestingengineering.com/science/dna-data-storage-...

DNA storage in thermoresponsive microcapsules

https://www.nature.com/articles/s41565-023-01377-4

stavros
You're not paying for the disk, you're paying for the disk to be there when you need it.

> Ere by Fate or Happy Accident

Doesn't "ere" mean "before", not "either"?

> Forget data centers, DNA is the Future of Data Storage

Giving a much more literal meaning to "bit rot".

uncensoredjrk
Restic (only backing up important data) to Backblaze B2 has been the sweet spot for my use cases. If the full VM goes down, I'll spin up a new one and restore my data.
briffle
Exactly, setup scripts or some other way to automate the stuff you need from the bare OS, and skip backing up that whole OS, pagefile, etc.
graiz
I use backblaze.com - less than $100/year for unlimited backup on my main machine. Their business backup is $5/TB/Month, significantly less than S3.
tzs
I don't know about their business backup product, but the last time I considered their personal backup product restoring files involved decrypting the files on their server to produce a zip file for you to download.

Is that still the case, and does business backup also work like that?

seanalltogether
Since you mention buying a desktop as a local solution. You can instead get a single drive NAS for less then $200, or better yet a 2 drive one that you can put into a RAID config.
Octoth0rpe
Anecdata, but I've been very happy with pCloud. Notably it regularly goes on sale. I think I got my 2tb lifetime plan for ~$300, and it's been perfectly sufficient. I've had it for around 4 years so far, so it's more than paid for itself versus the dropbox subscriptions that it replaced.
xrd
That's good to hear, I had not heard of them before and was putting "they will go out of business in x years" into my risk assessment.
leoedin
How much trust do you have in the lifetime plan? Surely the only way that can work is if new customers keep arriving - if they ever stop gaining customers then the money stops flowing, and you'd better hope they didn't spend it all.
Octoth0rpe
> How much trust do you have in the lifetime plan?

Very low when I first bought into it, but yeah that was 4 years ago. I've MORE than paid off the up front cost. If they disappeared tomorrow, I'd be comfortable with what I paid. I'd note that my usage may imply a higher level of risk than others may be comfortable with. All of my data is fully local/synced on my personal machine, and not local/available as long as I have internet on phone/tablet. So the only way I'd lose data is if pCloud disappeared _and_ my personal machine blew up in the same day. If pcloud disappears, I'd just dump my local files into iCloud or dropbox or something; and if my personal machine blew up I'd replace it and hope I'd get the replacement before pCloud goes kaput :) Doesn't seem like a terrible strategy.

Also, pcloud does have a couple of mechanisms for getting additional revenue from existing customers. For example, you can increase the size of your lifetime plan for an additional one time payment, or add their crypto option. Also, some people presumably may pay month to month.

Lastly, the best thing I can do as a pcloud user who is banking on them not going out of business is to occasionally rep them in public so that they get more customers, which I'm doing right now :)

xrd
You captured exactly my own backup thinking! Thanks for writing this up.
throwawaaarrgh
What you want is an external SATA RAID array hot swap cage. Connect two drives and in your OS set up a mirrored array. If one drive fails the other keeps working (don't buy the same model/batch drive). Cheaper over time than monthly payment.
hestefisk
RAID is not a backup guarantee. Use zfs.
throwawaaarrgh
?????? How is ZFS better than mirroring??? For two drives??????
invertedohm
Personally I just push to an offsite restic backup on a cheap Hetzner storage box. It's not super fast and only hosted in EU AFAICT but it fits the bill for being cheap and reliable enough. It's just there for me to retrieve sentimental-value things like photos in the off chance all my other backup methods have failed.

Initial setup is a bit weird with their SSH key setup IIRC but manageable.

https://www.hetzner.com/storage/storage-box

xrd
Man, that pricing of 4 euro for 1TB is nice. I love this option.
anotherhue
You can sshfs or cifs to it. Latency hurts but it's good.
rodolphoarruda
I think a good way to go would be to get an used desktop and backup your VMs to it using rsync. "Pure sync is not backup", yes, that's right. So for a real backup you can use a service like Wasabi, for example, which will cost you around $5/TB per month with no extras fees for data transfers.

In case you lose any VMs, you could immediately access a local copy at your used desktop. In case your whole place/office is compromised, you could access Wasabi from a different location and download a copy of your VM.

Disclaimer: I'm a Wasabi MSP.

yjftsjthsd-h
If you're running a spare desktop with some disks in it, just run ZFS on that, take snapshots after every rsync, and ideally initiate rsync from the backup machine and don't have credentials to it anywhere else so that it can't (easily) be compromised at the same time as the other systems in the event of malware.

Though yes, you should also always have an offsite backup as well, which AFAIK wasabi is fine for (as long as you're aware of the 1TB/90d minimums).

aBioGuy
The cheapest and simplest strategy is to buy several external drives and leave them at work / someone else's house.
berkle4455
If you're not worried about physical/location risk, just buy an external HDD or a NAS.
kqr
What is your process for restoration testing? If you test restoring from backups frequently you can afford less durable backup media. If you test restoring only after a few years when shit has hit the fan you will want to pay for higher durability.
FrancoisLp
Backblaze is really nice, I used it for large backups (cold backups)
christophilus
Yeah. Backblaze or Wasabi would be my choice if you want cloud options. If you want to just toss it onto a drive in your basement, that's probably the cheapest option long-term.
hestefisk
Set up a physical server on hetzner with a bunch of large disks. Then use zfs send. Job done :)
opengears
Can you elaborate on this setup a bit? Any good documentation out there to read?
zendaven
IceDrive is a pCloud alternative you might want to look into. I personally use pCloud though.
SirMaster
>I would prefer to use a tool like rsync (and not an inflexible closed source GUI). So, things like Box don't appeal to me.

The good news is that there is a tool for this: rclone

https://rclone.org/docs/

It's basically rsync for cloud services and even supports your example of the Box cloud service.

You can do a lot of neat things with rclone like transparent encryption and even turning a cloud storage into a mounted local storage folder.

The other option would just be something simple like a RasPi or similar miniPC with a pair of HDDs mirrored, either automatically or manually by just backing up to both, or having the pc copy one to the other, with some retention rules etc.

Honestly for important data I would do both a cloud backup and local, unless you have too much data where cloud would be too expensive. In that case just keep the most important stuff in the cloud and keep that and the less important stuff local.