Thanks. I now get to the store, but it gets a 500 Internal Error trying to verify my key. Also, the Purchase link does embed my Arq 4 license key, but in the form, the upgrade license key field is empty, and I have to find it from my old .arqLicense file and paste it. I don't even know if I'm copying the right thing.
It should be fine. The problem is more likely that your stack isn't put together very well. Don't throw more hardware/dollars at it, refine your stack.
At a minimum, add some caching in front of WordPress. Even a 1 minute cache of pages will result in being able to host your WordPress site just about anywhere. Throw in a CDN, and you could almost host it on a Raspberry Pi and throw Reddit at it.
If you don't want to change your nameservers to Cloudflare (Takes up to 24 hours), try Kloudsec, and enable Offline Protection. Caches (like Cloudflare), and better, it archives your site even if your origin server is down.
For Kloudsec, all you have to do is to update your A records to point to the CDN IP.
A few notes on why I'd stick with CloudFlare personally: In my experience the NS transfer takes 20 minutes tops. Part of the idea with CloudFlare being nameserver on down is that DNS is a great thing to hit if you're performing a DDoS attack.
Also, CloudFlare does offer the offline mode feature, it's called "Always Online". It's also free for more than 1 page, unlike the Kloudsec one.
It also seems weird to me that they bill for their "webshield" on a per-attack basis... someone firing an automated scanner against you could be costing you money big time.
These guys also don't have the same reputation as CF for being bulletproof (though I haven't heard bad things about them yet either) and they have some limits on their free plan which CF does not have.
Interesting though - always glad to see some competition brewing. I know where I'm taking my business if CF steers me wrong somehow. This looks a lot cheaper than other alternatives.
Arq, (For backups to AWS - though obviously supports every cloud back end under the sun) and "Data Backup" by ProSoft engineering (For backups to USB) are my goto backup tools for day-day ensuring all my work documents are kept up to date.
Yes, I have Crashplan (for the last couple years, backblaze for the three years before that) - but the constant chewing up of CPU cycles gets annoying after a while. And both crashplan/backblaze "everything for $5" come with massive caveats (like deleting backups of Hard Drives that haven't been plugged in for 6 months - I've got Arq Backups of Hard Drives that I haven't plugged in for a couple years, safe and sound) - and I've never had an AWS backup bill in excess of $3.00, ARQ does a wicked good job of keeping your backups on a tight budget.
Also - awesome win for ARQ - when I moved to Singapore, I simply added a AWS Singapore S3 Bucket and wowza - fast backups on my gigabit ($49/month) link from MyRepublic. Really feel like I'm living in the future.
I think once I switch away from Aperture over to Photos, which presumably has a rock solid backup to iCloud photos, then simply doing a quarterly backup or so with CarbonCopyCloner + Arq to AWS + DataBackup to USB key will have my OS X backups covered.
Yea I think so, it's only for them to keep you from storing everything up there forever when it could just be stale data.
I think it's fair if these are supposed to be daily backups to do this esp with cloud data and having to provide "unlimited" space to everyone using the product.
Don't know whether I'm grand-fathered into a different terms but I've just pulled some files from a back-up on a machine that was last active 5.4 years ago.
This policy only affects devices that have not connected to CrashPlan Central in 6 months or longer. This does not affect volumes that have not connected to the device in that period of time. (i.e. an external hard drive that has not connected in 6 months.) Additionally, there is no minimum connection time for local CrashPlan backups.
It’s important for CrashPlan users to consistently connect their device(s). Part of CrashPlan’s ability to maintain the archive health and integrity relies upon regular connection from the device. CrashPlan is able to routinely perform maintenance on the archive by comparing checksums between both device and CrashPlan Central.
Since the infrastructure for CrashPlan's backup engine is the same between our Business/Consumer clients, we recommend that all users routinely connect their devices to the backup destinations. That being said, this policy only affects CrashPlan for Home subscribers at this time.
It took 3 years of disconnection before I got a notice the other day that one of my old backed up PCs was going to be deleted.
It makes sense; they're not an open-ended data archival company. If you don't have a copy of the data locally, you can't expect CrashPlan to keep the one true copy indefinitely. Connecting the data to the internet twice a year doesn't seem too onerous to show evidence of that, particularly for a cloud backup, the entire premise of which is that you're connected.
Absolutely agree with you - I think CrashPlan/BackBlaze are acting entirely reasonably when they delete old hard drives, particularly if they give a bit of grace after sending the email that they are about to nuke them.
It's that just for some of us, who like to archive something like a 100 GB Hard Drive onto Amazon Glacier, for $0.007/GB/Month. (Roughly $0.70/month + $5 upload fees for a 100 GB Hard Drive Archive) - and just leave it there, presumably for decades, are better served by Arq + Glacier than we are by CrashPlan/BackBlaze - they are entirely different tools for different purposes.
On the Flip Side, backing up users two 5+ Terabyte Hard Drives on S3 with Arq gets a little pricey... How crashplan/backblaze manage to do it for $5 is beyond me. Presumably it's because most its users are sending in < 100 Gigabytes (after deduping)
Yev from Backblaze here -> Absolutely. They key difference is backup vs. archive. Backblaze was designed as a backup solution, it's intended to be a 1:1 copy of your user data, and if your data set changes we change it on our end as well, with a 30-day history for accidental deletes. We need to reclaim that space to keep costs down, and we're not intended nor designed to be an archive (keeping data forever).
Backblaze B2 is designed differently and can be used as an archival system. The philosophies are different, but one of the reasons that we created it was to give folks the option of making actual archives they could keep in the cloud.
We hit the $5/month price-point by having our own server design, and by reclaiming space on occasion when data sets are removed. On the B2 side, since you're paying per GB, we can afford to keep that data for longer stretches. Hopefully Arq will integrate with B2 in the future and you'd be able to use their system to pick and choose what you want archived and have B2 as a possible repository.
I think your comparison is out of date [1], AWS S3 infrequent access has $0.0125 per GB now not $0.022+ that you have listed. Of course yours is still much better with $0.005 per GB.
Also, it does not look like you have any kind of consumer offering for it, as I have to contact sales to get anything at all. So there's really no point for Arq to even try adding support. I like your prices though.
We don't write out the infrequent access pricing as it's not what we're competing with, also why we don't price out Nearline or Glacier, it's not at 1:1 for what we're trying to offer. The consumer offering is available from the docs: https://www.backblaze.com/b2/docs/. You can use the web interface if you want, or tie in with CLIs/APIs. The sales channel is primarily for people that want to integrate it in to the apps that they are building. Hope that clears some things up!
”On the Flip Side, backing up users two 5+ Terabyte Hard Drives on S3 with Arq gets a little pricey”
I send my Arq backups to Amazon Cloud Drive using the ”unlimited” plan. It costs $59.99/year. So far I have only backed up < 100 GiB however, so I don’t know how well it handles backups that are multiple TiB.
Well, I am fairly certain that Amazon S3 will handle multi-terabyte backups, mostly because their pricing tiers are for 1 TB, 50TB, 500 TB, 1 Petabyte, 4 Petabytes, and 10 Petabytes - and they make more money the more you store. Performance (at least in Singapore) is also pretty awesome if you have a gigabit connection.
I'll be interested in hearing of any experiences (particularly around performance) of someone attempting to backup on the order of 10 TBytes on the Amazon Cloud Drive. My guess is that if more than very few people do this, then either (A) Amazon puts an end to "unlimited" (and yes, I appreciated the scare quotes), or, (B) They rate limit uploads after a certain size to the point at which it just frustrates people.
For some of us, dealing with a vendor who finds greater usage on your part to be a desirable behavior, such that they actually give you price breaks the more you use, creates a business relationship that is worth more than the several hundred dollars/year you'll end up saving. (Of course, this is coming from the guy who has a $36/year AWS bill, $24 of which is S3 storage)
(Side note - When talking about Storage, it's very rare to use GiB/TiB - Data rates and Storage are almost always GB or TB).
How much are you storing for those $3.00/month? Sounds too good to be true :-)
We're currently just rsyncing our pictures and stuff at home to two 3TB USB-drives (one active and one at my parents' place; using LUKS and btrfs with compression, snapshots). But even after running deduplication, they're filling up (raw files …), so I'm always on the lookout for other options. Upgrading to 2x4TB is a bit expensive, but I haven't yet found anything that'll cost me less than that while still having client-side encryption and Linux support. E.g. tarsnap seems to be about $250/TB-month, http://jotta.no/ is unlimited but has no Linux/encryption support, and I never understood Glacier pricing :-)
(and it's really convenient to be able to just restore from a local USB drive instead of having to wait for the network, though of course it's less convenient not having backups when travelling …)
AWS storage with S3 is $0.03/GB/Month. I have probably around 20 Gigabytes, stored in Singapore and US-East, versioned back 2+ years + another 58 Gigabytes in Personal Photos Backed up on Amazon Singapore (which I really should move to Glacier). It's compressed when stored on S3, so total storage is only 54 Gigabytes on Amazon Singapore, and 12 Gigabytes on Amazon-East.
I shoot a ton of pictures with my SLR, but, at the same time, I don't shoot raw, my camera is an EOS 10D (6.3 megapixel resolution) and I'm hyper aggressive about deleting all but the top 5% of my shots each day. I may shoot 300 pictures and keep 10-15.
So, I guess the major difference is I'm backing up about 78 Gigabytes of Data (though 20 of which is in two locations).
$50/year and you have to provide your own storage? Can somebody explain to me how they are competitive, i.e. in comparison with Crashplan or Backblaze?
edit: my bad, it's $50 for one time purchase, not a year.
It's a one time fee for really awesome software. Upgrades are every couple years, and only about $25. My backup bills on AWS have never been more than $3/month (I only backup my actual documents) - and you have total control - don't have to worry about the policy engine at BackBlaze/CrashPlan arbitrarily deleting old hard drives that haven't been plugged in.
Its $50 one time purchase for new purchase. The upgrade from Arq 4 is $25. During store checkout, you can also choose to add a "Lifetime Upgrades" at $30 which will give you free upgrades for Arq 6, 7, 8, 9, 10, 11, 12 etc.
I purchased the Lifetime Upgrades in September 2015. Now it tells me there is a $25 upgrade fee to go to version 5. I emailed support a few hours ago to confirm. If this is true, I am extremely disappointed and feel cheated. Notice on the form to buy a new license it doesn't say anything about "Lifetime Upgrades for this major version _only_ which will be obsolete in a few months". I sure hope this is just a bug that they will fix.
Client-side encryption is a big one. It also has a passable gui which can be helpful when restoring files, and does a pretty good job of scheduling and throttling upload speeds.
I had an older version that I upgraded to 4.x because they added support for uploading backups to Dropbox and I was already paying for a Dropbox pro account that had plenty of unused space. The ability to have versioned backups on various cloud services that don't support rsync is a big plus.
rsnapshot: "rsnapshot is a filesystem snapshot utility based on rsync. rsnapshot makes it easy to make periodic snapshots of local machines, and remote machines over ssh. The code makes extensive use of hard links whenever possible, to greatly reduce the disk space required."
Arq is like Tarsnap/Attic/Borg in that it uses shift-resistant block deduplication - it can efficiently cope with small modifications to existing files, where rsync would have to clone the entire file for each new version.
In the context of Time Machine, it does not use that form of deduplication, so rsync is capable of fulfilling the parent's requirements.
In the larger scope of things, that level of deduplication has never been something I've found all that valuable; but of course most binary files I backup are images and music files; not ones which change frequently (if ever).
Although the application itself is not open source, the developer has open-sourced a restore tool, to help ease the "what if they go out of business" concerns:
if a few years ago somebody said "just copy all your private data to some place on the internet... but encrypt it first" you would take it for the crazy argument it is.
What do you think is crazy about it? If it's encrypted, it can't be read. If it's on "some place on the internet", it can only be as redundant or more redundant as any other offsite solution.
With arq there are multiple layers that would have to be broken to get to the data. Still a risk, obviously - and one worth considering. I bet in the scheme of things though there are much easier ways to access this data available now.
Any agent capable of ever reading AES encrypted files with 256 bit keys is also an agent capable of opening your safe deposit box, requesting your tapes from a data vault provider, or coming over to your house with a rubber hose.
Short of a vulnerability in AES (in which case we have more problems than a few copies of the Anarchists Handbook in our backup files), cracking proper encryption is simply not feasible.
You could use rsync and s3cmd in tandem to create a decent backup solution. Of course, it does require the command line and some scripting chops, but I've done it in the past to keep daily, weekly, and monthly logical MySQL backups on S3.
If you are interested in something similar to what you describe, there is always Dirvish. It's basically a program that manages multiple snapshots and backups using rsync and hardlinks.
If I understand well, Arq does not backup the whole computer. I am looking for a tool that allows for recovery of single files, but also that has a backup of the whole machine, so that if the HD crashes it is possible to rebuild it verbatim. What does HN suggest for such a tool? I used Norton Ghost for years (but it has been discontinued).
Window's folks generally refer to this as "ghosting", after the discontinued Norton Ghost. I have used Acronis in the past, but there are alternatives:
On Windows, my favorite by far is ShadowProtect. It's actually a sector-by-sector backup that pretends to be a file-by-file backup when you restore a file.
What's great about this is when you modify a huge file, it only backs up the actual sectors you changed. (Most Windows backup programs detect changes on a file-by-file basis.) I have it set to run a full incremental backup every 15 minutes; the backup typically takes 15-30 seconds and is unnoticeable when it happens.
Even though it's a sector backup, you can still restore specific files or do a complete system recovery, either to the same/compatible device or with a Hardware Independent Restore to different hardware. I've done each of these many times.
On OSX, I'm using Time Machine, but I wish there were something as good as ShadowProtect. It is a bummer when I touch a few sectors in a huge file like a VM disk image, and the best Time Machine can do is back up the entire file again.
I could not recommend Carbon Copy Cloner enough. It makes your external hard disk bootable and verbatim has the same files as your mac. This is for Mac only. Think of a dd with a nice interface. Costs $40.
Do you really need to protect the entire system? Or have you not captured the setup of your system in a provisioning tool like ansible or chef or the like? And then protect only the necessary files to restore the configuration and user data of the system.
Protecting the entire system should be something that is done near line so if you have a catastrophic loss your time to recovery is less than that if everything was stored somewhere in the cloud. Or Just not do it at all and rely on a tool manage the configuration of your system.
Data files and configuration are really the best thing to protect with a tool like this. If you have a total loss your playbook should include something like replacing the failed hardware, Installing and patching the OS, replaying configuration of the system using ansible or chef, restoring data files.
To me this is the fundamental gap that cloud backup solutions need to fill to really capture the consumer market well. The SMB market already has this as you goto IT and get your system reloaded and then restore your user data, its pretty much the standard for larger corps.
How much data are you backing up? The compression and deduplication seem pretty good in Arq. My monthly AWS (and now Google Nearline) bill is usually around $2, so I hit the breakeven point vs the $5/month services pretty quickly.
My current backup is around 300 Gigs. I see the big advantage of Arq (over Backblaze) is the fact that you can choose your host. However their own hosting solution for $10/mo with a 250GB data cap is quite expensive. I can see that for smaller amounts of data Arq is more interesting, especially since it also stores history (which Backblaze does not).
It's worth noting that Arq can back up to Amazon Cloud Drive, which offers unlimited storage for $59.99 a year. As soon as they offered that I switched from Glacier storage; Cloud Driver is faster, cheaper and less fiddly.
That is quite good! Unfortunately, in France there is no Unlimited Everything plan. The prices are similar to the UK (comment from mobiuscog).
However there is one last quirk I have with Arq + Hosting Service. If you need to retrieve your backup _fast_ you will have to reach for a paid service such as AWS Snowball which seems quite expensive. I can't to seem to find more recent information about them shipping smaller disks, just some old articles.
I didn't even know they offered that until now. I viewed the prime advantage of Arq was selecting your host. And really that is "hosts", since you can have multiple backup sets and different hosts. Stuff that I often want to retrieve I might have on Google Drive as well as one of the "cold storage" options.
I've been using Arq for years and absolutely love it. Worth every penny. It is extremely well-built software — it's FAST, doesn't hog resources, and feels very polished & reliable.
I like that I can backup to multiple destinations (AWS S3/Glacier, Dropbox, Google Drive, even my own server via SFTP). IMO you can never have too many backups.
I use it along with Backblaze (and will be setting up Time Machine & Super Duper or Carbon Copy Cloner this week, after putting it off forever).
Arq's author has said on Twitter that the B2 API doesn't support all commands Arq needs, so even if B2 support was planned (which it isn't) it wouldn't be possible yet: https://twitter.com/arqbackup/status/717756616578301952
Yev from Backblaze -> We're working on a few APIs and some should be aligned with what ARQ is looking for. We're hoping that they do integrate with us, giving folks an inexpensive alternative!
FWIW I've been a big fan of Syncovery (formerly SuperFlexible File Synchronizer) for years. It's a Swiss Army Knife of backup/recovery. https://www.syncovery.com/
Syncovery looks pretty awesome for what it does, I just installed it to take a look, - Directory Synchronizaton - but it's achilles heel for file backups (as opposed to directory/disk cloning) - is appears to be it's lack of any type of versioning. I.E. If I make changes to a file each day, and do backups each day - no way to go back to a version of the file a couple weeks ago?
I was a user of Arq 3, but it was slow and seemed to eat up resources on my computer. I eventually turned it off, deleted the backups and deleted it. Not a great backup plan.
So I'm keen to know how much faster Arq 4 was, and in turn Arq 5. I'd be happy to try again (I think I'll have to pay full price again as I'm not an Arq 4 user) - but might wait until someone can let me know just how much faster it really is.
This whole thread has also reminded me to run my backups to local Time Machine!
I have nothing to add except that I've been a happy customer of Arq for years, and Stefan and team have provided us very helpful and personal support by email on the rare occasions when we needed it.
It appears Arq's being destroyed by HN currently -- I downloaded the trial for Mac, but when I press "Start Trial" I get "Failed to create trial - A server with the specified hostname could not be found." I'll try again tomorrow.
What compression was used before Arq 5? lz4 is super fast, but not particularly space-efficient compared to some slower compression algorithms. Since Arq customers are the ones paying the storage bills, this doesn't seem like an entirely costless decision--Arq is now faster, but you should expect your storage bills to go up a bit due to the lower compression.
I use Borg backup with lz4 compression, so I definitely don't think this is the wrong decision, just something to keep in mind (and, it does seem like something that could and maybe should be user-configurable).
I'm also curious about the choice of LZ4. It's fast, but its compression is pretty awful.
For example, on simple, regular text (JSON) files I'm seeing about 45% worse compression than plain gzip (default compression level). I hope at least it's using the highest compression level, but I've found that "lz4 -9" is about as slow as "gzip -6", still with worse compression.
I'd be happier if the choice of compression was dependent on the size of files. The larger a file, the more you gain from compression.
Does Arq skip compression for already-compressed files (.gz, .xz, .bz2 etc.)?
Curious decision, as lz4's terrible compression ratio burns you both in monthly storage fees, as well as Glacier (or other service) restore fees.
Most of my backup set is pictures and compressed files, so maybe the lz4 decision was because they figured the majority of their customers fell in that category where no compression algorithm is going to help?
I use Arq to backup all my stuff hourly to my NAS at home through SFTP and the NAS then backups the most important stuff to AWS daily. That works really well and keeps the cost really low because i need the NAS anyway for streaming media an such.
193 comments
[ 3.2 ms ] story [ 239 ms ] threadAnd the format of the stored data is available, which is a nice safety feature in case of major problems.
"Error establishing a database connection"
[0] https://www.arqbackup.com/
I was excluding cache.noindex from my other backup system (carbon copy cloner). Is it still safe to do so?
https://gist.github.com/15dde3fb013581b6aa59bf0b0c3a701e
They're pretty reputable for free speech and fighting for their customers too, so I trust them to a decent degree.
For Kloudsec, all you have to do is to update your A records to point to the CDN IP.
Also, CloudFlare does offer the offline mode feature, it's called "Always Online". It's also free for more than 1 page, unlike the Kloudsec one.
It also seems weird to me that they bill for their "webshield" on a per-attack basis... someone firing an automated scanner against you could be costing you money big time.
These guys also don't have the same reputation as CF for being bulletproof (though I haven't heard bad things about them yet either) and they have some limits on their free plan which CF does not have.
Interesting though - always glad to see some competition brewing. I know where I'm taking my business if CF steers me wrong somehow. This looks a lot cheaper than other alternatives.
Yes, I have Crashplan (for the last couple years, backblaze for the three years before that) - but the constant chewing up of CPU cycles gets annoying after a while. And both crashplan/backblaze "everything for $5" come with massive caveats (like deleting backups of Hard Drives that haven't been plugged in for 6 months - I've got Arq Backups of Hard Drives that I haven't plugged in for a couple years, safe and sound) - and I've never had an AWS backup bill in excess of $3.00, ARQ does a wicked good job of keeping your backups on a tight budget.
Also - awesome win for ARQ - when I moved to Singapore, I simply added a AWS Singapore S3 Bucket and wowza - fast backups on my gigabit ($49/month) link from MyRepublic. Really feel like I'm living in the future.
I think once I switch away from Aperture over to Photos, which presumably has a rock solid backup to iCloud photos, then simply doing a quarterly backup or so with CarbonCopyCloner + Arq to AWS + DataBackup to USB key will have my OS X backups covered.
Wait - what?
If your hard disk gets lost or destroyed, how do you protect those backups?
For backblaze it's even less time.
I think it's fair if these are supposed to be daily backups to do this esp with cloud data and having to provide "unlimited" space to everyone using the product.
Wanted to jump in here to confirm.
This policy only affects devices that have not connected to CrashPlan Central in 6 months or longer. This does not affect volumes that have not connected to the device in that period of time. (i.e. an external hard drive that has not connected in 6 months.) Additionally, there is no minimum connection time for local CrashPlan backups.
It’s important for CrashPlan users to consistently connect their device(s). Part of CrashPlan’s ability to maintain the archive health and integrity relies upon regular connection from the device. CrashPlan is able to routinely perform maintenance on the archive by comparing checksums between both device and CrashPlan Central.
https://support.code42.com/Administrator/3/Monitoring_And_Ma...
Please let me know if I can provide additional clarity.
Best regards,
Jarrod
No way. What ?
You're saying that if I dutifully pay my $X/mo for unlimited backup space, but fail to connect, or perform an update, the remote data is removed ?
What ?
--
[1] At least they had that 30 days limit a few years ago, I haven't used them in a while.
It makes sense; they're not an open-ended data archival company. If you don't have a copy of the data locally, you can't expect CrashPlan to keep the one true copy indefinitely. Connecting the data to the internet twice a year doesn't seem too onerous to show evidence of that, particularly for a cloud backup, the entire premise of which is that you're connected.
It's that just for some of us, who like to archive something like a 100 GB Hard Drive onto Amazon Glacier, for $0.007/GB/Month. (Roughly $0.70/month + $5 upload fees for a 100 GB Hard Drive Archive) - and just leave it there, presumably for decades, are better served by Arq + Glacier than we are by CrashPlan/BackBlaze - they are entirely different tools for different purposes.
On the Flip Side, backing up users two 5+ Terabyte Hard Drives on S3 with Arq gets a little pricey... How crashplan/backblaze manage to do it for $5 is beyond me. Presumably it's because most its users are sending in < 100 Gigabytes (after deduping)
Backblaze B2 is designed differently and can be used as an archival system. The philosophies are different, but one of the reasons that we created it was to give folks the option of making actual archives they could keep in the cloud.
We hit the $5/month price-point by having our own server design, and by reclaiming space on occasion when data sets are removed. On the B2 side, since you're paying per GB, we can afford to keep that data for longer stretches. Hopefully Arq will integrate with B2 in the future and you'd be able to use their system to pick and choose what you want archived and have B2 as a possible repository.
Also, it does not look like you have any kind of consumer offering for it, as I have to contact sales to get anything at all. So there's really no point for Arq to even try adding support. I like your prices though.
[1] https://www.backblaze.com/b2/cloud-storage-providers.html
These pages make it clear how it works and it seems reasonable:
https://help.backblaze.com/hc/en-us/articles/217665398-Backi...
https://help.backblaze.com/hc/en-us/articles/217664898-What-...
I send my Arq backups to Amazon Cloud Drive using the ”unlimited” plan. It costs $59.99/year. So far I have only backed up < 100 GiB however, so I don’t know how well it handles backups that are multiple TiB.
[Edited to correct confusing typo.]
I'll be interested in hearing of any experiences (particularly around performance) of someone attempting to backup on the order of 10 TBytes on the Amazon Cloud Drive. My guess is that if more than very few people do this, then either (A) Amazon puts an end to "unlimited" (and yes, I appreciated the scare quotes), or, (B) They rate limit uploads after a certain size to the point at which it just frustrates people.
For some of us, dealing with a vendor who finds greater usage on your part to be a desirable behavior, such that they actually give you price breaks the more you use, creates a business relationship that is worth more than the several hundred dollars/year you'll end up saving. (Of course, this is coming from the guy who has a $36/year AWS bill, $24 of which is S3 storage)
(Side note - When talking about Storage, it's very rare to use GiB/TiB - Data rates and Storage are almost always GB or TB).
We're currently just rsyncing our pictures and stuff at home to two 3TB USB-drives (one active and one at my parents' place; using LUKS and btrfs with compression, snapshots). But even after running deduplication, they're filling up (raw files …), so I'm always on the lookout for other options. Upgrading to 2x4TB is a bit expensive, but I haven't yet found anything that'll cost me less than that while still having client-side encryption and Linux support. E.g. tarsnap seems to be about $250/TB-month, http://jotta.no/ is unlimited but has no Linux/encryption support, and I never understood Glacier pricing :-)
(and it's really convenient to be able to just restore from a local USB drive instead of having to wait for the network, though of course it's less convenient not having backups when travelling …)
I shoot a ton of pictures with my SLR, but, at the same time, I don't shoot raw, my camera is an EOS 10D (6.3 megapixel resolution) and I'm hyper aggressive about deleting all but the top 5% of my shots each day. I may shoot 300 pictures and keep 10-15.
So, I guess the major difference is I'm backing up about 78 Gigabytes of Data (though 20 of which is in two locations).
edit: my bad, it's $50 for one time purchase, not a year.
edit: You get a free upgrade if there is a major version released within 90 days of your order (if you don't order lifetime upgrade of course)
On the features page, https://www.arqbackup.com/features/, under "Limit Network Impact", the image lists whether to use all wireless or no wireless.
I don't have the software so I can't confirm.
> No wireless. Less space than a nomad. Lame.
https://twitter.com/arqbackup/status/719924424263249920
Just wondering about the technical aspects.
I had an older version that I upgraded to 4.x because they added support for uploading backups to Dropbox and I was already paying for a Dropbox pro account that had plenty of unused space. The ability to have versioned backups on various cloud services that don't support rsync is a big plus.
Have a look at the `--link-dest` option to rsync.
http://rsnapshot.org
In the larger scope of things, that level of deduplication has never been something I've found all that valuable; but of course most binary files I backup are images and music files; not ones which change frequently (if ever).
Although the application itself is not open source, the developer has open-sourced a restore tool, to help ease the "what if they go out of business" concerns:
https://github.com/sreitshamer/arq_restore
Also, somebody else has created a Go utility that can read the backup format:
https://github.com/asimihsan/arqinator
Finally, if you really wanted to do it all yourself, the data format is documented here:
http://www.haystacksoftware.com/arq/s3_data_format.txt
if a few years ago somebody said "just copy all your private data to some place on the internet... but encrypt it first" you would take it for the crazy argument it is.
Yet. It can't be read yet.
Short of a vulnerability in AES (in which case we have more problems than a few copies of the Anarchists Handbook in our backup files), cracking proper encryption is simply not feasible.
http://www.dirvish.org
the physical death of a drive is not the only reason you may find yourself needing a bootable backup:
* accidentally deletion of system files * theft, fire * system update that goes wrong
http://www.tenforums.com/tutorials/5495-system-image-create-...
http://www.econtechnologies.com/chronosync/overview.html
SuperDuper is the easiest to use, and I would recommend it to anyone as their first choice.
CCC and ChronoSync both have more advanced features: including bootable network copies, and backing the recovery partitions.
ChronoSync also offers two-way folder sync's, enterprise features, and more.
http://alternativeto.net/software/norton-ghost/?platform=win...
[1] https://www.veeam.com/endpoint-backup-free.html
On Windows, my favorite by far is ShadowProtect. It's actually a sector-by-sector backup that pretends to be a file-by-file backup when you restore a file.
What's great about this is when you modify a huge file, it only backs up the actual sectors you changed. (Most Windows backup programs detect changes on a file-by-file basis.) I have it set to run a full incremental backup every 15 minutes; the backup typically takes 15-30 seconds and is unnoticeable when it happens.
Even though it's a sector backup, you can still restore specific files or do a complete system recovery, either to the same/compatible device or with a Hardware Independent Restore to different hardware. I've done each of these many times.
On OSX, I'm using Time Machine, but I wish there were something as good as ShadowProtect. It is a bummer when I touch a few sectors in a huge file like a VM disk image, and the best Time Machine can do is back up the entire file again.
You could exclude those kinds of files from Time Machine, and use your VM's own backup system.
https://bombich.com/
Protecting the entire system should be something that is done near line so if you have a catastrophic loss your time to recovery is less than that if everything was stored somewhere in the cloud. Or Just not do it at all and rely on a tool manage the configuration of your system.
Data files and configuration are really the best thing to protect with a tool like this. If you have a total loss your playbook should include something like replacing the failed hardware, Installing and patching the OS, replaying configuration of the system using ansible or chef, restoring data files.
To me this is the fundamental gap that cloud backup solutions need to fill to really capture the consumer market well. The SMB market already has this as you goto IT and get your system reloaded and then restore your user data, its pretty much the standard for larger corps.
Failed to activate. The remote name could not be resolved. store.arqbackup.com
This being put aside I find their pricing a bit excessive in comparison to Backblaze for example.
However there is one last quirk I have with Arq + Hosting Service. If you need to retrieve your backup _fast_ you will have to reach for a paid service such as AWS Snowball which seems quite expensive. I can't to seem to find more recent information about them shipping smaller disks, just some old articles.
I like that I can backup to multiple destinations (AWS S3/Glacier, Dropbox, Google Drive, even my own server via SFTP). IMO you can never have too many backups.
I use it along with Backblaze (and will be setting up Time Machine & Super Duper or Carbon Copy Cloner this week, after putting it off forever).
Congrats to the Haystack team!
For Arq team (saw one or two here), is B2 on the roadmap?
FWIW I've been a big fan of Syncovery (formerly SuperFlexible File Synchronizer) for years. It's a Swiss Army Knife of backup/recovery. https://www.syncovery.com/
The announcement doesn't say, but I'm hoping they have reduced the amount of space needed for the client-side cache (currently 18GB (!) on my laptop).
So I'm keen to know how much faster Arq 4 was, and in turn Arq 5. I'd be happy to try again (I think I'll have to pay full price again as I'm not an Arq 4 user) - but might wait until someone can let me know just how much faster it really is.
This whole thread has also reminded me to run my backups to local Time Machine!
I use Borg backup with lz4 compression, so I definitely don't think this is the wrong decision, just something to keep in mind (and, it does seem like something that could and maybe should be user-configurable).
For example, on simple, regular text (JSON) files I'm seeing about 45% worse compression than plain gzip (default compression level). I hope at least it's using the highest compression level, but I've found that "lz4 -9" is about as slow as "gzip -6", still with worse compression.
I'd be happier if the choice of compression was dependent on the size of files. The larger a file, the more you gain from compression.
Does Arq skip compression for already-compressed files (.gz, .xz, .bz2 etc.)?
Most of my backup set is pictures and compressed files, so maybe the lz4 decision was because they figured the majority of their customers fell in that category where no compression algorithm is going to help?