After all of these years I am surprised at the number of new clients that are written against an SFTP back-end.
For instance, foldersync[1], restic[2], rclone[3][4], and for ESX(i) (!!) I have just learned of "VerticalBackup"[5].
It makes me happy that a stock, standard OpenSSH serving up what is basically a UNIX home directory is still so powerful and useful.
Contrast this with the elegance and state of the art that is ZFS and how immutable snapshots simultaneously solve both retention schedules and ransomware/mallory[6].
@rsync I’ve read you’re a major FreeBSD shop. What’s your currently feeling about FreeBSD? How’s the current state of their OS support methodology? How’s stability of the OS these days? Your thoughts on ZFS now separated from FreeBSD. Etc.
You’re one of the few individuals who have the breadth and depth of experience with FreeBSD for commercial use, I’m hoping to hear you’re thoughts.
Most of my career has been working for shops running FreeBSD.
FreeBSD is doing just fine. ZFS change is not interesting. Previously upstream was Illinois, now it's Linux. Support for releases changed but for the better. Stability is still great, but it will be a while before I see servers with 10 years uptime on modern kernels again. If I'm lucky it won't happen because they're getting patched...
Ssh is just so ubiquitous. I actually write software that replicates our zfs snapshots off site using ssh as the transfer mechanism. Makes it easy to transfer snapshots between Linux and Solaris.
The best client I have used is lftp. [1] It support chroot sftp, can spawn multiple threads per job and even per file. I can max out any internet link with it. It is super easy to write automation with it.
Say I generate logs at say 1 GB/hr and store them in files named `logs.$(date +%Y%m%dT%H)` and I have it set up to take snapshots every 4 hours and delete files older than 5 hours, then I could have essentially unlimited storage and only ever pay for 5 GB/month?
This is the big one. Even for backblaze b2, it's the transfer that means 2-3tb data doesn't make sense on a personal budget (eg digital negatives and a few 4k videos). A restore become (too) expensive.
Wow, the prices are down to 2cents/GB, cheaper than standard AWS + without the fees. [1]
Back in 2015, it was 20cents per GB. [2]
Besides ZFS with snapshots, one of their benefits vs "dumb" cloud storage is you can run a server side component, e.g. borg which requires both a client and server. They even have slightly reduced pricing (which isn't that big a deal anymore, it was 3 cents vs 8-20 back in 2016 [3]) if you only need borg and don't need the configuration support. [4]
EDIT: And, tons of other things I forgot about: git, s3cmd, and others! [5]
for smaller storage servers, up to 2TB, Time4VPS beats Hetzner on price.
I've been using one for afew years now and it's been great. I run borg backups, rsync some stuff, and serve a calibre server out of a 1TB storage server.
That snapshot[0] surely doesn't reflect the reality. Last Modified date is "today" for all snapshots when you advertize an immutable storage.
You should also add the current date in the picture (just in case).
I used to use this for <10GB with the old pricing model. Stopped when the family needed something they could use (ended up on Office Home + OneDrive). I could definitely find a use for 1-5GB of storage, but the minimum is 500GB ($10/USD month or $108/USD annually).
Sorry John, looks like you had to chase me a bit due to an expired credit card. Hope I didn't cost too much for support.
The most useful thing to me would be being able to use git clone and rclone to pull files closer to me, but I bet it's not an intended use case to use rsync.net as a proxy.
Yes. Our web based admin portal / management interface has 2FA options inside.
There are also options to lock your entire account (or just your .ssh/authorized_keys file) to be read-only. The snapshots are always immutable, but you can toggle your entire account to be so, as well.
Finally, inside the web based management is an "idle alert" where you can alert yourself (via email, SMS, pushover or webhook - or any combination thereof) to your account being unchanged for a (user configurable) amount of time.
Nice! These points seem like things you might want to put on the web site. (Did I miss it? I think I read through all of the public tech details/support pages though.)
Yes. You can specify any arbitrary schedule of days/weeks/months/quarters/years.
They are efficiently stored so if your data doesn't change much, they will take up almost no space - even though they present your entire filesystem from that date.
Go to the "Simple Pricing" page. Click the 0-999 GB for 2 cents/GB link. That's great pricing.
Select 15 GB.
Get message telling me the minimum is 500 GB.
That's not the kind of company I'm going to trust with my data. What other kind of BS are they going to try to pull if they're willing to do something like this before I'm even a customer?
Maybe they don't want small customers like me. That's okay. Just be honest about it on your website.
You can already have something like this for free with Yandex Disk or Google Drive. rclone will let you mount the so called "cloud storage" locally and will encrypt everything you upload (be aware that encryption has to be configured separately as a remote on top of existing remote).
I seriously doubt this is the case. That's not how overhead works. Amortized across all users of the service, it will be well under 30 cents per user-month for this kind of service.
It's not a customer overhead issue - or, rather, that's not the computation we're doing. It's a payment processing issue - if a payment costs 20-30 cents per process, plus the percentage (which can get eye-poppingly large when people use rewards cards) we can't, then, charge people 40 or 90 or 110 cents per month.
As I have mentioned elsewhere, however, if this is a showstopper for you, just email info@rsync.net and we'll figure out a way to give you a 100 GB (or whatever) account that you pay annually or something.
"That's not the kind of company I'm going to trust with my data. What other kind of BS are they going to try to pull if they're willing to do something like this before I'm even a customer?"
I am sorry if that is confusing - it is not in any way meant to be a dark pattern or misleading.
We can't charge people, for instance, 30 cents per month (as in your example) - the fees or processing your card (or paypal payment) would be larger than that.
The minimum order of 500 GB computes to $10 per month. If you have a very tight budget, please email info@rsync.net and I will personally figure out a way (annual payment, whatever) to make it work for you. It will be larger than 15 GB, however, unless you want to pay 5 years in advance.
I was just about to change the wording on the pricing page to be "500-9999 GB" but that's even more confusing, IMO ... I think most people understand that processing fees dictate a minimum order ...
>We can't charge people, for instance, 30 cents per month (as in your example) - the fees or processing your card (or paypal payment) would be larger than that.
You can - you just need to eliminate the card processing from most transactions in these cases. Allow users to top up a balance with a minimum buy-in that meets your processing threshold, and decrement that balance as services are consumed.
This does require some degree of effort in terms of implementation and accounting, but it drastically increases the appeal of your service to small businesses and personal users.
> This does require some degree of effort in terms of implementation and accounting
I feel like you're really downplaying what would be involved. It is a great suggestion, but the effort to manage and implement doesn't sound trivial. Personally, the few systems that exclusively use this model drive me nuts. Instead of just paying for what I want I have to add to this account, then pay for what I want.
For the business owner, they now have to account for and secure new account balances. At a large scale (Starbucks has $1.6 billion [1]), you can likely cover any additional regulatory and accounting costs and also make money. On a small scale it can be a liability that costs more than it brings in.
Dealing with customer accounts that carry a credit is a rather mundane and common bookkeeping exercise. Experienced accountants are unlikely to balk at such a suggestion.
For the record, I doubt that you were trying to mislead anyone. I clicked through because I thought to myself, "There's no way they can sell 15 GB of storage for 30 cents/month."
It looks like a good service for those with moderate and large data storage needs, but it would be better to be clear about that from the start. I know a lot of people that will test these services with smaller amounts of data before making a commitment.
I was similarly surprised as the original commenter (without so much drama). I am just starting to look for Offline storage ideas for important files for my very very early startup And I thought rsync.net was very cool. However I think right now I will have at most 10 GB, but given that there is no option for that, I will have to concentrate the development efforts to integrate with GCS + S3 to have the offsite redundancy I have.
I think you guys could benefit form "early adoption" people like me. My company might not be a very big client now, as with any startup, we plan to grow and thus our offsite storage requirements. But for me, I prefer to spend the dev-time to integrate with a provider early once, preferably in something that will last for a long time (so I don't have to spend dev time again soon)
I feel that you guys/gals/polar bears are great. Just had to say. You really can see the dedication and lack of cynicalness. That's all. Not a customer just a commenter. Data is already safe here. But you seem really great
Just to give an anecdote - rsync.net has been in this business for a long time. From their own site:
> rsync.net began providing cloud storage for offsite backups in the fall of 2001, for our original corporate parent, JohnCompanies.
> In 2006, we became a stand-alone firm dedicated only to offsite backup. We provide this simple product and nothing else.
I set up rsync.net with a previous employer a decade ago and I am quite certain that all of the automation set up at that time is probably still functioning flawlessly.
Never had a single issue, it Just Worked from day one and provided us with off-site backups using trivial technology that took me scant minutes to set up.
It may not have the glitz of Amazon S3 and such, but the raw simplicity of the product + the stellar track record of the provider is very compelling.
Can you just let people prepay a certain amount and deduct from that every month? You don't want to charge 30 cents a month, but I'd happily prepay $10-15 and have it deducted from the total every month.
If the problem is not wanting to support a bunch of small accounts, I totally get that, but if the problem is credit card fees every month, there are lots of ways to avoid that.
Wasabi was the same way when I was trying to find a secondary cloud to host personal backups.
$6 /mo minimum storage charge, and they have a 90 day minimum. That, combined with the free trial ending as soon as I put my credit card in meant they tried charging me $18 for the GB of test data I uploaded... I should have read the fine print more closely, apparently.
Would mounting an SFTP file system using FUSE and then mounting encfs or veracrypt on top of that be reasonably reliable? I'm always concerned about my data at rest on servers others have access to.
Yes. sshfs/FUSE is a very common use-case among rsync.net customers. However, borg (and to a lesser degree, restic) is the typical solution for encryption at rest.
Just signed up for 100gb Borg hosting for $18 for a year! That ain't bad. Some lints (no zfs snapshots) but Borg should have snapshots covered. Waiting for activation and will start playing around...
If you need your data right now and your network connection isn't fast enough to do it in a timely manner, we will dump your data to a bare SATA drive and fedex it to you.
Alternatively, if you have many TB to upload and your network connection makes that unworkable, we will accept your bare SATA drive for the onloading process.
Wow, much cheaper than it used to be. I left them as a customer some years back purely based on price, but now I'd consider moving back. Other than being undercut by AWS (now also GCP) they were great.
I did some searching around and finally settled on rsync as a backup system about 5 years ago, and I've been happy since. I don't understand the fixation on finding the lowest price solution - pay more, get more. With rsync you get their in-house hardware and engineers on the support desk. I use it to back up my FreeNAS ZFS shares, which is dead-simple process, and a bunch of web servers. Getting historical snapshots is genius. I also love being able to use any SFTP client I like instead of some dedicated client or a web interface. My data is not available through a web page - that's exactly the way I like it. Bravo Rsync, keep up the good work.
75 comments
[ 2.8 ms ] story [ 109 ms ] threadFor instance, foldersync[1], restic[2], rclone[3][4], and for ESX(i) (!!) I have just learned of "VerticalBackup"[5].
It makes me happy that a stock, standard OpenSSH serving up what is basically a UNIX home directory is still so powerful and useful.
Contrast this with the elegance and state of the art that is ZFS and how immutable snapshots simultaneously solve both retention schedules and ransomware/mallory[6].
[1] https://www.tacit.dk/foldersync/
[2] https://restic.net/
[3] https://rclone.org/
[4] https://rsync.net/products/rclone.html
[5] https://verticalbackup.com/
[6] https://www.rsync.net/resources/howto/snapshots.html
You’re one of the few individuals who have the breadth and depth of experience with FreeBSD for commercial use, I’m hoping to hear you’re thoughts.
FreeBSD is doing just fine. ZFS change is not interesting. Previously upstream was Illinois, now it's Linux. Support for releases changed but for the better. Stability is still great, but it will be a while before I see servers with 10 years uptime on modern kernels again. If I'm lucky it won't happen because they're getting patched...
[1] - https://serverfault.com/questions/836103/replace-scp-with-sf...
S3/Google give around 2 cents/GB a month (lower for cold storage) but you also get to access it through the web... what's the benefit with this?
You get 7 free snapshots that do not count against your quota.
Additional snapshots are charged based on the byte diffs, not total size.
It uses a standard protocol and anything that uses ssh can conceivably communicate with it.
Oh, and you are not charged for transfer.
Say I generate logs at say 1 GB/hr and store them in files named `logs.$(date +%Y%m%dT%H)` and I have it set up to take snapshots every 4 hours and delete files older than 5 hours, then I could have essentially unlimited storage and only ever pay for 5 GB/month?
This is the big one. Even for backblaze b2, it's the transfer that means 2-3tb data doesn't make sense on a personal budget (eg digital negatives and a few 4k videos). A restore become (too) expensive.
Back in 2015, it was 20cents per GB. [2]
Besides ZFS with snapshots, one of their benefits vs "dumb" cloud storage is you can run a server side component, e.g. borg which requires both a client and server. They even have slightly reduced pricing (which isn't that big a deal anymore, it was 3 cents vs 8-20 back in 2016 [3]) if you only need borg and don't need the configuration support. [4]
EDIT: And, tons of other things I forgot about: git, s3cmd, and others! [5]
[1] https://www.rsync.net/pricing.html
[2] https://web.archive.org/web/20150509033924/https://www.rsync...
[3] https://web.archive.org/web/20160926192841/http://rsync.net/...
[4] http://rsync.net/products/borg.html
[5] https://www.rsync.net/resources/howto/remote_commands.html
The reason being, anything s3cmd can do, rclone can do better. Right ?
But is it better than s5cmd?
I've been using one for afew years now and it's been great. I run borg backups, rsync some stuff, and serve a calibre server out of a 1TB storage server.
[0] https://www.rsync.net/images/duck2.png on https://archive.is/3kK28
Sorry John, looks like you had to chase me a bit due to an expired credit card. Hope I didn't cost too much for support.
The most useful thing to me would be being able to use git clone and rclone to pull files closer to me, but I bet it's not an intended use case to use rsync.net as a proxy.
Couldn't find anything on the site. Sure, I get that it's probably primarily built for unattended automated backups, but still.
There are also options to lock your entire account (or just your .ssh/authorized_keys file) to be read-only. The snapshots are always immutable, but you can toggle your entire account to be so, as well.
Finally, inside the web based management is an "idle alert" where you can alert yourself (via email, SMS, pushover or webhook - or any combination thereof) to your account being unchanged for a (user configurable) amount of time.
Btw: I found https://secured.fyi/Backup/rsync.net while googling this topic, it seems at least partially incorrect.
> We are NOT reselling another cloud - we built and maintain our own infrastructure and have complete control over our platform.
> We've been doing this since 2001.
https://www.rsync.net/resources/notices/canary.txt
They are efficiently stored so if your data doesn't change much, they will take up almost no space - even though they present your entire filesystem from that date.
Go to the "Simple Pricing" page. Click the 0-999 GB for 2 cents/GB link. That's great pricing.
Select 15 GB.
Get message telling me the minimum is 500 GB.
That's not the kind of company I'm going to trust with my data. What other kind of BS are they going to try to pull if they're willing to do something like this before I'm even a customer?
Maybe they don't want small customers like me. That's okay. Just be honest about it on your website.
Otherwise I don’t see a problem with a minimum of 500GB to keep things efficient.
On the other hand I believe there is a micro market waiting, paying 1-3 usd/month for 15GB.
They could probably be more clear about minimums, but the minimum isn't unreasonable.
As I have mentioned elsewhere, however, if this is a showstopper for you, just email info@rsync.net and we'll figure out a way to give you a 100 GB (or whatever) account that you pay annually or something.
I am sorry if that is confusing - it is not in any way meant to be a dark pattern or misleading.
We can't charge people, for instance, 30 cents per month (as in your example) - the fees or processing your card (or paypal payment) would be larger than that.
The minimum order of 500 GB computes to $10 per month. If you have a very tight budget, please email info@rsync.net and I will personally figure out a way (annual payment, whatever) to make it work for you. It will be larger than 15 GB, however, unless you want to pay 5 years in advance.
I was just about to change the wording on the pricing page to be "500-9999 GB" but that's even more confusing, IMO ... I think most people understand that processing fees dictate a minimum order ...
You can - you just need to eliminate the card processing from most transactions in these cases. Allow users to top up a balance with a minimum buy-in that meets your processing threshold, and decrement that balance as services are consumed.
This does require some degree of effort in terms of implementation and accounting, but it drastically increases the appeal of your service to small businesses and personal users.
I feel like you're really downplaying what would be involved. It is a great suggestion, but the effort to manage and implement doesn't sound trivial. Personally, the few systems that exclusively use this model drive me nuts. Instead of just paying for what I want I have to add to this account, then pay for what I want.
For the business owner, they now have to account for and secure new account balances. At a large scale (Starbucks has $1.6 billion [1]), you can likely cover any additional regulatory and accounting costs and also make money. On a small scale it can be a liability that costs more than it brings in.
[1] https://mobile.twitter.com/jp_koning/status/1160387043790413...
It looks like a good service for those with moderate and large data storage needs, but it would be better to be clear about that from the start. I know a lot of people that will test these services with smaller amounts of data before making a commitment.
I think you guys could benefit form "early adoption" people like me. My company might not be a very big client now, as with any startup, we plan to grow and thus our offsite storage requirements. But for me, I prefer to spend the dev-time to integrate with a provider early once, preferably in something that will last for a long time (so I don't have to spend dev time again soon)
$15/yr for 50?
> rsync.net began providing cloud storage for offsite backups in the fall of 2001, for our original corporate parent, JohnCompanies.
> In 2006, we became a stand-alone firm dedicated only to offsite backup. We provide this simple product and nothing else.
I set up rsync.net with a previous employer a decade ago and I am quite certain that all of the automation set up at that time is probably still functioning flawlessly.
Never had a single issue, it Just Worked from day one and provided us with off-site backups using trivial technology that took me scant minutes to set up.
It may not have the glitz of Amazon S3 and such, but the raw simplicity of the product + the stellar track record of the provider is very compelling.
If the problem is not wanting to support a bunch of small accounts, I totally get that, but if the problem is credit card fees every month, there are lots of ways to avoid that.
$6 /mo minimum storage charge, and they have a 90 day minimum. That, combined with the free trial ending as soon as I put my credit card in meant they tried charging me $18 for the GB of test data I uploaded... I should have read the fine print more closely, apparently.
I recall offering to donate you the .io domain at some point too!
Alternatively, if you have many TB to upload and your network connection makes that unworkable, we will accept your bare SATA drive for the onloading process.