Hi, author here. I know that some scp flags are not handled correctly or that I don't provide a nice story to list and delete files, but I wanted to explain what I was doing early. I plan to write a bit more, e.g. about how I use Docker but I needed some kind of introduction first.
To use scp instead of a custom protocol (or even HTTP) is the neat thing here. The possibilities are infinite.
Also, this makes me think about what today's services that run on HTTP could run on top of basic Unix tools (like scp) instead of building their own infrastructure on top of HTTP or something else (I couldn't think of any, though, so I'm probably wrong in thinking this).
I really wish that more services that dealt with documents just exported either a synthetic WebDAV or SSH server.
Mounting either of those server-types as a directory is simple and straightforward on Windows, OSX, or Linux, which is something that can't be said for pretty much anything else.
I really wonder why S3 isn't (mainly) a WebDAV service, for that matter...
The beauty of Tarsnap is to provide a tar-like command-line tool. Here, while the later goal is to provide storage in a more generally accessible way, the beauty is to use scp as-is, which you already have (if you're a developer).
A client-side encryption tool could be implemented and use any scp accessible hosts, but that would not be really a Tarsnap equivalent: there is still the problem of avoiding redundant storage and traffic. Bup could be a solution here.
Did I understand your purpose wrong, or can the same thing be achieved by running sshd on a (cluster of) machine(s) that mount a GlusterFS/Ceph/OtherDistributedRedundantFS volume?
That would be similar. Indeed the redundant backend can be implemented in various ways. Now I would really like to experiment with Bup to provide historization in addition to deduplication (again this could be implemented with some underlying file systems, but I really like Git/Bup).
How would this be different than using Amazon S3 with s3cmd (https://github.com/s3tools/s3cmd)? I understand scp is stock in all Linux distros, but Amazon S3 is extremely cost competitive, allows archival to Glacier, and allows for immediate verification of data integrity. It only takes moments to install s3cmd from the command line.
Please don't think I'm being a debbie downer. I could be wrong.
The bare idea is very much similar to S3, i.e. a file store. I hope to bring a different value with things like webhooks, Docker images rebuilt automatically as files are uploaded (e.g. for reconfiguration prior to deployment on a cluster of machines), ... agreed these additional features are vaporware but I get started :)
Now, I really think that re-using scp is neat. For instance if you have tools to manage a cluster of machines, it is entirely possible you are already using SSH deploy keys (i.e. you have tools to manage SSH-based credentials). But sure you are probably also using other services that requires API keys.
11 comments
[ 3.0 ms ] story [ 34.4 ms ] threadAlso, this makes me think about what today's services that run on HTTP could run on top of basic Unix tools (like scp) instead of building their own infrastructure on top of HTTP or something else (I couldn't think of any, though, so I'm probably wrong in thinking this).
Mounting either of those server-types as a directory is simple and straightforward on Windows, OSX, or Linux, which is something that can't be said for pretty much anything else.
I really wonder why S3 isn't (mainly) a WebDAV service, for that matter...
A client-side encryption tool could be implemented and use any scp accessible hosts, but that would not be really a Tarsnap equivalent: there is still the problem of avoiding redundant storage and traffic. Bup could be a solution here.
Please don't think I'm being a debbie downer. I could be wrong.
TL;DR Why is this better than S3?
Now, I really think that re-using scp is neat. For instance if you have tools to manage a cluster of machines, it is entirely possible you are already using SSH deploy keys (i.e. you have tools to manage SSH-based credentials). But sure you are probably also using other services that requires API keys.