HN Explorer

Ask HN: Why don't more software projects use BitTorrent for downloads?

90 points by nerdponx ↗ HN
It seems like it would be great for open source projects with limited funding to use BitTorrent for downloads. However I only see a handful of projects using this model, e.g. LibreOffice and GIMP. Whereas a lot of Linux-based OSes just host their disk images over HTTP, which I imagine could incur substantial hosting costs over time.

Why don't more software projects do this? Are there any big downsides to distributing torrents as opposed to traditional FTP/HTTP downloads?

92 comments

[ 5.1 ms ] story [ 143 ms ] thread
linkdd
When distributing over torrent, you need at least one seeder with a good connection. You obviously. So you still need to host a server with your package on it, and configure a torrent client to seed it.

I know a lot of HTTP/FTP storage space providers, or I can just put my binary on Github (Github Pages, or Release asset), or host it on a CDN like netlify. I think there is more solutions than real numbers between 0 and 1.

But I don't know a single "torrent seedbox" provider. Setting it up yourself is not complicated, but since it's DIY, this alone explains why it's not more wide-spread.

charcircuit
You don't need a seedbox. You just make the torrent file reference the normal HTTP download.
PaulHoule
It's not really worth it for small downloads. I looked at one of my projects and it is an 880kb download.

If you are talking a gigabyte or more there is a big advantage in terms of (1) a completely reliable download, and (2) being able to split up the download into multiple streams. At home I have two ADSL lines and a load balancer and with bittorrent I get double-speed downloads.

stevenwoo
Why don't large companies use it for updates? I have a lot of trouble downloading the 10+ gigabyte IOS downloads on DSL for some reason but torrenting works fine for larger files and stopping and resuming does not break torrenting unlike Apple's update process.
PaulHoule
I don't know how they do it but when I download from Steam there are multiple streams and I get about 200% of the throughput of a single ADSL.
Arie
Steam downloads are plain HTTP in separate file chunks. Easy to cache and load balance across connections.
duffyjp
I don't know if this would be of interest to you, but https://lancache.net/ is really fun to setup. I assigned an old (unreliable) 500gb SSD to a Linux VM and run Lancache on it. A couple tweaks to Pi Hole and every system in your house will use with no configuration (and fallback to Steam if it's down).

If you only have a single gaming system it won't matter but if you have a few it's fun to watch the second one get 100MB/s Steam downloads in a house with 20MB/s internet.

jaybeavers
Taxes killed p2p...

https://youtu.be/5-sF7N1bfv0

jaybeavers
Or more clearly stated, tax havens and offshoring revenue killed p2p.
jldugger
Even without the tax havens, the jurisdictional issues seem killer -- if I pay for software in Kansas and download blocks from Texas, California, Puerto Rico and Canada, who all do I owe sales tax to?
retrac
Some have. Blizzard used a variant of BitTorrent about 10 - 15 years ago to distribute the multi-gigabyte patches for their online games. Peak demand was very high with hundreds of thousands of users all on the same day.

It was built into the game installer / launcher. I think they've switched away from that since. I remember it causing some consternation at the time given some people pay by the byte for network service and felt it was unfair.

llui85
Musescore is planning to use P2P sharing to download sample libraries (>1gb) and updates in version 4 (coming in the next few months).
Terry_Roll
Its redundancy. Just because the download is small, doesnt mean your webserver providing the downloads doesnt go down.
toast0
Sure, but if the webserver providing the downloads goes down, usually so does the website providing the torrent link.
NullPrefix
Magnet link would still show up in the search engine cache.
Terry_Roll
Theres other ways if you can control the firewall ports, ie keep one open and then that could also be a website with links. It depends on how much extra thats built into an app.
ralusek
The best downside that I've heard, at least for doing this automatically, is that not all users have cheap/available upload bandwidth. As long as you're able to opt in as a seeder, it's great.
garbagecoder
Until everyone has synchronous internet it won’t need to be a thing. I’ve pointed out many times that between BitTorrent, tor, and bitcoin we’ve already done alleged web3, which is what we used to call the Dark Web.
unforeseen9991
Plus a whole number of other distributed projects over the years. Everything web3 is just hyped ideas that are already out there, in some cases for 10+ years.
betwixthewires
That's a good point, you're right. I hadn't thought of it this way. What we want already exists, we just have to use it.
wmf
There's a ton of free HTTP mirroring available for open source; BitTorrent really has no advantage. Bandwidth is so cheap in general that BitTorrent is not needed.
cjbprime
For the least friction, you could make the download page use Webtorrent, which is a BitTorrent bridge using WebRTC datachannels.
Froedlich
A lot of ISPs nuke anything that looks like a torrent and send angry messages to their customers.
LeoPanthera
Is that actually true? Because I thought it was "content" owners hunting for torrents containing their stuff and then sending DMCA complaints (or local equivalent) to the ISPs of everyone in that particular swarm.

Are there really ISPs that just blanket block all torrents? I've never used one that does.

Shared404
There's some local to me that don't necessarily block torrents, but will get upset.

Mostly just means I set up a VPN through Linode or whatever.

banana_giraffe
And some corporate environments might be ok with non-business uses of Internet data, but uncomfortable talks will be had if they detect BitTorrent.
pabs3
Unfortunately web browsers don't support it, so it requires an external client to download.
TheFreim
The Brave browser has a BitTorrent client built in, though I've never used it. Not a major player in the browser market, though.
chc4
There is WebTorrent, which uses WebRTC. instant.io uses it for file sharing, for example.
pabs3
I got the impression that WebTorrent users can't download from BitTorrent users and vice versa, is that true?
sergiotapia
Last I checked webtorrent users can download from webtorrent users. So they are isolated like that.
hiimshort
If I remember correctly, there is a server bridge that enables communication between webtorrent and normal bittorrent clients.
pabs3
Got a link to that?
baobabKoodaa
Why would you use BitTorrent to distribute files P2P if you're just going to place a server between the peers? Wouldn't it be better to just have the server distribute the files in that case?
SahAssar
It's used to bridge clients until more clients start supporting webtorrent directly. libtorrent (which for example deluge and qbittorrent use) already supports webtorrent (not sure which ones turn it on by default though).

The goal is basically to not have to use the bridge in the future.

Also one plus with a bridge compared to just downloading directly is that if a file becomes popular your server does not need to shoulder the full load since the webtorrent swarm can handle most of the load.

belltaco
That and blocked on many firewalls, especially corporate ones.

Also, Opera used to support torrent downloads.

1over137
>Unfortunately web browsers don't support it, so it requires an external client to download.

I guess I'm old, but it used to be common to - gasp! - install multiple applications. Now, the browser is basically everything.

pabs3
To be clear, I'm old too, vastly prefer applications rather than web browsers.
smt88
There's no reason to be condescending. Adding download friction for $0 of benefit for a developer makes no sense. I have had a BitTorrent client installed for more than 20 years, and I use it perhaps once a year these days (and only to download old video games that I can't get anywhere else).
saboot
It's been so long since I used a torrent that I don't know what program I'd use. I last used mutorrent but read they were putting crypto mining code in their client.
pabs3
rtorrent or transmission are good CLI and GUI open source clients.
NackerHughes
qBittorrent is cross-platform, open-source and malware-free. µTorrent has been not recommended for a while.
mobilio
Brave support that native.

Try it.

pabs3
A lot of Linux based OSes also have BitTorrent downloads.
badrabbit
How is it better than HTTPS+CDNs both performance wise and implementation/maintainance complexity wise?
gbraad
Blocked or otherwise impeded in some places: eg, unable to request peers from a tracker.
HellsMaddy
GitHub/GitLab releases makes it easy, free, and has virtually no overhead to distribute software to millions of users globally. Setting up torrents and making sure they stay seeded costs time and money, and torrents are less convenient for most users.
nhoughto
Yep came to say the same, because a lot of software is already hosted in a lower friction way by GitHub and similar. So unless you have a need/want to host it yourself and maybe benefit from the optimizations then why bother?
dagmx
Those aren't necessarily mutually exclusive though. The Git hosts could also provide their releases by seeding them as an option.
sodality2
You can add the web URL seed to the torrent, pretty cool feature.
newaccount74
We've had a few users complain that they had issues downloading large files from Github -- either S3 was blocked alltogether (China) or just very slow (Australia). That was a couple of years ago, don't know what the situation is like today.

I think offering BT as an alternative would be a good idea, especially for downloads > 100MB.

d--b
Raspberry pi OS has both.

Regarding hosting, I am not exactly sure that it changes anything. You still have to seed the torrent, so the file needs to be hosted somewhere. What is reduced is the bandwidth, but you rarely pay for bandwidth.

I do like the torrent option, but it’s a little more work for the person who downloads.

soperj
This is something I've wondered specifically for Musescore 4. They're releasing their Orchestral library as a separate 7.5gb download, but instead of using bit-torrent, they're making you download a download manager. Seems so bizarre.
bombcar
Could the download manager be a torrent client in disguise?
soperj
Why would they do that though?
cuteboy19
If it's not p2p then there is no point to bittorrent
LeoPanthera
Windows 10 claims it distributes updates in a peer-to-peer manner. Or at least it used to. Does that still happen?
waste_monk
Delivery Optimisation? Yes, that's still a thing.

It doesn't have to be internet based, you can turn it down to just peers on your local network, or disable it entirely (if you have WSUS infrastructure and no need for it).

u801e
Maybe not for individual software projects, but I would like to see something like this for installing and updating packages via the OS's package manager. Given the fact that package repositories have mirror servers, it should definitely be possible.
perihelions
Sounds like something that would have significant security impact, wouldn't it?
Jtsummers
BT is resistant to content spoofing. So unless the users download arbitrary .torrent files from untrusted sources, it will be hard to inject malware this way. You'd need to find hash collisions for the contents of the torrent. It would have some privacy implications, as you end up announcing "I'm using this OS!" to your peers (well, it's a high probability that you are at least).
perihelions
The same's true for ordinary package managers, but they've come with CVE's that allowed circumventing checksum validation [0]. Software mirror security doesn't seem to me (non-expert) a trivial or solved problem.

[0] https://news.ycombinator.com/item?id=8330386

banana_giraffe
I suspect a big part of the reason is "more work for very little gain" since most users will just download over HTTP given the choice.

I'd love to see some stats from somewhere like Internet Archive that offers both, it'd be great to be proven wrong, but I suspect a number basically 0% after rounding of their data is downloaded via peer-to-peer BitTorrent traffic.

shiado
Many ISPs give customers much better download speeds than upload speeds. I remember personally having problems using bittorrent and seeding too much destroying the network for all using it.
WatchDog
There are so many free CDNs and mirrors, especially for hosting open source. Also most open source projects aren't particularly bandwidth hungry, why bother?
gorgoiler
Spotify used to use BitTorrent, or something closely related.

Back in the early days, colleagues in my office would be falsely accused by IT of file sharing, when the only crime they were really committing was listening to Jack Johnson.

jldugger
I'm pretty sure OSS gets a free ride on mirrors, because of peering agreements: if traffic between networks is equal, nobody has a case that they're the stronger party on negotiations, so no money changes hands. If there's an asymmetry, theres' an argument that one side has more demand or need for peering and use that as leverage to demand payment. So smaller ISPs that want to balance out a lopsided consumer traffic, will set up mirrors to protect themselves from this.

Since OSS is free to redistribute, a loose federation of mirrors has set up to better match ISPs who want to send more bytes with projects that need to send a lot of bytes.

> Are there any big downsides to distributing torrents as opposed to traditional FTP/HTTP

Analytics is the big reason Mozilla shut down their community FTP mirrors, and bit torrent doesn't really solve that portion.

pugworthy
Valve hired Bram Cohen back in 2004, and AFAIK it is/was core to Steam's distribution system.
bbulkow
Bittorrent clients for windows and mac are generally malware infested pits of adware. Wherever i try to be virtuous and use one, i have to clean whatever i installed. I am sure i am not the only one.

Why not build your own install using the bittorrent libraries? The security model is more complicated, for one. If i was an architect and proposed that, i would have to understand the attack vectors and present why they are than the well known solution. And taking control of the software update channel is a massive risk.

Second, i have less control of the user experience. With http, i can pay for the right amount of bandwidth, or time on a cdn. If i implement bittorrent, i still have to have buy capacity, i just have a more complicated model for how much to buy.

Suggested updates can be spread over time - and need to be, for canary purposes. I think Android often pushes an app update over 4 or 5 days, by default? Steady state infra capacity, or even better, low priority which and be interrupted, is cheap.

Given the complexity and business risk (people can't download our software! Our binary got hijacked! Two code paths to test?) And the inexpensive nature of mirrors, and the competence of cdns, there are rarely causes where it would make sense.

Mandatum
Deluge, qBittorrent, Transmission, oh my!
AlchemistCamp
Blizzard has used BT for binary downloads for years. None of your fears came to pass.