Makes me wonder how much unnecessary load these streaming services are placing on global networks. Problems created by copyright. How much cheaper, easier and more efficient would things be if we were allowed to use torrents for media distribution?
A CDN box at a datacenter or ISP is more efficient than uploading through the last mile, especially if that last mile is a half-duplex shared medium like cable. I'm pro-p2p, pro-home-servers, and pro-piracy, but I don't think it's necessarily more efficient in terms of network load.
Not in comparison to other customers closer than the ISP’s data center. BitTorrent has significant overhead but can scale really well for extremely popular downloads like the latest WoW patch at it’s peak. It’s really hard to beat when your exchanging data with a dozen people in the same apartment complex.
Only if that apartment complex is wired with ethernet and a local switch [0]. GPON, cable, and DSL would all still go back to the ISP's core, transiting the last mile link twice. There are definitely efficiency gains to be had with decentralized identifiers and distributed systems, but with the centralized nature of the physical Internet they're not always a win.
[0] or roommates want the same files and their router supports hairpin NAT or the software uses local segment discovery.
The network load from P2P is about the same as it is from a very well distributed CDN. In theory the CDN could be a little better, e.g. if the network topology is such that a user uploading to another user on the same segment has to make a round trip through the head end, that will be somewhat less efficient than a CDN box which is already at the head end. But not by a lot. Linearly worse, not polynomially or exponentially worse.
And that's assuming the CDN is perfect. That it has a node in every place there are users. Which none of them actually do because the world is really big.
Worse, the places without them are the places with the slowest networks in general. If you're using twice as much bandwidth on some 100Gbps piece of fiber in New York City, the users aren't even going to notice that and the ISP can just send more light through the glass. But when there are a hundred users sharing some 40Mbps microwave link in some third world village and all trying to download the same thing, there isn't going to be a CDN node there and only having to suck it through the straw once multiplies the efficiency improvement by the number of users on the far end of the line, which in that case would be a factor of a hundred.
Moreover, you can get the efficiency benefit of the CDN by replacing the CDN box with a box in the same place that seeds popular content using a P2P protocol, without losing the efficiency benefits of P2P for the people not near one of those.
Thank you for saying, succinctly and clearly, what I've always figured was the case.
However.
If the majority of traffic came p2p, then the asymmetry of upload/download bandwidth on shared media like cable modems would start to be a significant issue, wouldn't it? Only on the local segment, but on _every_ such segment.
In that case, the seedbox-as-CDN-node route would be a no-brainer for ISPs to install.
Cable connections are typically asymmetric, but the average user doesn't max out their connection 100% of the time. Even if you have a 100 Mbps down 10 Mbps up connection, your average transfer rate might be 5 Mbps because most of the time your connection is idle. Which means you can feed it from peers with an upload cap of 5 Mbps, in theory, if they kept their uplinks saturated all the time. Or an upload cap of 10 Mbps if they all kept their uplinks half saturated all the time.
And you can still get 100 Mbps downloads that way by downloading from multiple peers at once, which doesn't change the average transfer rate required from each peer because then twenty people upload at 5 Mbps for 10 seconds instead of one peer having to upload at 5 Mbps for 200 seconds to transfer the same amount of data.
But the asymmetric link speed is artificial anyway. There is no technical reason they couldn't make fully symmetric cable modems -- in fact they did so in DOCSIS 4.0, though it's not widely deployed yet.
The main reason they're hesitant to offer it is, ironically, that P2P isn't sufficiently widely used. Because then hosting is scarce and people will have the incentive to do high volume hosting on their own connections. Which is bad for the ISP who offers a fast uplink, because they rely on over-subscribing their connections to offer the prices they do. If you give someone a fast uplink, it's quite easy to saturate it 24/7 by serving files to millions of users. So the ISPs don't want to do that.
But if P2P distribution was the default then that wouldn't be a problem because nobody would need to do that. You'd upload your thing once or some small number of times and then the users could get it from each other instead of millions of people all trying to get it from the one host. Everyone could have a 100 Mbps uplink and the average upload speed would still equal the average download speed, e.g. 5 Mbps, because that's all you need when the load is evenly distributed.
Streaming services are, for the most part, optimized to only send you a portion of the file at a time, send you lower bandwidth versions when the throughput drops below a threshold, or in the case of COVID lockdowns everyone temporarily dropped the max resolution of their services because there was a sudden surge in home media streaming.
When it comes to video torrents, I suspect a lot of people would start at looking for 4k rips (even if their devices didn't support it), or fall back to standard bluray rips. That's a huge amount of bandwidth when most people probably are fine with DVD or upscaled DVD quality.
I doubt those are set to the same quality. 1080p should use about twice the bandwidth for the video at most, but it's often less because there's less fine detail the higher you go, and also the audio data won't change in size.
I just did a quick test with x264 at quality 20 on two different clips. On a very high motion animated sequence, it came out at 2GB/3.4GB per hour. A live-action clip of people walking and talking came out at .65GB/1.3GB per hour. So that's 40% less and 50% less, respectively.
And with h.265 you can get something like h.264 720p size with h.264 1080p quality, everything else held equal.
Torrent is still better experience (in my particular case, lol).
Tried to watch Ted Lasso at peak lock down. Apple TV+ bumped the quality something to 480p since it was evening in nationwide lockdown so all streaming services are down to 40mbps (tho realistically Apple was barely moving at 3-4mbps). They do let you downloading episodes, but that's still at 1080p.
Fired up WebTorrent and was watching it glorious 4k in matter of minutes...
An article about the oldest torrent and nary a magnet link in sight? Sure, there's the hash in the picture, but that's weak.
What's the second oldest torrent? That's probably a better measure of the network health, because it doesn't get attention from articles about the oldest active torrent.
according to the thread here: https://news.ycombinator.com/item?id=10962253 an article about the then believed to be oldest torrent, turned out to be a few months less old than this one, so we need to look for the third oldest now to get a good answer.
on piratebay i found a torrent from 2008 that still works.
The first torrent I, and probably quite a few others, ever downloaded was some Linux distro that Bram Cohen shared with the internet as a demo of the protocol.
The first torrent I downloaded was one of the demo videos he put on his home page in 2001 (I think there were two). Never seen a reference to them since.
BakaBT still has the three torrents uploaded in december 2003 that the article mentions, and they are still up and well. 25 more between Jan.1 and Apr.30 of 2004, all alive as well.
Though I don't know if it is a good indicator. Even though, at the time, BakaBT was semi-private and not private, lessening the incentives to seed as one could download without limit by logging out, it still has a system that rewards seeding torrents with no/few seders.
Articles about torrents almost never contain a link to the torrent, even though magnet links are more authoritative than their own crappy https urls that can be arbitrarily changed. Journalists would rather mediate your access to source material than help you experience it directly. See also: Wikileaks.
It's funny to be part of that history. As the article says, "for a file to live on, at least one person has to keep sharing it", and for a while, I was that person (see the comments on that HN post).
About 10-15 years ago I went to download a torrent. It never had many seeders and got stuck at 94.8%. It was there for a few years. One day one of the very few, if not the only, humans with the whole torrent must have connected as it was 100%. Serendipity.
And then about 5 years later I lost the files in a hard drive crash. Tried to grab the torrent again and all this time later it is sitting at, you guessed it, 94.8%.
A (mostly bootleg) video bundle from a not super popular band. I'm pretty sure everything in there is easily accessible via Youtube now but it's the principle of the thing
I once loooong ago tried downloading a rare music album but it was stuck at 99.8% with 0 seeders and 2 peers (including me) for months so I was pretty bummed.
But I neglected to actually check the download folder. When I did, I found out it actually downloaded all the important files, just the logo of the upload group was still a .part file…
So I quickly downloaded another more popular release by the same group, copied over the logo and lo and behold, now it got to 100%!
That's nice! When it happens with me, it's like a torrent with 1 large file + the unnecessary files (txt, nfo from the upload group), and the missing .1% is on the file I want :/
But then, occasionally 1 or 2 gets completed, and then I'm happy
My torrent client (deluge) gives the option to check which files in the torrent I want to download when I add it. So bittorrent already has this facility, and it's probably how this situation arose in the first place.
I was talking about zipping all the files into one big zip/xz/whatever, torrents don't allow those to be broken up into "little files". All bittorrent clients I've used have the ability to select which files (-not- in an archive) you want to download, but default to "all"
I got a free Star Wars Jedi Knight: Jedi Academy CD set in 2003 because second disk had a deep scratch making less than megabyte of a 700MB data file unreadable. Found a torrent that wasnt a repack and it synced perfectly at 90 something % :-)
I've had that happen more than a couple of times at this point (often on Chinese or Japanese torrents), and I find it very strange. I always assume that some long replaced-but-not-thrown-away computer was turned on to find some old file, and it immediately and automatically resumed its mission to send bits of a 90s soap opera intercontinentally. Then after the owner finds what they need, the computer goes off and into the closet again, as I send those bits back to China, to the other 5 people who are silly enough to leave these things open for years.
I've kept sharing thousands, maybe tens of thousands of torrents over the years, keeping them even across Windows reinstalls. Btw it makes uTorrent start in about a minute.
Point being, not all of these torrents can be shared at the same time, so uTorrent uses some algorithm to cycle over the seeded files in small groups over time. Maybe that's what's happening in this scenario.
Some trackers also have re-seed requests which means that the tracker knows which users downloaded the torrent in the past and when it's no longer seeded and someone requests a reupload, those uses get that request into their inbox, so there's a chance that some of them have kept the files and can thus re-seed the torrent.
"We couldn't afford a wind machine for this shot, so we bought a leaf blower from a hardware store. We kept the receipt, so we can return it after we're done."
The stores don't like it. I know art directors who are banned from Ikea and Home Depot for returning ~$10,000 in a single month. At higher levels it's mostly Amazon returns. Mind you plenty of above board renting goes on as well
If they catch you it's fraud. If they catch you much less often (or cheaper) than doing it the 'right' way then the for profit group will default to the cheaper option.
This would explain why I've just seen it pulse towards the top of my seeded torrents today. I will be dragging this torrent around in my seeds until the day I die (or until some companies get together to eliminate torrents).
Regardless of what people think about copyright infringement, I see torrenting as an essential tool for the transmission and preservation of information.
In this day and age of convoluted terms of services (TOS/TOC), xyz as a service/subscription, and just plain DRM, no one really owns anything anymore. No one can be sure what version they do have is the original and immutable version. Edits and omissions happen because of licensing issues and just plan social activism. Problematic content should be acknowledged but not purged and erased.
It’s vital people have the means to share data regardless of what the powers that be allow to be shared.
By problematic he means stupid things like GTA Vice City losing parts of its original soundtrack due to idiotic copyright licensing issues and people having their games replaced with this "updated" version.
Also, you can't steal data. Copying is not stealing.
Jumping the fence at a concert and sitting in an empty seat is not stealing. But that doesn't mean it is moral, legal, or to the benefit of the musicians + concert venue that invested in the act.
> Jumping the fence at a concert and sitting in an empty seat is not stealing.
Of course not. That's trespassing, a real crime unlike copyright infringement. Physical space at the venue is limited and the actual artists are the most scarce resources there is. It makes sense to pay to see artists you love play live.
No idea why you're applying physical world concepts to digital "content" which can be infintely copied. It doesn't work. Mental gymnastics like copyright exist purely to enable rent seeking monopolists. It's not even morally defensible. Artists receive what, 30% of the profits of their record sales?
Artists receive a fraction of their profits because the distribution network is immensely valuable to them.
The 30% is pulled out of your ass. If I did all my own editing and released songs on Spotify without any assistance I would get a much higher cut. If I don't like Spotify I am allowed to sell mp3s on my own website. However - if I was trying to produce a high quality album a year and coordinate worldwide tours then it is worth it to me to share my profits with hundreds of talented artists/marketers/engineers on my team or in contract with my team.
ASCAP, SEACAM and BMG I think are the big three licensing orgs (USA). When you buy licenses from them they give you some paperwork that explains where the fees go. They even have a person explain it to you on the phone. (They took steps to pre-emptively high prices) I've cancelled those contracts awhile ago so I don't have handy numbers. It might even be on their sites?
Something like 10,000 peers have been keeping it alive for the past two years. Of course this mention is enough to drastically increase it's popularity.
I'm a member of a private torrent site that has been around for 15 years whose very mission is preservation. Everything is well seeded even the obscure stuff.
They are, but it's a little difficult (hence the longevity.) Now that you know it exists, a little determination will get you there, and it'd love to have you.
Yes, it's good enough that this discussion can only possibly refer to one place.
Is there a word for this interaction, where an act is taking place with the knowledge that the action is characterized as 'wrong' but done anyway to achieve an outcome that is ultimately benevolent and a benefit to posterity?
I was pretty disappointed when a games archive torrent site I frequented went down. It had every PS1 game in a nicely organized bundle, and PS2 games. Hundreds of terabytes of archived organized torrents, all gone now.
Same, and the outage yesterday gave me the willies. I've accumulated a vast upload in that time, and I'm only waiting to move somewhere without ISPs with packet limits to flesh out the disk shelves and get the better part of it archived.
The commercial web devoured the concept of the internet as a place where information was freely shared between any and all minds. I exported my bookmarks frequently all the way back to the 90s, and practically none of the dozens and dozens of communities that I was a part of as late as the 00s are still around.
I remember the year when the ad people started bullying the fan sites off the web because they were "stealing from creators"... by driving clicks away from traffic whoring aggregators that were worse, less curated, depersonalized, less focused in every way than what they cannibalized.
The modern web feels so empty and full of darkness in comparison. Nothing in it seems like it matters unless it enriches someone else's mission of accumulating wealth, and everything else is villainized that doesn't signal loudly for the sake of the dragons how much it costs to keep the lights on.
And I just realized the irony of saying this on HN.
I think it was originally for the benefit of the church, who didn't want people remixing the bible. They just had to add sweetener for the publishers so that they'd cooperate in censorship that would otherwise be to their disadvantage.
In Germany, using torrent means 700€ fine (ISP reports).
The goal of this might be that they...
- Want to punish people that can't do tunneling
- Want to prevent content distribution
- found a way to get money from foreigners that don't know (This happened to a friend of min, 1 month after he got installed in Hamburg)
I think the OP is overdramatising a little. Torrenting content that you're allowed to redistribute is of course perfectly fine – what is happening is the usual procedure of rightsholders attempting to log IPs that are illegally uploading copyrighted content, obtaining the contact data of the person behind the IP from the ISP and then sending out a cease-and-desist order.
Are you sure it isnt the case of a Prenda Law copycat? https://en.wikipedia.org/wiki/Prenda_Law We had one in Poland few years back after UPC customer data leak (name/IP). They filed court pre trial papers and started extorting people, finally vanished after one of the prosecutors got interested.
My partner and I were talking about Paprika. And how the file we downloaded didn't work halfway through. Realising we must have downloaded the same torrent long time ago. It was slightly romantic. Don't think I've ever really considered seeders and peers (prob peoplewith similar tasteas yours).
What immediately came to mind wasn't this but an Aussie matrix spoof Computer Boy. Congrats go out to the Fanimatrix crew and BitTorrent for holding up as well as it has. There seems to be less BT piracy from what I can see/guess but what exists I wish was less of ripping off blockbusters and more of preserving abandoned gems.
Wow! This short film is genuinely one of my favourites (and is generally under appreciated). The lead actor went on to be a very successful stage performer and I think he shied away from this film a bit. I remember pestering the director (who played Agent Smith) about 10 years ago to release a 1080p version. So happy to see someone else talking about it.
Oh and I can't write a comment about it and not acknowledge the tragic passing of Michael Falzon. What a loss
105 comments
[ 2.5 ms ] story [ 162 ms ] threadFor reference, a server in Germany in 2003 had bandwidth prices between €.8 and €2 per gigabyte [0]
[0] https://web.archive.org/web/20030810125406/http://www.hetzne...
[0] or roommates want the same files and their router supports hairpin NAT or the software uses local segment discovery.
And that's assuming the CDN is perfect. That it has a node in every place there are users. Which none of them actually do because the world is really big.
Worse, the places without them are the places with the slowest networks in general. If you're using twice as much bandwidth on some 100Gbps piece of fiber in New York City, the users aren't even going to notice that and the ISP can just send more light through the glass. But when there are a hundred users sharing some 40Mbps microwave link in some third world village and all trying to download the same thing, there isn't going to be a CDN node there and only having to suck it through the straw once multiplies the efficiency improvement by the number of users on the far end of the line, which in that case would be a factor of a hundred.
Moreover, you can get the efficiency benefit of the CDN by replacing the CDN box with a box in the same place that seeds popular content using a P2P protocol, without losing the efficiency benefits of P2P for the people not near one of those.
However.
If the majority of traffic came p2p, then the asymmetry of upload/download bandwidth on shared media like cable modems would start to be a significant issue, wouldn't it? Only on the local segment, but on _every_ such segment.
In that case, the seedbox-as-CDN-node route would be a no-brainer for ISPs to install.
Are we reinventing the topology of Usenet?
And you can still get 100 Mbps downloads that way by downloading from multiple peers at once, which doesn't change the average transfer rate required from each peer because then twenty people upload at 5 Mbps for 10 seconds instead of one peer having to upload at 5 Mbps for 200 seconds to transfer the same amount of data.
But the asymmetric link speed is artificial anyway. There is no technical reason they couldn't make fully symmetric cable modems -- in fact they did so in DOCSIS 4.0, though it's not widely deployed yet.
The main reason they're hesitant to offer it is, ironically, that P2P isn't sufficiently widely used. Because then hosting is scarce and people will have the incentive to do high volume hosting on their own connections. Which is bad for the ISP who offers a fast uplink, because they rely on over-subscribing their connections to offer the prices they do. If you give someone a fast uplink, it's quite easy to saturate it 24/7 by serving files to millions of users. So the ISPs don't want to do that.
But if P2P distribution was the default then that wouldn't be a problem because nobody would need to do that. You'd upload your thing once or some small number of times and then the users could get it from each other instead of millions of people all trying to get it from the one host. Everyone could have a 100 Mbps uplink and the average upload speed would still equal the average download speed, e.g. 5 Mbps, because that's all you need when the load is evenly distributed.
When it comes to video torrents, I suspect a lot of people would start at looking for 4k rips (even if their devices didn't support it), or fall back to standard bluray rips. That's a huge amount of bandwidth when most people probably are fine with DVD or upscaled DVD quality.
I haven't really seen much 4k to know how I'll feel about that jump. Though I've definitely seen some split-screen streams where I wished it was 4k.
I just did a quick test with x264 at quality 20 on two different clips. On a very high motion animated sequence, it came out at 2GB/3.4GB per hour. A live-action clip of people walking and talking came out at .65GB/1.3GB per hour. So that's 40% less and 50% less, respectively.
And with h.265 you can get something like h.264 720p size with h.264 1080p quality, everything else held equal.
Tried to watch Ted Lasso at peak lock down. Apple TV+ bumped the quality something to 480p since it was evening in nationwide lockdown so all streaming services are down to 40mbps (tho realistically Apple was barely moving at 3-4mbps). They do let you downloading episodes, but that's still at 1080p.
Fired up WebTorrent and was watching it glorious 4k in matter of minutes...
What's the second oldest torrent? That's probably a better measure of the network health, because it doesn't get attention from articles about the oldest active torrent.
according to the thread here: https://news.ycombinator.com/item?id=10962253 an article about the then believed to be oldest torrent, turned out to be a few months less old than this one, so we need to look for the third oldest now to get a good answer.
on piratebay i found a torrent from 2008 that still works.
Though I don't know if it is a good indicator. Even though, at the time, BakaBT was semi-private and not private, lessening the incentives to seed as one could download without limit by logging out, it still has a system that rewards seeding torrents with no/few seders.
It's funny to be part of that history. As the article says, "for a file to live on, at least one person has to keep sharing it", and for a while, I was that person (see the comments on that HN post).
And then about 5 years later I lost the files in a hard drive crash. Tried to grab the torrent again and all this time later it is sitting at, you guessed it, 94.8%.
But I neglected to actually check the download folder. When I did, I found out it actually downloaded all the important files, just the logo of the upload group was still a .part file…
So I quickly downloaded another more popular release by the same group, copied over the logo and lo and behold, now it got to 100%!
That day I actually felt like a hacker…
But then, occasionally 1 or 2 gets completed, and then I'm happy
Point being, not all of these torrents can be shared at the same time, so uTorrent uses some algorithm to cycle over the seeded files in small groups over time. Maybe that's what's happening in this scenario.
Some trackers also have re-seed requests which means that the tracker knows which users downloaded the torrent in the past and when it's no longer seeded and someone requests a reupload, those uses get that request into their inbox, so there's a chance that some of them have kept the files and can thus re-seed the torrent.
https://www.youtube.com/watch?v=G025oxyWv0E
I have worked on $100 million+ budget projects with Netflix and Apple. The amount of buying items to return within 30 days is absurd
In this day and age of convoluted terms of services (TOS/TOC), xyz as a service/subscription, and just plain DRM, no one really owns anything anymore. No one can be sure what version they do have is the original and immutable version. Edits and omissions happen because of licensing issues and just plan social activism. Problematic content should be acknowledged but not purged and erased.
It’s vital people have the means to share data regardless of what the powers that be allow to be shared.
/soapbox
By problematic content, do you mean stolen content? What do you suggest other than purging and erasing stolen content?
Also, you can't steal data. Copying is not stealing.
Of course not. That's trespassing, a real crime unlike copyright infringement. Physical space at the venue is limited and the actual artists are the most scarce resources there is. It makes sense to pay to see artists you love play live.
No idea why you're applying physical world concepts to digital "content" which can be infintely copied. It doesn't work. Mental gymnastics like copyright exist purely to enable rent seeking monopolists. It's not even morally defensible. Artists receive what, 30% of the profits of their record sales?
The 30% is pulled out of your ass. If I did all my own editing and released songs on Spotify without any assistance I would get a much higher cut. If I don't like Spotify I am allowed to sell mp3s on my own website. However - if I was trying to produce a high quality album a year and coordinate worldwide tours then it is worth it to me to share my profits with hundreds of talented artists/marketers/engineers on my team or in contract with my team.
I pulled it off a search engine. And that was one of the better figures. Do you have better data?
As famous quote goes, 'Slavery, Apartheid and Holocaust were legal'
> (imitating a woman defending ridiculous divorce settlements): "That's what the law says!"
> (himself): "A hundred years ago I could beat you with a mop handle, that's what the law said! Doesn't make it right!"
[0]: https://youtu.be/x0gaYyNk7QA?t=603
https://imgur.com/a/ur0dUb8
Something like 10,000 peers have been keeping it alive for the past two years. Of course this mention is enough to drastically increase it's popularity.
But it's impressive for a public tracker.
Yes, it's good enough that this discussion can only possibly refer to one place.
I remember the year when the ad people started bullying the fan sites off the web because they were "stealing from creators"... by driving clicks away from traffic whoring aggregators that were worse, less curated, depersonalized, less focused in every way than what they cannibalized.
The modern web feels so empty and full of darkness in comparison. Nothing in it seems like it matters unless it enriches someone else's mission of accumulating wealth, and everything else is villainized that doesn't signal loudly for the sake of the dragons how much it costs to keep the lights on.
And I just realized the irony of saying this on HN.
I don't even go close to a torrent site anymore.
It's just not worth being accused of something.
And to the Copyright Net Nannies--what's the outcome?
Musicians are paid less than before the crackdown.
Books are selling at an all time low.
List (as of 2019) of public domain versions.
https://www.cambridge.org/bibles/about/rights-and-permission...
Behind all of those is lawyers. Contemporary copyright exists exclusively for the benefit of lawyers and people who can afford lawyers.
The gov DNS block sites, but that’s it really.
I hope is the 3rd
Even for legitimate purposes?
Oh and I can't write a comment about it and not acknowledge the tragic passing of Michael Falzon. What a loss