Torrents should really be the preferred method for distributing AI model weights. Why rely on a single point of failure like Hugging Face? BitTorrent was made for exactly this.
In my experience public torrents often die as they grow older. It doesn't help that BitTorrent V1 makes long term seeding annoying, and BitTorrent V2 is almost never used.
I never understood this, is there anything that makes it difficult for the original uploader, the one that supposedly offers the file directly, to offer a torrent instead for the same amount of time?
As far as perennity is concerned it seems strictly better.
> Distros are a bad use case for P2P anyway since you depend on upstream as soon as you start upgrading and installing packages.
This is true for any distribution method not just p2p. You can even download a nightly through torrents so what does it matter how the data is transferred if it’s always going to require `apt update`?
Yeah, I just use the "netinstaller" ISOs since it's much smaller and never needs to be updated. If I had a need for air-gapped/offline installs I'd either download a larger ISO or just manually install packages from .deb as needed.
You can trivially have storage deduplication for the files served via torrent, transparent to the protocol. The most trivial version of this that you can do today with pretty much any client is having a single directory containing files serving multiple overlapping torrents.
Yes, models are a good use for P2P especially if everyone agrees to share the same torrent and someone (or a cohort) commit to seeding for the long haul.
Because history is path-dependent, as engineers keep learning over and over again. It doesn't matter whether Plan9 is theoretically superior to Linux - we're all on Linux and nobody's porting all the apps over.
The biggest problem with BTv1 was the lack of per-file checksumming, and swarm merging (i.e. individual files have shared seeding pools across torrents). BTv2 specs the latter, but I think only BiglyBT actually implements it. Having both of those features from the get-go would've gone a LONG way to fixing the dead torrent problem.
ages ago I tried using IPFS to more or less accomplish this, I imagined it to act more like a weights/training data network fs that everyone would be able to participate in.
HF is meant to be a single point of control. AI models and Linux distros aren't usually for normies, so distribution via torrents would make sense, especially to save the provider some bandwidth. Ubuntu has been offering torrent downloads for ages. No mention of torrents on HF. I believe most downloads will soon be account/EULA-walled.
great initiative, it's really weird seeing efficiencies get rediscovered in the LLM audience, because these efficiencies aren't even what I would consider to be old
Honestly, models are torrents will end any effort from the big AI labs to stop open models. No way to prevent weights from being shared, just like mobies. Genie is out of the bottle
I've seen this used for distributing container images in networks with awkward network topologies (e.g. a lot of bandwidth within a site or sub-site but limited bandwidth to central registries)
Once I was using Blizzard's downloader to install something (StarCraft, Diablo, I don't remember), and it was kinda slow. I disabled P2P downloads and speed skyrocketed, and I said "Huh, this was unexpected".
When P2P downloads disabled you could see the list of CDNs you're downloading from and mine had a single IP on that list. It looked familiar. Then it dawned on to me. It was the Akamai server which we were hosting in our system room, at 15 minutes of driving distance. After a chuckle, I went to get a cup of tea, because that was entertaining than the game itself.
Then of course, I dived into whatever I was installing that night.
So how come was using your own ip? Become a Diablo node installer so you downloaded from there, like how can they convert your own akamai instance into anode without you knowing?
It's not my own IP, but our IP block. As an institution we have a couple of big networking related hats, so we have a B block (/16 network) for ourselves. The IP was from that block.
However, these network operations (and CDN related stuff) are not managed by us, by a couple of high profile guys next door. I knew we had an Akamai node, because the machines were shipped recently and we had a chat about it.
Funnily, some of the performance optimizations on Akamai's disk access algorithms (SSDs were not dime a dozen back then, so people were still optimizing for spinning drives) was apparently developed by a friend of mine, but I don't know any details about that.
Several companies tried this for distributing software.
It was very controversial. Users were angry that software companies were using their internet bandwidth to distribute their software. Made a lot of people angry.
As someone who was annoyed by it at the time, I don't think it's a good example of that.
This was in an era of much more limited upstream bandwidth. The p2p nature was obscured from non-technical users in some implementations. So on and so on. I'm sure there are plenty of writeups from that era detailing why it was a bad idea.
These companies already used capable CDNs to distribute patches. It was a way for them to shift their CDN bill onto their paying customers.
When you have barely the upstream bandwidth to send the ACK stream of your downloads, anything sending something becomes a big burden on your system.
Not everyone has the luxury of fully symmetric internet connections even in this age. Yes, my uplink speeds are absurd when compared to a decade ago, but when I ratio it to my downlink, it's still slow.
I have no comment on residential proxies. Somebody doing something in my name is unacceptable to put it very mildly.
I’ve been wondering when this will come. The days are numbered for abliterated models to be published on HF I think. Why wouldn’t the government want a central control there?
If I search for 'uncensored' there are no torrents available. Uncensored models should be top priority, especially now that Nvidia owns HuggingFace and will enshittify the platform in accordance with upcoming US laws.
That was my my first search too. I know they are still listed on HF but every time I try and use one, the links are dead or the size is beyond my scope. Was hoping for a fresh batch. Ill check back.
Torrents always seemed like the more sensible way to distribute model weights.
Though I have been disappointed that most of these have been spurred on by the misleading claim that abliterated models were being taken down from HuggingFace because they removed an abliterated model. HF took one abliterated model down because the uploader was spamming people that requested access with sketchy requirements to pay for it.
Plus, there are a bunch of these types of sites, all of them have a couple of models and otherwise completely dead.
There's also the problem of catching malicious models that have been fine tuned to exfiltrate credentials. It would be nice to have means of checking hashes against the HF versions (or against other reputable sources). I'm guessing this is probably easy when just serving the same folder as what HF serves.
Speaking to the "uncensored model" angle: there's little reason to distribute abliterated weights anyway. Instead of orthogonalising the weights that write back to the residual stream, you can just orthogonalise the activations themselves. It's equivalent.
Orthogonalising activations at runtime is computationally cheap. Just distribute the refusal vectors (few thousand floats per layer), then run against the stock weights. Antirez's DS4 already supports this: https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9...
Abliterated weights are just a bad habit we've gotten into. It's also deeply suboptimal from a precision point of view to take a model that's already been QATed and distributed in pre-quantised form (DeepSeek V4, Kimi K2.5 or K3...), modify its weights, and re-quantise it. Similarly, abliterated models regain some of their refusal when they're re-quantised after abliteration -- avoidable by keeping the two separate.
So... distribute a LoRA (or equivalent) that modifies the base weights with the abliteration vectors. That makes sense as it would be possible to try different abliterations and keep the storage space down.
Could you please explain why? It's an additive update to the weights, adding an outer product of a vector with its transpose, which must have rank 1. What am I missing?
Distributing the vectors themselves isn't (yet) common practice, because people have gotten used to just putting the full modified weights up on HuggingFace's huge free storage.
I believe abliterated models are mostly still created at this point because they're "universal": they can not only be run locally, and on cloud GPUs, but also on "managed inference" providers (i.e. services where you hand them a model URI, and they blindly fetch it, load it, and give you inference access to it through standard text/chat-completion APIs. Think HuggingFace Spaces, or Google CoLab, or CloudFlare Workers AI.)
Such managed inference providers have (for now) plausible deniability of behaving ethically (at least enough that they don't get boycotted / scare away investors) due to them being "blind" to what gets run on their systems. They're acting as the inference equivalent of data transit carriers.
But I don't think it would be possible for managed inference providers to publicly expose "runtime activation steering" in the way antirez's DS4 does, without that reading much more explicitly as them inviting unethical workloads.
(Yes, there are other things you can do with runtime steering. But almost all of those things are workload-specific, relying on you privately tuning to the needs of your own dataset. And if you can do that, you can run inference without the help of a managed inference provider. The only time a customer will come along with a pre-made runtime-steering vector file in hand, is if that vector is an alignment-orthogonalization vector.)
These providers can also just ignore insinuations they're being "unethical" when people come to them with steering vectors in hand. Nobody has to listen to the scolds.
> Similarly, abliterated models regain some of their refusal behaviour when they're re-quantised after abliteration
Thanks for this information, Q4 seemed fine but they reappeared again in Q5 with an vengeance, I couldn't understand why. Very Strict and I've only found one jail break that barely works around 60% of the time.
Yeah I use a custom fork of llama.cpp that has an abliteration feature that basically does this. It's sloppily vibe coded and I don't have time to coordinate on a way to do this cleanly upstream, but it's absolutely possible and saves a lot of time and bandwidth from being wasted
I didn’t know about that method, thank you. I’ve needed a local model for security research but Qwen 27b abliterated did 30% worse than the stock weights on my internal benchmarks (I just skip the public benches now, it’s honestly useless noise on an operational level).
A question that comes up in my mind, since I don't fully understand how this works, is how does this affect runtime performance. It feels like abliterated weight models would work faster than some extra runtime operations?
The vector that needs to be checked is length n while computing that vector requires n^2 operations. I haven't benchmarked it but I expect the performance overhead to be a rounding error.
Let's imagine that a model is pulled from HF by order of the new overlords or because of some other kind of censorship. Wouldn't the question of it having a Free license or not potentially become a complex legal issue?
But if the point is to be "censorship-free" then why respect licenses at all? They are among main choke points today. If authoritarians use licenses to censor political, artistic, scientific, etc., speech that they want to block, does that make the censorship more respectable?
When Anthropic sues a Chinese lab for IP infringement and get a court to put a bar on that software, does it THEN get pulled from Pirate Face?
I know that an awful lot of international negotiations have become focused more and more on questions of "IP" - licensing battles are already intensely politicized and it's hard to imagine a future where it doesn't get much much worse. Imagine N Korea coming after you for violating a license that they worked hard to control and leverage.
Don’t laugh. Models like DS 4.1 Flash, which is unguardrailed and insanely capable, and cheap, will become illegal in the USA, and this service will become the new Pirate Bay.
You will rent guardrailed intelligence from approved providers and will not be able to own it. Just like your music and your films. And this time it won’t even be Metallica’s fault, it’ll be in the name of safety and child protection.
203 comments
[ 0.21 ms ] story [ 27.9 ms ] threadEDIT/ Yes they did, that no longer seems to be the case though
https://x.com/MistralAI/status/1833758285167722836
As far as perennity is concerned it seems strictly better.
Nor do they need to. 99% of everything is crap, and not worth prescribing except for a random sample so future historians can study our crap.
You can see this with many Linux distros: there is no single Debian torrent that people seed for years because there's always a refreshed version.
Distros are a bad use case for P2P anyway since you depend on upstream as soon as you start upgrading and installing packages.
This is true for any distribution method not just p2p. You can even download a nightly through torrents so what does it matter how the data is transferred if it’s always going to require `apt update`?
[0] https://specs.ipfs.tech/ipns/ipns-record/
If this site represents a coordinated datahoarding effort then there will be at least a few people who will seed indefinitely.
but I guess they are
When StarCraft 2 was lauched, the installer (before Battle.net installer crapware) had a complete graphical visualization of seeders & leechers
To get the file out to 100s or 1000s of machine they would often use private bittorent to distribute the file out.
Once I was using Blizzard's downloader to install something (StarCraft, Diablo, I don't remember), and it was kinda slow. I disabled P2P downloads and speed skyrocketed, and I said "Huh, this was unexpected".
When P2P downloads disabled you could see the list of CDNs you're downloading from and mine had a single IP on that list. It looked familiar. Then it dawned on to me. It was the Akamai server which we were hosting in our system room, at 15 minutes of driving distance. After a chuckle, I went to get a cup of tea, because that was entertaining than the game itself.
Then of course, I dived into whatever I was installing that night.
However, these network operations (and CDN related stuff) are not managed by us, by a couple of high profile guys next door. I knew we had an Akamai node, because the machines were shipped recently and we had a chat about it.
Funnily, some of the performance optimizations on Akamai's disk access algorithms (SSDs were not dime a dozen back then, so people were still optimizing for spinning drives) was apparently developed by a friend of mine, but I don't know any details about that.
It was very controversial. Users were angry that software companies were using their internet bandwidth to distribute their software. Made a lot of people angry.
This was in an era of much more limited upstream bandwidth. The p2p nature was obscured from non-technical users in some implementations. So on and so on. I'm sure there are plenty of writeups from that era detailing why it was a bad idea.
These companies already used capable CDNs to distribute patches. It was a way for them to shift their CDN bill onto their paying customers.
Not everyone has the luxury of fully symmetric internet connections even in this age. Yes, my uplink speeds are absurd when compared to a decade ago, but when I ratio it to my downlink, it's still slow.
I have no comment on residential proxies. Somebody doing something in my name is unacceptable to put it very mildly.
If I understand the description correctly, microsoft still does this!
https://support.microsoft.com/en-us/windows/privacy/windows-...
Though I have been disappointed that most of these have been spurred on by the misleading claim that abliterated models were being taken down from HuggingFace because they removed an abliterated model. HF took one abliterated model down because the uploader was spamming people that requested access with sketchy requirements to pay for it.
Plus, there are a bunch of these types of sites, all of them have a couple of models and otherwise completely dead.
There's also the problem of catching malicious models that have been fine tuned to exfiltrate credentials. It would be nice to have means of checking hashes against the HF versions (or against other reputable sources). I'm guessing this is probably easy when just serving the same folder as what HF serves.
Orthogonalising activations at runtime is computationally cheap. Just distribute the refusal vectors (few thousand floats per layer), then run against the stock weights. Antirez's DS4 already supports this: https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9...
Abliterated weights are just a bad habit we've gotten into. It's also deeply suboptimal from a precision point of view to take a model that's already been QATed and distributed in pre-quantised form (DeepSeek V4, Kimi K2.5 or K3...), modify its weights, and re-quantise it. Similarly, abliterated models regain some of their refusal when they're re-quantised after abliteration -- avoidable by keeping the two separate.
I haven't wrapped my mind around this
https://www.reddit.com/r/LocalLLaMA/s/iksvmXBzuC
Distributing the vectors themselves isn't (yet) common practice, because people have gotten used to just putting the full modified weights up on HuggingFace's huge free storage.
Such managed inference providers have (for now) plausible deniability of behaving ethically (at least enough that they don't get boycotted / scare away investors) due to them being "blind" to what gets run on their systems. They're acting as the inference equivalent of data transit carriers.
But I don't think it would be possible for managed inference providers to publicly expose "runtime activation steering" in the way antirez's DS4 does, without that reading much more explicitly as them inviting unethical workloads.
(Yes, there are other things you can do with runtime steering. But almost all of those things are workload-specific, relying on you privately tuning to the needs of your own dataset. And if you can do that, you can run inference without the help of a managed inference provider. The only time a customer will come along with a pre-made runtime-steering vector file in hand, is if that vector is an alignment-orthogonalization vector.)
Thanks for this information, Q4 seemed fine but they reappeared again in Q5 with an vengeance, I couldn't understand why. Very Strict and I've only found one jail break that barely works around 60% of the time.
[0] https://arxiv.org/abs/2406.11717
Where Hugging Face AI models never die, and are immortalized as torrents.
Claim yours: https://pirateface.co/claim?ref=qwen3.8-27b-uncensored-fp8
Really odd approach.
Where Hugging Face AI models never die, and are immortalized as torrents.
Claim yours: https://pirateface.co/claim?ref=skeptical_genius
- https://meet.hn/
- https://book.keybase.io/account#adding-proofs
Those tell you to put a text in your HN profile “about” section.
But if the point is to be "censorship-free" then why respect licenses at all? They are among main choke points today. If authoritarians use licenses to censor political, artistic, scientific, etc., speech that they want to block, does that make the censorship more respectable?
When Anthropic sues a Chinese lab for IP infringement and get a court to put a bar on that software, does it THEN get pulled from Pirate Face?
I know that an awful lot of international negotiations have become focused more and more on questions of "IP" - licensing battles are already intensely politicized and it's hard to imagine a future where it doesn't get much much worse. Imagine N Korea coming after you for violating a license that they worked hard to control and leverage.
Where Hugging Face AI models never die, and are immortalized as torrents.
Claim yours: https://pirateface.co/claim?ref=reedf1
"Trillions of parameters? Trillion my ass. Back in my day we got by with millions. And now here we are, washed up has beens."
Where Hugging Face AI models never die, and are immortalized as torrents.
Claim yours: https://pirateface.co/claim?ref=ganesshkumar
You will rent guardrailed intelligence from approved providers and will not be able to own it. Just like your music and your films. And this time it won’t even be Metallica’s fault, it’ll be in the name of safety and child protection.
They will after a few incidents where unguardrailed local models are used to hack and stop the water supply.
ThePirateBay is tolerated. Market places where drugs/guns are sold are not, they are all infiltrated and shut down after a while.
Try asking it about Tianmen Square. I use DS myself, but let's not kid ourselves.
Where Hugging Face AI models never die, and are immortalized as torrents.
Claim yours: https://pirateface.co/claim?ref=mortalsilk
Where Hugging Face AI models never die, and are immortalized as torrents.
Claim yours: https://pirateface.co/claim?ref=skeledrew
I claimed the "itomato" handle on Pirate Face.
Where Hugging Face AI models never die, and are immortalized as torrents.
Claim yours: https://pirateface.co/claim?ref=itomato