With this in mind, having LLM-powered NPCs would be a beautiful thing. There's been limited testing on PC games (PUBG Ally), but I can imagine it will be the standard.
For somebody who hasn't kept up with all the different open source models and what token rates mean, how would using this compare to something like codex or claude? I assume it would be much slower and less capable..?
The model they're running is Qwen 3.8 27b. It's maybe somewhere around Opus 4.6 level, it's quite good. I didn't check the benchmarks, that's from memory/vibes.
The ~issue is though that they're using a pretty low quant, 2 bit. 4 bit are _pretty_ close to full ability, maybe degraded a bit. 2 bit is _noticably_ worse. It's about as small of a quant as you can get while still pretending it works, for this model.
This won't replace claude for you, unless you baby it a lot and are okay with it being kind or derpy.
Qwen3.8-27B is surprisingly effective, somewhere in the vicinity of Opus 4.6, but it needs to do a lot of inference-time computation. i.e. anything moderately involved requires xhigh reasoning and it will spend literally tens of thousands of tokens turning over a problem before emitting the answer. The answer/code/whatever is often surprisingly good, but if you’re in the low tens of tok/s you’ll be waiting a long time to get it.
As sibling said, you won’t hit the same quality using a 2 bit quant like this project, which is unfortunate but that’s the reality today.
The project's primary site should not be GitHub, and I am serious. Playstation 5 is not some historical curiosity and Sony might strike the page down any hour now.
Those projects should default to hosts less geared toward corporate governancy; custom internet domain being the first requirement.
I'm sure Sony has an opinion of the jailbreaks that enable this, but they really don't give a shit about anyone running llama.cpp. Why would they? They're just glad you're not pirating games.
There's no feasible way to avoid takedowns/DMCA in 2026 unless you do TOR shenanigans. Even with a custom domain, Sony could just DMCA the web host for the project.
Any recommendations for code that’ll one shot pull a GitHub repo zip from an repo url and announce a magnet torrent?
Looking for similar UX to https://archive.softwareheritage.org/save/ (which archives an source control repo on demand from a url, but is still a single point of failure)
Torrents. Got the leaked GTA6 videos by torrenting and they're still easily available on public trackers. Don't see why projects like these couldn't be distributed similarly if shit hits the fan.
>Radicle is an open source, peer-to-peer code collaboration stack built on Git. Unlike centralized code hosting platforms, there is no single entity controlling the network. Repositories are replicated across peers in a decentralized manner, and users are in full control of their data and workflow.
It'd be kind of interesting if someone started a public git host whose main value add was being judgement proof, like Anna's Archive, torrent sites, etc. The methods exist, it's just a matter of implementation and getting a critical mass. Obviously TOR-hidden-service first, but then making it available to naive hidden browsers would be its own public service.
I hate the centralization of centralized git watering holes, but it is what it is. And frankly getting older and not as able to keep up, having all the cool projects in one place would massively help my street cred.
That's why I specified open internet, because while it's possible to make hidden services, the Domain Name System will pretty much always be subject to censorship.
The problem, of course, still arises when you want to guide users to your hidden service, when sharing the link anywhere will get it taken down. This already happens with magnet links.
That was my point though. Anna's Archive maintains several "open web" domains, so when one ends up getting censored it's not a big deal, they just spin up more.
If every cybersecurity researcher starts hosting their most potent source code on a single platform, you're gonna start seeing enforcement speed up to the point where new domains are taken down in days (or faster), and there will be substantial efforts to attack the source with raids and prison time. If it gets bad enough, you will literally start to see US agents traveling internationally to perform targeted strikes. This is assuming the platform wouldn't start censoring code based on its provenance or purpose, but that would make everyone leave.
It's been proven time and time again that the only way to pull something like this off is to be completely untraceable. If the feds can ever find you, that's it. Which creates difficulty because in order for somebody to be able to visit your site with a normal web browser, they need to be able to trace something, and that can always be attacked. Even if that doesn't reach all the way to the source, it means you have to both produce infinite new traceable points and somehow distribute the knowledge to your clearweb users too.
We're talking about a centralized code hosting platform generally immune to takedowns, where yes, software that runs on the PS5 could be hosted, but also that would have far wider implications. I am not implying this particular software is so risky and illegal that the feds will be trying to personally strike the authors and uploaders. I am saying that such a service is fundamentally really really unsafe to operate because of the types of software it would inevitably attract, and how interesting that would make it for national security.
The "but don't you know git is distributed" commentary always sounds clever, but it really isn't.
GitHub isn't just used to host your actual git work tree and everyone knows that, it's an issue tracker, an easy way for projects to get contributions as everyone already has a GitHub account, an easy way to receive bug reports, CI, a website for your project, a popularity metric like GitHub stars (For better or worse).
Nothing you've said invalidates my point. I'm not saying that everyone must use it distributed, just that it is under"known" and underused this way, given other problems that centralization has caused.
It is just a git repository. Sure, you may lost some issues if it is stuck down and no one made a backup, but the important part, the code is now spread all over the internet. Should they want to migrate, all they need is to ask those who used GitHub as the origin to do a "git remote origin set-url".
That's one of the best things with git (among with other DVCSs), it is incredibly resilient. I remember during an interview of Linus Torvalds about backups, he says he doesn't backup his work in the traditional sense, he has no real reason to when everything he does is copied millions of time.
The real point of GitHub for such a project is because GitHub is really popular, and it has a social network aspect to it. So yes, you are going to lose it if you get kicked out of GitHub, but by choosing a lesser known platform you won't get it in the first place. Also, their website is not on GitHub, so kicking them out of GitHub won't make them disappear completely.
Conceivably this should also work on the BC250 cards, which are ex-cryptocurrency mining motherboard built around AMD's "Cyan Skillfish" APU which is a cut-down version of the PlayStation 5's custom silicon. You can still pick them up on ebay for $150-$200.
Flashing the bios takes ten minutes with a thumb drive.
It’s hard to get wrong but guess some people would be nervous enough about that to pay $20 extra.
I'm sure someone will point out some spec that's better on the PS5, but the extra $200 will make up the difference if you spend more time looking than the 15 seconds I just did.
>I'm sure someone will point out some spec that's better on the PS5
They're not even in the same ballpark - the PS5 is more than an order of magnitude better in GPU performance.
Despite how much it does matter, RAM is not the only thing you need for running LLMs - that N150 is not going to run anything but really tiny models at any sort of reasonable speed - certainly nothing that approaches using all of its ram. The GPU in the PS5 is ~25x faster. And if you're running games the RAM doesn't really affect performance at all once you have "enough" for a given game.
You're right that you probably could build something quite a bit faster than a PS5 for the same price though - if you can build/find everything but the GPU for ~250 USD and slap a 16gb 9060xt (~25tflops @32bit compared to the PS5 gpus ~10tflops @32bit) in it for another ~350 USD it should theoretically do a better job - the GPU in the PS5 was on par with mid-high end PC GPUs on launch, but that was like 5 years ago.
N150 isn't really what you'd want to use for a gaming machine. At a budget of mid $300s - the best (new) Mini PC you're going to find right now is probably this unit[1] with a Ryzen 7 7735U (680M iGPU) and 16GB RAM. Not the most powerful device - but I have a similar one I use to play emulator games, indie games on steam - throw Steam OS on it and it's a great living room device. Expand that budget to $530 and you can get this guy[2] with a Ryzen 7 255 (780M iGPU) - which will play modern games (on low settings for many).
It's been a little bit since any major hacks have hit the news but this is a time honored tradition partcularly with the Playstation. Way back when the PS3 arrived the US government was considering export sanctions because they were too powerful. I believe some universities even kept a cluster of PS3s for their distibuted computing courses or for research. Sony even had an approved way to run Linux.
Probably, the benefit of using PS5 for this purpose would be the same as a standalone Mac mini 16GB($900); it comes with the rest of the computer. From [1][2]:
A 20 tps Qwen3.8-27B at 2 bits quantization is limited but not useless. I've been putting one on "read this codebase and figure out XYZ" for me, where the typos are less consequential.
Even though it only runs qwen 3.8 27b at a 2 bit quant, it's still highly viable, especially for things where you just need a constant token stream of slightly below average intelligence, ie this would be great for things like:
- monitoring server error rates
- reviewing security footage
- watching plant growth / controlling watering
etc. Any long running task you can now just slap a PS5 on it. 70 cents a day in power cost means you now get ~2M output tokens per day (~$3.50 worth of usage). It's not the worst thing in the world if you have one lying around. 2bit isn't even that terrible - it's ~80% the accuracy of 27b at full quant, which should put it at around GPT-5 levels of intelligence (which honestly is crazy in its own right).
Wonder if there is an advantage to chain multiple PS5 with this and spread the load over, or just have it stand by for an orchestrator to use it for those tasks across. Cost definitely comes in a good factor, especially leaving it just sitting at home.
82 comments
[ 0.26 ms ] story [ 17.4 ms ] threadhttps://www.ncsa.illinois.edu/2003/05/27/playing-the-superco...
The ~issue is though that they're using a pretty low quant, 2 bit. 4 bit are _pretty_ close to full ability, maybe degraded a bit. 2 bit is _noticably_ worse. It's about as small of a quant as you can get while still pretending it works, for this model.
This won't replace claude for you, unless you baby it a lot and are okay with it being kind or derpy.
As sibling said, you won’t hit the same quality using a 2 bit quant like this project, which is unfortunate but that’s the reality today.
Those projects should default to hosts less geared toward corporate governancy; custom internet domain being the first requirement.
Looking for similar UX to https://archive.softwareheritage.org/save/ (which archives an source control repo on demand from a url, but is still a single point of failure)
And nothing is stopping you from renting a VPS from a provider based in that country and running gitlab or something.
The worst part is sharing around a sketchy URL but, that just comes with the territory when you're in the DMCA dodging parts of the internet.
Renting a VPS from a country that genuinely doesn't is pretty risky for numerous reasons, least of all international sanctions.
Private keys and private endpoints is all we need
>Radicle is an open source, peer-to-peer code collaboration stack built on Git. Unlike centralized code hosting platforms, there is no single entity controlling the network. Repositories are replicated across peers in a decentralized manner, and users are in full control of their data and workflow.
I struggle to think of anywhere on the entire open internet you could put something like this where it could not be taken down.
I hate the centralization of centralized git watering holes, but it is what it is. And frankly getting older and not as able to keep up, having all the cool projects in one place would massively help my street cred.
The problem, of course, still arises when you want to guide users to your hidden service, when sharing the link anywhere will get it taken down. This already happens with magnet links.
It's been proven time and time again that the only way to pull something like this off is to be completely untraceable. If the feds can ever find you, that's it. Which creates difficulty because in order for somebody to be able to visit your site with a normal web browser, they need to be able to trace something, and that can always be attacked. Even if that doesn't reach all the way to the source, it means you have to both produce infinite new traceable points and somehow distribute the knowledge to your clearweb users too.
GitHub isn't just used to host your actual git work tree and everyone knows that, it's an issue tracker, an easy way for projects to get contributions as everyone already has a GitHub account, an easy way to receive bug reports, CI, a website for your project, a popularity metric like GitHub stars (For better or worse).
That's one of the best things with git (among with other DVCSs), it is incredibly resilient. I remember during an interview of Linus Torvalds about backups, he says he doesn't backup his work in the traditional sense, he has no real reason to when everything he does is copied millions of time.
The real point of GitHub for such a project is because GitHub is really popular, and it has a social network aspect to it. So yes, you are going to lose it if you get kicked out of GitHub, but by choosing a lesser known platform you won't get it in the first place. Also, their website is not on GitHub, so kicking them out of GitHub won't make them disappear completely.
And at that price point or a little bit more you might as well just get an RTX 3060 and save yourself a lot of hassle (you also get DLSS, DLAA etc.)
I'm sure someone will point out some spec that's better on the PS5, but the extra $200 will make up the difference if you spend more time looking than the 15 seconds I just did.
(or if you just build a PC from parts yourself)
They're not even in the same ballpark - the PS5 is more than an order of magnitude better in GPU performance.
Despite how much it does matter, RAM is not the only thing you need for running LLMs - that N150 is not going to run anything but really tiny models at any sort of reasonable speed - certainly nothing that approaches using all of its ram. The GPU in the PS5 is ~25x faster. And if you're running games the RAM doesn't really affect performance at all once you have "enough" for a given game.
You're right that you probably could build something quite a bit faster than a PS5 for the same price though - if you can build/find everything but the GPU for ~250 USD and slap a 16gb 9060xt (~25tflops @32bit compared to the PS5 gpus ~10tflops @32bit) in it for another ~350 USD it should theoretically do a better job - the GPU in the PS5 was on par with mid-high end PC GPUs on launch, but that was like 5 years ago.
[1] https://www.amazon.com/dp/B0GGM6BKN9 [2] https://www.amazon.com/dp/B0FX2TV3HX
Shameless plug - I methodically rank the best possible Mini PC deals on https://MiniPCs.zip for this exact purpose.
Ever since RAM prices grew 6x, these old appliances suddenly became a lot better value for money.
Tools like these will probably just spike the prices of second hand PS5s though.
2: https://www.techpowerup.com/gpu-specs/playstation-5-gpu.c348...
The readme oversells it by pretending it would be something actually usable, when instead it is a cool achievement to make it happen.
- monitoring server error rates
- reviewing security footage
- watching plant growth / controlling watering
etc. Any long running task you can now just slap a PS5 on it. 70 cents a day in power cost means you now get ~2M output tokens per day (~$3.50 worth of usage). It's not the worst thing in the world if you have one lying around. 2bit isn't even that terrible - it's ~80% the accuracy of 27b at full quant, which should put it at around GPT-5 levels of intelligence (which honestly is crazy in its own right).