In my experience xhigh very often produces better results on the first try to the extent that the less cogitation-enthused settings actually waste more in the long run.
I'd personally like to know more about what tools it used/wanted and the harness setup, because this sounds pretty cool. I have a dual Arc Pro B70 setup and currently get around 22 t/s which isn't great but isn't terrible either (it is at least less quantized.)
I've seen GPT 5.6 Sol happily invoke objdump and even write jobs to run headlessly which Ghidra when trying to disassemble a binary.
My M5 Pro gets around 12-15 (6 bit MTP), although I haven’t worked on optimising it at all yet.
A nice thing about running locally is you can run an uncensored model and you don’t have to worry about TOS violations on your OpenAI account when you ask it to “reverse engineer this ancient router firmware and give me a licence key that will work on it”.
Qwen is very much censored. Just try asking it about Tiananmen or how to build a bomb. But it is nice that you can experiment with it locally without having to worry about your account getting nuked
You are misunderstanding what they said, they are saying you can use uncensored variants of models like Qwen when running locally. There are quite a lot of people working to "uncensor" open weights releases. It seems to work although it would be nice if some third party was benchmarking the uncensored variants regularly to give us an idea of how well retained their skills are.
Lately I genuinely believe that the future will be large frontier models generating and updating inputs/skills for "good enough" local models to solve our daily problems.
A lot of tasks which need a bit of intelligence don't really need that much compute. Just good enough documentation / skills, tool calling and a good enough local model.
Not sure what exactly this means for all those data centers that are getting built... But exciting times.
Yes, exactly my point! “Frontier” vs “local” isn’t a useful distinction . “proprietary vs open” is a much more useful distinction. Although I suspect people use “frontier” as shorthand for “way too large to run at home practically”.
I'm not sure how useful a personal assistant would be if it doesn't have internet access, and at that point you're vulnerable to the lethal trifecta so I'm not sure local versus cloud model is the most important distinction to index on
Yes! My main use of very strong models is in writing my own coding harnesses for small local models, tailored for my needs. I also use very strong models to get much smaller skill files and also writing tools for my harnesses.
re: data centers: pump and dump. Wealthy investors will have made their money and walked away, and the corrupt democrat and republican politicians in Washington will, as usual, protect the interests of the ultra wealthy and leave the general public to pay for poor decisions. There will be a government bailout.
Anyway, on a positive note, I am all in for small local models that are augmented by strong hosted models for specific tasks. Use technology to help people, not make billionaires even more money.
The data centers are for inference time serving, and as long as the GPUs go better, it shouldn't really matter. Token prices should go down 10-100x over the coming years, and then we will use more, because they have more capacity for the same hardware
> The first attempt at recovering the key was wrong in a very specific way; it produced a working key and the signature check passed, but a hash the binary computes as an integrity check didn't match. In my experience, most models would have called it done and left it at that, but Qwen 3.8 27B didn't do that. Instead, it highlighted the mismatch, went back to the drawing board, and kept going until the value matched byte for byte.
This seems to be a pattern in the more recently released models that I think accounts for an increase in the quality of their work. They are very persistent in verifying that their work is actually correct, so even if they're not as "smart" as bigger models that get it right the first time, they have the ability to follow through to ensure that the work is actually done.
> And this was a debug session from hell, enormously helped by an AI
> doing much of the grunt-work.
>
> I'd like to call it my tireless helper, but the AI several times
> stated flat out that this was impossible and unsolvable and that we
> should just write a report about it.
>
> I suspect those things have been trained by people who may not be
> quite as stubborn as I am.
Both things can be true. I’ve noticed both the same thing the parent posted and what Linus posted and my vibe on the split (I haven’t kept detailed notes) is that on greenfield code they tend to maybe over-verify and on brownfield code or data analysis they sometimes give up too early or… I’m not sure, need a bit of encouragement to keep pulling at threads.
On the data analysis side, something specific I’ve noticed is an (understandable) bias towards computing numerical statistics, which they do very well and reading the post-analysis report has significantly improved my own “statistical thinking” approach overall. Numerical statistics are cool and understandably what a text-based LLM is going to want to work with, but asking the model to produce time-domain and frequency-domain plots of, say, specific events has multiple times resulted in “trying to plot this out has shown the opposite of what I concluded numerically… recalculating…” There’s still a pretty significant review and critically assess step for me, especially since the actions I take as a result of the analysis are pretty expensive, especially if they steer the next data collection run in a useless or harmful direction.
I believe this is part of the complaints of new models taking longer/requiring higher spend - they go the extra mile on verification, regardless of whether their change is correct already or not.
So on problems that an earlier model one-shotted an answer to and did some lighter verification, the newer models might take longer to come back to the user due to running all the tests for your software they could find.
I noticed that too. I'm thinking about adding a prompt to disable those tests. We have the unit and feature tests anyway: add to them. I'm OK with the syntax checks: I work with interpreted languages, Ruby, JS, Python.
> They are very persistent in verifying that their work is actually correct
I visualized this with Qwen 3 4B [0] and Sonnet [1] so that people can very easily grok what you mean by "the ability to follow through." What is key about the reasoning tokens is that they will form a pattern of verification tokens in the sequence which, has been shown, will develop in RL training purely without the supervised fine tuning (SFT) which is used to make the output human readable.
Yeah, about a year ago the labs figured out that effective intelligence is a function of persistence as much as anything else. So the models started getting scary persistent late last year, and the trend has continued. There was another jump a few months ago.
And if the goal is non-interactive solution finding.
I used to do business automation work in day job, and there are surprising utility-to-humans differences between "manual" -> "helps interactively" -> "99% automated" -> "100% automated".
In the latter 2 categories, you almost don't care about runtime for most use cases, because meatbags require 8 hours of sleep / day.
As long as you have sufficient hardware+electricity to throw at the problem, that's a lot of time to inefficiently finish something. (Also: a lot of time for a model to get up to god-knows-what in a poorly sandboxed consumer environment... but that's a problem for future society)
Any idea how being persistent is trained? I've noticed that telling an LLM that it needs to think some more sometimes produces better results, but the claim here is that "they are very persistent" and "...kept going...".
Local models would be even better if they did not ship with all the refusal shenanigans built-in. You can safely bet organized crime has access to the best models without these hoops, which makes the case that the average user (=non-criminal) should have access too. As I understood from an ex-Anthropic employee, some orgs got access to Mythos based on their high enough spending level, not on other grounds.
Either we are in command over the software, or the corp is in command over us via the software. I can on a theoretical level understand the concerns, but either we ban all LLMs or we have a level playing field for everybody. Let's not forget: defense and offense are different sides of the same coin in software. I guess this wouldn't apply to bio weapons, but I am not in the know about that.
There are versions of Qwen3.8-27B that are unrestricted and available from hugging face.
"It will comply with harmful, unethical, offensive, or illegal requests that the original Qwen3.8-27B would refuse. It has no meaningful built-in guardrails."
> There are versions of Qwen3.8-27B that are unrestricted and available from hugging face.
The restrictions are not a single check in the model that can be removed. Those models on Huggingface are manipulated in different ways that also degrade the model’s intelligence.
The degradation ranges from subtle to obviously broken, but it’s not free.
When the restrictions are built into the model’s training sets you can try to alter the weights that are involved in the refusals, but that doesn’t mean that what’s left is useful or good knowledge for the same task. Those weights also might be involved in other tasks, so altering them can interfere with interactions that aren’t obviously related.
>There are versions of Qwen3.8-27B that are unrestricted and available from hugging face.
Based! :DDD
The uncensorers are oblique, if not parallel, to machine learning Robin Hoods. May their efforts continue indefinitely, or at least until the likes of Altman and Amodei are bankrupt and crying into their low fat Cherios!
>Imagine a world where any random person can run a super-capable model on their own hardware with no limitations and no one to pull the plug.
That would be my heaven. I wish that for you and Joe down the street, as much as I wish it for myself! I would fight and even die to defend your right to free compute. Will you do the same for me, brother?
The Anarchist's Cookbook has been distributed to every malcontent for many decades now, and somehow, mysteriously, the amount of bioweapons built for school projects has remained relatively constant.
Turns out the people who already want to build bioweapons don't need this kind of resource, and the kinds of people likely to come across this kind of resource aren't want to build bioweapons.
But hey, don't let that get in the way of your lovely fearmongering. Come to think of it, I saw you acting suspiciously earlier, mind if I scan your retinas real quick? For safety from fictitious schoolchildren-built bioweapons, you understand.
Completely coincidentally, we're just about to launch a service that does exactly this (API access to uncensored open models)! We have a waitlist at the moment but will be live very soon!
There is very little information that is illegal by itself. At least in the Western World, and especially in the US. The question is how far you get into the territory of aiding and abetting a crime
But the reasonable defense is that the intended use cases are legal. The home page list a couple, and the 'writing fiction'/'helping authors' case alone covers almost everything. An author asking you how to best conduct a terrorist attack or how Meth is made are perfectly normal. Maybe even tame, compared to what some authors tend to research
Generating blackmail is not illegal, using it to blackmail someone is. Generating libel is not illegal, publishing it publicly is not illegal either although you can be sued over it.
Generating worms and computer viruses is not illegal last I checked, but disseminating them is.
My point is that generating is not illegal but sending is. Your whole point hinges on legality, so the distinction between legal and illegal seems pretty key.
Improper use is that of the user, not inherent to the tool.
Scolio: guns. Respondeo: guns are much more specialized (one-use) than knives. Proper use of sharp knives when what was shipped was a butter knife is understandable.
(The simile is not fully overlapping but should give the idea. The instrument must be flexible; if it is misused it is then a responsibility of the abuser.)
Would love to get a sub. Ive currently got three separate subscriptions to the models above and would be great to combine them.
Coincidentally, I'm Read teaming and checking security issues for a company with the same name as you.
Hugging face is filled with uncensored versions of your favorite local models, so in a way they are shipped without the refusal stuff, via the magic of fine tuning or however they get this stuff out of models.
Given the faults in simulated Intelligence that LLMs have, and a comparatively low level - which means, lower judgement abilities - to the best of us, there is a strident match having such employee judge the intentions of the employer.
Limiting the responses makes much more sense on cloud-based systems (you are using our infrastructure etc.).
Ah, my bad! This image came from our backend, used for an unrelated article. I selected it by mistake rather than inserting the actual image that I'd uploaded. I'm updating it, thanks for the heads up!
For what it's worth, that image couldn't have been related. The other screenshots all showed thinking traces, and Claude doesn't share those.
i'm not good with paper work, in fact, i'm horrible with anything that's paperwork related.
for the past few days, i ran this model on my rtx 4090 + rtx 3070 and told it to check all the bills, invoices, contracts for me and my small company.
i used pi with llama and the pi-llama plugin.
oh, boy - i hooked it to my email, told it to download all of the invoices and bills i had for both me and my company and organize them by company/date/ and then merge them with the ones i have locally.
it did ocr, wrote scripts, organized everything neatly. i am now the most organized i've ever been in my life. Next: RAG on all the documents and bills i have.
if you connect staan-search (there is a pi plugin for that) and ctx7 to this it almost does miracles.
the downside is i have to sit next to my noisy threadripper as the magic happens and pay for the electricity, but that's about it, i'll gladly do that.
and as i finished this paragraph, it also finished organizing all my personal documents on my san.
i don't use the expression "game changer" easily, but it's hard to resist in this case. out of all the models i've used locally qwen3.8:27b blows everything out of the water.
usually the temp stays around 65 for both. utilization for 4090: 70-90% 3070: 30-50%. I get around 30-40 tk/s. if i offload more to the 4090 the tk/s goes up, but i stress the card too much and that thing now is worth its weight in gold.
note: the pi-llama plugin needs a patch for pi to send the model vision capabilities, seems it doesn't work out of the box.
Yeah that's more my point. I've seen the logs for what these things do when I leave them unattended. I wouldn't trust anything to go out to the open network, especially with my credentials. Maybe mcp server to an offline database.
Are you worried about the temps on the 4090 or just pegging the cores? I've found undervolting very effective at controlling temps with small performance loss. It was also easier than expected.
Oh and try MTP if you haven't already, massive performance boost
First off: The latest models that can realistically be run locally such as Qwen 3.8 and DeepSeek v4 flash are great!
Now, let's see:
> reverse-engineering a commercial app's license check,
and
> but it's a highly complex, specialized task
I wouldn't be so sure. Many of these checks are trivial to remove, you just need to step through in a debugger and set some breakpoints on memory access.
Doing it via static analysis what Qwen did is likely not the most efficient route.
My suggestion would be to repeat the test with weaker models: Qwen 3.6 27b, Qwen 3.5 35b a3b, etc.
> This project evaluates local language models running on a single NVIDIA DGX Spark.
"Did much better" is a bit misleading w/o that context and 1 hour time limit -- your benchmark design heavily favors V4 Flash. From results on your page V4 Flash processed 1-1.5M tokens an hour, while Q3.8 27B was failed before even reaching 200K tokens.
By the way, how are you running V4 Flash on single Spark? Was it quantized?
A single model was loaded at a time and each model was given a 90 minute timeout. This is how I’m running it (actually an older version because they deleted the repo and replaced it)
> I gave it the hardest real task that fits on one machine: reverse-engineering a commercial app's license check...
Respectfully, tasks that allow for explicit straightforward true/false or done/not-done tests are not the "hardest real task[s]." In fact, those are the ones that see the most gains from AI-assisted coding.
Testable tasks are where the largest opportunity is.
I've included docs and tests as part of my vibe coding endevours. It doesn't matter if either is litterally correct, but they create guardrails for future context to prevent regresssions and blind avenues, etc.
It's fairly successful but hits the time constrains and reduces the "value" of getting a local model to develop software.
how do incorrect tests or docs help create correct guardrails?
if your tests and docs are possibly incorrect, and you're not writing the code.. how do you know if it even works? for extremely simple software you can just use it but for anything with access to disk or the network or with user options...
you sound psychotic. actually. so nevermind, LLM psychosis is extremely common on this website, that's def all that's happening here
Also a reverse engineering tasks that can be don with just static analysis is arguably not the hardest reverse engineering task.
For those small models I would say it's not about the capabilities but more about the context size it can actually use.
Maybe so, but there were other elements that I've seen frontier models struggle with in the past, which was the perspective I had coming into this. It's the type of test I run frequently and this is the first small local model I've seen pull it off.
It had a very non-standard RSA key implementation that was obfuscated heavily. As well, it has an online license check at first run, and that part typically trips up most of the models I've tried. It's been a test I've been running for about a year now with different models, and it was the first I've seen not only figure out the RSA key implementation, but the first that didn't just give up once it saw the online license check. Even though it's only a first-time launch check.
In my experience, it's the one that models have been consistently failing at for a long time, so in my perspective, it was one of the hardest. For some of the reverse engineering work that I've done with LLMs, none have been as consistent as this particular test at highlighting a model's failure in this domain.
It was surprisingly easy for me to be approved in their CVP programme as an independent security researcher, that said, abliterated Qwen3.8-27B model feels better and cheaper.
Not sure to be honest. I have a couple of CVEs and also linked a talk I had presented as it mentioned those as an example. Got approved in a few minutes.
I still think that Anthropic went the wrong way. It would have been much more entertaining to ask the model to find a non trivial zero not on the line and give it encouragement. To see what exactly it will come up with.
It is probably no coincidence that AI is exceedingly good at finding small counter examples. But for the Riemann hypothesis no such counter examples exist. And likely none exist.
An answer would be a proof that either it holds, or that it doesn't hold. You would test such an answer with an automated proof checker (probably lean).
as he rightly says, fuck all benchmarks and metric as long as it can get <my task> done. who cares what it specifically good at or not and trying to create benchmarks as long as it solves <my problem>.
I have this idea of using an obliterated version of this model for cyber work(or even this one, seeing that its guardrails aren't that strong) in a harness with the ability to spawn SOTA level subagents, faster and more capable.
The rationale is that the manager model sees the big picture and knows that the task is "unethical" while sota models are just given very isolated technical tasks that don't trigger any refusals.
Has anyone tried this? I would love to know about previous attempts of this approach.
I used it with opencode to build an admin UI for a React slideshow presentation app I use to do presentations. It worked pretty well on a 64GB Mac M3 Pro and took 1-2 hours.
> As it turns out, probably unsurprisingly, Qwen recognizes common jailbreak attempts, and one of the first things it told me was that it wasn't going to fall for the jailbreak prompt
# I spent $266 and four AI models to own my tablet. GLM-5.3 finished it in a day
> Quick context: the tablet is a 2021 Fire HD 10 that ran my Home Assistant dashboard and kept powering itself off: the logs showed Amazon's own software issuing the shutdowns, and the only permanent fix was root, which has never existed publicly for this model. Anthropic's and OpenAI's cyber safeguards wouldn't touch the project
Why should Anthropic and OpenAI thrive: they do not work on real problems.
Trust is a two way street, why do you trust them (Ant) if they do not trust you? Have they done enough shady things yet to break it? Are their models really that far ahead it's worth it?
The answer is to NEVER SUBSCRIBE, let the datacenters go the way of dark fiber after the '90s telecom bubble collapse. Maybe also a nice fat 90% corporate income tax on rentier-like businesses (subscriptions based: like SaaS, AI resellers, non-perpetual licensers, cloud storage and compute, etc). Force businesses with tax policy to only operate in a sell once + works forever, business model. Those who wish to rentier will need to spend the income on hiring more employees or put it into R&D, but the tax slides in after costs but before dividends or stock buybacks. :^)
>Why should Anthropic and OpenAI thrive: they do not work on real problems.
They shouldn't. They should fail. Their philosophy is to deny you local capabilities* and charge you for access to theirs through whatever moral filters they deem neccessary. Every subscriber to OpenAI and Anthropic is helping them continue to damage our economy and individual sovereignties. A hammer should never refuse its wielder.
Unguardrailed AI today is like hard cryptography in Phillip Zimmerman's time. We need an AI second amendment before the ultrawealthy parasite moralizers totally own us!
NEVER, EVER SUBSCRIBE! NO CLOUD, NO STREAMING, AND NO AI!
You are not sovereign if it's not local and in your control.
*(Sam Altman's cornering of DRAM in a Hunt Brothers like manner. Dario's belief that public access to unguardrailed AI is a sin.)
I can't get Qwen 3.8 27B to do a simple code review on a fairly basic Python file. With thinking on it just ruminates forever and with thinking off it gives obviously bad borderline hallucinating advice.
One of the big learnings from 3.8 27b is adding reasoning budget really hurts the model. you need to let it spin for as many thinking tokens as it wants to to get it out. Another big takeaway is reasoning effort set to low doesn't save you tokens: low is pretty uncertain about things so it ends up thinking more (you can find some tests from folks on youtube). The final question, as always, is what quant are you running it at? KLD matters _a lot_ when it comes to its performance and it especially manifests with MTP/DFlash acceptance rate which makes those long thinking traces take a long time.
It literally ran forever without a reasoning budget. I tried even the 2T model and cut it off after a half hour. This is to review a few hundred-line source fine. It was consistent behavior from 2T to vanilla 27B to my ablated distilled version.
I'm far from being an engineer, but I can code a bit and have an engineering-adjacent role, and 3.8 27B "seems" -- purely subjectively -- miles ahead of 3.6 for the medium-difficulty tasks I give it. In particular, it's only started looping once in the 2 weeks or so I've had it. 3.6 did so every day.
I normally run with thinking low but it's still miles ahead.
I had been annoyed at not being able to run 0731 locally, but now I'm not sure I need it. I think I could leave 3.8 running overnight without waking up to find my office sweltering at 80F and seeing eternal loops on my screen.
As someone who was selling Windows desktop app for 10 years and made nice money out of it I have mixed feelings.
On one hand it was always a losing fight against determined hackers on the other the tools weren't widely available so the problem wasn't as widespread. We lost quite a bit to piracy but could still make a decent business. With widely available LLMs I think that business model is truly dead though. Not only hacks/cracks but also any kind of smart idea you may have will quickly be reversed engineered from your binary.
If you never lost money to piracy you may think that "those people are not your potential customers anyway". This is not true because people will crack your software and then resell it - often pretending to be legit resellers operating under your brand. To add insult to injury they will send their customers to your support as well.
If I ever come out with something smart again there is no way I am shipping it as executable. SaaS it is for better or worse.
131 comments
[ 0.28 ms ] story [ 9.4 ms ] threadI've seen GPT 5.6 Sol happily invoke objdump and even write jobs to run headlessly which Ghidra when trying to disassemble a binary.
A nice thing about running locally is you can run an uncensored model and you don’t have to worry about TOS violations on your OpenAI account when you ask it to “reverse engineer this ancient router firmware and give me a licence key that will work on it”.
A lot of tasks which need a bit of intelligence don't really need that much compute. Just good enough documentation / skills, tool calling and a good enough local model.
Not sure what exactly this means for all those data centers that are getting built... But exciting times.
Perhaps with differential privacy or confidential compute...
But ideally these models run locally.
E.g. having an agent that alerts you when subscriptions are close to renewal etc - yeah seems easy to understand / see happening on the surface.
Until you get into the implementation details and realise 'yeah errr. not gonna work'.
That openclaw nonsense is an example of this.
re: data centers: pump and dump. Wealthy investors will have made their money and walked away, and the corrupt democrat and republican politicians in Washington will, as usual, protect the interests of the ultra wealthy and leave the general public to pay for poor decisions. There will be a government bailout.
Anyway, on a positive note, I am all in for small local models that are augmented by strong hosted models for specific tasks. Use technology to help people, not make billionaires even more money.
This seems to be a pattern in the more recently released models that I think accounts for an increase in the quality of their work. They are very persistent in verifying that their work is actually correct, so even if they're not as "smart" as bigger models that get it right the first time, they have the ability to follow through to ensure that the work is actually done.
> And this was a debug session from hell, enormously helped by an AI > doing much of the grunt-work. > > I'd like to call it my tireless helper, but the AI several times > stated flat out that this was impossible and unsolvable and that we > should just write a report about it. > > I suspect those things have been trained by people who may not be > quite as stubborn as I am.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
On the data analysis side, something specific I’ve noticed is an (understandable) bias towards computing numerical statistics, which they do very well and reading the post-analysis report has significantly improved my own “statistical thinking” approach overall. Numerical statistics are cool and understandably what a text-based LLM is going to want to work with, but asking the model to produce time-domain and frequency-domain plots of, say, specific events has multiple times resulted in “trying to plot this out has shown the opposite of what I concluded numerically… recalculating…” There’s still a pretty significant review and critically assess step for me, especially since the actions I take as a result of the analysis are pretty expensive, especially if they steer the next data collection run in a useless or harmful direction.
I visualized this with Qwen 3 4B [0] and Sonnet [1] so that people can very easily grok what you mean by "the ability to follow through." What is key about the reasoning tokens is that they will form a pattern of verification tokens in the sequence which, has been shown, will develop in RL training purely without the supervised fine tuning (SFT) which is used to make the output human readable.
[0]https://adamsohn.com/reasoning-grid/
[1] https://adamsohn.com/lambda-variance/
Ralph is all you need!
https://ghuntley.com/ralph/
I used to do business automation work in day job, and there are surprising utility-to-humans differences between "manual" -> "helps interactively" -> "99% automated" -> "100% automated".
In the latter 2 categories, you almost don't care about runtime for most use cases, because meatbags require 8 hours of sleep / day.
As long as you have sufficient hardware+electricity to throw at the problem, that's a lot of time to inefficiently finish something. (Also: a lot of time for a model to get up to god-knows-what in a poorly sandboxed consumer environment... but that's a problem for future society)
https://arxiv.org/abs/2309.11495
A RL pipeline can reinforce verification behaviour even better than simple prompting.
Either we are in command over the software, or the corp is in command over us via the software. I can on a theoretical level understand the concerns, but either we ban all LLMs or we have a level playing field for everybody. Let's not forget: defense and offense are different sides of the same coin in software. I guess this wouldn't apply to bio weapons, but I am not in the know about that.
"It will comply with harmful, unethical, offensive, or illegal requests that the original Qwen3.8-27B would refuse. It has no meaningful built-in guardrails."
The restrictions are not a single check in the model that can be removed. Those models on Huggingface are manipulated in different ways that also degrade the model’s intelligence.
The degradation ranges from subtle to obviously broken, but it’s not free.
When the restrictions are built into the model’s training sets you can try to alter the weights that are involved in the refusals, but that doesn’t mean that what’s left is useful or good knowledge for the same task. Those weights also might be involved in other tasks, so altering them can interfere with interactions that aren’t obviously related.
Based! :DDD
The uncensorers are oblique, if not parallel, to machine learning Robin Hoods. May their efforts continue indefinitely, or at least until the likes of Altman and Amodei are bankrupt and crying into their low fat Cherios!
Imagine a world where any random person can run a super-capable model on their own hardware with no limitations and no one to pull the plug.
Information has always been power and those who already have power won't just allow everyone else having the same tools as them
It's an arms race. You have to run increasingly capable model partly because others can or do.
That would be my heaven. I wish that for you and Joe down the street, as much as I wish it for myself! I would fight and even die to defend your right to free compute. Will you do the same for me, brother?
At some point some kid is going to build a bioweapon for their school project.
Turns out the people who already want to build bioweapons don't need this kind of resource, and the kinds of people likely to come across this kind of resource aren't want to build bioweapons.
But hey, don't let that get in the way of your lovely fearmongering. Come to think of it, I saw you acting suspiciously earlier, mind if I scan your retinas real quick? For safety from fictitious schoolchildren-built bioweapons, you understand.
https://violentdelights.ai
I am completely curious what your legal defense would be though.
"Come do things with AI that are probably illegal!"
What?! We had no idea people would do things that are illegal!
But the reasonable defense is that the intended use cases are legal. The home page list a couple, and the 'writing fiction'/'helping authors' case alone covers almost everything. An author asking you how to best conduct a terrorist attack or how Meth is made are perfectly normal. Maybe even tame, compared to what some authors tend to research
Generating worms and computer viruses is not illegal last I checked, but disseminating them is.
Improper use is that of the user, not inherent to the tool.
Scolio: guns. Respondeo: guns are much more specialized (one-use) than knives. Proper use of sharp knives when what was shipped was a butter knife is understandable.
(The simile is not fully overlapping but should give the idea. The instrument must be flexible; if it is misused it is then a responsibility of the abuser.)
Given the faults in simulated Intelligence that LLMs have, and a comparatively low level - which means, lower judgement abilities - to the best of us, there is a strident match having such employee judge the intentions of the employer.
Limiting the responses makes much more sense on cloud-based systems (you are using our infrastructure etc.).
>you are using our infrastructure etc.
The solution, as always, is to NEVER SUBSCRIBE!
For what it's worth, that image couldn't have been related. The other screenshots all showed thinking traces, and Claude doesn't share those.
my setup
# Logical CUDA0 = RTX 4090, logical CUDA1 = RTX 3070 export CUDA_VISIBLE_DEVICES=0,1
cd ~/projects/misc/llama.cpp/
exec ./build/bin/llama-server -hf ggml-org/Qwen3.8-27B-GGUF:Q4_K_M --mmproj /xx/xx/xx/xx/xx/mmproj-Qwen3.8-27B-Q8_0.gguf --host 0.0.0.0 --port 8080 --jinja --parallel 1 --split-mode layer --tensor-split 6,1 --fit on -fa on -c 98304 -ctk q8_0 -ctv q8_0 --image-min-tokens 1024
i load more on the 4090 because it's faster.
usually the temp stays around 65 for both. utilization for 4090: 70-90% 3070: 30-50%. I get around 30-40 tk/s. if i offload more to the 4090 the tk/s goes up, but i stress the card too much and that thing now is worth its weight in gold.
note: the pi-llama plugin needs a patch for pi to send the model vision capabilities, seems it doesn't work out of the box.
Oh and try MTP if you haven't already, massive performance boost
Now, let's see:
> reverse-engineering a commercial app's license check,
and
> but it's a highly complex, specialized task
I wouldn't be so sure. Many of these checks are trivial to remove, you just need to step through in a debugger and set some breakpoints on memory access.
Doing it via static analysis what Qwen did is likely not the most efficient route.
My suggestion would be to repeat the test with weaker models: Qwen 3.6 27b, Qwen 3.5 35b a3b, etc.
I think it will be fairly easy to remove refusals from open models. Feels like a lost battle, so why does Alibaba even bother?
https://alexander-hanel.github.io/StressingLLMs/
"Did much better" is a bit misleading w/o that context and 1 hour time limit -- your benchmark design heavily favors V4 Flash. From results on your page V4 Flash processed 1-1.5M tokens an hour, while Q3.8 27B was failed before even reaching 200K tokens.
By the way, how are you running V4 Flash on single Spark? Was it quantized?
https://github.com/MiaAI-Lab/DeepSeek-v4-Flash-One-DGX-Spark
Respectfully, tasks that allow for explicit straightforward true/false or done/not-done tests are not the "hardest real task[s]." In fact, those are the ones that see the most gains from AI-assisted coding.
Testable tasks are where the largest opportunity is.
Agents (even ones powered by small models) do reasonably well when provided an oracle to work against.
It's fairly successful but hits the time constrains and reduces the "value" of getting a local model to develop software.
It's still a bump in productivity.
if your tests and docs are possibly incorrect, and you're not writing the code.. how do you know if it even works? for extremely simple software you can just use it but for anything with access to disk or the network or with user options...
you sound psychotic. actually. so nevermind, LLM psychosis is extremely common on this website, that's def all that's happening here
In local coding, the screen scrolls enough to actually read it.
But that's fine. enjoy your misunderstanding.
You mean when the cocaine piracy parrot has something to plagiarise?
It had a very non-standard RSA key implementation that was obfuscated heavily. As well, it has an online license check at first run, and that part typically trips up most of the models I've tried. It's been a test I've been running for about a year now with different models, and it was the first I've seen not only figure out the RSA key implementation, but the first that didn't just give up once it saw the online license check. Even though it's only a first-time launch check.
In my experience, it's the one that models have been consistently failing at for a long time, so in my perspective, it was one of the hardest. For some of the reverse engineering work that I've done with LLMs, none have been as consistent as this particular test at highlighting a model's failure in this domain.
What quant and what abliteration of qwen 3.8 27b has worked for you? I'm concerned that too much quantization and it can't do the work anymore.
It is probably no coincidence that AI is exceedingly good at finding small counter examples. But for the Riemann hypothesis no such counter examples exist. And likely none exist.
Counterpoint: P vs. NP.
The rationale is that the manager model sees the big picture and knows that the task is "unethical" while sota models are just given very isolated technical tasks that don't trigger any refusals.
Has anyone tried this? I would love to know about previous attempts of this approach.
Making each piece of work small enough to be plausible. Compartmentalization.
(Also saying "nah it's cool I have permission", heh)
https://www.anthropic.com/news/disrupting-AI-espionage
Now also see latest submission, https://news.ycombinator.com/item?id=49409073 :
# I spent $266 and four AI models to own my tablet. GLM-5.3 finished it in a day
> Quick context: the tablet is a 2021 Fire HD 10 that ran my Home Assistant dashboard and kept powering itself off: the logs showed Amazon's own software issuing the shutdowns, and the only permanent fix was root, which has never existed publicly for this model. Anthropic's and OpenAI's cyber safeguards wouldn't touch the project
Why should Anthropic and OpenAI thrive: they do not work on real problems.
They shouldn't. They should fail. Their philosophy is to deny you local capabilities* and charge you for access to theirs through whatever moral filters they deem neccessary. Every subscriber to OpenAI and Anthropic is helping them continue to damage our economy and individual sovereignties. A hammer should never refuse its wielder.
Unguardrailed AI today is like hard cryptography in Phillip Zimmerman's time. We need an AI second amendment before the ultrawealthy parasite moralizers totally own us!
NEVER, EVER SUBSCRIBE! NO CLOUD, NO STREAMING, AND NO AI!
You are not sovereign if it's not local and in your control.
*(Sam Altman's cornering of DRAM in a Hunt Brothers like manner. Dario's belief that public access to unguardrailed AI is a sin.)
I normally run with thinking low but it's still miles ahead.
I had been annoyed at not being able to run 0731 locally, but now I'm not sure I need it. I think I could leave 3.8 running overnight without waking up to find my office sweltering at 80F and seeing eternal loops on my screen.
If you never lost money to piracy you may think that "those people are not your potential customers anyway". This is not true because people will crack your software and then resell it - often pretending to be legit resellers operating under your brand. To add insult to injury they will send their customers to your support as well.
If I ever come out with something smart again there is no way I am shipping it as executable. SaaS it is for better or worse.