Perhaps DeepSeek Search will be a strong competitor for google, from the link: DeepSeek Search is an AI-native, general-purpose search engine that indexes hundreds of billions of web pages. It supports multilingual search and returns highly relevant, authoritative, and up-to-date results in real time.
I may be missing something so apologies if I'm telling you something you already know but: there are plenty of web search APIs for agents - the Brave one is the one I use: https://brave.com/search/api/
I already know the Brave Search API, but thanks for telling me. The issue is that it's expensive. It's 5 USD per 1000 requests, while DeepSeek's is 0.75 USD per 1000 requests.
I gave it a kick and got mixed results. For example, searching for a Microsoft product couldn't find its landing page, and searching for a niche open source package landed in the Debian repos for the top hits, but the home page was in the first 10 hits.
Lovely to see they're doing their own search though, I love how many folk are actively working on search clones at the moment.
I haven't really tested it. But they might be doing keyword search like ceramic.ai instead of semantic search, so that's why the results might be worse, but I'm not sure.
I really like the having DeepSeek Flash + Opus 5.5 as advisor/reviewer at the moment. I run through a $200 Claude Code subscription in less than 24 hours now just for a single project, and I hate fiddling with different accounts every day. Now that I get a 200 USD api credits inlcuded with the subscriptions I have rewritten my coding agent workflow completely.
I swear I have the same question in my head always when I read such statements! I have never once reached 90% that includes everyday usage across multiple repos, artefact generation, device debugging, long running harness work!
I am not saying OP is not telling the truth. I just really want to understand!
I decided to try something a little weird research wise the other day analyzing about a thousand older research papers and burned through my $100 chatgpt sub + 3 resets in three hours and only finished half of them. It's the first time I've done that, though.
It’s because they don’t know what they’re doing. It’s like when openAI was using 10,000 agent clusters to solve problems… then realised the model could do the same work on its own and solve the same problem.
Anyone who tells you they have a “team of subagents” and “code reviewer” agents and “project manager” agents doesn’t know what they are doing. That’s how they burn thousands of dollars in a day.
Do you have a source for OpenAI discovering they can solve Navier Stokes with a single agent instead of a 10k swarm?
I think I’ll trust the experience of the people in the labs who have unlimited token budgets and are running experiments continuously over some anonymous rando slinging mud on the internet.
Also my own experience, which says having code reviewer agents is one billion percent useful. That comment alone completely disqualifies you as a reliable source of info.
As a dev who hasn't used LLMs as an embedded part of their work, curious:
- How long do you reckon it would take a human to do the task?
- What percentage of features/ feature-extensions you add would have been done at all if you didn't have agentic assistance? Is there a tension between llm productivity gains and 'because we can' feature additions?
- Are many of the tasks for throwaway v1 features, exploratory tries, or redos of the same?
- How speced out is the work you give it, eg did this task have pages of specs, how long did the spec take to write?
- How deeply will you manually review the work and how much time will it take? Will you read all the code / review at a high level and interrogate with llm / have the llm review and self iterate?
For what I am currently doing, years, if not a decade or maybe even two. Claude has already been running for 3 months constantly working on this project. I've been trying to do this several times before, but the scope outgrew my available time/life, too big for one person. But now I can actually build a production release by just setting the direction and let Claude to all the wiring, plumbing, tests etc.
I do now add all the major features I want, and write a task list where I add a few more minor tasks each week that just come to my mind. I need to write them down before I forget. I often ask Claude to implement several different implementations/algorithms and compare them. There are plenty of trade-offs, which usually end up in building configuration profiles, so the application can work optimally in multiple environments.
The specification is actually really large, and has grown over time as I've been learning and new ideas has come to my mind. I do not review/audit the code, it's massive would take me months, no chance! I do however use code inspection/audit tools like TSAN, ASAN and more. I will also spend time on full code reviews with the coming Fable 5.5 when a release is about ready. I do however review behavior together with Claude, and this is also where the specification grow, because this and that should be possible, but not this and that. User stories also helps.
Keep doing what you're doing, this guy clearly is just revving his engine because he likes the way it sounds and can afford the gasoline. I wouldnt be asking people like this for advise, they're completely off the rails.
There's no reason to blow through tokens this way, there's a 99% chance nobody uses the output (except maybe themselves).
I have some fairly large personal software projects and I still don't understand what useful work people are getting out of burning all these tokens on unattended agents.
What exactly are they doing for you that's giving you $200 of value in such a short timeframe?
Not OP, but I used to run out in a few days, for me it was a matter of suboptimal code review practices with many subagents looking at different aspects, often expanded to look at more than the diff.
I can drain a $200 OAI plan in 2 hours flat working on my 10 projects simultaneously.
Maybe you're not being ambitious enough with these things? They are at their most fun when you spec out sprawling architecture documents and watch 1+8 astras buzzsaw through it.
I'm only at 260B for the year, though. I'm sure someone who has used more will be along shortly to chime in
E: The other subthread talking about tests is correct. Expensive models should not waste a single second waiting on tests or validation. If you cannot assign a single cheap model to be the test bitch, then just tell them to frontload everything and clean up later as a single wave.
Rewrite this 250kloc native codebase in 5 different languages on 5 platforms (Windows, Linux, MacOS, iOS, Android) using these 10 different design methodologies and architectures. Use 32 subagents. Perform extensive before/after benchmarking and report the results here.
That should keep 'em busy for a few weeks worth of tokens! I feel so productive!
Interesting, coming from a similar situation, I'm interested what workflow you landed on. Opus planning + available to be called for deep thinking + opus reviwer? or?
By not using existing tools and one shooting everything through the API - I.e. instead of decompile via Ghidra they will try to get source via capstone genersted assembly.
It is like putting a jet engine on a shipping container and wondering why it needs to be constantly kept in afterburner to at least scrape on the road while smarter users put wheels on the shipping container.
Yeah the Claude/anthropic usage is quite arbitrary, I also run through my entire 20x pro plan in 1-2 days. Codex has slightly better usage levels, but is annoying to use for many reasons. DeepSeek flash (direct or via opencode go) is the best value model I've found so far.
I'm baffled, my $20 is enough for me to do some consulting, personal projects and side work, and still cannot outran my quotas, I'm building different kind of things: flutter apps, dotnet backend, database, front end, architecture diagrams, code exploration, tests, PowerShell scripts.
How people spend more than 200 usd monthly is beyond my understanding, are you guys so inefficient with the usage? or I am the opposite?
Without any context for what you're working on, I'd have to assume the reason you don't hit the $20 subscription limit is that you're either not using it very often, or you're working on some relatively boring CRUD app that doesn't require it to think very deeply.
> I run through a $200 Claude Code subscription in less than 24 hours now just for a single project
I would be worried about your methodology when the subscription party ends. Make no mistake, one day, the party is ending and the people who are dining out on approaches that burn massive tokens are in for a rude shock.
This and (the not quite yet open) Kimi K2.8 are my main coding engines, and I find them incredibly cost efficient. I find Deepseek in particular is really good with Svelte work.
Right now I use each of them in their own respective coding harnesses, deepseek-harness and kimi-code, but I do wonder what they are like inside something like opencode.
Both the free (seems limited to ~200k context) and metered variants in opencodego show the release date as 1788998400000 → 2026-09-10, which implies they're not the newer checkpoint
At the beginning of the month I realized I had more tokens and more capabilities on Opencode at 8$/month via DS 4.1 flash, than I had on a 200$ account in June with the american providers.
Literally both more tokens and capabilities than Opus 4.8/GPT 5.5.
Even if you pay per token, you're still unlikely to outspend american providers.
What does it matter if you’re not hitting your cap with OpenAI/Anthropic in the first place? This thread is apparently populated with the highest of powerusers running multiple OpenClaws. Us peasants aren’t concerned with meters or cost/token - just quality - and nobody comes close to OpenAI or Anthropic in that regard.
Pretty funny to say "us peasants aren't concerned with ... cost/token", as if most people ("peasants") won't find it absolutely insane to be paying $200/mo for a subscription.
Large corporations pay API rates: Claude teams for example can buy subscriptions for its members, until they reach 150 of them after which they need to migrate to org.
They are showing this new version scoring significantly higher across multiple coding benchmarks, beating Sol 5.6 (6.1 is the latest) across 10/11 benchmarks.
DS isn't better than Sol and Gemini in the benchmarks, but after working with them all for a while now I'm beginning to understand why people find DS _feels_ so much better.
First, it's the tok/s and TTFT from providers. Deepseek blazes through work - at quite good quality - and you don't find yourself waiting for your agents to finish.
Second is the openness. Being able to see the CoT and not being randomly blocked for cyber review. You're never wondering if it's stuck or if it will randomly fail. Combined with the great cost value.. My next subscription will probably be Z.ai
Feels better to me in the sense that it doesn't try to be too clever. When I ask it to do things it actually generates code that a developer would write. That still lets me build software in layers like I do mentally. And it keeps you in control of where abstractions are.
Some people would say that you don't need this at all with the frontier llms. But when extending existing software or writing software that fits into an existing ecosystem, I find I still need to offer a lot of guidance. And it's better that the context window isn't flooded with code that was reverted after I looked at it.
> Some people would say that you don't need this at all with the frontier LLMs
Hit the nail on the head. Development methodologies pivoting at the whim of frontier models have been one of the things that have been worrying me as of late.
it doesn't worry me too much tbh. This is really just cutting corners and going this route leaves you with some kind of additive ship of theseus piece of software you don't understand and require your frontier model to work with, which just... becomes more additive and gross.
right now we're in some proverbial heyday of everyone seeing how absolutely grand of a thing they can make with frontier models.
over time i expect some old design philosophies to take hold with people who use LLM's successfully and find a place again paired with developers who understand why such layered methodologies are important... because this whole AI thing wasn't the silver bullet CEO's and crazy execs thought it was. They will eventually get sick of the support burden of badly designed stuff. i anticipate a bit of pendulum swinging back towards "still using AI, but we need smart people to do it because it's expensive and we want it done right". Right now a lot of companies are being extremely reckless imo and they haven't fully realized/felt the cost implications of trainwreck software yet.
In a way, that’s the situation today. You can pay an experienced developer $$$$ to do it right, or you can outsource it and pay $$ but pay extra for more layers of review, have lots of unfixed bugs and more tech debt.
You are proposing that instead we pay the frontier model to clean it up. If it can, I expect it to cost somewhere in the neighborhood of an experienced dev because that’s the alternative. That’s the way the frontier AI companies will extracting the value of the work from you.
> I expect it to cost somewhere in the neighborhood of an experienced dev because that’s the alternative.
That's only true if a single company owns the frontier, otherwise the alternative is "a rival frontier model" and the cost will be the marginal price of the compute + amortized training costs.
Asking the LLM to 'just' fix 'it', or whatever, is going to get you lost in the woods.
You're car will have spun off the road and you'll find it harder and harder to control it and keep it on the road because you have less and less visibility into the vehicle, nor will you be able to clearly see out your windscreen to use the car and road metaphor. You'll be driving blind eventually, unless your working on some simple closed toy problems and/or other small systems.
Building large systems means understanding the system. Without understanding the system, you lose control of it.
The LLM can probably help you understand the system to gain/regain some control of it again, but now you're becoming a system specialist rather than someone who's repeating, "fix it" and "make it good".
There are a whooooole lot of people writing software who, despite spending years on the craft, never really get any further than McGuyver mode.
Self-trained people (like me) are particularly susceptible to it and we're heavily represented in the software world. A lot of us either recognise that (or are coached) and invest the time learning the benefits, and limitations of, software architecture. And, importantly, keep reflecting, learning and updating our practise.
I keep going back to the YOLO mode every other year and try out new languages to see if we still need the structure and abstractions. Then VMs came along and all I can say is they're both even more important and now we have an army of virtual "ADHD idiot juniors developers" at our finger tips it's even easier to tell the good and bad abstractions from each other.
I can not rely on Opus 5.5+ to "fix" whatever crops up. Should it find a security related bug one too many times, it gets the Anthropic Shock Collar [cyber]! and I have to pick Opus 4.8 or older to continue and they try to up-sell Mythos which I am not permitted to use without going through their verification program that they will not offer to me. Thankfully for most of my hobby tasks (thats all I do now) Opus 4.8 is usually mostly good enough. In my opinion their newer models are just not something I can rely on.
IMO the fundamental principles and "conflicts" are still the same, but many have forgotten about them with all the hype and sense of speed. The industry will slowly rediscover them.
Buggy slop products will burn buyers and then people will buy less. The very-early-80s video game companies shoveled out slop, including a few games that wouldn't even launch. Which led to the Great Video Game Crash of 1983.
> haven't fully realized/felt the cost implications of trainwreck software yet
As long as the API is private or not dependent on by automated systems and you have good test coverage, you can get away with bad code IMO since you can always refactor. If it looks stupid but it works (and can be refactored/fixed/improved later if needed) it aint stupid.
I just use a suite of tests to enforce the mechanical standards (dry, encapsulation, etc) and arch documents for shape + reviews of new work against the two. I’m not seeing any significant drift.
Yea, this is one thing that gets me. Vibe coding and freestyle prompting gets wildly different results. Starting from a set of specs and expectations routinely gets me reviewable and we'll tested code. Sure, running the full test suite alot slows things down and uses more tokens, but when I hit a five hour or weekly window, I just roll up my sleeves and start working without the model because the code is layed out the way I expected and is designed to fit my meatbag workflow and pace.
When is the last time you used a frontier model? Any chance you work someplace that forces you to use last year's model (or any American open weight model)?
I use them every day, in systems programming. They produce a solution, but it’s often several orders of magnitude away from an optimal one in various dimensions. That’s still quite useful, but for some problems that doesn’t cut it.
To reach a more optimal solution, significant human steering is reqiured for every tiny decision. It’s the same as how early models needed hand holding on the basics, playing out again in the finer details. Human experts need to inject “eureka moments” from the outside.
And no feedback loop that I’ve tried has been able to conjure an understanding of these finer details that, as far as I can tell, just aren’t represented in the weights.
Modeling the basics of software engineering was easy, since things like writing code that passes some tests is easy to RL. But for anything not easily RL-able, we’ve basically ran out of human generated tokens to distill from.
Without some breakthrough in generalization, I remain unconvinced that we’ll be able to model the finer details of software engineering by looping the existing training corpus back into itself. Garbage in, garbage out.
Interesting. I’ve been using GLM 5.3 Flash for a couple months and found it to be great. What has it been doing that you found to be “too clever”? It sometimes gets stuck in loops. I once asked it to do a code review of a simple pattern and it went off on a 45 odyssey trying to analyze too much. But other than those, it’s been a pretty good daily workhorse. And you can’t beat the price. Even Deepseek 4.1 Flash is more expensive. And I switched to GLM 5.3 Flash from DS 4 Flash, my previous daily driver, which I also found quite good. Anyway, my point isn’t to start a “which model is better” subthread, but was just curious what behavior you’ve seen where it’s “too clever.”
Anecdotal but I was doing regular adversarial reviews of new work using about 5 models (mostly the open Chinese ones like DS, GLM, Qwen, Kimi, etc) and GLM routinely took much longer for similar results and the same instructions.
It would just loop on things and go down rabbit holes more. DS flash has been so consistently good I've kinda settled on it for 90% of my work.
I still use adversarial review for bigger changes where I want to make sure any issues are caught early, but I genuinely feel as tho if no new models ever get made I'd be fine with just using DS forever.
What quant were you using? I’ve been using Q4_K_S for a while now and it’s been rock solid, but having tried Q3_XS, that would loop like crazy. Maybe try a bigger quantity size?
It wastes my time. A lot! In different ways. That's the tl;dr. Really.
Keeps repeating things I settled as accepted or rejected. But what really gets on my nerves? The language it uses – that aweful, useless typical LLM languages! No AGENTS.md instruction, no per session instruction, no just two-prompts-prior instruction seems to work, hell sometimes instruction regarding that in same prompt doesn't work. GOOD/BAD example along with instruction don't work. That wastes time and zones you out! Not to mention the fatigue of to and fro. DS has been working much better. Hell Gemini 3.1 Pro works so much better than GLM. I don't have extended experience with OpenAI and Anthromic models but the free claide.ai works quite well and better than GLM 5.3 (not even flash). Albeit I must say I rarely use claude.ai for coding, but technical tasks? All the time!
GLM's best response: "No excuse. My fault. You exactly asked me not to do exactly that and so did the AGENTS.md. I failed". Love it. Except that if I press I will get that response every 5 prompts.
In my experience GLM and Claude models behave in ways that steadily drift from your directives and speak in similarly mannered "Claudese", leading me to believe that glm was more or less trained to mimic Claude. I think this reflects a very strong gravitational pull anchored at the system prompt level to specific voice and behavior directives directives. I find other models like DS and Gemini are more compliant and drift much less from my custom behavior and voice directives.
2 of the 3 responses to your comment are [dead], and it's not clear to me why.
Fwiw, to elaborate on mine, a small fast 8B-ish model should be able to few-shot simple text style transfer, such as converting one model's complex careful style to another model's punchy compact style. Perhaps also a 2B fine tune. So you might tweak your harness to transfer and present a preferred style.
I briefly tried GLM 5.3 Flash and the first thing I noticed was how slow it was followed quickly by its single-mindedness. With DS 4.1 Flash I find that it understands that not all the detail in my prompt is important. Sometimes I like to give a quick example of what I'm looking for and it can take the hint whereas GLM will seemingly reshape its worldview around this tiny detail.
DS just handles ambiguity and my bursts of detail with much more grace. I'm only using "max effort" and the CoT is reasonable compared to GLM.
> Feels better to me in the sense that it doesn't try to be too clever. When I ask it to do things it actually generates code that a developer would write. That still lets me build software in layers like I do mentally. And it keeps you in control of where abstractions are.
yes!So it's not just me. In terms of how intelligent they want to be GPT is at the top then Claude. And actually when I was playing with Sol it meant that many tasks it would just try to do something else because it thought I don't know what I want. Even trying to nudge it back in the right direction wouldn't work.
I guess the models just have the cultures of their creatures.
LLMs don't seem to be really that good at managing complex and conflicting needs, especially if it grows in increments, a ticket here and there. (To be fair people often suck at it too.)
I have a theory that mess in an LLM-generated codebase sort of grows exponentially. Each hack or sloppy abstraction chips away at the coherence of the project as a whole and makes the next feature harder to do. But perhaps more importantly, it demonstrates to the LLM, which is a probabilistic machine despite its impressive capabilities, that quality of the code is low, and it generates more low-quality hacks.
And eventually you reach knots upon knots that you have to untangle yourself because the LLM becomes too confused by the code it wrote itself. As models get better, the problem seems to be getting better. I'm needing to do this more and more rarely, and the tradeoff is still clearly worth it.
But I've seen Fable 5.1 absolutely lobotomized by someone who fills each chat context with half a million tokens of insipid garbage, and I don't see why this couldn't happen when coding as well. I'm not sure a solution to this general problem that LLMs can simulate idiots really well will be fixed any time soon.
Be the product manager. Just like real iterative development, you have to take time occasionally to reduce technical debt explicitly.
Adding unit tests, increasing modularity, reducing duplication, and the like are all things you can tell the LLM to do explicitly. And it is happy to do it.
And you can have coding standards as part of your design criteria. Set and document these early in the project and audit the code at least enough to see they are being enforced.
> Adding unit tests, increasing modularity, reducing duplication
Agents know these principles already and they are frequently the problem themselves.
Too strongly-insisting on eg DRY can lead to tortured abstractions shoehorning too much together. Agents need to be able to distinguish between incidentally similar code vs conceptually redundant code.
Excessive modularity, eg. militant single-responsibility principle can lead to abstraction hell that is impossible to reason about at once.
Unit tests are obviously good. Coding agents write a copious amounts of them, often baking you into poorly-chosen designs made without your knowledge.
I wish benchmarks tracked how often you have to undo changes the model made on its own. Sometimes the less ambitious model is just easier to work with.
This is it for me, it feels like someone developing alongside me, rather than something like Claude which is constantly fighting to degrade code to a point where it and only it knows what the fuck is going on anymore, which would be defensible if it was at least good code but it isn't.
Huh, doesn't DeepSeek take the longest because it overthinks way too much? Haven't tested this release yet but was very disappointed with past ones because it takes an eternity to do anything because it constantly overthinks and questions itself way too much instead of just starting.
It thinks a bit more, but if the cache reads are free or almost free, and it blasts through 10x faster than GPT Sol/Terra/Luna, it doesn't really matter. Just don't use the max thinking for implementation, go with high or low.
I support people making choices with full knowledge of what they are doing and what the consequences are, as much as it is possible. What I oppose is making choices without such knowledge.
Assuming canadian or dane, but maybe there are other places he’s threatened. There are canadian inference providers running deepseek, and definitely eu ones idk about ones based in denmark.
The economic sovereignty of any nation, but especially smaller ones like canada and denmark, is harmed by sending our money out of the country when there are options available in country.
> The economic sovereignty of any nation, but especially smaller ones like canada and denmark, is harmed by sending our money out of the country when there are options available in country.
I respectfully disagree. I subscribe to the comparative advantage economic principle, so I'm fine with international trade.
What I am saying isn’t inconsistent with comparative advantage. International trade is fine, but only when it’s not a commodity easily obtained inside your own nation. Open model inference is a commodity. There is no real lost opportunity cost from using canadian owned GPUs or chinese owned ones. In fact, as a country with abundant cheap power, and physical space, canada’s probably a better place than many to run inference. That’s what I meant by “when there are options in country”.
I hear you, but as I said, I hope that buying from Deepseek gets more Deepseek. Maybe they get the same amount (or more) from a Canadian provider, and if so I might reconsider.
The 'clean power' would factor in a well, but I haven't really looked in to it. I know China is installing a ton of renewable and has a carbon market while Canada's building a new pipeline and building Alberta data centres running on natural gas, so I'm not willing to take on faith that switching is cleaner.
Fair point. Personally i don’t see value in closed models, be they chinese or american. So, it’s pretty much against my religion to pay the model creator to provide inference. Even a situation where model makers license models to providers seems better than deepseek hosted deepseek provided models.
Also, re: power. I said canada has lots of cheap power, clean was your word.
RE: clean power - China is by far the biggest leader in getting renewables off the ground, but they are also still comfortably the biggest polluter, so it's difficult to compare.
The unfortunate truth behind at least the solar PV boom in China is that it's driven in large part by the exploitation of Uyghurs in the Xinjiang region. One of the reasons they can sustain the pace they're going at is because while nowadays PV modules exports increasingly gravitate towards offshore fabs (Vietnam, Malaysia), domestically all the biggest PV players make full use of modern slavery within Xinjiang's re-education camps.
>International trade is fine, but only when it’s not a commodity easily obtained inside your own nation
It might be a minor nitpick, but that part is just not how the macroeconomic principle works. Abundance of a good/service within a country is irrelevant to the concept of a comparative advantage - this is counter-intuitive but but that was David Ricardo's entire point. It's not about whether production is more efficient domestically, it's whether comparative allocation of resources between multiple countries is more efficient by specialization within the different goods.
Even if you can produce "good 1" cheaper in Country A vs Country B, if Country A can instead produce "good 2" that Country B cannot, it's better for Country A to produce "good 2" domestically and import "good 1".
Now, I still agree with the original notion. It's unnecessarily risky to use services from "democratically-challenged" nations (especially imperialist ones), when safer options are available. But there's no need to twist macro 101 concepts to justify that view. Modern trade policy takes more into account than basic market efficiency.
China has annexed (or it claims so) several countries already, so it sounds like you're only concerned with yourself - probably not what you wanted to achieve.
(But I'd concede that in the field of genocide China doesn't come up as bad)
To add to this, Fireworks (and many competitors, e.g. enumerated here[0]) offer zero-data retention inference, to anyone. DeepSeek cloud does not. It's part of their terms that anything you send them is fair game for them to train models on.
It’s pretty clear despite what they want you to think Anthropic and OpenAI train on everyone’s data who has a subsidized plan. Only way you get ZDR is by paying API prices. Also note that it is zero retention, not zero training.
> "Also note that it is zero retention, not zero training"
The "green" badges explicitly say the opposite:
> "Private: this provider does not train on prompts and does not retain prompt data."
I.e. for the example of Fireworks, this is the precise language they use to describe ZDR:
> "More technically: prompt and generation data exist only in volatile memory for the duration of the request. If prompt caching is active, some prompt data (and associated KV caches) can be stored in volatile memory for several minutes. In either case, prompt and generation data are not logged into any persistent storage."
Spoiler, it's not clear by any means. There's no evidence to back up the claim, it's just their feelings... which is fine, but I definitely wouldn't call it clear.
I'd rather my data go to China and directly to labs that release open weight models (DeepSeek is MIT licensed still); instead of closed labs and the NSA/CIA; who has far better ability to hurt me than China does.
Without getting too much into politics; over the past several decades, one country is going around starting wars, regime changes and other interventionism, or initiating economic warfare (Huawei sanctions because they were getting too competitive to US companies; GPU restrictions; tariffs; etc), and that's not one administration: it's US policy over the past few years.
The reason that US and its allies/friendlies always score high on these kinds of ratings while their enemies/adversaries always score low is because these institutions and 'experts' are directly funded by and beholden to these countries/interests.
Just look at Russia - it's score is highest in the 90s - the most corrupt and destitute time in its modern history, also happens to be when it was deemed somewhat friendly to US and friends.
This number reports, basically, how good it is to live inside a country, while the comment that you are replying to is entirely a list of foreign policy complaints (regime changes, etc).
It's kinda like meeting someone on the street who's protesting about animal cruelty in slaughterhouses, slyly handing them a chart showing them the nutrition facts of beef, and then walking off without another word as though you've settled your case.
Like, I get the point you're trying to make, but I hope you're aware it's a non-sequitur in context?
> Like, I get the point you're trying to make, but I hope you're aware it's a non-sequitur in context?
Your hope should have been that amelius was not aware that it's a non-sequitur, because that is correctable. But then they kept posting and removed all doubt.
have a look at everywhere at the human cost of what the orange man is doing depending on his morning coffee. much better to send data to china than to US right now.
Have you ever wondered why Manus founders gave up billions of dollars from Meta and “voluntarily” moved back to Beijing? One country does things out in the open, while the other operates in the shadows. The latter worries me more.
It worries me far more what people are ok with us doing in broad daylight.
> Have you ever wondered why Manus founders gave up billions of dollars from Meta and “voluntarily” moved back to Beijing?
This country wouldn't be such a shithole if we were able to bring the wealthy to heel. I suspect most people on earth see China as the more competently governed country. Understandably, this forum tends to select for people who view private investment positively, but this is clearly insufficient by itself to produce functional societies.
You mean the project that literally had an anus as its mascot? Wasn’t that a joke project? Maybe the government was admonishing them for naming their company “man-anus”
I mean, someone has to say it, but Anthropic is the one who has an anus as a mascot. More specifically, Claude. It even winks at you, and clinches, too. I'm sorry that I have to be the one to report this to anyone reading this, I truly am. As the saying goes, don't kill the messenger.
I live in the US right now, my surface of attack from the US govt & ad companies is infinitely larger even if I decide to move to China some time in the future
The biggest problem with this is that I can't run my company's own internal benchmarks against this API, because they will be trained into the next version of the model, making them worthless.
> I'd rather my data go to China and directly to labs that release open weight models (DeepSeek is MIT licensed still); instead of closed labs and the NSA/CIA; who has far better ability to hurt me than China does.
Thank you. I've been shouting this very obvious thing from rooftops
I work in silicon valley and I feel everyone around me is suffering from China-BAD psychosis and dismiss Chinese releases
No Fireworks for me. They invited me to test Firepass, screwed up the onboarding, charged me money (when the account was supposed to be gratis) and when I asked them to fix that after I used up my $$, they said they closed the program. After that, no emails were responded to, and we were on a person-to-person basis. Not sure what their deal is, but I'm off Fireworks.
to add to the corrent sibling answer, the unhinging is called abliteration by the wider community but it's hard to find someone who will serve that over API.
Depends on the question. Most models won't tell you how to make weapons because they're trained not to, but the political stuff is usually blocked on the API level
It doesn’t have to be better, it just had to be nearly equivalent. I’ve been doing a lot of vulnerability work lately (trying to get ahead of the bots finding exploits in our code) and the frontier models feel like a chaperone at a church dance: get too close and they show up with the ruler and enforce the safety margin. Using deepseek and glm and qwen, I’ve gotten nearly equivalent results without the constraints. I still use frontier models for things but outside of a company environment, I’m happy to spend 1/10 the money, get the same results, and not have try and trick the AI into working.
Yeah I've been doing this since even before you could run DS via claude code, it's damn near the same quality with plenty instances where claude had to eat a hat and a shoe and use DS corrections.
American folks really have a distorted sense of reality and few have their own mind, they listen to influencer noise and once self thinkers pick up the quality of things like DS, they will echo the statements.
In practice, it does help to immediately be able to tell if you missed something or wrote something incorrect in your prompt, compared to if you didn't have access to the CoT. If you see that it goes off wondering about stuff or thinking about stuff that you know are already solved/known, you cancel the execution, edit the prompt and redo, then observe the CoT again, until you've resolved any "misdirections".
Both OpenAI and Anthropic hides so much of that, that you don't really know if it misunderstood something or you got something wrong, until all the turns are complete or you directly see tool calls that implicitly tells you something is wrong.
You don't need to read any papers to see this effect concretely when developing together with LLMs.
I can think of plenty of tasks that even the closed frontier models fail miserably at. From my perspective we have a long way to go for these models to get better.
If I were part of a cabal that is interested in preserving control and trying to secure trillions of dollars, it would be a minor effort to nudge and bend the curve against competitors like the Asian AI complex.
Z.ai does offer access to DS AFAIK, so I guess that's what they mean? Maybe the feeling is that it provides more value that DS's own platform since you also get GLM, not sure.
I don't think it is the z.ai subscription that offers other models (at least I don't see any on mine). The Qwen token plan has a lot of chinese models including GLM and Deepseek and will probably be the one I use once my z.ai sub gets near the end.
It happens very frequently if you happen to work on anything low level, e.g. a C code base, a database code base, etc. that deals with memory, and these have nothing to do with cyber security. It is extremely annoying and makes these models unusable.
I like it because it does the thing I want it to do rather than moralizing.
No, I don't care about license provenance for all the components of a 30 line script that i'm using in my house, for myself to sync up some folders. No, i'm not going to find and archive the licenses for all the previous projects leading to this for future legal disclosure.
Yes, I own the DVD for the movie i'm transcoding. No I don't want to talk about the legality of rebroadcast.
Deepseek (and most chinese models as a whole) don't care . oAI and Anthropic models want to make it point #1.
... which is extra hilarious knowing the laws broken to create the damn models in the first place.
there's a literal meme about this floating around the harness/AI chats for the past few months[0]
and really , for 90% of standard agentic work, deepseek flash is more than good enough.
its the main driver for our Personal Assistant offering and users love it.
Google's Argon model via antigravity is unusable, for a small code review I asked, already spent like 10% of the weekly usage, in less than 10 mins. Very useless imho.
I always find DS or Qwen to be very spot on on audit changes or plans for just a few cents.
DeepSeek feels like an excellent sparring partner.
The others feel like mind readers.
I don't have to worry about any complex context engineering / multi agent stuff if I am using DeepSeek because the sequential stuff is fast enough. I don't have to bother about touching a million context length because its cheap enough.
Where it seems to lag behind is 3d, computer use, motion graphics, design but the initial demos seem like the vision model has improved substantially.
> Being able to see the CoT and not being randomly blocked for cyber review.
I had a bad experience with MiMo from Xiaomi. I had it trigger unsafe content blocks while fixing visual bugs in a completely vanilla space game it just wrote from scratch in the same session. It also got triggered for changes in some other benign greenfield app.
I kind of get it. We don't need the best of the best for everything, specially if cost is a big factor. Maybe DS is not at the top of the food chain, but it works for most tasks, and that's enough for a lot of work
zcode was not long ago revealed to upload your entire project folder to their server encrypted using a key you don't have. And yet you still want to subscribe?
The subscription plan from Z.ai doesn't stop you from using harnesses other than Z Code though?
And I am not sure why you are so worked up over it -- even meta-commentary wise, you have been shitting on everyone in this thread and your comments are in general very inflammatory. Stop that.
i let 4 different agents review the same dirty code base (200+ files changed, c++, game engine):
- deepseek v4.1 flash via deepseek API: took 20m to find one p1 issue that was relevant and a p2 issue that was irrelevant
- glm 5.3 flash NVFP4 via TP2 DGX Sparks: took 1h 5m to find a p3 documentation issue
- gpt 5.6 sol: took 5m30s for one p1 and one p2 issue. both relevant
- gpt 6.1 sol: took 4m30s for two p1 and one p2 issue. both relevant.
Worth to mention: none of those findings overlapped.
I let deepseek v4.1 flash and gpt 6.1 sol both plan fixes for the summary of the findings. in this particular scenario: gpt 6.1 sol planned to increase a buffer size from 128 to 256 to fit the 191 elements that blew past a test scenario. it included some headroom. deepseek v4.1 flash on the other hand decided to make it exactly 191 to satisfy the test, for example.
The TPS decode on deepseek is impressive. I never had a second / feeling of being left aone and though "damn this thing is working". Despite all my efforts, it took ~4x longer in a review, found less and planned worse than GPT 6.1 Sol. Wall clock time. I don't have token counts for comparison. If I were to grade those models from 1-10 (10 = top tier GG), I'd give deepseek a solid 6.5 and 6.1 sol a 7.5-8.0 maybe.
That may be the case, but simply running the same code review twice with the same model may give very different results as well. Impossible to know with the information we have.
yeah, there is definitely still a lot of vibes to all this still, I'm working on evals next to learn and leverage, to reduce the vibes component to the claims
today we can use some anecdata based on our observations, this also helps shape the dimensions we want to eval the agents-tasks on
In your case, is it a public repository that the llms might have found the issues for online?
I too am thinking about creating some benchmarks that I can understand myself. The LLM 'cheating' by finding known issues through some sort of history or online source is a path you want to protect against. Then again, you DO want them to be able to look online for people having similar issues.
I meant things like bugs. So lets say some open source project has a bug that is later found, you could theoretically use that in a benchmark for an LLM to see if it can find it.
Pelican took 48s for me on high, OpenRouter's default effort for this model. I also asked for a Breakout game to vary from just pelicans :). One shot, 69s, about 25k tokens, and it launched.
LLMs are non-deterministic, you could run the same prompt on the same code base with the same models, and have different result. I don't think only one run with each model is enough to draw conclusions.
New model comes out and someone says, "it can (or can't) draw ducks on bikes" or some esoteric thing. But it's not enough statistical power to claim anything other than a personal anecdote.
I got a flat tire in downtown today. Therefore, driving in downtown causes flat tires.
Honest question about the non-deterministic nature of llms. Every programming language provides some way to control the random seed. Is this not true in GPU programming land?
Yes, but for one thing, floating-point math is non-associative so even if you use the exact same random seed, you can get different results on different implementation kernels.
For another, the dependence on a random seed is just one example of how you can get different behavior. You could just as well get different results due to e.g. a tiny difference in prompt wording or punctuation or whitespace.
If a model passes some test on 70% of random seeds, and fails on the other 30%, then knowing the rough probability of success is much more useful than knowing one random sample of the outcome.
And empirically this kind of probabilistic success is quite common, because many LLM benchmarks report "pass@k" success rates that are much higher than the "pass@1" success rate on a single attempt.
I don’t understand what “floating-point math is non-associative” has to do with the random selection of possible result token set after the soft max operation. The weights of the model are set and don’t change during the many matrix operations(which is deterministic regardless of the floating point encoding scheme).
It depends on the execution order. Model computations can be scheduled differently depending on what else is going on. You need extra effort to guarantee the same order of execution. For example if addition saturates at 8, then 4+5-2 is non-associative.
It doesn't really matter for most purposes. Determinism doesn't make them any less chaotic, it's not like fixing the seed makes the output any more predictable. Some percentage of seeds will behave in ways you don't like for a particular input, and you can't know which beforehand.
On some models you can control the temperature so it responds the same every time to a specific prompt but the “randomness” of llms is usually a feature not a bug.
Probably a week ago someone posted a thread with the title "Why isn't the industry freaking out about DeepSeek 4.1 Flash?"
I have been intrigued about Chinese models for a long time, but never tried anything except an early Qwen. My daily driver was Luna, with occasional elevation to Terra or Sol - all on 5.6 as I found the output of 6 jarringly incomprehensible.
I still had 95 USD on my OpenRouter account, so I thought I would give DeepSeek Flash 4.1 a try.
I think now I can understand the person who created that thread.
Here is the setup I settled on, after a lot of refactoring and calibration of the workflow:
I have a "plan" prompt which explains to the agent (Luna) that I want to have a discussion about an idea I have in mind. The agent should use the Socratic method to pull out all the necessary info from me using pointed questions. When the agent feels it has a complete picture, it should elevate to a higher level model (Sol), and ask its review. If Sol has complaints, Luna keeps asking me until the points raised by Sol are clarified. The review rounds with Sol continue until Sol says "ready to implement".
At this point Luna creates four files in a directory I specify:
1. PLAN.md: the implementation plan with all the details that came out from the design discussion
2. PHASES.md: implementation phases
3. DECISIONS.md: design decisions (this serves the same role as ADRs aka. architectural decision records)
4. DIARY.md: append-only log of what happened during implementation
Files 2-4 are written by the implementing agent. Luna only adds headers to explain the purpose of the files.
Once these files are there, I start up a new Pi session, switch to Deepseek 4.1 Flash and then point it to this plan folder.
And then it goes brrrrr... :-)
Here is what I implemented in the past week using this method:
1. A capture/analysis/decompilation pipeline for C64 demos: I ask the agent to start a selected demo through VICE, and say "on" or "off" when I want it to start/stop capture. The capture process uses VICE's builtin trace functionality: the trace log is piped into Clojure where it is saved into an optimized binary format (chunks of trace events). Once I have everything captured, I unleash my analysis pipeline which checks the write patterns of the executed code: what it wrote (VIC, SID, CIA, memory) and when (rasterline, cycle). Then it splits the execution trace into segments by type deduced from the write patterns (decruncher, loader, raster, music) and then decompiles each segment into both traditional ASM files and a s-exp based syntax which roundtrips back into the in-memory representation. Once I have the results of the analysis pipeline, I can tell the agent to analyze any demopart and it can explain how the effects are built down to the tiniest timing detail. As a first test, I asked it to create a clean-room implementation of Fairlight's legendary intro and two days later I had it, with everything in sexp assembly, including the music routine and its data. (The reason it took so long was that this was the first decomp and it provided a lot of feedback to the pipeline development.) Finally I had it write an assembler, linker and packager which creates a PRG from the assembled and linked artifact, then runs it through VICE. Done.
2. A DSP compiler which lets me compile sexp forms of a C-like DSP language into native DSP kernels running on the JVM, JS and WASM. Currently I have about 600 tests which verify that the behavior of the three targets matches that of a reference interpreter written in Clojure. I have a full numeric tower with i32, f32, f64, support for arrays and structs, typical control flow constructs, and a library of DSP building blocks like oscillators, filters, LFOs, delays and a chorus effect. By the time this is ready (end of the weekend?), I will be able to build synthesizers like Synth1, Surge XT or Vital in Clojure/Script, compile the DSP kernels in the browser, choosing between JS or WASM...
Isn't there a lot of overlap between PLAN, PHASES and DECISION that pollutes the context ? Whereas a simple PLAN.MD to put the harness on track and an ARCHITECTURE as a single source of truth to prevent it go offtrack would suffice and gives it more space for higher level of reasoning ?
Have you thoroughly tested a slightly simpler workflow with one less stage but a frontier model doing the initial planning? Luna is amazing but it's a deliberate trade off designed for speed and cost savings. You can also give them a rule about speaking clearly.
I wrote a Pi extension called "model-escalation" which lets the model request an automatic switch to a smarter model for the next turn if it thinks the problem it is struggling with is too complex or if the user explicitly asks it to do so.
With this extension in hand, I could split the work between Luna and Sol, resulting in a ~95%-5% split. I don't have to switch to Sol when review time comes, Luna does it automatically. Sol tells its remarks, then Luna gets the control back and continues the back&forth (if there are still complaints).
I have started phasing into DeepSeek v4 (Dwarfstar quant from Antirez) as my daily driver and it's going very well. It can one shot iOS apps on my Mac Studio, it's a little slow and sometimes needs retries but can definitely do medium difficulty real work for me.
I didn't try the last flash model because I would need to use SSD streaming (I have 128 GB), but the benches for this release are so compelling (and it using less tokens to get tasks done) it might be worth the throughput hit. I will probably give it a try when I have some downtime.
It feels very freeing to be doing useful work on a local model on my own hardware with my own coding harness. The TCO argument is starting to look compelling too.
Try the Qwen 3.8 flash next model on DwarfStar - you will be very surprised. I’ve been using the 4 bit quant almost exclusively after having DS v4 flash be my daily driver. Qwen is faster and seems to be much better at coding tasks.
I've been using DS v4.1 Flash daily for nearly a month now on Agent Swarm experiments. I've tested it against GLM-5.3-flash and Opus 5.5. DS 4.1 comes out tops for me. Opus 5.5 is still the smartest, but DS 4.1's speed and price more than make up for the "intelligence" gap. I've ran Chief/Builder/Reviewer swarms autonomously for multiple days. I think my new workflow is going to have DS 4.1 Flash do all the engineering and just have Opus review its work. 4.1 Flash really is a very impressive model. Kudos to the Deepseek team.
Huh? This is a new checkpoint. It has identical architecture, only different weights, and most likely just more post-training.
Much like Qwen 3.8 Next; which is qwen4-exp in a lot of the code/patches, there's usually a tick-tock cycle with LLMs. That said, it's hard to know for closed frontier models.
My current experiment is running a Chief of Staff pattern within herdr, nothing too fancy. You should be able to find quite a few resources on the net.
Its a great update, I ran a benchmark comparing verbosity and intelligence compared to old weights.
- Reasoning effort = 1 is now actually usable and produces 2.5x less tokens than old weights, without intelligence lost.
- Overall 20% less tokens at each reasoning effort.
- Same intelligence and same tool-calling pattern.
I wonder how you would be able to reliably rely on a model like that for agentic work, even for implementation only after careful planning, how are you making sure you are not wasting tokens reviewing and correcting it over and over again ?
Yeah, I have the same question. I'd like to incorporate cheap & fast models like DS into my workflow to reduce my reliance on OpenAI/Anthropic, but after trying several such models and having to have a frontier model review/fix/redo everything multiple times per task, I gave up and simply have the frontier models (Sol/Opus) do everything. All of this fits pretty comfortably in my monthly subscription.
Maybe the equation changes if I exceed my subscription usage, and need to fill the gap with something else. But it seems like a huge waste of your time to go out of your way to use a cheap model if you have the tokens to spare in your subscription.
Does DeepSeek provide the inference code along with the weights? I read here recently that for models where only the weights are released, providers have to implement which could be different than what the labs themselves use.
366 comments
[ 9.1 ms ] story [ 11.9 ms ] threadhttps://www.reddit.com/r/DeepSeek/comments/1x84wke/deepseek_...
Lovely to see they're doing their own search though, I love how many folk are actively working on search clones at the moment.
How?
I am not saying OP is not telling the truth. I just really want to understand!
Anyone who tells you they have a “team of subagents” and “code reviewer” agents and “project manager” agents doesn’t know what they are doing. That’s how they burn thousands of dollars in a day.
I think I’ll trust the experience of the people in the labs who have unlimited token budgets and are running experiments continuously over some anonymous rando slinging mud on the internet.
Also my own experience, which says having code reviewer agents is one billion percent useful. That comment alone completely disqualifies you as a reliable source of info.
This is how one of my claude code session looks like right now, one prompt from yesterday:
- How long do you reckon it would take a human to do the task?
- What percentage of features/ feature-extensions you add would have been done at all if you didn't have agentic assistance? Is there a tension between llm productivity gains and 'because we can' feature additions?
- Are many of the tasks for throwaway v1 features, exploratory tries, or redos of the same?
- How speced out is the work you give it, eg did this task have pages of specs, how long did the spec take to write?
- How deeply will you manually review the work and how much time will it take? Will you read all the code / review at a high level and interrogate with llm / have the llm review and self iterate?
I do now add all the major features I want, and write a task list where I add a few more minor tasks each week that just come to my mind. I need to write them down before I forget. I often ask Claude to implement several different implementations/algorithms and compare them. There are plenty of trade-offs, which usually end up in building configuration profiles, so the application can work optimally in multiple environments.
The specification is actually really large, and has grown over time as I've been learning and new ideas has come to my mind. I do not review/audit the code, it's massive would take me months, no chance! I do however use code inspection/audit tools like TSAN, ASAN and more. I will also spend time on full code reviews with the coming Fable 5.5 when a release is about ready. I do however review behavior together with Claude, and this is also where the specification grow, because this and that should be possible, but not this and that. User stories also helps.
There's no reason to blow through tokens this way, there's a 99% chance nobody uses the output (except maybe themselves).
That’s the “how” we’re wondering about.
What exactly are they doing for you that's giving you $200 of value in such a short timeframe?
Maybe you're not being ambitious enough with these things? They are at their most fun when you spec out sprawling architecture documents and watch 1+8 astras buzzsaw through it.
I'm only at 260B for the year, though. I'm sure someone who has used more will be along shortly to chime in
E: The other subthread talking about tests is correct. Expensive models should not waste a single second waiting on tests or validation. If you cannot assign a single cheap model to be the test bitch, then just tell them to frontload everything and clean up later as a single wave.
Rewrite this 250kloc native codebase in 5 different languages on 5 platforms (Windows, Linux, MacOS, iOS, Android) using these 10 different design methodologies and architectures. Use 32 subagents. Perform extensive before/after benchmarking and report the results here.
That should keep 'em busy for a few weeks worth of tokens! I feel so productive!
It is like putting a jet engine on a shipping container and wondering why it needs to be constantly kept in afterburner to at least scrape on the road while smarter users put wheels on the shipping container.
How people spend more than 200 usd monthly is beyond my understanding, are you guys so inefficient with the usage? or I am the opposite?
I would be worried about your methodology when the subscription party ends. Make no mistake, one day, the party is ending and the people who are dining out on approaches that burn massive tokens are in for a rude shock.
Right now I use each of them in their own respective coding harnesses, deepseek-harness and kimi-code, but I do wonder what they are like inside something like opencode.
At the beginning of the month I realized I had more tokens and more capabilities on Opencode at 8$/month via DS 4.1 flash, than I had on a 200$ account in June with the american providers.
Literally both more tokens and capabilities than Opus 4.8/GPT 5.5.
Even if you pay per token, you're still unlikely to outspend american providers.
And southern Europe isn't exactly the poorest region around.
If you can get away with paying that much to a swe in Italy then why aren't Italian tech companies ruling the world on the back of ~free labor?
They are showing this new version scoring significantly higher across multiple coding benchmarks, beating Sol 5.6 (6.1 is the latest) across 10/11 benchmarks.
First, it's the tok/s and TTFT from providers. Deepseek blazes through work - at quite good quality - and you don't find yourself waiting for your agents to finish.
Second is the openness. Being able to see the CoT and not being randomly blocked for cyber review. You're never wondering if it's stuck or if it will randomly fail. Combined with the great cost value.. My next subscription will probably be Z.ai
Some people would say that you don't need this at all with the frontier llms. But when extending existing software or writing software that fits into an existing ecosystem, I find I still need to offer a lot of guidance. And it's better that the context window isn't flooded with code that was reverted after I looked at it.
Hit the nail on the head. Development methodologies pivoting at the whim of frontier models have been one of the things that have been worrying me as of late.
In a way, that’s the situation today. You can pay an experienced developer $$$$ to do it right, or you can outsource it and pay $$ but pay extra for more layers of review, have lots of unfixed bugs and more tech debt.
You are proposing that instead we pay the frontier model to clean it up. If it can, I expect it to cost somewhere in the neighborhood of an experienced dev because that’s the alternative. That’s the way the frontier AI companies will extracting the value of the work from you.
But there's two ways this could be true: Frontier models costs rise, or (more likely) experience dev costs fall
That's only true if a single company owns the frontier, otherwise the alternative is "a rival frontier model" and the cost will be the marginal price of the compute + amortized training costs.
You're car will have spun off the road and you'll find it harder and harder to control it and keep it on the road because you have less and less visibility into the vehicle, nor will you be able to clearly see out your windscreen to use the car and road metaphor. You'll be driving blind eventually, unless your working on some simple closed toy problems and/or other small systems.
Building large systems means understanding the system. Without understanding the system, you lose control of it.
The LLM can probably help you understand the system to gain/regain some control of it again, but now you're becoming a system specialist rather than someone who's repeating, "fix it" and "make it good".
Self-trained people (like me) are particularly susceptible to it and we're heavily represented in the software world. A lot of us either recognise that (or are coached) and invest the time learning the benefits, and limitations of, software architecture. And, importantly, keep reflecting, learning and updating our practise.
I keep going back to the YOLO mode every other year and try out new languages to see if we still need the structure and abstractions. Then VMs came along and all I can say is they're both even more important and now we have an army of virtual "ADHD idiot juniors developers" at our finger tips it's even easier to tell the good and bad abstractions from each other.
They already did the bare minimum allocation to both despite engineering screaming at them.
I highly doubt executives are going to accept slower release and sale cycles in order to provide better support.
If anything, I expect this trend to accelerate.
Sales are out of their control.
If there’s a market for quality and you don’t have it, sales are going down no matter what the PHB wants.
If there’s not, watch out for the vibe-coded competitor.
As long as the API is private or not dependent on by automated systems and you have good test coverage, you can get away with bad code IMO since you can always refactor. If it looks stupid but it works (and can be refactored/fixed/improved later if needed) it aint stupid.
To reach a more optimal solution, significant human steering is reqiured for every tiny decision. It’s the same as how early models needed hand holding on the basics, playing out again in the finer details. Human experts need to inject “eureka moments” from the outside.
And no feedback loop that I’ve tried has been able to conjure an understanding of these finer details that, as far as I can tell, just aren’t represented in the weights.
Modeling the basics of software engineering was easy, since things like writing code that passes some tests is easy to RL. But for anything not easily RL-able, we’ve basically ran out of human generated tokens to distill from.
Without some breakthrough in generalization, I remain unconvinced that we’ll be able to model the finer details of software engineering by looping the existing training corpus back into itself. Garbage in, garbage out.
It would just loop on things and go down rabbit holes more. DS flash has been so consistently good I've kinda settled on it for 90% of my work.
I still use adversarial review for bigger changes where I want to make sure any issues are caught early, but I genuinely feel as tho if no new models ever get made I'd be fine with just using DS forever.
Keeps repeating things I settled as accepted or rejected. But what really gets on my nerves? The language it uses – that aweful, useless typical LLM languages! No AGENTS.md instruction, no per session instruction, no just two-prompts-prior instruction seems to work, hell sometimes instruction regarding that in same prompt doesn't work. GOOD/BAD example along with instruction don't work. That wastes time and zones you out! Not to mention the fatigue of to and fro. DS has been working much better. Hell Gemini 3.1 Pro works so much better than GLM. I don't have extended experience with OpenAI and Anthromic models but the free claide.ai works quite well and better than GLM 5.3 (not even flash). Albeit I must say I rarely use claude.ai for coding, but technical tasks? All the time!
GLM's best response: "No excuse. My fault. You exactly asked me not to do exactly that and so did the AGENTS.md. I failed". Love it. Except that if I press I will get that response every 5 prompts.
Fwiw, to elaborate on mine, a small fast 8B-ish model should be able to few-shot simple text style transfer, such as converting one model's complex careful style to another model's punchy compact style. Perhaps also a 2B fine tune. So you might tweak your harness to transfer and present a preferred style.
DS just handles ambiguity and my bursts of detail with much more grace. I'm only using "max effort" and the CoT is reasonable compared to GLM.
This really depends on the provider, it's a very fast model with ours
yes!So it's not just me. In terms of how intelligent they want to be GPT is at the top then Claude. And actually when I was playing with Sol it meant that many tasks it would just try to do something else because it thought I don't know what I want. Even trying to nudge it back in the right direction wouldn't work.
I guess the models just have the cultures of their creatures.
I have a theory that mess in an LLM-generated codebase sort of grows exponentially. Each hack or sloppy abstraction chips away at the coherence of the project as a whole and makes the next feature harder to do. But perhaps more importantly, it demonstrates to the LLM, which is a probabilistic machine despite its impressive capabilities, that quality of the code is low, and it generates more low-quality hacks.
And eventually you reach knots upon knots that you have to untangle yourself because the LLM becomes too confused by the code it wrote itself. As models get better, the problem seems to be getting better. I'm needing to do this more and more rarely, and the tradeoff is still clearly worth it.
But I've seen Fable 5.1 absolutely lobotomized by someone who fills each chat context with half a million tokens of insipid garbage, and I don't see why this couldn't happen when coding as well. I'm not sure a solution to this general problem that LLMs can simulate idiots really well will be fixed any time soon.
Adding unit tests, increasing modularity, reducing duplication, and the like are all things you can tell the LLM to do explicitly. And it is happy to do it.
And you can have coding standards as part of your design criteria. Set and document these early in the project and audit the code at least enough to see they are being enforced.
Agents know these principles already and they are frequently the problem themselves.
Too strongly-insisting on eg DRY can lead to tortured abstractions shoehorning too much together. Agents need to be able to distinguish between incidentally similar code vs conceptually redundant code.
Excessive modularity, eg. militant single-responsibility principle can lead to abstraction hell that is impossible to reason about at once.
Unit tests are obviously good. Coding agents write a copious amounts of them, often baking you into poorly-chosen designs made without your knowledge.
Plus only one country has threatened to annex mine and it wasn't China.
The economic sovereignty of any nation, but especially smaller ones like canada and denmark, is harmed by sending our money out of the country when there are options available in country.
I respectfully disagree. I subscribe to the comparative advantage economic principle, so I'm fine with international trade.
The 'clean power' would factor in a well, but I haven't really looked in to it. I know China is installing a ton of renewable and has a carbon market while Canada's building a new pipeline and building Alberta data centres running on natural gas, so I'm not willing to take on faith that switching is cleaner.
Also, re: power. I said canada has lots of cheap power, clean was your word.
The unfortunate truth behind at least the solar PV boom in China is that it's driven in large part by the exploitation of Uyghurs in the Xinjiang region. One of the reasons they can sustain the pace they're going at is because while nowadays PV modules exports increasingly gravitate towards offshore fabs (Vietnam, Malaysia), domestically all the biggest PV players make full use of modern slavery within Xinjiang's re-education camps.
more on the topic: https://www.shu.ac.uk/-/media/home/research/helena-kennedy-c...
It might be a minor nitpick, but that part is just not how the macroeconomic principle works. Abundance of a good/service within a country is irrelevant to the concept of a comparative advantage - this is counter-intuitive but but that was David Ricardo's entire point. It's not about whether production is more efficient domestically, it's whether comparative allocation of resources between multiple countries is more efficient by specialization within the different goods.
Even if you can produce "good 1" cheaper in Country A vs Country B, if Country A can instead produce "good 2" that Country B cannot, it's better for Country A to produce "good 2" domestically and import "good 1".
Now, I still agree with the original notion. It's unnecessarily risky to use services from "democratically-challenged" nations (especially imperialist ones), when safer options are available. But there's no need to twist macro 101 concepts to justify that view. Modern trade policy takes more into account than basic market efficiency.
As someone who's in the market for Canadian inference (data sovereignty reasons), the pickings are slim for DSF 4.1 man.
(But I'd concede that in the field of genocide China doesn't come up as bad)
[0] https://openrouter.ai/deepseek/deepseek-v4.1-flash
(The badges next to companies indicate their privacy terms, i.e. green/yellow/red)
The "green" badges explicitly say the opposite:
> "Private: this provider does not train on prompts and does not retain prompt data."
I.e. for the example of Fireworks, this is the precise language they use to describe ZDR:
> "More technically: prompt and generation data exist only in volatile memory for the duration of the request. If prompt caching is active, some prompt data (and associated KV caches) can be stored in volatile memory for several minutes. In either case, prompt and generation data are not logged into any persistent storage."
https://docs.fireworks.ai/guides/security_compliance/data_ha...
Without getting too much into politics; over the past several decades, one country is going around starting wars, regime changes and other interventionism, or initiating economic warfare (Huawei sanctions because they were getting too competitive to US companies; GPU restrictions; tariffs; etc), and that's not one administration: it's US policy over the past few years.
https://ourworldindata.org/grapher/human-rights-index-vdem
Yes, people have put thought into this. This is why I trust institutions and not random posters on the internet.
Just look at Russia - it's score is highest in the 90s - the most corrupt and destitute time in its modern history, also happens to be when it was deemed somewhat friendly to US and friends.
It's kinda like meeting someone on the street who's protesting about animal cruelty in slaughterhouses, slyly handing them a chart showing them the nutrition facts of beef, and then walking off without another word as though you've settled your case.
Like, I get the point you're trying to make, but I hope you're aware it's a non-sequitur in context?
Your hope should have been that amelius was not aware that it's a non-sequitur, because that is correctable. But then they kept posting and removed all doubt.
https://www.prisonstudies.org/world-prison-data/highest-lowe...
https://www.theguardian.com/us-news/2022/sep/27/slavery-loop...
In Arkansas, agricultural work accounted for 17% of all prison labor assignments in 2021. https://www.prisonlegalnews.org/media/publications/Captive_L...
> Have you ever wondered why Manus founders gave up billions of dollars from Meta and “voluntarily” moved back to Beijing?
This country wouldn't be such a shithole if we were able to bring the wealthy to heel. I suspect most people on earth see China as the more competently governed country. Understandably, this forum tends to select for people who view private investment positively, but this is clearly insufficient by itself to produce functional societies.
boy oh boy, if you think the US doesn't do anything in the shadows. The last 100 years of world geopolictical history would beg to disagree
Only thing is they aren't clear on the SOC2 certification. Am I being naive?
Thank you. I've been shouting this very obvious thing from rooftops
I work in silicon valley and I feel everyone around me is suffering from China-BAD psychosis and dismiss Chinese releases
Wait wut? You do realize that they are two different companies, right?
It is very capable for a lot of tasks, not only coding like the other open models.
It doesn’t have to be better, it just had to be nearly equivalent. I’ve been doing a lot of vulnerability work lately (trying to get ahead of the bots finding exploits in our code) and the frontier models feel like a chaperone at a church dance: get too close and they show up with the ruler and enforce the safety margin. Using deepseek and glm and qwen, I’ve gotten nearly equivalent results without the constraints. I still use frontier models for things but outside of a company environment, I’m happy to spend 1/10 the money, get the same results, and not have try and trick the AI into working.
American folks really have a distorted sense of reality and few have their own mind, they listen to influencer noise and once self thinkers pick up the quality of things like DS, they will echo the statements.
Dont assume that is how your model decides...
https://arxiv.org/abs/2305.04388
https://arxiv.org/abs/2307.13702
https://arxiv.org/abs/2503.08679
I love COT and gives me a good idea about the thought process. Sure it doesn’t work always
Both OpenAI and Anthropic hides so much of that, that you don't really know if it misunderstood something or you got something wrong, until all the turns are complete or you directly see tool calls that implicitly tells you something is wrong.
You don't need to read any papers to see this effect concretely when developing together with LLMs.
It's only for long running/very difficult cases that the differences are here, and they are small.
Wait... you just spent 3 paragraphs singing the praises of DS... and then you say you're going to subscribe to the one that makes GLM. ?
(I do tend to agree with you, though I tend to favor GLM-5.3-flash over DS 4.1 flash.)
debugging reasoning traces more effectively seems useful to find prompt issues writing custom harnesses even if it runs on stronger model later
Is DeepSeek: DeepSeek v4.1 Flash 1017 any better?
[1] - https://www.youtube.com/watch?v=P4dTq4X8bqk
please don't subject yourself to this. z.ai subscriptions are not a good value. just pay API rates, or use a subscription like opencode go.
No, I don't care about license provenance for all the components of a 30 line script that i'm using in my house, for myself to sync up some folders. No, i'm not going to find and archive the licenses for all the previous projects leading to this for future legal disclosure.
Yes, I own the DVD for the movie i'm transcoding. No I don't want to talk about the legality of rebroadcast.
Deepseek (and most chinese models as a whole) don't care . oAI and Anthropic models want to make it point #1.
... which is extra hilarious knowing the laws broken to create the damn models in the first place.
there's a literal meme about this floating around the harness/AI chats for the past few months[0]
[0]: https://i.ibb.co/SwH4Sgwq/796779070-28483090404651613-920066...
I always find DS or Qwen to be very spot on on audit changes or plans for just a few cents.
DeepSeek feels like an excellent sparring partner. The others feel like mind readers.
I don't have to worry about any complex context engineering / multi agent stuff if I am using DeepSeek because the sequential stuff is fast enough. I don't have to bother about touching a million context length because its cheap enough.
Where it seems to lag behind is 3d, computer use, motion graphics, design but the initial demos seem like the vision model has improved substantially.
I had a bad experience with MiMo from Xiaomi. I had it trigger unsafe content blocks while fixing visual bugs in a completely vanilla space game it just wrote from scratch in the same session. It also got triggered for changes in some other benign greenfield app.
Also z.ai doesn't offer Deepseek models.
And I am not sure why you are so worked up over it -- even meta-commentary wise, you have been shitting on everyone in this thread and your comments are in general very inflammatory. Stop that.
i let 4 different agents review the same dirty code base (200+ files changed, c++, game engine):
- deepseek v4.1 flash via deepseek API: took 20m to find one p1 issue that was relevant and a p2 issue that was irrelevant
- glm 5.3 flash NVFP4 via TP2 DGX Sparks: took 1h 5m to find a p3 documentation issue
- gpt 5.6 sol: took 5m30s for one p1 and one p2 issue. both relevant
- gpt 6.1 sol: took 4m30s for two p1 and one p2 issue. both relevant.
Worth to mention: none of those findings overlapped.
I let deepseek v4.1 flash and gpt 6.1 sol both plan fixes for the summary of the findings. in this particular scenario: gpt 6.1 sol planned to increase a buffer size from 128 to 256 to fit the 191 elements that blew past a test scenario. it included some headroom. deepseek v4.1 flash on the other hand decided to make it exactly 191 to satisfy the test, for example.
The TPS decode on deepseek is impressive. I never had a second / feeling of being left aone and though "damn this thing is working". Despite all my efforts, it took ~4x longer in a review, found less and planned worse than GPT 6.1 Sol. Wall clock time. I don't have token counts for comparison. If I were to grade those models from 1-10 (10 = top tier GG), I'd give deepseek a solid 6.5 and 6.1 sol a 7.5-8.0 maybe.
This read like a conversation you were having in your head -- unfiltered.
The lower the number the higher the priority, e.g. if production is broken someone might say "This is p -1" (higher than your highest priority).
P0 is "do this immediately", P1 is very important, P2/P3 are like "nice to haves." P100 would mean something like "we will never do this"
At 3am,
P0 = get people out of bed and fix this now.
P1 = we can fix this in the morning.
P2 = we can fix this before CoB today.
P3 = push it into the current sprint.
P4 = put it in backlog.
This is why using multiple model families is important. Don't limit yourself by being a stan for this or that company.
today we can use some anecdata based on our observations, this also helps shape the dimensions we want to eval the agents-tasks on
I too am thinking about creating some benchmarks that I can understand myself. The LLM 'cheating' by finding known issues through some sort of history or online source is a path you want to protect against. Then again, you DO want them to be able to look online for people having similar issues.
Yet to be obsolete.
- For some reason choosing only a subset of issues.
- Different LLMs actually learn different things.
I suspect the former since they do this for me and it's extremely annoying sometimes
But try it on high instead of max if possible. max is known for being overly verbose to the point of being detrimental compared to high at times.
For example Simmons svg pelican took 11 minutes on max and 1 minute on high. And they are comparable.
I recall the difference being minimal in 4.0.
New model comes out and someone says, "it can (or can't) draw ducks on bikes" or some esoteric thing. But it's not enough statistical power to claim anything other than a personal anecdote.
I got a flat tire in downtown today. Therefore, driving in downtown causes flat tires.
For another, the dependence on a random seed is just one example of how you can get different behavior. You could just as well get different results due to e.g. a tiny difference in prompt wording or punctuation or whitespace.
If a model passes some test on 70% of random seeds, and fails on the other 30%, then knowing the rough probability of success is much more useful than knowing one random sample of the outcome.
And empirically this kind of probabilistic success is quite common, because many LLM benchmarks report "pass@k" success rates that are much higher than the "pass@1" success rate on a single attempt.
This is not true. Non determinism only arises via sampling, and to a much lesser extent, CUDA kernel launch subtleties.
Were you using the same setup to call these models? was it pi/opencode or was it codex/claude etc etc.
Because I've started to see larger and larger drifts in the results depending on the agent it self rather then the model.
I have been intrigued about Chinese models for a long time, but never tried anything except an early Qwen. My daily driver was Luna, with occasional elevation to Terra or Sol - all on 5.6 as I found the output of 6 jarringly incomprehensible.
I still had 95 USD on my OpenRouter account, so I thought I would give DeepSeek Flash 4.1 a try.
I think now I can understand the person who created that thread.
Here is the setup I settled on, after a lot of refactoring and calibration of the workflow:
I have a "plan" prompt which explains to the agent (Luna) that I want to have a discussion about an idea I have in mind. The agent should use the Socratic method to pull out all the necessary info from me using pointed questions. When the agent feels it has a complete picture, it should elevate to a higher level model (Sol), and ask its review. If Sol has complaints, Luna keeps asking me until the points raised by Sol are clarified. The review rounds with Sol continue until Sol says "ready to implement".
At this point Luna creates four files in a directory I specify:
1. PLAN.md: the implementation plan with all the details that came out from the design discussion 2. PHASES.md: implementation phases 3. DECISIONS.md: design decisions (this serves the same role as ADRs aka. architectural decision records) 4. DIARY.md: append-only log of what happened during implementation
Files 2-4 are written by the implementing agent. Luna only adds headers to explain the purpose of the files.
Once these files are there, I start up a new Pi session, switch to Deepseek 4.1 Flash and then point it to this plan folder.
And then it goes brrrrr... :-)
Here is what I implemented in the past week using this method:
1. A capture/analysis/decompilation pipeline for C64 demos: I ask the agent to start a selected demo through VICE, and say "on" or "off" when I want it to start/stop capture. The capture process uses VICE's builtin trace functionality: the trace log is piped into Clojure where it is saved into an optimized binary format (chunks of trace events). Once I have everything captured, I unleash my analysis pipeline which checks the write patterns of the executed code: what it wrote (VIC, SID, CIA, memory) and when (rasterline, cycle). Then it splits the execution trace into segments by type deduced from the write patterns (decruncher, loader, raster, music) and then decompiles each segment into both traditional ASM files and a s-exp based syntax which roundtrips back into the in-memory representation. Once I have the results of the analysis pipeline, I can tell the agent to analyze any demopart and it can explain how the effects are built down to the tiniest timing detail. As a first test, I asked it to create a clean-room implementation of Fairlight's legendary intro and two days later I had it, with everything in sexp assembly, including the music routine and its data. (The reason it took so long was that this was the first decomp and it provided a lot of feedback to the pipeline development.) Finally I had it write an assembler, linker and packager which creates a PRG from the assembled and linked artifact, then runs it through VICE. Done.
2. A DSP compiler which lets me compile sexp forms of a C-like DSP language into native DSP kernels running on the JVM, JS and WASM. Currently I have about 600 tests which verify that the behavior of the three targets matches that of a reference interpreter written in Clojure. I have a full numeric tower with i32, f32, f64, support for arrays and structs, typical control flow constructs, and a library of DSP building blocks like oscillators, filters, LFOs, delays and a chorus effect. By the time this is ready (end of the weekend?), I will be able to build synthesizers like Synth1, Surge XT or Vital in Clojure/Script, compile the DSP kernels in the browser, choosing between JS or WASM...
I wrote a Pi extension called "model-escalation" which lets the model request an automatic switch to a smarter model for the next turn if it thinks the problem it is struggling with is too complex or if the user explicitly asks it to do so.
With this extension in hand, I could split the work between Luna and Sol, resulting in a ~95%-5% split. I don't have to switch to Sol when review time comes, Luna does it automatically. Sol tells its remarks, then Luna gets the control back and continues the back&forth (if there are still complaints).
I didn't try the last flash model because I would need to use SSD streaming (I have 128 GB), but the benches for this release are so compelling (and it using less tokens to get tasks done) it might be worth the throughput hit. I will probably give it a try when I have some downtime.
It feels very freeing to be doing useful work on a local model on my own hardware with my own coding harness. The TCO argument is starting to look compelling too.
Much like Qwen 3.8 Next; which is qwen4-exp in a lot of the code/patches, there's usually a tick-tock cycle with LLMs. That said, it's hard to know for closed frontier models.
DeepSeek delivery deliberately uses confusing version numbers.
- Reasoning effort = 1 is now actually usable and produces 2.5x less tokens than old weights, without intelligence lost. - Overall 20% less tokens at each reasoning effort. - Same intelligence and same tool-calling pattern.
More details in blog post https://sference.com/blog/deepseek-v41f-1017-benchmark
Maybe the equation changes if I exceed my subscription usage, and need to fill the gap with something else. But it seems like a huge waste of your time to go out of your way to use a cheap model if you have the tokens to spare in your subscription.
Definitely an improvement on deepseek-v4-flash-0731: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
Looks much better.
https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/...
> providers have to implement which could be different than what the labs themselves use.
I'd imagine (most) providers just take implementation by vLLM/SGLang.