It surprises me that this concept took as long as it did to gain traction in… hacker news. 15 years ago folks here were compiling kernels and gentoo distros. Lately it’s been “you should just pay the man, it’s cheaper than running these things yourself”
True, I see so many posts along the lines of 'I am a quasi programming god with no time on my hands (apart from time wasted here) so I will gladly pay $10,000 a month to Anthropic if it makes me 1% more effective than if I were to use GLM'
Actually, if you break tasks down into small enough units, there's not much difference between open models and closed models. The only reason to use agents is the hope that they'll work with natural language input—but that's where the difficulty lies.
For example, if you modify things at the level of small functions, open models seem to perform just as wel
To be honest, I am surprised by how good DeepSeek V4 Flash is. I use Claude Code and Codex on Claude 5 Opus and GPT-5.6 Sol most of the time, but when I use DS V4 Flash I don't feel like it's really that bad. And with oh-my-pi and just plain pi it's pretty good. To be honest the frontier models are much better at tool calling so in an assistant flow they're better but I did the dumb thing and optimized the harness for the model instead, rewriting the tools so they match what it guesses at, and DS V4 Flash does just fine.
The TTFT and tok/s are much higher on the small model so that makes it competitive for a bunch of things. It feels like what old Sonnet used to by the end of last year which is honestly damned good.
This is only a thinly veiled ad. It's fine, I was curious about this exact setup, anyways.
What would be useful is a cost metric. I'm curious how much I'd be willing to spend as a premium to not have those companies piping my conversations directly to the NSA. Maybe only some conversations? Claude and OpenAI are heavily subsidized, by all accounts, so Kimi K3 on a private endpoint might end up costing more or less - that's what I want to know.
I love OpenCode and the blankness of it too. Clean, light, and manual. It's a good change of pace from what we are used to on the internet (very cool but slow). Also, can you hmu with that modal plan XD
The reason Claude code is so popular is because it’s really good at taking super vague human prose “Claude build me a million dollar SaaS”-type prompts and spitting out thousands of lines of code which cover tons of surface-level edge cases, build in tons of functionality, etc
The smaller/open models are less good at that. But that’s not how software development is done. You don’t prompt a whole app and be done with it. If you’re using it as an aid to traditional software dev, iterating on small, targeted functions, GLM works as good if not better than Claude. Anthropic expects low quality prompts. If you rubber duck GLM, you get absolutely pristine output in most cases.
Are local models more snappy? I'm at a stage where I can work with the output of LLMs, and the next win is really just getting things written out quickly.
Depends on your setup. If you drop $10k on an RTX Pro 6000 then yeah Qwen 35B MoE will absolutely fly.
If you have a pair of 3090s and run Qwen 27B, or an old Threadripper with heaps of system RAM and Deepseek or MiniMax or Kimi, no it won't be as fast as Claude.
Most local LLM nerds are not running locally for superior speed, we're doing it for sovereignty and/or privacy, or maybe just because it's fun which accidentally became useful this year.
I had what I thought was a completely ridiculous prompt: “Build me a replacement for Microsoft Word.” ChatGPT did what I thought was the right thing: it responded asking along the lines of “you don’t really want that do you?” and explained how Word is decades of corner cases, bug fixes, obscure features, and business processes that have been built around it.
I was stunned that Claude simply started spinning its wheels in an attempt to actually build something.
I’ve enjoyed using Kimi K3. The biggest issue for me so far is finding a good harness. Claude Code is excellent for tasks that require backgrounding processes/agents, or fanning out. Codex and OpenCode I’ve found to be less capable.
I know the author has a long history here and I'm sure the post is a genuine reflection. But how is posting "I used my own product and it felt really good" anything other than self promotion?
I mean, many posts on HN are self promotion; whether they reach the front page discerns the interesting ones from the ones that are not.
An interesting self promotion is still interesting.
Classic HN. Usually people write longer blogs to hide the self-promotion/products promotion. Better hide the promotion into 2000 character mumble jumble.
Hello! It probably doesn’t make much of a difference now that the post is flagged (I assume that it has something to do with people thinking it’s an ad?) but just for the record, I really didn’t intend this as an “ad”. No one asked me to write this and I didn’t plan on writing it. It basically happened just how I described it - I went home, had some extra energy, wanted to build something that needed both 1) an LLM to help me build the thing and 2) an LLM to use within the thing I was building. So I decided to try the Modal endpoint bc it was top of mind and I didn’t want to upgrade my Claude account. Had I not thought it was cool I would not have written about it, but what made me write the thing wasn’t “Modal is cool”, it was the feeling I describe in the post. I was a little worried it might come across as an ad and considered removing the links but figured, “oh well”. It was a nice plus that it involved the company I’m at doing something cool, but that wasn’t really the point. In any case, I’m glad some people found it interesting, and maybe this was flagged for some other reason, but thought I might as well respond, even if I may not convince anyone :) Possibly even given the above it’s not HN-appropriate somehow but I thought that the sentiment might be appreciated
40 comments
[ 0.23 ms ] story [ 18.4 ms ] threadFor example, if you modify things at the level of small functions, open models seem to perform just as wel
Now I can't wait for someone to distill K3 into a Qwen 3.6 27b or Poolside S 2.1 sized models for a proper fast local Composer 2.5 replacement.
The TTFT and tok/s are much higher on the small model so that makes it competitive for a bunch of things. It feels like what old Sonnet used to by the end of last year which is honestly damned good.
What would be useful is a cost metric. I'm curious how much I'd be willing to spend as a premium to not have those companies piping my conversations directly to the NSA. Maybe only some conversations? Claude and OpenAI are heavily subsidized, by all accounts, so Kimi K3 on a private endpoint might end up costing more or less - that's what I want to know.
The smaller/open models are less good at that. But that’s not how software development is done. You don’t prompt a whole app and be done with it. If you’re using it as an aid to traditional software dev, iterating on small, targeted functions, GLM works as good if not better than Claude. Anthropic expects low quality prompts. If you rubber duck GLM, you get absolutely pristine output in most cases.
If you have a pair of 3090s and run Qwen 27B, or an old Threadripper with heaps of system RAM and Deepseek or MiniMax or Kimi, no it won't be as fast as Claude.
Most local LLM nerds are not running locally for superior speed, we're doing it for sovereignty and/or privacy, or maybe just because it's fun which accidentally became useful this year.
I was stunned that Claude simply started spinning its wheels in an attempt to actually build something.
If it sounds good it must be good. The code this new model writes is incredible! It told me so!
Harness-aside, I get this feeling sometimes when I swap from a big frontier model to something more nimble like Composer.
I can get into a better thinking and q&a loop with fast models, similar to how I can flow through a file more easily with vim.
I know the author has a long history here and I'm sure the post is a genuine reflection. But how is posting "I used my own product and it felt really good" anything other than self promotion?
https://news.ycombinator.com/item?id=22878515