My question on all of the “can’t work with big codebases” is how would a codebase that was designed for an LLM look like? Composed of many many small functions that can be composed together?
I'm going a little offtopic here, but I disagree with the OPs use of the term "PhD-level knowledge", although I have a huge amount of respect for antirez (beside that we are born in the same island).
This phrasing can be misleading and points to a broader misunderstanding about the nature of doctoral studies, which it has been influenced by the marketing and hype discourse surrounding AI labs.
The assertion that there is a defined "PhD-level knowledge" is pretty useless. The primary purpose of a PhD is not simply to acquire a vast amount of pre-existing knowledge, but rather to learn how to conduct research.
Except during the data science craze of 2015s, there was never a situation that you could just have a phd in any field and get any "phd level job", so whatever pedantic idea you have of what phds learn, not a single person who's hiring phds agrees with you. On the contrary, even most phd professors treat you as only a vessel of the very specific topic you studied during your phd. Go try to get a postdoc in a top lab when your PhD was not exactly what they work on already. I know I tried! Then gave up.
Quite. "PhD-level knowledge" is the introduction to one's PhD thesis. The point of doing a PhD is to extend knowledge beyond what is already known, i.e that which cannot be known by an LLM.
Unlike OP, from my still limited but intense month or so diving into this topic so far, I had better luck with Gemini 2.5 PRO and Opus 4 on more abstract level like architecture etc. and then dealing input to Sonnet for coding. I found 2.5 PRO, and to a lesser degree Opus, were hit or miss; A lot of instances of them circling around the issue and correcting itself when coding (Gemini especially so), whereas Sonnet would cut to the chase, but needed explicit take on it to be efficient.
“Always be part of the loop by moving code by hand from your terminal to the LLM web interface: this guarantees that you follow every process. You are still the coder, but augmented.”
I agree with this, but this is why I use a CLI. You can pipe files instead of copying and pasting.
Since I’ve heard Gemini-cli is not yet up to snuff, has anyone tried opencode+gemini? I’ve heard that with opencode you can login with Google account (have NOT confirmed this, but if anyone has any experience, pls advise) so not sure if that would get extra mileage from Gemini’s limits vs using a Gemini api key?
Whether it's vibe coding, agentic coding, or copy pasting from the web interface to your editor, it's still sad to see the normalization of private (i.e., paid) LLM models. I like the progress that LLMs introduce and I see them as a powerful tool, but I cannot understand how programmers (whether complete nobodies or popular figures) dont mind adding a strong dependency on a third party in order to keep programming. Programming used to be (and still is, to a large extent) an activity that can be done with open and free tools. I am afraid that in a few years, that will no longer be possible (as in most programmers will be so tied to a paid LLM, that not using them would be like not using an IDE or vim nowadays), since everyone is using private LLMs. The excuse "but you earn six figures, what' $200/month to you?" doesn't really capture the issue here.
Where is the strong dependency? I can point Cursor at Openrouter and use any LLM I want, from multiple providers. Every LLM provider supports the same OpenAI completions API.
I wish the stuff on top - deep research modes, multimodal, voice etc. had a more unified API as well but there's very little lock-in right now.
The advantage of having smarter models is greater than the risk/harm of them being closed source, especially so when speed of execution is a major factor.
What I thought you were going to say is “Gemini- wha?”.
I’ve used Gemini 2.5 PRO and would definitely not use it for most of my coding tasks. Yes, it’s better at hard things, but it’s not great at normal things.
I’ve not used Claude 4 Opus yet- I know it’s great at large context- but Claude 4 Sonnet Thinking is mostly good unless the task is too complex, and Claude 4 Sonnet is good for basic operations in 1-2 files- beyond that it’s challenged and makes awful mistakes.
If you are so tied to an LLM that you cannot program without one then you are not a programmer. It is not the same as with an IDE or whatever, those are essentially ergonomic and do not speak to your actual competence. This will probably be an unpopular take but its just reality.
Us FORTH and LISP hackers will be doing free range code forever.
We can use cheap hardware that can be fixed with soldering irons and oscilloscopes.
People said for decades our projects just become weird DSLs. And now whatever little thing I want to do in any mainstream language involves learning some weird library DSL.
And now people be needing 24h GPU farm access to handle code.
In 50 years my grandkids that wish to will be able to build, repair and program computers with a garage workbench and old wrinkled books. I know most of the software economy will end up in the hands of major corporations capable of paying through the nose for black box low code solutions.
Doesn't matter. Knowledge will set you free if you know where to look.
> The excuse "but you earn six figures, what' $200/month to you?" doesn't really capture the issue here.
Why?
If I want to pick up many hobbies, not to mention lines of professional work, I have to pay for tools. Why is programming any different? Why should it be?
Complaining that tools that improve your life cost money is... weird, IMO. What's the alternative? A world in which people gift you life-and-work-improving tools for free? For no reason? That doesn't exist.
> Programming used to be (and still is, to a large extent) an activity that can be done with open and free tools.
Btw, I think this was actually less true in the past. Compilers used to cost money in the 70s/80s. I think it's actually cyclical - most likely tools will cost money today, but then down the line, things will start getting cheaper again until they're free.
That's a code time dependency which is of least concern. That's like saying how can a company hire a developer, they are now having a dependency.
Code written will continue to run, you can use alternate LLM to further write code. Unless you are a pure vibe coder, you can still type the code. IF programmer stop learning how to write code, that's on them and not Claude's or antirez responsibility.
We are using best tool available to us, right now it is Claude Code, tomorrow it may be something from OpenAI, Meta or Deepseek.
I share uour concerns absolutely, and would run an open model local (or self hosted), but for now the reality is that what is available as such is not satisfactory compared to the closed frontier models only available through SaaS.
I hope this will change before (captured) regulation strangles open models.
I've personally felt this way about many proprietary tech ecosystems in the past. Still do. I don't want to invest my energy to learn something if the carpet can be pulled from under my feet. And it does happen.
But that is my personal value judgement. And it doesn't mean other people will think the same. Luckily tech is a big space.
The software is largely there: you can run Ollama, vLLM or whatever else you please today.
The models are somewhat getting there: even the smaller ones like Qwen3-30B-A3B and Devstral-23B are okay for some use cases and can run decently fast. They’re not amazing, but better than much larger models a year or two ago.
The hardware is absolutely not there: most development laptops will be too weak to run a bunch of tools, IDEs and local services alongside a LLM and will struggle to do everything at the pace of those cloud services.
Even if you seek compromise and get a pair of Nvidia L4 cards or something similar and put them on a server somewhere, the aforementioned Qwen3-30B-A3B will run at around 60 tokens/second for a single query but slow down as you throw a bunch of developers at it that all need chat and autocomplete. The smaller Devstral model will more than halve the performance at the starting point because it’s dense.
Tools like GitHub Copilot allow an Ollama connection pretty easily, Continue.dev also does but can be a bit buggy (their VS Code implementation is better than their JetBrains one), whereas the likes of RooCode only seem viable with cloud models cause they generate large system prompts and need more performance than you can squeeze out of somewhat modest hardware.
That said, with more MoE models and better training, things seem hopeful. Just look at the recent ERNIE-4.5 release, their model is a bit smaller than Qwen3 but has largely comparable benchmark results.
Those Intel Arc Pro B60 cards can’t come soon enough. Someone needs to at least provide a passable alternative to Nvidia, nothing more.
> Programming used to be (and still is, to a large extent) an activity that can be done with open and free tools.
This was largely not the case before 2000, and may not be the case after, say 2030. It may well be that we are living in extraordinary times.
Before 2000 one had to buy expensive compilers and hardware to do serious programming. The rise of the commercial internet has made desktop computers cheaper, and the free software movement (again mostly by means of commercial companies enjoying the benefits) has made a lot of free software available.
However, we now have mostly smartphones, and serious security risks to deal with.
Genuine question as I don't understand the deeper aspects of this, but is it possible that we will see AI specific hardware in the future that will make local AI's more possible?
My general thinking is that we're using graphics processors which _work_ but aren't really designed for AI (lack of memory, etc.).
I find agentic coding to be best when using one branch per conversation. Even if that conversation is only a single bugfix, branch it. Then do 2 or 3 iterations of that same conversation across multiple branches and choose the best result of the 3 and destroy the other two.
Can anyone recommend a workflow / tools that accomplishes a slightly more augmented version of antirez’ workflow & suggestions minus the copy-pasting?
I am on board to agree that pure LLM + pure original full code as context is the best path at the moment, but I’d love to be able to use some shortcuts like quickly applying changes, checkpoints, etc.
My persistent (and not unfounded?) worry is that all the major tools & plugins (Cursor, Cline/Roo) all play games with their own sub-prompts and context “efficiency”.
I use the agent panel in my editor of choice (Zed).
For each task I always start with a new (empty) context and manually tag relevant files to include (this is trivial since I know the codebase well).
First I use Claude 4 Sonnet in thinking mode (I could also use Gemini 2.5 Pro or Opus as per antirez' recommendations) to come up with a detailed plan on how to implement something (research/planning phase). I provide feedback and we iterate on the plan.
Then, in the same conversation I switch to Sonnet 4 non-thinking and tell it to implement what we just devised.
I manually review the changes and trst them. If something needs fixing or (more often) if I notice I missed some edge case/caveat, I tell it to do that (still same convo).
Commit, clear convo, next task.
For research that isn't directly tied to the code, I use ChatGPT or Claude (web apps) to brainstorm ideas, and sometimes copy/pasre these into the editor agent as starting point.
Github Copilot's Edit mode allows you to manually specify the context, and it runs only once each time to write code with diff checking, without entering a agent loop.
I think all conversations about coding with LLMs, vibe coding, etc. need to note the domain and choice of programming language.
IMHO those two variables are 10x (maybe 100x) more explanatory than any vibe coding setup one can concoct.
Anyone who is befuddled by how the other person {loves, hates} using LLMs to code should ask what kind of problem they are working on and then try to tackle the same problem with AI to get a better sense for their perspective.
Until then, every one of these threads will have dozens of messages saying variations of "you're just not using it right" and "I tried and it sucks", which at this point are just noise, not signal.
Have used Claude's GitHub action quite a bit now (10-20 issue implementations, a bit more PR reviews), and it is hit and miss so agree with the enhanced coding rather than just letting it run loose.
When the change is very small, self-contained feature/refactor it can mostly work alone, if you have tests that cover the feature then it is relatively safe (and you can do other stuff because it is running in an action, which is a big plus...write the issue and you are done, sometimes I have had Claude write the issue too).
When it gets to a more medium size, it will often produce something that will appear to work but actually doesn't. Maybe I don't have test coverage and it is my fault but it will do this the majority of the time. I have tried writing the issue myself, adding more info to claude.md, letting claude write the issue so it is a language it understands but nothing works, and it is quite frustrating because you spend time on the review and then see something wrong.
And anything bigger, unsurprisingly, it doesn't do well.
PR reviews are good for small/medium tasks too. Bar is lower here though, much is useless but it does catch things I have missed.
So, imo, still quite a way from being able to do things independently. For small tasks, I just get Claude to write the issue, and wait for the PR...that is great. For medium (which is most tasks), I don't need to do much actual coding, just directing Claude...but that means my productivity is still way up.
I did try Gemini but I found that when you let it off the leash and accept all edits, it would go wild. We have Copilot at work reviewing PRs, and it isn't so great. Maybe Gemini better on large codebases where, I assume, Claude will struggle.
The problem here is the infrastructure required to demo the changes to the user. Like yeah you made a code-change, but now I have to pull it, maybe setup data to get it in the right state, check if it's functioning how I want it to. Looking at the code it produced in a diff can waste a lot of your time if it doesn't even work as expected.
I currently use LLMs as a glorified Stack Overflow. If I want to start integrating an LLM like Gemini 2.5 PRO into my IDE (I use Visual Studio Code), whats the best way to do this? I don't want to use a platform like Cursor or Claude Code which takes me away from my IDE.
Lovely post @antirez. I like the idea that LLMs should be directly accessing my codebase and there should be no agents in between. Basically no software that filters what the LLM sees.
That said, are there tools that make going through a codebase easier for LLMs? I guess tools like Claude Code simply grep through the codebase and find out what Claude needs. Is that good enough or are there tools which keep a much more thorough view of the codebase?
I used a similar setup until a few weeks ago, but coding agents became good enough recently.
I don’t find context management and copy pasting fun, I will let GitHub Copilot Insiders or Claude Code do it. I’m still very much in the loop while doing vibe coding.
Of course it depends on the code base, and Redis may not benefit much from coding agents.
But I don’t think one should reject vibe coding at this stage, it can be useful when you know what the LLMs are doing.
I have found that if I ask the LLM to first _describe_ to me what it wants to do without writing any code, then the subsequent code generated has much higher quality. I will ask for a detailed description of the things it wants to do, give it some feedback and after a couple of iterations, tell it to go ahead and implement it.
IMO Claude code was a huge step up. We have a large and well structured python code base revolving mostly around large and complicated adapter pattern Claude is almost fully capable to implement a new adapter if given the right prompt/resources.
What is the overall feedback loop with LLMs writing code? Do they learn as they go like we do? Do they just learn from reading code on GitHub? If the latter, what happens as less and less code gets written by human experts? Do the LLMs then stagnate in their progress and start to degrade? Kind of like making analog copies of analog copies of analog copies?
72 comments
[ 2.7 ms ] story [ 68.8 ms ] threadI thought large contexts are not necessarily better and sometimes have opposite effect ?
I've been going down to sonnet for coding over opus. maybe i am just writing dumb code
This phrasing can be misleading and points to a broader misunderstanding about the nature of doctoral studies, which it has been influenced by the marketing and hype discourse surrounding AI labs.
The assertion that there is a defined "PhD-level knowledge" is pretty useless. The primary purpose of a PhD is not simply to acquire a vast amount of pre-existing knowledge, but rather to learn how to conduct research.
I find it very sad that people who have been really productive without "AI" now go out of their way to find small anecdotal evidence for "AI".
I agree with this, but this is why I use a CLI. You can pipe files instead of copying and pasting.
Whether it's vibe coding, agentic coding, or copy pasting from the web interface to your editor, it's still sad to see the normalization of private (i.e., paid) LLM models. I like the progress that LLMs introduce and I see them as a powerful tool, but I cannot understand how programmers (whether complete nobodies or popular figures) dont mind adding a strong dependency on a third party in order to keep programming. Programming used to be (and still is, to a large extent) an activity that can be done with open and free tools. I am afraid that in a few years, that will no longer be possible (as in most programmers will be so tied to a paid LLM, that not using them would be like not using an IDE or vim nowadays), since everyone is using private LLMs. The excuse "but you earn six figures, what' $200/month to you?" doesn't really capture the issue here.
I wish the stuff on top - deep research modes, multimodal, voice etc. had a more unified API as well but there's very little lock-in right now.
What I thought you were going to say is “Gemini- wha?”.
I’ve used Gemini 2.5 PRO and would definitely not use it for most of my coding tasks. Yes, it’s better at hard things, but it’s not great at normal things.
I’ve not used Claude 4 Opus yet- I know it’s great at large context- but Claude 4 Sonnet Thinking is mostly good unless the task is too complex, and Claude 4 Sonnet is good for basic operations in 1-2 files- beyond that it’s challenged and makes awful mistakes.
We can use cheap hardware that can be fixed with soldering irons and oscilloscopes.
People said for decades our projects just become weird DSLs. And now whatever little thing I want to do in any mainstream language involves learning some weird library DSL.
And now people be needing 24h GPU farm access to handle code.
In 50 years my grandkids that wish to will be able to build, repair and program computers with a garage workbench and old wrinkled books. I know most of the software economy will end up in the hands of major corporations capable of paying through the nose for black box low code solutions.
Doesn't matter. Knowledge will set you free if you know where to look.
Why?
If I want to pick up many hobbies, not to mention lines of professional work, I have to pay for tools. Why is programming any different? Why should it be?
Complaining that tools that improve your life cost money is... weird, IMO. What's the alternative? A world in which people gift you life-and-work-improving tools for free? For no reason? That doesn't exist.
> Programming used to be (and still is, to a large extent) an activity that can be done with open and free tools.
Btw, I think this was actually less true in the past. Compilers used to cost money in the 70s/80s. I think it's actually cyclical - most likely tools will cost money today, but then down the line, things will start getting cheaper again until they're free.
Code written will continue to run, you can use alternate LLM to further write code. Unless you are a pure vibe coder, you can still type the code. IF programmer stop learning how to write code, that's on them and not Claude's or antirez responsibility.
We are using best tool available to us, right now it is Claude Code, tomorrow it may be something from OpenAI, Meta or Deepseek.
I hope this will change before (captured) regulation strangles open models.
But that is my personal value judgement. And it doesn't mean other people will think the same. Luckily tech is a big space.
The models are somewhat getting there: even the smaller ones like Qwen3-30B-A3B and Devstral-23B are okay for some use cases and can run decently fast. They’re not amazing, but better than much larger models a year or two ago.
The hardware is absolutely not there: most development laptops will be too weak to run a bunch of tools, IDEs and local services alongside a LLM and will struggle to do everything at the pace of those cloud services.
Even if you seek compromise and get a pair of Nvidia L4 cards or something similar and put them on a server somewhere, the aforementioned Qwen3-30B-A3B will run at around 60 tokens/second for a single query but slow down as you throw a bunch of developers at it that all need chat and autocomplete. The smaller Devstral model will more than halve the performance at the starting point because it’s dense.
Tools like GitHub Copilot allow an Ollama connection pretty easily, Continue.dev also does but can be a bit buggy (their VS Code implementation is better than their JetBrains one), whereas the likes of RooCode only seem viable with cloud models cause they generate large system prompts and need more performance than you can squeeze out of somewhat modest hardware.
That said, with more MoE models and better training, things seem hopeful. Just look at the recent ERNIE-4.5 release, their model is a bit smaller than Qwen3 but has largely comparable benchmark results.
Those Intel Arc Pro B60 cards can’t come soon enough. Someone needs to at least provide a passable alternative to Nvidia, nothing more.
This was largely not the case before 2000, and may not be the case after, say 2030. It may well be that we are living in extraordinary times.
Before 2000 one had to buy expensive compilers and hardware to do serious programming. The rise of the commercial internet has made desktop computers cheaper, and the free software movement (again mostly by means of commercial companies enjoying the benefits) has made a lot of free software available.
However, we now have mostly smartphones, and serious security risks to deal with.
My general thinking is that we're using graphics processors which _work_ but aren't really designed for AI (lack of memory, etc.).
I am on board to agree that pure LLM + pure original full code as context is the best path at the moment, but I’d love to be able to use some shortcuts like quickly applying changes, checkpoints, etc.
My persistent (and not unfounded?) worry is that all the major tools & plugins (Cursor, Cline/Roo) all play games with their own sub-prompts and context “efficiency”.
What’s the purest solution?
For each task I always start with a new (empty) context and manually tag relevant files to include (this is trivial since I know the codebase well).
First I use Claude 4 Sonnet in thinking mode (I could also use Gemini 2.5 Pro or Opus as per antirez' recommendations) to come up with a detailed plan on how to implement something (research/planning phase). I provide feedback and we iterate on the plan.
Then, in the same conversation I switch to Sonnet 4 non-thinking and tell it to implement what we just devised.
I manually review the changes and trst them. If something needs fixing or (more often) if I notice I missed some edge case/caveat, I tell it to do that (still same convo).
Commit, clear convo, next task.
For research that isn't directly tied to the code, I use ChatGPT or Claude (web apps) to brainstorm ideas, and sometimes copy/pasre these into the editor agent as starting point.
IMHO those two variables are 10x (maybe 100x) more explanatory than any vibe coding setup one can concoct.
Anyone who is befuddled by how the other person {loves, hates} using LLMs to code should ask what kind of problem they are working on and then try to tackle the same problem with AI to get a better sense for their perspective.
Until then, every one of these threads will have dozens of messages saying variations of "you're just not using it right" and "I tried and it sucks", which at this point are just noise, not signal.
When the change is very small, self-contained feature/refactor it can mostly work alone, if you have tests that cover the feature then it is relatively safe (and you can do other stuff because it is running in an action, which is a big plus...write the issue and you are done, sometimes I have had Claude write the issue too).
When it gets to a more medium size, it will often produce something that will appear to work but actually doesn't. Maybe I don't have test coverage and it is my fault but it will do this the majority of the time. I have tried writing the issue myself, adding more info to claude.md, letting claude write the issue so it is a language it understands but nothing works, and it is quite frustrating because you spend time on the review and then see something wrong.
And anything bigger, unsurprisingly, it doesn't do well.
PR reviews are good for small/medium tasks too. Bar is lower here though, much is useless but it does catch things I have missed.
So, imo, still quite a way from being able to do things independently. For small tasks, I just get Claude to write the issue, and wait for the PR...that is great. For medium (which is most tasks), I don't need to do much actual coding, just directing Claude...but that means my productivity is still way up.
I did try Gemini but I found that when you let it off the leash and accept all edits, it would go wild. We have Copilot at work reviewing PRs, and it isn't so great. Maybe Gemini better on large codebases where, I assume, Claude will struggle.
https://cloud.google.com/gemini/docs/codeassist/write-code-g...
That said, are there tools that make going through a codebase easier for LLMs? I guess tools like Claude Code simply grep through the codebase and find out what Claude needs. Is that good enough or are there tools which keep a much more thorough view of the codebase?
I used a similar setup until a few weeks ago, but coding agents became good enough recently.
I don’t find context management and copy pasting fun, I will let GitHub Copilot Insiders or Claude Code do it. I’m still very much in the loop while doing vibe coding.
Of course it depends on the code base, and Redis may not benefit much from coding agents.
But I don’t think one should reject vibe coding at this stage, it can be useful when you know what the LLMs are doing.
antirez is a big fuggin deal on HN.
I’m sort of curious if the AI doubting set will show up in force or not.