Weird to see the use case referenced specifically code search when that's a very targeted one rather than what general purpose agents (or RAG) use cases might target.
grep was invented at a time when computers had very small amounts of memory, so small that you might not even be able to load a full text file. So you had tools that would edit one line at a time, or search through a text file one line at a time.
LLMs have a similar issue with their context windows. Go back to GPT-2 and you wouldn't have been able to load a text file into its memory. Slowly the memory is increasing, same as it did for the early computers.
I don't find this surprising. We are constantly finding workarounds for technical limitations, then ditch them when the limitation no longer exists. We will probably be saying the same thing for LLMs in a few years (when a new machine learning related TLA becomes the hype).
>The winners will not be the ones who maintain the biggest vector databases, but the ones who design the smartest agents to traverse abundant context and connect meaning across documents.
So if one were building say a memory system for an AI chat bot, how would you save all the data related to a user? Mother's name, favorite meals, allergies? If not a Vector database like pinecone, then what? Just a big .txt file per user?
This glosses over a fundamental scaling problem that undermines the entire argument. The author's main example is Claude Code searching through local codebases with grep and ripgrep, then extrapolates this to claim RAG is dead for all document retrieval. That's a massive logical leap.
Grep works great when you have thousands of files on a local filesystem that you can scan in milliseconds. But most enterprise RAG use cases involve millions of documents across distributed systems. Even with 2M token context windows, you can't fit an entire enterprise knowledge base into context. The author acknowledges this briefly ("might still use hybrid search") but then continues arguing RAG is obsolete.
The bigger issue is semantic understanding. Grep does exact keyword matching. If a user searches for "revenue growth drivers" and the document discusses "factors contributing to increased sales," grep returns nothing. This is the vocabulary mismatch problem that embeddings actually solve. The author spent half the article complaining about RAG's limitations with this exact scenario (his $5.1B litigation example), then proposes grep as the solution, which would perform even worse.
Also, the claim that "agentic search" replaces RAG is misleading. Recent research shows agentic RAG systems embed agents INTO the RAG pipeline to improve retrieval, they don't replace chunking and embeddings. LlamaIndex's "agentic retrieval" still uses vector databases and hybrid search, just with smarter routing.
Context windows are impressive, but they're not magic. The article reads like someone who solved a specific problem (code search) and declared victory over a much broader domain.
Cursor’s use of grep is bad. It finds definitions way slower and less accurately than I do using IDE indexing, which is frustratingly “right there.” Crazy that there’s not even LSP support in there.
>The author spent half the article complaining about RAG's limitations with this exact scenario (his $5.1B litigation example), then proposes grep as the solution, which would perform even worse.
Yeah I found this very confusing. Sad to see such a poor quality article being promoted to this extent.
Code is also unique in its suitability for agentic grep retrieval, especially when combined with a language server. Code enforces structure, semantics, and consistency in a way that is much easier to navigate than the complexities of natural language.
We're processing tenders for the construction industry - this comes with a 'free' bucket sort from the start, namely that people practically always operate only on a single tender.
Still, that single tender can be on the order of a billion tokens. Even if the LLM supported that insane context window, it's roughly 4GB that need to be moved and with current LLM prices, inference would be thousands of dollars. I detailed this a bit more at https://www.tenderstrike.com/en/blog/billion-token-tender-ra...
And that's just one (though granted, a very large) tender.
For the corpus of a larger company, you'd probably be looking at trillions of tokens.
While I agree that delivering tiny, chopped up parts of context to the LLM might not be a good strategy anymore, sending thousands of ultimately irrelevant pages isn't either, and embeddings definitely give you a much superior search experience compared to (only) classic BM25 text search.
Agentic search with a handful of basic tools (drawn from BM25, semantic search, tags, SQL, knowledge graph, and a handful of custom retrieval functions) blows the lid off RAG in my experience.
The downside is it takes longer. A single “investigation” can easily use 20-30 different function calls. RAG is like a static one-shot version of this and while the results are inferior the process is also a lot faster.
This is a great example of a piece with enough meaningful and useful content in it that it's very clear the author had something of value to deliver, and I'm grateful for that... but enough repetitive LLM-output that I'm very annoyed by the end.
Actually, let me be specific: everything from "The Rise of Retrieval-Augmented Generation" up to "The Fundamental Limitations of RAG for Complex Documents" is good and fine as given, then from "The Emergence of Agentic Search - A New Paradigm" to "The Claude Code Insight: Why Context Changes Everything" (okay, so the tone of these generated headings is cringey but not entirely beyond the pale) is also workable. Everything else should have been cut. The last four paragraphs are embarrassing and I really want to caution non-native English speakers: you may not intuitively pick up on the associations that your reader has built with this loudly LLM prose style, but they're closer to quotidian versions of the [NYT] delusion reporting than you likely mean to associate with your ideas.
That's quite the over-generalization. RAG fundamentally is: topic -> search -> context -> output. Agents can enhance it by iterating in a loop, but what's inside the loop is not going away.
This reads like someone AI-generated prose to defend something they want to invest in and decry something it competes with. It does not come off as honest, written by a human, or useful to anyone outside of the specific, narrow contexts the "author" sees for the technologies mentioned.
Frankly, reading through this at makes me feel as though I am a business analyst or engineering manager being presented with a project proposal from someone very worried that a competing proposal will take away their chance to shine.
As it reaches the end, I feel like I'm reading the same thing, but presented to a Buzzfeed reader.
Saying that RAG alone is complex and should be superseded by agentic search is a bit weak. Agentic search makes more sense when your pipeline becomes more complicated: RAG+MCP+Client calls, it is then when you can see that LLM starts behaving erratically and cannot answer the question well. You then want better control over streams of content and intents which could be solved by smaller agents looping over the data.
After building a few RAG based apps I was curious to try the ClaudeCode based approach that is mentioned by the author. So I built a python service that exposes ripgrep to a rest api: https://github.com/masterkram/jaguar
This makes it possible to quickly deploy this on coolify and quickly build an agent that can use ripgrep on any of your uploaded files.
These types of articles regularly come from people who don't actually build SCALE systems with LLMs. Or, people who want to sell you on a new tech.
And the frustrating thing is: They ain't even wrong.
Top-K RAG via vector search is not a sufficient solution. It never really was for most interesting use-cases.
Of course, take easiest and most structured - in a sense perfectly indexed - data (code repos) and claim that "RAG is dead". Again.
Now try this with billions of unstructured tokens where the LLM really needs to do something with the entire context (like, confirm that something is NOT in the documents), where even the best LLM loses context coherence after like 64k tokens for complex tasks.
Good luck!
The truth is: Whether its Agentic RAG, Graph RAG, or a combination of these with ye olde top-k RAG - it's still RAG.
You are going to Retrieve, and then you are going to use a system of LLM agents to generate stuff with it. You may now be able to do the first step smarter. It's still Rag tho.
The latest Antrophic whoopsy showed that they also haven't solved the context rot issue. Yes you can get a 1M context scaled version of Claude, but then the small/detail scale performance is so garbage that misrouted customers loose their effin mind.
"My LLM is just gonna ripgrep through millions of technical doc pdfs identified only via undecipherable number-based filenames and inconsistent folder structures"
I agree. Permit me to rephrase. From this learning adventure https://www.infoq.com/articles/architecting-rag-pipeline/ I came to understand what many now call context rot. If you want quality answers, you still need relevance reranking and filtering no matter how big your context window becomes. Whether that happens in a search that is upfront in a one shot prompt or iteratively in a long session through an agentic system is merely an implementation detail.
People say “agents not RAG”, but one framing is that this describes RAG where the database is a file system and bash is the query language (with other cli tools installed it can use, including curl, jq, grep). With writing its own notes on the filesystem structure and maintaining them as a way to “index the database”
It is still using code to selectively grab the chunks of data it needs rather than than putting everything in context. It’s just better RAG?
49 comments
[ 5.9 ms ] story [ 79.9 ms ] threadIt's pretty much the same process I would use in an unfamiliar code base. Just ctrl+f the file system till I find the right starting point.
These corpora have a high degree of semantic ambiguity among other tricky and difficult to alleviate issues.
Other types of text are far more amenable to RAG and some are large enough that RAG will probably be the best approach for a good while.
For example: maintenance manuals and regulation compendiums.
LLMs have a similar issue with their context windows. Go back to GPT-2 and you wouldn't have been able to load a text file into its memory. Slowly the memory is increasing, same as it did for the early computers.
So if one were building say a memory system for an AI chat bot, how would you save all the data related to a user? Mother's name, favorite meals, allergies? If not a Vector database like pinecone, then what? Just a big .txt file per user?
Grep works great when you have thousands of files on a local filesystem that you can scan in milliseconds. But most enterprise RAG use cases involve millions of documents across distributed systems. Even with 2M token context windows, you can't fit an entire enterprise knowledge base into context. The author acknowledges this briefly ("might still use hybrid search") but then continues arguing RAG is obsolete.
The bigger issue is semantic understanding. Grep does exact keyword matching. If a user searches for "revenue growth drivers" and the document discusses "factors contributing to increased sales," grep returns nothing. This is the vocabulary mismatch problem that embeddings actually solve. The author spent half the article complaining about RAG's limitations with this exact scenario (his $5.1B litigation example), then proposes grep as the solution, which would perform even worse.
Also, the claim that "agentic search" replaces RAG is misleading. Recent research shows agentic RAG systems embed agents INTO the RAG pipeline to improve retrieval, they don't replace chunking and embeddings. LlamaIndex's "agentic retrieval" still uses vector databases and hybrid search, just with smarter routing.
Context windows are impressive, but they're not magic. The article reads like someone who solved a specific problem (code search) and declared victory over a much broader domain.
Claude Code is better, but still frustrating.
Yeah I found this very confusing. Sad to see such a poor quality article being promoted to this extent.
Still, that single tender can be on the order of a billion tokens. Even if the LLM supported that insane context window, it's roughly 4GB that need to be moved and with current LLM prices, inference would be thousands of dollars. I detailed this a bit more at https://www.tenderstrike.com/en/blog/billion-token-tender-ra...
And that's just one (though granted, a very large) tender.
For the corpus of a larger company, you'd probably be looking at trillions of tokens.
While I agree that delivering tiny, chopped up parts of context to the LLM might not be a good strategy anymore, sending thousands of ultimately irrelevant pages isn't either, and embeddings definitely give you a much superior search experience compared to (only) classic BM25 text search.
Actually, let me be specific: everything from "The Rise of Retrieval-Augmented Generation" up to "The Fundamental Limitations of RAG for Complex Documents" is good and fine as given, then from "The Emergence of Agentic Search - A New Paradigm" to "The Claude Code Insight: Why Context Changes Everything" (okay, so the tone of these generated headings is cringey but not entirely beyond the pale) is also workable. Everything else should have been cut. The last four paragraphs are embarrassing and I really want to caution non-native English speakers: you may not intuitively pick up on the associations that your reader has built with this loudly LLM prose style, but they're closer to quotidian versions of the [NYT] delusion reporting than you likely mean to associate with your ideas.
[NYT]: https://www.nytimes.com/2025/08/08/technology/ai-chatbots-de...
Frankly, reading through this at makes me feel as though I am a business analyst or engineering manager being presented with a project proposal from someone very worried that a competing proposal will take away their chance to shine.
As it reaches the end, I feel like I'm reading the same thing, but presented to a Buzzfeed reader.
This makes it possible to quickly deploy this on coolify and quickly build an agent that can use ripgrep on any of your uploaded files.
These types of articles regularly come from people who don't actually build SCALE systems with LLMs. Or, people who want to sell you on a new tech. And the frustrating thing is: They ain't even wrong.
Top-K RAG via vector search is not a sufficient solution. It never really was for most interesting use-cases.
Of course, take easiest and most structured - in a sense perfectly indexed - data (code repos) and claim that "RAG is dead". Again. Now try this with billions of unstructured tokens where the LLM really needs to do something with the entire context (like, confirm that something is NOT in the documents), where even the best LLM loses context coherence after like 64k tokens for complex tasks. Good luck!
The truth is: Whether its Agentic RAG, Graph RAG, or a combination of these with ye olde top-k RAG - it's still RAG. You are going to Retrieve, and then you are going to use a system of LLM agents to generate stuff with it. You may now be able to do the first step smarter. It's still Rag tho.
The latest Antrophic whoopsy showed that they also haven't solved the context rot issue. Yes you can get a 1M context scaled version of Claude, but then the small/detail scale performance is so garbage that misrouted customers loose their effin mind.
"My LLM is just gonna ripgrep through millions of technical doc pdfs identified only via undecipherable number-based filenames and inconsistent folder structures"
lol, and also, lmao