The entire site looks like it was generated with Claude.
The site hijacks scrolling.
I scanned through paragraph after paragraph of overbloated and repetitive descriptions of what and how RAG works compared to how "modern agents" need to access documents.
What I never saw at any point was whatever this "alternative to RAG" was supposed to be.
Honestly the entire body of copy feels like it was llm generated.
I read the first paragraph or two, but then I stopped for two reasons:
A) searching using BM25 and via vector embeddings are two different methods of search; both have their uses, many practical use cases benefit from having both. The search method is independent of who calls it when, i.e. whether you just have a static pipeline, or a dynamic one where the search tool calls them.
B) you're describing the static pipeline as if it were SOTA. We dropped that well over a year ago IIRC. There might still be cases where that is good enough, but in general, agentic search has pretty much become the default since LLMs became reasonably good at tool calling.
Firstly, I think it would be useful for each document to first be neatly converted to markdown, including detailed Latex/Mermaid/captions for images. This would be at the time of insertion into the store.
Secondly, I think vectoring should never apply over simple chunks. A document's markdown should be separated into distinct subtopical assortments by a reasoning LLM, with each subtopic having a subtitle and text. Each such subtopic must then be added to the BM25 and vector stores as a row. Speaking of which, don't overlook BM25 search co-integration besides vector search.
Thirdly, I think agentic RAG should apply only as an optional final step to refine what was generated using BM25+vector search.
11 comments
[ 0.70 ms ] story [ 39.3 ms ] threadI scanned through paragraph after paragraph of overbloated and repetitive descriptions of what and how RAG works compared to how "modern agents" need to access documents.
What I never saw at any point was whatever this "alternative to RAG" was supposed to be.
Honestly the entire body of copy feels like it was llm generated.
A) searching using BM25 and via vector embeddings are two different methods of search; both have their uses, many practical use cases benefit from having both. The search method is independent of who calls it when, i.e. whether you just have a static pipeline, or a dynamic one where the search tool calls them.
B) you're describing the static pipeline as if it were SOTA. We dropped that well over a year ago IIRC. There might still be cases where that is good enough, but in general, agentic search has pretty much become the default since LLMs became reasonably good at tool calling.
Secondly, I think vectoring should never apply over simple chunks. A document's markdown should be separated into distinct subtopical assortments by a reasoning LLM, with each subtopic having a subtitle and text. Each such subtopic must then be added to the BM25 and vector stores as a row. Speaking of which, don't overlook BM25 search co-integration besides vector search.
Thirdly, I think agentic RAG should apply only as an optional final step to refine what was generated using BM25+vector search.