Show HN: Is grep enough? A transparent benchmark for agentic code navigation (entelligentsia.github.io)
Felt LSP Servers were too complex. Bash tools alone too brutish. Wanted to see what if it is a tree-sitter as a firstclass tool. Ran a bench over 10 large codebases [bitcoin, django, rails, redis,...] at 5 levels of exploration complexity each. That 150 context isolated runs over the last few days. Sharing the results with full tarnsparency. All scripts, docker image scripts, all transcrpts. There is a TL;DR; but I hope you don't leave it at that. Has been quite a bit of work. Repo links are on the site.
3 comments
[ 2.7 ms ] story [ 19.4 ms ] threadI hope someone with a large budget can reproduce these with latest Opus/gpt.
My gut feeling is that higher reasoning models tend to use grep more effectively. But intuitively lsp should still win there.