Show HN: Is grep enough? A transparent benchmark for agentic code navigation (entelligentsia.github.io)

3 points by bonigv ↗ HN
Felt LSP Servers were too complex. Bash tools alone too brutish. Wanted to see what if it is a tree-sitter as a firstclass tool. Ran a bench over 10 large codebases [bitcoin, django, rails, redis,...] at 5 levels of exploration complexity each. That 150 context isolated runs over the last few days. Sharing the results with full tarnsparency. All scripts, docker image scripts, all transcrpts. There is a TL;DR; but I hope you don't leave it at that. Has been quite a bit of work. Repo links are on the site.

3 comments

[ 2.7 ms ] story [ 19.4 ms ] thread
This is nicely put together, it does make sense that lsps help more as complexity grows because makes navigation across symbols easier.

I hope someone with a large budget can reproduce these with latest Opus/gpt.

My gut feeling is that higher reasoning models tend to use grep more effectively. But intuitively lsp should still win there.

(comment deleted)