Show HN: A MitM proxy to see what your LLM tools are sending (github.com)
I built this out of curiosity about what Claude Code was actually sending to the API. Turns out, watching your tokens tick up in real-time is oddly satisfying.
Sherlock sits between your LLM tools and the API, showing you every request with a live dashboard, and auto-saved copies of every prompt as markdown and json.
39 comments
[ 6.3 ms ] story [ 84.2 ms ] threadI'm surprised that there isn't a stronger demand for enterprise-wide tools like this. Yes, there are a few solutions, but when you contrast the new standard of "give everyone at the company agentic AI capabilities" with the prior paradigm of strong data governance (at least at larger orgs), it's a stark difference.
I think we're not far from the pendulum swinging back a bit. Not just because AI can't be used for everything, but because the governance on widespread AI use (without severely limiting what tools can actually do) is a difficult and ongoing problem.
When I work with AI on large, tricky code bases I try to do a collaboration where it hands off things to me that may result in large number of tokens (excess tool calls, unprecise searches, verbose output, reading large files without a range specified, etc.).
This will help narrow down exactly which to still handle manually to best keep within token budgets.
Note: "yourusername" in install git clone instructions should be replaced.
[0]https://news.ycombinator.com/item?id=46782091
If you build a DIY proxy you can also mess with the prompt on the wire. Cut out portions of the system prompt etc. Or redirect it to a different endpoint based on specific conditions etc.
I know and trust mitmproxy. I'm warier and less likely to use a new, unknown tool that has such broad security/privacy implications. Especially these days with so many vibe-coded projects being released (no idea if that's the case here, but it's a concern I have nonetheless).
build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/build/lib/sherlock
https://github.com/quilrai/LLMWatcher
here is my take on the same thing, but as a mac app and using BASE_URL for intercepting codex, claude code and hooks for cursor.
It shells out to mitmproxy with "--set", "ssl_insecure=true"
This took all of 5 minutes to find reading through main.py on my phone.
https://github.com/jmuncor/sherlock/blob/fb76605fabbda351828...
Edit: In case it’s not clear, you should not use this.
https://imgur.com/a/Ztyw5x5
E.g. if a request contains confidential information (whatever you define that to be), then block it?
https://news.ycombinator.com/item?id=46820977