Launch HN: HyperProbe (YC S26) – Agents that do read-only debugging in prod (hyperprobe.co)
When prod breaks, it lets Cursor, Claude, and others drop virtual breakpoints or probes safely in your running code, and extract the exact variable values that logs don’t have.
All this saves time and effort for engineers who’d otherwise dig through logs and traces or redeploy with console.logs or print statements until they find the root cause.
Here is the link to the video that explains this: https://www.youtube.com/watch?v=ivV7I--ta5c
Agents write most of our code now. This shrinks the useful context engineers need to debug AI written code, a problem not helped by the limited telemetry added in the same code by AI.
So when something breaks in prod, the first instinct for an engineer is to open logs or throw them to your agents. But if the line you are looking for is not there, agents will start guessing the root cause on non-existent data, forcing you to add a log, and redeploy.
This analysis-inference loop of agents with existing data does not come cheap, burning a lot of tokens. And the add log, redeploy cycle is so slow and painful that it makes engineers hate on-call.
Our approach lets agents capture telemetry on-demand at the exact moment and point of failure, killing the log-redeploy cycle and getting the most accurate RCA while burning fewer tokens.
The obvious problem is making it work on a running service. You can't pause a live service the way you'd pause a debugger on your laptop. Getting the value out of a running process safely, without pausing a thread or slowing the host is the challenge.We are making this happen.
Before this I ran engineering at a 100 member team. Then Karan and I spent three years on HyperTest which was a testing tool.
At HyperTest, we turned production traffic into integration tests using OpenTelemetry. That was production instrumentation too. The hard parts of pulling real runtime state out of a running service without breaking it, were the hard parts we learnt to put together.
We learnt some other lessons the hard way too. HyperTest tried to prevent bugs with better tests, and adoption was a fight every time. Calls kept getting cancelled because teams were firefighting production. Testing was hygiene. Broken prod was hair on fire. This made us see where priorities lie.
This seeded the idea of building a truly autonomous on-call agent i.e. one that takes an alert, probes, diagnoses and fixes it in a few minutes. But this is how it works as of now:
You talk to your coding agent the way you already do. Tell it what's wrong: "checkout returns 200 but some users are seeing their order fail, find out why." It locates the line in your local code, connects to us over MCP, and drops a probe on that line in the running service. The probe is read-only and sits dormant until real traffic hits. When hit, it captures the local variables at every frame of the call stack at that exact moment. It hands them to the agent, which diagnoses with real data.
There are two pieces. An SDK that runs inside your service, and an MCP server your coding agent talks to. The SDK is what makes setting probes (virtual breakpoints, log or metric) possible without a redeploy. In Node and Python it hooks in-process. In Java it attaches as a JVM agent, instrumenting at the bytecode level. Either way the service keeps running and serving traffic. Nothing pauses.
When your agent wants to look at a line, it calls the MCP server, which tells the SDK to place a probe there. When a request hits the line, the SDK captures what the probe asked for, sanitizes it in-process, and streams it back to the agent via the MCP.
This can run inside prod, so a probe can read any value sitting in that variable. We ensure redaction happens in-process, inside your own container's memory. This is before anything goes on the wire. Keys like pass...
27 comments
[ 2.4 ms ] story [ 53.7 ms ] threadHow does it work? Using the NodeJs inspector API or other language equivalent to drop breakpoints? Those APIs are unavailable in many serverless environments and are challenging to use alongside bundlers.
If you don’t know how it broke, and you don’t know how you fixed it, what exactly is it you think you understand about your application?
For people that don't have these neat observability tools (like me), I've been using https://shellshare.net (disclaimer: I made it).
This is a single command to share a terminal live with e2e encryption. Originally it was for teaching classes or helping colleagues, but it's also very helpful for agents. I SSH into prod and run:
> npx shellshare exec --json -- tail /var/log/my-app.log
This generates a URL, then I can tell any agent:
> monitor <URL>, instructions in https://shellshare.net/llms.txt
They can see the output live. No need to install anything in the agent's machine. Next shellshare version it will be just "monitor <URL>" and the agent's instructions will be in the URL itself.
Nothing even near what you've guys done, but it has been helpful for me. Best of luck in your startup!
A useful adjunct to this kind of production in-memory debugging is a read-only agent locked down role in AWS or equivalent. I believe amazon has just set up some kind of a wizard for configuring a role like this. It really gives agents the ability to relatively safely look at the prod setup without exposing sensitive details or making changes, especially with e.g. terraform
Consider putting this near the beginning rather than 2/3 of the way down your pitch. I nearly stopped reading because these dramatic 1-2 sentence paragraphs are unpleasantly like listening to TV commercials. I think your target audience should not be CTOs or their direct reports, but engineers themselves, and I think you need a more focused pitch that takes less time to get to the point.
Anyway, an MCP-managed passive debugger seems like a useful tool. Best of luck with it.
> Every log-and-trace tool hands the agent data that already exists and asks it to reason backward to what probably happened
If the app is using a decent instrumentation tool, the data shows what 'actually' happened, not what 'probably' happened.
> "checkout returns 200 but some users are seeing their order fail, find out why."
Does this tool only exist to shore up poor system design? Failing orders at any e-commerce business I've worked with, large and small, are a huge red flag. Typically that is one of the first actions that is logged and traced (alongside onboarding/login), and the metrics are actively monitored. Returning 200 for failure and not catching that error is very bad API design.
Similarly, putting engineers in a situation where debugging requires accessing unknown amounts of live sensitive customer data is generally considered bad practice (even if it happens often IRL) -- in a hurry to debug, it's easy to miss that a property should have been redacted; by then it's too late and sensitive data is exposed. Plus, in most systems with significant usage the volume of trace data is prohibitive to individually examine and search through. That's why Rollbar etc aggregate errors and captured data to identify patterns before a human (or agent, or tool) ever takes a look at it. A single captured instance can also be very misleading as to the true cause.
How are you addressing these common concerns?
Running an autonomous pipeline for eight months, the three incidents that cost me the most days all had the surface error naming the wrong subsystem:
- "x264: malloc of size N failed / incorrect parameters" — I read it as a codec or bad-args bug and went looking there. It was RAM exhaustion. The encoder was the victim, not the cause.
- A 22x slowdown in an LLM step that was indistinguishable from a hang. It was swap: the model no longer fit in RAM, and the page file did the rest.
- A 27-minute "freeze" in a background job. The process was healthy; the pipe was buffering, so nothing appeared until exit.
In all three the logs were complete and the metrics were green. The mistake was in the inference drawn from them — and an agent will produce that wrong inference far faster than I did, with better prose attached to it.
So: does HyperProbe ever return "I don't know — here are two competing hypotheses and the cheapest check that separates them"? The discriminating check is the part I'd pay for. A single confident answer that's wrong is worse than no answer, because it sends a human down a road with the agent's credibility behind it.
When I looked into this a while back I explored using ptrace() to add breakpoints and even add functions at specific line numbers. But ptrace is so slow, and it doesn't work with bytecode-in-VM setups.
What were some of the requirements you guys had when building HyperProbe? I can see low latency was one.
we work at the application layer by hooking into production grade tooling if available (inspector in v8, sys.monitoring in python) or bytecode manipulation(jvm)
since we arent controlling the application for a different process, we dont need to freeze the app to get the current app state
requirements we had in mind in order of importance
1. safety -> user app needs to function as usual no matter what happens, there shouldnt be an error in the user's app because of us
2. zero idle footprint -> if no probe is active, cpu/memory differency in the user app should be immeasurable
3. zero latency footprint at non probe paths while other probes are active
4. measure mem/cpu footprint directly or via a proxy like eventloop lag and have guardrails around it. suspend probes or even lose snapshot data if guardrail conditions meet
5. minimal mem/cpu footprint for active probes
6. minimal latency foot print for active probe paths
1. What makes "read-only" a guarantee rather than a convention? In Python a plain attribute read can hit a @property that lazy-loads from the DB; in Java a getter can mutate state or take a lock. If the capture expression permits attribute access at all, read-only becomes a property of the code being probed rather than of your SDK. Do you restrict the expression grammar, or is it best-effort?
2. What's the shape of what comes back through MCP? A captured frame can serialize into something enormous, and an agent will cheerfully spend its entire context on one request object. Can you project at capture time (user.id rather than user), or does trimming happen after the full payload is already built?