Show HN: Haystack – Review the PRs that need human attention (haystackeditor.com)

45 points by akshaysg ↗ HN
Hey HN! We're building Haystack (https://haystackeditor.com/) to help teams deal with the explosion in the number of pull requests that need to be reviewed due to the rise of coding agents.

Haystack replaces the GitHub PR review system with a queue that triages each PR before a human has to read any diffs. It looks at the diffs, the codebase, and the coding-agent conversation that produced the PR. Haystack then routes it into one of three buckets:

1. Safe to merge. This means the PR has enough evidence behind it that the team can merge it without another human's review.

Some examples:

-- A small UI copy change that includes a screenshot showing the final state

-- A backend change where the author clearly tested the important paths and ran the changes in a real environment

2. Needs fixes. This means that the PR has bugs or violates a rule in your codebase and therefore the PR needs to be fixed by the author.

Some examples:

-- The agent was asked to make loading a large table faster by adding pagination, but the PR still loads every result at once and "implements" pagination in the UI

-- The PR silently catches an error instead of logging, surfacing, or handling it. This violates the team's "no silent error swallowing" rule

3. Needs human review. This means that the PR could not be sufficiently verified by the author or is touching a sensitive part of the codebase (determined by user-input guidelines) and thus requires human review.

Some examples:

-- The PR changes a significant amount of logic in billing

-- The PR changes an important user flow like onboarding, but the author only ran unit tests and never opened the app to check the flow end-to-end. That violates the team's rule that high-impact user-facing changes need manual verification.

Instead of starting with line-by-line diffs, Haystack immediately tells the reviewer the goal behind the PR, what design decisions the author made (informed by their coding-agent conversation), and how much the author did to verify that the pull request works (e.g. run scripts, checked the frontend, etc.).

In this way, review shifts from "what changed?" to "is this the right behavior and is there evidence that it works?".

Here's a quick demo: https://www.tella.tv/video/streamlining-code-reviews-with-ha...

We previously launched Haystack as a tool for understanding large PRs (https://news.ycombinator.com/item?id=45201703). As many of you can probably relate to, the release of Opus 4.5 completely shattered our conception of how fast an engineer could craft a PR.

And as coding agents got even better from 4.5, we realized that pull requests did not scale along with our coding velocity. With each member of our team being able to pump out more than 20 pull requests a day, code review quickly became cognitively exhausting and less helpful.

After talking with other folks, we learned many feel similarly, and currently face the binary option of either not doing review at all or trying to keep up with a fire hose of pull requests.

Haystack is our attempt at a third path. We still believe in code review, but as coding agents produce more code, human reviewer attention becomes more valuable and more expensive.

Haystack helps teams spend that attention on the PRs where a human can meaningfully change the outcome of that PR. And for such PRs, Haystack shows the reviewer what the PR intended to do, whether the author showed that it works, and what design decisions need a second pair of eyes.

We're still quite early and are figuring out whether Haystack truly makes code review better. We would love any and all feedback!

8 comments

[ 3.2 ms ] story [ 28.0 ms ] thread
I like this idea. To be blunt, would this have more features than hooking up Claude/Gemini/Codex and saying "If - at any point - you're unsure, step back and let a human review"
Man, it's such a shame that you pivoted away from the canvas-based editor concept, it was such a pleasure to use, it's so much better than tabs.

https://github.com/haystackeditor/haystack-editor

It's still probably the best tool to navigate, visualize and understand complex codebases, which is particularly important now with AI coded repos. I keep looking for alternatives but they are all notably worse.

About a month ago I spent a few frustrating hours building it from source for my system and making it work, and I've enjoyed using it as my main IDE since.

I wish I had the time to make a fork and bring in a newer version of VSCode. If anyone takes it up I might help at least.

Just to say great idea. But the name "Haystack" is used by several dozen things FWIW :)
Let AI review the code written by AI? interesting...
I think that AI coding is only going to grow in usage. I also think that AI more-or-less writes working code that is faithful to the user's prompt.

At the same time, AI does not write code that's easy for humans to review, as it writes very verbosely and with the goal of getting the job done vs producing readable code.

I really don't think humans should be reviewing the absolute mountains of LLM-produced code; doing so would only exhaust someone's cognitive budget.

Therefore, I think that bug finding should be left to AI to review, while architectural decisions should be reviewed by humans.

Yes, I think that having the code completely checked by AI is not a good idea, but an AI that says, "Check these," because they are noteworthy. This is my idea of a future where I hope AI is like the movie "Limitless." That is, it supports you in improving yourself and giving you greater capabilities, not in replacing you entirely.
Congratulations, this is a nice concept, this is actually usefull for large corporates with distibuted teams. The only issue is it is very hard these days to sell developer tools to enterprises, every tool any corporate is using these days are trying to inflate their offering with AI stuff. But if you can close them, definately this will be usefull. One more thing, your pricing is quite aggressive at 30/seat.