Rashomon is an independent execution recorder for coding agents that notifies users whenever there is a discrepancy between what the agent said it did and what we capture as a ground truth.
My team and I were working on a different project and we faced a problem with being able to verify what an agent did, rather than relying on its own transcript.
It aims to capture items like shell commands, exit codes, file writes, tool calls, tests, and subagents. We build a timeline of the events but we do not store any prompts, responses, file contents, or tool outputs.
To get started, you can install it is as a Claude Code plugin or from the command line.
I would appreciate feedback on any edge cases as well as what execution items people want captured that we do not do already.
A couple of months ago, I gave access to my coding agent to do work on my repos. I was worried about hallucinations so I manually checked each transcript against modifications to see whether what the ge t said it did actually happened. I found that in many cases, my coding agent either failed to disclose what it actually did or found workarounds to things like failed tests instead of telling me upfront. That led me to the idea for Rashomon.
Rashomon is an independent execution recorder for coding agents that notifies users whenever there is a discrepancy between what the agent said it did and what we capture as a ground truth.
It aims to capture items like shell commands, exit codes, file writes, tool calls, tests, and subagents. We build a timeline of the events but we do not store any prompts, responses, file contents, or tool outputs.
Currently, we do not capture network codes, prompts, responses file contents, and tool outputs. We plan to implement some of those based on feedback recently, but please let us know whether you would find that usefull.
To get started, you can install it is as a Claude Code plugin or from the command line.
I would appreciate feedback on the solution, but I also would want to see whether anyone can break a test on purpose and see whether Rashomon can capture everything you would like to see in order to trust the independent record.
4 comments
[ 1.1 ms ] story [ 36.2 ms ] threadMy team and I were working on a different project and we faced a problem with being able to verify what an agent did, rather than relying on its own transcript.
It aims to capture items like shell commands, exit codes, file writes, tool calls, tests, and subagents. We build a timeline of the events but we do not store any prompts, responses, file contents, or tool outputs.
To get started, you can install it is as a Claude Code plugin or from the command line.
I would appreciate feedback on any edge cases as well as what execution items people want captured that we do not do already.
Rashomon is an independent execution recorder for coding agents that notifies users whenever there is a discrepancy between what the agent said it did and what we capture as a ground truth.
It aims to capture items like shell commands, exit codes, file writes, tool calls, tests, and subagents. We build a timeline of the events but we do not store any prompts, responses, file contents, or tool outputs.
Currently, we do not capture network codes, prompts, responses file contents, and tool outputs. We plan to implement some of those based on feedback recently, but please let us know whether you would find that usefull.
To get started, you can install it is as a Claude Code plugin or from the command line.
I would appreciate feedback on the solution, but I also would want to see whether anyone can break a test on purpose and see whether Rashomon can capture everything you would like to see in order to trust the independent record.