HN Explorer

Codex's precision and attention to detail is *crazy* when set up correctly

5 points by ditchfieldcaleb ↗ HN
Lately I've been working on a Tower Defense game with Codex, in part to learn how game development works and in part to see how far I can get using just Codex, no manual coding at all. I've got my AGENTS md & my CODESTYLE md & six other ALLCAPS md files etc, and am working on some refactoring to keep the codebase clean & file sizes low, etc.

And then I see this in the ExecPlan for my latest refactor:

---

# Observations

- Observation: The refactor made the screenshots pixel-identical after the baseline was recaptured correctly.

Evidence: sha256sum screenshots/before-implementation-x.png screenshots/after-implementation-x.png reported matching hashes for before/after pairs 1, 2, and 3.

---

Which is crazy! I've never told Codex to do an sha compare on before/after screenshots of the app, but I do have instructions in my PLANS.md to take before & after screenshots of the webapp for the game to make sure we avoid frontend regressions (it uses GPT-Image-2 for analysis). So for non-frontend impacting changes, of course nothing should be different between screenshots taken at identical timestamps into the game start.

But doing an explicit SHA compare - that's just...not something I would've ever thought of. Wild.

3 comments

[ 3.2 ms ] story [ 15.1 ms ] thread
[deleted]
(comment deleted)
TacticalCoder
> But doing an explicit SHA compare - that's just...not something I would've ever thought of. Wild.

If I'm not mistaken SGI (Silicon Graphics, Inc.) was already doing that to prevent regression 40 years ago: maybe not SHA but they were taking "screenshots" of the entire screen at a time t and some kind of checksum to then verify (without having to compare every single pixel in the happy case) that enhancement/optimization to their rendering pipeline not supposed to change the output indeed did indeed generate the exact same image as before.

It's basically a 40 years old technique: not too sure what's that wild about it.

daemonologist
I theorize that this kind of thing - verification approaches that do already exist but which a human would not usually reach for - is a result of RLVR. I've noticed Claude Opus a couple of times trying to use `nm` to check that its changes made it into the binary (and often tying itself into knots in the process), and have stuffed some extra coaxings along the lines of "don't try to be clever" into its prompt.