HN Explorer

User-guide-driven development with coding agents

1 points by hosamsh ↗ HN
Since agents now own most, if not all of the coding process, I've been wondering what would be the best durable and reviewable source of truth for the human side to own.

One of the ideas I've been experimenting with is user-guide-driven development, where I write a user guide and mockups first, and let the agents iterate from there until they match the guide. I've almost stopped using unit tests and create user-experience instead, i.e. a context-less agent goes through the guide and tries to follow it literally before the work is considered done.

I've been using it in small utils, and feel more in control of the process. tbh though, I'm not sure how far this approach can scale. I'm curious where people think this breaks.

2 comments

[ 0.26 ms ] story [ 13.6 ms ] thread
gojkoa
the problem with user guides in general as a driver of specs (this is coming from surviving two such attempts 20 years ago) is that user guides tend to be written for an external audience, hiding a lot of complexity and underlying details. Sometimes this is to keep user docs focused on a specific task, sometimes this is because you don't want to disclose internal workings of a system to users).

Specs can influence (and potentially be even a source for generating parts of) user docs, but user docs can't replace specs.

On the other hand, the idea to write the guide, do mockups etc first is a great way to think through what the user would experience before you do the specs. We have something similar with agents creating demo pages so we can check them, and they become parts of the spec (and something to match the final product against).

But that's not a good fit for backend processes, cross-functional concerns (performance/security) etc, or things you do because your business needs it and not the users (payment fraud prevention) that also need to be specified

hosamsh
great points! The difference between using this system and a spec is exactly as you said, to capture the user outcome first and work backwards from there. I don't like specs in general because they are prone to become bloated with implementation details, and they never worked as a reliable source of truth in my experience, even prior to the AI generated code.

Your points on impl. complexity, backend processes and cross functional concerns make total sense. I'd try approaching these from an end experience point of view as well. For example, the operator guide could detail steps to run a benchmark tool and inspect performance metrics. I am not sure how much that can cover though, but the core idea is that end users, admins, operators, devs are all users of the system and their experience is documented first.