HN Explorer

Ask HN: How do you go from writing code to deploying with agents?

17 points by sakuraiben ↗ HN
Been doing hardcode agent driven coding for the past year. I feel like I now have to test all my code locally with agents reviewing them and ideally testing them. Unfortunately I feel like the traditional CI/CD testing is starting to feel like a formality. Would love to know what everyone is doing, and if I'm not the only one making a mistake.

16 comments

[ 2.4 ms ] story [ 116 ms ] thread
mr1337
Testing is not a formality. In fact, it's better and easier than ever. Tell your agents to develop your code-base with Red-Green-Refactor Test-driven development. Write failing tests first and run them, then write the code to make the tests pass, then refactor the code as needed. I used to hate writing tests before, but now that I can offload that (and the development, honestly) to AI, it's nothing to keep up with it. Your codebase quality will increase with TDD, because it will reduce the number of things that inadvertently break during updates. Things will break loudly and can be easily fixed (again, by AI) before shipping. If something does manage to get through, add more tests to cover it for next time.

This actually makes the LLMs better at coding as well. They can verify their own results and iterate with far less manual validation.

mackatsol
More testing is better, so far! I'm using the same red-green-refactor TDD idea, and having Claude use Superpowers code review and running Coderabbit CLI on it all before pushing to CI/CD, which also runs a series of tests (mostly security ones). Someone else posted that whenever something broke they would ask 'Why was it built that way?' to try to have the bot fix it's own coding patterns.. as well as adding a regression test to ensure it did not happen again. I also have it do dry-runs and then do live on one item before batching. Catches a lot of things! Oh, and insist that it RTFM's the API if you're using one.
knighthacker
We still create a ton of end-to-end tests and run them a lot. What changed for us is the review process. While our whole team uses agents to do reviews, we also ask other team members to review the coding session itself rather than the diff. That team member can ask the coding session questions about the architecture decisions, scalability, edge cases, ..etc.

Disclosure: I'm building AQ (aq.dev), which is partly why I'm deep in this. We wrote up the session-review practice here: https://aq.dev/guides/how-to-review-an-ai-coding-session/. The practice works with any agents too, nothing tool-specific about it.

jmathai
I have Claude implementing features from a spec, running tests, creating a build, and then publishing to TestFlight. All I have to do is verify that the build works once I get the TestFlight notification.

I have found a key is to use end-to-end tests and not unit tests.

This has been working well so far after around 100 pull requests for an app I recently decided to make. So far it’s been good.

It’s free, no subscription.

[1] https://trysojourn.app

I’ve done similar with backend projects utilizing GitHub actions to run tests and publish to staging for verification.

alekstret
I'm building solo and am extensionally using Claude Code for development and testing. The following scheme finally works great for me.

First of all, the agent writes code. After that, it creates tests and verifies that all of them work correctly by cracking the checks and rerunning the test suite.

Secondly, it gives me the ready to test code + set up environment (stage). I'm checking that the code actually works, and if not, we are making some fixes until I'm fully satisfied with the result.

Thirdly, the agent starts an external review using skill for code review and usually makes some additional fixes to codebase and corrects tests again.

After all, we are ready to commit and push the feature to the main branch. Before merging, we start CI/CD - which is important because it does a type check - and wait until the run finishes successfully.

That's how it happens in my case.

Quick note: this plan was built after many iterations of coding and testing, and it has finally proved that it works.

kojeovo
do you read the code and test code? how do you know that the tests are any good? in my experience a lot of the time it will just mock the expected outcome and then it passes but it doesn't actually test behaviour.
ygjb
I use a combination of automated and manual testing. I spent a bunch of time setting up playwright testing at the start of my project, and then I shunt that through ZAP as well so that it can have a good understanding of how things work.

If gives me a pretty solid map of what is working well, and aids in driving through security testing.

Still not a replacement for manual testing. Us meatbags still have a knack for royally messing up the expected use cases :P

kojeovo
Sadly I can't rely on manual testing in my professional work although for personal projects it's how I do it too
alekstret
Most of the time it works as you said, but I implemented one rule that fixed these issues for me: my agent doesn't count a test until it has watched it fail.

The process works the following way: the agent writes the test, then it deletes the code it covers, reruns the test, confirms it goes red, and finally puts the code back.

Here is what made me start doing this. An e2e test for a touch gesture was green for weeks. I deleted the gesture handler, and the test stayed green, because Playwright's synthetic touch isn't actually a touch, so the test had never once exercised the thing it was named after. Reading it would never have shown me that.

It's discipline, not tooling. I don't run mutation testing; it's manual and only around whatever I just changed. And to be straight about your first question: it's the agent reading that test code, not me. Which is exactly why I don't rely on reading to catch this. Breaking the code and watching the test go red doesn't care who read it.

cyanydeez
super-dev.sh > super-test.sh > super-build.sh > super-e2e.sh > super-deploy.sh ? super-e2e2e.sh