HN Explorer

Show HN: Cost.dev (YC W21) – making agents cost-aware and cheaper to call (cost.dev)

48 points by akh ↗ HN
We launched Infracost on HN five years ago (https://news.ycombinator.com/item?id=26064588) where our CLI generated cost estimates for infra-as-code, e.g. "this Terraform PR adds $400/mo". The idea was to shift cloud costs (FinOps) left, so engineers get visibility of costs before deployment and make better decisions.

Earlier this year we started seeing agent traffic in our logs and it looked like coding agents were calling our CLI. But that CLI wasn't designed with coding agents in mind. We went down a philosophical rabbit hole to see if a CLI is even needed anymore given that Claude, Copilot et al. already follow best practices. Ultimately we decided to create a new CLI from the ground up with coding agents in mind for two reasons:

1. We optimized the CLI for agent callers and cut Claude's output token usage by up to 79% and API cost by up to 67% versus a bare-Claude baseline. We wrote a blog documenting our lessons on optimizing user token usage when designing a CLI, e.g. using predicate flags so the agent doesn't compose jq | python | wc pipelines, output format that strips JSON's redundant field names. The blog is here: https://www.infracost.io/resources/blog/we-cut-claude-s-toke...

2. With cloud costs, precision matters. Telling a coding agent "make this Terraform cost-optimized" can be expensive and lossy. You burn tokens loading code and policy context into every conversation. Your agent could make up a price and you wouldn't know because it's difficult to verify that across the ~10M price points that AWS, Azure and Google have. The CLI runs static analysis on the code, uses the latest prices from cloud vendors, and passes that context to the coding agent.

So that's what we're launching today - Cost.dev: https://cost.dev/.

- It runs locally. Your code never leaves your machine, you get a fast feedback loop, and you're not burning API calls per character when you want to fetch prices.

- The CLI does the deterministic work. Fetching price points, scanning the code, validating fixes. The coding agent does the natural-language part. You don't have to trust the LLM to remember the rules, and can verify it called the right CLI command.

- It provides a consistent rule layer across every tool you use. Get cost estimates in your IDE and your coding agent with a single install. We support Claude Code, GitHub Copilot, Cursor, Windsurf, OpenAI Codex, Gemini CLI, as well as IDEs like VS Code and JetBrains

Before we keep building more in that direction, I want to sanity-check with HN: is "agents writing IaC in prod" actually a thing yet, or am I betting on a future that's still a year out? I know software developers are using coding agents heavily, but are platform/infra folks doing that for prod too? Also, if you have any feedback on Cost.dev, I'd love to hear it!

26 comments

[ 4.3 ms ] story [ 44.7 ms ] thread
eugeneonai
The 79% / 67% reduction generalizes broader than IaC. Any CLI agents shell out to (curl, jq, grep, kubectl, gh, psql) burns the same token tax — verbose JSON, free-form text output, agent-composed pipelines. A predicate-flag + compact-output redesign would land on all of those.

  Direct answer to your question: agents-writing-IaC-in-prod is rare today but not zero. I see more "agent reviews the IaC PR a human wrote", which Cost.dev sounds well-suited to since verification runs locally and the agent only consumes the result. Even if the prod-IaC path takes another year, the design pattern earns its keep on every agent-shellout you already do. One question: does the CLI surface its cache state to the agent, or does each invocation start fresh Repeated price-fetches across a single agent run would be the obvious next-tier savings.
glenngillen
We do cache the results locally so that we're not repeatedly hitting our pricing API. The LLM doesn't access that cache directly though as it'd suffer the token tax you mention. Instead we optimised our CLI to return agent optimised results. We're constantly iterating and improving on it, but it already reduces the tokens usage very significantly. I wrote about it here: https://www.infracost.io/resources/blog/we-cut-claude-s-toke...

We've found even more improvements since that post so those will be shipping soon too.

zuzululu
Not really seeing the point I just use openrouter if I'm penny pinching
5701652400
I don't know how they can justify 250 USD / month bill. let alone 1000 USD / month.
5701652400
why would anyone need 10,000 runs a month? do people modify their infrastructure 10,000 times a month?
zane_shu
The useful split here seems to be: let the CLI do price lookup and validation, and let the agent decide which diff to make. The thing I’d watch is how visible the source of the estimate is in review — if a PR says “saved $X”, reviewers need to see which prices/rules produced that number.
jing09928
The interesting bit is making cloud cost a first-class constraint for the agent loop, not just a post-hoc report. I'd be curious how you handle confidence/uncertainty in estimates, since a wrong cheap-looking recommendation can be worse than no estimate in infra PRs.
lavaman131
This is an interesting approach. I think people still underestimate how quickly token limits and context bloat become the bottleneck when you start running agents in production loops. Seems very relevant topic also given the environment. Will check it out
akh
Thanks - indeed, it's also difficult to estimate how much the agents will cost in advance, and how much changes could reduce or increase costs.
photonair
Tokens can get expended very quickly driving costs up substantially since there is no clarity in its usage If your service can cut the use substantially, for a large company with a huge AI bill, the savings could make up for the cost of the service which I think is pricey in my opinion. But I can see how it could work out.
tuo-lei
how are you handling errors? when an agent gets a flag wrong, cli help text is usually massive. could eat a lot of the savings on retries.
glenngillen
It's been a lot of trial & error. A quick aside: running these tests/evals/call them what you will at scale has been fascinating to me. Going back and trawling through the logs has been like speed-running through hundreds of usability tests with people, full of the same types of "aha! Of course you'd try and do that, why didn't I think of that already?" moments of insight and inspiration.

Which is also how we've gone about working out how to improve the CLI. It's usually one or more of:

* rethinking the subcommands and hierarchy to something more obvious and aligned to the task

* providing clear documentation upfront (i.e, in the skills file)

* keeping help text concise, but not too concise. You can't assume the reader is already a power user and it's simply looking for a reminder/reference. So include usage examples for common use cases

* where possible on errors, suggest the likely commands the person meant.

* In general offer affordances on what likely next steps will be. This goes for help output, success, and errors.

> cli help text is usually massive

That doesn't have to be true.

> could eat a lot of the savings on retries

This doesn't have to be true either. You don't need to give the same full help output on every single error, once they've got it once they've got it. Also the size of the entire help output for most CLIs is generally insignificant compared to even just a couple of source files in most repos.

yohji1984
I'm wondering why all these token-saving solutions focus their benchmarks exclusively on simple Q&A tasks. If their tools truly saved money in real, long-term programming tasks, they would have definitely published those benchmark results instead of just Q&A tests, especially since a simple code editing benchmark with a hidden eval harness is very easy to design. Personally, asking a coding agent questions without any code editing is a very rare case for me
glenngillen
I did exactly that and it's all covered in the blog post. There's no hidden eval harness, it's in the same codebase as the CLI so others can reproduce and/or extend as they see fit. It also includes code editing tasks and measures them too. The only asterisk on the code editing is I didn't automate the reporting of accuracy because the test only uses Claude and having it judge it's own work seemed dubious, and having our existing parsers + policy checks verify Claude's output in a benchmark test like this might look like we were cooking the books in our favor (i.e., we're testing and verifying using our own system which obviously we will always get 100% on). Writing up a whole new independent Terraform parser or test harness to verify the results was beyond the scope of what I was willing to do for this just right now. So I opted for a "just assume Claude always gets it right", and we reported on just the token differences to get there.
bohdanstefaniuk
I would like to see not only token-saving in regular chats but also in codding sessions. Also it would be nice to have some kind of revenue attribution. For example we have a project which earns X amount of money and I would like to see if our token usage actually help us moving our revenue. Or at least what part of our revenue tokens cost