Show HN: Infracost (YC W21): Be proactive with your cloud costs
We launched in February 2021 (https://news.ycombinator.com/item?id=26064588), and Infracost is now being actively used by over 3,000 companies. However, there is a shift happening in the cloud cost management space. New teams, called FinOps teams (a combination of "Finance" and "DevOps"), are being formed within companies to manage cloud costs.
One of the first tasks assigned to these teams is to determine "who is using what" - that is, which teams, business units, products, etc. are spending the most on cloud. To accomplish this, they use tags. Tags are labels that all cloud resources should have and are key-value pairs. For example, a server could be tagged with: product=HackerNews; environment=production; team=blueTeam. So if resources are not tagged properly, then you can’t tell who is using what.
However, FinOps teams face challenges because their tools are reactive. These tools begin by analyzing cloud bills and providing visibility of tags from there. This means that they are looking at resources that are already running in production and costing money. A customer recently shared, “I want all resources to be properly tagged. But if they are not, I would rather a resource not be tagged at all than be tagged incorrectly.”
My "aha" moment! FinOps teams can define a tagging policy that can be validated in CI/CD before resources are launched. This is important because if code is shipped with the wrong tags, FinOps teams will have to fight for sprint time to fix them. Even if you shut down an untagged resource directly in the cloud, the next time Terraform runs, the resource will launch again with no tag. You need to fix the issue at its root.
I’d love your feedback on our solution to the tagging problem. You define your tag key-value policy in our SaaS product, and Infracost checks all Terraform resources per change. If anything fails the policy, it posts a comment with the details of which resources need tags, and what the allowed values are. Once fixed, it will let the code be shipped to production.
Try it out by going to https://dashboard.infracost.io/, setting up with the GitHub app or GitLab app, and defining your tagging policy. It will then scan your repository and inform you of any missing tags and their file and line number. You can use the free trial, but if you need more time, please message me and I’ll extend it for you.
I would also love to hear how others ensure that the correct tag keys and values are applied to all resources, and whether this is done proactively or reactively. Additionally, I would be interested in hearing about any lessons learned in the process.
Cheers
37 comments
[ 4.5 ms ] story [ 94.4 ms ] threadWhat remains seems more like organizational dynamics, but what are your thoughts?
The testing analogy is a good one as this feature also shows the engineers the current "failing policies" on the main branch too, so whilst they could merge the pull request without fixing the tagging issue, it'll just get added to the list. And maybe like tests, they group them into one task and go through to fix them all every so often to get the main branch back to green!
I've used this in practice in a company of ~80 developers at the time, applied it because read about it in some Dropbox papers, and have since seen it work in 2 other companies. Skip your flaky tests!!
I think the easier we make it for engineers to fix tagging issues, the more likely it'll be for engineers to take action. Send me an email asking me to read the company's wiki page on tagging policy and I'll delete the email; tell me I have a typo on line 8 as soon as I open my pull request, I'll fix it and move on.
We're using service control policies to enforce tagging on certain resource types, and retroactively for the rest.
Considering to use a "shift-left" tool as well, but it would need to support Terraform, CDK, Serverless and Cloudformation.
In case it's the latter, I can recommend Spacelift[0] - a specialized CI/CD tool for IaC and supports all the tools you've mentioned. It basically helps you build policies and orchestrate your infra (don't want to go into too much detail in this comment) to scale it to bigger teams and setups. Policies to enforce tagging would indeed be a good example.
It integrates with infracost too, but obviously just for the tools infracost works with, no CloudFormation.
Disclaimer: Work at Spacelift so obviously take the recommendation with a grain of salt, but I do legitimately think it's a great tool.
[0]: https://spacelift.io
P.S. Congrats on the Show HN Infracost team!
I like the idea of implementing tagging enforcement in the pipeline. In a perfect world you would use cloud policies to do this, but in practice this is a big loser in AWS where a staggering number of resources are created by one API call and then tagged as a followup API call, meaning an SCP to prevent launch of untagged resources won't ever work.
Estimating costs for serverless technologies upfront is definitely challenging. We're thinking of bringing in the last 30 days of usage for these resources to give engineers some visibility.
I have experience interacting with a logging system, where any diff to the logged data would need a tag like `log_size_increase=3 bytes` – the CICD system would then turn this, with the data already available, into an estimate of the overall extra storage needed.
Perhaps the same could be done. Rather than figuring out "usage" of some serverless systems, which is a very vague question and therefore hard to answer, perhaps it could be more specific. For example, how many requests per second is it expected to receive? Or, which other serverless functions call it (and therefore which will it necessarily have the scale of). Or, what increase in usage would be expected for this change.
We have a bot at work that sends you (or a DL with a bunch of people) a nastygram if you forget to tag your resources, but it doesn't know this. So if CloudFormation isn't done, you'll get the email and then have to respond to everyone with a screenshot showing that you didn't in fact goof it up. I wonder if you can make it so EventBridge (or however it's implemented, I'm not sure) delay an event for 30 seconds so they don't actually look until CF is done tagging.
As far as a solution to consistent tagging -- if I am understanding the problem space correctly -- something like Cloud Custodian could possibly help. It's open source and you can set up auto-tagging policies as well as use Cloud Custodian to backfill tags. These policies use lambda functions to respond to certain actions (ie, spinning up an ec2 instance, etc) and auto-tag with the resource creator/owner.
Backfilling tags works, but the issue is if Terraform isn't updated, it causes drift - it's much better to fix it at the root, so that's what Infracost helps with
It is a bit like complaining about making tractor engines more efficient when you didn’t have those problems using animals and ploughs.
Anyway the point is also not that cloud didn't solve any issues but that by solving 1 issue, you get 5 new issues, and you solve one of those and you get again 5 new ones. And the loop keeps going endlessly.
But how often are these things truly *necessary"? I think they seem more necessary than they really are because they are so readily available.
That's the feedback we got from users, so with Infracost, the Pull Request itself tells the engineer what needs to be done, along with exact code line numbers etc before going any further, so everything is fixed within the same pull request. Also, it works across all cloud providers, so FinOps can set central tags in a uniform manner no matter where the engineers are launching resources.