Thankfully Google has some basic protection for it. I accidentally commited my google api token, as part of some OTEL trace JSON file, and within a few minutes my key was automatically locked by google, and marked as leaked (with exact link pointing where it has happened).
I understand that cloud resources and automatically stopping them beyond a certain spend is problematic and challenging in many ways, e.g. do you just destroy provisioned computer, storage, data?
But for those stupid API keys the corporations have zero excuse not to have configurable limits with a sensible default.
> Conclusion: Always set billing caps and alerts on cloud API keys.
Sadly, way easier said than done in the case of GCP. Been a proper reason for me to avoid GCP deployments with LLM use-cases for smaller projects.
I remember looking into this a while back assuming it would be a sane feature to expect. But for some reason it's surprisingly non-trivial with GCP to set budgets. Especially if the only thing you want is a Gemini API key with finite spending.
IIRC you could either set (rate) limits on quotas, but quotas are extremely granular (like, per region per model) meaning you need to both set tons of values and understand which quotas to relax. Or alternatively do some bubblegum-and-ducktape like solution where you build an event-driven pipeline to react to cost increases in your own project.
I understand that exact budgets are hard to enforce in real-time, especially for their more complex infra offerings.
However, (1) even if it's not exactly real-time, but instead enforced every hour that's already going to go a long way, and (2) PAYG LLM usage is billed rather linearly by the amount of tokens you use, so if there would be an easy way to set a dollar-amount and have that expressed as budgets that would already get you part of the way there.
Anyway, the current state of GCP budgeting it makes me avoid it for production usage until I'm ready to commit spending significant effort to harden it. For all small projects, the free tier tokens are a safe bet, but their extremely low rate-limits make them rarely a good fit.
Yeah, I couldn't figure out how to set billing caps on the gemini API. Here's what the chatbot said:
Me: Help me cap gemini API request costs ... limit total billing for this project to max $100 a month
GC: Hello! While it's not possible to set a hard spending cap on Gemini API requests, you can set up billing alerts to monitor your costs and avoid surprises.
Me: How to set hard budget limit tied to billing account
GC: Based on your account information, it is not possible to set a hard budget limit that automatically stops charges for a billing account.
This reads like an “incident without guardrails”: per-project caps/quotas, anomaly alerts (minutes), env-split keys, and an automated kill-switch should be defaults for usage-based APIs. Billing emails are post-mortems.
17 comments
[ 6.8 ms ] story [ 34.0 ms ] threadAs far as I saw you can only set up billing alerts, no hard limit.
[1] https://news.ycombinator.com/item?id=47156925
https://github.com/coollabsio/llmhorrors.com/blob/main/CLAUD...
The whole website seems to be focused on promoting the author and their projects more than sharing the information. Just link to the original.
https://www.reddit.com/r/googlecloud/comments/1reqtvi/82000_...
Posted to HN twice recently.
https://news.ycombinator.com/item?id=47231708
https://news.ycombinator.com/item?id=47184182
its googles blunder that they allowed public tokens to be used for paid functionality.
But for those stupid API keys the corporations have zero excuse not to have configurable limits with a sensible default.
> Conclusion: Always set billing caps and alerts on cloud API keys.
Sadly, way easier said than done in the case of GCP. Been a proper reason for me to avoid GCP deployments with LLM use-cases for smaller projects.
I remember looking into this a while back assuming it would be a sane feature to expect. But for some reason it's surprisingly non-trivial with GCP to set budgets. Especially if the only thing you want is a Gemini API key with finite spending.
IIRC you could either set (rate) limits on quotas, but quotas are extremely granular (like, per region per model) meaning you need to both set tons of values and understand which quotas to relax. Or alternatively do some bubblegum-and-ducktape like solution where you build an event-driven pipeline to react to cost increases in your own project.
I understand that exact budgets are hard to enforce in real-time, especially for their more complex infra offerings.
However, (1) even if it's not exactly real-time, but instead enforced every hour that's already going to go a long way, and (2) PAYG LLM usage is billed rather linearly by the amount of tokens you use, so if there would be an easy way to set a dollar-amount and have that expressed as budgets that would already get you part of the way there.
Anyway, the current state of GCP budgeting it makes me avoid it for production usage until I'm ready to commit spending significant effort to harden it. For all small projects, the free tier tokens are a safe bet, but their extremely low rate-limits make them rarely a good fit.
Me: Help me cap gemini API request costs ... limit total billing for this project to max $100 a month
GC: Hello! While it's not possible to set a hard spending cap on Gemini API requests, you can set up billing alerts to monitor your costs and avoid surprises.
Me: How to set hard budget limit tied to billing account
GC: Based on your account information, it is not possible to set a hard budget limit that automatically stops charges for a billing account.
Me: How to set quota for gemini api?
GC: Sorry, I'm not able to answer that question.