Spent this weekend building a one-file OpenRouter client — figured I'd open-source it.
We deal with a lot of dynamic structured data, where different LLM calls have different priorities — speed, latency, cost.
After a year using OpenRouter heavily in production, I hit a bunch of real-world problems — bad providers, unreliable fallbacks, weird edge cases with schemas — so I baked all of that into this.
We needed strong guarantees on output and simple, reliable handling around it.
Fine-tune per call — optimize for speed, latency, or price
Ignore bad providers (beyond account settings)
Fallback cleanly to JSON mode (180+ models)
Add new models declaratively, no client changes needed
It handles:
Strict structured outputs (zod-to-json-schema)
Fallback to JSON if needed
Provider sorting and fallback models
Unified one-line abstraction (makeLLMCall)
Real error handling and dynamic model swapping
We run over 50k OpenRouter requests/day through this in production.
Tried to make it stupid simple to read and extend - coming up with the best API abstractions I could.
(On a personal note, as an overworked founder, it felt great to step away from growth and customer support for a bit and spend a day building clean APIs again — reminded me of the simpler days at CMU, grinding through Joshua Bloch’s API design classes. Honestly feels like this could make a good assignment problem.)
1 comment
[ 2.9 ms ] story [ 7.9 ms ] threadWe deal with a lot of dynamic structured data, where different LLM calls have different priorities — speed, latency, cost. After a year using OpenRouter heavily in production, I hit a bunch of real-world problems — bad providers, unreliable fallbacks, weird edge cases with schemas — so I baked all of that into this. We needed strong guarantees on output and simple, reliable handling around it.
The primary motivations:
Explicit structure enforcement (Zod + JSON Schema)
Fine-tune per call — optimize for speed, latency, or price
Ignore bad providers (beyond account settings)
Fallback cleanly to JSON mode (180+ models)
Add new models declaratively, no client changes needed
It handles:
Strict structured outputs (zod-to-json-schema)
Fallback to JSON if needed
Provider sorting and fallback models
Unified one-line abstraction (makeLLMCall)
Real error handling and dynamic model swapping
We run over 50k OpenRouter requests/day through this in production. Tried to make it stupid simple to read and extend - coming up with the best API abstractions I could.
(On a personal note, as an overworked founder, it felt great to step away from growth and customer support for a bit and spend a day building clean APIs again — reminded me of the simpler days at CMU, grinding through Joshua Bloch’s API design classes. Honestly feels like this could make a good assignment problem.)