Hey I just created a router that's completely free for Gemini, Anthropic, and OpenAI models. It lets you set a SLA on how quickly you want a response to start getting processed and looks for cheaper inference in that time. If it succeeds then it charges on the savings but if it doesn't find cheaper inference or you just make a default request then it is still entirely free. There is also a typed SDK for TypeScript and Python with it. Let me know what you all think, you can test it out without any credit cards as long as you don't set the start_within parameter to a SLA and instead just set it to default.
There are a lot of areas I can already see for improvements but really want to know what everyone thinks.
1 comment
[ 0.56 ms ] story [ 9.0 ms ] threadThere are a lot of areas I can already see for improvements but really want to know what everyone thinks.