HN Explorer

Show HN: Smart model routing directly in Claude, Codex and Cursor (github.com)

214 points by adchurch ↗ HN
We built a model router that plugs into coding agents (e.g. Claude Code, Codex, Cursor, etc.) and intelligently sends requests to the best model to serve them. Here's a quick demo of running it locally: https://www.youtube.com/watch?v=isKhAyivtfM.

At Weave, we write most of our code with AI, and it's been getting more expensive. This came to a head when Opus 4.7 was released and, thanks to its tokenizer changes, our costs shot up. We knew we didn't need Opus for everything but we didn't want to lose out on the intelligence for the cases where you really need it. So we decided to build a model router to handle this for us.

The Weave Router acts as an Anthropic/OpenAI endpoint specifically for coding agents. It looks at every inference request and intelligently (more on that in a sec) decides what model to send it to, handling all the translations required along the way. So it can use faster/cheaper models (e.g. DeepSeek v4, GLM 5.2, Kimi K2.6) when possible, and frontier models (Opus 4.8 & GPT 5.5 (& Fable whenever it's back)) when necessary.

How do we know what model to route to? We trained an RL model on tens of thousands (so far!) of agent traces. We reward the routing model when it selects an LLM that successfully completes the given task.

Here's an example: if you ask the router to plan a complex change, it will (probably) route that request to Opus 4.8. Subagents exploring the codebase to gather context will be routed to more suitable models (e.g. DeepSeek V4 Flash). Then when you have the plan ready to implement, it will be (most likely) be handed to a quicker model (e.g. GLM 5.2) to carry it out.

We've been using this internally for the last month or so. We've saved 40% on tokens vs. what we otherwise would have paid, with no noticeable differences in quality or velocity.

The router is source-available under Elastic License 2.0, so you can self-host it. Or if you prefer, you can also use our hosted version: weaverouter.com.

I'll be here to answer any questions you may have!

56 comments

[ 0.25 ms ] story [ 90.5 ms ] thread
stpedgwdgfhgdd
The thing I do not get with these routers is that you will have more cache misses (5min ttl). And if there is one thing i’ve learned; using the cache is crucial.

How does this router translate to $$$ when developing?

port11
Artefact-based workflows solve this problem, and I think it’s more effective to go in that direction.

I still have Claude Code because Opus makes good plans, but I hand the plan over to M3 on Pi with 99.9% cache hits on a long session. Lovely. Pi then makes a summary file that Opus can use to review the code/context.

But you do need them to write down their stuff, so that compaction and clear sessions can work off a nice, concise document.

And if you are simply using Claude Code, then /advisor is what you want: a sub-agent with a much cleaner context is spawned to handle something -> not cached per se, but much cheaper to run.

I’d stay away from workflows that automatically route between models unless you can afford the cache misses. That’s also why GLM 5.x is costing me much more, I don’t get good caching with it.

_pdp_
Cool.. but I still don't get how this is going to save money. It seems to me that it might actually burn more money just because the whole system now seems to be coming from different LLMs.

Also, small LLMs are prone to stop before completion, throw errors and produce loops. Is this factored in the design of the tool? I am not sure.

edit: spellcheck

arendtio
What is the difference from Cursors 'auto' mode?
ai_slop_hater
Isn't this more expensive than always using the same model, since, as I understand, by routing to different models you give up on cache?
debarshri
It is funny. We are building something similar.
spqw
This + making sure common requests are saved as reusable skills and scripts would probably save a large part of my token usage

As prices increase we will see more of these tools to optimise and make the best use of token budget

g00k
Man, I'm not so sure if I'd use something like this because the way I prompt already changes based upon what model I am using. I'm not convinced it would route to the right model based on my diction or whatever.
stronglikedan
> Man, I'm not so sure if I'd use something like this because the way I prompt already changes based upon what model I am using.

Perhaps you're just not the best use case. It may work better when Average Joe is the one prompting.

emilio_srg2
but this means you work with API pricing rather than subscription pricing. Isn’t it better to use claude or codex CLI etc directly in terms of cost?
alansaber
"We reward the routing model when it selects an LLM that achieves the task successfully" sounds pretty oversimplified
gautam_io
This is cool!

Will this use my Claude Pro/Max subscription? Or will it always use the API billing "pay as you go"?

slopinthebag
> At Weave, we write ~all our code with AI

This is probably not a very effective way of marketing imo. At least, it turns me completely off.

suyash
I would rather just use OpenCode - leverage AI models, even can host locally or paid ones with ease.
k9294
What about request caching? If you swap to a cheaper model mid execution it might cost more that to make multiple requests to the already cached provider?
mkagenius
We have created Murmur[1] which kind of works with your existing subscription (having API key is not mandatory). You can just tag @copilot @codex from claude code to delegate work to them. (it can also do it on its own too btw)

1. https://github.com/instavm/murmur - Murmur

iluvcommunism
This is basically what I need, a router. I’m tired of changing intelligence & speed levels manually.
peterbell_nyc
I auto tune my prompts to a locked model version based on production data used as evals with holdback data. I think the use case for this would be one off interactive prompts? For now I just run those all against an Opus 4.8 MAX and I'm sure I could downtune, although for interactive my opening prompt isn't always reflective of my overall goals for the multi turn session.

I'm just trying to figure out why on the fly routing would beat testing and tuning and locking models and versions for each class of call, with evals and auto tunes running to explore more possible models for commonly run classes of prompt over time . . .

gopher_space
"Based on your subscription tier and local hardware here's a list of models that fit and process definitions your biggest brain will comfortably handle."

I guess that sounds a lot like moving your evals and auto tunes to a third-party, but I don't have the time, budget, or inclination to create a system like this out of whole cloth and then keep it relevant.

I could see something that provides on-the-fly routing information being useful, but actual decision-making is too dependent on context.

bijowo1676
How come data privacy and confidentiality is not an issue with services like these?

Do people voluntarily use these proxies/routers, knowing their prompts, outputs and code will be seen by other people ?

I get it might be ok for personal projects, but for anything that makes money and is a part of business... this must be big no-no ?

jakozaur
It's rather hard to do at the proxy level with agentic coding, such as Claude Code or similar. These are long-chained sessions of tool use that heavily rely on prompt caching. Changing mid-flight is costly.

It looks like much more context is required to decide on the best model (e.g., summarizing logs might use a cheap model, whereas you likely want Opus/Mythos/GPT 5.6 to debug multithreading logic). In an agentic system, a decision about the model may be embedded in the decision to orchestrate the model.

GodelNumbering
This would not work in the way that shows any significant genuine benefit IMO. Caching and optimum routing of a single request are at odds with each other. Higher the distinct model count in a conversation, more cache misses you accept.

Based on what OP said elsewhere in the discussion "threshold to switch to another model will be higher" means that essentially you reduce the workflow into two models at most. The two model primitive, one planner and one executor, is already sufficient for such a use case.

For lower than 2 models, it's just a simple single model cache-preserving conversation which arguably doesn't need another layer. For larger than 2 models, you are likely paying a large aggregate cache penalty that negates most of the gains

reliablereason
Wont this kill the kv cache?

Also i am pretty sure neither open ai or anthropic leets you seed the agents own tokens.

treexs
Ahh been working on the same thing for a while now but haven't launched yet
jmalicki
> with no noticeable differences in quality or velocity.

Have you done any A/B tests on this with evidence? (That's one thing I'd be very interested to see for claims like this - I'm not necessarily doubting you, it just seems like it could be useful to understand claims of quality/efficiency)

nikcub
I'm glad there are more attempts at solving model routing, as costs (at API rates) has really become an issue. Some feedback:

1. Reiterate the cache issue from other comments already here. there is a lot of optimisation in harnesses around caching and a proxy model blows that up

2. Coding agents are model aware - they already route code discovery to mini / flash models, planning to heavy models, workflow design to ultra, implementation to mid / high etc. They know when they're exploring, planning, implementing, reviewing etc. and which model class to select and when it fails.

With a proxy you're breaking this control loop and feedback. It doesn't know, for ex. that it just attempted with deepseek v4 and it failed, lets try Opus?

3. How are you going to RL improvements and prevent the router becoming stale? You only have access to your own internal prompts and ~thousands of samples.

This is RL'd on one orgs codebase. There are going to be a lot of prompts you haven't seen before and have no insight to on how to route correctly, and you have no insight into users HF to improve your own model. Orgs aren't going to share their traces with you, so you need other sources to train on and improve

There are also new model releases every week that you need to keep up with - whats the story going to be here

4. Publish evals by running terminalbench / deepswe bench. Show us the performance / cost / time chart vs the other agent and model sets. If you can show gains there, you have a very simple value prop to sell where you can charge for a % of the saved costs