Show HN: GoModel – an open-source AI gateway in Go (github.com)

how does this compare to bifrost - another golang router?

Curious how the semantic caching layer works.. are you embedding requests on the gateway side and doing a vector similarity lookup before proxying? And if so, how do you handle cache invalidation when the underlying model changes or gets updated?

Expectable, given that LiteLLM seems to be implemented in Python.

However kudos for the project, we need more alternatives in compiled languages.

This is really useful. I've been building an AI platform (HOCKS AI) where I route different tasks to different providers — free OpenRouter models for chat/code gen, Gemini for vision tasks. The biggest pain point has been exactly what you describe: switching models without changing app code.

One thing I'd love to see is built-in cost tracking per model/route. When you're mixing free and paid models, knowing exactly where your spend goes is critical. Do you have plans for that in the dashboard?

Any plans for AI provider subscription compatibility? Eg ChatGPT, GH Copilot etc ? (Ala opencode)

I don't see any significant advantage over mature routers like Bifrost.

Are there even any benchmarks?

  - image size: 16.96 MB vs Bifrost's 69.84 MB. It matters for sidecar, edge, and cold-start scenarios.
  - per-tenant keys, guardrails, and audit logs are all in the OSS repo - not gated.
  - AI interaction visualization that makes debugging individual request/response flows much easier.

Nice one! Let's say I'm serving local models via vllm (because ollama comes with huge performance hits), how would I implement that in gomodel?

  docker run --rm -p 8080:8080 \
    -e VLLM_BASE_URL=http://host.docker.internal:18000/v1 \
    -e VLLM_BASEMENT_BASE_URL=http://host.docker.internal:18000/v1 \
    enterpilot/gomodel:latest

Does this have a unified API? In playing around with some of these, including unified libraries to work with various providers, I've found you are, at some point, still forced to do provider-specific works for things such as setting temperatures, setting reasoning effort, setting tool choice modes, etc.

What I'd like is for a proxy or library to provide a truly unified API where it will really let me integrate once and then never have to bother with provider quirks myself.

Also, are you also planning on doing an open-source rug pull like so many projects out there, including litellm?

[dead]

Are these kinds of libraries a temporary phenomenon? It strikes me as weird that providers haven't settled on a single API by now. Of course they aren't interested in making it easier for customers to switch away from them, but if a proprietary API was a critical part of your business plan, you probably weren't going to make it anyway.

(I'm asking only about the compatibility layer; the other tracking features would be useful even if there were only one cloud LLM API.)

I wrote a similar golang gateway, with the understanding that having solid API gateway features is important.

https://sbproxy.dev - engine is fully open source.

Another reason golang is interesting for the gateway is having clear control of the supply chain at compile time. Tools like LiteLLM the supply chain attacks can have more impact at runtime, where the compiled binary helps.

I have written and maintained AI proxies. They are not terribly complex except the inconsistent structure of input and output that changes on each model and provider release. I figure that if there is a not a < 24 hour turn around for new model integration the project is not properly maintained.

Governance is the biggest concern at this point - with proper logging, and integration to 3rd party services that provide inspection and DLP type threat mitigation.

Looks nice, thanks for open sourcing and sharing.

I'm all in on Go and integrating AI up and down our systems for https://housecat.com/ and am currently familiar and happy with:

https://github.com/boldsoftware/shelley -- full Go-based coding agent with LLM gateway.

https://github.com/maragudk/gai -- provides Go interfaces around Anthropic / OpenAI / Google.

Adding this to the list as well as bifrost to look into.

Any other Go-based AI / LLM tools folks are happy with?

I'll second the request to add support for harnesses with subscriptions, specifically Claude Code, into the mix.

This is awesome work, thanks for sharing!

How do you plan on keeping up with upstream changes from the API providers? I have implemented something similar, and the biggest issue I have faced with go is that providers don’t usually have sdk’s (compared to javascript and python), and there is work involved in staying up to date at each release.

[dead]

it's nice that it supports different providers

looks interesting, will defo give it a try. thanks for open-sourcing it!

Hey, this looks super nice. I do like the 'compact' feel of this. Reminds me of Traefik. It seems very promising indeed!

One problem I have is that yes, LiteLLM key creation is easier than creating it directly at the providers and managing it there for team members and test environments, but if I had a way of generating keys via vault, it would be perfect and such a relief in many ways.

I see what I need on your roadmap, but miss integration with service where I can inspect and debug completion traffic, and I don't see if I would be able to track usage from individual end-users through a header.

Thank you and godspeed!

Given this app seems to expose itself via REST calls, why would anyone care that it’s written in Go? I guess it matters to potential contributors but the majority of interest would be from users.

Really slick and lightweight! My biggest question is about the upstream API evolution. Keeping up with every provider's updates feels like a full-time job—how do you automate or manage the tracking of these constant protocol shifts?

Love the progress!! keey going

Nice work. How are you doing the cache key when the prompt has a timestamp or session id in it? Regex pass to strip volatile fields before hashing, or do you make the user declare what's stable upfront? I've tried both in my own stuff and neither ends up clean.

[dead]

[flagged]

[dead]