HN Explorer

Best Open Source Offline AI Agent

3 points by Ms-J ↗ HN
With the rise of AI agents, I've been looking for an agent that can use a combination of local models or models that are hosted on servers I control.

For the local models, the agent must be able to run completely offline. When using remote models, it must only send network traffic to my servers. No checking for "updates" or pulling any type of dependencies from elsewhere.

When trying Opencode, it was a failure.

See: https://github.com/Chetic/opencode-offline

"This is the offline fork of OpenCode. Upstream OpenCode requires internet access for several core functions: the web UI proxies all requests to app.opencode.ai, ripgrep and LSP servers (TypeScript, Python, C++, Rust) are downloaded on first use, and model metadata is fetched from models.dev. In air-gapped or restricted network environments, this means no web interface (HTTP 500 on every request), no file search, no code intelligence, and no model definitions."

The problem is this project isn't maintained to standards and still uses Opencodes base.

Thanks for any suggestions.

4 comments

[ 4.0 ms ] story [ 21.2 ms ] thread
josefcub
Try charmbracelet's crush, found here:

https://github.com/charmbracelet/crush

Crush is pretty new, but getting better all the time. It's written in Go, so no node hijinks to get it working. It works fine with my ollama or llama-server localhost endpoints, and I've used it to make up a couple of internal projects without any issues.

It does have internal telemetry and such (including updating its list of external models it can use) that can be turned off in the crush.json configuration file.

If you're on a Mac, you can install via homebrew or use the more traditional route via Github.

Archerlm
From what I know of, Ollama works offline, but if your ram is only 8 GB and without a GPU, latency would be severely limiting. The closest local model I can suggest is LM Studio (no cloud dependencies, own model registry, etc.). If that doesn't work out for you then GPT4All, but its models are quite small.
verdverm
I have a custom agent setup and just added ollama support. The main issue with local is model quality, it's just not there for the most part. I have had some good results using gemma4 31b on cpu, but the latency is really long. I do plan to use it for higher level automation that could run for a day or two without me needing to look at it (what would be ~minutes to hours with Gemini flash, i.e.)