g023
- Karma
- 0
- Created
- ()
- Submissions
- 0
-
A high-performance local LLM server providing drop-in API compatibility with Ollama and OpenAI, built on llama.cpp's llama-server. Features automatic VRAM management, Hugging Face integration, and modular architecture.…
- Show HN: Standalone TurboQuant KV Cache Inference (github.com)
Implements TurboQuant (ICLR 2026, arXiv:2504.19874) KV cache compression directly inside a Transformers inference script. All algorithms are self-contained. Minimal dependencies. - uses…
-
An offline first focused agentic CLI application powered by local Ollama models. Integrated advanced memory management. Minimal dependencies. MIT licensed. DEFAULT USES A 4b Qwen 3.5 MODEL.…
-
g023's OllamaMan - Ollama Manager OS style GUI management of ollama Server - Open Source LLM Management using PHP/JS/Sqlite and a web browser. Integrated apps for chat, terminal viewer, model management (can pull or…