Flint – local AI runtime for Rust (no internet, no API keys) (flint.brantandco.com) 1 points by Lbrant 4mo ago ↗ HN
[–] Lbrant 4mo ago ↗ I wanted to run AI in my Rust app without sending data to OpenAI. Built Flint — drop it in and run any GGUF model locally on device.cargo install flint-ai flint use TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUFRuns on Apple Silicon via Metal, NVIDIA via CUDA, AMD via ROCm, or any CPU as fallback. Models store in ~/.flint/models and persist across projects.Still early (v0.1.0) but works. Would love feedback from anyone who tries it.
2 comments
[ 3.3 ms ] story [ 14.1 ms ] threadcargo install flint-ai flint use TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF
Runs on Apple Silicon via Metal, NVIDIA via CUDA, AMD via ROCm, or any CPU as fallback. Models store in ~/.flint/models and persist across projects.
Still early (v0.1.0) but works. Would love feedback from anyone who tries it.