Ask HN: Is anybody using llama.cpp for production?

11 points by HardikVala ↗ HN
Considering the server version of llama.cpp for a commercial use case over bulkier options like vllm. But wondering if it's been battle-tested in production environments.

3 comments

[ 3.3 ms ] story [ 21.3 ms ] thread