Does the size of your llama matter? What's Ollama?, can I plug it to Chroma? (

9 points by torrmal ↗ HN
If you have not seen it, this pretty cool open source project Ollama: It let's you run big LLMs in computers as small as your mac book!!

https://lu.ma/yourdatayourmodel

Ollama is launching support for Linux based machines this Wednesday at the SF AI Collective (come: https://lu.ma/yourdatayourmodel), alongside the founder of Chroma and a few more interesting people, join us in socializing and brainstorming about the future of all this!

5 comments

[ 2.9 ms ] story [ 178 ms ] thread
(comment deleted)
My Llama is huge! 70B parameters....Maybe I should try Ollama...
well, I am not sure how small you can turn your 70b model into a small one, but its a great question for the Ollama creators: https://lu.ma/yourdatayourmodel
actually, yes they do have support for Llama2 70B, you need 39GB of ram, compared to 130GB that it originally takes.
(comment deleted)