HN Explorer

6 comments

[ 3.4 ms ] story [ 22.2 ms ] thread
trzy
Simple API server for LLaVA based on llama.cpp. The llama.cpp project will add server and interactive support for LLaVA but I just couldn't wait. Runs great on my M1 Pro Macbook w/ 32gb RAM. Enjoy! :)
v3ss0n
We already have ollama,LocalAI.
trzy
Neither of which run LLaVA as far as I can tell. Use this until that happens.

Also, this is way easier to set up. No dependencies. Easy.

v3ss0n
why not? both uses llamacpp as backend
trzy
llama.cpp does not have interactive mode support for LLaVA nor the web server yet. So if they're just invoking the app, there is a lot of overhead for each call until they add that in. This server doesn't have that issue.
juliangoldsmith
Glad to see you're still working on something interesting. I saw the game console you developed many years ago. That paper demystified HDLs and the hardware side of the Z80 for me.