LocalAI – Run any model – LLMs, vision, voice, image, video – on any hardware (github.com)

I can't recommend LocalAI enough. I am thoroughly enjoying it in all aspects. Especially the memory management and ability to load and boot models to and from memory as needed, right from the clients that use them.