Yeah, we are on it. In the current version, things are hardcoded and implicit (we are also in experimental preview), but soon it will be configurable and explicit.
You can run the same model in production, using Docker Model Runner in Docker CE. This is assuming, the model works for your production use case :)
You can check out my blog post on publishing models to Docker Hub using Docker Model Runner :) https://www.docker.com/blog/publish-ai-models-on-docker-hub/
Unfortunately not, since the container won't have access to the Apple silicon GPU. That's why in our architecture, we have to run llama.cpp as a host process and wire it up with the rest of the Docker Desktop…
(disclaimer: I'm leading the Docker Model Runner team at Docker) We decided to start with Apple silicon Macs, because they provide one of the worst experiences of running LLMs in a containerized form, while at the same…
Better late than never ;)
(disclaimer: I'm leading the Docker Model Runner team at Docker) We are not packaging models as Docker images, since indeed that is the wrong fit and comes with all kinds of technical problems. It also feels wrong to…
(disclaimer: I'm leading the Docker Model Runner team at Docker) It's fine to disagree of course, but we envision Docker as a tool that has a higher abstraction level than just container management. That's why having a…
(disclaimer: I'm leading the Docker Model Runner team at Docker) You were always able to manually install Docker CE within WSL2 on Windows. But if you want to have an integrated Docker experience on the Windows host,…
Yeah, we are on it. In the current version, things are hardcoded and implicit (we are also in experimental preview), but soon it will be configurable and explicit.
You can run the same model in production, using Docker Model Runner in Docker CE. This is assuming, the model works for your production use case :)
You can check out my blog post on publishing models to Docker Hub using Docker Model Runner :) https://www.docker.com/blog/publish-ai-models-on-docker-hub/
Unfortunately not, since the container won't have access to the Apple silicon GPU. That's why in our architecture, we have to run llama.cpp as a host process and wire it up with the rest of the Docker Desktop…
(disclaimer: I'm leading the Docker Model Runner team at Docker) We decided to start with Apple silicon Macs, because they provide one of the worst experiences of running LLMs in a containerized form, while at the same…
Better late than never ;)
(disclaimer: I'm leading the Docker Model Runner team at Docker) We are not packaging models as Docker images, since indeed that is the wrong fit and comes with all kinds of technical problems. It also feels wrong to…
(disclaimer: I'm leading the Docker Model Runner team at Docker) It's fine to disagree of course, but we envision Docker as a tool that has a higher abstraction level than just container management. That's why having a…
(disclaimer: I'm leading the Docker Model Runner team at Docker) You were always able to manually install Docker CE within WSL2 on Windows. But if you want to have an integrated Docker experience on the Windows host,…