Ollama
hosomaki.ollama
A Docker Compose file for Ollama alone, generated from the catalog and checked by the safety inspector. No account needed.
Local runtime that downloads, quantises and runs open-weight language models, exposing them over its own HTTP API and an OpenAI-compatible endpoint. Also serves embedding models.
Ollama manages a local library of model weights, pulling them from its registry in quantised form and loading them into memory on demand when a request arrives. A Modelfile fixes the base weights, system prompt and sampling parameters for a named variant, which makes a customised model reproducible across machines. The API covers generation, chat and embeddings, and a compatibility layer accepts OpenAI-format requests so existing clients need only a changed base URL. A first deployment pulls one small model and issues a completion; GPU acceleration is used automatically where host drivers are present.
You know it worked when
- The service answers a version request over HTTP.
- A pulled model appears in the local model list.
- A generation request returns tokens from that model.
- An OpenAI-format request to the compatibility endpoint returns the response shape a client library expects.
- An embedding request returns a vector of the model's stated dimensionality.
Known sharp edges
- The first pull of a model transfers several gigabytes and the API returns nothing useful until it finishes.
- Models load on first request and are evicted after an idle timeout, so the first call after a pause is markedly slower than those following it.
- GPU acceleration needs matching host drivers and container runtime configuration; without them the service falls back to CPU silently.
- The API has no authentication and must not be reachable beyond the deployment network without a proxy in front of it.