Local inference
Ollama
The simplest way to pull and run open-weight models locally, with an OpenAI-compatible endpoint.
- Category
- Local inference
- Pricing
- Open source
- Runs
- Runs locally
- Interface
- CLI, API
- Source
- Open source
What Ollama is
Ollama makes running a local model a single command: it handles download, quantisation choice, and memory sizing for you. It serves an OpenAI-compatible endpoint, so existing code can point at localhost with no other change, and runs on macOS, Linux, and Windows.
Best for
Anyone trying local inference for the first time, and for local development against a model that costs nothing per call.
Consider something else if
Optimised for one user at a time; serving concurrent production traffic wants a batching engine instead.
Ollama alternatives
The closest options in local inference, on the axes that actually separate them.
| Tool | Best for | Pricing | Runs |
|---|---|---|---|
| Ollamaopen source | Anyone trying local inference for the first time, and for local development against a model that costs nothing per call. | Open source | Runs locally |
| LM Studio | Running models locally without the terminal, and comparing several quickly in one interface. | Free tier | Runs locally |
| llama.cppopen source | Squeezing a model onto constrained hardware, and embedding inference into your own application. | Open source | Runs locally |
| vLLMopen source | Serving an open-weight model to real traffic on GPUs you operate. | Open source | Self-hosted |
| MLXopen source | Getting the most out of a Mac, and local fine-tuning on Apple silicon. | Open source | Runs locally |
Choosing within local inference
One user or many
Single-user tools optimise for a fast start and easy model switching. Serving engines optimise throughput across concurrent requests via batching. Using the first for a production endpoint wastes most of your GPU.
Your hardware
Apple silicon, NVIDIA, and CPU-only are genuinely different targets. Some tools cover all three; others are built for one and are much faster on it.
Questions
What is Ollama?
Ollama makes running a local model a single command: it handles download, quantisation choice, and memory sizing for you. It serves an OpenAI-compatible endpoint, so existing code can point at localhost with no other change, and runs on macOS, Linux, and Windows. It is open source and runs locally.
What are the alternatives to Ollama?
The closest alternatives are LM Studio, llama.cpp, vLLM, MLX. They sit in the same category — local inference — and differ mainly on hosting model, pricing shape, and how much they abstract away.
Is Ollama the right choice?
Anyone trying local inference for the first time, and for local development against a model that costs nothing per call. The main caveat: Optimised for one user at a time; serving concurrent production traffic wants a batching engine instead.
Whatever you build on, the model is the line item that scales. See what each one costs per million tokens, or price your own workload.