Home › Runtimes › Ollama vs vLLM
Ollama vs vLLM (2026): One Machine vs Many Requests
Ollama = app + CLI that runs models on your own computer. vLLM = serving engine for many requests, datacenter-oriented. Different jobs. Ollama’s docs are about running models on your computer (default: 1 parallel request, RAM scales with context). vLLM’s docs are about serving throughput (continuous batching, PagedAttention) on datacenter GPUs — Apple Silicon isn’t in its core. On one card doing personal inference, the batching machinery mostly idles.
Docs check date: 2026-10-23 — every quoted cell below comes from an official docs page or the project README we fetched on that date (links at the bottom). Cells marked "no public data" mean we looked and could not extract a citable statement — a gap printed as a gap, not a zero.
| What we could verify | Ollama | vLLM |
|---|---|---|
| What it is (their own words) | “Download Ollama and run a model on your computer” — model library, CLI, local or cloud models[1] | “State-of-the-art serving throughput” — PagedAttention KV-memory management, continuous batching, chunked prefill, prefix caching[4] |
| Official GPU platforms | Metal (Apple GPUs) via the Metal API; NVIDIA CUDA with a compute-capability table that lists RTX 50xx; AMD ROCm on Linux (v7 driver) plus additional AMD support via Vulkan[5] | NVIDIA CUDA compute capability ≥7.5 (T4 → B200), AMD ROCm ≥6.3 (MI200s/MI300/MI350, RX 7900), Intel XPU; Apple Silicon is NOT in core — a community-maintained vLLM-Metal plugin on MLX[6] |
| On an Apple Silicon Mac | “Ollama supports GPU acceleration on Apple devices via the Metal API” — a section of its own in the GPU docs; no per-chip memory guidance there[5] | Listed as a GPU variant, but the install page routes it to vLLM-Metal — “a community-maintained hardware plugin that uses MLX as the compute backend”, not core vLLM[6] |
| Model format | GGUF import — “Ollama does not quantize GGUF models during import… quantize them first with a GGUF tool such as llama.cpp’s llama-quantize”; plus its own registry[7] | Hugging Face weights first; GGUF listed among quantisation options (with FP8, GPTQ/AWQ, compressed-tensors…)[4] |
| When the model exceeds VRAM | Scheduler “leverages available VRAM data reported by the GPU libraries to make optimal scheduling decisions”; no per-model GiB table in the docs we fetched — that gap is what this site fills[5] | no public data (we checked, 2026-10-23) — install docs are GPU-backend oriented; no CPU-offload statement we could cite |
| KV-cache quantisation switch | OLLAMA_KV_CACHE_TYPE sets K/V cache quantisation; default f16; docs state it is a global option across models[8] | “Quantized KV Cache” is a listed quantisation feature in the docs[4] |
| Parallel requests | OLLAMA_NUM_PARALLEL default 1; docs warn required RAM scales by OLLAMA_NUM_PARALLEL × OLLAMA_CONTEXT_LENGTH[8] | Continuous batching of incoming requests is a headline design feature[4] |
| OpenAI-compatible API | “Use Ollama’s API or compatible OpenAI and Anthropic clients” — API reference has an OpenAI-compatibility section[9] | OpenAI-compatible serving endpoints are documented (chat completion, responses, embeddings)[4] |
The number none of the four publishes
“How many GiB does this model need at q4_K_M with my context?” appears in none of the docs quoted above — each runtime publishes capabilities, not per-model memory needs. That table is our dataset: weights + KV-cache math per model, with sources and check dates — open the calculator. Speed (tokens/s) we deliberately do not publish either: nothing official, nothing repeated.
FAQ
Which is faster?
We print no tokens/s figure. Neither project publishes a comparable performance table we could verify as of 2026-10-23, and third-party benchmarks mix model, quant and context into numbers nobody can reproduce. What is citable is what each engine officially supports and what it says happens when a model exceeds VRAM — the table above. Fit comes first: our calculator computes GiB need from the sourced weights figures.
Does it fit on my 24GB (or 16/32/48GB) card?
The GPU question is runtime-independent: weights at your quant plus KV cache at your context. Start there — worked example for a 24GB card · model pages — then take the answer to whichever runtime's platform table above covers your hardware.
Why so many “no public data” cells?
Because we quote only official docs fetched the same day. Where a vendor page is silent or a doc path 404'd on us, the cell says so instead of repeating a blog’s guess. Gaps stay visible — that is the whole honesty model of this site.
Sources for this page
- Ollama docs — home — fetched 2026-10-23
- vLLM docs — stable home — fetched 2026-10-23
- Ollama docs — GPU support — fetched 2026-10-23
- vLLM docs — GPU installation — fetched 2026-10-23
- Ollama docs — import a model — fetched 2026-10-23
- Ollama docs — FAQ (env vars) — fetched 2026-10-23
- Ollama docs — OpenAI compatibility — fetched 2026-10-23
More runtime compares: Ollama vs llama.cpp · Ollama vs LM Studio · vLLM vs llama.cpp. All four engines in one 8-row table: runtime comparison table. Choosing cloud GPUs instead: RunPod vs Vast.ai. On a Mac instead of a card, where system memory doubles as GPU memory: Mac mini M4 16GB · Mac mini M4 24GB · Mac mini M4 Pro 48GB · MacBook Pro M4 16GB.