Home › Runtime compares › Ollama vs llama.cpp
Ollama vs llama.cpp (2026): What Their Docs Actually Say
Ollama = app + CLI that runs models on your own computer. llama.cpp = plain-C/C++ inference engine, the GGUF ecosystem's engine room. One lineage, two layers. Ollama’s own import docs point quantisation work at llama.cpp’s llama-quantize, so the real question is which layer you want to work in: the packaged app with a registry and sane defaults, or the raw engine with the backend table and the documented CPU+GPU hybrid fallback. What fits on your card is GGUF math either way — that part you can compute before installing either.
Docs check date: 2026-10-23 — every quoted cell below comes from an official docs page or the project README we fetched on that date (links at the bottom). Cells marked "no public data" mean we looked and could not extract a citable statement — a gap printed as a gap, not a zero.
| What we could verify | Ollama | llama.cpp |
|---|---|---|
| What it is (their own words) | “Download Ollama and run a model on your computer” — model library, CLI, local or cloud models[1] | “Enable LLM (and VLM) inference with minimal setup and state-of-the-art performance on a wide range of hardware — locally and in the cloud”; plain C/C++ without dependencies[2] |
| Official GPU platforms | Metal (Apple GPUs) via the Metal API; NVIDIA CUDA with a compute-capability table that lists RTX 50xx; AMD ROCm on Linux (v7 driver) plus additional AMD support via Vulkan[5] | README backend table: Metal (Apple silicon, “first-class citizen”), CUDA, HIP (AMD), MUSA, Vulkan, SYCL, OpenCL, OpenVINO, CANN, Hexagon and more[2] |
| On an Apple Silicon Mac | “Ollama supports GPU acceleration on Apple devices via the Metal API” — a section of its own in the GPU docs; no per-chip memory guidance there[5] | README: “Apple silicon is a first-class citizen” (ARM NEON, Accelerate, Metal); Metal listed for Apple Silicon in its backend table[2] |
| Model format | GGUF import — “Ollama does not quantize GGUF models during import… quantize them first with a GGUF tool such as llama.cpp’s llama-quantize”; plus its own registry[7] | GGUF native — README shows pulling a GGUF repo straight from Hugging Face[2] |
| When the model exceeds VRAM | Scheduler “leverages available VRAM data reported by the GPU libraries to make optimal scheduling decisions”; no per-model GiB table in the docs we fetched — that gap is what this site fills[5] | “CPU+GPU hybrid inference to partially accelerate models larger than the total VRAM capacity” — an official statement, not folklore[2] |
| KV-cache quantisation switch | OLLAMA_KV_CACHE_TYPE sets K/V cache quantisation; default f16; docs state it is a global option across models[8] | no public data (we checked, 2026-10-23) — the docs paths we tried 404’d at fetch time; the switch may exist, we quote only what we fetched |
| Parallel requests | OLLAMA_NUM_PARALLEL default 1; docs warn required RAM scales by OLLAMA_NUM_PARALLEL × OLLAMA_CONTEXT_LENGTH[8] | no public data (we checked, 2026-10-23) in the README we fetched |
| OpenAI-compatible API | “Use Ollama’s API or compatible OpenAI and Anthropic clients” — API reference has an OpenAI-compatibility section[9] | README: “Launch OpenAI-compatible API server” (llama serve)[2] |
The number none of the four publishes
“How many GiB does this model need at q4_K_M with my context?” appears in none of the docs quoted above — each runtime publishes capabilities, not per-model memory needs. That table is our dataset: weights + KV-cache math per model, with sources and check dates — open the calculator. Speed (tokens/s) we deliberately do not publish either: nothing official, nothing repeated.
FAQ
Which is faster?
We print no tokens/s figure. Neither project publishes a comparable performance table we could verify as of 2026-10-23, and third-party benchmarks mix model, quant and context into numbers nobody can reproduce. What is citable is what each engine officially supports and what it says happens when a model exceeds VRAM — the table above. Fit comes first: our calculator computes GiB need from the sourced weights figures.
Does it fit on my 24GB (or 16/32/48GB) card?
The GPU question is runtime-independent: weights at your quant plus KV cache at your context. Start there — worked example for a 24GB card · model pages — then take the answer to whichever runtime's platform table above covers your hardware.
Why so many “no public data” cells?
Because we quote only official docs fetched the same day. Where a vendor page is silent or a doc path 404'd on us, the cell says so instead of repeating a blog’s guess. Gaps stay visible — that is the whole honesty model of this site.
Sources for this page
- Ollama docs — home — fetched 2026-10-23
- llama.cpp README (master) — fetched 2026-10-23
- Ollama docs — GPU support — fetched 2026-10-23
- Ollama docs — import a model — fetched 2026-10-23
- Ollama docs — FAQ (env vars) — fetched 2026-10-23
- Ollama docs — OpenAI compatibility — fetched 2026-10-23
More runtime compares: Ollama vs LM Studio · Ollama vs vLLM · vLLM vs llama.cpp. Choosing cloud GPUs instead: RunPod vs Vast.ai. On a Mac instead of a card, where system memory doubles as GPU memory: Mac mini M4 16GB · Mac mini M4 24GB · Mac mini M4 Pro 48GB · MacBook Pro M4 16GB.