Home › Runtimes › vLLM vs llama.cpp
vLLM vs llama.cpp (2026): Two Engines, Two Different Jobs
vLLM = serving engine for many requests, datacenter-oriented. llama.cpp = plain-C/C++ inference engine, the GGUF ecosystem's engine room. Two engines that share a file format and little else. llama.cpp’s README promises inference “with minimal setup… on a wide range of hardware” and documents CPU+GPU hybrid inference for models larger than total VRAM; vLLM’s docs promise “state-of-the-art serving throughput” — PagedAttention and continuous batching — on CUDA ≥7.5, ROCm ≥6.3 and Intel XPU. One person on one card with a model that almost fits is llama.cpp’s stated use case; there the batching machinery mostly idles. Neither publishes a comparable speed table, so this page prints none.
Docs check date: 2026-10-23 — every quoted cell below comes from an official docs page or the project README we fetched on that date (links at the bottom). Cells marked "no public data" mean we looked and could not extract a citable statement — a gap printed as a gap, not a zero.
| What we could verify | vLLM | llama.cpp |
|---|---|---|
| What it is (their own words) | “State-of-the-art serving throughput” — PagedAttention KV-memory management, continuous batching, chunked prefill, prefix caching[4] | “Enable LLM (and VLM) inference with minimal setup and state-of-the-art performance on a wide range of hardware — locally and in the cloud”; plain C/C++ without dependencies[2] |
| Official GPU platforms | NVIDIA CUDA compute capability ≥7.5 (T4 → B200), AMD ROCm ≥6.3 (MI200s/MI300/MI350, RX 7900), Intel XPU; Apple Silicon is NOT in core — a community-maintained vLLM-Metal plugin on MLX[6] | README backend table: Metal (Apple silicon, “first-class citizen”), CUDA, HIP (AMD), MUSA, Vulkan, SYCL, OpenCL, OpenVINO, CANN, Hexagon and more[2] |
| On an Apple Silicon Mac | Listed as a GPU variant, but the install page routes it to vLLM-Metal — “a community-maintained hardware plugin that uses MLX as the compute backend”, not core vLLM[6] | README: “Apple silicon is a first-class citizen” (ARM NEON, Accelerate, Metal); Metal listed for Apple Silicon in its backend table[2] |
| Model format | Hugging Face weights first; GGUF listed among quantisation options (with FP8, GPTQ/AWQ, compressed-tensors…)[4] | GGUF native — README shows pulling a GGUF repo straight from Hugging Face[2] |
| When the model exceeds VRAM | no public data (we checked, 2026-10-23) — install docs are GPU-backend oriented; no CPU-offload statement we could cite | “CPU+GPU hybrid inference to partially accelerate models larger than the total VRAM capacity” — an official statement, not folklore[2] |
| KV-cache quantisation switch | “Quantized KV Cache” is a listed quantisation feature in the docs[4] | no public data (we checked, 2026-10-23) — the docs paths we tried 404’d at fetch time; the switch may exist, we quote only what we fetched |
| Parallel requests | Continuous batching of incoming requests is a headline design feature[4] | no public data (we checked, 2026-10-23) in the README we fetched |
| OpenAI-compatible API | OpenAI-compatible serving endpoints are documented (chat completion, responses, embeddings)[4] | README: “Launch OpenAI-compatible API server” (llama serve)[2] |
The number none of the four publishes
“How many GiB does this model need at q4_K_M with my context?” appears in none of the docs quoted above — each runtime publishes capabilities, not per-model memory needs. That table is our dataset: weights + KV-cache math per model, with sources and check dates — open the calculator. Speed (tokens/s) we deliberately do not publish either: nothing official, nothing repeated.
FAQ
Which is faster?
We print no tokens/s figure. Neither project publishes a comparable performance table we could verify as of 2026-10-23, and third-party benchmarks mix model, quant and context into numbers nobody can reproduce. What is citable is what each engine officially supports and what it says happens when a model exceeds VRAM — the table above. Fit comes first: our calculator computes GiB need from the sourced weights figures.
Does it fit on my 24GB (or 16/32/48GB) card?
The GPU question is runtime-independent: weights at your quant plus KV cache at your context. Start there — worked example for a 24GB card · model pages — then take the answer to whichever runtime's platform table above covers your hardware.
Why so many “no public data” cells?
Because we quote only official docs fetched the same day. Where a vendor page is silent or a doc path 404'd on us, the cell says so instead of repeating a blog’s guess. Gaps stay visible — that is the whole honesty model of this site.
Sources for this page
- llama.cpp README (master) — fetched 2026-10-23
- vLLM docs — stable home — fetched 2026-10-23
- vLLM docs — GPU installation — fetched 2026-10-23
More runtime compares: Ollama vs llama.cpp · Ollama vs LM Studio · Ollama vs vLLM. All four engines in one 8-row table: runtime comparison table. Choosing cloud GPUs instead: RunPod vs Vast.ai. On a Mac instead of a card, where system memory doubles as GPU memory: Mac mini M4 16GB · Mac mini M4 24GB · Mac mini M4 Pro 48GB · MacBook Pro M4 16GB.