Home › VRAM sizes
What your VRAM size can run
46 open-weight models × nine quantisations, computed against five frame-buffer sizes at an 8k context window. One number per cell: how many models fit. Each size has its own page with the model names and the near misses.
Check date: 2026-10-24 — GiB figures are computed from the modelvram.com open dataset (CC BY 4.0 / MIT, retrieved 2026-10-20) by the same formula the calculator uses: weights at your quant + KV cache at 8k context. Card capacities are vendor spec-sheet numbers recorded 2026-10-20 (cross-checked against the Wikipedia GPU series tables). Budget rule: ours, labelled above (2026-10-24).
The matrix
| VRAM | native | bf16 | fp8 | q8_0 | q6_k | q5_k_m | q4_k_m | q3_k_m | q2_k | largest that fits q4_K_M | near misses ≤2 GiB |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 8GB | 2 | 2 | 4 | 4 | 5 | 9 | 9 | 10 | 10 | Ornith 1.5 9B | 1 |
| 12GB | 4 | 4 | 9 | 9 | 10 | 10 | 10 | 11 | 11 | Gemma 4 12B | 1 |
| 16GB | 5 | 4 | 10 | 10 | 10 | 11 | 11 | 15 | 20 | gpt-oss-20b | 1 |
| 24GB | 10 | 9 | 11 | 11 | 12 | 19 | 22 | 22 | 22 | Ornith 1.5 35B-A3B (MoE) | 0 |
| 32GB | 11 | 10 | 15 | 14 | 22 | 22 | 22 | 22 | 22 | Ornith 1.5 35B-A3B (MoE) | 0 |
Cells are counts of dataset models whose weights-at-that-quant + KV-cache-at-8k total at or below the nominal size. The budget rule is ours and it is optimistic by design on discrete cards — read it once, above the first table on any size page.
The budget rule, in one paragraph
Our budget for a discrete card is the frame-buffer size on the vendor’s spec sheet, spent in full: we deduct nothing for the OS or the display driver, and we say so out loud, because there is no published figure to deduct by — llama.cpp’s docs (README plus the CUDA/Metal build pages we fetched 2026-10-23) name levers that move a run’s VRAM (—ngl, GGML_CUDA_FORCE_MMQ, the unified-memory fallback) but never an OS-by-OS ‘usable VRAM’ number, and Ollama’s FAQ tells you to inspect what it has loaded, not what you have free. So every table on this page is an upper bound, our heuristic: a desktop compositor and a connected display take a slice of it, and how much is no public data (we checked, 2026-10-24).
Pick your size
8GB — 9 models at q4_K_M · 12GB — 10 models at q4_K_M · 16GB — 11 models at q4_K_M · 24GB — 22 models at q4_K_M · 32GB — 22 models at q4_K_M
FAQ
Which VRAM size should I buy for local LLMs?
The model decides, not the card. Read what the model you actually want costs at your quant in the calculator, then buy the size that clears it with headroom — the rows above are the same arithmetic for five sizes. Two of those sizes have their own card pages: RTX 4090 24GB · RTX 5090 32GB.
Why do the counts change so much between quants?
Because a quantisation multiplies the weight tensor and the KV cache is charged separately: at 8k context the same model can cost twice the GiB at bf16 as at q4_K_M. The counts above are computed, not estimated — every cell is the same formula the calculator uses.
Is a 16GB Mac the same as a 16GB card?
No, and that is where most of the confusion lives. On a Mac the number is shared with macOS and your apps, so the budget we apply there is smaller by our own rule (two thirds up to 36GB, three quarters above) while on a discrete card we spend the nominal figure in full and label it as our heuristic. Apple does not publish a GPU-usable figure for any Mac — the Mac pages say so on every page.
No tokens/s figures anywhere in this cluster: no runtime publishes a per-card throughput table we could verify, so we print the arithmetic that is decidable. Pages for individual models: model VRAM pages; sizes on the buy side: under $500 · under $1000 · under $1500 · used GPU break-even.