Home › VRAM sizes › 16GB
What a 16GB card actually runs
11 of 46 dataset models fit at q4_K_M with an 8k context window. Cards in this class we can name: RTX 4060 Ti 16GB, RTX 5080 16GB.
Check date: 2026-10-24 — GiB figures are computed from the modelvram.com open dataset (CC BY 4.0 / MIT, retrieved 2026-10-20) by the same formula the calculator uses: weights at your quant + KV cache at 8k context. Card capacities are vendor spec-sheet numbers recorded 2026-10-20 (cross-checked against the Wikipedia GPU series tables). Budget rule: ours, labelled above (2026-10-24).
The budget we spend from, in one paragraph
Our budget for a discrete card is the frame-buffer size on the vendor’s spec sheet, spent in full: we deduct nothing for the OS or the display driver, and we say so out loud, because there is no published figure to deduct by — llama.cpp’s docs (README plus the CUDA/Metal build pages we fetched 2026-10-23) name levers that move a run’s VRAM (—ngl, GGML_CUDA_FORCE_MMQ, the unified-memory fallback) but never an OS-by-OS ‘usable VRAM’ number, and Ollama’s FAQ tells you to inspect what it has loaded, not what you have free. So every table on this page is an upper bound, our heuristic: a desktop compositor and a connected display take a slice of it, and how much is no public data (we checked, 2026-10-24).
Models that fit 16GB at q4_K_M (8k context, GiB)
| model (largest first) | params | needs | headroom left |
|---|---|---|---|
| gpt-oss-20b | 2.09148e+10B | 13.7 | 2.3 |
| Gemma 4 12B | 1.19597e+10B | 8.6 | 7.4 |
| Ornith 1.5 9B | 9.6531e+09B | 6.8 | 9.2 |
| Qwen3.5 9B | 9.6531e+09B | 6.8 | 9.2 |
| Qwen3 8B | 8.19074e+09B | 6.8 | 9.2 |
| Llama 3.1 8B | 8.03026e+09B | 6.6 | 9.4 |
| Gemma 4 E4B | 7.99616e+09B | 5.6 | 10.4 |
| LightOnOCR-3 4B | 4.53927e+09B | 3.6 | 12.4 |
| Nemotron 3 Nano 4B | 3.97356e+09B | 3.1 | 12.9 |
| MiniCPM5 2B | 2.51676e+09B | 2.4 | 13.6 |
| Limite 1B Violetto | 1.03525e+09B | 1.3 | 14.7 |
Need = weights at q4_K_M + KV cache at 8k context, from the modelvram.com dataset (retrieved 2026-10-20, CC BY 4.0). Headroom is against the nominal 16 GiB, not against what your desktop leaves free — see the rule above.
The near misses (the honest part)
These miss your 16GB at q4_K_M by 2 GiB or less, at their smallest shipped quant nothing else helps:
| model | needs | verdict |
|---|---|---|
| Gemma 4 26B-A4B (MoE) | 17.0 | short by ≈1.0 GiB |
A card with no model page above still means “no dedicated page yet”, not “not in the dataset”: 46 models are in the calculator, 13 have their own page.
What each quant buys you at 16GB
Count of dataset models that fit at that exact quantisation, same 8k context: native 5 · bf16 4 · fp8 10 · q8_0 10 · q6_k 10 · q5_k_m 11 · q4_k_m 11 · q3_k_m 15 · q2_k 20.
Not a discrete card?
A Mac with 16GB of unified memory is not this page: the same number has to hold macOS and your apps too, so the budget we apply there is 10.7 GiB, not 16 GiB — our rule, labelled on those pages. Apple pages at this size: Mac mini (M4, 2024) 16GB · MacBook Pro (14-inch, M4, 2024) 16GB.
Head-to-heads in this class
RTX 5080 16GB vs RTX 4090 24GB · RTX 3090 24GB vs RTX 5080 16GB · RTX 3060 12GB vs RTX 4060 Ti 16GB · model VRAM pages · used-card break-even
FAQ
Is 16GB of VRAM enough for a local LLM?
For 11 of the 46 models in our dataset, yes, at q4_K_M with 8k context — that is exactly the table above, and the near-miss table shows the ones that fall within 2 GiB of the edge. If you have one model in mind, ask the calculator: it answers for 46 models at nine quantisations and any context length.
How much does quantisation actually save on a 16GB card?
On this budget the same dataset fits: native 5 · bf16 4 · fp8 10 · q8_0 10 · q6_k 10 · q5_k_m 11 · q4_k_m 11 · q3_k_m 15 · q2_k 20 models (q4_K_M is the default most people start from). We publish counts, not quality judgements — which quant is acceptable is a task-by-task call and no source we trust scores it for your prompt.
Does 16GB of VRAM mean 16GB for my model?
No. We budget the full 16 GiB because nobody publishes an OS-by-OS usable figure (no public data (we checked, 2026-10-24)); a compositor and a connected display take a slice of it, so treat these numbers as an upper bound and keep headroom. Context is the other bill: the tables are written at 8k, and the KV cache grows with every extra token window — the calculator scales it.
Other sizes
8GB · 12GB · 24GB · 32GB · all five in one matrix
GiB figures computed from the modelvram.com open dataset (CC BY 4.0 / MIT, retrieved 2026-10-20) by the calculator formula: weights at your quant + KV cache at 8k context. Card capacities: vendor spec sheets, captured 2026-10-20. Budget rule: ours, stated on the page. Check date 2026-10-24.