Home › VRAM sizes › 12GB
What a 12GB card actually runs
10 of 46 dataset models fit at q4_K_M with an 8k context window. Cards in this class we can name: RTX 3060 12GB.
Check date: 2026-10-24 — GiB figures are computed from the modelvram.com open dataset (CC BY 4.0 / MIT, retrieved 2026-10-20) by the same formula the calculator uses: weights at your quant + KV cache at 8k context. Card capacities are vendor spec-sheet numbers recorded 2026-10-20 (cross-checked against the Wikipedia GPU series tables). Budget rule: ours, labelled above (2026-10-24).
The budget we spend from, in one paragraph
Our budget for a discrete card is the frame-buffer size on the vendor’s spec sheet, spent in full: we deduct nothing for the OS or the display driver, and we say so out loud, because there is no published figure to deduct by — llama.cpp’s docs (README plus the CUDA/Metal build pages we fetched 2026-10-23) name levers that move a run’s VRAM (—ngl, GGML_CUDA_FORCE_MMQ, the unified-memory fallback) but never an OS-by-OS ‘usable VRAM’ number, and Ollama’s FAQ tells you to inspect what it has loaded, not what you have free. So every table on this page is an upper bound, our heuristic: a desktop compositor and a connected display take a slice of it, and how much is no public data (we checked, 2026-10-24).
Models that fit 12GB at q4_K_M (8k context, GiB)
| model (largest first) | params | needs | headroom left |
|---|---|---|---|
| Gemma 4 12B | 1.19597e+10B | 8.6 | 3.4 |
| Ornith 1.5 9B | 9.6531e+09B | 6.8 | 5.2 |
| Qwen3.5 9B | 9.6531e+09B | 6.8 | 5.2 |
| Qwen3 8B | 8.19074e+09B | 6.8 | 5.2 |
| Llama 3.1 8B | 8.03026e+09B | 6.6 | 5.4 |
| Gemma 4 E4B | 7.99616e+09B | 5.6 | 6.4 |
| LightOnOCR-3 4B | 4.53927e+09B | 3.6 | 8.4 |
| Nemotron 3 Nano 4B | 3.97356e+09B | 3.1 | 8.9 |
| MiniCPM5 2B | 2.51676e+09B | 2.4 | 9.6 |
| Limite 1B Violetto | 1.03525e+09B | 1.3 | 10.7 |
Need = weights at q4_K_M + KV cache at 8k context, from the modelvram.com dataset (retrieved 2026-10-20, CC BY 4.0). Headroom is against the nominal 12 GiB, not against what your desktop leaves free — see the rule above.
The near misses (the honest part)
These miss your 12GB at q4_K_M by 2 GiB or less, at their smallest shipped quant nothing else helps:
| model | needs | verdict |
|---|---|---|
| gpt-oss-20b | 13.7 | short by ≈1.7 GiB |
A card with no model page above still means “no dedicated page yet”, not “not in the dataset”: 46 models are in the calculator, 13 have their own page.
What each quant buys you at 12GB
Count of dataset models that fit at that exact quantisation, same 8k context: native 4 · bf16 4 · fp8 9 · q8_0 9 · q6_k 10 · q5_k_m 10 · q4_k_m 10 · q3_k_m 11 · q2_k 11.
Not a discrete card?
A Mac with 12GB of unified memory is not this page: the same number has to hold macOS and your apps too, so the budget we apply there is 8.0 GiB, not 12 GiB — our rule, labelled on those pages. Apple pages at this size: the Mac pages (the machines we could quote Apple on are 16/24/48GB).
Head-to-heads in this class
RTX 3060 12GB vs RTX 4060 Ti 16GB · model VRAM pages · used-card break-even
FAQ
Is 12GB of VRAM enough for a local LLM?
For 10 of the 46 models in our dataset, yes, at q4_K_M with 8k context — that is exactly the table above, and the near-miss table shows the ones that fall within 2 GiB of the edge. If you have one model in mind, ask the calculator: it answers for 46 models at nine quantisations and any context length.
How much does quantisation actually save on a 12GB card?
On this budget the same dataset fits: native 4 · bf16 4 · fp8 9 · q8_0 9 · q6_k 10 · q5_k_m 10 · q4_k_m 10 · q3_k_m 11 · q2_k 11 models (q4_K_M is the default most people start from). We publish counts, not quality judgements — which quant is acceptable is a task-by-task call and no source we trust scores it for your prompt.
Does 12GB of VRAM mean 12GB for my model?
No. We budget the full 12 GiB because nobody publishes an OS-by-OS usable figure (no public data (we checked, 2026-10-24)); a compositor and a connected display take a slice of it, so treat these numbers as an upper bound and keep headroom. Context is the other bill: the tables are written at 8k, and the KV cache grows with every extra token window — the calculator scales it.
Other sizes
8GB · 16GB · 24GB · 32GB · all five in one matrix
GiB figures computed from the modelvram.com open dataset (CC BY 4.0 / MIT, retrieved 2026-10-20) by the calculator formula: weights at your quant + KV cache at 8k context. Card capacities: vendor spec sheets, captured 2026-10-20. Budget rule: ours, stated on the page. Check date 2026-10-24.