Home › VRAM sizes › 24GB
What a 24GB card actually runs
22 of 46 dataset models fit at q4_K_M with an 8k context window. Cards in this class we can name: RTX 3090 24GB, RTX 4090 24GB.
Check date: 2026-10-24 — GiB figures are computed from the modelvram.com open dataset (CC BY 4.0 / MIT, retrieved 2026-10-20) by the same formula the calculator uses: weights at your quant + KV cache at 8k context. Card capacities are vendor spec-sheet numbers recorded 2026-10-20 (cross-checked against the Wikipedia GPU series tables). Budget rule: ours, labelled above (2026-10-24).
The budget we spend from, in one paragraph
Our budget for a discrete card is the frame-buffer size on the vendor’s spec sheet, spent in full: we deduct nothing for the OS or the display driver, and we say so out loud, because there is no published figure to deduct by — llama.cpp’s docs (README plus the CUDA/Metal build pages we fetched 2026-10-23) name levers that move a run’s VRAM (—ngl, GGML_CUDA_FORCE_MMQ, the unified-memory fallback) but never an OS-by-OS ‘usable VRAM’ number, and Ollama’s FAQ tells you to inspect what it has loaded, not what you have free. So every table on this page is an upper bound, our heuristic: a desktop compositor and a connected display take a slice of it, and how much is no public data (we checked, 2026-10-24).
Models that fit 24GB at q4_K_M (8k context, GiB)
| model (largest first) | params | needs | headroom left |
|---|---|---|---|
| Ornith 1.5 35B-A3B (MoE) | 3.59518e+10B | 22.9 | 1.1 |
| Qwen3.6 35B-A3B (MoE) | 3.59518e+10B | 22.9 | 1.1 |
| Nemotron 3 Nano 30B-A3B (MoE) | 3.15779e+10B | 20.1 | 3.9 |
| Gemma 4 31B | 3.12731e+10B | 21.9 | 2.1 |
| GLM-4.7 Flash | 3.12215e+10B | 20.3 | 3.7 |
| Xing 4.0 29B-A4B (MoE) | 3.1215e+10B | 20.2 | 3.8 |
| Qwen3 30B-A3B (MoE) | 3.05321e+10B | 20.2 | 3.8 |
| Muse Glimmer 30B | 2.97766e+10B | 19.2 | 4.8 |
| Qwen3.6 27B | 2.77814e+10B | 18.3 | 5.7 |
| Qwen3.8 27B | 2.77814e+10B | 18.3 | 5.7 |
| Gemma 4 26B-A4B (MoE) | 2.58059e+10B | 17.0 | 7.0 |
| gpt-oss-20b | 2.09148e+10B | 13.7 | 10.3 |
| Gemma 4 12B | 1.19597e+10B | 8.6 | 15.4 |
| Ornith 1.5 9B | 9.6531e+09B | 6.8 | 17.2 |
| +8 more in the calculator | |||
Need = weights at q4_K_M + KV cache at 8k context, from the modelvram.com dataset (retrieved 2026-10-20, CC BY 4.0). Headroom is against the nominal 24 GiB, not against what your desktop leaves free — see the rule above.
The near misses (the honest part)
These miss your 24GB at q4_K_M by 2 GiB or less, at their smallest shipped quant nothing else helps:
| model | needs | verdict |
|---|---|---|
| none within 2 GiB | — | — |
A card with no model page above still means “no dedicated page yet”, not “not in the dataset”: 46 models are in the calculator, 13 have their own page.
What each quant buys you at 24GB
Count of dataset models that fit at that exact quantisation, same 8k context: native 10 · bf16 9 · fp8 11 · q8_0 11 · q6_k 12 · q5_k_m 19 · q4_k_m 22 · q3_k_m 22 · q2_k 22.
Not a discrete card?
A Mac with 24GB of unified memory is not this page: the same number has to hold macOS and your apps too, so the budget we apply there is 16.0 GiB, not 24 GiB — our rule, labelled on those pages. Apple pages at this size: Mac mini (M4, 2024) 24GB.
Head-to-heads in this class
RTX 3090 24GB vs RTX 4090 24GB · RTX 4090 24GB vs RTX 5090 32GB · RTX 3090 24GB vs RTX 5090 32GB · RTX 5080 16GB vs RTX 4090 24GB · RTX 4090 24GB vs A100 80GB · RTX 4090 24GB vs Mac 128GB unified · RTX 3090 24GB vs RTX 5080 16GB · RTX 3090 24GB vs Mac 64GB unified · model VRAM pages · used-card break-even
FAQ
Is 24GB of VRAM enough for a local LLM?
For 22 of the 46 models in our dataset, yes, at q4_K_M with 8k context — that is exactly the table above, and the near-miss table shows the ones that fall within 2 GiB of the edge. If you have one model in mind, ask the calculator: it answers for 46 models at nine quantisations and any context length.
How much does quantisation actually save on a 24GB card?
On this budget the same dataset fits: native 10 · bf16 9 · fp8 11 · q8_0 11 · q6_k 12 · q5_k_m 19 · q4_k_m 22 · q3_k_m 22 · q2_k 22 models (q4_K_M is the default most people start from). We publish counts, not quality judgements — which quant is acceptable is a task-by-task call and no source we trust scores it for your prompt.
Does 24GB of VRAM mean 24GB for my model?
No. We budget the full 24 GiB because nobody publishes an OS-by-OS usable figure (no public data (we checked, 2026-10-24)); a compositor and a connected display take a slice of it, so treat these numbers as an upper bound and keep headroom. Context is the other bill: the tables are written at 8k, and the KV cache grows with every extra token window — the calculator scales it.
Other sizes
8GB · 12GB · 16GB · 32GB · all five in one matrix
GiB figures computed from the modelvram.com open dataset (CC BY 4.0 / MIT, retrieved 2026-10-20) by the calculator formula: weights at your quant + KV cache at 8k context. Card capacities: vendor spec sheets, captured 2026-10-20. Budget rule: ours, stated on the page. Check date 2026-10-24.