Home › GPUs › RTX 4090 24GB
RTX 4090 24GB for local LLMs
Usable memory: 24 GiB. Launch MSRP $1,599 (Wikipedia GeForce RTX 40 series (launch 2022-10-12), retrieved 2026-10-20).
Models that fit (best sensible quant, 8k context)
| model | quant | need GiB |
|---|---|---|
| LightOnOCR-3 4B | native | 10.1 |
| Limite 1B Violetto | native | 2.8 |
| Ornith 1.5 35B-A3B (MoE) | q4_k_m | 22.9 |
| Ornith 1.5 9B | native | 20.6 |
| Qwen3.6 35B-A3B (MoE) | q4_k_m | 22.9 |
| Qwen3.6 27B | q5_k_m | 21.2 |
| Qwen3.8 27B | q5_k_m | 21.2 |
| Qwen3.5 9B | native | 20.6 |
| GLM-4.7 Flash | q5_k_m | 23.6 |
| Gemma 4 31B | q4_k_m | 21.9 |
| Gemma 4 26B-A4B (MoE) | q6_k | 22.7 |
| Gemma 4 12B | q8_0 | 14.2 |
| Gemma 4 E4B | native | 17.1 |
| Nemotron 3 Nano 4B | native | 8.8 |
| Nemotron 3 Nano 30B-A3B (MoE) | q5_k_m | 23.5 |
| Xing 4.0 29B-A4B (MoE) | q5_k_m | 23.6 |
| Muse Glimmer 30B | q5_k_m | 22.3 |
| MiniCPM5 2B | native | 6.0 |
| Llama 3.1 8B | native | 18.1 |
| Qwen3 8B | native | 18.5 |
| Qwen3 30B-A3B (MoE) | q5_k_m | 23.5 |
| gpt-oss-20b | q8_0 | 23.5 |
Rent-vs-buy
Cheapest rental with ≥24GB: RunPod RTX 5090 32GB @ $0.99/h (modelvram.com public table (self-stated check 2026-09-29, retrieved 2026-10-20). Partial re-verify 2026-10-20: Lambda A100 40GB $1.99/h CONFIRMED on lambda.ai/pricing (fetched directly); Lambda H100 rows ambiguous on-page ($3.99 vs $4.09 configs) → not adopted; RunPod & Vast pricing pages are JS-rendered, no extractable static figures — verify rates visually before spending).
Head-to-heads
compare · compare · compare · compare · compare
FAQ
Can the RTX 4090 24GB run large local LLMs?
It fits 22 of the 46 models in our dataset at a usable quantisation with 8k context — table above. Dataset: modelvram.com CC BY 4.0, retrieved 2026-10-20.
Should I buy a RTX 4090 24GB or rent instead?
Rent-vs-buy is a break-even question: Cheapest rental with ≥24GB: RunPod RTX 5090 32GB @ $0.99/h (modelvram.com public table (self-stated check 2026-09-29, retrieved 2026-10-20). Partial re-verify 2026-10-20: Lambda A100 40GB $1.99/h CONFIRMED on lambda.ai/pricing (fetched directly); Lambda H100 rows ambiguous on-page ($3.99 vs $4.09 configs) → not adopted; RunPod & Vast pricing pages are JS-rendered, no extractable static figures — verify rates visually before spending). Divide your street price by that rate; below ~500–1000 hours/quarter renting usually wins, above that buying. Enter real numbers in the calculator.