How Much VRAM Do You Need for Local LLMs in 2026? (Tested + Tabled)

Every week on r/LocalLLaMA the same thread appears: "I have an X GB GPU, what can I run?" Most answers are guesses, opinions, or links to outdated wikis. This article gives you the calibrated answer — for every popular local AI model in 2026, exactly how much VRAM you need and what tokens/second you'll get on which GPU class.

Numbers come from public llama.cpp / ComfyUI benchmarks, the 9bench GPU-class lookup table (50+ calibrated entries), and r/LocalLLaMA community submissions. Where workloads vary (fp16 vs NF4 vs Q4 quantization), we list each path separately.

The big picture: VRAM tiers in 2026

The 12 GB tier is the 2026 sweet spot. Above it you're paying premium for a small list of additional capabilities (Llama 30B, Flux fp16). Below it you're constantly fighting OOM errors with --lowvram flags and disabled features.

Per-model VRAM requirements (sorted by what you'd actually run)

Text LLMs

Vision / multimodal

Image generation

Video generation (the new frontier in 2026)

Audio / TTS

Quantization 101 — why "Q4" matters more than "GB"

All the numbers above assume Q4 quantization (also written Q4_K_M or NF4 depending on framework). Q4 means each weight is stored as a 4-bit integer with a scaling factor instead of fp16 (16 bits). That's a 4× memory reduction with ~95% quality retention for most models.

Other quantization paths to know:

• fp16 / bf16: Full quality. 2× the VRAM of Q4. Use only if VRAM allows.
• fp8: 8-bit floating point. ~98% quality of fp16, 1.5× VRAM of Q4. Modern Hopper / Ada GPUs accelerate this directly.
• Q4_K_M: Most popular Q4 variant (llama.cpp). Balanced quality.
• NF4: Used by Bitsandbytes / Forge / ComfyUI for Flux. Similar to Q4_K_M.
• GGUF Q4: Same Q4 quality, packaged in llama.cpp's GGUF format. Most local LLM tools use this.
• AWQ / GPTQ: Activation-aware quants that beat Q4_K_M on quality benchmarks but require pre-computed quants per model. Usable in vLLM, ExLlamaV2.
• Q3 / Q2: 3-bit / 2-bit. Q3 is borderline usable for 70B+ models when 24 GB VRAM is your only option. Q2 is usually too lossy for serious work.

Practical rule of thumb: Q4_K_M is what 95% of local LLM users run. Don't worry about quant choices unless you're squeezing a 70B into 24 GB (try Q3) or optimizing throughput on a 24 GB+ workstation (try AWQ).

The KV cache trap

"But the weights are only 4.5 GB, why does my 7B model fill 8 GB VRAM?"

Answer: KV cache. Each token your model generates has to remember the keys and values from prior tokens (this is what makes attention work). The KV cache grows with context length:

• Llama 7B at 2K context: ~0.5 GB KV cache
• Llama 7B at 8K context: ~2 GB KV cache
• Llama 7B at 32K context: ~8 GB KV cache (yes, more than the weights!)
• Llama 70B at 8K context: ~5 GB KV cache

For most chat use cases, 4K context is plenty and KV cache is small. But if you're doing long-document summarization, code-base RAG, or 32K-token "agentic loops", VRAM requirements can double. Modern frameworks support KV cache quantization (Q4 KV cache) which cuts this in half with minimal quality loss.

Apple Silicon: unified memory works differently

On Apple Silicon (M1/M2/M3/M4 series), there's no separate "VRAM" — the GPU shares system RAM via unified memory architecture. By default, macOS reserves about 75% of total RAM for GPU use. So:

• M3 Max 36 GB: ~27 GB usable as VRAM
• M3 Max 64 GB: ~48 GB usable as VRAM (runs Llama 70B!)
• M3 Max 96 GB: ~72 GB usable
• M3 Max 128 GB: ~96 GB usable
• M3 Ultra 192 GB: ~144 GB usable (runs 120B+ models)

You can manually raise the GPU memory limit with sudo sysctl iogpu.wired_limit_mb=<value>. Most Mac LLM tools (LM Studio, Ollama, MLX) document this. It's the single most-skipped setup step on Mac.

What if you're VRAM-poor? The workarounds that actually work

1. Quantization is your friend

Q4 instead of fp16 = 4× less VRAM. AWQ / GPTQ instead of Q4_K_M = sometimes better quality at same VRAM. Don't run fp16 unless you have 24 GB+ to spare.

2. CPU offload (last resort, not first)

llama.cpp / Ollama / LM Studio can offload some layers to CPU RAM if VRAM is full. Speed cost: 3-10× slowdown. Use only if a model truly doesn't fit.

3. Reduce context length

Setting max context from 8K to 4K saves significant VRAM. Most chats don't need more. Code agents and document QA do — there's no free lunch there.

4. KV cache quantization

Modern llama.cpp supports Q8/Q4 KV cache. Cuts KV cache size in half with negligible quality impact. Set --cache-type-k q4_0 --cache-type-v q4_0.

5. Skip the refiner / fp16 image gen

SDXL refiner doubles peak VRAM. Disable it if you're on 8 GB. Use Flux.1-schnell (4 steps) instead of Flux.1-dev (20 steps). Quality drop is real but small for many use cases.

6. Use cloud for the rare big-model jobs

Renting an H100 hour costs $2-4 in 2026. If you only need Llama 70B 5 hours/month, cloud is cheaper than buying a 48 GB GPU. Local makes sense for daily use; cloud makes sense for spikes.

How 9bench tells you what fits your VRAM

Run 9bench.com in your browser. The result page detects your GPU via WEBGL_debug_renderer_info, looks up its known VRAM (50+ GPU classes calibrated), and shows you a feasibility table with:

• Llama 7B / 13B / 70B / Qwen2.5-Coder 32B / Qwen2-VL 7B (text + multimodal LLMs)
• SDXL, SD 1.5, Flux.1-dev, Flux.1-schnell (image gen)
• HunyuanVideo, LTX-Video (video gen)
• Whisper (audio)

Each row shows YES / MAYBE / NO with calibrated tokens-per-second or seconds-per-image for your specific GPU class. No download. No signup. 15 seconds.

Common questions

"Does the GPU brand matter?" For VRAM calculations, no — GB is GB. For speed, yes: NVIDIA + CUDA gets the most ecosystem support; AMD ROCm works on Linux, is flaky on Windows; Apple Silicon excels at memory capacity but pays a 2-4× speed penalty vs same-VRAM NVIDIA on most workloads.

"Can I add more VRAM to my GPU?" No, not on consumer cards. VRAM is soldered. (There are mod communities for the RTX 3070→16 GB but it's specialty work.) Buy a card with the VRAM you need from the start.

"Is unified memory always 'good enough' as VRAM?" For inference: yes. For training: no. Apple Silicon LLM inference is competitive, but research papers ship CUDA code, and MLX still lacks some kernels. If you do research, NVIDIA. If you do inference + content creation, Apple is fine.

"What about Strix Halo / AMD Ryzen AI Max+ 395?" The new AMD APU with up to 96 GB unified memory (LPDDR5X 8000) is genuinely interesting for local AI in 2026. Performance is closer to M3 Pro than M3 Max in current Linux benchmarks, but VRAM capacity is competitive with M3 Max. Watch this space — by mid-2026 there should be enough public benchmarks to call it definitively.

"How much RAM do I need (separate from VRAM)?" 16 GB system RAM for casual local LLM, 32 GB for serious work, 64 GB if you do offloading. Apple unified memory makes this simpler — one number for everything.