2026-09-29
Qwen3 VRAM Requirements (2026): How to Run Every Size Locally with Ollama
Qwen3 has quietly become the default open-weight family for home labs: Apache 2.0 licensed, sizes from 0.6B to 235B, a genuinely useful mixture-of-experts (MoE) variant that gives 30B-class quality at 8B-class speed, and first-class Ollama support. The question everyone asks before pulling a 20 GB download is the same one: will it fit on my GPU?
This guide answers that for every Qwen3 size, with the real numbers — actual GGUF download sizes, KV-cache overhead, and the context-window default that makes people wrongly conclude the model is broken. I'll also flag where the marketing math ("30B model, only 3B active!") will mislead you on memory.
The 30-Second Answer
- 8 GB GPU (RTX 3060 Ti, RTX 4060):
qwen3:8b— the sweet spot for most people. - 12 GB GPU (RTX 3060 12GB):
qwen3:8bcomfortably,qwen3:14bat the edge. - 16 GB GPU (RTX 4060 Ti 16GB, RTX 5060 Ti 16GB):
qwen3:14bwith room for real context. - 24 GB GPU (RTX 3090, RTX 4090):
qwen3:32b(tight) or — better —qwen3:30b-a3b, the MoE. - Mac with 16 GB unified memory:
qwen3:8b; 32 GB:qwen3:14b; 64 GB+:qwen3:32bor the MoE.
Rule of thumb: at FP16 a model needs roughly 2 GB of VRAM per billion parameters; at 4-bit (what Ollama pulls by default) roughly 0.5 GB per billion — plus 15–20% for KV cache and overhead. That overhead line is where most "it should fit" calculations die.
Qwen3 VRAM Requirements: The Full Table
These are Ollama's official tags, which download pre-quantized Q4_K_M GGUF files. Download size ≠ VRAM needed — you must leave headroom for the KV cache or you'll hit out-of-memory the moment you paste a long document.
| Ollama command | Download size | Minimum practical VRAM | Sensible GPU |
|---|---|---|---|
ollama pull qwen3:0.6b |
~523 MB | 2–4 GB | Anything, even iGPU/CPU |
ollama pull qwen3:1.7b |
~1.4 GB | 4 GB | GTX 1650 class |
ollama pull qwen3:4b |
~2.5 GB | 6 GB | RTX 3050 / 3060 |
ollama pull qwen3:8b |
~5.2 GB | 8 GB | RTX 3060 Ti / 4060 |
ollama pull qwen3:14b |
~9.3 GB | 12–16 GB | RTX 4060 Ti 16GB / 4070 |
ollama pull qwen3:32b |
~20 GB | 24 GB (tight) | RTX 3090 / 4090 |
ollama pull qwen3:30b (30B-A3B MoE) |
~19 GB | 20–24 GB | RTX 3090 / 4090 |
ollama pull qwen3:235b (235B-A22B MoE) |
~142 GB | Not viable on consumer hardware | 2× H100 territory |
Two things in that table trip people up, so let's spell them out.
Why the downloads are bigger than the 4-bit math says
Qwen3-8B at pure INT4 computes to about 4 GB, but the actual GGUF download is 5.2 GB — 20–30% larger. That's Q4_K_M's mixed-precision scheme keeping sensitive tensors (embeddings, attention output layers) at higher precision to protect quality. It's a real cost worth knowing before you assume a model fits your card by parameter math alone.
MoE models: size by TOTAL parameters, not active
Qwen3-30B-A3B activates only ~3B parameters per token (8 of 128 experts), so it generates tokens about as fast as an 8B model. But all 30B parameters must sit in memory — the ~19 GB download reflects that. The "A3B" buys you speed and quality-per-watt, not a smaller memory footprint. Same story at the top end: 235B-A22B needs 140 GB+ resident regardless of its 22B active figure.
Which GPU Should You Buy for Qwen3?
If you're shopping specifically to run Qwen3, here's how the tiers shake out on price/performance (US street prices, late September 2026; community throughput figures, not my benchmarks):
| Budget | Card | Best Qwen3 fit | Reported speed |
|---|---|---|---|
| ~$300 | RTX 3060 12GB | qwen3:8b Q4 | ~30 tok/s |
| ~$480–530 | RTX 5060 Ti 16GB | qwen3:14b Q4 | ~28 tok/s class |
| ~$700–800 (used) | RTX 3090 24GB | qwen3:30b-a3b | ~25 tok/s class |
| ~$1,600+ | RTX 4090 / 5090 | qwen3:32b Q4 or 30b-a3b | 22–25+ tok/s |
The value pick for this model family is a used RTX 3090: 24 GB is exactly the threshold that unlocks the 30B-A3B MoE, which is the best quality-per-dollar model Qwen ships. We compared the used-3090 route against buying new in our used RTX 3090 vs RTX 5060 Ti 16GB comparison.
On Apple Silicon, unified memory is your VRAM. Community numbers: M1/M2 16GB runs qwen3:8b at ~18 tok/s; M3 Max 64GB runs qwen3:32b at ~18 tok/s. If you're weighing a Mac against a dedicated GPU box, see our Mac mini M4 Pro vs Ryzen AI Max mini-PC breakdown.
Step-by-Step: Running Qwen3 with Ollama
1. Install Ollama
# macOS / Linux
curl -fsSL https://ollama.com/install.sh | sh
ollama --version
Windows users: grab the installer from ollama.com — it runs natively with CUDA, no WSL2 required.
2. Pull the size that fits your card
ollama pull qwen3:8b # 8 GB VRAM
ollama pull qwen3:14b # 12–16 GB VRAM
ollama pull qwen3:30b # 24 GB VRAM — the MoE, best 24GB choice
Always use an explicit size tag. The untagged qwen3 resolves to a default that can change between Ollama releases.
3. Fix the context window before you do anything else
This is the single most common "Qwen3 is broken" report, and it's almost never the model. Ollama defaults to a tiny context window (2,048–4,096 tokens) regardless of the 128K the model supports. Exceed it and Ollama silently truncates from the beginning — no error, no warning. The model appears to "forget" your instructions or the top of a pasted file.
Fix it durably with a Modelfile:
cat > Modelfile << 'EOF'
FROM qwen3:8b
PARAMETER num_ctx 32768
EOF
ollama create qwen3-32k -f Modelfile
ollama run qwen3-32k
Budget for it: going from 4K to 32K context on the 8B model adds roughly 1–2 GB of KV cache. That's why an 8 GB card running the 8B model is comfortable at 8–16K context but starts spilling to system RAM at 32K. For most chat, coding, and RAG work, 16K–32K is the sweet spot; reserve 128K for whole-repo tasks on hardware that can afford it.
4. Toggle thinking mode
Qwen3's party trick is a switchable chain-of-thought mode. Inside an interactive session:
/set think # deep reasoning — math, multi-step coding
/set nothink # fast direct answers — chat, RAG, summarization
You can also toggle per-message with /think and /no_think in your prompt. Thinking mode burns more tokens (slower, more VRAM for the longer generations) but measurably helps on multi-step problems.
5. Use the API
Ollama exposes an OpenAI-compatible endpoint on localhost:11434 automatically — point Continue.dev, Open WebUI, or your own scripts at it with no extra configuration.
Dense 32B vs MoE 30B-A3B: Which One on a 24 GB Card?
Both fit (barely) on 24 GB. They're different tools:
| Qwen3-32B (dense) | Qwen3-30B-A3B (MoE) | |
|---|---|---|
| Download (Q4_K_M) | ~20 GB | ~19 GB |
| Active params/token | 32B | ~3B |
| Generation speed | ~22 tok/s (4090 class) | ~25 tok/s, feels much snappier |
| Context headroom on 24 GB | 3–4 GB — a few thousand tokens | Similar, slightly better |
| Best at | Consistent quality, fine-tuning target | Throughput, interactive use |
My take for a home lab: pull the MoE first. You get near-32B quality at markedly better interactive speed, and speed is what you feel every day. Choose the dense 32B if you plan to fine-tune or need maximally predictable latency.
The honest caveat for both: ~20 GB of weights on a 24 GB card leaves only 3–4 GB for KV cache. That caps context at a few thousand tokens before OOM — short of what serious coding sessions need. If long context on a 30B-class model is your actual requirement, you want 32 GB+ (RTX 5090, dual GPUs, or a 64 GB Mac).
When Local Isn't Enough
Three things push you off a local box: a model your GPU can't fit even at 4-bit (the 235B MoE), context lengths your KV-cache headroom can't cover, or more than a couple of concurrent users. Ollama is built for a single interactive session; for real concurrency you graduate to vLLM (vllm serve Qwen/Qwen3-32B --quantization fp8) on a rented H100-class GPU — cloud GPU rentals started around $0.72/hr as of mid-September 2026. For a homelab, that's usually a burst tool, not a replacement.
FAQ
How much VRAM does Qwen3-8B need?
The Q4_K_M build Ollama pulls is a 5.2 GB download; with KV cache and overhead you want 8 GB of VRAM for comfortable use at moderate context. It runs on a 6 GB card with reduced context, and on any 16 GB Apple Silicon Mac.
Can I run Qwen3-32B on an RTX 4090?
Yes — the ~20 GB Q4_K_M fits in 24 GB, but with only 3–4 GB left for KV cache, expect to cap context at a few thousand tokens. For the same card, qwen3:30b-a3b gives comparable quality with better interactive speed.
Is Qwen3 free for commercial use?
Yes. All open-weight Qwen3 models ship under Apache 2.0 — no usage restrictions, no license fees. Note that Alibaba's newest flagship (Qwen3.7-Max) is API-only with no open weights, so it can't run locally.
Why does Qwen3 forget the start of my prompt?
Ollama's default context window (2,048–4,096 tokens), not the model. Ollama truncates silently from the top when you exceed it. Set num_ctx explicitly — 16K–32K covers most real work — ideally baked into a Modelfile so it survives model reloads.
What's the cheapest GPU that runs Qwen3 well?
A ~$300 RTX 3060 12GB runs qwen3:8b at ~30 tok/s (community figures) — fast enough that it doesn't feel like a compromise. The next meaningful step up is 16 GB (RTX 5060 Ti 16GB) for the 14B, then 24 GB for the 30B-class models.
Can Qwen3 run on a NAS or mini-PC without a GPU?
Small sizes, yes — the 0.6B/1.7B/4B run CPU-only at usable speeds for background automation. We covered the full setup in our Ollama on a Synology NAS guide. For interactive chat, 8B on CPU is the practical ceiling and it's slow.
Prices and throughput figures reflect late-September 2026 street prices and community-reported numbers (r/LocalLLaMA, Hugging Face discussions); your hardware, quantization, and context settings will move them. Nothing here is a paid placement.