Qwen 3

What is Qwen 3?

Qwen 3 is Alibaba's MoE open-weights language model. Released in 2025-04, it ships in 8 sizes (0.6B, 1.7B, 4B, 8B, 14B, 30B, 32B, 235B) and is licensed Apache 2.0 (most sizes). The most popular use case is best all-round local LLM in 2025-2026.

VRAM and hardware

The smallest 0.6B size needs at least 1 GB VRAM; the largest 235B needs around 5 GB.

Size	Min VRAM (Q4_K_M)	Recommended VRAM	Tokens/sec on 3090
0.6B	1 GB	2 GB	~34 tok/s
1.7B	1 GB	2 GB	~34 tok/s
4B	1 GB	2 GB	~34 tok/s
8B	5 GB	8 GB	~25 tok/s
14B	9 GB	14 GB	~22 tok/s
30B	18 GB	27 GB	~18 tok/s
32B	20 GB	30 GB	~17 tok/s
235B	130 GB	195 GB	~11 tok/s

How to run Qwen 3 locally

Option 1: Ollama (simplest)

ollama pull qwen-3
ollama run qwen-3

Option 2: Mullama (production)

mullama pull qwen-3
mullama run qwen-3

Option 3: llama.cpp (CLI)

# Download a GGUF from Hugging Face (search "qwen-3 gguf")
./llama-cli -m qwen-3.Q4_K_M.gguf -p "Hello, AI!"

Option 4: Python with Mullama or llama-cpp-python

from mullama import Model, Context
model = Model.load("qwen-3.Q4_K_M.gguf", n_gpu_layers=99)
ctx = Context(model, n_ctx=4096)
print(ctx.generate("Hello, AI!", 256))

What you can build with Qwen 3

• chat
• code
• reasoning
• RAG
• agentic

Hardware it fits on

• Apple Silicon (8GB): 0.6B in MLX
• Apple Silicon (16GB): 1.7B in MLX
• Apple Silicon (32GB): 4B in MLX or GGUF
• Apple Silicon (64GB+): 235B in MLX
• RTX 3090 (24GB): 4B at Q4_K_M
• RTX 4090 (24GB): same as 3090, ~30% faster
• RTX 5090 (32GB): 4B at Q8_0
• 2× RTX 6000 Ada (96GB total): 235B at Q4_K_M

Related models in the Qwen family

• Qwen 2.5 — 0.5B, 1.5B, 3B, 7B, 14B, 32B, 72B · strongest 7B-32B on consumer hardware
• Qwen 2.5-Coder — 0.5B, 1.5B, 3B, 7B, 14B, 32B · best open-weight code model in 2024-2025
• Qwen 3-Coder — 30B, 480B · long-context code agent workflows
• Qwen 3 VL — 2B, 4B, 8B, 30B, 32B, 235B · frontier-class vision-language with thinking

Other models you might consider

• Llama 3.1 (Meta) — 8B, general-purpose chat, agentic workflows, code
• Llama 3.2 (Meta) — 1B, low-end hardware, mobile, edge deployment
• Llama 3.2 Vision (Meta) — 11B, vision + chat on desktop GPUs
• Llama 3.3 (Meta) — 70B, best-in-class 70B for desktops
• Llama 4 (Meta) — Scout 17B (109B MoE), vision + long context (10M tokens)
• DeepSeek R1 (DeepSeek) — 1.5B, reasoning, math, step-by-step problem solving