Six months into 2026, the local AI ecosystem has crystallized into a clear three-tier structure. This month’s report covers the state of each tier, the major model releases of June 2026, and what to buy if you’re starting fresh today.
The three tiers
Tier 1: Frontier (256GB+ VRAM or multi-GPU)
Who it’s for: Research labs, well-funded startups, large enterprise deployments.
Hardware:
- NVIDIA H100 80GB (~$30,000 used, 2026)
- NVIDIA B200 (~$40,000 new, frontier-class)
- Multi-GPU: 4× RTX 6000 Ada (192GB total, ~$27,000)
- Apple M3 Ultra / M4 Ultra Mac Studio (192GB, ~$4,500)
Best models: GLM-5.2 (744B/40B MoE), Kimi K2.7 Code (MoE), minimax M2.7 (480B), Qwen 3 480B, DeepSeek R1 671B.
Real-world takeaway: Frontier-class inference is now possible locally. You can serve models that match GPT-4o, Claude 3.5 Sonnet, and o1 on local hardware. The trade-off is the upfront cost.
Tier 2: Single-GPU (24GB VRAM) — the sweet spot
Who it’s for: Individual developers, small teams, AI-curious professionals, serious hobbyists.
Hardware:
- Used RTX 3090 (24GB, ~$700 in 2026)
- RTX 4090 (24GB, ~$1,600 new)
- RTX 5090 (32GB, ~$2,500 new)
- RTX 5070 Ti (16GB, ~$750)
- Apple M4 Max (36GB, ~$2,400 new)
Best models:
- Qwen 3 32B at Q4 — best all-round 24GB model
- GLM-4.7 Flash 30B at Q4 — strongest 30B in 2026
- DeepSeek R1 distilled 32B at Q4 — best reasoning
- Qwen 2.5-Coder 32B at Q4 — best coding
- Mistral Small 24B at Q4 — polished chat
Real-world takeaway: A used 3090 for $700 gives you Qwen 3 32B at 15-20 tok/s. That handles 90% of professional local AI use cases in 2026. The 24GB tier is the right answer for almost everyone.
Tier 3: Edge (phones, SBCs, embedded)
Who it’s for: App developers, hobbyists, anyone who needs on-device inference.
Hardware:
- Modern phone (iPhone 15 Pro, Pixel 8 Pro) — 1-3B models at 15-25 tok/s
- Mac M-series base model (8GB) — 1-3B models at 30-50 tok/s
- Raspberry Pi 5 (8GB) — 1B models at 5-8 tok/s
- Steam Deck OLED — 1-3B models at 8-20 tok/s
- Jetson Orin Nano 8GB — 1-3B models at 5-12 tok/s
Best models:
- LFM2.5 Thinking 1.2B — best 1B with reasoning
- Llama 3.2 1B / 3B — best general small models
- Qwen 2.5 0.5B / 1.5B / 3B — best multilingual small
- SmolLM2 1.7B — best 1B from HuggingFace
- Moondream 1.8B — best small vision-language model
- Llamafu / MLC LLM — frameworks for on-device apps
Real-world takeaway: Edge inference is real in 2026. A 1B model on a phone is good enough for chat UI, voice assistant, and simple agents. The hardware is what you already have.
What changed in June 2026
Big model releases
- GLM-5.2 from Z.ai — 744B/40B MoE. State-of-the-art on SWE-Bench Pro. Tier 1 hardware.
- Kimi K2.7 Code from Moonshot — code-specialized MoE. Tier 1 hardware.
- minimax M2.7 — 480B. Among the most-pulled models in mid-2026. Tier 1.
- Qwen3-Coder 480B — Qwen’s flagship code agent model. Tier 1.
- GLM-4.7 Flash 30B — the strongest 30B class model. Tier 2 (24GB sweet spot).
Trend: MoE is the new normal
Every frontier release in 2026 is MoE: GLM-5, Kimi K2, minimax M2, Qwen 3 235B, DeepSeek V3. MoE delivers frontier-class quality with substantially lower per-token compute cost. For local deployment, the active-parameter count matters more than the total count.
Trend: Reasoning models are everywhere
DeepSeek R1 distilled 32B remains the strongest reasoning model in the 24GB tier. Qwen 3 with thinking mode is competitive. GLM-4.7 Flash brings strong reasoning to 30B. LFM2.5 Thinking adds reasoning to 1.2B.
Trend: Apple Silicon catches up
The M3 Ultra and M4 Ultra Mac Studio are now credible Tier 1 hardware at a fraction of the cost of an H100 cluster. 192GB unified memory, frontier-class inference, 4-5x cheaper than the equivalent NVIDIA setup. For teams that don’t need CUDA-specific libraries, the Mac Studio is the new default.
What to buy in 2026
If you have $700 to spend
- Used RTX 3090 (24GB). Run Qwen 3 32B, GLM-4.7 Flash, DeepSeek R1 32B distilled at 15-20 tok/s. The best value in local AI.
If you have $2,500 to spend
- RTX 5090 (32GB). 30-50% faster than 3090, 32GB instead of 24GB. The best single-GPU consumer card.
- Mac Studio M4 Max (36GB). Best for Mac-native workflows (MLX, MPS). Slightly slower than 5090 on raw tok/s, better for unified-memory workflows.
If you have $5,000 to spend
- Mac Studio M4 Ultra (128GB). Run frontier models at FP16, no quantization. Best in class for Apple-native deployments.
- 2× RTX 6000 Ada workstation (96GB total). For NVIDIA-specific workflows (vLLM, TensorRT-LLM).
If you have $30,000+ to spend
- NVIDIA H100 80GB server. Frontier-class inference at scale.
- NVIDIA B200 server. Cutting-edge 2026 hardware.
Open questions for the rest of 2026
- Will Apple Silicon keep pace with NVIDIA? The M4 Ultra is competitive with consumer NVIDIA in 2026. Can Apple keep this up in the M5 generation?
- Will frontier MoE models keep scaling? The active parameter count seems to be plateauing around 30-40B. Will we see a breakthrough in 2026-H2?
- Will edge inference get traction? Phones and SBCs can run 1-3B models today. Will app developers actually ship on-device AI features?
What we’re watching
- The release of GLM-5.3, Kimi K3, minimax M3, and Qwen 4
- Apple’s M5 generation
- NVIDIA’s RTX 6000 Ada successor
- The first truly capable video generation on consumer hardware
See also
- Best local LLM 2026 — annual best-of
- State of Local AI 2026 (annual) — the full annual report
- Local LLM on RTX 3090 — $700 build
- Local LLM on Apple Silicon — Mac setup
About this series
State of Local AI is a monthly long-form report on the local AI ecosystem. Covers the major model releases, hardware trends, ecosystem shifts, and what to buy. Published the last Monday of each month. Subscribe via RSS.