Local LLM on Apple M3 Ultra

Is the Apple M3 Ultra the right card for this?

Unified memory changes the arithmetic. There is no separate VRAM pool to fill, so the whole 192 GB is addressable by the model, and at 60 W this machine draws less than a single desktop GPU. The trade is bandwidth: 800 GB/s is a fraction of what a data-centre part delivers, so very large models load and run rather than run fast. At about $4000, or roughly $21 per gigabyte of usable memory, it is the cheapest way to hold a model this size in one box. Catalogue throughput class: FP16 70B: 18-25, FP16 405B: 6-10.

One other card here carries 192 GB, so the two load the same weights and differ only in how fast they read them. At 800 GB/s this is not the quickest of that group, so its argument is price rather than speed.

A 2024 part sits in the middle of this guide's range: old enough that backend support has stopped moving and the community has written up the quirks, new enough that nothing treats it as legacy. That is the least risky place on the calendar to buy from if you want the setup to work the first time.

What the price gap buys

Nothing else in this guide holds 192 GB for less. The fastest alternative at this capacity is the NVIDIA B200, which costs $36000 more for 900% more bandwidth and not one extra model. If the models you want already fit here, the money buys speed and nothing else.

At 2 years old this is still a current retail part, so the price above is a shop price rather than a listing price and stock is the usual constraint rather than condition. Buying new also means the 60 W figure is the number your power supply has to meet from day one.

Where the Apple M3 Ultra sits in this guide

Across the 20 cards catalogued here, the Apple M3 Ultra ranks 11th on memory bandwidth at 800 GB/s, 15th cheapest at about $4000, and 1st on bandwidth per watt at 13.3 GB/s per watt. Those three positions, not the capacity figure, are what separate it from other 192 GB parts.

Unified LPDDR5 is shared between the CPU and the GPU on this part, which is what makes 192 GB usable by a model at all on a machine this size. The trade is 800 GB/s: enough to hold very large models, not enough to run them at the speed a discrete card manages on smaller ones.

On Apple hardware the fast path is MLX, Apple's own array framework, with Ollama as the simpler alternative running through Metal. Our card records for Apple parts cite mlx-lm benchmarks for exactly this reason: the same machine gives materially different numbers depending on which of the two you use.

Which catalogue models fit on Apple M3 Ultra?

Of the 46 models in our catalogue with per-size memory figures, 45 have at least one size that fits in 192 GB at Q4_K_M with room for context. The largest of each are below, biggest first.

Model	Largest size that fits	Weights at Q4_K_M
DeepSeek Coder V2	236B	140 GB
Qwen 3	235B	130 GB
minimax M2	230B	130 GB
minimax M2.1	230B	130 GB
minimax M2.5	230B	130 GB
minimax M3	230B	130 GB
Qwen 3 VL	235B	130 GB
Mistral Large	123B	70 GB

A further 37 smaller models also fit; see the model catalogue.

The next sizes up are out of reach without offloading: Llama 3.1 405B at 240 GB, Hermes 3 405B at 240 GB, Qwen 3-Coder 480B at 280 GB.

Expected tokens per second on Apple M3 Ultra

Memory bandwidth, not compute, is the bottleneck for single-stream token generation. The Apple M3 Ultra's 800 GB/s puts the following ceiling on a batch of one at 2048 tokens of context.

Model	Quantization	Status	Approx tokens/sec
Llama 3.2 1B	Q4_K_M	Fits in VRAM	~144 tok/s
Llama 3.1 8B	Q4_K_M	Fits in VRAM	~48 tok/s
Llama 3.1 8B	Q8_0	Fits in VRAM	~36 tok/s
Mistral Nemo 12B	Q4_K_M	Fits in VRAM	~32 tok/s
Qwen 2.5 14B	Q4_K_M	Fits in VRAM	~28 tok/s
Qwen 3 32B	Q4_K_M	Fits in VRAM	~14 tok/s
DeepSeek R1 distilled 32B	Q4_K_M	Fits in VRAM	~14 tok/s
Llama 3.1 70B	Q4_K_M	Fits in VRAM	~8 tok/s
Llama 4 Scout 109B (MoE)	Q4_K_M	Fits in VRAM	~6 tok/s

These come from a bandwidth heuristic (tokens/sec ≈ bandwidth in GB/s × a per-model efficiency factor), not from a test rig in our office. Real numbers move with model architecture, batch size and KV-cache size. Treat them as an order of magnitude, and treat the "reported throughput class" row in the table above as the community-measured figure.

Build recommendations for Apple M3 Ultra

Best model to download first

Llama 3.1 8B to check the stack works, then the largest model in the table above that you actually need. At 192 GB the constraint stops being capacity. Start with ollama pull llama3.1:8b.

Recommended inference backend

For Apple Silicon, use MLX (best performance) or Ollama (simplest setup). Mullama works via llama.cpp but won't beat MLX on Apple Silicon.

Sources

[1] apple.com/mac-studio · [2] mlx-lm benchmarks · VRAM figures for catalogue models from src/data/models.json (2026-06-29); card specifications from src/data/gpus.json (2026-06-29).