Local LLM on Steam Deck OLED (AMD APU)

Is the Steam Deck OLED (AMD APU) the right card for this?

A 15 W handheld running a language model is a demonstration more than a workstation, and it is a real one: 16 GB of Unified LPDDR5 is shared with the operating system and the game you were playing, and 89 GB/s is the constraint on every token. Expect small models, short contexts and warm hands. At about $549 it is not bought for inference, but it can do it. Catalogue throughput class: Q4 7B: 8-12, Q4 3B: 15-22.

4 other cards here carry 16 GB, so all of them load the same weights and differ only in how fast they read them. At 89 GB/s this is not the quickest of that group, so its argument is price rather than speed.

A 2023 part sits in the middle of this guide's range: old enough that backend support has stopped moving and the community has written up the quirks, new enough that nothing treats it as legacy. That is the least risky place on the calendar to buy from if you want the setup to work the first time.

What the price gap buys

The cheapest 16 GB option in this guide is the NVIDIA RTX 4060 Ti 16GB at about $450. This one costs $99 more and reads memory 69% slower, so on inference alone the cheaper card wins outright. Whatever case there is for this one is a case about the rest of the device, not about the models it can run.

At about $549 this is an impulse purchase next to every other entry in this guide, and it is sold as a complete device rather than as a component. That changes the question from "will it fit in my machine" to "is what it can run worth having", which the model table below answers directly.

Where the Steam Deck OLED (AMD APU) sits in this guide

Across the 20 cards catalogued here, the Steam Deck OLED (AMD APU) ranks 18th on memory bandwidth at 89 GB/s, 5th cheapest at about $549, and 5th on bandwidth per watt at 5.9 GB/s per watt. Those three positions, not the capacity figure, are what separate it from other 16 GB parts.

Unified LPDDR5 is shared between the CPU and the GPU on this part, which is what makes 16 GB usable by a model at all on a machine this size. The trade is 89 GB/s: enough to hold very large models, not enough to run them at the speed a discrete card manages on smaller ones.

Which catalogue models fit on Steam Deck OLED (AMD APU)?

Of the 46 models in our catalogue with per-size memory figures, 32 have at least one size that fits in 16 GB at Q4_K_M with room for context. The largest of each are below, biggest first.

Model	Largest size that fits	Weights at Q4_K_M
GPT-OSS	20B	14 GB
StarCoder2	15B	10 GB
LLaVA	13B	10 GB
DeepSeek Coder V2	16B	10 GB
Qwen 2.5	14B	9 GB
Qwen 2.5-Coder	14B	9 GB
Qwen 3	14B	9 GB
DeepSeek R1	14B	9 GB

A further 24 smaller models also fit; see the model catalogue.

The next sizes up are out of reach without offloading: Mistral Small 24B at 15 GB, Magistral 24B at 15 GB, Devstral 24B at 15 GB.

Expected tokens per second on Steam Deck OLED (AMD APU)

Memory bandwidth, not compute, is the bottleneck for single-stream token generation. The Steam Deck OLED (AMD APU)'s 89 GB/s puts the following ceiling on a batch of one at 2048 tokens of context.

Model	Quantization	Status	Approx tokens/sec
Llama 3.2 1B	Q4_K_M	Fits in VRAM	~16 tok/s
Llama 3.1 8B	Q4_K_M	Fits in VRAM	~5 tok/s
Llama 3.1 8B	Q8_0	Fits in VRAM	~4 tok/s
Mistral Nemo 12B	Q4_K_M	Fits in VRAM	~4 tok/s
Qwen 2.5 14B	Q4_K_M	Fits in VRAM	~3 tok/s
Qwen 3 32B	Q4_K_M	Needs CPU offload	~0 tok/s

These come from a bandwidth heuristic (tokens/sec ≈ bandwidth in GB/s × a per-model efficiency factor), not from a test rig in our office. Real numbers move with model architecture, batch size and KV-cache size. Treat them as an order of magnitude, and treat the "reported throughput class" row in the table above as the community-measured figure.

Build recommendations for Steam Deck OLED (AMD APU)

Best model to download first

Llama 3.1 8B or Mistral Nemo 12B. Start with ollama pull llama3.1:8b.

Recommended inference backend

Use Ollama for general use or Mullama if you need a drop-in Ollama alternative with native bindings for 6 languages. For production serving on multi-GPU setups, see vLLM.

Sources

[1] steamdeck.com · [2] community benchmarks · VRAM figures for catalogue models from src/data/models.json (2026-06-29); card specifications from src/data/gpus.json (2026-06-29).