Local LLM on Raspberry Pi 5 (8GB)

Is the Raspberry Pi 5 (8GB) the right card for this?

There is no VRAM here at all; the model lives in the same LPDDR4X (shared) the operating system uses, and 25 GB/s is the hard ceiling on generation speed. That puts it firmly in tiny-model territory, and it is genuinely useful there: a 5 W board that costs about $80 and answers offline is a different proposition from a graphics card. Catalogue throughput class: Q4 1B: 5-8, Q4 3B: 1-2.

A 2023 part sits in the middle of this guide's range: old enough that backend support has stopped moving and the community has written up the quirks, new enough that nothing treats it as legacy. That is the least risky place on the calendar to buy from if you want the setup to work the first time.

What the price gap buys

No other card in this guide carries this memory arrangement, so there is no like-for-like price comparison to make. The nearest capacities are listed at the foot of this page, and moving to any of them changes which models fit, not just how fast they run.

At about $80 this is an impulse purchase next to every other entry in this guide, and it is sold as a complete device rather than as a component. That changes the question from "will it fit in my machine" to "is what it can run worth having", which the model table below answers directly.

Where the Raspberry Pi 5 (8GB) sits in this guide

Across the 20 cards catalogued here, the Raspberry Pi 5 (8GB) ranks 20th on memory bandwidth at 25 GB/s, 1st cheapest at about $80, and 7th on bandwidth per watt at 5.0 GB/s per watt. Those three positions, not the capacity figure, are what separate it from other shared-memory parts.

LPDDR4X (shared) is the same memory the operating system runs from, and at 25 GB/s it is the single number that decides everything on this board. There is no memory hierarchy to optimise around here; there is one pool, and it is slow.

There is no GPU path worth using here. llama.cpp compiled for ARM CPU inference is the whole story, and the interesting question is not which backend but which model is small enough to be useful at this speed.

Which catalogue models fit on Raspberry Pi 5 (8GB)?

No model in our catalogue fits in dedicated video memory on this device, because it has none. Everything runs from system RAM, which means the practical limit is how much RAM the board has and how patient you are. See the model catalogue for the smallest entries.

Expected tokens per second on Raspberry Pi 5 (8GB)

Memory bandwidth, not compute, is the bottleneck for single-stream token generation. The Raspberry Pi 5 (8GB)'s 25 GB/s puts the following ceiling on a batch of one at 2048 tokens of context.

Model	Quantization	Status	Approx tokens/sec
Llama 3.2 1B	Q4_K_M	System RAM only	~1 tok/s

These come from a bandwidth heuristic (tokens/sec ≈ bandwidth in GB/s × a per-model efficiency factor), not from a test rig in our office. Real numbers move with model architecture, batch size and KV-cache size. Treat them as an order of magnitude, and treat the "reported throughput class" row in the table above as the community-measured figure.

Build recommendations for Raspberry Pi 5 (8GB)

Best model to download first

CPU inference. Use the smallest model that fits in your system RAM: Llama 3.2 1B or SmolLM2 1.7B.

Recommended inference backend

Use Ollama for general use or Mullama if you need a drop-in Ollama alternative with native bindings for 6 languages. For production serving on multi-GPU setups, see vLLM.

Sources

[1] raspberrypi.com · [2] community benchmarks · VRAM figures for catalogue models from src/data/models.json (2026-06-29); card specifications from src/data/gpus.json (2026-06-29).