Local AI Glossary
What is Memory bandwidth?
Source: local-llm.net/glossary
The rate at which data can be read from or written to memory, measured in GB/s. Memory bandwidth is the primary bottleneck for LLM token generation speed. NVIDIA GPUs offer 300-1,000+ GB/s, Apple Silicon offers 100-800 GB/s, and DDR5 RAM offers 50-100 GB/s. Higher bandwidth means faster tok/s.
How Memory bandwidth fits in the local AI stack
Memory bandwidth is part of the broader local AI ecosystem. See the related guides, comparisons, and tools below to see how it applies in practice.
See also
- Full Local AI Glossary
- All learn topics
- What is Local AI?
- Quantization explained
- Hardware requirements
- How to choose a local LLM