Local AI Glossary
What is KV-cache (Key-Value cache)?
Source: local-llm.net/glossary
A memory structure that stores the key and value tensors computed during attention for all previously processed tokens. This cache avoids recomputing attention over the entire context at each generation step, making autoregressive generation computationally feasible. The KV-cache grows with context length and is a significant consumer of memory.
How KV-cache (Key-Value cache) fits in the local AI stack
KV-cache (Key-Value cache) is part of the broader local AI ecosystem. See the related guides, comparisons, and tools below to see how it applies in practice.
See also
- Full Local AI Glossary
- All learn topics
- What is Local AI?
- Quantization explained
- Hardware requirements
- How to choose a local LLM