Local AI Glossary
What is Attention mechanism?
Source: local-llm.net/glossary
The core innovation of the Transformer architecture. Attention allows each token in a sequence to "attend to" (consider the relevance of) every other token, creating a weighted representation of context. Self-attention computes query, key, and value matrices for each token, enabling the model to focus on the most relevant parts of the input.
How Attention mechanism fits in the local AI stack
Attention mechanism is part of the broader local AI ecosystem. See the related guides, comparisons, and tools below to see how it applies in practice.
See also
- Full Local AI Glossary
- All learn topics
- What is Local AI?
- Quantization explained
- Hardware requirements
- How to choose a local LLM