Local AI Glossary
What is Speculative decoding?
Source: local-llm.net/glossary
An inference optimization where a smaller, faster "draft" model generates candidate tokens that are then verified by the larger "target" model in parallel. Tokens that the target model agrees with are accepted immediately, speeding up generation without any quality loss. Effective when the draft model is a good approximator of the target.
How Speculative decoding fits in the local AI stack
Speculative decoding is part of the broader local AI ecosystem. See the related guides, comparisons, and tools below to see how it applies in practice.
See also
- Full Local AI Glossary
- All learn topics
- What is Local AI?
- Quantization explained
- Hardware requirements
- How to choose a local LLM