Local AI Glossary
What is ExLlamaV2?
Source: local-llm.net/glossary
A high-performance CUDA-only inference engine for running quantized LLMs on NVIDIA GPUs. Known for achieving the fastest token generation speeds, particularly with EXL2 and GPTQ quantized models. Does not support CPU inference or non-NVIDIA hardware.
How ExLlamaV2 fits in the local AI stack
ExLlamaV2 is part of the broader local AI ecosystem. See the related guides, comparisons, and tools below to see how it applies in practice.
See also
- Full Local AI Glossary
- All learn topics
- What is Local AI?
- Quantization explained
- Hardware requirements
- How to choose a local LLM