Local AI Glossary
What is AWQ 4-bit?
Source: huggingface/AutoAWQ
A 4-bit variant of Activation-Aware Weight Quantization, optimized for fast inference on NVIDIA Ampere and newer GPUs. Used in vLLM and TGI for production serving. Achieves better quality per byte than GPTQ at 4-bit.
How AWQ 4-bit fits in the local AI stack
AWQ 4-bit is part of the broader local AI ecosystem. See the related guides, comparisons, and tools below to see how it applies in practice.
See also
- Full Local AI Glossary
- All learn topics
- What is Local AI?
- Quantization explained
- Hardware requirements
- How to choose a local LLM