Local AI Glossary
What is GPTQ (Generalized Post-Training Quantization)?
Source: local-llm.net/glossary
A GPU-optimized quantization method that uses calibration data and second-order error correction to minimize quality loss during quantization. GPTQ models are stored in SafeTensors format and run on NVIDIA GPUs via ExLlamaV2, vLLM, or AutoGPTQ. Typically used for 4-bit quantization.
How GPTQ (Generalized Post-Training Quantization) fits in the local AI stack
GPTQ (Generalized Post-Training Quantization) is part of the broader local AI ecosystem. See the related guides, comparisons, and tools below to see how it applies in practice.
See also
- Full Local AI Glossary
- All learn topics
- What is Local AI?
- Quantization explained
- Hardware requirements
- How to choose a local LLM