Local AI Glossary
What is Tensor parallelism?
Source: local-llm.net/glossary
A model parallelism strategy where individual layers are split across multiple GPUs, with each GPU computing a portion of every layer. More communication-intensive than pipeline parallelism but eliminates pipeline bubbles. Requires fast inter-GPU connections (NVLink or high-bandwidth PCIe). Supported by vLLM.
How Tensor parallelism fits in the local AI stack
Tensor parallelism is part of the broader local AI ecosystem. See the related guides, comparisons, and tools below to see how it applies in practice.
See also
- Full Local AI Glossary
- All learn topics
- What is Local AI?
- Quantization explained
- Hardware requirements
- How to choose a local LLM