AI glossary
What is quantisation?
Short answer
Storing model weights with fewer bits (e.g. 4-bit) to save memory and speed up inference.
For example
Storing each weight in 4 bits instead of 16 lets a model that needed a large server fit on a laptop, with only a small drop in quality.
Learn it in a free lesson
- Distillation and the rise of small modelsAdvanced · 7 min
- Open-weight models and running AI yourselfIntermediate · 6 min
- Fine-tuning on a budget: LoRA & quantisationAdvanced · 6 min
Related terms
Questions
What is quantisation in simple terms?
Storing model weights with fewer bits (e.g. 4-bit) to save memory and speed up inference.
What is an example of quantisation?
Storing each weight in 4 bits instead of 16 lets a model that needed a large server fit on a laptop, with only a small drop in quality.
Where can I learn about quantisation for free?
AI in Bed explains it in the free lesson "Distillation and the rise of small models" (Advanced level, 7 minutes), with no sign-up.