AI glossary
What is inference?
Short answer
Using a trained model to make predictions, with its parameters frozen.
For example
Every time you send a message to a chatbot and it replies, that reply is inference: the model is used, not changed.
Learn it in a free lesson
- How machines learnBeginner · 5 min
- How LLMs are trainedAdvanced · 7 min
- Fine-tuning on a budget: LoRA & quantisationAdvanced · 6 min
- How models are served: prefill, decode and the KV cacheAdvanced · 8 min
Related terms
Questions
What is inference in simple terms?
Using a trained model to make predictions, with its parameters frozen.
What is an example of inference?
Every time you send a message to a chatbot and it replies, that reply is inference: the model is used, not changed.
Where can I learn about inference for free?
AI in Bed explains it in the free lesson "How machines learn" (Beginner level, 5 minutes), with no sign-up.