AI glossary
What is eval?
Short answer
A test suite for an AI model or app: example inputs plus a way to judge whether each output is good.
For example
An eval for a support bot might be 200 real customer questions plus a check, by a person or another model, of whether each answer is correct and polite.
Learn it in a free lesson
- How we test language modelsIntermediate · 7 min
- Keeping AI on our side: alignment and interpretabilityAdvanced · 8 min
- Scaling laws, evals & safetyAdvanced · 7 min
Questions
What is eval in simple terms?
A test suite for an AI model or app: example inputs plus a way to judge whether each output is good.
What is an example of eval?
An eval for a support bot might be 200 real customer questions plus a check, by a person or another model, of whether each answer is correct and polite.
Where can I learn about eval for free?
AI in Bed explains it in the free lesson "How we test language models" (Intermediate level, 7 minutes), with no sign-up.