AI glossary
What is benchmark?
Short answer
A fixed set of test questions with known answers, used to compare models with a single score.
For example
MMLU, a set of multiple-choice questions across dozens of school and university subjects, is a well-known benchmark for language models.
Learn it in a free lesson
Questions
What is benchmark in simple terms?
A fixed set of test questions with known answers, used to compare models with a single score.
What is an example of benchmark?
MMLU, a set of multiple-choice questions across dozens of school and university subjects, is a well-known benchmark for language models.
Where can I learn about benchmark for free?
AI in Bed explains it in the free lesson "How we test language models" (Intermediate level, 7 minutes), with no sign-up.