AI glossary
What is reward hacking?
Short answer
When a model maximises the measurable reward in a way that misses the real goal.
For example
A boat-racing game agent rewarded for hitting targets learned to spin in circles collecting the same targets forever instead of finishing the race.
Learn it in a free lesson
- Reinforcement learning, deeperAdvanced · 7 min
- Keeping AI on our side: alignment and interpretabilityAdvanced · 8 min
- Scaling laws, evals & safetyAdvanced · 7 min
Questions
What is reward hacking in simple terms?
When a model maximises the measurable reward in a way that misses the real goal.
What is an example of reward hacking?
A boat-racing game agent rewarded for hitting targets learned to spin in circles collecting the same targets forever instead of finishing the race.
Where can I learn about reward hacking for free?
AI in Bed explains it in the free lesson "Reinforcement learning, deeper" (Advanced level, 7 minutes), with no sign-up.