AI glossary
What is KV cache?
Short answer
Stored keys and values from earlier tokens that a model reuses while writing, so it only does fresh work for each new token. Faster, but uses memory.
For example
When a chatbot writes its 500th word, the KV cache means it doesn't re-process the first 499 from scratch; it reuses what it already computed.
Learn it in a free lesson
Related terms
Questions
What is KV cache in simple terms?
Stored keys and values from earlier tokens that a model reuses while writing, so it only does fresh work for each new token. Faster, but uses memory.
What is an example of KV cache?
When a chatbot writes its 500th word, the KV cache means it doesn't re-process the first 499 from scratch; it reuses what it already computed.
Where can I learn about KV cache for free?
AI in Bed explains it in the free lesson "How models are served: prefill, decode and the KV cache" (Advanced level, 8 minutes), with no sign-up.