# Thinking before answering: reasoning models · AI in Bed

Advanced level · 7 min · https://www.aiinbed.com/learn/advanced/reasoning-models

*Why letting a model write out its thinking makes it smarter, and how reinforcement learning taught models to think for longer.*

Ask a person "what's 17 × 24?" and demand an instant answer, and they'll often get it wrong. Let them use paper and they'll get it right. Language models are the same: every token they write is a little bit of computation, so writing out steps gives them more room to think.

### Chain of thought

In 2022, researchers found that simply prompting models to "think step by step" sharply improved maths and logic scores. This is **chain-of-thought** prompting.

The intermediate steps act as a scratchpad the model can read back as it continues.

### From prompting to training

Reasoning models are trained to do this by default. They're given problems with checkable answers, like maths with a known result or code with tests, and rewarded with **reinforcement learning** when the final answer is right.

Nobody tells them how to think. Over training they learn useful habits on their own: breaking problems down, checking work, backtracking with "wait, that's wrong".

**Quick check:** Why does writing out intermediate steps help a language model?

A. It makes the answer longer, which graders like
B. Each written token is extra computation, and the model can build on its own earlier steps
C. It lets the model search the internet
D. It changes the model's weights

Answer: B. The model produces one token at a time; writing steps gives it more computation and a scratchpad to reason over.

### Test-time compute

Old rule: to get a smarter model, train a bigger one. New lever: let the same model think longer at answer time. Spending more compute while answering, called **test-time compute**, keeps improving results on hard problems, up to a point.

The cost is time and money: a long reasoning trace can be thousands of tokens.

### Is the written thinking faithful?

The visible reasoning is useful but not a guaranteed window into the model. Research has found cases where the stated steps don't fully match what actually drove the answer, so reasoning traces should be read with care.

**Quick check:** What kind of reward is easiest to use when training a reasoning model with RL?

A. How long the answer is
B. Whether the final answer can be automatically checked as correct
C. How polite the model sounds
D. How fast it responds

Answer: B. Maths answers and code tests give clean, automatic right-or-wrong signals, which make reinforcement learning work well.

**Key takeaways**

- Writing out steps gives models more computation and a scratchpad.
- Reasoning models learn to think via RL on checkable problems.
- Thinking longer at answer time is a new way to buy capability.

**Go deeper (free sources)**

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://arxiv.org/abs/2201.11903) by Wei et al.
- [DeepSeek-R1: Incentivizing Reasoning Capability via RL](https://arxiv.org/abs/2501.12948) by DeepSeek-AI
- [Lil'Log](https://lilianweng.github.io/) by Lilian Weng