# Overfitting & the Goldilocks model · AI in Bed

Intermediate level · 5 min · https://www.aiinbed.com/learn/intermediate/overfitting

*Too simple misses the pattern, too complex memorises the noise. How to land just right.*

A student who memorises every past exam word for word might ace those papers and then flunk the real exam. Models do the same thing. It's called **overfitting**, and fighting it is a huge part of machine learning.

*Figure: Underfit (too simple), just right, and overfit (wiggles through every point, including the noise).*

*Interactive explainer: Slide from too simple to too complex and watch the test error.*

### Underfitting

The model is too simple to capture the pattern. It does badly on training data *and* new data.

Fix: a bigger model, better features, or train longer.

### Overfitting

The model memorises the training data, noise and all. Training error is tiny but validation error is high.

Signature: the training loss keeps falling while the validation loss starts *rising*.

**Quick check:** Training accuracy: 99%. Validation accuracy: 71%. What's going on?

A. Underfitting
B. Overfitting
C. A perfect model
D. The validation set is broken for sure

Answer: B. A big gap between training and validation performance is the classic sign of overfitting.

### Ways to fight overfitting

**More data**: the best cure when you can get it.

**Data augmentation**: flip, crop, rotate images to make "new" examples.

**Regularisation** (L1/L2): penalise large weights so the model stays simple.

**Dropout**: randomly switch off neurons during training so none can be relied on too much.

**Early stopping**: stop training when validation loss stops improving.

### The bias–variance trade-off

**Bias**: error from wrong assumptions (too simple → underfit).

**Variance**: error from being too sensitive to the specific training data (too complex → overfit).

Good models balance the two.

> Pillow thought: huge modern networks are so big they *could* memorise everything, yet they often generalise well anyway. Why exactly is still an active research question.

**Quick check:** Which of these is NOT a way to reduce overfitting?

A. Dropout
B. Early stopping
C. Training on the test set
D. Data augmentation

Answer: C. Training on the test set doesn't fix overfitting; it just destroys your ability to measure it honestly.

**Key takeaways**

- Underfit = too simple; overfit = memorised the noise.
- Watch for the gap between training and validation scores.
- Fight overfitting with more data, augmentation, regularisation, dropout, early stopping.

**Go deeper (free sources)**

- [ML Crash Course: Overfitting & regularisation](https://developers.google.com/machine-learning/crash-course) by Google
- [scikit-learn: Underfitting vs. Overfitting example](https://scikit-learn.org/stable/auto_examples/model_selection/plot_underfitting_overfitting.html) by scikit-learn developers
- [Dropout paper (JMLR 2014)](https://jmlr.org/papers/v15/srivastava14a.html) by Srivastava, Hinton et al.
