# Diffusion: art from noise · AI in Bed

Advanced level · 6 min · https://www.aiinbed.com/learn/advanced/diffusion

*How Stable Diffusion and friends turn static into pictures, one denoising step at a time.*

Image generators like Stable Diffusion, DALL·E and Midjourney are built on a beautifully odd idea: teach a network to remove noise, then start from *pure* noise and let it keep cleaning until a picture appears.

*Figure: Forward process adds noise step by step. The model learns the reverse: predicting and removing noise to recover an image.*

*Interactive explainer: Run the reverse process from static to an image.*

### Forward: destroy

Take a real image and add a little Gaussian noise. Repeat for, say, 1,000 steps until it's pure static.

This direction needs no learning; it's just maths.

### Reverse: create

Train a network (often a U-Net or a transformer) to look at a noisy image and predict **the noise** that was added.

Subtract the predicted noise, step by step, and an image emerges from static.

**Quick check:** What is a diffusion model usually trained to predict?

A. The final image in one step
B. The noise that was added to an image
C. The caption of the image
D. The next pixel

Answer: B. The standard DDPM objective trains the network to predict the added noise at each step.

### Text-to-image

To follow a prompt, the denoiser is **conditioned** on a text embedding (e.g. from CLIP or a T5 encoder) via cross-attention.

**Classifier-free guidance** pushes outputs harder towards the prompt; the "guidance scale" slider controls how strongly.

### Latent diffusion

Denoising full-resolution pixels is expensive. Stable Diffusion works in a compressed **latent space** from an autoencoder, then decodes to pixels at the end.

Same idea, far cheaper, which is why it runs on consumer GPUs.

> Pillow thought: the same denoising idea now generates video, audio, 3-D shapes and even protein structures.

**Quick check:** Why does Stable Diffusion work in latent space?

A. To make images blurrier
B. It's much cheaper than denoising every pixel directly
C. Latent space contains the text prompt
D. Pixels can't contain noise

Answer: B. Working on a compressed representation slashes compute while keeping image quality high.

**Key takeaways**

- Forward process: add noise. Reverse process: learn to remove it.
- Models predict noise, then subtract it step by step.
- Text conditioning + guidance steer the image.
- Latent diffusion makes it efficient.

**Go deeper (free sources)**

- [Denoising Diffusion Probabilistic Models](https://arxiv.org/abs/2006.11239) by Ho, Jain & Abbeel, 2020
- [What are Diffusion Models?](https://lilianweng.github.io/posts/2021-07-11-diffusion-models/) by Lilian Weng
- [The Illustrated Stable Diffusion](https://jalammar.github.io/illustrated-stable-diffusion/) by Jay Alammar
- [Hugging Face Diffusion Models Course](https://huggingface.co/learn/diffusion-course) by Hugging Face
