Neural networks, gently
Tiny calculators, wired in layers, that together can recognise a face or write a poem.
Neural networks are loosely inspired by brains, but don't let the name intimidate you. Each "neuron" is a very simple calculator. The magic comes from wiring thousands or billions of them together.
βοΈ Weights: how much each input matters
Deciding whether to go for a run? Inputs: is it sunny, am I tired, is my friend coming.
Each input gets a weight. Sunshine might matter a lot (big weight), tiredness might count against it (negative weight).
β Add it up, then decide
The neuron adds up all the weighted inputs plus a bias (its baseline mood).
Then an activation function decides how strongly to "fire". A popular one, ReLU, simply says: if the total is negative, output 0; otherwise pass it through.
π₯ Layers make it powerful
Neurons are stacked in layers. The first layer sees raw input (pixels), middle ("hidden") layers build up features, and the last layer gives the answer.
In an image model, early layers might detect edges, middle layers shapes like eyes, and later layers whole faces.
What are a neural network's "weights"?
π How big are they?
A network that reads handwritten digits might have ~100,000 weights.
Modern large language models have billions to trillions. Same idea, wildly bigger.
Why do we need an activation function like ReLU between layers?
β¨ Before you drift off
- A neuron = weighted sum of inputs + bias β activation function.
- Layers build from simple features up to complex ones.
- Weights are learned during training.
π Go deeper (free & open)
- Neural networks (chapters 1β4) β Β· 3Blue1Brown
- TensorFlow Playground β Β· Daniel Smilkov & Shan Carter (Google) Β· Apache 2.0, interactive in your browser
- Neural Networks and Deep Learning (free online book) β Β· Michael Nielsen Β· CC BY-NC 3.0