# Lines and curves: regression · AI in Bed

Intermediate level · 6 min · https://www.aiinbed.com/learn/intermediate/regression

*The humble straight line is the ancestor of every neural network. Linear and logistic regression, explained.*

Before neural networks, there was the line. Understanding the two classic models, **linear regression** and **logistic regression**, gives you 80% of the intuition for everything that follows.

*Figure: Left: linear regression fits a line through points to predict a number. Right: logistic regression squashes a line into an S-curve to predict a probability.*

**Fit a line with least squares**

```python
# Hours studied -> exam score. Find the best line y = w*x + b.
xs = [1, 2, 3, 4, 5, 6]
ys = [52, 55, 61, 64, 70, 73]

n = len(xs)
mean_x = sum(xs) / n
mean_y = sum(ys) / n
w = sum((x - mean_x) * (y - mean_y) for x, y in zip(xs, ys)) / sum((x - mean_x) ** 2 for x in xs)
b = mean_y - w * mean_x

mse = sum((w * x + b - y) ** 2 for x, y in zip(xs, ys)) / n
print(f"score = {w:.2f} * hours + {b:.2f}")
print(f"mean squared error: {mse:.2f}")
print(f"prediction for 8 hours: {w * 8 + b:.1f}")
```

Change the data and run again. Try adding an outlier like (7, 20) and see how much one bad point moves the line.

### Linear regression

Predict a number from inputs with `y = w·x + b`. With many inputs: `y = w₁x₁ + w₂x₂ + … + b`.

Example: house price = 3,000 × size + 20,000 × bedrooms + 50,000.

"Training" means finding the `w`s and `b` that make the line fit the data best.

### Measuring fit: Mean Squared Error

For each point, take the gap between prediction and truth, square it (so negatives don't cancel and big misses hurt more), then average.

`MSE = average of (prediction − truth)²`. Smaller is better.

**Quick check:** Why does MSE square the errors instead of just averaging them?

A. Squaring makes the maths impossible
B. So positive and negative errors don't cancel out, and big errors are punished more
C. To make the number smaller
D. Squaring is required by law

Answer: B. An error of +5 and −5 would average to 0 without squaring. Squaring also makes one huge miss count more than several tiny ones.

### Logistic regression (it's for classification!)

Despite the name, it predicts **yes/no**. Take the linear score and pass it through the **sigmoid** function `σ(z) = 1 / (1 + e⁻ᶻ)`.

Sigmoid squashes any number into 0–1, which we read as a probability: 0.92 → "92% likely spam".

### The link to neural networks

A single neuron with a sigmoid activation *is* logistic regression.

A neural network is many of these stacked and connected. So you already understand the building block.

**Quick check:** Logistic regression outputs…

A. Any real number like house prices
B. A probability between 0 and 1
C. A list of words
D. A straight line only

Answer: B. The sigmoid squashes the output into (0, 1), so it's used for classification probabilities.

**Key takeaways**

- Linear regression predicts numbers: y = w·x + b.
- MSE measures fit by averaging squared errors.
- Logistic regression = linear score + sigmoid → probability.
- One sigmoid neuron is logistic regression.

**Go deeper (free sources)**

- [ML Crash Course: Linear & logistic regression](https://developers.google.com/machine-learning/crash-course) by Google
- [CS229 Lecture Notes](https://cs229.stanford.edu/) by Stanford University (Andrew Ng et al.)
- [Dive into Deep Learning, ch. 3–4](https://d2l.ai/) by Zhang, Lipton, Li & Smola
