Intermediate · 01

Lines and curves: regression

The humble straight line is the ancestor of every neural network. Linear and logistic regression, explained.

6 min read · Tap underlined words for a definition

Before , there was the line. Understanding the two classic models, linear regression and logistic regression, gives you 80% of the intuition for everything that follows.

Linear → a numberhouse size →Logistic → a probability0.510suspicious words →
Left: linear regression fits a line through points to predict a number. Right: logistic regression squashes a line into an S-curve to predict a probability.
Run itFit a line with least squares
Edit anything, then run. Python runs in your browser.

Change the data and run again. Try adding an outlier like (7, 20) and see how much one bad point moves the line.

Linear regression

Predict a number from inputs with y = w·x + b. With many inputs: y = w₁x₁ + w₂x₂ + … + b.

Example: house price = 3,000 × size + 20,000 × bedrooms + 50,000.

"Training" means finding the ws and b that make the line fit the data best.

Measuring fit: Mean Squared Error

For each point, take the gap between prediction and truth, square it (so negatives don't cancel and big misses hurt more), then average.

MSE = average of (prediction − truth)². Smaller is better.

Quick check

Why does MSE square the errors instead of just averaging them?

Logistic regression (it's for classification!)

Despite the name, it predicts yes/no. Take the linear score and pass it through the sigmoid function σ(z) = 1 / (1 + e⁻ᶻ).

Sigmoid squashes any number into 0–1, which we read as a probability: 0.92 → "92% likely spam".

The link to neural networks

A single neuron with a sigmoid activation is logistic .

A neural network is many of these stacked and connected. So you already understand the building block.

Quick check

Logistic regression outputs…

Before you drift off

  • Linear regression predicts numbers: y = w·x + b.
  • MSE measures fit by averaging squared errors.
  • Logistic regression = linear score + sigmoid → probability.
  • One sigmoid neuron is logistic regression.

Terms in this lesson

Go deeper, free and open

That's the lesson

Lines and curves: regression · 6 min

Each lesson you finish lights a star.

Finish & continue · Up nextGradient descent: rolling downhillIntermediate · 6 min

Spotted a mistake? Suggest an edit