Intermediate · 12

Finding groups: clustering

How a computer can sort things into groups when nobody has told it what the groups are, with k-means step by step.

6 min read · Tap underlined words for a definition

Most models learn from labelled examples: this email is spam, that one isn't. Unsupervised learning has no labels at all. The model looks for structure on its own. The most common job is clustering: putting similar things into groups.

k-means in four steps

1. Choose how many groups you want. That number is k.

2. Drop k starting points, called centroids, somewhere among the data.

3. Assign every data point to its nearest centroid.

4. Move each centroid to the average position of its points. Repeat steps 3 and 4 until nothing changes.

Run itk-means on a line of numbers

import random
random.seed(1)
# Minutes customers spend in an app each day (1D, easy to see)
points = [2, 3, 4, 5, 20, 22, 25, 24, 50, 55, 52, 58]
k = 3
centres = random.sample(points, k)
for step in range(5):
    groups = {i: [] for i in range(k)}
    for x in points:  # assign each point to its nearest centre
        nearest = min(range(k), key=lambda i: abs(x - centres[i]))
        groups[nearest].append(x)
    # move each centre to the mean of its group
    centres = [sum(g) / len(g) if g else centres[i] for i, g in groups.items()]
    print(f"step {step + 1}: centres = {[round(c, 1) for c in centres]}")
for i in range(k):
    print(f"cluster {i}: {groups[i]}")

Edit anything, then run. Python runs in your browser.

The centres settle after a couple of steps. Try k = 2 or a different seed: the groups can change, because k-means depends on where it starts.

Quick check

In k-means, what happens to each centroid after the points have been assigned?

Choosing k: the elbow idea

Run k-means for k = 1, 2, 3 and so on. For each, measure how far points sit from their centroid on average.

More clusters always shrinks that distance, but at some point the gains flatten out. Plot it and look for the bend, the elbow. That is often a sensible k. It is a rule of thumb, not a law, and sometimes there is no clear bend.

Customer segments: grouping shoppers by what and how often they buy.
Grouping documents: turn each article into an , then cluster to find topics.
Spotting oddities: a point far from every centroid may be worth a closer look.
Compressing images: replace thousands of colours with the k most typical ones.

Where k-means struggles

It assumes groups are roughly round blobs of similar size. Long, curved or nested shapes confuse it.

Results depend on the starting centroids, so libraries run it several times and keep the best. A smarter start called k-means++ helps.

Features on big scales dominate distance. Income in pounds will drown out age in years unless you rescale them first.

Clusters have no names. A human still has to look and decide what each group means.

Quick check

You cluster customers by income (in pounds) and age (in years) without rescaling. What is the likely problem?

Before you drift off

Clustering finds groups in data with no labels, a kind of unsupervised learning.
k-means repeats two steps: assign points to the nearest centroid, then move centroids to the mean.
You must choose k, scale your features, and interpret the groups yourself.

Terms in this lesson

Go deeper, free and open

Clustering · scikit-learn
Clustering course · Google · Free

That's the lesson

Finding groups: clustering · 6 min

Finish & continue · Up nextHow text becomes tokensIntermediate · 7 min

Was this lesson clear?

Spotted a mistake? Suggest an edit

Previous: Decision trees and random forests