# Embeddings: meaning as maps · AI in Bed

Intermediate level · 5 min · https://www.aiinbed.com/learn/intermediate/embeddings

*How AI turns words, images and songs into points in space where closeness means similarity.*

Computers only understand numbers. So how does a model know that "puppy" is close to "dog" but far from "spreadsheet"? It gives every word a list of numbers, an **embedding**, so that similar meanings land near each other.

*Figure: A 2-D sketch of embedding space (real ones have hundreds of dimensions). Related words cluster, and directions can carry meaning.*

*Interactive explainer: Explore a small map of word meanings.*

### Words as coordinates

Each token becomes a vector, e.g. 768 numbers long.

These numbers are learned during training. Nobody hand-picks them; they emerge from which words appear in similar contexts.

### Maths on meaning

Famous result from word2vec: `king − man + woman ≈ queen`.

Directions in the space can capture concepts like gender, tense, or country → capital.

**Quick check:** In a good embedding space, "cat" and "kitten" should be…

A. Very far apart
B. Close together
C. Exactly the same point
D. Unrelated, embeddings are random

Answer: B. Similar meanings end up near each other. That's the whole point of embeddings.

### Measuring closeness

**Cosine similarity** compares the angle between two vectors: 1 = same direction, 0 = unrelated.

It's how semantic search finds documents that *mean* the same thing even with different words.

### Embeddings everywhere

Not just words: sentences, images (CLIP puts images and captions in the same space), products, songs, users.

Recommendation systems, search engines and RAG (Advanced level) all run on embeddings.

> Pillow thought: "You shall know a word by the company it keeps." Linguist J.R. Firth said this in 1957, decades before it became the core idea of word embeddings.

**Quick check:** Where do embedding values come from?

A. A dictionary written by linguists
B. They're learned from data during training
C. Random numbers that never change
D. The alphabetical order of words

Answer: B. Embeddings are parameters too. Training shapes them so that useful relationships emerge.

**Key takeaways**

- Embeddings turn things into vectors where closeness = similarity.
- They're learned, not hand-written.
- Cosine similarity measures how close two embeddings are.
- They power search, recommendations and RAG.

**Go deeper (free sources)**

- [The Illustrated Word2vec](https://jalammar.github.io/illustrated-word2vec/) by Jay Alammar
- [Hugging Face LLM Course: Semantic search with embeddings](https://huggingface.co/learn/llm-course) by Hugging Face
- [Embedding Projector (interactive)](https://projector.tensorflow.org/) by TensorFlow / Google
