Glossary
Every bit of jargon, in plain English
The terms used across the lessons. Tap a term for an example and the lessons that teach it. In a lesson, tap an underlined word to see its definition.
- Activation function
- A function applied to a neuron's weighted sum (e.g. ReLU, sigmoid) that adds non-linearity, letting stacked layers learn complex patterns.
- Agent
- An AI system that reasons and takes actions, often by calling tools in a loop, to complete a goal.
- AGI
- Artificial general intelligence: a hypothetical AI that could learn and do almost any intellectual task a person can. There is no agreed definition or test, and none exists today.
- AI winter
- A period when hype around AI faded and funding and interest dropped sharply, as happened in the 1970s and again around the late 1980s.
- Alignment
- Making an AI system do what its developers and users actually intend, including what they didn't spell out.
- Artificial intelligence
- Computer systems that do tasks we would call intelligent if a person did them, such as recognising speech, translating, recommending or answering questions. Most AI today is machine learning.
- Attention
- A mechanism that lets each token weigh and pull information from other tokens. The heart of the transformer.
- Augmentation
- Using AI to help a person do a task better or faster while they stay in charge, as opposed to automation, where the machine does the task instead.
- Backpropagation
- The algorithm that computes every weight's gradient by passing error signals backwards through a network using the chain rule.
- Batch / mini-batch
- A small group of training examples processed together in one optimisation step.
- Benchmark
- A fixed set of test questions with known answers, used to compare models with a single score.
- Bias (in data)
- Systematic skew in data or outcomes that can make a model unfair or unreliable for some groups.
- Chain of thought
- Having a model write out intermediate reasoning steps before its final answer, which improves accuracy on hard problems.
- Chatbot
- A program you talk to in everyday language. Modern chatbots such as ChatGPT, Claude and Gemini are built on large language models.
- Classification
- Predicting a category, like spam vs. not spam.
- Clustering
- Unsupervised learning that groups similar data points together without any labels, for example with k-means.
- CNN
- Convolutional neural network. Uses sliding filters to detect local patterns, mainly in images.
- Collaborative filtering
- A recommendation method that suggests items based on what people with similar behaviour liked, without needing to know what the items are.
- Computer vision
- The field of AI that gets computers to understand images and video, for example recognising faces, reading signs or spotting tumours in scans.
- Context window
- How many tokens an LLM can consider at once, both prompt and response.
- Decision tree
- A model that predicts by asking a chain of yes or no questions about the data, each answer leading down a branch to a final prediction.
- Deep learning
- Machine learning with neural networks that have many layers. It powers modern speech recognition, image recognition and chatbots.
- Deepfake
- A realistic fake image, video or voice of a real person, made with AI.
- Diffusion model
- A generative model that learns to turn noise into data by reversing a gradual noising process.
- Embedding
- A vector of numbers representing something (a word, image, user) so that similar things are close together.
- Epoch
- One full pass through the training dataset.
- Eval
- A test suite for an AI model or app: example inputs plus a way to judge whether each output is good.
- Expert system
- An older style of AI built from hand-written if-then rules supplied by human experts, popular in the 1980s.
- Few-shot prompting
- Including a few examples of the desired output in a prompt so the model follows the pattern.
- Fine-tuning
- Further training a pre-trained model on a smaller, task-specific dataset.
- Generative AI
- AI that creates new content, such as text, images, music or code, rather than only sorting or predicting labels. Chatbots and image generators are examples.
- Gradient
- The slope of the loss with respect to each parameter; it points in the direction of steepest increase.
- Gradient descent
- An optimisation method that repeatedly steps parameters opposite to the gradient to reduce loss.
- Hallucination
- When a model confidently produces false or invented information.
- Inference
- Using a trained model to make predictions, with its parameters frozen.
- Interpretability
- Research into what is happening inside a model's internal computations, in human-understandable terms.
- Knowledge distillation
- Training a small student model to copy the soft probability answers of a larger teacher model.
- KV cache
- Stored keys and values from earlier tokens that a model reuses while writing, so it only does fresh work for each new token. Faster, but uses memory.
- Learning rate
- The step size used in gradient descent.
- LLM
- Large language model. A big transformer trained on text to predict the next token.
- LoRA
- Low-Rank Adaptation. Fine-tunes by training small low-rank matrices while freezing the base weights.
- Loss function
- A number measuring how wrong a model's predictions are. Training minimises it.
- Machine learning
- The branch of AI where computers learn patterns from examples instead of following hand-written rules, then use those patterns to make predictions on new data.
- Mixture of experts
- A model design where a router sends each token to only a few of many sub-networks (experts), so it can store a lot but compute little per token.
- Multimodal
- Able to work with more than one kind of data, such as text, images and audio.
- Natural language processing
- The field of AI that works with human language: understanding, translating, summarising and generating text or speech. Often shortened to NLP.
- Neural network
- A model made of layers of simple connected units (neurons) whose connection strengths are learned from data. Loosely inspired by the brain.
- Open-weight model
- A model whose learned weights are published so anyone can download and run it, usually under a licence.
- Overfitting
- When a model memorises training data, including its noise, and performs poorly on new data.
- Parameters / weights
- The learned numbers inside a model that determine its behaviour.
- Precision
- Of the items predicted positive, the fraction that truly are positive: TP / (TP + FP).
- Prompt
- The input text you give an LLM to steer its output.
- Prompt injection
- Instructions hidden in content a model reads (a web page, an email) that try to hijack its behaviour.
- Quantisation
- Storing model weights with fewer bits (e.g. 4-bit) to save memory and speed up inference.
- RAG
- Retrieval-augmented generation. Fetches relevant documents and includes them in the prompt so answers are grounded.
- Random forest
- An ensemble of many decision trees, each trained on a random resample of the data and features, that vote together for a more reliable prediction.
- Recall
- Of the truly positive items, the fraction the model caught: TP / (TP + FN).
- Recommender system
- A program that suggests things you might like, such as films or products, based on your past choices and those of similar people.
- Regression
- Predicting a continuous number, like a price or temperature.
- Regularisation
- Techniques (L2 penalty, dropout, etc.) that discourage overly complex models to reduce overfitting.
- Reinforcement learning
- Learning to act by trial and error to maximise reward from an environment.
- Reward hacking
- When a model maximises the measurable reward in a way that misses the real goal.
- RLHF
- Reinforcement learning from human feedback. Uses human preference data to fine-tune model behaviour.
- Speculative decoding
- A speed-up where a small draft model guesses several tokens ahead and the big model checks them all at once, keeping the ones it agrees with.
- Supervised learning
- Learning from labelled examples, where each input comes with the right answer, so the model learns to map inputs to answers.
- Temperature
- A sampling setting controlling randomness. Lower is more deterministic, higher is more varied.
- Test-time compute
- Computation spent while answering, for example letting a model think longer, to get better results.
- Token
- A chunk of text (a word, part of a word or punctuation) that language models read and write.
- Training data
- The examples a model learns from. Its quality, size and biases shape everything the model does.
- Transformer
- A neural network architecture built on attention, used by virtually all modern LLMs.
- Unsupervised learning
- Learning from data with no labels, finding structure such as groups or patterns on its own.
- Validation set
- Data held out from training to tune settings and check for overfitting.
Want more? Google's Machine Learning Glossary covers hundreds of terms.
Learn the ideas behind the words
- Learn AI in 30 daysFree plan
- AI for everyone: no coding neededTrack
- What is an LLM? How ChatGPT worksBeginner
- How to write good AI promptsBeginner
- Open-weight models: run AI yourselfIntermediate
- How LLMs are trainedAdvanced
- LoRA and quantisation explainedAdvanced
- Reinforcement learning explainedAdvanced
- What is AI alignment?Advanced
- Free machine learning courseLevel
- Free advanced AI course: LLMs and agentsLevel