# RAG & AI agents · AI in Bed

Advanced level · 7 min · https://www.aiinbed.com/learn/advanced/rag-and-agents

*Giving models fresh knowledge with retrieval, and the ability to act with tools.*

An LLM's knowledge is frozen at training time, and it can't see your company's private documents. Two big ideas fix this: **retrieval-augmented generation** (look things up first) and **agents** (use tools in a loop).

*Figure: RAG: embed the question, find the most similar document chunks, paste them into the prompt, then generate a grounded answer.*

*Interactive explainer: Ask a shop assistant bot a question with and without retrieval.*

### How RAG works

**Index**: split documents into chunks, embed each chunk, store the vectors in a vector database.

**Retrieve**: embed the user's question and fetch the top-k most similar chunks.

**Generate**: put those chunks in the prompt and ask the model to answer *using them*, ideally with citations.

### Making RAG good

Chunk size matters: too big and you dilute relevance, too small and you lose context.

**Hybrid search** (keywords + embeddings) and a **re-ranker** often beat embeddings alone.

Evaluate retrieval separately from generation: if the right chunk never arrives, the model can't use it.

**Quick check:** What's the main benefit of RAG over fine-tuning for new facts?

A. It makes the model smaller
B. You can update knowledge by changing documents, without retraining, and cite sources
C. It removes the need for embeddings
D. It makes the model faster at maths

Answer: B. RAG keeps knowledge outside the weights, so it's easy to update and lets answers point to their sources.

### Agents: models that act

An agent is an LLM in a loop: **think → choose a tool → observe the result → think again**, until the task is done.

Tools can be web search, a calculator, a code interpreter, an API, or a browser. The ReAct paper popularised this reason-and-act pattern.

### Tool calling & protocols

Modern APIs let you describe tools with a JSON schema; the model replies with a structured call, your code runs it, and you feed back the result.

Open standards like the **Model Context Protocol (MCP)** let any tool or data source plug into any compatible AI app.

> Pillow thought: agents fail in long, quiet ways. A small mistake at step 3 snowballs by step 30. Good agent design is mostly about checkpoints, verification and knowing when to ask a human.

**Quick check:** What is the core loop of an AI agent?

A. Train → test → deploy
B. Reason → act with a tool → observe → repeat
C. Embed → cluster → label
D. Encrypt → send → decrypt

Answer: B. Agents alternate between reasoning and taking actions, using each observation to decide the next step.

**Key takeaways**

- RAG = retrieve relevant chunks, then generate grounded answers.
- Chunking, hybrid search and re-ranking make or break RAG.
- Agents = LLMs that reason and use tools in a loop.
- MCP standardises how tools plug into AI apps.

**Go deeper (free sources)**

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP](https://arxiv.org/abs/2005.11401) by Lewis et al., 2020
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://arxiv.org/abs/2210.03629) by Yao et al., 2022
- [LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/) by Lilian Weng
- [Model Context Protocol](https://modelcontextprotocol.io/) by Open standard, MIT licensed
- [Hugging Face AI Agents Course](https://huggingface.co/learn/agents-course) by Hugging Face
