# Open-weight models and running AI yourself · AI in Bed

Intermediate level · 6 min · https://www.aiinbed.com/learn/intermediate/open-models

*What "open" really means for AI models, why it matters, and how a laptop can run a capable chatbot.*

Some AI models are only available through a company's app or API. Others publish their **weights**, the learned numbers, so anyone can download and run them. These are called **open-weight models**.

### Open weights isn't always open source

Truly open-source AI would also share the training code and enough detail about the data to rebuild the model. Many "open" models share only the weights, often under a licence with conditions.

Always read the licence before building a product on one.

### Why people want open models

**Privacy:** data never leaves your machine. **Control:** the model won't change under you overnight. **Customisation:** you can fine-tune it on your own data. **Research:** scientists can look inside.

**Quick check:** What does an open-weight model let you do that an API-only model doesn't?

A. Get answers without any computer
B. Download the model and run it on your own hardware
C. Guarantee it never makes mistakes
D. Use it without electricity

Answer: B. With the weights you can run, inspect and fine-tune the model yourself.

### Fitting a model on a laptop

A model with 8 billion parameters stored as 16-bit numbers needs about 16 GB just for weights. **Quantisation** stores each weight in 4 bits instead, cutting that to about 4 to 5 GB with a small quality loss.

Tools like llama.cpp and Ollama do this for you, so many modern laptops can run a decent chatbot offline.

**How much memory does a model need?**

```python
def memory_gb(billion_params, bits):
    return billion_params * 1e9 * bits / 8 / 1e9

for size in [1, 8, 70]:
    row = [f"{bits}-bit: {memory_gb(size, bits):5.1f} GB" for bits in (16, 8, 4)]
    print(f"{size:>3}B params  " + "   ".join(row))
```

Weights only. Running the model needs some extra memory for the conversation, so add 10 to 20 percent.

**Quick check:** Roughly how much memory do the weights of an 8-billion-parameter model need at 4 bits?

A. About 32 GB
B. About 16 GB
C. About 4 GB
D. About 4 MB

Answer: C. 8 billion × 4 bits = 32 billion bits = 4 billion bytes, about 4 GB.

**Key takeaways**

- Open-weight models publish their learned numbers; open source goes further.
- Open models give privacy, control and room to customise.
- Quantisation shrinks models enough to run on ordinary laptops.

**Go deeper (free sources)**

- [The Open Source AI Definition](https://opensource.org/ai/open-source-ai-definition) by Open Source Initiative
- [llama.cpp](https://github.com/ggml-org/llama.cpp) by ggml-org
- [Ollama](https://ollama.com/) by Ollama