# Ollama

Ollama is a local large language model (LLM) runtime that downloads, runs, and serves open models such as Llama, Qwen, Gemma, and DeepSeek on your own hardware. It exposes a simple REST API and a command-line interface, so you can run inference privately without sending data to a third-party provider.

## Software included

| Component | Version |
| --- | --- |
| Ollama | 0.31.2 |
| Ubuntu | 24.04 LTS |

No models are pre-loaded. You pull the models you need on first boot (see [Getting started](https://docs.zcp.zsoftly.ca/public-cloud/marketplace/ollama/#getting-started)).

## Requirements

Ollama runs on CPU and RAM. A GPU is optional and speeds up inference. Two things decide the plan you need:

- RAM has to hold the model you run (about the model file size plus headroom for the context). A model that does not fit in RAM will not run.
- Storage has to hold the model files you pull, plus the operating system and working space. Add the size of every extra model you keep, because they add up fast.

Size the instance for the largest model you plan to run. The figures below use Q4 quantization, Ollama’s default.

| Model size | Example | Model file (approx.) | Minimum RAM | Suggested disk |
| --- | --- | --- | --- | --- |
| 1-3B | Llama 3.2 3B | 2-3 GB | 8 GB | 20 GB |
| 7-8B | Llama 3.1 8B | 5 GB | 8 GB | 30 GB |
| 13-14B | Llama 2 13B | 9 GB | 16 GB | 40 GB |
| 32-34B | Qwen 2.5 32B | 20 GB | 32 GB | 60 GB |
| 70B | Llama 3.3 70B | 40 GB | 64 GB | 100 GB |

## Recommended plans by region

Plan names and storage tiers differ by region. The general-purpose plans below give each model enough RAM. Plans cover CPU and memory only, so choose a root disk to hold the model files on the OS Disk Offering step. The disk sizes in the table above are starting sizes. See [Instance Types](https://docs.zcp.zsoftly.ca/public-cloud/compute/instance-types/) and the [pricing page](https://zcp.zsoftly.ca/pricing) for current specs and prices.

| Model size | YOW-1 (Ottawa) | YUL-1 (Montréal) |
| --- | --- | --- |
| 1-8B | `ci1c.l` (4 vCPU, 8 GB) | `ca2c.l` (4 vCPU, 8 GB) |
| 13-14B | `ci1c.xl` (4 vCPU, 16 GB) | `ca2c.xl` (4 vCPU, 16 GB) |
| 32-34B | `ci1c.2xl` (8 vCPU, 32 GB) | `ca2c.2xl` (8 vCPU, 32 GB) |
| 70B | `ci1c.4xl` (16 vCPU, 64 GB) | `ca2c.4xl` (16 vCPU, 64 GB) |

Adjust from there:

- Same RAM, lower cost: the memory-focused family gives the same RAM with fewer vCPUs, good for occasional use. Use `cim1c.*` in YOW-1 or `cam2c.*` in YUL-1 (for example `cim1c.xl` or `cam2c.xl` for 64 GB).
- Faster responses: add vCPUs with the CPU-focused family (`cac1c.*` in YOW-1, `cac2c.*` in YUL-1). Public-cloud GPU VM inference is not currently available in YOW-1 or YUL-1. For larger interactive models, contact ZSoftly about private or dedicated GPU capacity, or use a managed inference service.
- Several large models on one instance: attach a [block storage volume](https://docs.zcp.zsoftly.ca/public-cloud/compute/settings/block-storage/) instead of moving to a bigger plan.

## Getting started

### 1. Connect to your VM

```bash
ssh ubuntu@<your-vm-ip>
```

### 2. Wait for first-boot configuration

On the first boot, a setup script starts the service. This takes under a minute. Track progress:

```bash
journalctl -u ollama-first-boot.service -f
```

The login message (MOTD) confirms when Ollama is ready.

### 3. Pull a model and run a prompt

```bash
ollama pull llama3.2
ollama run llama3.2 "Hello, what can you do?"
```

### 4. Use the API

The API listens on port `11434`. Test it from the VM:

```bash
curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2",
  "prompt": "Why is the sky blue?",
  "stream": false
}'
```

From another host, use an SSH tunnel or a restricted private path. Do not expose port `11434` to the public internet.

## Managing Ollama

Ollama runs as a systemd service.

```bash
# Check status
systemctl status ollama


# Restart
sudo systemctl restart ollama


# View logs
sudo journalctl -u ollama -f


# List, pull, and remove models
ollama list
ollama pull qwen2.5
ollama rm qwen2.5
```

Models are stored in `/usr/share/ollama/.ollama/models`. A summary of URLs and commands is written to `/etc/ollama/info.txt`.

## Security

Ollama’s local API has no built-in authentication. Keep it private unless you place authentication in front of it. Before sharing the VM, remove any broad guest firewall rule for port `11434` and choose an SSH tunnel or a restricted source CIDR.

To restrict the API to a specific IP:

```bash
sudo ufw delete allow 11434/tcp
sudo ufw allow from <trusted-ip> to any port 11434 proto tcp
```

To reach the API without opening the firewall, use an SSH tunnel:

```bash
# Run this on your local machine
ssh -L 11434:localhost:11434 ubuntu@<your-vm-ip>
```

For production use, place Ollama behind a reverse proxy that enforces TLS and authentication, or restrict access to trusted IPs only.

## Next steps

- [Run Ollama Chat and Inference on ZCP](https://docs.zcp.zsoftly.ca/tutorials/ollama-chat-and-inference)
- [Run Open WebUI With Ollama on ZCP](https://docs.zcp.zsoftly.ca/tutorials/open-webui-with-ollama)
- [Ollama documentation](https://docs.ollama.com/)
- [Ollama API reference](https://docs.ollama.com/api)