# How to run an LLM on your own computer with Ollama (Windows, macOS, Linux)

> OnPrem World · https://onpremworld.com/en/guides/install-ollama · Updated: 2026-10-14 · CC BY 4.0

**Install Ollama, then run one command: `ollama run` followed by a model name.** On Windows and macOS you download the installer from ollama.com (or use the official one-line script), and on Linux you run a single `curl` command. Ollama downloads the model the first time you run it, loads it onto your GPU if it fits, and opens a chat in your terminal. The only real decision is which model to run, and that depends on how much memory you have: an 8B model at Ollama's default quantization needs about 5.9 GiB by our estimate, which fits almost any recent gaming GPU or a 16 GB Mac. This guide follows the official Ollama documentation as of October 2026 (version 0.40.4), and every step links to it in the sources at the end.

## Before you start: what you need

- **Windows**: Windows 10 22H2 or newer, Home or Pro. NVIDIA cards need driver 551.61 or newer; AMD cards need a ROCm v7 (HIP 7) driver stack, or a Vulkan-capable Radeon driver. The installer needs at least 4 GB of disk space for itself, and models take tens of gigabytes more.
- **macOS**: macOS 14 Sonoma or newer. Apple Silicon (M-series) Macs get GPU acceleration; Intel Macs run on the CPU only.
- **Linux**: any modern 64-bit distribution with systemd. NVIDIA GPUs need compute capability 5.0 or newer and driver 550 or newer; AMD GPUs need the ROCm v7 driver.
- **Memory**: the model has to fit in your GPU's VRAM (or, on a Mac, in the share of unified memory the GPU can use) to run at full speed. Look up your card or Mac on our [hardware pages](https://onpremworld.com/en/hardware.md) and see what fits in the [calculator](https://onpremworld.com/en/can-i-run.md).

## Step 1: Install Ollama

### Windows

1. Go to [ollama.com/download](https://ollama.com/download) and download **OllamaSetup.exe**.
2. Run it. It does not need administrator rights: it installs into your user folder and adds Ollama to your PATH.
3. Ollama starts in the background and shows an icon in the system tray. Open PowerShell or Windows Terminal to use the `ollama` command.

If you prefer the command line, the official PowerShell one-liner does the same thing:

```powershell
irm https://ollama.com/install.ps1 | iex
```

### macOS

1. Download **Ollama.dmg** from [ollama.com/download](https://ollama.com/download).
2. Open it and drag Ollama into your Applications folder.
3. Start Ollama. On first launch it offers to link the `ollama` command into `/usr/local/bin` so you can use it from Terminal; accept.

The official install script also works on macOS:

```bash
curl -fsSL https://ollama.com/install.sh | sh
```

Homebrew packages exist, but the Ollama project lists them as community integrations, not as an official install method.

### Linux

Run the official script:

```bash
curl -fsSL https://ollama.com/install.sh | sh
```

It installs the `ollama` binary, creates an `ollama` system user and a systemd service, and fetches the extra ROCm package if it detects an AMD GPU. When it finishes, the API is listening on `127.0.0.1:11434`. Check the service with:

```bash
sudo systemctl status ollama
```

If you would rather not pipe a script into your shell, the [Linux docs](https://docs.ollama.com/linux) describe a manual install from the release archive (now a `.tar.zst` file) and the systemd unit to create.

### Check that it works

On any system, open a terminal and run:

```bash
ollama -v
```

You should see the version number. If the command is not found on Windows, close and reopen the terminal so it picks up the new PATH.

## Step 2: Pick a model that fits your memory

Ollama's library tags most models in several sizes. The default tag of a model is usually a 4-bit Q4_K_M build: on the [qwen3 tags page](https://ollama.com/library/qwen3/tags), for example, `qwen3:8b` and `qwen3:8b-q4_K_M` are the same file. Here is what some popular models need at that quantization with a 4K context, which is Ollama's default context on GPUs with less than 24 GiB of VRAM:

| Model | Parameters | Memory needed (Q4_K_M, 4K) | Command |
|---|---:|---:|---|
| [Llama 3.2 3B Instruct](https://onpremworld.com/en/models/llama-3.2-3b.md) | 3.21B | 2.8 GiB | `ollama run llama3.2:3b` |
| [Gemma 4 E4B IT](https://onpremworld.com/en/models/gemma-4-e4b.md) | 8B | 5.4 GiB | `ollama run gemma4:e4b` |
| [Qwen3-8B](https://onpremworld.com/en/models/qwen3-8b.md) | 8.2B | 5.9 GiB | `ollama run qwen3:8b` |
| [Qwen3.5-9B](https://onpremworld.com/en/models/qwen3.5-9b.md) | 9B | 6.0 GiB | `ollama run qwen3.5:9b` |
| [Gemma 4 12B Unified IT](https://onpremworld.com/en/models/gemma-4-12b.md) | 11.95B | 7.9 GiB | `ollama run gemma4:12b` |
| [gpt-oss-20b](https://onpremworld.com/en/models/gpt-oss-20b.md) | 21B (3.6B active) | 13.0 GiB | `ollama run gpt-oss:20b` |
| [Qwen3.6-27B](https://onpremworld.com/en/models/qwen3.6-27b.md) | 27B | 16.8 GiB | `ollama run qwen3.6:27b` |
| [Qwen3.6-35B-A3B](https://onpremworld.com/en/models/qwen3.6-35b-a3b.md) | 35B (3B active) | 21.3 GiB | `ollama run qwen3.6:35b` |
| [Gemma 4 31B IT](https://onpremworld.com/en/models/gemma-4-31b.md) | 30.7B | 19.6 GiB | `ollama run gemma4:31b` |

*Estimates from our [methodology](https://onpremworld.com/en/methodology.md), using data as of 2026-10-16. Real speed depends on the runtime, drivers and prompt length.*

A simple rule: pick the largest model whose memory figure is comfortably below your VRAM. On a Mac, compare it with the share of memory macOS lets the GPU use by default rather than the full amount; Apple does not publish that share, and our estimates assume 75% (our [Mac guide](https://onpremworld.com/en/guides/mac-local-llm.md) explains why). Leave a gigabyte or two for your desktop and apps. If you have a 12 GB card, start with an 8–12B model; with 24 GB you can run the 27–35B class. Each model's page shows the memory at every quantization and context length, and which GPUs and Macs can run it.

Two families deserve a note. Mixture-of-experts models such as [gpt-oss-20b](https://onpremworld.com/en/models/gpt-oss-20b.md) and [Qwen3.6-35B-A3B](https://onpremworld.com/en/models/qwen3.6-35b-a3b.md) need memory for all their parameters but only compute a few billion per token, so they answer much faster than dense models of the same size. And on Apple Silicon, since Ollama 0.40.0 (September 2026), model architectures supported by Apple's MLX framework run on MLX by default; the rest run on llama.cpp with Metal. Some library tags therefore carry both a GGUF and an MLX file, and their sizes can differ.

## Step 3: Run it

```bash
ollama run qwen3:8b
```

The first run downloads the model (several gigabytes, so it takes a while), then drops you into a chat prompt. Type a question and press Enter. A few commands work inside the chat:

| Inside `ollama run` | What it does |
|---|---|
| `/?` | Lists the available commands |
| `/set parameter num_ctx 8192` | Sets the context length for this session |
| `"""` | Starts and ends a multi-line message |
| `/bye` | Leaves the chat (Ctrl+D also works) |

On Windows and macOS you can also chat in the Ollama app's own window instead of the terminal; it uses the same models.

## Everyday commands

| Command | What it does |
|---|---|
| `ollama pull qwen3:8b` | Downloads a model without starting a chat |
| `ollama list` | Lists the models you have downloaded |
| `ollama ps` | Shows loaded models, how much runs on GPU vs CPU, and their context length |
| `ollama stop qwen3:8b` | Unloads a model from memory |
| `ollama rm qwen3:8b` | Deletes a model from disk |
| `ollama show qwen3:8b` | Shows a model's details, such as parameters and quantization |

Ollama keeps a model in memory for 5 minutes after the last request, then unloads it. Set `OLLAMA_KEEP_ALIVE` (for example `24h`) to change that.

## Choose a quantization

The default Q4_K_M build is a good balance of size and quality. If you have memory to spare, a higher-precision tag such as `qwen3:8b-q8_0` is closer to the original model: it stores about 8.5 bits per weight instead of 4.85, so the weights grow from 4.6 GiB to 8.1 GiB. The `-fp16` tag is the unquantized version at 15.3 GiB. Not every model publishes every tag, so check the model's tags page in the Ollama library. Our [quantization guide](https://onpremworld.com/en/guides/quantization-explained.md) explains what the names mean and how much quality you trade.

## Context length: the setting most people miss

The context length is how many tokens of conversation, documents or code the model can see at once. Ollama picks a default from your VRAM:

| Your VRAM | Default context |
|---|---|
| Less than 24 GiB | 4K tokens |
| 24 to 48 GiB | 32K tokens |
| 48 GiB or more | 256K tokens |

4K is fine for short chats but too small for long documents, coding tools or agents; Ollama's own documentation recommends at least 64,000 tokens for those tasks. You can raise it in three ways:

- **In the app**: Settings → Context length slider.
- **For the whole server**: set the `OLLAMA_CONTEXT_LENGTH` environment variable, for example `OLLAMA_CONTEXT_LENGTH=32768`, then restart Ollama.
- **For one session**: `/set parameter num_ctx 32768` inside `ollama run`, or `"options": {"num_ctx": 32768}` in an API request.

Bigger contexts cost memory. For [Qwen3-8B](https://onpremworld.com/en/models/qwen3-8b.md) our estimate grows from 5.9 GiB at 4K to 9.9 GiB at 32K. Check the CONTEXT column of `ollama ps` to confirm what was actually applied.

## Check that the GPU is doing the work

Run `ollama ps` while a model is loaded:

```text
NAME        ID            SIZE    PROCESSOR   UNTIL
llama3:70b  bcfb190ca3a7  42 GB   100% GPU    4 minutes from now
```

`100% GPU` is what you want. `100% CPU` means the model runs from system memory, and a split such as `48%/52% CPU/GPU` means part of it did not fit on the card. Both are much slower. Use a smaller model or quantization, or a shorter context. If even small models show CPU, check the GPU requirements:

- **NVIDIA**: compute capability 5.0 or newer and driver 550 or newer (551.61 on Windows).
- **AMD**: the ROCm v7 driver and a card on Ollama's supported list. On cards outside that list, Ollama's Vulkan backend, enabled by default on Windows and Linux, is the fallback.
- **Intel**: Ollama supports Intel GPUs through Vulkan.

Logs help when something goes wrong. They are in `~/.ollama/logs` on macOS and `%LOCALAPPDATA%\Ollama` on Windows; on Linux run `journalctl -e -u ollama`.

## Use Ollama from other apps: the local API

Ollama runs a server at `http://localhost:11434`. Other tools (chat front ends, coding assistants, your own scripts) talk to it through the native API or an OpenAI-compatible one:

```bash
curl http://localhost:11434/api/chat -d '{
  "model": "qwen3:8b",
  "messages": [{ "role": "user", "content": "Why is the sky blue?" }],
  "stream": false
}'
```

```bash
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{ "model": "qwen3:8b", "messages": [{ "role": "user", "content": "Say this is a test" }] }'
```

Libraries that speak the OpenAI API only need the base URL `http://localhost:11434/v1/` and any placeholder API key; Ollama ignores it. The local API needs no authentication, which is why it listens only on your own machine by default.

To let other computers use it, turn on **Expose Ollama to the network** in the app's settings, or set `OLLAMA_HOST=0.0.0.0:11434` and restart. On Linux, add that line to the service with `sudo systemctl edit ollama.service` under `[Service]` as `Environment="OLLAMA_HOST=0.0.0.0:11434"`, then run `sudo systemctl daemon-reload` and `sudo systemctl restart ollama`. Anyone who can reach that port can use your models, so do this only on a trusted network. For a shared office setup with user accounts, see our guide to [a private ChatGPT with Open WebUI](https://onpremworld.com/en/guides/private-chatgpt-open-webui.md).

## Keep it local: cloud models and how to turn them off

Ollama also offers cloud models, with names that contain `cloud`, such as `gemma4:cloud`. They run on Ollama's servers, not on your machine, even when you call them through `localhost:11434`, and they require `ollama signin`. If your goal is to keep everything on your own hardware, switch cloud features off:

- **In the app**: Settings → turn off **Cloud**.
- **With an environment variable**: `OLLAMA_NO_CLOUD=1`.
- **In a config file**: add `{"disable_ollama_cloud": true}` to `~/.ollama/server.json`.

Restart Ollama afterwards; the log then shows `Ollama cloud disabled: true`. Turning cloud off also disables Ollama's web search feature.

## Move models to another drive

Models are large. To store them elsewhere, set `OLLAMA_MODELS` to a folder of your choice, or use **Model location** in the app's settings. On Windows: open Settings (or Control Panel on Windows 10), search for *environment variables*, choose *Edit environment variables for your account*, add `OLLAMA_MODELS`, then quit and restart Ollama from the tray. On Linux the `ollama` service user must own the new folder: `sudo chown -R ollama:ollama <directory>`.

## Update and uninstall

The Windows and macOS apps download updates automatically; click the tray or menu-bar icon and choose **Restart to update**. On Linux, re-run the install script. To uninstall, use *Add or remove programs* on Windows, delete the app and `~/.ollama` on macOS (the [macOS docs](https://docs.ollama.com/macos) list every folder), and follow the uninstall steps in the [Linux docs](https://docs.ollama.com/linux) to remove the service and files.

## Where to go next

- Not sure your hardware is enough? Read [choosing a GPU for local LLMs](https://onpremworld.com/en/guides/choosing-a-gpu.md) or [running LLMs on a Mac](https://onpremworld.com/en/guides/mac-local-llm.md).
- Wondering whether Ollama is the right tool? Compare it with LM Studio, llama.cpp and vLLM in [which runtime for which job](https://onpremworld.com/en/guides/ollama-vs-lm-studio-vs-llama-cpp-vs-vllm.md).
- Want to share a model with your team? Set up [Open WebUI on top of Ollama](https://onpremworld.com/en/guides/private-chatgpt-open-webui.md).

## Frequently asked questions

### Is Ollama free, and does it send my prompts anywhere?

Ollama is free, open-source software under the MIT licence. Models you download run on your own machine and the local API does not require an account. The exception is cloud models, whose names contain 'cloud' (for example gemma4:cloud): they run on Ollama's servers and need you to sign in. You can turn cloud features off entirely with the Cloud toggle in the app's settings or the OLLAMA_NO_CLOUD=1 environment variable.

### How much memory do I need to run a model with Ollama?

Roughly the model's file size plus room for the context. By our estimates, an 8B model such as Qwen3-8B needs about 5.9 GiB at Ollama's default Q4_K_M quantization with a 4K context, so a GPU with 8 GB or a Mac with 16 GB is a comfortable start. Our model pages list the memory for every quantization and context length.

### How do I know if Ollama is using my GPU?

Run ollama ps while a model is loaded. The PROCESSOR column shows 100% GPU when the whole model is on the graphics card, 100% CPU when it runs from system memory, and a split such as 48%/52% CPU/GPU when it did not fit entirely. A split or CPU-only result usually means the model or the context is too big for your VRAM, or the GPU driver is not supported.

### What is Ollama's default context length and how do I change it?

It depends on your VRAM: 4K tokens below 24 GiB, 32K from 24 to 48 GiB and 256K at 48 GiB or more. Change it with the Context length slider in the app's settings, the OLLAMA_CONTEXT_LENGTH environment variable, or /set parameter num_ctx inside an ollama run session. A longer context uses more memory.

### Where does Ollama store downloaded models?

In ~/.ollama/models on macOS, /usr/share/ollama/.ollama/models on Linux and C:\Users\<username>\.ollama\models on Windows. Set the OLLAMA_MODELS environment variable, or the Model location setting in the app, to use another drive.

### Can other computers on my network use my Ollama?

Yes, but it is off by default: Ollama listens only on 127.0.0.1 port 11434. Turn on Expose Ollama to the network in the app, or set OLLAMA_HOST=0.0.0.0:11434, then restart Ollama. The local API has no authentication, so only do this on a network you trust or behind a front end with logins.

## Sources

- https://docs.ollama.com/quickstart
- https://docs.ollama.com/windows
- https://docs.ollama.com/macos
- https://docs.ollama.com/linux
- https://docs.ollama.com/gpu
- https://docs.ollama.com/cli
- https://docs.ollama.com/context-length
- https://docs.ollama.com/faq
- https://docs.ollama.com/api/introduction
- https://docs.ollama.com/api/openai-compatibility
- https://docs.ollama.com/cloud
- https://ollama.com/download
- https://ollama.com/library/qwen3/tags
- https://github.com/ollama/ollama/releases/tag/v0.40.0
- https://github.com/ollama/ollama/releases/tag/v0.40.4
- https://github.com/ollama/ollama/blob/main/LICENSE