How to run an LLM on your own computer with Ollama (Windows, macOS, Linux)

Install Ollama, then run one command: ollama run followed by a model name. On Windows and macOS you download the installer from ollama.com (or use the official one-line script), and on Linux you run a single curl command. Ollama downloads the model the first time you run it, loads it onto your GPU if it fits, and opens a chat in your terminal. The only real decision is which model to run, and that depends on how much memory you have: an 8B model at Ollama's default quantization needs about 5.9 GiB by our estimate, which fits almost any recent gaming GPU or a 16 GB Mac. This guide follows the official Ollama documentation as of October 2026 (version 0.40.4), and every step links to it in the sources at the end.

Before you start: what you need

• Windows: Windows 10 22H2 or newer, Home or Pro. NVIDIA cards need driver 551.61 or newer; AMD cards need a ROCm v7 (HIP 7) driver stack, or a Vulkan-capable Radeon driver. The installer needs at least 4 GB of disk space for itself, and models take tens of gigabytes more.
• macOS: macOS 14 Sonoma or newer. Apple Silicon (M-series) Macs get GPU acceleration; Intel Macs run on the CPU only.
• Linux: any modern 64-bit distribution with systemd. NVIDIA GPUs need compute capability 5.0 or newer and driver 550 or newer; AMD GPUs need the ROCm v7 driver.
• Memory: the model has to fit in your GPU's VRAM (or, on a Mac, in the share of unified memory the GPU can use) to run at full speed. Look up your card or Mac on our hardware pages and see what fits in the calculator.

Step 1: Install Ollama

Windows

1. Go to ollama.com/download and download OllamaSetup.exe.
2. Run it. It does not need administrator rights: it installs into your user folder and adds Ollama to your PATH.
3. Ollama starts in the background and shows an icon in the system tray. Open PowerShell or Windows Terminal to use the ollama command.

If you prefer the command line, the official PowerShell one-liner does the same thing:

irm https://ollama.com/install.ps1 | iex

macOS

1. Download Ollama.dmg from ollama.com/download.
2. Open it and drag Ollama into your Applications folder.
3. Start Ollama. On first launch it offers to link the ollama command into /usr/local/bin so you can use it from Terminal; accept.

The official install script also works on macOS:

curl -fsSL https://ollama.com/install.sh | sh

Homebrew packages exist, but the Ollama project lists them as community integrations, not as an official install method.

Linux

Run the official script:

curl -fsSL https://ollama.com/install.sh | sh

It installs the ollama binary, creates an ollama system user and a systemd service, and fetches the extra ROCm package if it detects an AMD GPU. When it finishes, the API is listening on 127.0.0.1:11434. Check the service with:

sudo systemctl status ollama

If you would rather not pipe a script into your shell, the Linux docs describe a manual install from the release archive (now a .tar.zst file) and the systemd unit to create.

Check that it works

On any system, open a terminal and run:

ollama -v

You should see the version number. If the command is not found on Windows, close and reopen the terminal so it picks up the new PATH.

Step 2: Pick a model that fits your memory

Ollama's library tags most models in several sizes. The default tag of a model is usually a 4-bit Q4_K_M build: on the qwen3 tags page, for example, qwen3:8b and qwen3:8b-q4_K_M are the same file. Here is what some popular models need at that quantization with a 4K context, which is Ollama's default context on GPUs with less than 24 GiB of VRAM:

Estimates from our methodology, using data as of 2026-10-14. Real speed depends on the runtime, drivers and prompt length.

A simple rule: pick the largest model whose memory figure is comfortably below your VRAM. On a Mac, compare it with the share of memory macOS lets the GPU use by default rather than the full amount; Apple does not publish that share, and our estimates assume 75% (our Mac guide explains why). Leave a gigabyte or two for your desktop and apps. If you have a 12 GB card, start with an 8–12B model; with 24 GB you can run the 27–35B class. Each model's page shows the memory at every quantization and context length, and which GPUs and Macs can run it.

Two families deserve a note. Mixture-of-experts models such as gpt-oss-20b and Qwen3.6-35B-A3B need memory for all their parameters but only compute a few billion per token, so they answer much faster than dense models of the same size. And on Apple Silicon, since Ollama 0.40.0 (September 2026), model architectures supported by Apple's MLX framework run on MLX by default; the rest run on llama.cpp with Metal. Some library tags therefore carry both a GGUF and an MLX file, and their sizes can differ.

Step 3: Run it

ollama run qwen3:8b

The first run downloads the model (several gigabytes, so it takes a while), then drops you into a chat prompt. Type a question and press Enter. A few commands work inside the chat:

On Windows and macOS you can also chat in the Ollama app's own window instead of the terminal; it uses the same models.

Everyday commands

Ollama keeps a model in memory for 5 minutes after the last request, then unloads it. Set OLLAMA_KEEP_ALIVE (for example 24h) to change that.

Choose a quantization

The default Q4_K_M build is a good balance of size and quality. If you have memory to spare, a higher-precision tag such as qwen3:8b-q8_0 is closer to the original model: it stores about 8.5 bits per weight instead of 4.85, so the weights grow from 4.6 GiB to 8.1 GiB. The -fp16 tag is the unquantized version at 15.3 GiB. Not every model publishes every tag, so check the model's tags page in the Ollama library. Our quantization guide explains what the names mean and how much quality you trade.

Context length: the setting most people miss

The context length is how many tokens of conversation, documents or code the model can see at once. Ollama picks a default from your VRAM:

4K is fine for short chats but too small for long documents, coding tools or agents; Ollama's own documentation recommends at least 64,000 tokens for those tasks. You can raise it in three ways:

• In the app: Settings → Context length slider.
• For the whole server: set the OLLAMA_CONTEXT_LENGTH environment variable, for example OLLAMA_CONTEXT_LENGTH=32768, then restart Ollama.
• For one session: /set parameter num_ctx 32768 inside ollama run, or "options": {"num_ctx": 32768} in an API request.

Bigger contexts cost memory. For Qwen3-8B our estimate grows from 5.9 GiB at 4K to 9.9 GiB at 32K. Check the CONTEXT column of ollama ps to confirm what was actually applied.

Check that the GPU is doing the work

Run ollama ps while a model is loaded:

NAME        ID            SIZE    PROCESSOR   UNTIL
llama3:70b  bcfb190ca3a7  42 GB   100% GPU    4 minutes from now

100% GPU is what you want. 100% CPU means the model runs from system memory, and a split such as 48%/52% CPU/GPU means part of it did not fit on the card. Both are much slower. Use a smaller model or quantization, or a shorter context. If even small models show CPU, check the GPU requirements:

• NVIDIA: compute capability 5.0 or newer and driver 550 or newer (551.61 on Windows).
• AMD: the ROCm v7 driver and a card on Ollama's supported list. On cards outside that list, Ollama's Vulkan backend, enabled by default on Windows and Linux, is the fallback.
• Intel: Ollama supports Intel GPUs through Vulkan.

Logs help when something goes wrong. They are in ~/.ollama/logs on macOS and %LOCALAPPDATA%\Ollama on Windows; on Linux run journalctl -e -u ollama.

Use Ollama from other apps: the local API

Ollama runs a server at http://localhost:11434. Other tools (chat front ends, coding assistants, your own scripts) talk to it through the native API or an OpenAI-compatible one:

curl http://localhost:11434/api/chat -d '{
  "model": "qwen3:8b",
  "messages": [{ "role": "user", "content": "Why is the sky blue?" }],
  "stream": false
}'

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{ "model": "qwen3:8b", "messages": [{ "role": "user", "content": "Say this is a test" }] }'

Libraries that speak the OpenAI API only need the base URL http://localhost:11434/v1/ and any placeholder API key; Ollama ignores it. The local API needs no authentication, which is why it listens only on your own machine by default.

To let other computers use it, turn on Expose Ollama to the network in the app's settings, or set OLLAMA_HOST=0.0.0.0:11434 and restart. On Linux, add that line to the service with sudo systemctl edit ollama.service under [Service] as Environment="OLLAMA_HOST=0.0.0.0:11434", then run sudo systemctl daemon-reload and sudo systemctl restart ollama. Anyone who can reach that port can use your models, so do this only on a trusted network. For a shared office setup with user accounts, see our guide to a private ChatGPT with Open WebUI.

Keep it local: cloud models and how to turn them off

Ollama also offers cloud models, with names that contain cloud, such as gemma4:cloud. They run on Ollama's servers, not on your machine, even when you call them through localhost:11434, and they require ollama signin. If your goal is to keep everything on your own hardware, switch cloud features off:

• In the app: Settings → turn off Cloud.
• With an environment variable: OLLAMA_NO_CLOUD=1.
• In a config file: add {"disable_ollama_cloud": true} to ~/.ollama/server.json.

Restart Ollama afterwards; the log then shows Ollama cloud disabled: true. Turning cloud off also disables Ollama's web search feature.

Move models to another drive

Models are large. To store them elsewhere, set OLLAMA_MODELS to a folder of your choice, or use Model location in the app's settings. On Windows: open Settings (or Control Panel on Windows 10), search for environment variables, choose Edit environment variables for your account, add OLLAMA_MODELS, then quit and restart Ollama from the tray. On Linux the ollama service user must own the new folder: sudo chown -R ollama:ollama <directory>.

Update and uninstall

The Windows and macOS apps download updates automatically; click the tray or menu-bar icon and choose Restart to update. On Linux, re-run the install script. To uninstall, use Add or remove programs on Windows, delete the app and ~/.ollama on macOS (the macOS docs list every folder), and follow the uninstall steps in the Linux docs to remove the service and files.

Where to go next

• Not sure your hardware is enough? Read choosing a GPU for local LLMs or running LLMs on a Mac.
• Wondering whether Ollama is the right tool? Compare it with LM Studio, llama.cpp and vLLM in which runtime for which job.
• Want to share a model with your team? Set up Open WebUI on top of Ollama.