Ollama vs LM Studio vs llama.cpp vs vLLM: which runtime for which job

Use Ollama or LM Studio to run models on your own computer, llama.cpp when you want full control or unusual hardware, and vLLM when one GPU server has to answer many people at once. Ollama is the quickest start: one install, one command, and a background service that other apps can call. LM Studio is the most complete graphical app, with a model browser, memory warnings and Apple's MLX engine on Macs. llama.cpp is the open-source engine underneath much of the local-AI world; using it directly gives you every option and the widest hardware support. vLLM is a different kind of tool: a Linux serving engine built to keep a GPU busy with many simultaneous requests, which is what an office server needs. All four expose an OpenAI-compatible API, so the apps you put on top can move between them.

At a glance

Ollama: the quickest way to run a model

Ollama installs as a background service with a curated model library. You pick a model by name and size, it downloads the right file and loads it onto your GPU, and any app on your machine can call it at localhost:11434, through Ollama's own API or the OpenAI-compatible /v1 endpoints.

ollama run qwen3:8b

Strengths: the shortest path from nothing to a working model; tags that encode size and quantization; an always-on API; GPU support for NVIDIA, AMD (ROCm v7) and, through Vulkan, other GPUs including Intel; and on Apple Silicon, Apple's MLX engine by default for supported model architectures since version 0.40.0.

Limits: fewer low-level knobs than llama.cpp, though the important ones (context length, parallel requests, KV-cache type, keep-alive) are available as settings or environment variables. The default context is only 4K tokens on GPUs with less than 24 GiB of VRAM, which is too small for documents and coding tools until you raise it. Requests to the same model are processed one at a time by default (OLLAMA_NUM_PARALLEL is 1); additional requests queue.

Our Ollama guide covers installation on each operating system step by step.

LM Studio: the graphical workbench

LM Studio is a desktop app for finding, downloading and chatting with models. The Discover tab searches Hugging Face, the model loader lets you set the context length and GPU offload, and LM Studio warns you before loading a model that is likely to run out of memory. On Apple Silicon it runs both GGUF models (through llama.cpp) and MLX models.

It also works without the window. The lms command-line tool ships with the app, and a headless daemon called llmster runs LM Studio's engine on servers without a desktop:

lms get openai/gpt-oss-20b
lms load openai/gpt-oss-20b
lms server start

The server listens on http://localhost:1234 with OpenAI-compatible endpoints. A Serve on Local Network switch, or lms server start --bind 0.0.0.0, makes it reachable from other machines; LM Studio's documentation recommends turning on authentication when you do.

Licensing: since July 2025, LM Studio has been free for use at home and at work. The desktop app is proprietary: its terms allow use for personal and internal business purposes and forbid redistributing or reverse-engineering it. The lms CLI, the JavaScript and Python SDKs and the MLX engine are MIT-licensed.

Requirements: on a Mac, Apple Silicon and macOS 14 or newer (Intel Macs are not supported); on Windows, an x64 CPU with AVX2 or an ARM device; on Linux, an x64 or ARM64 system. LM Studio recommends 16 GB of RAM, and on Windows at least 4 GB of dedicated VRAM.

llama.cpp: the engine underneath

llama.cpp is the C/C++ inference engine that created the GGUF format. Using it directly gives you early support for new models (new architectures often land here first), every tuning option, and the widest hardware coverage: its README lists backends for NVIDIA (CUDA), AMD (HIP), Apple (Metal), Intel (SYCL), any GPU with Vulkan, and more.

Install it from a package manager, or with the new official installer:

brew install llama.cpp

winget install llama.cpp

curl -LsSf https://llama.app/install.sh | sh

Then download a model from Hugging Face and chat with it, or start an OpenAI-compatible server with a built-in web chat:

llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF

Recent versions do a lot automatically. The context size defaults to the model's own maximum, GPU offload (-ngl) defaults to auto, and a --fit option, on by default, shrinks settings you have not set (down to a 4,096-token context) so the model fits in memory. Flash attention is auto, and the KV cache can be quantized with --cache-type-k and --cache-type-v.

One change to watch: the server's default port moved from 8080 to 9931 on 9 October 2026 (pull request #30159, nightly build b11521 onward). Release v0.6.0 and the Docker images still use 8080. If a guide or app expects 8080, pass --port 8080.

vLLM: serving many users from a GPU server

vLLM is built for throughput. Its PagedAttention memory management and continuous batching let one GPU serve many requests at once, which matters when a team shares one machine. It also splits large models across several GPUs with tensor parallelism (--tensor-parallel-size).

It is also the most demanding to run:

• Linux only. On Windows, use WSL. On macOS the official build is CPU-only and must be compiled from source; the community vLLM-Metal plugin uses the Apple GPU.
• Supported accelerators: NVIDIA GPUs with compute capability 7.5 or newer (RTX 20 series and later, T4, A100, H100 and so on), AMD GPUs with ROCm 6.3 or newer (including Radeon RX 7900 and RX 9000 series and Ryzen AI MAX), Intel GPUs, and Google TPUs, plus CPU builds.
• Python 3.11 to 3.14 per the documentation.

The recommended NVIDIA install uses uv:

uv venv --python 3.12 --seed
source .venv/bin/activate
uv pip install vllm --torch-backend=auto
vllm download-kernels

Then serve a model from Hugging Face; the server listens on http://localhost:8000:

vllm serve Qwen/Qwen2.5-1.5B-Instruct

Two things surprise newcomers. First, vLLM reserves most of the GPU's memory at start-up for its KV cache: --gpu-memory-utilization defaults to 0.92, so the GPU is not available for much else. Second, vLLM loads Hugging Face weights rather than GGUF files: you run Qwen/Qwen3-8B rather than an Ollama tag such as qwen3:8b, and quantized variants come in formats such as AWQ, GPTQ or FP8. Use --max-model-len to cap the context and the memory it reserves.

Serving more than one person

For a household or a small team, Ollama or LM Studio behind a chat front end is usually enough: requests queue, and each answer streams at single-user speed. In Ollama, raise OLLAMA_NUM_PARALLEL to process several requests to the same model at once; Ollama's documentation notes that the memory needed scales with the number of parallel requests times the context length. When many people work at the same time and waiting in a queue becomes the problem, vLLM's continuous batching serves them together and gets far more total throughput from the same GPU. Our guide to a private ChatGPT for your office builds the small-team setup with Open WebUI and Ollama.

The same model in each runtime

Each runtime names models differently. For Qwen3-8B:

Memory needs are the same whichever runtime you use: see the model page or the calculator.

Which one should you use?

1. You want to try local AI tonight: Ollama, or LM Studio if you prefer a graphical app.
2. You have a Mac: Ollama or LM Studio; both use MLX on Apple Silicon. vLLM is not a practical choice.
3. You want to compare models and quantizations: LM Studio, whose model browser and memory warnings make it easy.
4. You have an AMD or Intel GPU that other tools do not support well: llama.cpp with its Vulkan, HIP or SYCL backend.
5. You are building apps or agents on one machine: Ollama or llama.cpp's server, both always-on APIs.
6. You are serving a team from a Linux GPU server: Ollama for a handful of users; vLLM when many people use it at once.

There is no lock-in. All four speak the OpenAI API format, so chat front ends, coding tools and your own scripts can point at a different runtime by changing one URL.