Running LLMs on a Mac: unified memory, how much RAM to buy, MLX vs llama.cpp

Any Apple Silicon Mac can run local LLMs, and the amount of memory you buy decides how big a model it can hold. Macs have no separate video memory: the CPU and GPU share one pool of unified memory, so a 64 GB Mac can give its GPU far more memory than any consumer graphics card has. Two limits apply. macOS lets the GPU use only part of that memory by default, and the speed of generation follows the chip's memory bandwidth, which ranges from 153 GB/s on a base M5 to 1,200 GB/s on an M5 Ultra. As a rule of thumb, 16 GB runs 7–12B models, 32 GB handles today's popular 27–35B models at 4-bit, 64 GB reaches 70B, and 128 GB or more is for the large mixture-of-experts models. For software, the easiest start is Ollama or LM Studio, both of which can use Apple's MLX framework or llama.cpp underneath. The numbers below come from our hardware and model data; check any configuration in the calculator.

How unified memory works, and the GPU limit

On a PC, a model has to fit in the graphics card's VRAM, typically 8 to 32 GB. On an Apple Silicon Mac, the CPU and GPU read the same memory. Apple's MLX documentation describes it this way: both processors have direct access to the same memory pool, so arrays never need to be copied between them. For local AI, that means the memory you configure when you buy the Mac is, mostly, memory the model can use.

Mostly, because macOS caps how much memory the GPU can wire. Metal reports the cap as recommendedMaxWorkingSetSize, which Apple documents only as an approximation of how much the GPU can allocate without hurting performance; it does not publish a percentage. Measurements reported by llama.cpp's maintainer give an idea:

• On a 32 GB M1 Pro the reported limit was 21,845 MB, about two-thirds of the memory (PR #1826).
• In his guide to running gpt-oss, a 192 GB Mac's default is given as 154 GB, about 80% (discussion #15396).

Our estimates assume the GPU can use 75% of a Mac's memory, which is in the middle of that range. Smaller Macs may get somewhat less.

Raising the limit

If a model fits in your Mac's memory but not under the default cap, you can raise the cap from Terminal. The value is in megabytes; this example allows 56 GB on a 64 GB Mac (56 × 1024 = 57344):

sudo sysctl iogpu.wired_limit_mb=57344

The setting does not survive a reboot. The mlx-lm documentation advises a value larger than the model but smaller than the machine's total memory; in practice leave several gigabytes for macOS and your apps, or the system will start swapping. Watch the Memory Pressure graph in Activity Monitor while the model runs: memory pressure combines free memory, swap rate, wired memory and file cache, and a rising graph means you have pushed too far.

How much memory to buy

These tables show the highest-quality quantization (up to Q8_0) that fits each configuration with an 8K context, and the estimated generation speed. They are estimates, not benchmarks, and assume the default GPU memory limit described above.

16 GB: M5

Estimates from our methodology, using data as of 2026-10-14. Real speed depends on the runtime, drivers and prompt length.

A 16 GB Mac is a capable machine for 7–12B models. It runs out at the 20B class: even gpt-oss-20b, an efficient MoE model, needs about 13.1 GiB by our estimate, more than the GPU's default share of 16 GB.

32 GB: M5

Estimates from our methodology, using data as of 2026-10-14. Real speed depends on the runtime, drivers and prompt length.

32 GB is the most useful step up. It opens the 24–35B class, which is where local models start to feel close to hosted assistants for writing and coding. Note the speed column: on a base M5, a dense 32B model is slow, while the MoE models run several times faster.

64 GB: M5 Pro

Estimates from our methodology, using data as of 2026-10-14. Real speed depends on the runtime, drivers and prompt length.

64 GB brings 70B dense models within reach at 4-bit, though slowly on a Pro chip, and lets 27–35B models run at Q8_0, close to their original quality. The large MoE models, such as gpt-oss-120b, still do not fit.

128 GB: M5 Max (18-core CPU / 40-core GPU)

Estimates from our methodology, using data as of 2026-10-14. Real speed depends on the runtime, drivers and prompt length.

128 GB is where a Mac does something no single consumer GPU can: it runs 100–120B mixture-of-experts models such as gpt-oss-120b, GLM-4.5-Air and Qwen3.5-122B-A10B at usable speeds, because only a few billion parameters are active per token.

256 GB and 512 GB: M5 Ultra

The Ultra chips go further still: up to 512 GB on the M3 Ultra and M5 Ultra. Even so, the very largest open models are a stretch. DeepSeek-R1-0528 (671B) needs about 398.8 GiB at Q4_K_M by our estimate, which is more than the GPU's default share of 512 GB; it only fits if you raise the limit and leave macOS little room.

Why bandwidth decides speed

When a model generates text, every new token requires reading all of its active weights from memory, so generation speed is capped by memory bandwidth. Apple Silicon chips differ widely:

Here is the same dense model, Qwen3-32B at Q4_K_M, across Macs that can hold it, with two PC graphics cards for reference:

Estimates from our methodology, using data as of 2026-10-14. Real speed depends on the runtime, drivers and prompt length.

The chip tier matters more than the generation: an M5 Max has about four times the bandwidth of a base M5, and an M5 Ultra about twice an M5 Max. For dense models, a Max or Ultra chip is the difference between a usable assistant and a slow one.

Mixture-of-experts models change the picture, because only their active parameters are read per token. Here is gpt-oss-120b (117B (5.1B active)) on large-memory machines, including two unified-memory PCs and a workstation GPU:

Estimates from our methodology, using data as of 2026-10-14. Real speed depends on the runtime, drivers and prompt length.

Treat MoE estimates as upper bounds: routing between experts adds work the bandwidth formula does not count.

Prompt processing and the M5's Neural Accelerators

Generation is only half the story. Before the first word of the answer, the model must process your whole prompt, and that step depends on compute rather than bandwidth. It matters when you paste long documents or use coding tools that send large contexts. Apple's M5 generation adds Neural Accelerators to each GPU core. In Apple's own tests with MLX on an M5 MacBook Pro, time to first token was up to about four times faster than on an M4, while generation was 19–27% faster, matching the M5's higher memory bandwidth. MLX uses the accelerators on macOS 26.2 or later. Our estimates cover generation only.

MLX or llama.cpp?

Both run models on the Mac's GPU. The differences are in formats, quantization options and which apps use them.

In practice:

• Use the app you like and let it choose. Since Ollama 0.40.0 (September 2026), models whose architecture MLX supports run on MLX by default on Apple Silicon, and the rest run on llama.cpp. LM Studio offers both engines and lets you pick MLX or GGUF downloads.
• Expect MLX to shine on prompt processing, especially on M5-generation chips. Ollama's announcement of its MLX engine reported large prefill and decode gains on Qwen3.5-35B-A3B, but that test compared a different quantization (NVFP4 on MLX against Q4_K_M before) on an unnamed chip, so treat it as an indication, not a benchmark.
• Choose llama.cpp for coverage. New models usually appear as GGUF files first, and GGUF offers the widest range of quantization levels, which helps when you need a model to squeeze into memory.

Getting started

The easy way: Ollama or LM Studio

Install Ollama and run a model that fits your memory, for example:

ollama run qwen3.6:35b

Ollama requires macOS 14 Sonoma or newer and uses the GPU only on Apple Silicon. LM Studio also requires macOS 14 or newer on Apple Silicon and recommends 16 GB of memory or more; its comparison with other runtimes is in our runtime guide.

MLX directly, with mlx-lm

MLX needs Apple Silicon, macOS 14 or newer and a native arm64 Python (3.10 or newer for MLX; the current mlx-lm release requires 3.11 or newer). Check that your Python is native; this should print arm:

python -c "import platform; print(platform.processor())"

Then install and run. Without --model, mlx-lm uses its default model, mlx-community/Llama-3.2-3B-Instruct-4bit:

pip install mlx-lm
mlx_lm.generate --prompt "How tall is Mt Everest?"
mlx_lm.chat

To serve a model to other apps through an OpenAI-style API on localhost:8080:

mlx_lm.server --model mlx-community/Mistral-7B-Instruct-v0.3-4bit

The project's own documentation says this server is not recommended for production because it implements only basic security checks; for a shared setup, use a full server such as Ollama or LM Studio.

llama.cpp directly

brew install llama.cpp
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF

-hf downloads a GGUF model from Hugging Face; replace the repository with the model you want. llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF starts an OpenAI-compatible server with a built-in web chat. Check the port it prints at startup: release v0.6.0 (which Homebrew installs today) listens on 8080, while builds from 9 October 2026 onwards default to 9931. You can always set it with --port.

Tips for getting more out of your Mac

• Close memory-hungry apps before loading a large model. Browsers with many tabs can hold several gigabytes.
• Watch the context length. The KV cache grows with every token of context. For Qwen3-32B our estimate rises from 21.9 GiB at 8K to 27.9 GiB at 32K. Models with hybrid attention, such as Qwen3.6-27B, grow much more slowly.
• Quantize the KV cache if you need long contexts: in Ollama, set OLLAMA_KV_CACHE_TYPE=q8_0, which uses about half the memory of the default f16 cache; mlx-lm's server has a --kv-bits option.
• Prefer MoE models on lower-bandwidth chips. On a base or Pro chip, a 30–35B MoE model is far more pleasant to use than a 32B dense one.

Mac or PC?

A Mac gives you more model memory per dollar and per watt; a PC graphics card gives you more bandwidth per gigabyte. If the models you want fit in 24–32 GB, a GPU will usually answer faster. If you want 70B dense models or 100B-plus MoE models on one quiet machine, a Mac with 64–128 GB or more is the simpler path. Our GPU guide covers the PC side of the decision.