# Running LLMs on a Mac: unified memory, how much RAM to buy, MLX vs llama.cpp

> OnPrem World · https://onpremworld.com/en/guides/mac-local-llm · Updated: 2026-10-14 · CC BY 4.0

**Any Apple Silicon Mac can run local LLMs, and the amount of memory you buy decides how big a model it can hold.** Macs have no separate video memory: the CPU and GPU share one pool of unified memory, so a 64 GB Mac can give its GPU far more memory than any consumer graphics card has. Two limits apply. macOS lets the GPU use only part of that memory by default, and the speed of generation follows the chip's memory bandwidth, which ranges from 153 GB/s on a base M5 to 1,200 GB/s on an M5 Ultra. As a rule of thumb, 16 GB runs 7–12B models, 32 GB handles today's popular 27–35B models at 4-bit, 64 GB reaches 70B, and 128 GB or more is for the large mixture-of-experts models. For software, the easiest start is Ollama or LM Studio, both of which can use Apple's MLX framework or llama.cpp underneath. The numbers below come from our [hardware](https://onpremworld.com/en/hardware.md) and [model](https://onpremworld.com/en/models.md) data; check any configuration in the [calculator](https://onpremworld.com/en/can-i-run.md).

## How unified memory works, and the GPU limit

On a PC, a model has to fit in the graphics card's VRAM, typically 8 to 32 GB. On an Apple Silicon Mac, the CPU and GPU read the same memory. Apple's MLX documentation describes it this way: both processors have direct access to the same memory pool, so arrays never need to be copied between them. For local AI, that means the memory you configure when you buy the Mac is, mostly, memory the model can use.

Mostly, because macOS caps how much memory the GPU can wire. Metal reports the cap as `recommendedMaxWorkingSetSize`, which Apple documents only as an approximation of how much the GPU can allocate without hurting performance; it does not publish a percentage. Measurements reported by llama.cpp's maintainer give an idea:

- On a 32 GB M1 Pro the reported limit was 21,845 MB, about two-thirds of the memory ([PR #1826](https://github.com/ggml-org/llama.cpp/pull/1826#issuecomment-1595028325)).
- In his guide to running gpt-oss, a 192 GB Mac's default is given as 154 GB, about 80% ([discussion #15396](https://github.com/ggml-org/llama.cpp/discussions/15396)).

Our estimates assume the GPU can use 75% of a Mac's memory, which is in the middle of that range. Smaller Macs may get somewhat less.

### Raising the limit

If a model fits in your Mac's memory but not under the default cap, you can raise the cap from Terminal. The value is in megabytes; this example allows 56 GB on a 64 GB Mac (56 × 1024 = 57344):

```bash
sudo sysctl iogpu.wired_limit_mb=57344
```

The setting does not survive a reboot. The mlx-lm documentation advises a value larger than the model but smaller than the machine's total memory; in practice leave several gigabytes for macOS and your apps, or the system will start swapping. Watch the Memory Pressure graph in Activity Monitor while the model runs: memory pressure combines free memory, swap rate, wired memory and file cache, and a rising graph means you have pushed too far.

## How much memory to buy

These tables show the highest-quality quantization (up to Q8_0) that fits each configuration with an 8K context, and the estimated generation speed. They are estimates, not benchmarks, and assume the default GPU memory limit described above.

### 16 GB: [M5](https://onpremworld.com/en/hardware/apple-m5.md)

| Model | Parameters | Best quant that fits | Memory needed | tok/s (est.) |
|---|---:|---|---:|---:|
| [Llama 3.2 3B Instruct](https://onpremworld.com/en/models/llama-3.2-3b.md) | 3.21B | Q8_0 | 4.7 GiB | 29 |
| [Gemma 4 E4B IT](https://onpremworld.com/en/models/gemma-4-e4b.md) | 8B | Q8_0 | 9.1 GiB | 12 |
| [Qwen3-8B](https://onpremworld.com/en/models/qwen3-8b.md) | 8.2B | Q8_0 | 10.1 GiB | 11 |
| [Llama 3.1 8B Instruct](https://onpremworld.com/en/models/llama-3.1-8b.md) | 8B | Q8_0 | 9.8 GiB | 12 |
| [Gemma 3 12B IT](https://onpremworld.com/en/models/gemma-3-12b.md) | 12B | Q6_K | 11.0 GiB | 10 |
| [Phi-4 (14B)](https://onpremworld.com/en/models/phi-4.md) | 14B | Q5_K_M | 11.8 GiB | 10.0 |
| [gpt-oss-20b](https://onpremworld.com/en/models/gpt-oss-20b.md) | 21B (3.6B active) | Doesn't fit | 13.1 GiB | – |
| [Mistral Small 3.2 24B Instruct (2506)](https://onpremworld.com/en/models/mistral-small-3.2-24b.md) | 24B | Doesn't fit | 16.0 GiB | – |

*Estimates from our [methodology](https://onpremworld.com/en/methodology.md), using data as of 2026-10-14. Real speed depends on the runtime, drivers and prompt length.*

A 16 GB Mac is a capable machine for 7–12B models. It runs out at the 20B class: even [gpt-oss-20b](https://onpremworld.com/en/models/gpt-oss-20b.md), an efficient MoE model, needs about 13.1 GiB by our estimate, more than the GPU's default share of 16 GB.

### 32 GB: [M5](https://onpremworld.com/en/hardware/apple-m5.md)

| Model | Parameters | Best quant that fits | Memory needed | tok/s (est.) |
|---|---:|---|---:|---:|
| [Qwen3-8B](https://onpremworld.com/en/models/qwen3-8b.md) | 8.2B | Q8_0 | 10.1 GiB | 11 |
| [Gemma 3 12B IT](https://onpremworld.com/en/models/gemma-3-12b.md) | 12B | Q8_0 | 13.8 GiB | 7.8 |
| [gpt-oss-20b](https://onpremworld.com/en/models/gpt-oss-20b.md) | 21B (3.6B active) | Q8_0 | 22.5 GiB | 26 |
| [Mistral Small 3.2 24B Instruct (2506)](https://onpremworld.com/en/models/mistral-small-3.2-24b.md) | 24B | Q6_K | 21.1 GiB | 5.0 |
| [Qwen3.6-27B](https://onpremworld.com/en/models/qwen3.6-27b.md) | 27B | Q6_K | 22.7 GiB | 4.5 |
| [Qwen3-32B](https://onpremworld.com/en/models/qwen3-32b.md) | 32.8B | Q4_K_M | 21.9 GiB | 5.0 |
| [Qwen3.6-35B-A3B](https://onpremworld.com/en/models/qwen3.6-35b-a3b.md) | 35B (3B active) | Q4_K_M | 21.4 GiB | 55 |
| [Gemma 4 31B IT](https://onpremworld.com/en/models/gemma-4-31b.md) | 30.7B | Q5_K_M | 22.9 GiB | 4.6 |

*Estimates from our [methodology](https://onpremworld.com/en/methodology.md), using data as of 2026-10-14. Real speed depends on the runtime, drivers and prompt length.*

32 GB is the most useful step up. It opens the 24–35B class, which is where local models start to feel close to hosted assistants for writing and coding. Note the speed column: on a base M5, a dense 32B model is slow, while the MoE models run several times faster.

### 64 GB: [M5 Pro](https://onpremworld.com/en/hardware/apple-m5-pro.md)

| Model | Parameters | Best quant that fits | Memory needed | tok/s (est.) |
|---|---:|---|---:|---:|
| [Qwen3.6-27B](https://onpremworld.com/en/models/qwen3.6-27b.md) | 27B | Q8_0 | 29.1 GiB | 7.0 |
| [Qwen3-32B](https://onpremworld.com/en/models/qwen3-32b.md) | 32.8B | Q8_0 | 36.6 GiB | 5.7 |
| [Qwen3.6-35B-A3B](https://onpremworld.com/en/models/qwen3.6-35b-a3b.md) | 35B (3B active) | Q8_0 | 37.0 GiB | 63 |
| [Gemma 4 31B IT](https://onpremworld.com/en/models/gemma-4-31b.md) | 30.7B | Q8_0 | 33.5 GiB | 6.1 |
| [Llama 3.3 70B Instruct](https://onpremworld.com/en/models/llama-3.3-70b.md) | 70B | Q4_K_M | 44.5 GiB | 4.7 |
| [Qwen3-Coder-Next (80B-A3B)](https://onpremworld.com/en/models/qwen3-coder-next.md) | 80B (3B active) | Doesn't fit | 48.1 GiB | – |
| [gpt-oss-120b](https://onpremworld.com/en/models/gpt-oss-120b.md) | 117B (5.1B active) | Doesn't fit | 70.1 GiB | – |
| [GLM-4.5-Air](https://onpremworld.com/en/models/glm-4.5-air.md) | 106B (12B active) | Doesn't fit | 64.8 GiB | – |

*Estimates from our [methodology](https://onpremworld.com/en/methodology.md), using data as of 2026-10-14. Real speed depends on the runtime, drivers and prompt length.*

64 GB brings 70B dense models within reach at 4-bit, though slowly on a Pro chip, and lets 27–35B models run at Q8_0, close to their original quality. The large MoE models, such as [gpt-oss-120b](https://onpremworld.com/en/models/gpt-oss-120b.md), still do not fit.

### 128 GB: [M5 Max (18-core CPU / 40-core GPU)](https://onpremworld.com/en/hardware/apple-m5-max-40c-gpu.md)

| Model | Parameters | Best quant that fits | Memory needed | tok/s (est.) |
|---|---:|---|---:|---:|
| [Llama 3.3 70B Instruct](https://onpremworld.com/en/models/llama-3.3-70b.md) | 70B | Q8_0 | 75.7 GiB | 5.4 |
| [Llama 4 Scout 17B-16E Instruct](https://onpremworld.com/en/models/llama-4-scout.md) | 109B (17B active) | Q6_K | 89.8 GiB | 28 |
| [gpt-oss-120b](https://onpremworld.com/en/models/gpt-oss-120b.md) | 117B (5.1B active) | Q6_K | 95.0 GiB | 95 |
| [GLM-4.5-Air](https://onpremworld.com/en/models/glm-4.5-air.md) | 106B (12B active) | Q6_K | 87.3 GiB | 40 |
| [Mistral Small 4 119B (2603)](https://onpremworld.com/en/models/mistral-small-4-119b.md) | 119B (6.5B active) | Q5_K_M | 83.4 GiB | 86 |
| [Qwen3.5-122B-A10B](https://onpremworld.com/en/models/qwen3.5-122b-a10b.md) | 122B (10B active) | Q5_K_M | 85.5 GiB | 56 |
| [Qwen3-Coder-Next (80B-A3B)](https://onpremworld.com/en/models/qwen3-coder-next.md) | 80B (3B active) | Q8_0 | 83.8 GiB | 125 |
| [Qwen3.8-Flash-Next](https://onpremworld.com/en/models/qwen3.8-flash-next.md) | 180B (6B active) | Doesn't fit | 107.4 GiB | – |

*Estimates from our [methodology](https://onpremworld.com/en/methodology.md), using data as of 2026-10-14. Real speed depends on the runtime, drivers and prompt length.*

128 GB is where a Mac does something no single consumer GPU can: it runs 100–120B mixture-of-experts models such as gpt-oss-120b, GLM-4.5-Air and Qwen3.5-122B-A10B at usable speeds, because only a few billion parameters are active per token.

### 256 GB and 512 GB: [M5 Ultra](https://onpremworld.com/en/hardware/apple-m5-ultra.md)

The Ultra chips go further still: up to 512 GB on the [M3 Ultra](https://onpremworld.com/en/hardware/apple-m3-ultra.md) and [M5 Ultra](https://onpremworld.com/en/hardware/apple-m5-ultra.md). Even so, the very largest open models are a stretch. [DeepSeek-R1-0528 (671B)](https://onpremworld.com/en/models/deepseek-r1-0528.md) needs about 398.8 GiB at Q4_K_M by our estimate, which is more than the GPU's default share of 512 GB; it only fits if you raise the limit and leave macOS little room.

## Why bandwidth decides speed

When a model generates text, every new token requires reading all of its active weights from memory, so generation speed is capped by memory bandwidth. Apple Silicon chips differ widely:

| Hardware | Memory | Bandwidth | Launch price | US$ per GB |
|---|---:|---:|---:|---:|
| [M5](https://onpremworld.com/en/hardware/apple-m5.md) | 16–32 GB | 153 GB/s | US$1,599 | – |
| [M6 (16GB configuration, 153 GB/s)](https://onpremworld.com/en/hardware/apple-m6-16gb.md) | 16 GB | 153 GB/s | US$899 | US$56 |
| [M6 (24GB/32GB configurations, 170 GB/s)](https://onpremworld.com/en/hardware/apple-m6-24-32gb.md) | 24–32 GB | 170 GB/s | – | – |
| [M5 Pro](https://onpremworld.com/en/hardware/apple-m5-pro.md) | 24–64 GB | 307 GB/s | US$2,199 | – |
| [M5 Max (18-core CPU / 32-core GPU)](https://onpremworld.com/en/hardware/apple-m5-max-32c-gpu.md) | 36 GB | 460 GB/s | US$3,599 | US$100 |
| [M5 Max (18-core CPU / 40-core GPU)](https://onpremworld.com/en/hardware/apple-m5-max-40c-gpu.md) | 48–128 GB | 614 GB/s | US$4,099 | – |
| [M5 Ultra](https://onpremworld.com/en/hardware/apple-m5-ultra.md) | 96–512 GB | 1,200 GB/s | US$5,499 | – |
| [M4 Pro](https://onpremworld.com/en/hardware/apple-m4-pro.md) | 24–64 GB | 273 GB/s | US$1,399 | – |
| [M4 Max (16-core CPU / 40-core GPU)](https://onpremworld.com/en/hardware/apple-m4-max-40c-gpu.md) | 48–128 GB | 546 GB/s | US$3,699 | – |
| [M3 Ultra](https://onpremworld.com/en/hardware/apple-m3-ultra.md) | 96–512 GB | 819 GB/s | US$3,999 | – |

Here is the same dense model, [Qwen3-32B](https://onpremworld.com/en/models/qwen3-32b.md) at Q4_K_M, across Macs that can hold it, with two PC graphics cards for reference:

| Hardware | Bandwidth | Fits? (21.9 GiB) | tok/s (est.) |
|---|---:|---|---:|
| [M5](https://onpremworld.com/en/hardware/apple-m5.md) (32 GB) | 153 GB/s | Yes | 5.0 |
| [M6 (24GB/32GB configurations, 170 GB/s)](https://onpremworld.com/en/hardware/apple-m6-24-32gb.md) (32 GB) | 170 GB/s | Yes | 5.6 |
| [M4 Pro](https://onpremworld.com/en/hardware/apple-m4-pro.md) (48 GB) | 273 GB/s | Yes | 8.9 |
| [M5 Pro](https://onpremworld.com/en/hardware/apple-m5-pro.md) (48 GB) | 307 GB/s | Yes | 10 |
| [M5 Max (18-core CPU / 32-core GPU)](https://onpremworld.com/en/hardware/apple-m5-max-32c-gpu.md) (36 GB) | 460 GB/s | Yes | 15 |
| [M5 Max (18-core CPU / 40-core GPU)](https://onpremworld.com/en/hardware/apple-m5-max-40c-gpu.md) (64 GB) | 614 GB/s | Yes | 20 |
| [M3 Ultra](https://onpremworld.com/en/hardware/apple-m3-ultra.md) (96 GB) | 819 GB/s | Yes | 27 |
| [M5 Ultra](https://onpremworld.com/en/hardware/apple-m5-ultra.md) (96 GB) | 1,200 GB/s | Yes | 39 |
| [GeForce RTX 4090](https://onpremworld.com/en/hardware/rtx-4090.md) | 1,008 GB/s | Yes | 35 |
| [GeForce RTX 5090](https://onpremworld.com/en/hardware/rtx-5090.md) | 1,792 GB/s | Yes | 63 |

*Estimates from our [methodology](https://onpremworld.com/en/methodology.md), using data as of 2026-10-14. Real speed depends on the runtime, drivers and prompt length.*

The chip tier matters more than the generation: an M5 Max has about four times the bandwidth of a base M5, and an M5 Ultra about twice an M5 Max. For dense models, a Max or Ultra chip is the difference between a usable assistant and a slow one.

Mixture-of-experts models change the picture, because only their active parameters are read per token. Here is [gpt-oss-120b](https://onpremworld.com/en/models/gpt-oss-120b.md) (117B (5.1B active)) on large-memory machines, including two unified-memory PCs and a workstation GPU:

| Hardware | Bandwidth | Fits? (70.1 GiB) | tok/s (est.) |
|---|---:|---|---:|
| [M4 Max (16-core CPU / 40-core GPU)](https://onpremworld.com/en/hardware/apple-m4-max-40c-gpu.md) (128 GB) | 546 GB/s | Yes | 115 |
| [M5 Max (18-core CPU / 40-core GPU)](https://onpremworld.com/en/hardware/apple-m5-max-40c-gpu.md) (128 GB) | 614 GB/s | Yes | 129 |
| [M3 Ultra](https://onpremworld.com/en/hardware/apple-m3-ultra.md) (256 GB) | 819 GB/s | Yes | 172 |
| [M5 Ultra](https://onpremworld.com/en/hardware/apple-m5-ultra.md) (256 GB) | 1,200 GB/s | Yes | 252 |
| [DGX Spark](https://onpremworld.com/en/hardware/nvidia-dgx-spark.md) (128 GB) | 273 GB/s | Yes | 53 |
| [Ryzen AI Max+ 395 (Strix Halo) systems](https://onpremworld.com/en/hardware/amd-ryzen-ai-max-plus-395.md) (128 GB) | 256 GB/s | Yes | 50 |
| [RTX PRO 6000 Blackwell Workstation Edition](https://onpremworld.com/en/hardware/nvidia-rtx-pro-6000-blackwell.md) | 1,792 GB/s | Yes | 406 |

*Estimates from our [methodology](https://onpremworld.com/en/methodology.md), using data as of 2026-10-14. Real speed depends on the runtime, drivers and prompt length.*

Treat MoE estimates as upper bounds: routing between experts adds work the bandwidth formula does not count.

### Prompt processing and the M5's Neural Accelerators

Generation is only half the story. Before the first word of the answer, the model must process your whole prompt, and that step depends on compute rather than bandwidth. It matters when you paste long documents or use coding tools that send large contexts. Apple's M5 generation adds Neural Accelerators to each GPU core. In Apple's own tests with MLX on an M5 MacBook Pro, time to first token was up to about four times faster than on an M4, while generation was 19–27% faster, matching the M5's higher memory bandwidth. MLX uses the accelerators on macOS 26.2 or later. Our estimates cover generation only.

## MLX or llama.cpp?

Both run models on the Mac's GPU. The differences are in formats, quantization options and which apps use them.

| | MLX (mlx-lm) | llama.cpp |
|---|---|---|
| Made by | Apple machine learning research | The open-source ggml project |
| Model format | MLX weights, thousands published by the mlx-community organization on Hugging Face | GGUF files, the most widely published local format |
| Quantization | 4-bit with group size 64 by default; also other bit widths and the mxfp4, nvfp4 and mxfp8 formats | K-quants such as Q4_K_M, importance-matrix IQ quants, Q8_0 and more |
| Runs on | Apple Silicon (with builds for other platforms) | macOS, Windows and Linux, with many GPU backends; Metal is on by default on macOS |
| Install | `pip install mlx-lm` | `brew install llama.cpp` |
| Inside apps | Ollama (default on Apple Silicon for supported model architectures since version 0.40.0) and LM Studio (MLX engine since 0.3.4) | Ollama (for other models), LM Studio and many more |
| Licence | MIT | MIT |

In practice:

- **Use the app you like and let it choose.** Since Ollama 0.40.0 (September 2026), models whose architecture MLX supports run on MLX by default on Apple Silicon, and the rest run on llama.cpp. LM Studio offers both engines and lets you pick MLX or GGUF downloads.
- **Expect MLX to shine on prompt processing**, especially on M5-generation chips. Ollama's announcement of its MLX engine reported large prefill and decode gains on Qwen3.5-35B-A3B, but that test compared a different quantization (NVFP4 on MLX against Q4_K_M before) on an unnamed chip, so treat it as an indication, not a benchmark.
- **Choose llama.cpp for coverage.** New models usually appear as GGUF files first, and GGUF offers the widest range of quantization levels, which helps when you need a model to squeeze into memory.

## Getting started

### The easy way: Ollama or LM Studio

[Install Ollama](https://onpremworld.com/en/guides/install-ollama.md) and run a model that fits your memory, for example:

```bash
ollama run qwen3.6:35b
```

Ollama requires macOS 14 Sonoma or newer and uses the GPU only on Apple Silicon. LM Studio also requires macOS 14 or newer on Apple Silicon and recommends 16 GB of memory or more; its [comparison with other runtimes](https://onpremworld.com/en/guides/ollama-vs-lm-studio-vs-llama-cpp-vs-vllm.md) is in our runtime guide.

### MLX directly, with mlx-lm

MLX needs Apple Silicon, macOS 14 or newer and a native arm64 Python (3.10 or newer for MLX; the current mlx-lm release requires 3.11 or newer). Check that your Python is native; this should print `arm`:

```bash
python -c "import platform; print(platform.processor())"
```

Then install and run. Without `--model`, mlx-lm uses its default model, `mlx-community/Llama-3.2-3B-Instruct-4bit`:

```bash
pip install mlx-lm
mlx_lm.generate --prompt "How tall is Mt Everest?"
mlx_lm.chat
```

To serve a model to other apps through an OpenAI-style API on `localhost:8080`:

```bash
mlx_lm.server --model mlx-community/Mistral-7B-Instruct-v0.3-4bit
```

The project's own documentation says this server is not recommended for production because it implements only basic security checks; for a shared setup, use a full server such as Ollama or LM Studio.

### llama.cpp directly

```bash
brew install llama.cpp
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
```

`-hf` downloads a GGUF model from Hugging Face; replace the repository with the model you want. `llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF` starts an OpenAI-compatible server with a built-in web chat. Check the port it prints at startup: release v0.6.0 (which Homebrew installs today) listens on 8080, while builds from 9 October 2026 onwards default to 9931. You can always set it with `--port`.

## Tips for getting more out of your Mac

- **Close memory-hungry apps** before loading a large model. Browsers with many tabs can hold several gigabytes.
- **Watch the context length.** The KV cache grows with every token of context. For [Qwen3-32B](https://onpremworld.com/en/models/qwen3-32b.md) our estimate rises from 21.9 GiB at 8K to 27.9 GiB at 32K. Models with hybrid attention, such as [Qwen3.6-27B](https://onpremworld.com/en/models/qwen3.6-27b.md), grow much more slowly.
- **Quantize the KV cache** if you need long contexts: in Ollama, set `OLLAMA_KV_CACHE_TYPE=q8_0`, which uses about half the memory of the default f16 cache; mlx-lm's server has a `--kv-bits` option.
- **Prefer MoE models on lower-bandwidth chips.** On a base or Pro chip, a 30–35B MoE model is far more pleasant to use than a 32B dense one.

## Mac or PC?

A Mac gives you more model memory per dollar and per watt; a PC graphics card gives you more bandwidth per gigabyte. If the models you want fit in 24–32 GB, a GPU will usually answer faster. If you want 70B dense models or 100B-plus MoE models on one quiet machine, a Mac with 64–128 GB or more is the simpler path. Our [GPU guide](https://onpremworld.com/en/guides/choosing-a-gpu.md) covers the PC side of the decision.

## Frequently asked questions

### How much RAM does a Mac need to run local LLMs?

16 GB runs 7–12B models well. 32 GB is the practical minimum for the popular 27–35B models at 4-bit: by our estimate Qwen3-32B needs about 21.9 GiB at Q4_K_M with an 8K context. 64 GB fits 70B models at 4-bit, and 128 GB or more is for 100B-plus mixture-of-experts models such as gpt-oss-120b.

### Can a Mac use all of its memory for the model?

Not by default. macOS limits how much unified memory the GPU can wire; Apple does not publish the share, and measurements reported by llama.cpp's maintainer range from about two-thirds on a 32 GB Mac to about 80% on a 192 GB Mac. You can raise the limit with sudo sysctl iogpu.wired_limit_mb, but it resets at reboot and macOS still needs memory for itself.

### Is MLX faster than llama.cpp on a Mac?

Often, especially for reading long prompts on M5-generation chips, whose GPU Neural Accelerators MLX can use on macOS 26.2 or later. Generation speed is limited mainly by memory bandwidth for both. Results vary by model and quantization, so the practical answer is to try both: LM Studio and Ollama can run either engine.

### Is a Mac Studio better than a PC with an RTX 5090 for local AI?

It depends on model size. An RTX 5090 has more bandwidth (1,792 GB/s) but only 32 GB of memory, so models that fit run faster on it. A Mac Studio with M5 Ultra has 1,200 GB/s and up to 512 GB, so it can load models no single consumer GPU can, at lower speed for dense models.

### Can I run local LLMs on an Intel Mac?

Only slowly. Ollama runs on Intel Macs but on the CPU only, while MLX and LM Studio require Apple Silicon. For local AI, an Apple Silicon Mac is the realistic starting point.

### Does the M5's Neural Accelerator make answers generate faster?

Mostly it speeds up processing the prompt. Apple's own MLX tests on an M5 MacBook Pro showed time to first token up to about 4 times faster than on an M4, while generation was 19–27% faster, in line with the M5's higher memory bandwidth.

## Sources

- https://github.com/ml-explore/mlx
- https://ml-explore.github.io/mlx/build/html/install.html
- https://ml-explore.github.io/mlx/build/html/usage/unified_memory.html
- https://github.com/ml-explore/mlx-lm
- https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/SERVER.md
- https://huggingface.co/mlx-community
- https://developer.apple.com/documentation/metal/mtldevice/recommendedmaxworkingsetsize
- https://github.com/ggml-org/llama.cpp/pull/1826#issuecomment-1595028325
- https://github.com/ggml-org/llama.cpp/discussions/15396
- https://github.com/ggml-org/llama.cpp/discussions/2182
- https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md
- https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md
- https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md
- https://github.com/ollama/ollama/releases/tag/v0.40.0
- https://ollama.com/blog/mlx
- https://docs.ollama.com/macos
- https://lmstudio.ai/blog/lmstudio-v0.3.4
- https://lmstudio.ai/docs/app/system-requirements
- https://machinelearning.apple.com/research/exploring-llms-mlx-m5
- https://support.apple.com/guide/activity-monitor/view-memory-usage-actmntr1004/mac