# Llama 3.2 3B Instruct

> OnPrem World · https://onpremworld.com/en/models/llama-3.2-3b · Last updated: 2026-10-14 · CC BY 4.0

A 3B model for low-memory machines and quick summaries: small enough to run unquantized (FP16) on a 12 GB card or a 16 GB Mac.

|  |  |
|---|---|
| Developer | Meta |
| Parameters | 3.21B |
| Native context | 131,072 tokens |
| Licence | Llama 3.2 Community License |
| Commercial use | Yes |
| KV cache | All 28 layers (standard attention) |
| Released | 2024-09 |

```
ollama run llama3.2:3b
```

## Memory needed

Weights + KV cache + runtime overhead, in GiB. Lower quantization = less memory, slightly lower quality.

| Quantization | 4K | 8K | 32K |
|---|---|---|---|
| Q4_K_M | 2.8 GiB | 3.3 GiB | 5.9 GiB |
| Q5_K_M | 3.2 GiB | 3.6 GiB | 6.2 GiB |
| Q6_K | 3.5 GiB | 4.0 GiB | 6.6 GiB |
| Q8_0 | 4.3 GiB | 4.7 GiB | 7.3 GiB |
| FP16 | 7.2 GiB | 7.7 GiB | 10.3 GiB |

## Which hardware runs it (8K)

| Hardware | Memory | Best quant | tok/s (est.) |
|---|---|---|---|
| GeForce RTX 5090 | 32 GB | FP16 | 195 |
| RTX PRO 6000 Blackwell Workstation Edition | 96 GB | FP16 | 195 |
| RTX PRO 5000 72GB Blackwell | 72 GB | FP16 | 147 |
| M5 Ultra | 512 GB | FP16 | 121 |
| GeForce RTX 4090 | 24 GB | FP16 | 110 |
| GeForce RTX 5080 | 16 GB | FP16 | 105 |
| Radeon RX 7900 XTX | 24 GB | FP16 | 105 |
| RTX 6000 Ada Generation | 48 GB | FP16 | 105 |
| GeForce RTX 3090 | 24 GB | FP16 | 102 |
| GeForce RTX 5070 Ti | 16 GB | FP16 | 98 |
| Radeon RX 7900 XT | 20 GB | FP16 | 87 |
| RTX A6000 | 48 GB | FP16 | 84 |
| M3 Ultra | 512 GB | FP16 | 83 |
| GeForce RTX 4080 SUPER | 16 GB | FP16 | 80 |
| GeForce RTX 5070 | 12 GB | FP16 | 73 |
| Radeon RX 9070 XT | 16 GB | FP16 | 70 |
| Radeon AI PRO R9700 | 32 GB | FP16 | 70 |
| Arc Pro B70 | 32 GB | FP16 | 66 |
| M5 Max (18-core CPU / 40-core GPU) | 128 GB | FP16 | 62 |
| M4 Max (16-core CPU / 40-core GPU) | 128 GB | FP16 | 55 |
| Arc B580 | 12 GB | FP16 | 50 |
| Arc Pro B60 | 24 GB | FP16 | 50 |
| GeForce RTX 5060 Ti 16GB | 16 GB | FP16 | 49 |
| M5 Max (18-core CPU / 32-core GPU) | 36 GB | FP16 | 47 |
| M4 Max (14-core CPU / 32-core GPU) | 36 GB | FP16 | 42 |
| GeForce RTX 3060 (12 GB) | 12 GB | FP16 | 39 |
| Radeon RX 9060 XT 16GB | 16 GB | FP16 | 35 |
| GeForce RTX 4060 Ti 16GB | 16 GB | FP16 | 31 |
| M5 Pro | 64 GB | FP16 | 31 |
| RTX Spark (N1X) laptops & compact desktops | 128 GB | FP16 | 28 |
| M4 Pro | 64 GB | FP16 | 28 |
| DGX Spark | 128 GB | FP16 | 26 |
| Ryzen AI Max+ PRO 495 (Gorgon Halo) systems | 192 GB | FP16 | 26 |
| Ryzen AI Max+ 395 (Strix Halo) systems | 128 GB | FP16 | 24 |
| M6 (24GB/32GB configurations, 170 GB/s) | 32 GB | FP16 | 17 |
| M5 | 32 GB | FP16 | 15 |
| M6 (16GB configuration, 153 GB/s) | 16 GB | FP16 | 15 |

## Frequently asked questions

### How much VRAM does Llama 3.2 3B Instruct need?

With an 8K context, about 3.3 GiB at Q4_K_M and 4.7 GiB at Q8_0 (weights + KV cache + runtime overhead). Longer contexts need more memory for the KV cache.

### Can Llama 3.2 3B Instruct run on an GeForce RTX 4090?

Yes. Llama 3.2 3B Instruct fits on the GeForce RTX 4090 (24 GB) at FP16 with an 8K context, using about 7.7 GiB, at an estimated 110 tokens/s.

### Can I run Llama 3.2 3B Instruct on a Mac?

Yes. Llama 3.2 3B Instruct fits on the M3 Ultra (96 GB) at FP16 with an 8K context, using about 7.7 GiB, at an estimated 83 tokens/s.

### Can Llama 3.2 3B Instruct be used commercially?

Llama 3.2 3B Instruct is released under Llama 3.2 Community License. The licence allows commercial use (read its conditions).

## Sources

- https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/
- https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE
- https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD.md
- https://huggingface.co/unsloth/Llama-3.2-3B-Instruct/resolve/main/config.json
- https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
- https://ollama.com/library/llama3.2:3b/blobs/dde5aa3fc5ff
- https://huggingface.co/api/models/meta-llama/Llama-3.2-3B-Instruct?expand[]=safetensors
- https://ollama.com/library/llama3.2/tags