Llama 3.2 3B Instruct
Llama 3.2 3B Instruct by Meta (3.21B parameters). At Q4_K_M with an 8K context it needs about 3.3 GiB of memory. A 3B model for low-memory machines and quick summaries: small enough to run unquantized (FP16) on a 12 GB card or a 16 GB Mac.
Last updated: 2026-10-13
Model facts
Run it
ollama run llama3.2:3b
Memory needed
Weights + KV cache + runtime overhead, in GiB. Lower quantization = less memory, slightly lower quality.
| Quantization | 4K context | 8K context | 32K context |
|---|---|---|---|
| Q4_K_M | 2.8 GiB | 3.3 GiB | 5.9 GiB |
| Q5_K_M | 3.2 GiB | 3.6 GiB | 6.2 GiB |
| Q6_K | 3.5 GiB | 4.0 GiB | 6.6 GiB |
| Q8_0 | 4.3 GiB | 4.7 GiB | 7.3 GiB |
| FP16 | 7.2 GiB | 7.7 GiB | 10.3 GiB |
Which hardware runs it
Best quantization that fits each device with a 8K context. Estimates, not benchmarks. See the methodology.
Frequently asked questions
How much VRAM does Llama 3.2 3B Instruct need?
With an 8K context, about 3.3 GiB at Q4_K_M and 4.7 GiB at Q8_0 (weights + KV cache + runtime overhead). Longer contexts need more memory for the KV cache.
Can Llama 3.2 3B Instruct run on an GeForce RTX 4090?
Yes. Llama 3.2 3B Instruct fits on the GeForce RTX 4090 (24 GB) at FP16 with an 8K context, using about 7.7 GiB, at an estimated 110 tokens/s.
Can I run Llama 3.2 3B Instruct on a Mac?
Yes. Llama 3.2 3B Instruct fits on the M3 Ultra (96 GB) at FP16 with an 8K context, using about 7.7 GiB, at an estimated 83 tokens/s.
Can Llama 3.2 3B Instruct be used commercially?
Llama 3.2 3B Instruct is released under Llama 3.2 Community License. The licence allows commercial use (read its conditions).
Sources
- https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/
- https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE
- https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD.md
- https://huggingface.co/unsloth/Llama-3.2-3B-Instruct/resolve/main/config.json
- https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
- https://ollama.com/library/llama3.2:3b/blobs/dde5aa3fc5ff
- https://huggingface.co/api/models/meta-llama/Llama-3.2-3B-Instruct?expand[]=safetensors
- https://ollama.com/library/llama3.2/tags