Llama 3.1 8B Instruct
Llama 3.1 8B Instruct by Meta (8B parameters). At Q4_K_M with an 8K context it needs about 6.2 GiB of memory. Meta's 8B general-purpose model with a 128K context. It fits a 12 GB card even at 8-bit (Q8_0).
Last updated: 2026-10-13
Model facts
Run it
ollama run llama3.1:8b
Memory needed
Weights + KV cache + runtime overhead, in GiB. Lower quantization = less memory, slightly lower quality.
| Quantization | 4K context | 8K context | 32K context |
|---|---|---|---|
| Q4_K_M | 5.7 GiB | 6.2 GiB | 9.2 GiB |
| Q5_K_M | 6.6 GiB | 7.1 GiB | 10.1 GiB |
| Q6_K | 7.4 GiB | 7.9 GiB | 10.9 GiB |
| Q8_0 | 9.3 GiB | 9.8 GiB | 12.8 GiB |
| FP16 | 16.6 GiB | 17.1 GiB | 20.1 GiB |
Which hardware runs it
Best quantization that fits each device with a 8K context. Estimates, not benchmarks. See the methodology.
Frequently asked questions
How much VRAM does Llama 3.1 8B Instruct need?
With an 8K context, about 6.2 GiB at Q4_K_M and 9.8 GiB at Q8_0 (weights + KV cache + runtime overhead). Longer contexts need more memory for the KV cache.
Can Llama 3.1 8B Instruct run on an GeForce RTX 4090?
Yes. Llama 3.1 8B Instruct fits on the GeForce RTX 4090 (24 GB) at FP16 with an 8K context, using about 17.1 GiB, at an estimated 44 tokens/s.
Can I run Llama 3.1 8B Instruct on a Mac?
Yes. Llama 3.1 8B Instruct fits on the M3 Ultra (96 GB) at FP16 with an 8K context, using about 17.1 GiB, at an estimated 33 tokens/s.
Can Llama 3.1 8B Instruct be used commercially?
Llama 3.1 8B Instruct is released under Llama 3.1 Community License. The licence allows commercial use (read its conditions).
Sources
- https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/MODEL_CARD.md
- https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE
- https://huggingface.co/RedHatAI/Llama-3.1-8B-Instruct/resolve/main/config.json
- https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct
- https://huggingface.co/api/models/meta-llama/Llama-3.1-8B-Instruct?expand[]=safetensors
- https://ollama.com/library/llama3.1/tags