# Llama 3.1 8B Instruct

> OnPrem World · https://onpremworld.com/en/models/llama-3.1-8b · Last updated: 2026-10-16 · CC BY 4.0

Meta's 8B general-purpose model with a 128K context. It fits a 12 GB card even at 8-bit (Q8_0).

|  |  |
|---|---|
| Developer | Meta |
| Parameters | 8B |
| Native context | 131,072 tokens |
| Licence | Llama 3.1 Community License |
| Commercial use | Yes |
| KV cache | All 32 layers (standard attention) |
| Released | 2024-07 |

```
ollama run llama3.1:8b
```

## Memory needed

Weights + KV cache + runtime overhead, in GiB. Lower quantization = less memory, slightly lower quality.

| Quantization | 4K | 8K | 32K |
|---|---|---|---|
| Q4_K_M | 5.7 GiB | 6.2 GiB | 9.2 GiB |
| Q5_K_M | 6.6 GiB | 7.1 GiB | 10.1 GiB |
| Q6_K | 7.4 GiB | 7.9 GiB | 10.9 GiB |
| Q8_0 | 9.3 GiB | 9.8 GiB | 12.8 GiB |
| FP16 | 16.6 GiB | 17.1 GiB | 20.1 GiB |

## Which hardware runs it (8K)

| Hardware | Memory | Best quant | tok/s (est.) |
|---|---|---|---|
| DGX Station (GB300) | 252 GB | FP16 | 192 |
| GeForce RTX 5080 | 16 GB | Q8_0 | 68 |
| GeForce RTX 5090 | 32 GB | FP16 | 68 |
| RTX PRO 6000 Blackwell Workstation Edition | 96 GB | FP16 | 68 |
| RTX PRO 6000 Blackwell Max-Q Workstation Edition | 96 GB | FP16 | 68 |
| GeForce RTX 5070 Ti | 16 GB | Q8_0 | 64 |
| GeForce RTX 4080 SUPER | 16 GB | Q8_0 | 54 |
| RTX PRO 5000 72GB Blackwell | 72 GB | FP16 | 53 |
| RTX PRO 5000 48GB Blackwell | 48 GB | FP16 | 53 |
| GeForce RTX 5070 | 12 GB | Q8_0 | 50 |
| Radeon RX 9070 XT | 16 GB | Q8_0 | 48 |
| M5 Ultra | 512 GB | FP16 | 44 |
| GeForce RTX 4090 | 24 GB | FP16 | 41 |
| Radeon RX 7900 XTX | 24 GB | FP16 | 39 |
| RTX 6000 Ada Generation | 48 GB | FP16 | 39 |
| GeForce RTX 3090 | 24 GB | FP16 | 38 |
| RTX PRO 4500 Blackwell | 32 GB | FP16 | 36 |
| Arc B580 | 12 GB | Q8_0 | 35 |
| GeForce RTX 5060 Ti 16GB | 16 GB | Q8_0 | 34 |
| Radeon RX 7900 XT | 20 GB | FP16 | 33 |
| RTX A6000 | 48 GB | FP16 | 31 |
| M3 Ultra | 512 GB | FP16 | 31 |
| GeForce RTX 3060 (12 GB) | 12 GB | Q8_0 | 28 |
| RTX PRO 4000 Blackwell | 24 GB | FP16 | 28 |
| Radeon AI PRO R9700 | 32 GB | FP16 | 27 |
| Arc Pro B70 | 32 GB | FP16 | 25 |
| Radeon RX 9060 XT 16GB | 16 GB | Q8_0 | 25 |
| M5 Max (18-core CPU / 40-core GPU) | 128 GB | FP16 | 24 |
| GeForce RTX 4060 Ti 16GB | 16 GB | Q8_0 | 23 |
| M4 Max (16-core CPU / 40-core GPU) | 128 GB | FP16 | 21 |
| Arc Pro B60 | 24 GB | FP16 | 19 |
| M5 Max (18-core CPU / 32-core GPU) | 36 GB | FP16 | 18 |
| M4 Max (14-core CPU / 32-core GPU) | 36 GB | FP16 | 16 |
| M5 Pro | 64 GB | FP16 | 12 |
| M6 (16GB configuration, 153 GB/s) | 16 GB | Q8_0 | 11 |
| RTX Spark (N1X) laptops & compact desktops | 128 GB | FP16 | 11 |
| M4 Pro | 64 GB | FP16 | 11 |
| DGX Spark | 128 GB | FP16 | 10 |
| Ryzen AI Max+ PRO 495 (Gorgon Halo) systems | 192 GB | FP16 | 10 |
| Ryzen AI Max+ 395 (Strix Halo) systems | 128 GB | FP16 | 9.4 |
| M6 (24GB/32GB configurations, 170 GB/s) | 32 GB | FP16 | 6.8 |
| M5 | 32 GB | FP16 | 6.1 |

## Frequently asked questions

### How much VRAM does Llama 3.1 8B Instruct need?

With an 8K context, about 6.2 GiB at Q4_K_M and 9.8 GiB at Q8_0 (weights + KV cache + runtime overhead). Longer contexts need more memory for the KV cache.

### Can Llama 3.1 8B Instruct run on an GeForce RTX 4090?

Yes. Llama 3.1 8B Instruct fits on the GeForce RTX 4090 (24 GB) at FP16 with an 8K context, using about 17.1 GiB, at an estimated 41 tokens/s.

### Can I run Llama 3.1 8B Instruct on a Mac?

Yes. Llama 3.1 8B Instruct fits on the M3 Ultra (96 GB) at FP16 with an 8K context, using about 17.1 GiB, at an estimated 31 tokens/s.

### Can Llama 3.1 8B Instruct be used commercially?

Llama 3.1 8B Instruct is released under Llama 3.1 Community License. The licence allows commercial use (read its conditions).

## Sources

- https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/MODEL_CARD.md
- https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE
- https://huggingface.co/RedHatAI/Llama-3.1-8B-Instruct/resolve/main/config.json
- https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct
- https://huggingface.co/api/models/meta-llama/Llama-3.1-8B-Instruct?expand[]=safetensors
- https://ollama.com/library/llama3.1/tags