OnPrem World地端世界
中文

Llama 3.3 70B Instruct

Llama 3.3 70B Instruct by Meta (70B parameters). At Q4_K_M with an 8K context it needs about 44.5 GiB of memory. A dense 70B model: at 4-bit it needs a 48 GB card or a 64 GB unified-memory machine, and because every token reads all 70B weights, it is slow on low-bandwidth hardware.

Last updated: 2026-10-13

Model facts

DeveloperMeta
Parameters70B[src]
Native context131,072 tokens[src]
LicenceLlama 3.3 Community License[src]
Commercial useYes
KV cacheAll 80 layers (standard attention)[src]
Released2024-12

Good for: chat · RAG · multilingual (8 languages) · tool use

Run it

ollama run llama3.3:70b

Memory needed

Weights + KV cache + runtime overhead, in GiB. Lower quantization = less memory, slightly lower quality.

Quantization4K context8K context32K context
Q4_K_M43.2 GiB44.5 GiB52.0 GiB
Q5_K_M50.4 GiB51.7 GiB59.2 GiB
Q6_K58.1 GiB59.4 GiB66.9 GiB
Q8_074.5 GiB75.7 GiB83.2 GiB
FP16138.7 GiB139.9 GiB147.4 GiB

Which hardware runs it

Best quantization that fits each device with a 8K context. Estimates, not benchmarks. See the methodology.

HardwareMemoryBest quanttok/s (est.)
RTX PRO 6000 Blackwell Workstation Edition96 GBQ8_017
RTX PRO 5000 72GB Blackwell72 GBQ6_K16
RTX 6000 Ada Generation48 GBQ4_K_M16
RTX A600048 GBQ4_K_M13
M5 Ultra512 GBFP165.6
M5 Max (18-core CPU / 40-core GPU)128 GBQ8_05.4
M4 Max (16-core CPU / 40-core GPU)128 GBQ8_04.8
M5 Pro64 GBQ4_K_M4.7
M4 Pro64 GBQ4_K_M4.2
M3 Ultra512 GBFP163.8
RTX Spark (N1X) laptops & compact desktops128 GBQ8_02.4
DGX Spark128 GBQ8_02.2
Ryzen AI Max+ 395 (Strix Halo) systems128 GBQ8_02.1
Ryzen AI Max+ PRO 495 (Gorgon Halo) systems192 GBFP161.2
GeForce RTX 3060 (12 GB)12 GBDoesn't fit–
GeForce RTX 309024 GBDoesn't fit–
GeForce RTX 4060 Ti 16GB16 GBDoesn't fit–
GeForce RTX 4080 SUPER16 GBDoesn't fit–
GeForce RTX 409024 GBDoesn't fit–
GeForce RTX 5060 Ti 16GB16 GBDoesn't fit–
GeForce RTX 507012 GBDoesn't fit–
GeForce RTX 5070 Ti16 GBDoesn't fit–
GeForce RTX 508016 GBDoesn't fit–
GeForce RTX 509032 GBDoesn't fit–
Radeon RX 7900 XT20 GBDoesn't fit–
Radeon RX 7900 XTX24 GBDoesn't fit–
Radeon RX 9060 XT 16GB16 GBDoesn't fit–
Radeon RX 9070 XT16 GBDoesn't fit–
Arc B58012 GBDoesn't fit–
Radeon AI PRO R970032 GBDoesn't fit–
Arc Pro B6024 GBDoesn't fit–
Arc Pro B7032 GBDoesn't fit–
M4 Max (14-core CPU / 32-core GPU)36 GBDoesn't fit–
M532 GBDoesn't fit–
M5 Max (18-core CPU / 32-core GPU)36 GBDoesn't fit–
M6 (16GB configuration, 153 GB/s)16 GBDoesn't fit–
M6 (24GB/32GB configurations, 170 GB/s)32 GBDoesn't fit–

Frequently asked questions

How much VRAM does Llama 3.3 70B Instruct need?

With an 8K context, about 44.5 GiB at Q4_K_M and 75.7 GiB at Q8_0 (weights + KV cache + runtime overhead). Longer contexts need more memory for the KV cache.

Can Llama 3.3 70B Instruct run on an GeForce RTX 4090?

Llama 3.3 70B Instruct does not fit on the GeForce RTX 4090 (24 GB), even at Q4_K_M with an 8K context.

Can I run Llama 3.3 70B Instruct on a Mac?

Yes. Llama 3.3 70B Instruct fits on the M3 Ultra (96 GB) at Q6_K with an 8K context, using about 59.4 GiB, at an estimated 9.2 tokens/s.

Can Llama 3.3 70B Instruct be used commercially?

Llama 3.3 70B Instruct is released under Llama 3.3 Community License. The licence allows commercial use (read its conditions).

Sources