OnPrem World地端世界
中文

Mistral Small 3.2 24B Instruct (2506)

Mistral Small 3.2 24B Instruct (2506) by Mistral AI (24B parameters). At Q4_K_M with an 8K context it needs about 16.0 GiB of memory. Mistral's 24B with image input and function calling, under Apache-2.0; it fits a 16 GB card at 4-bit.

Last updated: 2026-10-13

Model facts

DeveloperMistral AI
Parameters24B[src]
Native context131,072 tokens[src]
LicenceApache-2.0[src]
Commercial useYes
KV cacheAll 40 layers (standard attention)[src]
Released2025-06

Good for: chat · instruction following · function calling · image understanding · European languages

Run it

ollama run mistral-small3.2:24b

Memory needed

Weights + KV cache + runtime overhead, in GiB. Lower quantization = less memory, slightly lower quality.

Quantization4K context8K context32K context
Q4_K_M15.4 GiB16.0 GiB19.7 GiB
Q5_K_M17.8 GiB18.4 GiB22.2 GiB
Q6_K20.5 GiB21.1 GiB24.8 GiB
Q8_026.1 GiB26.7 GiB30.4 GiB
FP1648.1 GiB48.7 GiB52.4 GiB

Which hardware runs it

Best quantization that fits each device with a 8K context. Estimates, not benchmarks. See the methodology.

HardwareMemoryBest quanttok/s (est.)
GeForce RTX 509032 GBQ8_049
GeForce RTX 508016 GBQ4_K_M46
GeForce RTX 5070 Ti16 GBQ4_K_M43
GeForce RTX 409024 GBQ6_K36
GeForce RTX 4080 SUPER16 GBQ4_K_M35
Radeon RX 7900 XTX24 GBQ6_K34
GeForce RTX 309024 GBQ6_K33
Radeon RX 7900 XT20 GBQ5_K_M33
Radeon RX 9070 XT16 GBQ4_K_M31
RTX 6000 Ada Generation48 GBQ8_026
RTX PRO 6000 Blackwell Workstation Edition96 GBFP1626
GeForce RTX 5060 Ti 16GB16 GBQ4_K_M22
RTX A600048 GBQ8_021
RTX PRO 5000 72GB Blackwell72 GBFP1620
Radeon AI PRO R970032 GBQ8_018
Arc Pro B7032 GBQ8_017
M5 Ultra512 GBFP1616
Arc Pro B6024 GBQ6_K16
Radeon RX 9060 XT 16GB16 GBQ4_K_M15
GeForce RTX 4060 Ti 16GB16 GBQ4_K_M14
M5 Max (18-core CPU / 32-core GPU)36 GBQ8_012
M3 Ultra512 GBFP1611
M4 Max (14-core CPU / 32-core GPU)36 GBQ8_010
M5 Max (18-core CPU / 40-core GPU)128 GBFP168.3
M5 Pro64 GBQ8_07.8
M4 Max (16-core CPU / 40-core GPU)128 GBFP167.4
M4 Pro64 GBQ8_07.0
M6 (24GB/32GB configurations, 170 GB/s)32 GBQ6_K5.6
M532 GBQ6_K5.0
RTX Spark (N1X) laptops & compact desktops128 GBFP163.8
DGX Spark128 GBFP163.4
Ryzen AI Max+ PRO 495 (Gorgon Halo) systems192 GBFP163.4
Ryzen AI Max+ 395 (Strix Halo) systems128 GBFP163.2
GeForce RTX 3060 (12 GB)12 GBDoesn't fit–
GeForce RTX 507012 GBDoesn't fit–
Arc B58012 GBDoesn't fit–
M6 (16GB configuration, 153 GB/s)16 GBDoesn't fit–

Frequently asked questions

How much VRAM does Mistral Small 3.2 24B Instruct (2506) need?

With an 8K context, about 16.0 GiB at Q4_K_M and 26.7 GiB at Q8_0 (weights + KV cache + runtime overhead). Longer contexts need more memory for the KV cache.

Can Mistral Small 3.2 24B Instruct (2506) run on an GeForce RTX 4090?

Yes. Mistral Small 3.2 24B Instruct (2506) fits on the GeForce RTX 4090 (24 GB) at Q6_K with an 8K context, using about 21.1 GiB, at an estimated 36 tokens/s.

Can I run Mistral Small 3.2 24B Instruct (2506) on a Mac?

Yes. Mistral Small 3.2 24B Instruct (2506) fits on the M3 Ultra (96 GB) at FP16 with an 8K context, using about 48.7 GiB, at an estimated 11 tokens/s.

Can Mistral Small 3.2 24B Instruct (2506) be used commercially?

Mistral Small 3.2 24B Instruct (2506) is released under Apache-2.0. The licence allows commercial use (read its conditions).

Sources