OnPrem World地端世界
中文

Devstral Small 2 24B Instruct (2512)

Devstral Small 2 24B Instruct (2512) by Mistral AI (24B parameters). At Q4_K_M with an 8K context it needs about 16.0 GiB of memory. Mistral's 24B model for coding agents, with a 256K context, under Apache-2.0; it fits a 16 GB card at 4-bit, and a 24 GB card leaves room for longer contexts.

Last updated: 2026-10-13

Model facts

DeveloperMistral AI
Parameters24B[src]
Native context262,144 tokens[src]
LicenceApache-2.0[src]
Commercial useYes
KV cacheAll 40 layers (standard attention)[src]
Released2025-12

Good for: agentic coding · software engineering · tool use · single 24 GB GPU / 32 GB Mac

Run it

ollama run devstral-small-2:24b

Memory needed

Weights + KV cache + runtime overhead, in GiB. Lower quantization = less memory, slightly lower quality.

Quantization4K context8K context32K context
Q4_K_M15.4 GiB16.0 GiB19.7 GiB
Q5_K_M17.8 GiB18.4 GiB22.2 GiB
Q6_K20.5 GiB21.1 GiB24.8 GiB
Q8_026.1 GiB26.7 GiB30.4 GiB
FP1648.1 GiB48.7 GiB52.4 GiB

Which hardware runs it

Best quantization that fits each device with a 8K context. Estimates, not benchmarks. See the methodology.

HardwareMemoryBest quanttok/s (est.)
GeForce RTX 509032 GBQ8_049
GeForce RTX 508016 GBQ4_K_M46
GeForce RTX 5070 Ti16 GBQ4_K_M43
GeForce RTX 409024 GBQ6_K36
GeForce RTX 4080 SUPER16 GBQ4_K_M35
Radeon RX 7900 XTX24 GBQ6_K34
GeForce RTX 309024 GBQ6_K33
Radeon RX 7900 XT20 GBQ5_K_M33
Radeon RX 9070 XT16 GBQ4_K_M31
RTX 6000 Ada Generation48 GBQ8_026
RTX PRO 6000 Blackwell Workstation Edition96 GBFP1626
GeForce RTX 5060 Ti 16GB16 GBQ4_K_M22
RTX A600048 GBQ8_021
RTX PRO 5000 72GB Blackwell72 GBFP1620
Radeon AI PRO R970032 GBQ8_018
Arc Pro B7032 GBQ8_017
M5 Ultra512 GBFP1616
Arc Pro B6024 GBQ6_K16
Radeon RX 9060 XT 16GB16 GBQ4_K_M15
GeForce RTX 4060 Ti 16GB16 GBQ4_K_M14
M5 Max (18-core CPU / 32-core GPU)36 GBQ8_012
M3 Ultra512 GBFP1611
M4 Max (14-core CPU / 32-core GPU)36 GBQ8_010
M5 Max (18-core CPU / 40-core GPU)128 GBFP168.3
M5 Pro64 GBQ8_07.8
M4 Max (16-core CPU / 40-core GPU)128 GBFP167.4
M4 Pro64 GBQ8_07.0
M6 (24GB/32GB configurations, 170 GB/s)32 GBQ6_K5.6
M532 GBQ6_K5.0
RTX Spark (N1X) laptops & compact desktops128 GBFP163.8
DGX Spark128 GBFP163.4
Ryzen AI Max+ PRO 495 (Gorgon Halo) systems192 GBFP163.4
Ryzen AI Max+ 395 (Strix Halo) systems128 GBFP163.2
GeForce RTX 3060 (12 GB)12 GBDoesn't fit–
GeForce RTX 507012 GBDoesn't fit–
Arc B58012 GBDoesn't fit–
M6 (16GB configuration, 153 GB/s)16 GBDoesn't fit–

Frequently asked questions

How much VRAM does Devstral Small 2 24B Instruct (2512) need?

With an 8K context, about 16.0 GiB at Q4_K_M and 26.7 GiB at Q8_0 (weights + KV cache + runtime overhead). Longer contexts need more memory for the KV cache.

Can Devstral Small 2 24B Instruct (2512) run on an GeForce RTX 4090?

Yes. Devstral Small 2 24B Instruct (2512) fits on the GeForce RTX 4090 (24 GB) at Q6_K with an 8K context, using about 21.1 GiB, at an estimated 36 tokens/s.

Can I run Devstral Small 2 24B Instruct (2512) on a Mac?

Yes. Devstral Small 2 24B Instruct (2512) fits on the M3 Ultra (96 GB) at FP16 with an 8K context, using about 48.7 GiB, at an estimated 11 tokens/s.

Can Devstral Small 2 24B Instruct (2512) be used commercially?

Devstral Small 2 24B Instruct (2512) is released under Apache-2.0. The licence allows commercial use (read its conditions).

Sources