GeForce RTX 4060 Ti 16GB
NVIDIA GeForce RTX 4060 Ti 16GB: 16 GB of memory at 288 GB/s. Below: the open models it can run, estimated at 16 GB with an 8K context. 16 GB of VRAM for $499 at launch fits 24B models at 4-bit, but at 288 GB/s it has the lowest bandwidth of the 16 GB cards here, so generation is slow.
Last updated: 2026-10-14
Specifications
What it can run
Best-quality quantization that fits with a 8K context, and the estimated generation speed. Estimates, not benchmarks. See the methodology.
| Model | Params | Best quant | Memory | tok/s (est.) |
|---|---|---|---|---|
| Gemma 4 26B A4B IT | 25.2B (3.8B) | Q4_K_M | 15.7 GiB | 88 |
| Mistral Small 3.2 24B Instruct (2506) | 24B | Q4_K_M | 16.0 GiB | 14 |
| Magistral Small 1.2 (2509) | 24B | Q4_K_M | 16.0 GiB | 14 |
| Devstral Small 2 24B Instruct (2512) | 24B | Q4_K_M | 16.0 GiB | 14 |
| gpt-oss-20b | 21B (3.6B) | Q5_K_M | 15.3 GiB | 79 |
| Phi-4 (14B) | 14B | Q6_K | 13.3 GiB | 17 |
| Gemma 3 12B IT | 12B | Q8_0 | 13.8 GiB | 16 |
| Gemma 4 12B Unified IT | 11.95B | Q8_0 | 13.3 GiB | 16 |
| Qwen3.5-9B | 9B | Q8_0 | 10.1 GiB | 21 |
| Qwen3-8B | 8.2B | Q8_0 | 10.1 GiB | 23 |
| DeepSeek-R1-0528-Qwen3-8B | 8.19B | Q8_0 | 10.1 GiB | 23 |
| Llama 3.1 8B Instruct | 8B | Q8_0 | 9.8 GiB | 24 |
| Gemma 4 E4B IT | 8B | Q8_0 | 9.1 GiB | 24 |
| Llama 3.2 3B Instruct | 3.21B | FP16 | 7.7 GiB | 31 |
| Kimi K2.6 | 1000B (32B) | Doesn't fit | – | – |
| DeepSeek-R1-0528 (671B) | 671B (37B) | Doesn't fit | – | – |
| DeepSeek-V4-Flash | 284B (13B) | Doesn't fit | – | – |
| Qwen3.8-Flash-Next | 180B (6B) | Doesn't fit | – | – |
| Qwen3.5-122B-A10B | 122B (10B) | Doesn't fit | – | – |
| Mistral Small 4 119B (2603) | 119B (6.5B) | Doesn't fit | – | – |
| gpt-oss-120b | 117B (5.1B) | Doesn't fit | – | – |
| Llama 4 Scout 17B-16E Instruct | 109B (17B) | Doesn't fit | – | – |
| GLM-4.5-Air | 106B (12B) | Doesn't fit | – | – |
| Qwen3-Coder-Next (80B-A3B) | 80B (3B) | Doesn't fit | – | – |
| Llama 3.3 70B Instruct | 70B | Doesn't fit | – | – |
| Qwen3.5-35B-A3B | 35B (3B) | Doesn't fit | – | – |
| Qwen3.6-35B-A3B | 35B (3B) | Doesn't fit | – | – |
| Qwen3-32B | 32.8B | Doesn't fit | – | – |
| DeepSeek-R1-Distill-Qwen-32B | 32.76B | Doesn't fit | – | – |
| Gemma 4 31B IT | 30.7B | Doesn't fit | – | – |
| Qwen3-30B-A3B (2507) | 30.5B (3.3B) | Doesn't fit | – | – |
| Qwen3-Coder-30B-A3B-Instruct | 30.5B (3.3B) | Doesn't fit | – | – |
| GLM-4.7-Flash (30B-A3B) | 30B (3B) | Doesn't fit | – | – |
| Qwen3.5-27B | 27B | Doesn't fit | – | – |
| Qwen3.6-27B | 27B | Doesn't fit | – | – |
| Qwen3.8-27B | 27B | Doesn't fit | – | – |
| Gemma 3 27B IT | 27B | Doesn't fit | – | – |
Prices
No prices recorded for this device yet. All prices →
Frequently asked questions
Which LLMs can the GeForce RTX 4060 Ti 16GB run?
With 16 GB and an 8K context, the largest models that fit include: Gemma 4 26B A4B IT, Mistral Small 3.2 24B Instruct (2506), Magistral Small 1.2 (2509), Devstral Small 2 24B Instruct (2512), gpt-oss-20b. See the full table below.
How fast is the GeForce RTX 4060 Ti 16GB for local LLMs?
Generation speed is mostly set by memory bandwidth (288 GB/s). For Llama 3.1 8B Instruct at Q8_0, expect roughly 24 tokens/s; bigger models are slower.
Can the GeForce RTX 4060 Ti 16GB run 70B models?
Llama 3.3 70B Instruct does not fit on the GeForce RTX 4060 Ti 16GB (16 GB), even at Q4_K_M with an 8K context.
Sources
- https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4060-4060ti/
- https://www.profesionalreview.com/2023/07/21/nvidia-rtx-4060-ti-16gb-review/
- https://nvidianews.nvidia.com/news/geforce-rtx-4060-family-is-here-nvidias-revolutionary-ada-lovelace-architecture-comes-to-core-gamers-everywhere-starting-at-299