RTX 6000 Ada Generation

Name: RTX 6000 Ada Generation
Brand: NVIDIA

NVIDIA RTX 6000 Ada Generation: 48 GB of memory at 960 GB/s. Below: the open models it can run, estimated at 48 GB with an 8K context. The same 48 GB as the RTX A6000 with 25% more bandwidth (960 GB/s), so 70B models at 4-bit generate proportionally faster.

Last updated: 2026-10-14

Specifications

Memory	48 GB GDDR6 (ECC)[src]
Memory bandwidth	960 GB/s[src]
Power	300 W[src]
Launch	2022-12[src]
Launch price	US$6,800[src]
Architecture	Ada Lovelace (AD102)[src]
Software	CUDA

What it can run

Best-quality quantization that fits with a 8K context, and the estimated generation speed. Estimates, not benchmarks. See the methodology.

Model	Params	Best quant	Memory	tok/s (est.)
Llama 3.3 70B Instruct	70B	Q4_K_M	44.5 GiB	16
Qwen3.5-35B-A3B	35B (3B)	Q8_0	37.0 GiB	211
Qwen3.6-35B-A3B	35B (3B)	Q8_0	37.0 GiB	211
Qwen3-32B	32.8B	Q8_0	36.6 GiB	19
DeepSeek-R1-Distill-Qwen-32B	32.76B	Q8_0	36.5 GiB	19
Gemma 4 31B IT	30.7B	Q8_0	33.5 GiB	21
Qwen3-30B-A3B (2507)	30.5B (3.3B)	Q8_0	32.9 GiB	192
Qwen3-Coder-30B-A3B-Instruct	30.5B (3.3B)	Q8_0	32.9 GiB	192
GLM-4.7-Flash (30B-A3B)	30B (3B)	Q8_0	32.1 GiB	211
Qwen3.5-27B	27B	Q8_0	29.1 GiB	23
Qwen3.6-27B	27B	Q8_0	29.1 GiB	23
Qwen3.8-27B	27B	Q8_0	29.1 GiB	23
Gemma 3 27B IT	27B	Q8_0	29.6 GiB	23
Gemma 4 26B A4B IT	25.2B (3.8B)	Q8_0	27.0 GiB	166
Mistral Small 3.2 24B Instruct (2506)	24B	Q8_0	26.7 GiB	26
Magistral Small 1.2 (2509)	24B	Q8_0	26.7 GiB	26
Devstral Small 2 24B Instruct (2512)	24B	Q8_0	26.7 GiB	26
gpt-oss-20b	21B (3.6B)	FP16	41.8 GiB	93
Phi-4 (14B)	14B	FP16	29.4 GiB	24
Gemma 3 12B IT	12B	FP16	24.8 GiB	28
Gemma 4 12B Unified IT	11.95B	FP16	24.2 GiB	28
Qwen3.5-9B	9B	FP16	18.4 GiB	37
Qwen3-8B	8.2B	FP16	17.7 GiB	41
DeepSeek-R1-0528-Qwen3-8B	8.19B	FP16	17.6 GiB	41
Llama 3.1 8B Instruct	8B	FP16	17.1 GiB	42
Gemma 4 E4B IT	8B	FP16	16.4 GiB	42
Llama 3.2 3B Instruct	3.21B	FP16	7.7 GiB	105
Kimi K2.6	1000B (32B)	Doesn't fit	–	–
DeepSeek-R1-0528 (671B)	671B (37B)	Doesn't fit	–	–
DeepSeek-V4-Flash	284B (13B)	Doesn't fit	–	–
Qwen3.8-Flash-Next	180B (6B)	Doesn't fit	–	–
Qwen3.5-122B-A10B	122B (10B)	Doesn't fit	–	–
Mistral Small 4 119B (2603)	119B (6.5B)	Doesn't fit	–	–
gpt-oss-120b	117B (5.1B)	Doesn't fit	–	–
Llama 4 Scout 17B-16E Instruct	109B (17B)	Doesn't fit	–	–
GLM-4.5-Air	106B (12B)	Doesn't fit	–	–
Qwen3-Coder-Next (80B-A3B)	80B (3B)	Doesn't fit	–	–

Prices

No prices recorded for this device yet. All prices →

Frequently asked questions

Which LLMs can the RTX 6000 Ada Generation run?

With 48 GB and an 8K context, the largest models that fit include: Llama 3.3 70B Instruct, Qwen3.5-35B-A3B, Qwen3.6-35B-A3B, Qwen3-32B, DeepSeek-R1-Distill-Qwen-32B. See the full table below.

How fast is the RTX 6000 Ada Generation for local LLMs?

Generation speed is mostly set by memory bandwidth (960 GB/s). For Llama 3.1 8B Instruct at FP16, expect roughly 42 tokens/s; bigger models are slower.

Can the RTX 6000 Ada Generation run 70B models?

Yes. Llama 3.3 70B Instruct fits on the RTX 6000 Ada Generation (48 GB) at Q4_K_M with an 8K context, using about 44.5 GiB, at an estimated 16 tokens/s.