Runtimes & apps Published: Nov 2, 2026

vLLM v0.33 drops CUDA 12.9, moves to ROCm 10.0 and fits Qwen3.8-Flash-Next on one DGX Spark

What happened

vLLM v0.33.0, released on November 2 with 927 commits from 389 contributors, publishes wheels and images for CUDA 13.0 only and upgrades to PyTorch 2.14. On AMD, the default image and wheels move to ROCm 10.0, with ROCm 7.2 still published, and a new 3-4 bit `ROCM_OCTAVE` KV cache holds 4.6 times the KV tokens of fp16 on an RX 7900 XTX. Packed NVFP4 embeddings now fit Qwen3.8-Flash-Next-NVFP4 on a single DGX Spark, and new models include Reflection Beam-1.0, Muse Spark Open, Recursive Flint and Step-5 Preview. Medusa, the MLP speculator and the legacy Ray compiled-graph executor are removed.

Why it matters locally: Anyone pinned to CUDA 12.9 has to change their setup before upgrading, while AMD owners and single-box DGX Spark users get longer contexts or bigger models from hardware they already have.

Three ways to read it

For the market

The ROCm 10.0 default and a 3-4 bit KV cache that works on a consumer RX 7900 XTX likely help AMD look like a practical serving option beyond its data center cards. Fitting a Qwen3.8-Flash-Next build on one DGX Spark could also make small desktop systems easier to justify for teams that do not want a rack.

In plain words

vLLM is the software many companies use to run an AI model as a service on their own graphics cards. This update stops supporting one older driver toolkit, packs the model's working notes more tightly on AMD cards, and lets a large model fit on one small NVIDIA desktop box. It is like repacking a suitcase so the same bag holds more.

For practitioners

Move to the CUDA 13.0 artifacts first, since the `-cu129` wheels and images are gone, and check that NIXL is 1.5 or newer if you use that connector. Replace Medusa, the MLP speculator or the Ray compiled-graph executor if you rely on them, and handle HTTP 503 for requests sent while generation is paused. Test `ROCM_OCTAVE` against your own quality checks before enabling it, and treat speed figures such as DeepSeek-V4.1-Flash with DSpark on GB300 going from 1373 to 3390 output tok/s as one PR's benchmark.