# vLLM v0.34 sizes the KV cache from real free memory, removes Model Runner V1 and adds Mistral Large 4

> OnPrem World · https://onpremworld.com/en/news/2026-11-vllm-v0-34-extensible-kv-cache · CC BY 4.0

Published: 2026-11-16 · Runtimes & apps

vLLM v0.34.0, released with 1,061 commits from 412 contributors, turns on the extensible KV cache by default: the cache is now sized from the GPU memory actually free after warmup, and `gpu_memory_utilization` defaults to 1.0. The release notes give MiniMax-M3 on an H200 as an example, going from 433k to 672k cached tokens. Model Runner V1 is removed, the default block size moves from 16 to 64, ROCm 7.2 images are no longer published and `huggingface_hub` 2.0.0 or newer is required. Ten new models are supported, including Mistral Large 4 (1T total, 52B active, in FP8 and NVFP4), MiniMax-M3.1-Flash, the 30B-A4B Nemotron-3.5-Transcribe speech model and Ricursive Ripple-65B Preview.

## Why it matters locally

Most deployments get a bigger KV cache and more throughput without changing anything, but several defaults changed at once, so setups that relied on Model Runner V1, block size 16, ROCm 7.2 or `fp_quant` checkpoints break on upgrade.

## For the market

Squeezing more cache out of the same card is a quiet way to lower the cost per user of a self-hosted model, which could matter more to buyers than any single new model. Supporting Mistral Large 4 on NVIDIA Blackwell, Hopper and AMD MI350 also keeps AMD in the conversation for very large open models.

## In plain words

vLLM is the software many companies use to serve an AI model to many people from their own graphics cards. This version measures how much memory is really left once the model is loaded and uses all of it for the model's short-term memory, instead of guessing and leaving some unused. It is like filling a shelf after the books are in, not before.

## For practitioners

Before upgrading, remove `VLLM_USE_V2_MODEL_RUNNER=0`, decide whether to pin `--block-size 16`, and move AMD hosts to the ROCm 10.0 wheels or images. If the GPU is shared with other processes or you care most about per-token latency, test with `--gpu-memory-utilization 0.9` or `--no-enable-extensible-kv-cache`, since the notes report TPOT rising 6–18% on some H200 models at concurrency 128. `fp_quant` checkpoints no longer load, Marlin W4A8 users must switch to `--quantization humming`, and on RDNA3 and RDNA4 cards TRITON_ATTN is now the default attention backend.

## Source

[vLLM v0.34.0 release notes (GitHub)](https://github.com/vllm-project/vllm/releases/tag/v0.34.0)

## On this site

- Hardware: [Radeon RX 7900 XTX](https://onpremworld.com/en/hardware/amd-rx-7900-xtx.md)

[All news](https://onpremworld.com/en/news.md)

Summaries are written by OnPrem World from the official source. We link to the original and never republish it.
