Runtimes & apps Published:
vLLM v0.31 adds `vllm preload` for fast restarts and gates per-request multimodal options
vLLM v0.31.0 introduced a `vllm preload` command that runs a daemon keeping already-quantized weights in GPU memory, so a restarted engine maps them instead of reloading from disk. It also brought draft-model speculative decoding to Model Runner V2 and more speed work for DeepSeek-V4.1-Flash. As a security change, per-request `mm_processor_kwargs` and `media_io_kwargs` are now rejected unless the server starts with `--trust-request-mm-kwargs`.
Why it matters locally
Restarting a local vLLM server to change settings no longer means waiting for the weights to load again, but clients that send per-request multimodal options will get errors until you opt in.
Source
Read the official source: vLLM v0.31.0 release notes (GitHub) ↗Summaries are written by OnPrem World from the official source. We link to the original and never republish it.
More news
- NVIDIA RTX Spark laptops with up to 128 GB of unified memory go on sale October 16
- AMD's next Ryzen AI Halo: Ryzen AI Max+ PRO 495 with up to 192 GB of unified memory
- Open WebUI 0.12 adds two-factor sign-in, shared chats you can reply in, and realtime voice
- Ollama 0.40.2 upgrades older model downloads in the background and keeps backups
- EmbeddingGemma 2: a 740M multimodal embedding model for local search and RAG