Local AI news
New open models, runtime releases and hardware for running AI on your own machine. Each item is a short summary in our own words, why it matters for local use, and a link to the official source.
Latest item: Oct 13, 2026 · RSS · JSON · Markdown
· Hardware
NVIDIA RTX Spark laptops with up to 128 GB of unified memory go on sale October 16
NVIDIA said on October 13 that the first RTX Spark laptops launch on October 16. RTX Spark is a single superchip for Windows 11 PCs that pairs a Grace CPU of up to 20 cores with a Blackwell RTX GPU, with up to 128 GB of unified memory and up to 1 petaflop of FP4 AI compute. NVIDIA says it runs models of up to 120B parameters with a 1M-token context, and laptop makers include ASUS, Dell, HP, Lenovo, Microsoft and MSI. The same announcement introduced Nemotron 3.5 Bolt, a 74B mixture-of-experts model with 9B active parameters that fits in 24 GB using 2-bit quantization.
Source: NVIDIA Blog
· Hardware
AMD's next Ryzen AI Halo: Ryzen AI Max+ PRO 495 with up to 192 GB of unified memory
AMD introduced its next Ryzen AI Halo developer platform on October 12, built on the Ryzen AI Max+ PRO 495 with 16 Zen 5 CPU cores, 40 RDNA 3.5 GPU compute units and up to 192 GB of LPDDR5x unified memory. Up to 160 GB of that memory can be assigned to the GPU, which AMD says is enough for models of 300 billion parameters and more at 4-bit quantization. It comes with a validated ROCm software stack and is sold through Micro Center, with Windows-based versions listed as coming soon.
Source: AMD blog
· Runtimes & apps
Open WebUI 0.12 adds two-factor sign-in, shared chats you can reply in, and realtime voice
Open WebUI v0.12.0 lets administrators require an authenticator app for every user, with recovery codes and a command-line reset for lost devices. Shared chats and folders can now allow replies, so several people write in one conversation and see each other's messages as they stream. A new Realtime call mode talks through an OpenAI Realtime voice model and hands real questions to the chat's selected model. The release also fixes several Ollama issues; for example, `max_tokens` sent through the API now limits reply length.
· Runtimes & apps
Ollama 0.40.2 upgrades older model downloads in the background and keeps backups
Ollama v0.40.2 upgrades models downloaded with earlier versions the first time you run them, for better performance and compatibility on its llama.cpp engine. To keep downgrades safe it leaves the original copy on disk as a backup for now, and the release notes include a command that removes only those backups. Ollama says a future release will delete them automatically.
· Models
EmbeddingGemma 2: a 740M multimodal embedding model for local search and RAG
Google DeepMind released EmbeddingGemma 2 on October 6, an Apache 2.0 embedding model that maps text, code, images, audio and video into one 768-dimensional space. Text-only use needs just the 270M-parameter text part, while the vision and audio encoders load only when needed, for 740M parameters in total. Vectors can be shortened to 512, 256 or 128 dimensions to save storage, and the context window is 8K tokens. Google lists Ollama, LM Studio, llama.cpp, MLX and vLLM among the supported runtimes.
Source: Google blog: EmbeddingGemma 2
· Runtimes & apps
llama.cpp v0.6.0 adds GLM-5.3-Flash, a decision-model API and faster Metal kernels
llama.cpp published v0.6.0 on October 5, one of its numbered releases alongside the daily builds. It adds support for Z.ai's 320B GLM-5.3-Flash and the Clef decision model, multi-token-prediction speculative decoding for Qwen4Exp models (about 1.5x faster decoding on DGX Spark), and a /v1/systemone server endpoint for decision models. On Apple GPUs it brings a new flash-attention kernel for F16 KV caches and matrix-multiplication kernels that are up to about 3x faster for speculative and batched decoding. The built-in web UI was also reworked around downloading models from Hugging Face.
· Runtimes & apps
vLLM v0.31 adds `vllm preload` for fast restarts and gates per-request multimodal options
vLLM v0.31.0 introduced a `vllm preload` command that runs a daemon keeping already-quantized weights in GPU memory, so a restarted engine maps them instead of reloading from disk. It also brought draft-model speculative decoding to Model Runner V2 and more speed work for DeepSeek-V4.1-Flash. As a security change, per-request `mm_processor_kwargs` and `media_io_kwargs` are now rejected unless the server starts with `--trust-request-mm-kwargs`.
· Runtimes & apps
Ollama 0.35 adds decision models that answer with probabilities instead of text
Ollama v0.35.0 added a /v1/systemone endpoint for decision models, based on TypeSafe's Jev API. Instead of writing prose, these models pick one of several options, give the probability that a condition is true, or score input against a rubric, returning probabilities with each answer. The first two are Nimble from Bespoke Labs and Tev1 from Together AI, aimed at jobs such as ticket triage, model routing and content classification.
· Runtimes & apps
Ollama 0.40 runs supported models on MLX by default on Apple Silicon
From v0.40.0, Ollama automatically runs model architectures that its MLX engine supports on Apple Silicon Macs, with no change to `ollama run`. The release names Qwen3.8, Gemma 4, Qwen3.6 and Qwen3.5 among the covered models, and adds MLX versions of its decision models and of the EmbeddingGemma 2 embedding model. Ollama says it will keep testing and enabling more models.
· Hardware
Mac mini with M6 and Mac Studio with M5 Ultra go on sale, from US$899 and US$5,499
Apple's new Mac mini and Mac Studio went on sale on September 22. The Mac mini starts at US$899 with M6 (configurable up to 32 GB) or US$1,699 with M5 Pro (up to 64 GB), and the Mac Studio starts at US$2,499 with M5 Max (up to 128 GB) or US$5,499 with M5 Ultra. Apple says the 512 GB Mac Studio configuration is coming in late October.
Source: Apple Newsroom
· Runtimes & apps
Open WebUI 0.11.4 shrinks its slim Docker image to about 175 MB
Open WebUI v0.11.4 cut its slim container image to about 175 MB, roughly 89% smaller than before, by leaving out the bundled local models and the libraries around them; the standard image also lost about 170 MB. The slim build has no built-in embedding or reranking model, so knowledge bases need embeddings from Ollama, OpenAI or Azure OpenAI. It also searches only through PostgreSQL with pgvector and cannot read PDF or Office files unless an external document extractor is configured.
· Runtimes & apps
Ollama 0.34.1: MLX safetensors import is stable, GGUF conversion moves to llama.cpp tools
Ollama v0.34.1 made `ollama create` from MLX safetensors a supported feature rather than an experiment, and improved MLX memory handling on Apple Silicon. The same release stopped converting safetensors into GGUF inside Ollama, so conversion and quantization now have to be done with llama.cpp's own tools first. Listing a large model library through /api/tags also became much faster, dropping from 3.1 s to 294 ms in Ollama's testing.
· Models
DeepSeek-V4.1-Flash: a 552B multimodal MoE with a quarter of the KV cache
DeepSeek released V4.1-Flash on September 10, with open weights on Hugging Face under the MIT licence. It is a 552B-parameter mixture-of-experts model that understands images natively and activates about 8B parameters per token while reading the prompt and 16B while generating. A new causal encoder-decoder layout and a second version of DeepSeek's compressed sparse attention cut the KV cache per token to about a quarter of V4-Flash's, with contexts of up to one million tokens.
· Runtimes & apps
vLLM v0.29 makes Model Runner V2 the default and starts retiring V1
vLLM v0.29.0 switched every model to Model Runner V2 by default, completing a rollout that began with pooling models. The project now considers V1 deprecated and aims to remove it in v0.32. The release also deprecates starting the server with `python -m vllm.entrypoints.openai.api_server` in favour of `vllm serve`, removes ten deprecated model architectures, and adds Qwen3.8-Flash-Next with BF16, FP8 and NVFP4 weights.
· Models
Z.ai releases GLM-5.3-Flash: a 320B multimodal MoE with 18B active, MIT licence
Z.ai released GLM-5.3-Flash on August 26, the first natively multimodal model in its GLM-5 series, with open weights on Hugging Face under the MIT licence. It has 320B parameters in total with 18B active per token and, for the first time in a GLM model, combines sparse and linear attention to cut the cost of long contexts. The model card lists vLLM, SGLang, KTransformers and Unsloth among the ways to run it. The bigger GLM-5.3, built on the GLM-5.2 base and improved through post-training, also has open weights, under a separate GLM-5.3 licence.
· Models
Qwen3.8-Flash-Next previews the Qwen4 architecture: 125B MoE with 6B active
Qwen released Qwen3.8-Flash-Next on August 26 as an experimental preview of the architecture that will underpin Qwen4. Its language model has 125B mixture-of-experts parameters with 6B active per token, plus a 51B n-gram embedding table and a 4B multi-token-prediction layer, about 180B parameters in the checkpoint. Attention is split between Gated DeltaNet linear layers and a new Qwen Sparse Attention that picks small blocks of context rather than single tokens, and the native context is 262,144 tokens. The weights use the Qwen Community License, which allows commercial use with conditions.
· Hardware
Apple announces M6 and M5 Ultra, with up to 512 GB of unified memory at 1.2 TB/s
Apple announced two chips on August 25: M6 for the new Mac mini and M5 Ultra for the new Mac Studio. M5 Ultra joins two dual-die M5 Max chips into Apple's first quad-die design, with up to 80 GPU cores, up to 512 GB of unified memory and up to 1.2 TB/s of memory bandwidth, 50% more than M3 Ultra. M6 is Apple's first 2 nm chip and supports up to 32 GB of memory at up to 170 GB/s. Apple says M5 Ultra can run language models with hundreds of billions of parameters entirely on the device.
Source: Apple Newsroom
· Runtimes & apps
Open WebUI 0.11.1 lets you approve each tool call before it runs
Open WebUI v0.11.1 lets you switch a conversation from running tools freely to asking first, so a model stops and waits for you to allow or deny each tool call. Administrators have to enable the feature, and your choice is remembered for the conversation. The release also adds a built-in tool that lets a model ask you up to three multiple-choice questions before it continues, and rebuilds streaming so replies send only new text instead of the whole message on every update.
· Runtimes & apps
Ollama 0.32.15 roughly halves time to first token by caching model metadata
Ollama v0.32.15 caches resolved model metadata between requests, which cut time to first token from about 995 ms to about 524 ms in Ollama's benchmarks. It also fixes a bug where chat and generate requests could hang after a parser error mid-stream, and handles Qwen 3.8 system messages that appear later in a conversation consistently. First-time desktop users get a new onboarding flow.
· Models
Qwen3.8-27B released: a dense 27B model that reads images and video, under Apache 2.0
Alibaba's Qwen team released Qwen3.8-27B on August 14, two days after the 2.4-trillion-parameter Qwen3.8 flagship, as the generation's compact dense model. It handles text, images and video, keeps thinking mode on by default and lets you tune reasoning depth per request. Only 16 of its 64 layers use standard attention; the rest are Gated DeltaNet linear-attention layers, and the native context is 262,144 tokens. Ollama added it the same day as qwen3.8:27b.
Summaries are written by OnPrem World from the official source. We link to the original and never republish it.