Models Published:
Qwen3.8-Flash-Next previews the Qwen4 architecture: 125B MoE with 6B active
Qwen released Qwen3.8-Flash-Next on August 26 as an experimental preview of the architecture that will underpin Qwen4. Its language model has 125B mixture-of-experts parameters with 6B active per token, plus a 51B n-gram embedding table and a 4B multi-token-prediction layer, about 180B parameters in the checkpoint. Attention is split between Gated DeltaNet linear layers and a new Qwen Sparse Attention that picks small blocks of context rather than single tokens, and the native context is 262,144 tokens. The weights use the Qwen Community License, which allows commercial use with conditions.
Why it matters locally
With only 6B parameters active per token it can generate quickly, but all ~180B stored parameters still have to fit in memory, far more than one consumer GPU holds; runtime support arrived in Ollama 0.33.1 (MLX engine) and llama.cpp v0.6.0.
Source
Read the official source: Qwen3.8-Flash-Next model card (Hugging Face) ↗Summaries are written by OnPrem World from the official source. We link to the original and never republish it.
On this site
More news
- NVIDIA RTX Spark laptops with up to 128 GB of unified memory go on sale October 16
- AMD's next Ryzen AI Halo: Ryzen AI Max+ PRO 495 with up to 192 GB of unified memory
- Open WebUI 0.12 adds two-factor sign-in, shared chats you can reply in, and realtime voice
- Ollama 0.40.2 upgrades older model downloads in the background and keeps backups
- EmbeddingGemma 2: a 740M multimodal embedding model for local search and RAG