Runtimes & apps Published: Oct 28, 2026

llama.cpp v0.8.0 adds realtime voice and text-to-speech endpoints to its server, plus LFM3 support

What happened

llama.cpp v0.8.0, released on October 28, adds a `/v1/realtime` endpoint that implements the OpenAI realtime protocol for conversation and transcription, a `POST /tts` endpoint for text-to-speech models, and an experimental streaming-audio API. It supports Liquid AI's LFM3 family, including LFM3-Audio-2B and LFM3-Omni-2B, which handle speech recognition, speech output and voice chat in one model, along with new architectures such as DeepSeek V4.1 Flash, Muse Spark Open and Flint. MoE experts kept in host memory now use a 50/50 SLRU cache instead of plain LRU, and the built-in web UI can browse and download models from Hugging Face, search conversations and compact them.

Why it matters locally: A local voice assistant that listens and talks back can now run on one open-source server with an OpenAI-style realtime API, without sending audio to the cloud.

Three ways to read it

For the market

Realtime voice has mostly been a cloud API feature, so an OpenAI-compatible realtime endpoint in llama.cpp could make it easier for app builders to switch between cloud and local backends. It also likely raises the bar for other local runtimes that do not yet serve speech in and out.

In plain words

One of the most widely used free programs for running AI on your own computer can now hold a spoken conversation: you talk, it listens, and it answers out loud. Everything stays on your machine, like a smart speaker that never phones home.

For practitioners

Point an existing OpenAI realtime client at `/v1/realtime` and test latency and turn-taking with LFM3-Audio-2B or LFM3-Omni-2B on your own hardware; the streaming-audio API is marked experimental. Before upgrading, note that the CLI now requires `--model` or `-hf`, and that tensor parallelism for `stepfun` and `laguna` models now spans 4 to 10 GPUs.