Research
VoiceChat-TTS: A Low-Latency Continuous Speech Synthesis Model for Interactive Agents
arXiv:2608.13831v1 Announce Type: cross Abstract: Spoken dialogue is a natural form of human--computer interaction, yet most speech language models remain limited to turn-based operation and lack real
arXiv:2608.13831v1 Announce Type: cross Abstract: Spoken dialogue is a natural form of human--computer interaction, yet most speech language models remain limited to turn-based operation and lack real-time adaptability, such as user barge-in. Recent duplex speech-to-speech and speech-to-text models reduce latency by replacing multi-stage pipelines, but often compromise speech quality because accurate ASR, interruption handling, and high-fidelity synthesis must be optimized jointly. We propose VoiceChat-TTS, a low-latency, continuous, and streamable text-to-speech model for interactive agents. VoiceChat-TTS is driven directly by LLM text-token streams, supports explicit interruption via control tokens, and produces silence when no textual input is available. The model enables always-on, responsive speech generation while preserving modularity and high speech quality, and it supports mid-utterance interruptions without resetting the KV cache.
Related
- RT-SEMamba: Real-Time Speech Enhancement Mamba via Progressive Knowledge Distillation
- MELD: Mel-Spectrogram-Based Speech Language Modeling with Discrete Latent Variables
- Reference-Based Prosody and Rhythm Evaluation for Spoken Dialogue Systems
- How Should LLMs Listen While Speaking? A Study of User-Stream Routing in Full-Duplex Spoken Dialogue
Source: arXiv cs.CL | 2026-08-17