Safety
Adaptive Duration Model for Text Speech Alignment
arXiv:2507.22612v3 Announce Type: replace-cross Abstract: Speech-to-text alignment is a critical component of neural text to speech (TTS) models. Autoregressive TTS models typically use an attention m
arXiv:2507.22612v3 Announce Type: replace-cross Abstract: Speech-to-text alignment is a critical component of neural text to speech (TTS) models. Autoregressive TTS models typically use an attention mechanism to learn these alignments on-line, while non-autoregressive end to end TTS models rely on durations extracted from external sources. In this paper, we propose a novel duration prediction framework that can give promising phoneme-level duration distribution with given text. In our experiments, the proposed duration model has more precise prediction and adaptation ability to conditions, compared to previous baseline models. Specifically, it makes a considerable improvement on phoneme-level alignment accuracy and makes the performance of zero-shot TTS models more robust to the mismatch between prompt audio and input audio.
Related
- Mask, Sample, Revise: A Revisable CTMC Inference Stack for Guided Discrete Flow Matching Text-to-Speech
- Continuous-Time Acoustic Modelling with Neural Controlled Differential Equations
- Gradient-Based Speech-to-Text Alignment for Any ASR Model: From CTC to Speech LLMs
- CTC-TTS: LLM-based dual-streaming text-to-speech with CTC alignment
Source: arXiv cs.AI | 2026-10-16