Causal Diffusion Decoding for 25Hz Semantic Tokens
Replace chunk-wise diffusion models with strictly causal flow matching decoders to eliminate lookahead latency in speech synthesis tokens.
Source paper
Hangrui Hu, Xinfa Zhu, Ting He, Dake Guo, Bin Zhang, Xiong Wang, Zhifang Guo, Ziyue Jiang, Hongkun Hao, Zishan Guo, Xinyu Zhang, Pei Zhang, Baosong Yang, Jin Xu, Jingren Zhou, Junyang Lin
Generated by AI. The paper’s authors did not write these.
Replace chunk-wise diffusion models with strictly causal flow matching decoders to eliminate lookahead latency in speech synthesis tokens.
A system that predicts a high-level semantic token at fixed intervals to condition the generation of finer-grained acoustic tokens, ensuring long-form stability in speech generation.
A synthetic data augmentation pipeline that generates multilingual speech in a target speaker's voice to explicitly align speaker embeddings with underperforming Romance languages.
The paper introduces Qwen3-TTS, a large-scale multilingual text-to-speech model designed for both high-fidelity voice cloning and ultra-low-latency streaming. By utilizing a dual-track autoregressive architecture and two specialized neural codecs (one for semantic quality, one for streaming speed), the model achieves state-of-the-art performance in zero-shot cloning and instruction following while enabling real-time synthesis.
Qwen3-TTS employs a dual-track Language Model (LM) that processes text and speech tokens concurrently. It utilizes two distinct tokenizers: a 25Hz semantic-rich single-codebook tokenizer with a Diffusion Transformer (DiT) decoder for high quality, and a 12Hz multi-codebook tokenizer with a causal ConvNet decoder for <100ms latency. The model is trained on 5 million hours of speech using a multi-stage pipeline that includes general pre-training, high-quality continual pre-training, and Direct Preference Optimization (DPO).