Skip to content

Source paper

Qwen3-TTS Technical Report

Hangrui Hu, Xinfa Zhu, Ting He and 13 more

Hangrui Hu, Xinfa Zhu, Ting He, Dake Guo, Bin Zhang, Xiong Wang, Zhifang Guo, Ziyue Jiang, Hongkun Hao, Zishan Guo, Xinyu Zhang, Pei Zhang, Baosong Yang, Jin Xu, Jingren Zhou, Junyang Lin

arXiv 2601.15621 · 202663 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

The paper introduces Qwen3-TTS, a large-scale multilingual text-to-speech model designed for both high-fidelity voice cloning and ultra-low-latency streaming. By utilizing a dual-track autoregressive architecture and two specialized neural codecs (one for semantic quality, one for streaming speed), the model achieves state-of-the-art performance in zero-shot cloning and instruction following while enabling real-time synthesis.

How it works (AI summary)

Qwen3-TTS employs a dual-track Language Model (LM) that processes text and speech tokens concurrently. It utilizes two distinct tokenizers: a 25Hz semantic-rich single-codebook tokenizer with a Diffusion Transformer (DiT) decoder for high quality, and a 12Hz multi-codebook tokenizer with a causal ConvNet decoder for <100ms latency. The model is trained on 5 million hours of speech using a multi-stage pipeline that includes general pre-training, high-quality continual pre-training, and Direct Preference Optimization (DPO).