Abstract
RobuTrans is a robust text-to-speech (TTS) model designed to eliminate common alignment errors like word skipping or repeating in standard Transformer-based architectures. By replacing soft attention with a duration-based hard alignment and a pseudo non-causal decoder, RobuTrans achieves state-of-the-art naturalness while maintaining high robustness on challenging inputs such as URLs and long numerical sequences.
Introduction
Neural TTS models, such as Tacotron 2 and TransformerTTS, have set high standards for speech naturalness. However, these models are often fragile due to their reliance on "soft attention" mechanisms inherited from Neural Machine Translation (NMT). While NMT requires flexibility for word reordering, speech is inherently monotonic. This mismatch leads to significant failure modes, including word skipping, repeating, and "unstoppable" predictions where the model fails to identify the end of a sequence.
Furthermore, standard position embeddings often fail to generalize to sequences longer than those encountered during training, leading to garbled output for long texts.
Methods
RobuTrans introduces several architectural innovations to enforce stability and robustness while preserving naturalness.
Duration-Based Hard Attention
To solve alignment issues, RobuTrans replaces soft attention with a Duration Predictor. This module determines the exact number of frames each phoneme should occupy. The encoder states are then replicated (tiled) according to these predicted durations, ensuring a strictly monotonic alignment between text and audio.
Pseudo Non-Causal Attention (PNCA)
While hard alignment ensures robustness, it can lead to robotic-sounding speech if the decoder loses the holistic context provided by soft attention. RobuTrans introduces Pseudo Non-Causal Attention (PNCA). Unlike standard causal decoders that only look at past frames, PNCA allows the decoder to attend to the entire sequence of encoder hidden states for every generated frame. This restores the "global" context necessary for natural prosody without reintroducing the risk of alignment errors.
Linguistic and Positional Enhancements
RobuTrans utilizes rich linguistic features—including phonemic categories and prosodic markers like tone and break indices—rather than raw phonemes. Additionally, the model replaces fixed position embeddings with 1D Convolutional Neural Networks (CNNs) in the pre-nets to capture relative positional information, enabling better generalization to long sequences.
Experimental Setup
The model was trained on a 20-hour internal US English dataset from a single professional speaker. Evaluation was conducted across two sets:
- General Set: 81 sentences for naturalness testing.
- Bad-case Set: 327 "stress-test" sentences containing URLs, long number sequences, and complex command lines designed to trigger failures in traditional models.
Key Results
RobuTrans demonstrated superior robustness and competitive naturalness:
- Robustness: In the 327-sentence stress test, RobuTrans achieved zero failures. In comparison, TransformerTTS failed on 237 cases and Tacotron 2 on 35.
- Naturalness: RobuTrans achieved a Mean Opinion Score (MOS) of 4.36, statistically equivalent to TransformerTTS (4.37) and Tacotron 2 (4.37). It also outperformed FastSpeech (4.31 MOS).
- Generalization: The model successfully synthesized sequences up to 30 seconds long, a task where baseline models typically fail due to positional embedding constraints.
Conclusion and Future Work
RobuTrans proves that explicitly constraining TTS models to follow monotonic alignments significantly improves robustness without sacrificing naturalness. By combining duration prediction with pseudo non-causal attention, the model bridges the gap between the stability of traditional systems and the quality of modern neural architectures.
Future research will focus on reducing the dependency on external alignment tools for training the Duration Predictor and exploring faster, non-autoregressive vocoders to replace the computationally intensive WaveNet.
Summary updated