TL;DR
This paper investigates how to improve the naturalness of synthesized speech by moving beyond a simple "accented" vs. "non-accented" distinction for prosody. They found that while improving binary accent prediction didn't help, introducing a third "contrastive" accent type significantly improved speech quality, suggesting that a hierarchical view of prosody is crucial for realistic TTS.
What Problem Does This Paper Address?
Generating natural-sounding prosody in Text-to-Speech (TTS) systems remains a significant challenge. A key aspect of prosody is prosodic prominence, which refers to the emphasis placed on certain syllables or words within an utterance. Current TTS systems often rely on a simplified model of prominence, typically distinguishing only between "accented" and "non-accented" words. This simplification is thought to "over-flatten" the complex, hierarchical nature of prosodic prominence as described by theories like Autosegmental-Metrical theory. This paper aims to address this limitation by exploring the impact of a more nuanced prosodic prominence model, specifically by introducing a "contrastive" accent type, on the perceived quality of synthesized speech.
Methods
The authors developed two main components for their TTS system:
- Pitch Accent Prediction: They trained and tested several predictors for identifying standard pitch accents. They experimented with a combination of "old" (information content, POS, word length) and "new" features, including a dynamic version of information content (Cached Information Content - CIC) and features derived from syntactic dependencies. They compared different machine learning models, with a Bagging of CART approach yielding the best results.
- Contrastive Accent Prediction: This predictor was trained to identify words that convey "contrastive" focus, meaning they contrast with other words or phrases in the discourse. This predictor uses a Support Vector Machine (SVM) and relies on semantic, syntactic, and morphological features. The authors acknowledge the difficulty of purely text-based contrast identification and suggest a two-step approach involving semantic contrast identification first.
The generated prosodic prominence patterns were then integrated into an HMM-based TTS system. The experiments focused on evaluating the impact of these improved prominence models on speech quality.
Datasets
- Training and Testing: The primary dataset used was the f2b subset of the Boston University Radio News corpus for pitch accent prediction. For contrastive accent prediction, they used excerpts of spontaneous dialogues where contrasting word pairs were marked.
- TTS System Training: The HMM-based TTS systems were trained on 2025 utterances from the Roger voice of the Blizzard 2008 speech data set. For Experiment 2, they augmented this with 1683 utterances from the "carrier sentences" corpus, which contains contrastive words recorded in specific templates.
Key Insights and Ideas
- Binary Accent Prediction is Insufficient: Improving the accuracy of simply distinguishing between accented and non-accented words did not lead to a noticeable improvement in synthesized speech quality. This suggests that the binary distinction is too simplistic.
- Hierarchical Prosody Matters: The results strongly support theories that view prosodic prominence as a hierarchical structure. Flattening this structure into a simple accent/non-accent distinction limits the naturalness of synthesized speech.
- Contrastive Accents Improve Quality: Introducing a third accent type, a "contrastive" accent, significantly improved the perceived quality and naturalness of the synthesized speech. This accent type is intended to mark words that carry contrastive focus.
- Phonetic Correlates of Contrast: While the paper notes that contrastive accents are often claimed to have specific phonetic peculiarities, they also acknowledge that their phonetic appearance might be context-dependent. However, their TTS system treated contrastive accents as having a distinct, strong realization.
Key Results
- Pitch Accent Predictor: The Bagging of CART predictor achieved 85.2% accuracy in a 10-fold cross-validation, with new features providing a 0.9% relative accuracy increase.
- Contrastive Accent Predictor: Achieved 74.4% precision and 22.4% recall in contrast identification in a leave-one-out evaluation.
- Experiment 1 (Pitch Accent Improvement): Listening tests with 30 native English speakers showed no significant difference in overall speech quality between a system with a basic pitch accent predictor and one with an improved predictor and more prosodic features. There was even a slight, non-significant preference for the system with the less sophisticated predictor.
- Experiment 2 (Contrastive Accent Introduction): A preference test showed a significant preference for synthesized speech that used contrastive accents (EmphC) over speech that used standard pitch accents for contrastive words (StdC). In a contrastive word detection test, listeners were able to identify the most prominent word significantly better than chance when contrastive accents were used.
Limitations and Future Work
- Limited Training Data for Contrast: The authors note that the contrastive accent predictor's performance is limited by the lack of corpora that can be used to train a semantic contrast tagger or a focus tagger. They suggest active-learning and semi-supervised learning approaches for training on smaller corpora.
- Single Training Voice: In Experiment 1, only one training voice was used, and it differed from the voice used for the pitch accent predictor. The authors suggest that more definitive conclusions might be drawn if the training speech dataset for the TTS system had manual pitch accent annotation.
- Handling Relative Prominence: The current HMM-based system struggles to handle the relative nature of phonetic realization, which is a characteristic of prosodic prominence. Future work could explore models that can better capture these context-dependent phonetic realizations.
- Further Exploration of Contrast: The success of contrastive accents suggests further research into modeling different degrees and types of emphasis and contrast.
Summary updated