TL;DR
This paper explores a more efficient way to train HMM-TTS systems by using automatically generated "matched PLE" pronunciations instead of manually transcribed ones. Surprisingly, this automated approach yields synthesis quality statistically indistinguishable from systems trained with costly manual transcriptions, suggesting a significant reduction in development time and expense.
What Problem Does This Paper Address?
Building high-quality Text-to-Speech (TTS) systems, particularly those based on Hidden Markov Models (HMMs), relies heavily on accurate phonetic transcriptions of voice corpora. However, two major challenges exist:
- Cost and Time: Manually transcribing speech is a labor-intensive and expensive process.
- Training-Synthesis Mismatch: Even manually corrected transcriptions can't perfectly capture the subtle variations in pronunciation that occur in natural, continuous speech. This "training-synthesis mismatch" can degrade the performance of machine learning models like HMMs.
The paper investigates whether an automated approach to generating these phonetic transcriptions, which aims to perfectly match the pronunciations used during synthesis, can achieve comparable quality to the traditional, manual method.
Methods
The core of the paper is a comparison between two approaches for generating phonetic transcriptions used to train HMM-TTS systems:
- Manual Phone Sequences System: This is the traditional approach, using voice corpora that have been manually transcribed and corrected by phonetic experts.
- Matched PLE System: This novel approach uses a TTS front-end to generate canonical pronunciations for words. Then, a set of manually written Post-Lexical Effects (PLE) rules are applied to model continuous speech phenomena (like vowel reduction or flapping). The resulting phone sequences, which precisely match what the TTS system will produce at synthesis time, are used for both voice corpus markup and synthesis.
The authors also conducted a pilot experiment to compare manual time alignments with automatically generated ones, finding no significant difference in quality. This led them to use automatic time alignments for both systems in the main evaluation.
Datasets
- Training Data: A voice corpus of 2,422 sentences spoken by a single female American English speaker. This corpus was manually transcribed using a proprietary but mostly conventional phone set.
- Evaluation Data: 42 utterances were synthesized for subjective evaluation. These utterances were chosen such that about half contained clear continuous speech effects handled by the PLE rules.
No new benchmarks were introduced; the paper focuses on comparing existing methodologies.
Key Insights and Ideas
- The "Matched PLE" Concept: The central idea is that if the pronunciations used for training exactly match the pronunciations generated during synthesis, the HMMs can learn to model the variations more effectively, even if those training pronunciations aren't "perfectly" accurate in an absolute sense.
- Automation vs. Manual Labor: The paper challenges the assumption that manual phonetic transcriptions are essential for high-quality HMM-TTS. It suggests that a well-defined set of automated rules (PLE) can achieve comparable results, drastically reducing costs.
- Listener Preference for "Deliberate" Speech: Interestingly, in cases where the matched PLE system produced slightly different (and sometimes more "explicit," e.g., with glottal stops) pronunciations than the manual system, listeners sometimes preferred the matched PLE output. This might be because the more deliberate enunciation is easier to process than smoother, quicker speech, especially for synthesized output.
Key Results
- Statistical Equivalence: The subjective evaluation showed that listeners preferred the manual system 43% of the time and the matched PLE system 41% of the time. This difference was statistically insignificant (p=0.42).
- No Significant Impact of Modeling Differences: A post-hoc analysis revealed that even when there were clear differences in how continuous speech effects were modeled between the two systems, it did not significantly impact overall listener preference. In fact, the matched PLE system's preference slightly increased in these cases.
- Objective Analysis of PLE Rules: The manually written PLE rules achieved a phone-level accuracy of 90.1% relative to manual phone sequences, compared to 83.0% for unmodified canonical pronunciations.
Limitations and Future Work
- Manual PLE Rules: While the PLE rules are manually written, the paper acknowledges that their quality is crucial. The authors suggest future work could explore automatically learning PLE rules, though they note challenges with data sparsity.
- Single Speaker/Language: The experiments were conducted on a single American English female voice. Future work should explore different voices, languages, and corpus sizes to generalize these findings.
- Fully Automatic Markup: The current matched PLE system still relied on manual pause locations. The authors propose replacing these with automatically detected pauses for a fully automatic phonetic markup system.
- Limited Choices in Forced Alignment: The paper suggests allowing the forced alignment process to make very limited, context-aware choices in phone sequences when it's likely to improve overall system quality.
Summary updated