Abstract
This paper presents an evaluation of the contextual factors of HMM-based speech synthesis and coding systems. Two experimental setups are proposed that are based on successive context addition from phonetic to full-context. The aim was to investigate the impact of the individual contextual factors on the speech quality. In that sense, important and unimportant (i.e., not having significant impact on speech quality, also called weak) contextual factors were identified. The results imply that in speech coding, the improvement in quality can be achieved just with reconstruction of syllable contexts. The sentence and utterance contexts are unimportant on the decoder side, and it is not necessary to deal with them. Although in speech coding the wider context was not necessary, in speech synthesis current syllable and utterance contexts are more important over others (previous and next word/phrase contexts).
Introduction
In HMM-based speech synthesis (HTS), "contextual factors"—also known as prosodic context—are used to describe the environment of a speech sound. These include everything from the neighboring phonemes to whether a syllable is stressed, the part-of-speech of the current word, and the total number of words in the sentence.
The problem is that the number of these factors is huge, and their combinations grow exponentially. This makes it difficult to know which factors are actually improving speech quality and which are just adding noise or complexity. For very low bit-rate (VLBR) speech coding, where a system must reconstruct speech from a tiny stream of symbols, knowing the "minimal viable context" is essential for efficiency.
Methods
The authors conducted two main experiments using a "successive context addition" approach. They started with a baseline of simple phonetic context and gradually added layers of linguistic information to see how each influenced quality.
- HTS Synthesis Experiment: They synthesized speech using various levels of context and conducted subjective listening tests. Fourteen listeners rated the "Degradation Mean Opinion Score" (DMOS) by comparing synthetic speech to natural recordings.
- VLBR Speech Coding Experiment: They simulated a speech codec where the encoder transmits phonetic symbols, pitch, and duration. The decoder then tries to reconstruct the speech. They used an objective measure called PESQ (Perceptual Evaluation of Speech Quality) to evaluate how adding context at the decoder side improved the output.
The study specifically looked at factors ranging from the phonetic level (current phoneme position) to the syllabic level (stress, accent), word/phrase level (part-of-speech, length), and finally the utterance level (total syllables in the sentence).
Key Insights and Results
The core finding of the study is the identification of "weak" versus "strong" contextual factors based on their relative contribution to overall speech quality.
- Phonetic Dominance: Phonetic context is the primary driver of quality, accounting for 43% of quality in synthesis and a staggering 80% in coding.
- The Power of the Syllable: In speech coding, the researchers found a "sweet spot" at the syllable level. Once the current and next syllable contexts are reconstructed, adding word or phrase information provides almost no perceptible improvement.
- Synthesis vs. Coding Needs: There is a divergence between synthesis and coding. For full text-to-speech synthesis, the "utterance" context (the very end of the context chain) is vital for naturalness, contributing about 15% to quality. In coding, where the system often has access to the original timing and pitch, that global context becomes redundant.
- Efficiency: The proposed VLBR coding system, using only syllable-level reconstruction, achieved a PESQ score comparable to the industry-standard LPC-10 codec. However, the VLBR system operates at approximately 100 bps, while LPC-10 requires 2400 bps—an order of magnitude difference in efficiency.
Conclusion
Many factors commonly used in HTS, such as the part-of-speech of the previous or next word, were identified as "weak," meaning they have no significant impact on the final speech quality. Identifying and removing these "unimportant" factors could actually improve synthesis by preventing the decision-tree clustering from "over-smoothing" the speech data. While this study focused on English and a single target voice (Roger), it provides a framework for optimizing the complexity of HMM-based speech systems.
Summary updated