Introduction
In the field of speech synthesis, conveying emphasis is essential for natural and expressive communication. While unit-selection synthesis can achieve high quality, it requires extensive databases of emphatic speech. Parametric synthesis using Hidden Markov Models (HMMs) offers greater flexibility but suffers from a "data sparseness" problem; emphasized words occur much less frequently than neutral ones in standard corpora. This imbalance often leads to poorly trained models or unnatural, robotic speech. This paper proposes a hierarchical HMM-based framework designed to synthesize high-quality emphatic English speech even when training data is limited.
Methodology
The proposed system separates the general phonetic structure of speech from the expressive layer of emphasis through a hierarchical approach. The framework consists of two main stages: training and synthesis.
Hierarchical Decision Trees
Instead of a single complex tree, the researchers first construct a general decision tree (DT) using a large neutral corpus. Each leaf node of this general tree is then expanded into an emphasis-related sub-tree. These sub-trees categorize speech into six distinct contexts:
- I-P-E: Inside the primary stressed syllable of an emphasized word.
- B-P-E / A-P-E: Before or after the primary syllable.
- N-B / N-A: Neutral words immediately adjacent to an emphasized word.
- E-P: All other contexts.
HMM Adaptation and Selection
To address the scarcity of emphatic data, the system uses Maximum Likelihood Linear Regression (MLLR) to adapt neutral HMMs. In cases where specific leaf nodes lack data, a cost function based on phonetic broad classes (e.g., nasals, plosives) selects the most appropriate neighboring HMM from the sub-tree to ensure phonetic consistency.
Parameter Compensation
A critical component of the system is the compensation model. It applies statistical factors to predicted pitch ($f_0$) and duration at the phone layer. This acts as a refinement step, adjusting parameters to mimic emphasis even when the underlying HMM might under-predict intensity due to data averaging.
Experimental Evaluation
The study utilized the CMU US ARCTIC clb corpus for neutral speech and a custom-designed corpus of 700 utterances (350 neutral, 350 emphatic) for the emphasis model. The hierarchical system was compared against a traditional adaptation baseline and a "two-pass" decision tree method.
Results and Discussion
The hierarchical approach demonstrated a significant improvement in both emphasis accuracy and naturalness. Both the hierarchical and two-pass systems achieved an impressive 98% accuracy in emphasis identification tests, far exceeding the 70% accuracy of the baseline adaptation system.
Subjective tests revealed that listeners had higher confidence in the emphasis produced by the hierarchical system. Furthermore, preference tests indicated that the hierarchical method was significantly more natural than the two-pass system. While adding expressiveness typically incurs a slight naturalness penalty compared to a purely neutral baseline, the hierarchical model struck a superior balance compared to prior state-of-the-art techniques.
Conclusion
By nesting emphasis-specific sub-trees within a general phonetic structure and employing a statistical compensation model, this research successfully addresses the data sparseness problem in emphatic speech synthesis. The hierarchical logic ensures that phonetic transitions remain natural while providing the necessary acoustic modulation for clear, identifiable emphasis.
Summary updated