A hierarchical viterbi algorithm for Mandarin hybrid speech synthesis system

Ran Zhang, Zhengqi Wen, Jianhua Tao, Ya Li, Bing Liu, Xiaoyan LouInterspeech 2014 · OpenReview

Introduction

In the field of Text-to-Speech (TTS), hybrid systems aim to marry the stability of Hidden Markov Models (HMMs) with the high-fidelity naturalness of unit-selection methods. While effective, these systems often struggle with local acoustic glitches. In Mandarin Chinese, listeners are particularly sensitive to transitions within Consecutive Voiced Segments (CVS)—regions where multiple voiced phones, such as vowels or voiced consonants, flow together with a continuous pitch.

Traditional unit selection relies on a standard Viterbi algorithm to minimize a global cost function. However, a globally optimal path may still contain local mismatches within CVS regions that the human ear finds jarring. Research indicates that over 75% of perceived unnaturalness in synthetic speech occurs within these specific segments. To address this, we present a Hierarchical Viterbi Algorithm (HVA) that prioritizes local smoothness in critical voiced regions without increasing overall computational complexity.

Methodology: The Hierarchical Approach

The HVA operates in two distinct rounds to ensure that local acoustic continuity is preserved where it is most audible.

CVS Prediction

The system first identifies CVS regions from the input text using a Multi-Space Distribution HMM (MSD-HMM). These regions are defined as sequences of two or more voiced phones with continuous pitch. The prediction model achieves an impressive 94% accuracy, providing a stable foundation for the subsequent search.

Two-Round Search Process

  1. Round 1 (Local Search): Instead of searching the entire utterance, the algorithm first performs a Viterbi search exclusively within each predicted CVS region. This step identifies the sub-path that minimizes local cost, effectively selecting long, continuous segments from the original recordings to maximize smoothness.
  2. Round 2 (Global Search): A second Viterbi search is conducted for the entire utterance. In this stage, the optimal sub-paths identified in Round 1 are treated as fixed anchors or high-priority candidates. This ensures that the global sequence respects the local smoothness established in the first round.
The two-round Hierarchical Viterbi search process.

Because the HVA is a stepwise refinement of the standard Viterbi process, it does not introduce additional computational overhead, making it a highly efficient optimization for real-time synthesis.

Experimental Evaluation

The system was trained on a phonetically balanced corpus of 10,000 Mandarin sentences. Acoustic features were analyzed at a 5ms frame shift, and 7-state left-to-right HMMs were employed for phone modeling.

Subjective Results

To evaluate the efficacy of the HVA, blind A/B preference tests were conducted with native Mandarin speakers. Participants compared speech synthesized using the standard Viterbi algorithm (VA) against the proposed HVA.

Subjective preference scores for Naturalness and Quality.

The results demonstrate a clear preference for the hierarchical approach. HVA was preferred over the traditional method 72.5% of the time for naturalness and 71.75% for overall speech quality. Listeners specifically noted a significant reduction in the local inconsistencies and "glitches" that often plague hybrid systems.

Conclusion and Future Directions

The Hierarchical Viterbi Algorithm successfully improves the naturalness of Mandarin speech synthesis by focusing optimization on the most perceptually sensitive regions. By sacrificing a small degree of mathematical global optimality for local acoustic smoothness, the system delivers a superior listening experience.

Future work will focus on refining cost functions to further reduce remaining artifacts and addressing the "over-smoothing" problem. Short voiced phones (like /l/, /m/, /n/) can lose natural variation during HMM training, occasionally leading to the exclusion of high-quality candidate units. Improving these specific phone models will be key to the next generation of hybrid synthesis.

Summary updated

BibTeX
@inproceedings{zhang2014hierarchical,
  author = {Ran Zhang and Zhengqi Wen and Jianhua Tao and Ya Li and Bing Liu and Xiaoyan Lou},
  title = {{A hierarchical viterbi algorithm for Mandarin hybrid speech synthesis system}},
  booktitle = {Interspeech},
  year = {2014},
  url = {https://openreview.net/forum?id=5zEXUG2yMvp},
}
Figure