Syllable HMM based Mandarin TTS and comparison with concatenative TTS

Zhiwei Shuang, Shiyin Kang, Qin Shi, Yong Qin, Lianhong CaiInterspeech 2009 · DOI

Abstract

Mandarin speech synthesis typically utilizes phone-based units, yet the language's tonal and syllable-centric structure suggests that larger units may be more effective. This paper presents a Mandarin Text-to-Speech (TTS) system based on syllable-level Hidden Markov Models (HMMs). By leveraging high-quality manual alignments from legacy concatenative systems and introducing a novel voiced/unvoiced (V/UV) decision refinement technique, the authors developed a compact 5.3MB model. Experimental results demonstrate that this system achieves subjective quality comparable to a massive 1GB concatenative baseline while offering superior prosodic smoothness.

Introduction

Mandarin is a tonal language where the syllable serves as the primary linguistic and prosodic unit. While standard HMM-based speech synthesis (HTS) systems often use phones or demi-syllables, these units can struggle to capture the strong intra-syllabic co-articulation characteristic of Mandarin. Furthermore, significant effort has historically been invested in manually aligning syllable-level corpora for concatenative synthesis systems—resources that are often underutilized in HMM training. This paper addresses these gaps by proposing a syllable-centric HMM framework that incorporates linguistic constraints to improve synthesis naturalness and efficiency.

Methodology

The proposed system architecture is divided into training and synthesis phases, integrating a high-quality front-end with an HMM-based back-end.

Figure 3: Overview of the Syllable HMM-based TTS system architecture, showing the training and synthesis pipelines.

Syllable-Level Modeling

The system employs 10-state left-to-right HMMs to model each of the approximately 1,300 tonal syllables in Mandarin. By using the syllable as the base unit, the model inherently captures the complex transitions between consonants and vowels that phone-based models might miss.

Leveraging Legacy Front-ends

A key innovation is the reuse of front-end analysis and manually verified alignments from a high-quality concatenative system. This provides the HMMs with "cleaner" segmentation data than typical automated phone-level tools, leading to more stable model parameters.

Voiced/Unvoiced (V/UV) Refinement

Standard HMMs often make V/UV decisions for each state independently, which can result in unnatural "flickering" or "muffling" within a syllable. Since Mandarin syllables generally follow a predictable unvoiced-to-voiced transition, the authors introduced a refinement step. They used Gaussian Mixture Models (GMMs) to model the "unvoiced percentage" of a syllable, enforcing a single transition point.

Figure 6: Comparison between the conventional HMM method and the proposed V/UV refinement. The proposed method (bottom) ensures a single, stable transition from unvoiced (u) to voiced (v), resulting in a cleaner waveform and F0 contour.

F0 Range Tuning

To prevent "half-pitch" errors during analysis, the fundamental frequency (F0) range was manually tuned to 60–450Hz for the target female speaker. This ensures that the pitch contours generated during synthesis remain within a natural human range.

Experimental Setup

The study utilized a high-quality female speaker corpus recorded at 16kHz.

  • HMM Training Set: 3,000 sentences, resulting in a model size of 5.3MB.
  • Concatenative Baseline: 5,000 sentences, requiring a 1GB data footprint.

Results and Discussion

A Mean Opinion Score (MOS) test involving 13 judges compared the two systems:

  • Concatenative TTS: 3.66 MOS
  • Syllable HMM TTS: 3.46 MOS

While the concatenative system maintained a slight lead in overall quality due to its use of original speech segments, the HMM system was noted for its smoother prosody. Concatenative systems frequently suffer from "pitch jumps" at segment boundaries, whereas the HMM approach generates continuous pitch contours. However, the HMM system exhibited "spectral over-smoothing," a common artifact where the averaging of data leads to a slightly "muffled" or less detailed vocal texture compared to the sharp realism of concatenative synthesis.

Conclusion

The syllable-based HMM approach demonstrates that a model roughly 200 times smaller than a traditional concatenative system can achieve nearly equivalent subjective quality. By enforcing linguistic constraints like the V/UV transition, the system improves the "crispness" of synthesized Mandarin. Future work will focus on mitigating spectral over-smoothing through advanced vocoders and scaling training to larger datasets to further bridge the quality gap.

Summary updated

BibTeX
@inproceedings{shuang2009syllable,
  author = {Zhiwei Shuang and Shiyin Kang and Qin Shi and Yong Qin and Lianhong Cai},
  title = {{Syllable HMM based Mandarin TTS and comparison with concatenative TTS}},
  booktitle = {Interspeech},
  year = {2009},
  doi = {10.21437/interspeech.2009-145},
}
Figure