A multi-space distribution (MSD) approach to speech recognition of tonal languages

Huanliang Wang, Yao Qian, Frank K. Soong, Jian-Lai Zhou, Jiqing HanInterspeech 2006 · OpenReview

Abstract

Tonal languages like Mandarin utilize pitch (F0) to distinguish word meanings. However, F0 is only present during voiced speech segments, creating a discontinuity problem for standard Hidden Markov Models (HMMs). This paper introduces a Multi-Space Distribution (MSD) approach that mathematically handles this intermittent nature by treating unvoiced regions as zero-dimensional spaces. Combined with stream-dependent state tying to separate spectral and pitch contexts, the proposed MSD-SD system achieves a significant reduction in tonal syllable error rates (TSER) over traditional interpolation-based methods.

Introduction

In tonal languages, the pitch contour (F0) is essential for distinguishing meaning. However, pitch is only physically present during "voiced" segments of speech, such as vowels. During "unvoiced" segments, like many consonants, pitch effectively disappears. This creates a "discontinuity" problem for standard Hidden Markov Models (HMMs), which typically require continuous feature vectors. Traditionally, researchers have used heuristics like interpolating pitch values for unvoiced regions, which are often inaccurate and introduce bias. This work seeks a mathematically rigorous way to model pitch that respects its intermittent nature.

Methods

The researchers propose two primary innovations to address the pitch modeling challenge:

Multi-Space Distribution (MSD)

Originally developed for speech synthesis, MSD allows the HMM to model observation spaces with variable dimensions. In this framework, the "unvoiced" region is treated as a zero-dimensional space, while the "voiced" region is a one-dimensional space. This allows the model to directly handle the original F0 data without artificial interpolation.

Figure 2: Illustration of the MSD-HMM framework mapping F0 contours to zero-dimensional (unvoiced) and one-dimensional (voiced) subspaces.

Stream-Dependent State Tying

Spectral features (representing the vocal tract) and pitch features (representing the vocal cords) are physically independent. The authors separate these into two distinct data streams and build independent decision trees for each. This allows the pitch model to cluster based on tonal context (neighboring tones) while the spectral model clusters based on phonetic context. The resulting model is integrated into a standard one-pass Viterbi decoder for continuous speech recognition.

Experimental Evaluation

The experiments were conducted using a speaker-independent, gender-dependent Mandarin read speech database consisting of 80 hours of training data from 250 male speakers. Two phone sets were tested: Ph97 (consonant/sonorant decomposition) and Ph187 (Initial/tonal Final decomposition).

The proposed MSD-SD (Stream-Dependent) system showed dramatic improvements in Tonal Syllable Error Rate (TSER) compared to both a baseline without pitch and conventional interpolation methods.

Figure 5: Comparison of Tonal Syllable Error Rates (TSER) across different model configurations and phone sets (Ph97 and Ph187).

As shown in Figure 5, the MSD-SD system reduced the error rate to 36.1% for Ph97 and 35.1% for Ph187. Compared to standard interpolation approaches, the MSD-SD method provided an absolute TSER reduction of 5.5% to 6.1%. When using the Minimum Description Length (MDL) criterion to automatically control model complexity, TSER was further reduced to approximately 33%.

Conclusion

By separating spectral and pitch streams, the model acknowledges the independent operation of the vocal tract and vocal cords. MSD allows the model to statistically handle missing data in unvoiced regions rather than relying on noisy interpolation. While this study focused on male speakers, the results demonstrate that physics-informed modeling and mathematically sound handling of discontinuous features can significantly enhance tonal language recognition. Future work may explore dynamic stream weighting to further optimize performance across various acoustic contexts.

Summary updated

BibTeX
@inproceedings{wang2006multi,
  author = {Huanliang Wang and Yao Qian and Frank K. Soong and Jian-Lai Zhou and Jiqing Han},
  title = {{A multi-space distribution (MSD) approach to speech recognition of tonal languages}},
  booktitle = {Interspeech},
  year = {2006},
  url = {https://openreview.net/forum?id=tYhdXoyEHF},
}
Figure