Tied-state multi-path HMnet model using three-domain successive state splitting

Soo-Young Suk, Hiroaki KojimaInterspeech 2009

TL;DR

This paper introduces a novel "three-domain successive state splitting" method to create more efficient and accurate multi-path Hidden Markov Network (HMnet) acoustic models for speech recognition. By splitting states based on context, time, and environment (like gender), they achieve better performance with fewer parameters compared to existing methods.

What Problem Does This Paper Address?

Automatic speech recognition (ASR) systems heavily rely on Hidden Markov Models (HMMs) to model the acoustic properties of speech. However, traditional HMMs often struggle with the variability inherent in real-world speech, such as different speaking styles, accents, and background noise. While multi-path HMMs have shown promise in capturing this variability by creating parallel model paths for different speech characteristics, they can lead to a significant increase in model size and training data requirements. This paper tackles the challenge of creating effective multi-path models that are both accurate and computationally efficient, avoiding the explosion of parameters that often accompanies them.

Methods

The core innovation is a "three-domain successive state splitting" (SSS) algorithm, extending the existing Phonetic Decision Tree SSS (PDT-SSS). PDT-SSS typically splits states based on phonetic context. The proposed method adds two new domains for splitting:

  • Time Domain: Allows splitting based on temporal characteristics of phonemes (e.g., duration).
  • Environment Domain: Enables splitting based on external factors like speaker gender, age, or acoustic conditions (e.g., background noise).

The algorithm iteratively splits states that exhibit high acoustic variation. It first trains gender-separated context-dependent models, then generates linguistic questions, and selects states for splitting based on Bhattacharyya distances. The process continues until a predefined number of states is reached or further splitting yields diminishing returns. This approach aims to create non-uniform, tied-state HMnet topologies, meaning not all phonemes will have the same number of parallel paths or states.

Datasets

  • Training and Testing: The Japanese Newspaper Article Sentences (JNAS) corpus was used.
  • Speakers: 100 male and 100 female speakers for training, with 5 male and 5 female speakers for testing.

Key Insights and Ideas

  • Environmental Splitting is Crucial: Incorporating an "environment domain" (specifically gender in this study) significantly improves model accuracy by allowing distinct paths for male and female speech.
  • Non-Uniformity is Efficient: Creating multi-path models where only states with high variation are split (resulting in non-uniform topologies) is more efficient than uniformly splitting all states.
  • Multi-path Benefits Small Mixture Counts: Multi-path models show a notable performance improvement with fewer Gaussian mixture components per state, suggesting they can capture variability more effectively without needing overly complex individual states.
  • HMnet's Parameter Efficiency: HMnet models, even with multi-path extensions, can achieve a much smaller total number of states (2000-5000) compared to traditional triphone models (over 22,000 states) while maintaining or improving performance.

Key Results

  • The proposed multi-path HMnet model consistently outperformed single-path models (monophone, triphone, extriphone) and a standard HMnet model in phoneme recognition accuracy.
  • For example, the multi-path HMnet achieved a higher Phoneme Accuracy Rate (PAR) than a single-path HMnet with the same number of states.
  • The performance improvement was particularly noticeable with one to four Gaussian mixtures per state.
  • The paper reports a significant phoneme error rate reduction: 9% for ExHMnet and 16% for HMnet when comparing their multi-path versions to their single-path counterparts with the same number of states.

Limitations and Future Work

  • Limited Environment Domains: While the paper demonstrates the concept with gender, it acknowledges that the environment domain can be extended to include age, region, microphone type, or noise conditions.
  • Future Work: The authors plan to apply their method to generate multi-duration models to better represent multiple speaking styles or speed variations in spontaneous speech.

Summary updated

BibTeX
@inproceedings{suk2009tied,
  author = {Soo-Young Suk and Hiroaki Kojima},
  title = {{Tied-state multi-path HMnet model using three-domain successive state splitting}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure