Simultaneous discriminative training and mixture splitting of HMMs for speech recognition

Muhammad Ali Tahir, Markus Nussbaum-Thom, Ralf Schlüter, Hermann NeyInterspeech 2012

TL;DR

This paper introduces a novel method for improving speech recognition accuracy by combining discriminative training with a technique called "mixture splitting" for acoustic models. By iteratively splitting and refining Gaussian mixture models using a log-linear framework and then applying Minimum Phone Error (MPE) training, they achieve significant reductions in word error rates, outperforming traditional methods.

What Problem Does This Paper Address?

Traditional speech recognition systems often use Gaussian Mixture Models (GMMs) to represent the acoustic characteristics of speech sounds (phonemes). These models are typically trained using Maximum Likelihood (ML) estimation, where individual Gaussian components within each state are iteratively split to better fit the training data. While discriminative training methods like Maximum Mutual Information (MMI) can improve performance, their effectiveness diminishes with a large number of Gaussian components, and they can get stuck in local optima. The challenge is to effectively leverage discriminative training to achieve high accuracy even with complex acoustic models containing millions of parameters, while also ensuring robustness and avoiding local optima.

Methods

The authors propose a combined approach:

  • Log-linear Acoustic Model: They convert the standard GMMs into a log-linear form. This is crucial because, for a single Gaussian density per state, the MMI optimization in this form is a convex problem, guaranteeing a global maximum.
  • Discriminative Training (MMI): They first train the single-density log-linear model using the frame-level MMI criterion. This provides a robust starting point.
  • Discriminative Splitting: Instead of just splitting densities based on ML criteria, they adapt the splitting process to the log-linear parameters. This involves duplicating existing log-linear parameters and adding small offsets, then iteratively refining them. This process is repeated until a desired number of densities (e.g., 64 per triphone state) is reached.
  • Minimum Phone Error (MPE) Training: After achieving the desired model complexity through discriminative splitting, they further refine the model using the MPE criterion. This is known to yield better word error rates than MMI.

The training process is visualized in Figure 1, showing the flow from MFCC extraction through ML training, MMI optimization, density splitting, and finally MPE optimization.

Datasets

  • Training and Evaluation Corpus: European Parliament Plenary Sessions (EPPS) English corpus.
  • Training data: 40.8 hours (older version), >100 hours (newer versions).
  • Evaluation data: 3.5 hours.
  • Characteristics: Clean conditions, British English.

Key Insights and Ideas

  • Convexity is Key: The log-linear formulation for single-density models is critical because it guarantees a global optimum for MMI training, avoiding the local optima problem common in GMM-based discriminative training.
  • Discriminative Splitting Matters: Applying discriminative principles not just to parameter estimation but also to the splitting of mixture components leads to better models than simply splitting based on ML criteria and then discriminatively training.
  • MMI as a Stepping Stone: While frame-level MMI isn't the best criterion for final WER, it serves as an excellent robust initial step for discriminative splitting due to its global optimality property for single densities.
  • MPE for Final Polish: The subsequent MPE training on the discriminatively split log-linear model yields significant improvements, demonstrating the synergy between the two discriminative approaches.
  • SAT Benefits: Speaker Adaptive Training (SAT) with MLLR/CMLLR further boosts performance, showing that these techniques complement the proposed acoustic modeling approach.

Key Results

The paper reports significant word error rate (WER) reductions:

  • Without Speaker Adaptive Training (SAT):
    • ML baseline (64 densities): 17.1% WER
    • Log-MPE (64 densities): 15.3% WER (1.8% absolute reduction)
  • With SAT MLLR and cMLLR:
    • ML baseline (64 densities): 13.6% WER
    • Log-MPE (64 densities): 12.1% WER (1.5% absolute reduction)

They also show that a model trained only by MPE on an ML-split GMM (without discriminative splitting) yields smaller improvements (1.3% without SAT, 1.1% with SAT), highlighting the benefit of their integrated discriminative splitting and log-linear training approach.

Limitations and Future Work

  • Robustness of Log-MPE: The authors note that log-linear lattice-based MPE training could be made more robust.
  • Feature Engineering: Investigating state-of-the-art features like MLP features is suggested.
  • Discriminative MLLR: Exploring discriminative training of MLLR matrices to better adapt them to the overall system is proposed.
  • Handling of Noise/Silence: Combining noise, silence, and hesitation phoneme states into a single state was a workaround; further investigation into better handling of these states is implied.

Summary updated

BibTeX
@inproceedings{tahir2012simultaneous,
  author = {Muhammad Ali Tahir and Markus Nussbaum-Thom and Ralf Schlüter and Hermann Ney},
  title = {{Simultaneous discriminative training and mixture splitting of HMMs for speech recognition}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure