Improving discriminative training for robust acoustic models in large vocabulary continuous speech recognition

Janne Pylkkönen, Mikko KurimoInterspeech 2012

TL;DR

This paper introduces a novel, simpler regularization method for discriminative training of acoustic models in speech recognition. By using a global regularization parameter instead of individual ones for each Gaussian, they achieve improved robustness to noisy conditions without sacrificing performance on clean speech. This work highlights the importance of discriminative training for real-world speech recognition robustness.

What Problem Does This Paper Address?

Real-world speech recognition systems often struggle with noisy environments, different microphones, or out-of-domain data. While discriminative training methods like MMI, MPE, and MPFE have shown promise in improving acoustic models compared to traditional maximum likelihood (ML) training, their impact on robustness to these "mismatched" conditions is less understood. Existing discriminative training techniques, particularly the Extended Baum-Welch (EBW) algorithm, rely on complex heuristics for regularization. This paper aims to simplify this regularization process and demonstrate its effectiveness in improving the robustness of acoustic models under noisy conditions.

Methods

The core of this paper lies in a new approach to controlling the Extended Baum-Welch (EBW) algorithm during discriminative training. Instead of using complex, Gaussian-specific heuristics and I-smoothing (a method to improve generalization), the authors propose using a global D parameter. This global parameter applies a uniform regularization to all Gaussians in the acoustic model.

To determine this global D, they introduce a novel method based on measuring the Kullback-Leibler Divergence (KLD) between the original and updated Gaussians after the first discriminative training update. The goal is to find a global D that results in a desired level of model change, as measured by the median KLD. This approach is computationally efficient as it only requires statistics from the initial model.

The paper experiments with three discriminative training criteria: Maximum Mutual Information (MMI), Minimum Phone Error (MPE), and Minimum Phone Frame Error (MPFE).

Datasets

  • Finnish Speecon Corpus: Used for training acoustic models on clean speech and for evaluating performance on clean, "Clean/Ch2" (medium distance microphone, some noise/reverberation), and car noise test sets.
  • English Wall Street Journal (WSJ) Corpus: Used for training acoustic models on clean speech and for evaluating performance on clean test sets and two noisy test sets (S7 and S8).

Key Insights and Ideas

  • Global Regularization is Key for Robustness: The most significant finding is that a simple, global regularization parameter for EBW training leads to improved robustness in noisy conditions, often outperforming traditional, more complex heuristics.
  • Simplicity Can Be Powerful: The proposed global D method, controlled by KLD, is computationally efficient and effective, demonstrating that complex regularization isn't always necessary for good performance.
  • MMI's Surprising Robustness: In the Finnish Speecon corpus, MMI surprisingly outperformed MPE and MPFE in mismatched conditions, suggesting that MMI might be a strong contender for robust recognition tasks. However, this wasn't consistently observed in the WSJ corpus.
  • Development Sets are Crucial: The optimal number of discriminative training iterations can vary significantly, and using a realistic development set is essential for selecting the best model, especially for criteria like MMI where accuracy can oscillate.

Key Results

  • Improved Performance in Noisy Conditions: Across both Finnish Speecon and WSJ corpora, the global D method consistently improved recognition accuracy (lower LER/WER) in mismatched noisy test sets compared to the baseline EBW with I-smoothing.
  • Comparable Performance on Clean Speech: The global D method achieved performance comparable to the baseline on clean speech, indicating no degradation in performance on ideal conditions.
  • Statistical Significance: Improvements in noisy tasks were statistically significant in many cases.
  • Specific Improvements: For example, in the Finnish Speecon evaluation set, MMI with global D achieved 20.4% LER in car noise, compared to 31.2% for ML. In WSJ, MPFE with global D achieved 17.7% WER on the Noisy S7 test set, compared to 22.9% for ML.

Limitations and Future Work

  • Optimal Iterations Vary: The paper notes that the optimal number of discriminative training iterations is task-dependent and not easily predictable, requiring development set tuning.
  • MMI Performance Variation: The inconsistent superiority of MMI across different corpora in noisy conditions warrants further investigation.
  • Future Directions: The authors suggest exploring the interaction of their proposed regularization method with noise-robust front-end features and noise compensation techniques. They also propose investigating its use in multicondition training.

Summary updated

BibTeX
@inproceedings{pylkkonen2012improving,
  author = {Janne Pylkkönen and Mikko Kurimo},
  title = {{Improving discriminative training for robust acoustic models in large vocabulary continuous speech recognition}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure