Selective MCE training strategy in Mandarin speech recognition

Jian-Lai Zhou, Eric Chang, Chao Huang2001 · DOI

TL;DR

This paper introduces two novel strategies, "selective training" and "divide and conquer," to improve Minimum Classification Error (MCE) training for Mandarin speech recognition. By focusing adjustments on erroneous parts of the acoustic model and partitioning feature spaces, they achieve significant error rate reductions, outperforming standard MCE.

What Problem Does This Paper Address?

Traditional Maximum Likelihood Estimation (MLE) training for speech recognition models, while effective, can be further improved by discriminative training methods like Minimum Classification Error (MCE). MCE aims to directly minimize classification errors. However, applying MCE to large vocabulary continuous speech recognition systems, especially for languages like Mandarin with complex syllable structures, presents challenges. A key issue is that MCE training, when applied globally, can sometimes degrade the performance of already correct acoustic model components ("over-training"), leading to unexpected misrecognitions. Furthermore, MCE training is computationally intensive, making it difficult to apply to massive datasets. This paper tackles these issues by proposing more efficient and targeted MCE training strategies.

Methods

The authors propose two main enhancements to the embedded string model-based MCE approach:

  • Selective MCE Training (Weighted MCE): Instead of adjusting all model parameters, this strategy focuses adjustments only on the parameters associated with "wrong models" or "wrong states" identified during decoding. This is motivated by the observation that errors in speech recognition are often localized. They introduce a weighted loss function where misrecognized frames contribute more to the error calculation. This aims for more effective convergence and reduced over-training.
  • Divide and Conquer Strategy: This approach addresses the over-training issue more directly. It involves splitting the acoustic feature space of a HMM state into multiple subspaces. Each subspace is then trained to better model a specific region of the feature space, allowing for more nuanced and accurate modeling. This is implemented by creating multiple versions of a HMM state, each optimized for a different region.

Datasets

The experiments were conducted using a large Mandarin speech corpus collected by Microsoft Research China.

  • Training Set: 250 speakers, 200 sentences per speaker, resulting in over 60,000 cross-syllable triphones. The phone set includes 185 initial and tonal final phones.
  • Testing Set 1: 30 speakers, 10 sentences per speaker (4097 characters/syllables total).
  • Testing Set 2: 25 speakers, 500 sentences per speaker.

The system was built using the HTK 2.2 toolkit.

Key Insights and Ideas

  • Error Localization is Key: The core insight is that speech recognition errors are not uniformly distributed. Focusing MCE adjustments on the specific parts of the acoustic model that are failing is more efficient and less prone to causing new errors.
  • Over-training is a Real Problem: Discriminative training, while powerful, can indeed make already good models worse by over-specializing them to specific training examples, leading to a degradation in generalization.
  • Feature Space Partitioning Helps: Dividing the complex acoustic feature space into smaller, more manageable regions allows HMM states to specialize and achieve better local modeling, ultimately improving overall accuracy.
  • Combined Approach is Synergistic: The selective training and divide and conquer strategies are not mutually exclusive; combining them yields greater benefits than either approach alone.

Key Results

The paper reports significant error rate reductions:

  • Selective MCE (Weighted MCE): Achieved about a 10% relative error rate reduction on a large training set.
  • Combined Approach (Selective MCE + Divide and Conquer): Achieved more than 14.5% error rate reduction in syllable loop recognition experiments.
  • Consistent improvements were observed across two separate testing sets.

Limitations and Future Work

  • Computational Cost: While the proposed methods aim to improve efficiency, MCE training remains computationally intensive, especially for very large datasets. The paper mentions using distributed computation to mitigate this.
  • Parameter Tuning: The selection of learning step size and training time are noted as difficult problems with no theoretical solutions, requiring careful experimental tuning.
  • Tone Recognition: While the paper focuses on syllable loop recognition, it acknowledges that tone recognition is a difficult topic and suggests future work could concentrate on analyzing base syllable errors to better understand misclassifications.
  • Generalization to Other Tasks: The effectiveness of these strategies is demonstrated on Mandarin syllable loop recognition. Further research could explore their applicability to other speech recognition tasks and languages.

Summary updated

Abstract

The use of discriminative training methods in sp eech recognition is a promising approach. The minimum classification error (MCE) based discriminative methods have been extensively studied and successfully applied to speech recognition [1][2][3], speaker recognition [4], and utterance verification [5][6]. Our goal is to modify the embedded string model based MCE algorithm to train a large number of crosssyllable triphones in a large vocabulary recognition system. In this paper, selective strategy about MCE based discriminative training method, in particular for Mandarin sp eech syllable loop recognition, is introduced. Here, we use syllable loop recognition task to evaluate performance of the acoustic model of an established large vocabulary continuous sp eech recognition system. Since decoding errors only occur in parts of the whole decoded sentence, it is reasonable to adjust only the parameters of the wrong models. As a result, we introduce the weighted MCE formulation, which can provide more effective convergence property and about 10% relative error rate reduction for a large training set. On the other hand, from our experiments, we observed that although the overall performance of recognition system is improved, some originally correct recognition results are misrecognized after discriminative training. To address this issue, we propose a divide and conquer strategy. The acoustic feature sp ace is divided into two or more sub-spaces according to discriminative training procedure. Combining the above two methods, we got more than 14.5% error rate reduction in syllable loop recognition experiments.
BibTeX
@misc{zhou2001selective,
  author = {Jian-Lai Zhou and Eric Chang and Chao Huang},
  title = {{Selective MCE training strategy in Mandarin speech recognition}},
  year = {2001},
  doi = {10.21437/eurospeech.2001-460},
}
Figure