A back-off discriminative acoustic model for automatic speech recognition

Hung-An Chang, James R. GlassInterspeech 2009 · DOI

Introduction

In Automatic Speech Recognition (ASR), context-dependent modeling is essential for capturing the phonetic variations caused by neighboring sounds. However, as the context window expands (e.g., from diphones to triphones), the number of possible phonetic combinations grows exponentially, leading to a severe data sparsity problem. Traditional ASR systems address this through clustering, where similar contexts are grouped together to share parameters. While effective for parameter estimation, clustering can cause the recognizer to lose fine-grained distinctions between phonetic units.

This paper introduces a "back-off" discriminative acoustic modeling scheme designed to maintain distinguishability between phonetic units even when training data is sparse. By combining specific phonetic models with broader, class-based back-off models and optimizing them via discriminative training, the researchers achieved significant performance gains over standard clustering techniques.

Methodology

The proposed approach draws inspiration from back-off techniques used in language modeling. The system calculates an acoustic score for a specific phonetic unit as a weighted combination of three components:

  1. Specific Context-Dependent Model: The model for a specific phone pair (e.g., 'k' followed by 'oy').
  2. Right-Context Generalized Model: A model where the right context is mapped to a broad phonetic class (e.g., 'k' followed by any {vowel}).
  3. Left-Context Generalized Model: A model where the left context is mapped to a broad phonetic class (e.g., any {stop} followed by 'oy').

To ensure these components work together effectively, the authors utilize Minimum Classification Error (MCE) training. Unlike standard Maximum Likelihood (ML) training, which treats sub-models independently, MCE optimizes parameters to minimize actual recognition errors. This discriminative approach "couples" the specific and broad models, allowing the broad models to provide a robust performance floor while specific models add precision where data is sufficient.

Experimental Setup

The model was evaluated using the MIT Lecture Corpus, a challenging dataset containing approximately 300 hours of real-world academic speech characterized by disfluencies, complex vocabulary, and spontaneous sentence structures. The researchers used a 119-hour subset for training and an 8-hour set for testing, ensuring no speaker overlap between the two sets.

Results and Discussion

The back-off models were compared against state-of-the-art clustering-based models using decision trees. The results demonstrate that the discriminative back-off approach significantly outperforms traditional methods.

Word Error Rate (WER) comparison between clustering-based models (C) and back-off models (B) using both Maximum Likelihood (ML) and Minimum Classification Error (MCE) training.

As shown in the experimental results, the best discriminative back-off model (MCE-B30,60,60) achieved a 30.1% WER on the test set. This represents a substantial improvement over the best discriminative clustering-based baseline (MCE-C60), which recorded a 32.7% WER. Statistical analysis using the McNemar test confirmed that this improvement is highly significant (p < 0.001).

Key insights from the results include:

  • Resilience to Overfitting: While traditional clustering models often overfit as complexity (number of Gaussian mixtures) increases, the back-off models remained robust, maintaining or improving performance with higher complexity.
  • Synergistic Training: The combination of back-off structures and discriminative training is vital; the MCE training forces the different levels of the back-off hierarchy to complement each other.

Conclusion and Future Work

The back-off discriminative acoustic model provides a powerful alternative to traditional clustering for handling data sparsity in ASR. By maintaining the distinguishability of phonetic units, it achieves a lower error rate on complex, real-world speech data. Future research will explore the use of multiple sets of broad phonetic classes (e.g., place vs. manner of articulation) and methods to optimize the computational cost associated with evaluating multiple sub-models during decoding.

Summary updated

BibTeX
@inproceedings{chang2009back,
  author = {Hung-An Chang and James R. Glass},
  title = {{A back-off discriminative acoustic model for automatic speech recognition}},
  booktitle = {Interspeech},
  year = {2009},
  doi = {10.21437/interspeech.2009-83},
}
Figure