Adaptive training using discriminative mapping transforms

Chandra Kant Raut, Kai Yu, Mark J. F. GalesInterspeech 2008 · DOI · OpenReview

This paper, presented at Interspeech 2008 by researchers from Cambridge University, addresses a fundamental challenge in speech recognition: balancing high accuracy with robustness. While discriminative training methods like Minimum Phone Error (MPE) significantly improve system performance, they are often fragile when applied to "found data"—such as telephone calls or broadcast news—where perfect transcripts are unavailable for speaker adaptation.

The Problem: Discriminative Fragility

Speaker Adaptive Training (SAT) is essential for handling diverse acoustic conditions. However, high-performance discriminative transforms are notoriously "brittle." In unsupervised adaptation scenarios, where the system must adapt using its own initial (and potentially incorrect) hypothesis, discriminative transforms tend to overfit to these errors. This leads to a performance degradation compared to simpler, more robust Maximum Likelihood (ML) methods.

Methods: Discriminative Mapping Transforms (DMT)

The authors propose a framework called DMT-based Discriminative Speaker Adaptive Training (DSAT). This approach decouples the speaker-specific adaptation from the discriminative refinement through a two-stage process:

  1. Robust Adaptation: At test time, the system first estimates a standard Maximum Likelihood Linear Regression (MLLR) transform. This transform is robust to transcription errors because it focuses on modeling the speaker's acoustic characteristics rather than distinguishing between similar words.
  2. Discriminative Mapping: The system then applies a pre-learned Discriminative Mapping Transform (DMT) to the MLLR transform.

The DMT is a speaker-independent mapping learned during the training phase using high-quality reference transcripts. It acts as a global function that "upgrades" a robust ML transform into a discriminative space. Because the DMT is learned on reliable data and is not re-estimated for each test speaker, it avoids the "overfitting trap" common in unsupervised discriminative adaptation.

Experimental Results

The method was evaluated on a large-scale English Conversational Telephone Speech (CTS) task involving nearly 300 hours of training data. The primary evaluation metric was the Word Error Rate (WER) on the eval03 test set.

Word Error Rate (WER) results for various training and adaptation schemes on the eval03 test set.

As shown in the results, the DMT-based approach achieved a WER of 25.3%, representing a 1.1% absolute improvement over standard MLLR-based discriminative SAT (26.4%) and a 1.7% improvement over the simple MLLR baseline (27.0%).

Notably, the standard Discriminative Linear Transform (DLT) failed significantly in the unsupervised setting, yielding a WER of 28.1%—worse than the baseline. This failure occurs despite DLT showing the highest phone correctness on training data, as illustrated below.

Phone correctness on training data across iterations, showing DLT's high training performance despite its poor generalization in unsupervised test scenarios.

Conclusion

The DMT-based DSAT framework successfully bridges the gap between discriminative accuracy and adaptive robustness. By utilizing speaker-independent mapping, the system can leverage complex parameterizations (using up to 1,000 regression classes) to capture fine acoustic nuances without succumbing to the errors inherent in unsupervised hypotheses. While a performance gap still exists between unsupervised and supervised (reference) adaptation, DMT provides a significant step forward for robust, high-accuracy speech recognition in real-world conditions.

Summary updated

Abstract

Speaker adaptive training (SAT) is a useful technique for building speech recognition systems on non-homogeneous data. When combining SAT with discriminative training criteria, maximum likelihood (ML) transforms are often used for unsupervised adaptation tasks. This is because discriminatively estimated transforms are highly sensitive to errors in the supervision hypothesis. In this paper, speaker adaptive training based on discriminative mapping transforms (DMTs) is proposed. DMTs are speaker-independent discriminative transforms that are applied to ML-estimated speaker-specific transforms. As DMTs are estimated during training, they are not affected by errors in the supervision hypothesis. The proposed method was evaluated on an English conversational telephone speech task. It was found to significantly outperform the standard discriminative SAT schemes.
BibTeX
@inproceedings{raut2008adaptive,
  author = {Chandra Kant Raut and Kai Yu and Mark J. F. Gales},
  title = {{Adaptive training using discriminative mapping transforms}},
  booktitle = {Interspeech},
  year = {2008},
  doi = {10.21437/interspeech.2008-378},
}
Figure