This paper, presented at Interspeech 2008 by researchers from Cambridge University, addresses a fundamental challenge in speech recognition: balancing high accuracy with robustness. While discriminative training methods like Minimum Phone Error (MPE) significantly improve system performance, they are often fragile when applied to "found data"—such as telephone calls or broadcast news—where perfect transcripts are unavailable for speaker adaptation.
The Problem: Discriminative Fragility
Speaker Adaptive Training (SAT) is essential for handling diverse acoustic conditions. However, high-performance discriminative transforms are notoriously "brittle." In unsupervised adaptation scenarios, where the system must adapt using its own initial (and potentially incorrect) hypothesis, discriminative transforms tend to overfit to these errors. This leads to a performance degradation compared to simpler, more robust Maximum Likelihood (ML) methods.
Methods: Discriminative Mapping Transforms (DMT)
The authors propose a framework called DMT-based Discriminative Speaker Adaptive Training (DSAT). This approach decouples the speaker-specific adaptation from the discriminative refinement through a two-stage process:
- Robust Adaptation: At test time, the system first estimates a standard Maximum Likelihood Linear Regression (MLLR) transform. This transform is robust to transcription errors because it focuses on modeling the speaker's acoustic characteristics rather than distinguishing between similar words.
- Discriminative Mapping: The system then applies a pre-learned Discriminative Mapping Transform (DMT) to the MLLR transform.
The DMT is a speaker-independent mapping learned during the training phase using high-quality reference transcripts. It acts as a global function that "upgrades" a robust ML transform into a discriminative space. Because the DMT is learned on reliable data and is not re-estimated for each test speaker, it avoids the "overfitting trap" common in unsupervised discriminative adaptation.
Experimental Results
The method was evaluated on a large-scale English Conversational Telephone Speech (CTS) task involving nearly 300 hours of training data. The primary evaluation metric was the Word Error Rate (WER) on the eval03 test set.
As shown in the results, the DMT-based approach achieved a WER of 25.3%, representing a 1.1% absolute improvement over standard MLLR-based discriminative SAT (26.4%) and a 1.7% improvement over the simple MLLR baseline (27.0%).
Notably, the standard Discriminative Linear Transform (DLT) failed significantly in the unsupervised setting, yielding a WER of 28.1%—worse than the baseline. This failure occurs despite DLT showing the highest phone correctness on training data, as illustrated below.
Conclusion
The DMT-based DSAT framework successfully bridges the gap between discriminative accuracy and adaptive robustness. By utilizing speaker-independent mapping, the system can leverage complex parameterizations (using up to 1,000 regression classes) to capture fine acoustic nuances without succumbing to the errors inherent in unsupervised hypotheses. While a performance gap still exists between unsupervised and supervised (reference) adaptation, DMT provides a significant step forward for robust, high-accuracy speech recognition in real-world conditions.
Summary updated