Discriminative adaptation based on fast combination of DMAP and dfMLLR

Lukáš Machlica, Zbyněk Zajíc, Luděk MüllerInterspeech 2010

TL;DR

This paper tackles the challenge of improving speech recognition accuracy by combining two discriminative adaptation techniques: DMAP and DfMLLR. They introduce a novel "one-pass" method that significantly reduces computational cost by transforming statistics instead of re-processing data, achieving performance comparable to a slower "two-pass" approach.

What Problem Does This Paper Address?

Speech recognition systems often struggle with variations in speakers, accents, and recording conditions. Adaptation techniques aim to adjust a pre-trained acoustic model to better match the characteristics of new speech data. While standard adaptation methods like Maximum Likelihood Estimation (MLE) are effective, discriminative training approaches, which consider not just correct transcriptions but also potential errors, offer further improvements.

This paper focuses on two discriminative adaptation techniques: Discriminative Maximum A-Posteriori (DMAP) and Discriminative Feature Maximum Likelihood Linear Regression (DfMLLR). DfMLLR is particularly useful when adaptation data is scarce, while DMAP excels with larger amounts of data. Combining them offers a powerful solution, but a naive combination requires processing the adaptation data twice (a "two-pass" approach), which is computationally expensive. The core problem addressed is how to effectively combine these discriminative techniques while minimizing the computational burden.

Methods

The paper investigates the combination of DMAP and DfMLLR.

  • DfMLLR: This technique adapts feature vectors by learning a linear transformation. It's designed to work well even with limited adaptation data.
  • DMAP: This method adapts HMM parameters (specifically GMM means, variances, and weights) using a discriminative criterion (MMI) that aims to increase the probability of correct transcriptions while decreasing the probability of incorrect ones. It generally requires more adaptation data than DfMLLR.

The authors propose a "one-pass" combination strategy. Instead of re-processing the adaptation data to accumulate new statistics for the second adaptation stage (DMAP), they transform the statistics already computed during the DfMLLR pass. This transformation accounts for the feature space changes introduced by DfMLLR, allowing the DMAP adaptation to proceed using these transformed statistics. This avoids the need for a second pass over the adaptation data, significantly reducing computation time. They also compare this to the standard "two-pass" combination where data is processed twice.

Datasets

The experiments were conducted on the SpeechDat-East corpus, specifically the Czech portion. This corpus contains telephone speech.

  • Training: The acoustic HMMs were trained on 700 speakers, with each speaker providing approximately 50 sentences.
  • Testing: 150 speakers were used for testing. For each speaker, 10 sentences (around 50 seconds) were used for adaptation, and the remaining sentences (around 200 seconds) were used for testing the adapted system.
  • Vocabulary: The vocabulary consisted of 7000 words.

Key Insights and Ideas

  • Complementarity of DfMLLR and DMAP: DfMLLR effectively shifts the acoustic space with limited data, making subsequent DMAP adaptation more precise and effective, especially when more data becomes available.
  • The Power of Statistics Transformation: The core innovation is realizing that instead of re-processing data, the statistics derived from the first adaptation pass (DfMLLR) can be transformed to be compatible with the second pass (DMAP). This is the key to the "one-pass" efficiency.
  • Discriminative Advantage: Discriminative techniques (DMAP, DfMLLR) consistently outperform their non-discriminative counterparts (MAP, fMLLR) in terms of accuracy.
  • Efficiency vs. Accuracy Trade-off: The "one-pass" combination achieves performance very close to the "two-pass" method but with significantly reduced computational cost, making it a more practical solution.

Key Results

  • Accuracy Improvement: Combining adaptation techniques, especially the discriminative ones, led to significant improvements in accuracy.
  • DfMLLR + DMAP Superiority: The combination of DfMLLR and DMAP achieved the highest accuracy among all tested methods, reaching 66.60% with the two-pass approach and 66.38% with the one-pass approach.
  • One-Pass Efficiency: The proposed one-pass combination of DfMLLR and DMAP yielded results very similar to the two-pass combination (66.38% vs. 66.60%) but with lower time consumption demands. This suggests the statistical transformation is highly effective.
  • Baseline vs. Adapted: The baseline SI model achieved 55.85% accuracy. Individual adaptation methods improved this, with DfMLLR reaching 65.81%. Combinations further boosted performance.

Limitations and Future Work

  • Limited Adaptation Data for Experiments: While the paper discusses the benefits of DMAP with large amounts of data, the experiments focused on a smaller adaptation data set (50 seconds) to highlight the combination's effectiveness. The full potential of DMAP with extensive data in combination with DfMLLR is not fully explored in the experimental results.
  • Parameter Tuning: The paper mentions setting parameters like $T_{jm}$ and $f$ to specific values (e.g., $T_{jm}=16$, $f=1$). Further research could explore optimal parameter tuning strategies for these combined methods.
  • Other Discriminative Criteria: The paper focuses on MMI-based DMAP. Exploring combinations with other discriminative criteria like MPE (Minimum Phone Error) could be a future direction.
  • Real-time Application: While the one-pass method significantly reduces computation, further optimization for real-time speech recognition applications could be investigated.

Summary updated

BibTeX
@inproceedings{machlica2010discriminative,
  author = {Lukáš Machlica and Zbyněk Zajíc and Luděk Müller},
  title = {{Discriminative adaptation based on fast combination of DMAP and dfMLLR}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure