Model-based approaches for degraded channel modelling in robust ASR

Mark J. F. Gales, Federico FlegoInterspeech 2012

TL;DR

This paper tackles the challenge of robust Automatic Speech Recognition (ASR) in the presence of both communication channel distortions and speaker variability. They propose and evaluate a novel "front-end" CMLLR (FE-CMLLR) approach that creates a consistent feature space for speaker adaptation, leading to improved performance compared to standard CMLLR methods.

What Problem Does This Paper Address?

Speech recognition systems often struggle when the speech signal is degraded by various factors, such as background noise or distortions introduced by communication channels (like phone lines). Additionally, different speakers have unique vocal characteristics. To handle these issues, researchers use techniques to adapt the acoustic models. This paper focuses on "model-based" adaptation, specifically addressing how to effectively combine models for communication channels (link models) with models for speaker adaptation. A key challenge arises when multiple channel distortions are present, as standard methods can create inconsistent "feature spaces" for speaker adaptation, limiting performance gains.

Methods

The paper explores two main categories of model-based adaptation: predictive and adaptive.

  • Predictive Approaches: These methods explicitly model the channel distortion. Vector Taylor Series (VTS) compensation is mentioned as an example, where the distortion is assumed to follow a specific mathematical form. However, the authors found VTS to be less effective for the communication channel distortions in their experiments.
  • Adaptive Approaches: These methods learn transformations to adapt the acoustic models. Maximum Likelihood Linear Regression (MLLR) and Constrained MLLR (CMLLR) are discussed. The core innovation presented is Front-End CMLLR (FE-CMLLR). Unlike standard CMLLR, which can create inconsistent feature spaces across different channel models (regression classes), FE-CMLLR uses a front-end GMM to determine transform weights. This results in a consistent feature space, making it more suitable for subsequent speaker adaptation. The paper also discusses different ways to combine link and speaker adaptation, including normalized-space adaptation, full-covariance adaptation, and implicit co-ordination, highlighting the trade-offs of each.

Datasets

The study uses data from the Robust Automatic Transcription of Speech (RATS) program for Arabic keyword-spotting. This involved retransmitting Levantine Arabic conversational telephone speech data through eight different communication channels (links) with varying distortions.

  • Training Data: 173 hours of data, including original clean speech and retransmitted speech across all eight channels.
  • Test Data: A held-out portion of the retransmitted data, with 2-2.5 hours per channel.
  • Language Model Training: Used clean Levantine Arabic transcriptions (excluding the test data), resulting in 1.6 million words.

Key Insights and Ideas

  • The Importance of a Consistent Feature Space: The most significant insight is that when combining channel and speaker adaptation, the consistency of the feature space created by the channel model is crucial for effective speaker adaptation. Standard CMLLR can create disparate spaces, hindering speaker adaptation gains.
  • FE-CMLLR Solves the Consistency Problem: FE-CMLLR's novel approach of using a front-end GMM to derive transforms leads to a unified and consistent feature space, directly benefiting speaker adaptation.
  • Trade-offs in Adaptation Strategies: The paper implicitly highlights that while more complex adaptation methods (like full-covariance) might seem powerful, they come with higher computational costs and don't necessarily solve the feature space consistency issue as elegantly as FE-CMLLR.
  • Real-World Challenge: The experiments are conducted on a highly challenging task (degraded communication channels with dialectal Arabic), demonstrating the practical relevance of the proposed methods.

Key Results

  • LAT Performance: Link Adaptive Training (LAT) using both CMLLR and FE-CMLLR provided significant gains over a speaker-independent (SI) system, especially on more distorted links. FE-CMLLR and CMLLR LAT systems showed similar average performance.
  • Speaker Adaptation (Test-time): Unsupervised speaker-level MLLR adaptation at test-time yielded small gains, but FE-CMLLR slightly outperformed CMLLR, suggesting its space is more co-ordinated.
  • Speaker Adaptive Training (SAT): Combining FE-CMLLR with Speaker Adaptive Training (LSAT) showed more pronounced benefits. The gain from using LSAT over SAT was significantly larger for FE-CMLLR (2.0% absolute) compared to CMLLR (0.3% absolute) on the low distortion link.
  • Overall Average Gain: LSAT using FE-CMLLR achieved an average performance gain of 1.8% absolute over the baseline SAT system.
  • Combination: Combining multiple LSAT systems further improved performance by 0.5%-1.1%, with an overall gain of 0.8% absolute.
  • WER: Even with these improvements, Word Error Rates (WER) remained high (60%-70%), reflecting the inherent difficulty of the task.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that the supervision for speaker adaptation (using outputs from other systems) had low accuracy, limiting the gains from unsupervised speaker-level adaptation.
  • The current FE-CMLLR uses a single GMM for all channels.
  • The performance gains, while statistically significant, are still modest due to the extreme difficulty of the task.
  • Future Work:
    • Investigating link-specific front-end GMMs for FE-CMLLR.
  • Exploring acoustic de-weighting on GMM likelihoods to create smoother feature spaces.
  • Extending FE-CMLLR to discriminative training, which could be related to region-dependent transforms (RDTs) but potentially link and region dependent.
  • Using ML FE-CMLLR transforms as an initialization for these more complex discriminative transforms.

Summary updated

BibTeX
@inproceedings{gales2012model,
  author = {Mark J. F. Gales and Federico Flego},
  title = {{Model-based approaches for degraded channel modelling in robust ASR}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure