Model-based approaches to adaptive training in reverberant environments

Yongqiang Wang, Mark J. F. GalesInterspeech 2012

TL;DR

This paper introduces a novel adaptive training method called Reverberant Adaptive Training (RAT) to improve speech recognition in noisy and reverberant environments. RAT builds upon existing techniques to create a "canonical" acoustic model that is robust to these distortions, leading to significantly better performance than previous methods.

What Problem Does This Paper Address?

Speech recognition systems often struggle in real-world environments due to the presence of background noise and reverberation (the echoes that bounce off surfaces in a room). While adaptive training techniques exist to help models adjust to new conditions, they haven't been as effective in handling the complex distortions introduced by reverberation. Existing methods either assume clean training data or try to compensate for noise and reverberation separately, which can limit their effectiveness. This paper aims to bridge this gap by developing a unified adaptive training approach that can effectively factor out both additive noise and reverberation.

Methods

The core of this work is the Reverberant Adaptive Training (RAT) algorithm. RAT extends the Reverberant VTS-Joint (RVTSJ) approach, which was previously used for model compensation during testing. In RAT, RVTSJ is used not only for testing but also for training. The key idea is to factor out unwanted additive and reverberant noise variations to create a canonical model that is neutral to these specific environmental conditions.

The process involves:

  • Model Compensation: Using Vector Taylor Series (VTS) to approximate the mismatch function caused by additive and reverberant noise. This allows for the compensation of acoustic model parameters.
  • Noise Estimation: Employing an Expectation-Maximization (EM) algorithm to estimate the parameters of the additive and reverberant noise models.
  • Canonical Model Estimation: Iteratively updating the canonical acoustic model parameters and the noise models. This is done by maximizing an auxiliary function that considers both the model and noise parameters. A key aspect here is the use of a VTS expansion to approximate the canonical model parameters, making the update tractable.
  • Initialization: A crucial step for the success of adaptive training is good initialization. The paper proposes initializing the canonical model parameters using existing VTS-based adaptive training (VAT) results and setting the noise parameters to represent a smooth trajectory.

Datasets

  • AURORA4 task (reverberant simulated version): This is the primary dataset used for evaluation. It's derived from the Wall Street Journal (WSJ0) 5k-word dictation task.
  • Training Data: Clean WSJ0 training set was filtered with two Room Impulse Responses (RIRs) ("officel" and "livl") and combined with six types of background noises at various Signal-to-Noise Ratios (SNRs).
  • Testing Data: 330 utterances were filtered with "officel" and "office2" (a held-out environment). For each RIR, two noise conditions were used: "clean" and "restaurant" (a specific noise from AURORA4 at varying SNRs).

Key Insights and Ideas

  • Unified approach for noise and reverberation: The paper successfully demonstrates that a single adaptive training framework (RAT) can effectively handle both additive noise and reverberation simultaneously, which is a significant improvement over methods that treat them separately.
  • Canonical model robustness: The goal of creating a "canonical" model that is neutral to specific noise conditions is achieved, leading to better generalization to unseen environments.
  • Importance of initialization: The success of adaptive training heavily relies on a good starting point for model parameters. The proposed initialization strategy using VAT and smooth noise trajectories proves effective.
  • RAT outperforms MST+adaptation: While multi-style training (MST) followed by adaptation (like CMLLR) shows promise, RAT achieves better overall performance, especially in environments not seen during training.

Key Results

  • Significant WER reduction: In experiments on the reverberant simulated AURORA4 task, the RAT system consistently outperformed other approaches.
  • Compared to VAT with RVTSJ adaptation, RAT yielded absolute gains of 0.5% to 1.3% for the "officel" environment and 2.3% to 3.7% for the "office2" environment.
  • On average, the RAT system achieved the best performance, yielding 1.9% absolute gains over the CMLLR-adapted MST system.
  • Robustness to unseen environments: The RAT system showed particular strength in the "office2" environment, which was not part of the training data, highlighting its generalization capabilities.

Limitations and Future Work

  • Computational cost: While not explicitly detailed as a limitation, adaptive training methods, especially those involving iterative EM steps, can be computationally intensive.
  • Approximation in auxiliary function: The paper mentions using an approximation of the auxiliary function for canonical model updates to manage computational complexity, particularly for large vocabulary tasks. This approximation might introduce some trade-offs.
  • Future work: The paper implicitly suggests that further exploration of the RAT framework, potentially with more advanced noise modeling or different initialization strategies, could lead to even greater improvements. The effectiveness of RAT in real-world, non-simulated reverberant conditions would also be a valuable area for future research.

Summary updated

BibTeX
@inproceedings{wang2012model,
  author = {Yongqiang Wang and Mark J. F. Gales},
  title = {{Model-based approaches to adaptive training in reverberant environments}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure