Adaptive stereo-based stochastic mapping

Shay Maymon, Pierre L. Dognin, Xiaodong Cui, Vaibhava GoelInterspeech 2013 · OpenReview

Introduction

Automatic Speech Recognition (ASR) systems frequently encounter performance degradation in real-world environments characterized by high noise levels, such as factories or moving vehicles. While Stereo-based Stochastic Mapping (SSM) has emerged as a robust solution by mapping noisy audio features to clean versions using Gaussian Mixture Models (GMMs) trained on stereo data, its static nature often fails when test-time noise conditions differ from training data. This paper introduces Adaptive Stereo-based Stochastic Mapping (A-SSM), which addresses this brittleness by updating the noise-compensation model dynamically during inference.

Methodology

The A-SSM framework treats GMM parameters as variables to be refined during the testing phase using an Expectation-Maximization (EM) procedure. This allows the system to adapt to specific noise conditions that were not fully captured during the initial training phase.

Mathematical Framework

The system models the joint distribution of clean and noisy speech features as a GMM. To estimate clean speech from noisy input, the researchers utilize a Minimum Mean Square Error (MMSE) criterion.

Test-Time Adaptation

The core innovation lies in refining the pre-trained GMM using the current test audio through two primary approaches:

  • Maximum Likelihood (ML): Updates parameters based solely on current noisy observations.
  • Maximum A Posteriori (MAP): Uses the original training data as a "prior" to update parameters based on new evidence. This approach is more robust, preventing the model from over-fitting to short snippets of audio.

A-SSM is designed to be complementary to standard adaptation techniques like feature-space Maximum Likelihood Linear Regression (fMLLR), providing additive gains in accuracy.

Experimental Setup

The acoustic model was trained on 280 hours of clean English speech. The SSM component was trained using 60 hours of stereo data artificially corrupted with 10 types of noise from the NOISEX-92 dataset (e.g., babble, factory, white noise) at SNRs ranging from 10dB to 25dB. Evaluation was conducted on "Set C," a challenging real-world dataset featuring humvee-tank noise environments with SNRs between 5dB and 8dB.

Results and Discussion

The performance was evaluated using Word Error Rate (WER). A-SSM demonstrated significant improvements over both the baseline and standard SSM, particularly when the noise environment was mismatched from the training set.

Word Error Rate (WER) comparison across different techniques.

As shown in the results, A-SSM achieved a 29.17% WER, representing a ~6% relative improvement over standard SSM (31.08%). When combined with fMLLR, the system reached its peak performance of 27.37% WER, confirming that A-SSM cleans noisy features in a way that linear transforms cannot fully capture on their own.

Key Insights

  • Unlabeled Adaptation: The system effectively utilizes the statistical structure of noisy input to refine internal mappings without needing ground-truth clean audio.
  • MAP Robustness: MAP estimation proved essential for stability; pure ML estimation can be erratic with short test utterances.
  • Speaker Sensitivity: The model is reset at speaker turns, suggesting the adaptation captures both environment-specific noise and speaker-specific characteristics.

Conclusion and Future Work

A-SSM significantly enhances ASR robustness in mismatched noise conditions through dynamic adaptation. While effective, the computational overhead of running EM iterations at test time presents a challenge for real-time applications. Future work will focus on optimizing these updates for lower latency and integrating automatic speaker diarization to handle speaker-specific resets autonomously.

Summary updated

BibTeX
@inproceedings{maymon2013adaptive,
  author = {Shay Maymon and Pierre L. Dognin and Xiaodong Cui and Vaibhava Goel},
  title = {{Adaptive stereo-based stochastic mapping}},
  booktitle = {Interspeech},
  year = {2013},
  url = {https://openreview.net/forum?id=x3y-4FM-G8x},
}
Figure