A comparative study of noise estimation algorithms for VTS-based robust speech recognition

Yong Zhao, Biing-Hwang JuangInterspeech 2010

TL;DR

This paper compares two popular methods for estimating noise in speech signals to improve automatic speech recognition (ASR) performance. The authors reveal a surprising mathematical connection between these methods and demonstrate that one approach, Iterative Root Finding (IRF), generally converges faster and performs better than the other, Twofold Expectation Maximization (TEM), for VTS-based adaptation.

What Problem Does This Paper Address?

Robustness in automatic speech recognition (ASR) is a major challenge, especially in noisy environments. A common strategy to combat noise is to adapt the acoustic models of the ASR system to the specific noise conditions of the input speech. Vector Taylor Series (VTS) is a powerful technique that provides a linear approximation to relate noisy speech features to their clean counterparts, enabling model adaptation. However, the effectiveness of VTS adaptation hinges on accurately estimating the parameters of the noise. This paper focuses on comparing two prominent algorithms for this crucial noise estimation step: Iterative Root Finding (IRF) and Twofold Expectation Maximization (TEM). The authors aim to understand their strengths, weaknesses, and the underlying relationship between them.

Methods

The paper delves into two noise estimation algorithms within the Vector Taylor Series (VTS) framework:

  • Iterative Root Finding (IRF): This method directly differentiates the auxiliary function used in Expectation-Maximization (EM) and approximates the root of the resulting non-linear derivative function through recursive refinements. It involves estimating noise and channel means, and then iteratively refining noise variances using either a fixed-point approximation or Newton's method.
  • Twofold Expectation Maximization (TEM): This approach treats clean speech and noise distributions as hidden variables alongside HMM states and mixture components. EM iterations are performed for both state-frame alignments and noise parameter re-estimation. The paper details how TEM estimates noise means and variances, and also discusses a method for estimating channel means.

A key contribution is the mathematical derivation showing a substantial connection between the IRF and TEM approaches, highlighting similarities in their update structures and suggesting a trade-off between them.

Datasets

  • Artificially corrupted samples: Used for evaluating the algorithms' performance and convergence by fitting a Gaussian Mixture Model (GMM). Clean speech data was artificially corrupted with additive noise.
  • Aurora 2 database: A standard benchmark for evaluating ASR systems in noisy conditions. The experiments used connected digits data with various additive noises and channel distortions across different Signal-to-Noise Ratios (SNRs).

Key Insights and Ideas

  • Surprising Connection: The mathematical derivations reveal a significant underlying connection between the IRF and TEM noise estimation algorithms. This suggests that they are not entirely distinct but rather represent different perspectives or approximations of a similar underlying process.
  • IRF's Efficiency: For VTS-based adaptation, the IRF approach generally converges much faster than TEM. This is a crucial practical advantage, as fewer iterations mean quicker adaptation and potentially real-time applications.
  • Performance Trade-offs: While both methods can achieve similar final accuracy, IRF demonstrates better performance in terms of both computational complexity (fewer iterations) and robustness to initial noise parameter settings.
  • TEM's Broader Applicability: The authors acknowledge that TEM might be more suitable for scenarios beyond VTS-based adaptation, where noisy speech statistics might not have a closed-form relationship with noise statistics.

Key Results

  • GMM Fitting Task: In a GMM fitting task with artificially corrupted data, IRF methods (IRF-1 and IRF-2) converged significantly faster (around 2-3 iterations) than TEM (around 9 iterations). All methods achieved comparable log-likelihood scores.
  • Aurora 2 Experiments:
    • On the Aurora 2 database, IRF-1 achieved a Word Error Rate (WER) of 8.31% when stopping at the minimum WER, compared to 8.32% for TEM.
  • When using a more practical stopping criterion (1% WER reduction), IRF-1 achieved a WER of 8.42% with only 2 re-estimation steps, while TEM required 8 steps for a WER of 9.24%. This highlights IRF's superior efficiency and accuracy in a more constrained setting.
  • The study also indicated that additional decoding passes interleaved with re-estimations did not significantly improve performance.

Limitations and Future Work

  • VTS Specificity: The comparison is primarily focused on VTS-based robust speech recognition. The authors suggest that TEM might be more broadly applicable when the relationship between noisy and clean speech statistics is not a closed function, implying future work could explore TEM in such contexts.
  • Initial Parameter Sensitivity: While IRF is less sensitive than TEM, the paper notes that TEM's convergence is significantly affected by initial noise mean and variance values, especially when they are small. Further investigation into robust initialization strategies for TEM could be beneficial.
  • Model Complexity: The paper uses GMMs for acoustic modeling. Future work could explore how these noise estimation techniques perform with more complex acoustic models.

Summary updated

BibTeX
@inproceedings{zhao2010comparative,
  author = {Yong Zhao and Biing-Hwang Juang},
  title = {{A comparative study of noise estimation algorithms for VTS-based robust speech recognition}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure