Accounting for the uncertainty of speech estimates in the complex domain for minimum mean square error speech enhancement

Ramón Fernandez Astudillo, Dorothea Kolossa, Reinhold OrglmeisterInterspeech 2009

TL;DR

This paper introduces a novel approach to speech enhancement by modeling the uncertainty of estimated speech signals in the complex Fourier domain. This complex-domain uncertainty representation leads to more accurate estimation error modeling and, when combined with uncertainty propagation techniques, significantly improves the robustness of automatic speech recognition (ASR) systems with minimal computational overhead.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems often struggle with noisy environments. Traditional speech enhancement methods aim to remove noise, but they often operate on point estimates of the clean speech signal, ignoring the inherent uncertainty introduced by the noise and the enhancement process itself. This paper addresses the challenge of effectively incorporating this uncertainty into the ASR pipeline. Specifically, it focuses on how the uncertainty generated by common Minimum Mean Square Error (MMSE) speech enhancement filters (like Wiener and Ephraim-Malah) in the time-frequency domain can be better modeled and utilized. The authors highlight that existing methods often consider uncertainty only in the amplitude domain, which can miss crucial information due to the natural asymmetry of amplitude distributions.

Methods

The core contribution of this paper lies in deriving closed-form solutions for the estimation error of MMSE speech enhancement filters in the complex Fourier domain.

  • Complex Domain Uncertainty: Instead of just estimating the clean speech signal's magnitude, the authors model the uncertainty of the complex Fourier coefficients. This is achieved by deriving the posterior distribution of the clean signal's Fourier coefficients given the noisy signal's coefficients.
  • Wiener Filter: For the Wiener filter, the authors show that the complex Gaussian posterior distribution allows for a direct calculation of the estimation error, which is shown to be equal to the variance of the posterior.
  • Ephraim-Malah Filters (MMSE-STSA and MMSE-LSA): For these filters, which operate on amplitude and log-amplitude respectively, the authors derive approximations for the estimation error in the complex domain. These approximations leverage the fact that the phase estimator is the same as the Wiener filter and that the amplitude posterior can be approximated.
  • Uncertainty Propagation: The paper then describes how this complex-domain uncertainty is propagated through the feature extraction process (e.g., mel-cepstral features). They provide formulas for propagating uncertainty through magnitude and magnitude-squared transformations, leading to probabilistic descriptions of the extracted features (mean and variance).
  • Uncertainty Decoding/Modified Imputation: Finally, they explain how these probabilistic features are used within the ASR system. Two methods are discussed: modified imputation (maximizing observation likelihood) and uncertainty decoding (marginalizing over uncertain features).

Datasets

The paper mentions using the AURORA5 and JEIDA databases for testing. The experiments were conducted using a MATLAB implementation of the AURORA5 task, with noise artificially added to clean speech files using segmental SNR.

Key Insights and Ideas

  • Complex Domain is Crucial: Modeling uncertainty in the complex Fourier domain, rather than just the amplitude domain, provides a more accurate representation of the estimation error. This is because the complex domain allows for a symmetric Gaussian posterior, simplifying error analysis.
  • Closed-Form Solutions for Error: Deriving closed-form solutions for the estimation error, even if approximate for some filters, offers valuable insight into the origin of uncertainty and its impact.
  • Uncertainty Propagation Enhances Robustness: Combining the complex-domain uncertainty estimates with uncertainty propagation through feature extraction and then using these probabilistic features in ASR significantly improves robustness against noise.
  • Minimal Computational Cost: The proposed methods achieve improved robustness with only a slight increase in computational cost, making them practical for real-world applications.

Key Results

The paper presents experimental results in terms of Word Error Rates (WER) on the AURORA5 database under various noise conditions (white noise and instationary street noise) and Signal-to-Noise Ratios (SNRs).

  • Significant WER Reduction: The tables show that using uncertainty propagation with either modified imputation or uncertainty decoding consistently leads to lower WERs compared to using the MMSE estimators without uncertainty handling. For example, in Table 1 (white noise, 0 dB SNR), the MMSE-STSA with uncertainty decoding achieves a WER of 16.4%, compared to 19.2% for the MMSE-STSA without it.
  • Consistent Improvement: The benefits of uncertainty propagation are observed across different MMSE estimators (Wiener, MMSE-STSA, MMSE-LSA) and noise types.

Limitations and Future Work

  • Approximations for Ephraim-Malah: While closed-form solutions are derived for the Wiener filter, approximations are used for the MMSE-STSA and MMSE-LSA estimators. The authors acknowledge that these approximations are based on assumptions about the posterior distribution.
  • Ignoring Noise Variance Uncertainty: The current error measure does not account for the uncertainty in estimating the noise variance ($\lambda_D$) or the speech presence probability.
  • Future Work: The authors suggest a more general framework that incorporates these additional sources of uncertainty. They also anticipate that a more generalized model will further improve recognition results. They are also investigating the propagation of uncertainty for other feature types like Perceptual-Linear-Prediction (PLP) and RASTA features.

Summary updated

BibTeX
@inproceedings{astudillo2009accounting,
  author = {Ramón Fernandez Astudillo and Dorothea Kolossa and Reinhold Orglmeister},
  title = {{Accounting for the uncertainty of speech estimates in the complex domain for minimum mean square error speech enhancement}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure