Decoding of Uncertain Features Using the Posterior Distribution of the Clean Data for Robust Speech Recognition

Ahmed Hussen Abdelaziz, Dorothea KolossaInterspeech 2012 · DOI

This paper, originally presented at INTERSPEECH 2012, introduces a refined approach to "uncertainty-of-observation" techniques in Automatic Speech Recognition (ASR). By leveraging the posterior distribution of clean speech features, the authors propose Significance Decoding (SD), a method that outperforms existing state-of-the-art techniques in noisy and reverberant environments.

Introduction

Standard ASR systems are typically trained on "clean" speech. When deployed in real-world scenarios—such as a voice assistant in a noisy car or a crowded street—there is a significant mismatch between the clean training data and the distorted "noisy" input.

Traditional methods attempt to either "clean" the audio first (enhancement) or adapt the model (e.g., MLLR). However, these often struggle when noise changes rapidly. A more robust framework is uncertainty-of-observation, which acknowledges that any "cleaned" speech estimate is inherently imperfect. Instead of passing a single "best guess" feature vector to the decoder, this framework passes a probability distribution representing the uncertainty about the original clean speech.

Methods

The paper focuses on the probabilistic family of uncertainty techniques. Prior to this work, two main approaches dominated:

  1. Uncertainty Decoding (UD): This method "smears" the ASR model's internal states by adding the estimated noise uncertainty to them.
  2. Modified Imputation (MI): This method attempts to find the most likely clean feature vector given the noisy observation and the current model state.

The Innovation: Significance Decoding (SD) The authors argue that UD is often too pessimistic (adding excessive uncertainty) while MI is too optimistic (ignoring residual uncertainty). SD calculates the posterior distribution of the clean feature by simultaneously considering the noisy observation, the estimated noise level, and the ASR model's expectations.

Probabilistic dependency graph for the uncertainty-of-observation framework.

This results in a refined distribution for the clean speech characterized by a state-dependent mean (similar to MI) and a reduced uncertainty covariance. Mathematically, SD unifies MI and UD into a single, more principled framework that filters noise uncertainty through the lens of the acoustic model.

Experimental Setup

The researchers evaluated their method using the Aurora-5 database, a standard benchmark for robust speech recognition. The evaluation included:

  • Training: 8,623 clean utterances.
  • Testing: 8,700 utterances (English digits).
  • Conditions: Interior noise (office/living room), car noise, street noise, and distortions from GSM network transmission.
  • SNR Levels: Ranging from 15 dB down to 0 dB, plus a noise-free baseline.

Key Results

The authors compared SD against Maximum Likelihood (ML), UD, and MI. SD consistently provided the highest word accuracy across all tested conditions.

Word accuracy (%) across different Signal-to-Noise Ratios (SNR).

Key findings include:

  • Superiority at Low SNR: At the most challenging level (0 dB SNR), SD achieved 62.05% accuracy, significantly outperforming standard ML (58.73%) and UD (60.11%).
  • Robustness to Reverberation: In the "Living Room" test case, which involves complex acoustic reflections, SD maintained a performance edge over MI, demonstrating its ability to handle non-stationary noise and reverberation better than imputation alone.
  • Performance Trend: The performance gap between SD and competing methods widened as the Signal-to-Noise Ratio (SNR) decreased, proving its particular value in high-noise scenarios.

Conclusion and Limitations

Significance Decoding represents a powerful new member of the probabilistic uncertainty family. By dynamically adjusting the "significance region" of features at every time step, it offers superior robustness to non-stationary noise.

However, the method currently relies on Gaussian assumptions for speech features and estimation errors, which may be less accurate in highly non-linear distortion environments. Additionally, its success is dependent on the accuracy of the uncertainty estimates provided by the speech enhancement front-end. Future work may involve applying this framework to more complex acoustic models beyond basic HMMs.

Summary updated

BibTeX
@inproceedings{abdelaziz2012decoding,
  author = {Ahmed Hussen Abdelaziz and Dorothea Kolossa},
  title = {{Decoding of Uncertain Features Using the Posterior Distribution of the Clean Data for Robust Speech Recognition}},
  booktitle = {Interspeech},
  year = {2012},
  doi = {10.21437/interspeech.2012-505},
}
Figure