Log-spectral feature reconstruction based on an occlusion model for noise robust speech recognition

Jose A. González, Antonio M. Peinado, Angel M. Gómez, Ning MaInterspeech 2012

TL;DR

This paper proposes a novel method to improve speech recognition in noisy environments by modeling speech distortion as an "occlusion" problem. Instead of trying to completely remove noise, it estimates which parts of the speech signal are masked by noise and reconstructs them using a minimum mean square error (MMSE) approach, showing improved robustness compared to existing missing-data techniques.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems struggle in real-world scenarios due to various distortions, especially additive noise. Traditional approaches to combat this include feature compensation (cleaning up speech features) and model adaptation (adjusting acoustic models). Feature compensation is attractive because it can be added as a front-end to existing systems. This paper focuses on feature compensation in the log-spectral domain, where noise can mask certain speech features. The challenge is to effectively reconstruct these "occluded" or masked speech features to improve recognition accuracy.

Methods

The core of the proposed method is a novel MMSE estimator for reconstructing noisy speech features in the log-spectral domain. The authors reframe the speech distortion problem as an "occlusion" problem, where some speech features are unaffected by noise, while others are completely masked. This is based on a simplified noise occlusion model: $y_i \approx \max(x_i, n_i)$, where $y_i$ is the observed noisy feature, $x_i$ is the clean speech feature, and $n_i$ is the noise feature.

The MMSE estimate of the clean speech feature vector is computed by leveraging a Gaussian Mixture Model (GMM) for clean speech features and assuming a Gaussian distribution for noise features. A key aspect is the derivation of the posterior probability $p(k|y, \lambda_x, \lambda_v)$ and the partial estimate $E[x_i|y, k, \lambda_x, \lambda_v]$ for each GMM component. The final estimator is a linear combination of two terms: one representing the undistorted speech and the other representing the estimate when speech is completely masked by noise. This approach is contrasted with traditional missing-data techniques, which often rely on binary masks (hard decisions) for feature reliability. The proposed method uses a "soft" approach, exploiting probabilities of feature occlusion.

Datasets

  • Aurora-2: Used for evaluating the proposed technique. This database consists of English connected digits distorted by various types of additive noise at different Signal-to-Noise Ratios (SNRs).
  • Aurora-4: Also used for evaluation. This is a large vocabulary database with similar noise conditions and microphone variations.

Key Insights and Ideas

  • Speech distortion as occlusion: Reframing noise distortion as an occlusion problem simplifies the modeling and allows for a more targeted reconstruction approach.
  • Soft vs. Hard Decisions: The proposed method's use of probabilistic occlusion estimates (soft decisions) is more robust than binary masks (hard decisions) used in traditional missing-data techniques, especially when noise estimation is imperfect.
  • MMSE Estimation with GMMs: Combining MMSE estimation with GMMs for clean speech and Gaussian models for noise provides a principled way to estimate masked features.
  • Linear Combination of Estimates: The final reconstruction is an intuitive linear combination of the observed feature and an estimate under total occlusion, weighted by the probability of these scenarios.

Key Results

  • Aurora-2 Performance: The proposed Spectral Reconstruction based on Occlusion model (SRO) consistently outperformed the Binary-Mask Missing-Data (BMD) and Soft-Mask Missing-Data (SMD) approaches across various SNRs. For example, at 0 dB SNR, SRO achieved 42.82% word accuracy, compared to 22.69% for the baseline and 25.55% for BMD.
  • Aurora-4 Performance: SRO also showed significant improvements over BMD and SMD on the Aurora-4 database, yielding average relative improvements of 10.90% and 1.77%, respectively.
  • Robustness to Noise Estimation Errors: The paper highlights that SRO is more robust to errors in noise estimation compared to BMD and SMD.

Limitations and Future Work

  • Assumption of Negligible Residual Term: The simplification of the noise distortion model by assuming the residual term $r_i$ is negligible might limit performance in certain scenarios.
  • Independence Assumptions: The assumption of independence among features for both speech and noise might not hold in all cases.
  • Future Work: The paper suggests that the proposed technique can be viewed as a robust method for soft mask estimation, implying potential for further exploration in that direction. The authors also mention the possibility of extending the approach to other types of distortions beyond additive noise.

Summary updated

BibTeX
@inproceedings{gonzalez2012log,
  author = {Jose A. González and Antonio M. Peinado and Angel M. Gómez and Ning Ma},
  title = {{Log-spectral feature reconstruction based on an occlusion model for noise robust speech recognition}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure