Robust feature extraction for speech recognition by enhancing auditory spectrum

Md Jahangir Alam, Patrick Kenny, Douglas O'ShaughnessyInterspeech 2012

TL;DR

This paper introduces a novel feature extraction method for speech recognition that significantly improves robustness against background noise and reverberation. By enhancing the auditory spectrum and normalizing features over short time windows, the proposed method achieves comparable or better performance than existing state-of-the-art techniques on challenging datasets.

What Problem Does This Paper Address?

Speech recognition systems often struggle in real-world environments due to the degradation of speech intelligibility caused by additive noise (like background chatter) and reverberation (echoes in a room). Traditional methods like MFCCs and PLPs perform well in clean, controlled conditions but falter when the testing environment differs from the training environment. This mismatch is a major barrier to deploying reliable speech recognition in everyday scenarios. Researchers have tried to tackle this by enhancing speech before feature extraction, modifying feature computation, or applying post-processing normalization techniques. However, finding a single approach that effectively handles both additive noise and reverberation while remaining computationally efficient is a significant challenge.

Methods

The proposed feature extractor combines several novel and existing techniques:

  • Compressive Gammachirp Filter-Bank (cGCFB): Instead of standard filter banks, this method uses a cGCFB, which is a generalization of the gammatone filter bank. It's designed to better mimic human auditory processing by allowing for more realistic and controllable frequency-domain asymmetry and modeling level dependency observed in the basilar membrane. This is applied to both speech and noise power spectra.
  • Auditory Spectrum Enhancement: A weighting rule is applied to the auditory spectrum based on the subband a posteriori signal-to-noise ratio (SNR). This rule enhances noisy parts of the spectrum while preserving clean speech, aiming to improve robustness without distorting clean speech.
  • Soft Speech Presence Probability (SPP)-based Noise Estimation: A computationally efficient method is used to estimate the noise power spectrum, which avoids bias correction terms and reduces overestimation of noise.
  • Short-Time Cepstral Mean and Scale Normalization (STCMSN): To normalize the extracted features, a short-time version of cepstral mean and scale normalization is employed. Unlike traditional methods that normalize over an entire utterance (assuming constant channel effects), STCMSN normalizes over a sliding window (1.5 seconds). This relaxes the constant channel assumption and reduces processing delay, making it more suitable for real-time applications.
  • Power Function Non-linearity: Instead of the logarithmic non-linearity used in MFCCs, a power function non-linearity is used, which has been found to be more robust to noise.

Datasets

  • AURORA-2 Corpus: Used for evaluating performance in additive noise conditions. It includes ten noise scenarios across three test sets (A, B, C) at various SNRs (clean, 20dB, 15dB, 10dB, 5dB, 0dB).
  • AURORA-5 Corpus (MRDs subset): Used for evaluating performance in reverberant acoustic conditions. This subset comprises real-time recordings from a meeting room with different microphones, simulating hands-free recording in a reverberant environment with minimal additive noise.

Key Insights and Ideas

  • Mimicking Human Auditory Processing is Key: The use of the cGCFB highlights the benefit of incorporating more biologically plausible auditory models into feature extraction for improved robustness.
  • Targeted Spectrum Enhancement: The subband SNR-based weighting rule effectively boosts the signal in noisy regions of the auditory spectrum without corrupting clean speech, demonstrating a smart way to handle noise.
  • Short-Time Normalization is Crucial for Real-Time: STCMSN's ability to normalize features over shorter, sliding windows is a significant practical advantage, enabling more responsive and efficient real-time systems by avoiding the need for full utterance analysis.
  • Unified Approach for Noise and Reverberation: The paper demonstrates that a single feature extraction framework can effectively address both additive noise and reverberation, a challenging task that often requires separate solutions.

Key Results

  • AURORA-2 (Additive Noise): The proposed method achieved comparable results to the PNCC and ETSI-AFE features, outperforming conventional MFCC and PLP. For example, at 0dB SNR, the proposed method achieved 53.97% word accuracy, while MFCC was 33.01% and PLP was 36.32%.
  • AURORA-5 (Reverberation): The proposed method consistently provided better word accuracy across all microphones compared to MFCC, PLP, PNCC, and ETSI-AFE. For instance, on Mic 6, the proposed method achieved 89.16% accuracy, significantly higher than ETSI-AFE (64.32%).
  • Computational Efficiency: The paper notes that the proposed method is computationally less expensive than PNCC and ETSI-AFE.

Limitations and Future Work

  • Limitations: While the paper shows good performance, the authors acknowledge that the ETSI-AFE front-end was not as effective in reverberant environments as in additive noise, which is consistent with other studies. The proposed method's performance in very low SNR conditions (e.g., 0dB) is still limited, though improved over some baselines.
  • Future Work: The authors suggest applying the proposed feature extractor to a large vocabulary continuous speech recognition (LVCSR) task to further validate its effectiveness in more complex scenarios.

Summary updated

BibTeX
@inproceedings{alam2012robust,
  author = {Md Jahangir Alam and Patrick Kenny and Douglas O'Shaughnessy},
  title = {{Robust feature extraction for speech recognition by enhancing auditory spectrum}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure