Normalization of spectro-temporal Gabor filter bank features for improved robust automatic speech recognition systems

Marc René Schädler, Birger KollmeierInterspeech 2012

TL;DR

This paper explores how to make automatic speech recognition (ASR) systems more robust to noisy environments by combining physiologically inspired Gabor filter bank (GBFB) features with normalization techniques. They found that normalizing these features significantly improves accuracy, especially in noisy conditions, outperforming standard MFCC features and even advanced ETSI AFE systems.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems still struggle to match human performance, particularly in acoustically challenging environments like those with background noise, spectral distortion, or reverberation. This performance gap can be as large as 15 dB between human and machine capabilities. While some approaches try to mimic the human auditory system, others focus on technical improvements like mean and variance normalization (MVN) or histogram equalization (HEQ) to clean up speech representations. This paper investigates how to best combine physiologically motivated features (Gabor filter banks) with these normalization techniques to enhance ASR robustness.

Methods

The core of this work involves comparing two types of features:

  • Mel Frequency Cepstral Coefficients (MFCCs): A standard feature extraction method that processes speech spectrally (using DCT) and temporally (using slope filters).
  • Gabor Filter Bank (GBFB) Features: A more recent approach inspired by the auditory system. These features use 2D-Gabor filters to simultaneously capture spectro-temporal patterns. Unlike MFCCs, GBFB features are not separable into distinct spectral and temporal processing steps.

The paper then applies two normalization techniques:

  • Mean and Variance Normalization (MVN): This standard technique aims to remove the mean and normalize the variance of each feature dimension. The authors interpret this as normalizing the "modulation depth" of the speech signal.
  • Histogram Equalization (HEQ): This method maps feature values to a specific reference distribution, normalizing all moments of the feature distribution.

The researchers systematically apply MVN and HEQ to both MFCC and GBFB features, exploring different stages of MFCC processing (spectral, temporal, or spectro-temporal) to understand the impact of normalization.

Datasets

The primary dataset used for evaluation is the Aurora 2 framework. This framework is designed for evaluating ASR robustness under noisy conditions. It includes:

  • English connected digits recognition task.
  • Contamination with eight different everyday background noises.
  • A range of Signal-to-Noise Ratios (SNRs) from 20 dB down to -5 dB.
  • Two training conditions: "clean" (only clean utterances) and "multi" (utterances with and without added noise).

The paper also references standard ETSI MFCC and ETSI Advanced Front-End (AFE) features as baselines.

Key Insights and Ideas

  • Normalization is Key: Both MFCC and GBFB features benefit significantly from normalization (MVN/HEQ), especially in noisy conditions. This suggests that even physiologically inspired features can be further improved by statistical cleaning.
  • GBFB Outperforms MFCC in Noise: Normalized GBFB features show superior performance compared to normalized MFCCs, particularly in multi-condition training scenarios and across a wide range of SNRs.
  • Spectro-Temporal Integration Matters: For MFCCs, normalizing the spectro-temporal patterns after both spectral and temporal processing yields the best results. This implies that the combined spectro-temporal information is most valuable when preserved before normalization.
  • MVN as Modulation Depth Normalization: The authors propose that MVN effectively normalizes the "modulation depth" of speech signals, which is crucial for handling variations caused by noise and reverberation.
  • GBFB Sensitivity: While GBFB features are robust, they might be more sensitive to matched SNR conditions, suggesting potential for context-dependent models in future work.

Key Results

  • Significant Robustness Gains: Normalized GBFB features improved robustness by 2.5-5 dB SNR compared to unnormalized versions on the digit recognition task.
  • Outperforming Baselines: Normalized GBFB features outperformed ETSI AFE features in noisy testing conditions with multi-style training.
  • MFCC Improvements: MVN improved MFCC robustness by 5-7 dB in clean training and 2-3 dB in multi-condition training.
  • GBFB vs. MFCC: With multi-condition training, normalized GBFBs improved ETSI MFCC features by 48%, outperforming the ETSI AFE.
  • HEQ vs. MVN: HEQ provided similar improvements to MVN, within a ±1 dB range.

Limitations and Future Work

  • SNR Sensitivity of GBFB: The paper notes that GBFB features, even when normalized, might be sensitive to mismatches in SNR between training and testing. This suggests that context-dependent models that adapt to specific SNRs could further enhance performance.
  • Evaluation of Context-Dependent Models: The authors suggest that evaluating GBFB features with MVN in context-dependent models would be a valuable next step.
  • Further Investigation of Matched SNR Conditions: The paper calls for further investigation into how normalized GBFB features perform in matched signal-to-noise ratio conditions.

Summary updated

BibTeX
@inproceedings{schadler2012normalization,
  author = {Marc René Schädler and Birger Kollmeier},
  title = {{Normalization of spectro-temporal Gabor filter bank features for improved robust automatic speech recognition systems}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure