Sub-band based log-energy and its dynamic range stretching for robust in-car speech recognition

Weifeng Li, Hervé BourlardInterspeech 2012

TL;DR

This paper tackles the problem of degraded speech recognition in noisy car environments by proposing a novel way to extract log-energy features. Instead of using the full audio spectrum, they focus on specific sub-bands with the most dynamic speech variations and then stretch this dynamic range, leading to significantly improved recognition accuracy.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems often rely on features like log-energy and its derivatives to understand speech. However, in noisy environments, especially in-car settings, background noise significantly reduces the dynamic range of these spectral features. This "flattening" effect makes it difficult for the ASR system to distinguish speech from noise, leading to a mismatch between clean training data and noisy testing data, and ultimately, poor recognition performance. The paper specifically addresses how this noise distortion impacts log-energy features and proposes a method to make them more robust and discriminative in such challenging conditions.

Methods

The authors propose a two-pronged approach to improve log-energy features:

  1. Sub-band Log-Energy Estimation: Instead of calculating log-energy from the entire audio spectrum, they first analyze the Mel-filterbank (MFB) outputs. They identify the sub-bands (specific frequency ranges within the MFB) that exhibit the largest dynamic variations over time. The rationale is that these sub-bands are more likely to contain salient speech information and be less contaminated by noise. The log-energy is then computed using only these selected sub-bands.
  2. Dynamic Range Stretching (DRS): After obtaining the sub-band log-energy, a non-linear spectral contrast stretching technique is applied. This process further enhances the dynamic variations of the speech signal, effectively stretching the range of the log-energy values. This aims to make the speech signal more prominent and easier for the ASR system to detect, especially in low Signal-to-Noise Ratio (SNR) conditions.

Datasets

  • CENSREC-2 in-car speech database: This is the primary dataset used for evaluation. It's a continuous digit recognition task collected in real car driving environments under various conditions (e.g., idling, low speed, high speed) and using different microphones (hands-free and close-talking).

Key Insights and Ideas

  • Noise distorts log-energy dynamics: The paper provides a theoretical analysis showing how background noise directly impacts the trajectories of conventional log-energy and its derivatives, making them less representative of the actual speech signal.
  • Focusing on dynamic sub-bands is key: By selecting sub-bands with the largest dynamic range, the proposed method effectively filters out less informative or heavily noise-corrupted spectral components.
  • Dynamic range stretching amplifies speech variations: The DRS technique is crucial for further boosting the discriminative power of the log-energy features, making speech dynamics more apparent even in noisy conditions.
  • Conventional log-energy can be detrimental: The results strongly suggest that in mismatched conditions (clean training, noisy testing), the standard log-energy features can actually hurt ASR performance and are better omitted if not improved.

Key Results

  • Significant relative improvement: The proposed sub-band log-energy with dynamic range stretching (proposedE+DRS) achieved an average relative improvement of 27.2% compared to the baseline front-ends (original log-energy).
  • Competitive performance: The proposed method achieved comparable average recognition performance to established normalization techniques (like MVN and HEQ) that normalize all MFCCs, demonstrating the effectiveness of improving just the log-energy feature.
  • Further gains with spectral stretching: When combined with a previously proposed non-linear spectral stretching of MFB outputs, the proposed log-energy and DRS achieved an even higher average relative improvement of 53.6%.
  • Performance degradation without log-energy: In mismatched conditions, removing the original log-energy (NoE) sometimes led to performance improvements, highlighting its detrimental effect when not properly handled.

Limitations and Future Work

  • Limitations: The paper focuses specifically on log-energy features and their dynamic range. While effective, it's one component of the overall ASR system. The performance might still be limited by other feature extraction or modeling aspects. The theoretical analysis relies on certain assumptions, such as statistical independence of noise and speech, and relatively stable noise energy.
  • Future Work: The authors suggest that further improvements can be achieved by incorporating their new MFCCs obtained through non-linear spectral contrast stretching, which they have explored in related work. The paper also implies that exploring other sub-band selection criteria or DRS techniques could yield further gains.

Summary updated

BibTeX
@inproceedings{li2012sub,
  author = {Weifeng Li and Hervé Bourlard},
  title = {{Sub-band based log-energy and its dynamic range stretching for robust in-car speech recognition}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure