Empirical mode decomposition for noise-robust automatic speech recognition

Kuo-Hao Wu, Chia-Ping ChenInterspeech 2010 · DOI

TL;DR

This paper introduces a noise-reduction technique for Automatic Speech Recognition (ASR) based on Empirical Mode Decomposition (EMD). By decomposing speech features into adaptive "Intrinsic Mode Functions," the researchers successfully filtered out high-oscillation noise from log-energy sequences, achieving up to a 41.9% relative improvement in recognition accuracy on noisy datasets.

What Problem Does This Paper Address?

The "Achilles' heel" of modern ASR systems is environmental noise. While a system might perform perfectly in a quiet lab, its accuracy plummets in a subway station or a busy street because of the mismatch between the clean data used for training and the noisy data encountered during testing.

Traditional methods like Fourier analysis assume signals are linear and stationary (their statistical properties don't change over time). However, speech is notoriously non-stationary, and noise is often non-linear. The authors identify a specific gap: standard post-processing techniques (like Cepstral Mean Subtraction) often fail to capture the complex, time-varying nature of noise embedded within speech features.

Methods

The authors propose using Empirical Mode Decomposition (EMD) to post-process the log-energy feature of speech.

  1. Adaptive Decomposition: Unlike Fourier transforms that use fixed sine and cosine waves, EMD is data-driven. It decomposes a signal into several Intrinsic Mode Functions (IMFs). Each IMF represents a specific scale of oscillation embedded within the data.
  2. The Sifting Process: To find these IMFs, the algorithm uses an iterative "sifting" process:
  3. Identify local maxima and minima.
  4. Create upper and lower envelopes using cubic splines.
  5. Subtract the mean of these envelopes from the signal.
  6. Repeat until the remaining signal satisfies the definition of an IMF (roughly: symmetric and having a similar number of zero-crossings and extrema).
  7. Noise Filtering: The core hypothesis is that the first few IMFs (the "low-order" ones) contain the highest oscillation frequencies, which typically correspond to noise. By subtracting the first one or two IMFs from the noisy log-energy sequence, the authors "smooth" the feature back toward its clean state.
  8. Dynamic Selection: They also developed a "dynamic" version that calculates the average oscillation frequency of the remainder to decide exactly how many IMFs to subtract for a specific utterance.

Datasets

The researchers evaluated their method using the Aurora 2.0 database, a standard benchmark for noise-robust ASR.

  • Training: Clean-training tasks (mismatched conditions).
  • Testing: Test sets corrupted by various noises (subway, babble, car, exhibition, restaurant, street, airport, and station) at Signal-to-Noise Ratios (SNRs) ranging from -5 dB to 20 dB.

Key Insights and Ideas

  • Noise as High-Frequency Oscillation: The paper treats noise not just as an additive constant, but as a high-frequency "jitter" on the feature sequence. EMD is uniquely suited to isolate this jitter because it adapts to the local fluctuations of the signal.
  • Log-Energy is the Weak Link: A pilot experiment showed that simply replacing noisy log-energy features with clean ones (while keeping other features noisy) boosted accuracy from 60.2% to 83.4%. This proved that cleaning the energy feature alone provides massive "bang for the buck."
  • The "Sweet Spot" for Subtraction: Subtracting too many IMFs removes actual speech information. The authors found that while subtracting two IMFs is great for very noisy signals (0 dB), subtracting only one is better for cleaner signals (20 dB). This led to the development of their dynamic thresholding method.

Key Results

The EMD-based approach was tested alongside standard techniques like Mean Subtraction (MS) and Mean-Variance Normalization (MVN):

  • Baseline Accuracy: 60.1% (Average across 0-20 dB SNR).
  • MVN alone: 69.7% accuracy.
  • MVN + EMD (Fixed 2 IMFs): 76.8% accuracy.
  • MVN + EMD (Dynamic): 78.0% accuracy.
  • Relative Improvement: The combination of MVN and EMD provided a 41.9% relative improvement in word error rate over the raw baseline.

The visual results (Figure 2 in the paper) are striking: the EMD-processed log-energy sequences at 0 dB SNR look remarkably similar to the clean-speech sequences, effectively "undoing" the corruption caused by the noise.

Limitations and Future Work

  • Feature Scope: This study focused almost exclusively on the log-energy feature. The authors acknowledge that EMD could potentially be applied to other features, such as individual MFCC (Mel-frequency cepstral coefficient) dimensions.
  • Computational Complexity: The "sifting" process is iterative and can be computationally expensive compared to simple linear filters. The paper does not provide a detailed analysis of the latency added to the ASR pipeline.
  • Weighting IMFs: Rather than just subtracting whole IMFs, the authors suggest that future work could involve using a "weighted sum" of IMFs to more finely tune the noise removal.

Summary updated

BibTeX
@inproceedings{wu2010empirical,
  author = {Kuo-Hao Wu and Chia-Ping Chen},
  title = {{Empirical mode decomposition for noise-robust automatic speech recognition}},
  booktitle = {Interspeech},
  year = {2010},
  doi = {10.21437/interspeech.2010-585},
}
Figure