A Model-Based Spectral Envelope Wiener Filter for Perceptually Motivated Speech Enhancement

Najib Hadir, Friedrich Faubel, Dietrich KlakowInterspeech 2011 · DOI

Introduction

Speech enhancement—removing background noise from a voice recording—is a classic challenge in signal processing. Traditional methods like spectral subtraction often suffer from two main issues:

  1. Musical Noise: They create artificial, tinkling artifacts that are distracting to the listener.
  2. Perceptual Mismatch: Many algorithms operate in linear frequency domains, whereas human hearing is logarithmic.

While researchers have tried using "log-Mel" features (the same features used in speech recognition) because they mimic human hearing, reconstructing speech directly from these features usually sounds "robotic" or unnatural. This is because log-Mel features represent the broad "envelope" of the speech (the vocal tract shape) but discard the "fine structure" (the pitch and harmonics) that makes a voice sound human.

Methodology: The SFE-WF Approach

The authors propose a hybrid approach called the Model-Based Spectral Envelope Wiener Filter (SFE-WF). The key innovation is determining the Wiener filter's response based on a modern Minimum Mean Square Error (MMSE) feature enhancement approach rather than older, less accurate log-normal approximations.

Block diagram of the proposed SFE-WF system.

The Core Workflow:

  1. Feature Enhancement: A Bayesian approach with a Gaussian Mixture Model (GMM) trained on clean speech is used to estimate what the clean speech should look like in the log-Mel domain, given a noisy input.
  2. Phase-Averaged Interaction: Unlike older models that ignore the phase difference between speech and noise, this method uses a "phase-averaged interaction function" to more accurately model how noise and speech combine.
  3. The "Spectral Envelope" Trick: Instead of trying to synthesize a new signal from the log-Mel features, the estimated clean features are used to calculate the frequency response of a Wiener filter.
  4. Filtering, Not Re-synthesis: This filter is then applied to the original noisy signal. Because the filter only modifies the broad spectral envelope, the "fine structure" of the original signal is preserved, maintaining a more natural voice quality.

Experimental Setup

The researchers evaluated their model using the Multi-Channel Wall Street Journal Audio-Visual (MC-WSJ-AV) corpus and the NOISEX-92 database. They tested against three noise types: "volvo" (stationary car noise), "factory2," and "babble" (non-stationary crowd noise) at 0dB, 5dB, and 10dB Signal-to-Noise Ratios (SNRs).

Key Results and Insights

The proposed SFE-WF algorithm consistently outperformed standard techniques like Spectral Subtraction (SS) and the widely-used Ephraim-Mallah log-MMSE estimator.

PESQ scores across different noise types and SNRs comparing SFE-WF to benchmarks.
  • PESQ Scores: In highly non-stationary "babble" noise at 0dB, the SFE-WF showed a 24% relative improvement over the log-MMSE benchmark.
  • Log Spectral Distance (LSD): The method achieved up to a 45% relative reduction in spectral distance compared to log-MMSE, indicating the enhanced signal was much closer to the original clean speech.
  • Real-time Performance: Despite using a complex 128-component GMM, the system ran at approximately 3x real-time on a 3.00GHz processor, suggesting it is viable for practical applications.

Key Insights:

  • Envelope vs. Fine Structure: Correcting only the spectral envelope (the formants) while leaving the fine structure alone avoids the "synthetic" sound common in model-based reconstruction.
  • Perceptual Domain Advantage: Operating in the log-Mel domain makes the speech distribution more "Gaussian" and easier for models to learn, while also being more relevant to how humans perceive sound.
  • Smoothing is Essential: To prevent the filter from "chattering," the authors used first-order recursive averaging on the power spectral densities.

Limitations and Future Work

At very low SNRs (0dB and below), the speech can still take on a "synthetic" character. Additionally, the current experiments used speaker-dependent GMMs; real-world applications would require more robust, speaker-independent models. The authors also noted that smaller "frame shifts" are necessary to reduce "crackling" sounds, which increases computational load.

Overall, the paper demonstrates that by focusing on the perceptually relevant spectral envelope, we can achieve cleaner speech without sacrificing the natural "human" quality of the voice.

Summary updated

BibTeX
@inproceedings{hadir2011model,
  author = {Najib Hadir and Friedrich Faubel and Dietrich Klakow},
  title = {{A Model-Based Spectral Envelope Wiener Filter for Perceptually Motivated Speech Enhancement}},
  booktitle = {Interspeech},
  year = {2011},
  doi = {10.21437/interspeech.2011-85},
}
Figure