Maximum a posteriori Speech Enhancement Based on Double Spectrum

Pejman Mowlaee, Daniel Scheran, Johannes Stahl, Sean U. N. Wood, W. Bastiaan KleijnInterspeech 2019 · DOI

Introduction

Traditional speech enhancement systems typically operate in the acoustic frequency domain using the Short-Time Fourier Transform (STFT). While effective for stationary noise, these methods often struggle to distinguish speech from non-stationary noise that overlaps in frequency. The modulation domain—which captures the temporal evolution of spectral envelopes—offers a promising alternative because human speech exhibits distinct temporal patterns, such as syllabic rates. However, standard modulation-domain techniques often suffer from a resolution trade-off between time and frequency. This paper introduces a framework based on the Double Spectrum (DS), a two-stage transform that utilizes pitch-synchronous processing to achieve superior separation of speech and noise.

The Double Spectrum Framework

The proposed system follows a classic analysis-modification-synthesis structure, but replaces the standard STFT with the Double Spectrum transform.

Figure 6: Block diagram of the proposed speech enhancement framework, illustrating the analysis, modification, and synthesis stages.

As shown in the system architecture above, the process begins with the Double Spectrum (DS) Transform:

  1. Stage 1: The input signal $x[n]$ is segmented into blocks based on the fundamental frequency ($f_0$), estimated via PEFAC. It then undergoes a pitch-synchronous Modulated Lapped Transform (MLT).
  2. Stage 2: A second transform (DCT-II) is applied to the MLT coefficients across consecutive frames. This results in a 3D tensor representing frequency, modulation, and time ($q, k, l$).

By making the transform pitch-synchronous, the DS framework concentrates speech energy into specific periodic components, effectively isolating it from aperiodic noise.

Statistical Modeling and MAP Estimation

A core contribution of this work is the statistical characterization of speech and noise within the DS domain. By analyzing the TIMIT database, the authors determined that speech coefficients are best modeled by a two-sided generalized Gamma distribution, while noise coefficients remain approximately Gaussian.

Figure 3: Statistical analysis of the DS domain. Left: Distribution fitting of DS coefficients against various models. Right: Kullback-Leibler Divergence (KLD) comparing the separability of speech and noise in the DS vs. STSM domains.

Using these models, the authors derived a Maximum a Posteriori (MAP) estimator. Unlike standard Wiener filters that assume Gaussianity for both speech and noise, the DS-MAP estimator exploits the "peaky" nature of the Gamma distribution. This allows the system to more aggressively suppress noise while preserving the sparse, high-energy components of speech.

Quantitatively, the DS domain demonstrates significantly higher separability than traditional methods. The Kullback-Leibler Divergence (KLD) analysis shows a value of 3.86 for the DS domain, compared to only 0.52 for the standard Short-Time Spectral Modulation (STSM) domain, proving that speech and noise are much easier to distinguish in this new representation.

Experimental Results

The framework was evaluated using the TIMIT corpus against various noise types (white, pink, factory, and babble) at SNRs from -5 dB to 10 dB.

  • Oracle Performance: In tests where speech and noise were known, DS-based masking significantly outperformed STSM-based masking in both perceived quality (PESQ) and intelligibility (SIIB).
  • Blind Enhancement: In realistic scenarios with unknown noise, the DS-MAP estimator provided a superior trade-off between Noise Attenuation (NA) and Speech-to-Speech Distortion (SSDR) compared to standard acoustic-domain MMSE-STSA and previous DS-Wiener approaches.
  • Qualitative Analysis: Spectrograms revealed that DS-MAP preserved harmonic structures much better than Modulation Spectral Subtraction, which often over-smooths the signal.

Conclusion

The Double Spectrum framework represents a significant step forward in modulation-domain speech enhancement. By combining pitch-synchronous transforms with accurate statistical modeling, the DS-MAP estimator achieves high-quality noise reduction with minimal distortion. Future work will focus on improving sign (phase) estimation of DS coefficients and optimizing the transform's computational complexity for real-time applications.

Summary updated

BibTeX
@inproceedings{mowlaee2019maximum,
  author = {Pejman Mowlaee and Daniel Scheran and Johannes Stahl and Sean U. N. Wood and W. Bastiaan Kleijn},
  title = {{Maximum a posteriori Speech Enhancement Based on Double Spectrum}},
  booktitle = {Interspeech},
  year = {2019},
  doi = {10.21437/interspeech.2019-1197},
}
Figure