Low-SNR, speaker-dependent speech enhancement using GMMs and MFCCs

Laura Boucheron, Phillip L. De LeonInterspeech 2012

TL;DR

This paper introduces a novel two-stage speech enhancement technique that leverages speaker-specific acoustic models (GMMs) and a new acoustic class mapping matrix (ACMM) to significantly improve speech quality in very low Signal-to-Noise Ratio (SNR) environments, outperforming existing methods. The approach models distinct "acoustic classes" within a speaker's voice to better reconstruct clean speech from noisy signals, achieving notable PESQ score improvements even at -10 dB SNR.

What Problem Does This Paper Address?

Enhancing speech in noisy environments is a long-standing challenge in audio processing. While traditional methods like spectral subtraction and Wiener filtering can improve speech quality, their effectiveness rapidly degrades at low SNRs (below 5 dB), often introducing undesirable artifacts like "musical noise." Existing advanced methods also struggle in these extreme conditions, and some require extensive clean speech data for training. This paper tackles the problem of achieving robust speech enhancement specifically for low-SNR scenarios, focusing on speaker-dependent models to capture individual vocal characteristics more effectively.

Methods

The proposed technique employs a two-stage approach:

  • Training Stage:
    • Acoustic Class Modeling: Gaussian Mixture Models (GMMs) are trained on Mel-Frequency Cepstral Coefficients (MFCCs) extracted from a speaker's clean speech. Each component density within the GMM represents a distinct "acoustic class" of the speaker's voice.
  • Acoustic Class Mapping Matrix (ACMM): A novel matrix, ACMM, is computed. This matrix probabilistically maps noisy acoustic classes (derived from noisy speech MFCCs) to the speaker's clean acoustic classes (derived from clean speech MFCCs). This mapping is crucial for understanding how noise perturbs the original speech characteristics.
  • Enhancement Stage:
    • Speech Analysis: MFCCs are extracted from the noisy speech signal.
  • Acoustic Class Identification: The most likely clean acoustic class for each speech frame is identified using the noisy MFCCs and the pre-computed ACMM.
  • MFCC Estimation ("Phroming"): Two methods, PMAX and PMIX, are proposed to estimate the clean speech MFCCs. PMIX, which uses a weighted mixture of acoustic class estimates, is found to be superior. This step essentially "morphs" the noisy MFCCs towards the characteristics of the identified clean acoustic class.
  • Inverse Transformation: The estimated clean MFCCs are transformed back into a time-domain waveform using a direct cepstral inversion method, incorporating the phase information from the noisy signal.

A key innovation is the separation of MFCCs into formant and pitch subvectors, allowing for independent modeling of vocal tract configuration and excitation, further improving enhancement.

Datasets

  • Training: The paper mentions using "sufficiently long and phonetically diverse time-aligned training signals" to develop the probabilistic models. Specific dataset names are not provided for training.
  • Evaluation: The TIMIT corpus (specifically ten randomly chosen speakers, five male and five female) was used for testing. The NOISEX-92 corpus was used for generating noise signals (babble and white noise).

Key Insights and Ideas

  • Speaker-Specific Acoustic Classes: Modeling speech as a collection of distinct "acoustic classes" for each speaker, rather than a monolithic entity, allows for more nuanced and accurate reconstruction, especially in challenging noise conditions.
  • The Power of the ACMM: The Acoustic Class Mapping Matrix (ACMM) is a critical novel component. It bridges the gap between the distorted acoustic classes in noisy speech and the original, cleaner classes, enabling a more informed estimation of clean speech features.
  • PMIX Superiority: The "Phromed Mixture" (PMIX) method, which averages estimates weighted by the ACMM, consistently outperforms the simpler "Phromed Maximum" (PMAX) method, suggesting that incorporating information from multiple potential acoustic classes leads to better results.
  • Formant/Pitch Separation: Separating MFCCs into formant and pitch components allows for independent modeling and enhancement of these crucial speech characteristics, leading to significant improvements. This acknowledges that different aspects of speech production are affected differently by noise.
  • Low-SNR Robustness: The method demonstrates remarkable robustness at very low SNRs (-10 dB), an area where many existing techniques falter significantly.

Key Results

  • The proposed method achieves significant improvements in Perceptual Evaluation of Speech Quality (PESQ) scores, outperforming standard methods like spectral subtraction, Wiener filtering, log-MMSE, and generalized subspace.
  • For input SNRs as low as -10 dB, the method shows improvements of 0.3-0.6 PESQ points over the unenhanced signal, depending on the noise type.
  • The system operates effectively across a wide input SNR range, from -10 dB to +35 dB, with performance tapering off at the extremes.
  • Qualitative evaluation indicates good noise reduction with minimal artifacts, notably the absence of musical noise often seen in spectral subtraction and Wiener filtering.
  • The PMIX method consistently outperforms the PMAX method.

Limitations and Future Work

  • Underconstrained Inversion: The paper acknowledges that the inversion of mel-cepstral coefficients can introduce degradation in PESQ (around 0.2 points at very high SNR), though this is masked by noise at lower SNRs.
  • Real-time Application: While the paper describes a two-stage process, the implementation details for real-time application (e.g., frame advance for enhancement) are briefly mentioned but not extensively detailed.
  • Future Work: The authors suggest exploring further research into the paper and discussion areas, implying potential for continued development and refinement of the proposed technique. The paper itself doesn't explicitly list future research directions beyond this general statement.

Summary updated

BibTeX
@inproceedings{boucheron2012low,
  author = {Laura Boucheron and Phillip L. De Leon},
  title = {{Low-SNR, speaker-dependent speech enhancement using GMMs and MFCCs}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure