A data-driven approach to speech enhancement using Gaussian process

Sukanya Sonowal, Kisoo Kwon, Nam Soo Kim, Jong Won ShinInterspeech 2014 · OpenReview

Abstract

This paper presents a novel data-driven approach to single-channel speech enhancement employing Gaussian process (GP) regression. Our approach is based on applying GP regression to estimate the "residual gain"—the difference between the optimal gain and that obtained from the traditional minimum mean-square error log-spectral amplitude (MMSE-LSA) estimator. The proposed system involves a cascaded structure consisting of two stages. In the first stage, the gain of the MMSE-LSA estimator is calculated. In the second stage, residual gains are estimated through GP and used to further refine the output. Experimental results demonstrate that the proposed approach produces superior speech quality compared to both traditional statistical modules and other data-driven techniques.

Introduction

Speech enhancement aims to improve the quality and intelligibility of audio signals degraded by noise, such as background chatter or environmental sounds. Traditionally, this problem has been addressed using statistical models like the MMSE-LSA estimator. While effective under specific mathematical assumptions, these models often struggle with the complex, non-stationary noise found in real-world environments.

Recent shifts toward data-driven machine learning methods offer more flexibility. However, many existing techniques rely on rigid structures like codebooks or look-up tables that fail to capture the highly non-linear relationships between noise levels and optimal gain. This research bridges that gap by utilizing Gaussian Process (GP) regression, a powerful non-parametric Bayesian tool, to model these complex functions.

System Architecture and Methods

The core innovation of this work is a "cascaded" two-stage architecture that combines classic signal processing with modern regression techniques.

Block diagram of the proposed two-stage speech enhancement system, illustrating the Training and Enhancement phases.

The Two-Stage Process

  1. Preliminary Stage: The system first applies the standard MMSE-LSA algorithm. This stage serves as a sophisticated feature extractor, providing an initial estimate of the speech and calculating signal-to-noise ratio (SNR) features.
  2. GP Regression Stage: Rather than predicting clean speech from scratch, the model focuses on the residual gain. This is the mathematical error left behind by the first stage. By learning to correct the specific errors of a known algorithm, the machine learning task becomes significantly more efficient.

Technical Innovations

To make Gaussian Processes computationally feasible for real-time speech processing, the authors implemented several optimizations:

  • Clustering for Scalability: Since GP complexity scales cubically with data size, the authors used Gaussian Mixture Models (GMMs) to cluster training data into 64 groups. A smaller, faster "local expert" GP was then trained for each cluster.
  • Spectro-Temporal Features: The model analyzes a window of neighboring time-frequency points. To manage this data, Principal Component Analysis (PCA) was used to compress 30 raw features into the 10 most significant components.

Experimental Setup

The system was trained using the TIMIT database (50 speakers) and tested on 10 speakers. Noise conditions were sourced from the Noisex92 database:

  • Matched Condition: Training and testing both utilized White Gaussian Noise.
  • Mismatched Condition: To test generalization, the model was trained on white noise but tested against F-16 cockpit noise and factory noise.

Key Results

The GP-based method consistently outperformed both the standard MMSE-LSA and baseline data-driven methods like Vector Quantization (VQ) across multiple objective metrics.

Performance comparison showing SegSNR Improvement, Log-Likelihood Ratio (LLR), and Cepstral Distance (CD) across different input SNR levels.
  • Objective Quality: In matched white noise at 0 dB SNR, the GP method achieved a PESQ score of 0.95, significantly higher than the 0.83 achieved by MMSE-LSA.
  • Robustness: In mismatched environments (e.g., F-16 noise at 0 dB), the GP method maintained its lead with a score of 2.09 versus 1.99 for the standard method.
  • Visual Clarity: Spectrogram analysis revealed that the GP method effectively suppressed "residual noise"—the digital chirping artifacts often introduced by other filters—resulting in a cleaner audio profile.

Conclusion and Future Work

By focusing on error-driven learning (residual gain) and utilizing the non-linear mapping capabilities of Gaussian Processes, this approach offers a significant step forward in speech enhancement. While the current model treats frequency bins independently, future work will explore "frequency banding" to exploit the natural harmonic correlations in human speech, potentially further reducing computational load and improving audio fidelity.

Summary updated

Abstract

This paper presents a novel data-driven approach to single channel speech enhancement employing Gaussian process (GP). Our approach is based on applying GP regression to estimate the residual gain with the input features being the a priori and a posteriori signal-to-noise ratios (SNRs). The residual gain is defined as the difference between the optimal gain and that obtained from the minimum mean-square error log-spectral amplitude (MMSE-LSA) estimator. Our proposed approach involves a cascaded structure consisting of two stages. At the first stage, the gain of the MMSE-LSA estimator is calculated in conjunction with the SNR features. In the second stage, the residual gains are estimated through GP and they are used to further enhance the output of the MMSE-LSA module. Experimental results show that the proposed approach produced better speech quality than not only the MMSE-LSA enhancement module but also the other data-driven technique.
BibTeX
@inproceedings{sonowal2014data,
  author = {Sukanya Sonowal and Kisoo Kwon and Nam Soo Kim and Jong Won Shin},
  title = {{A data-driven approach to speech enhancement using Gaussian process}},
  booktitle = {Interspeech},
  year = {2014},
  url = {https://openreview.net/forum?id=iCU4GXwydF},
}
Figure