Adaptive speech enhancement for speech separation in diffuse noise

Rong Hu, Yunxin ZhaoInterspeech 2006

TL;DR

This paper tackles the challenge of improving speech recognition accuracy after blind speech separation (BSS) in noisy environments. They propose a novel "divide and conquer" strategy that first adapts the BSS system to handle noise during separation and then applies adaptive speech enhancement to the separated signals, significantly boosting phone recognition accuracy.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems struggle in noisy environments, and even more so when trying to separate multiple overlapping speech sources (Blind Source Separation or BSS). While BSS algorithms aim to isolate individual speech signals, they are often degraded by ambient noise, and they are not inherently designed to suppress diffuse noise effectively. Furthermore, the adaptive nature of BSS algorithms means that the noise characteristics in their output signals change over time, making it difficult to apply standard stationary noise reduction techniques. This paper addresses the dual problem of improving BSS performance in the presence of diffuse noise and then enhancing the separated speech to make it suitable for ASR.

Methods

The authors propose a two-stage approach:

  1. Noise-Compensated Adaptive Decorrelation Filtering (NC-FADF): This is an adaptation of the Adaptive Decorrelation Filtering (ADF) algorithm. The key innovation here is a "divide and conquer" strategy. First, a fast noise compensation (NC) step is introduced to adapt the ADF filters, allowing them to focus on the separation task without being overly biased by the noise. This is achieved by estimating and compensating for noise-induced bias in the filter adaptation process.
  2. Adaptive Speech Enhancement: After the NC-FADF algorithm separates the speech, the resulting signals are still contaminated by noise. To address this, the paper proposes a post-processing step that adaptively enhances the separated speech. This involves tracking the time-varying noise auto-correlations in the ADF output and then applying standard speech enhancement techniques. Two methods are explored for this enhancement:
  3. Spectral Subtraction: A classic and computationally efficient method for noise reduction.
  4. Generalized Subspace Approach (TDC-GSub): A more sophisticated method known for its ability to handle colored noise.

The core idea is to treat the adaptation bias and output noise suppression as separate problems, with the adaptive enhancement step specifically designed to handle the time-varying noise introduced by the adaptive BSS filters.

Datasets

  • TIMIT Speech Data: Used as the clean speech source for generating mixtures.
  • Impulse Response Data: From a room with a reverberation time (T60) of 0.3 seconds, used to create realistic acoustic environments for speech mixtures.
  • Simulated Diffuse Noise: Created to be speech-shaped.
  • Real Diffuse Noise: Recorded in a computer lab environment with active air conditioning, ventilation, and multiple workstations.

The paper does not introduce a new benchmark dataset but uses existing data to simulate challenging noisy and reverberant conditions.

Key Insights and Ideas

  • "Divide and Conquer" for Noisy BSS: Separating the problem of adapting BSS filters in noise from the problem of enhancing the output noise is crucial for effective performance.
  • Time-Varying Noise in Adaptive BSS: The adaptive nature of BSS algorithms inherently introduces time-varying noise characteristics in their outputs, necessitating adaptive enhancement techniques.
  • NC-FADF for Improved Separation: The proposed noise compensation mechanism in the ADF adaptation significantly improves the separation quality, especially at low Signal-to-Noise Ratios (SNRs).
  • Post-Processing Enhancement is Key: While improved separation is important, applying a subsequent adaptive speech enhancement step is vital for achieving high ASR accuracy.
  • TDC-GSub Outperforms Spectral Subtraction: The generalized subspace approach provides superior noise reduction and leads to higher phone recognition accuracy compared to spectral subtraction, especially in challenging noise conditions.

Key Results

The proposed techniques significantly improved phone recognition accuracy on the outputs of the ADF separation system.

  • Combined NC-FADF and TDC-GSub: This combination achieved the highest performance.
  • Low SNR Performance: At low SNRs, the gains in phone accuracy were primarily attributed to the speech enhancement stage.
  • High SNR Performance: At high SNRs, the improvements were mainly due to the enhanced speech separation capabilities of the NC-FADF algorithm.
  • Figures 3 and 4 visually demonstrate substantial gains in phone accuracy for the enhanced separated speech compared to noisy mixtures and separated speech without enhancement, across various SNR levels for both simulated and real diffuse noises.

Limitations and Future Work

  • Limitations: The paper focuses on diffuse noise and does not explicitly address directional interferences, which might require different approaches. While musical noise is mentioned as a potential issue with spectral subtraction, it was not further processed to avoid introducing more residual noise.
  • Future Work: The authors suggest that further research could explore more advanced speech enhancement algorithms beyond spectral subtraction and TDC-GSub. Investigating methods to mitigate musical noise more effectively without compromising other performance aspects would also be beneficial. The paper also implies that the effectiveness of the approach could be tested with other BSS algorithms.

Summary updated

BibTeX
@inproceedings{hu2006adaptive,
  author = {Rong Hu and Yunxin Zhao},
  title = {{Adaptive speech enhancement for speech separation in diffuse noise}},
  booktitle = {Interspeech},
  year = {2006},
}
Figure