Modulation domain blind source separation for noisy speech mixture

Yi Zhang, Yunxin ZhaoInterspeech 2012

TL;DR

This paper introduces a novel blind speech separation (BSS) method that operates in the modulation domain to enhance robustness against noise. By leveraging modulation domain spectral subtraction and direction-of-arrival (DOA) estimation, the method significantly improves speech separation quality compared to conventional acoustic domain approaches, especially in challenging noisy environments.

What Problem Does This Paper Address?

Blind Source Separation (BSS) aims to disentangle individual speech signals from a mixture recorded by multiple microphones. While effective in clean conditions, existing BSS methods often struggle when significant background noise is present, leading to degraded performance. This paper tackles the challenge of robust BSS in noisy environments, specifically focusing on scenarios with two microphones and two sources, where the number of sources exceeds the number of sensors (an underdetermined BSS problem). The core issue is how to effectively extract clean speech signals when the mixture is corrupted by noise, which can obscure crucial cues like phase information needed for source localization and separation.

Methods

The proposed method consists of two main stages:

  1. Modulation Domain Real and Imaginary Spectral Subtraction (MRISS): This pre-processing step enhances both the magnitude and phase spectra of the noisy speech mixture. Operating in the modulation domain is key here, as it's hypothesized to preserve phase information better than traditional acoustic domain methods, which is crucial for DOA estimation.
  2. Modulation Domain Time-Frequency (T-F) Masking with DOA Estimation:
    • Direction-of-Arrival (DOA) Estimation: The paper utilizes the enhanced phase information to estimate the Direction of Arrival (DOA) of each speech source. This is achieved by analyzing the Inter-channel Phase Difference (IPD) in the modulation domain. They propose using an Asymmetric Laplacian Mixture Density (ALMD) model to capture the often asymmetric and long-tailed distribution of IPD data, which is more robust than standard Gaussian Mixture Models (GMMs) or K-means clustering.
  3. T-F Masking: Based on the estimated DOAs, T-F masks are computed. These masks are then applied in the modulation domain to recover the source signals. The authors argue that operating in the modulation frequency domain reduces distortion in the separated speech compared to acoustic domain masking.

The overall pipeline is illustrated in Figure 1, showing the transformation to the modulation domain, MRISS enhancement, DOA estimation, T-F masking, and finally, transformation back to the time domain.

Datasets

The experiments were conducted using:

  • Anechoic room impulse responses (RIRs): From the RWCP dataset.
  • Speech signals: From the TIMIT dataset (40 sentences, 2 male and 2 female speakers).
  • Noises: Five types of spatially incoherent additive noises were used: white, babble, pink, volvo, and Factory2.
  • Evaluation metrics: Perceptual Evaluation of Speech Quality (PESQ), Segmental Signal-to-Distortion Ratio (SDR), and Signal-to-Interference Ratio (SIR) gain.

The paper does not introduce a new benchmark dataset but rather uses existing ones to evaluate their proposed method against a baseline.

Key Insights and Ideas

  • Modulation Domain Sparsity: The paper posits and demonstrates that speech signals exhibit sparsity not only in the acoustic frequency domain but also in the modulation frequency domain. Crucially, they show that this sparsity can be stronger in the modulation domain, making it a more suitable space for certain signal processing tasks like separation.
  • Phase Preservation for DOA: The MRISS pre-processing step is vital. By enhancing both real and imaginary parts of the modulation spectra, it effectively recovers phase information that is often lost in noisy conditions. This recovered phase is the cornerstone for accurate DOA estimation.
  • ALMD for Robust DOA: The choice of an Asymmetric Laplacian Mixture Density (ALMD) for modeling IPD distributions is a significant methodological contribution. It allows for more accurate source localization when sources are close or when noise introduces asymmetry in the IPD data, outperforming traditional methods.
  • Reduced Distortion via Modulation Domain Processing: Performing T-F masking in the modulation domain is claimed to lead to less distortion in the separated speech compared to acoustic domain masking, likely due to the inherent properties of modulation features.

Key Results

The proposed method consistently outperformed the baseline (conventional acoustic-domain DOA-based separation) across all tested noise conditions and Signal-to-Noise Ratios (SNRs).

  • PESQ: In white noise at 0 dB SNR, the baseline method achieved a PESQ score of 1.323, while the proposed method achieved 1.859. The improvements were particularly significant at lower SNRs.
  • Segmental SDR: Similar trends were observed, with the proposed method showing substantial gains over the baseline, especially as SNR decreased. For example, in white noise at 0 dB SNR, the baseline SDR was -6.541 dB, while the proposed method achieved 1.130 dB.
  • SIR Gain: The proposed method demonstrated consistent and high SIR gains (e.g., ~17.3 dB in white noise at 0 dB SNR), indicating effective suppression of interference. The baseline method's performance degraded significantly at low SNRs.
  • Statistical Significance: The improvements were statistically significant in most cases, confirming the robustness and effectiveness of the proposed approach.

Limitations and Future Work

  • Limitations:
    • The study focused on a two-microphone, two-source scenario (underdetermined BSS). The performance in more complex scenarios with more sources or microphones was not explored due to space constraints.
  • The paper assumes the number of sources is known.
  • Future Work:
    • Investigating source number estimation.
  • Extending the method to underdetermined speech source separation (more sources than sensors).
  • Addressing speech separation in reverberant conditions.

Summary updated

BibTeX
@inproceedings{zhang2012modulation,
  author = {Yi Zhang and Yunxin Zhao},
  title = {{Modulation domain blind source separation for noisy speech mixture}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure