Adaptive blocking beamformer for speech separation

Ngoc Thuy Tran, William Cowley, André PollokInterspeech 2011

TL;DR

This paper introduces an "Adaptive Blocking" (AB) beamformer, a simpler yet effective method for separating speech in noisy and reverberant environments like meeting rooms. AB outperforms the established MVDR-GSC approach by offering improved signal-to-interference-plus-noise ratios (SINRs) and reduced distortion, especially in dynamic acoustic conditions.

What Problem Does This Paper Address?

Extracting a specific speaker's voice from a cacophony of other sounds in a real-world setting, like a meeting room, is a significant challenge. This problem is compounded by factors like echoes (reverberation), multiple speakers talking simultaneously (overlapping speech), and background noise (e.g., air conditioners). Traditional methods, while effective in controlled environments, often struggle when these conditions become dynamic and unpredictable. Existing adaptive beamforming techniques, like the Generalized Sidelobe Canceller (GSC) structure, are powerful but can be complex and sensitive to mismatches in the acoustic environment. The authors aim to develop a simpler, more robust adaptive beamformer that maintains flexibility while improving speech separation performance in these challenging scenarios.

Methods

The core innovation is the "Adaptive Blocking" (AB) beamformer. It's inspired by the GSC structure but simplified. The AB beamformer operates in two paths:

  • Top Path (Wu): This path is designed to maintain a unit gain for the desired speaker's signal, preventing distortion.
  • Bottom Path (WB): This path is the adaptive component. It's designed to "block" the desired signal (to avoid canceling it out) and simultaneously minimize the power of all unwanted signals (interferences and noise).

The key novelty lies in how the blocking weighting vector (WB) is calculated. It's derived as a closed-form solution using a Lagrange multiplier method in the frequency domain, minimizing the output power of interference plus noise subject to the constraint that it doesn't attenuate the desired signal. This calculation involves recursively updating covariance matrices (Rxx) and cross-correlation vectors (Dyx) using a forgetting factor (µ) and a window length (T) to adapt to changing acoustic conditions. The authors highlight that this simpler blocking mechanism contributes to AB's improved performance compared to MVDR-GSC, which relies on more complex blocking matrices.

Datasets

The paper uses both synthetic and real recordings for evaluation.

  • Synthetic Data: Generated using room impulse responses (RIRs) with a reverberation time of 0.3 seconds. These RIRs were dynamically generated to simulate speaker movement and changing acoustic environments.
  • Real Recordings: Derived from the AMI corpus, where single-speaker signals from a real microphone array were summed to simulate overlapping speech scenarios (OVL1 and OVL2).

The paper also mentions that input and output signals for these experiments are available at [15].

Key Insights and Ideas

  • Simplicity Enhances Robustness: The AB beamformer's simplified blocking path, compared to traditional GSC approaches, leads to more stable and effective interference cancellation, especially in dynamic environments.
  • Dynamic Environments are Key: The performance gains of AB are most pronounced when dealing with changing acoustic conditions, such as moving speakers, highlighting its suitability for real-world applications.
  • Training Data Matters: The effectiveness of adaptive beamformers is heavily reliant on the availability and quality of training data. Practical training strategies (like using segments where the desired speaker is inactive) are crucial for achieving good performance, though they may result in lower SINR gains compared to ideal, "genie-aided" training.
  • Frequency-Independent Gains: The improved SINR from AB is observed across a wide range of frequencies, not just at higher frequencies, indicating a more comprehensive signal enhancement.

Key Results

  • Significant SINR Improvement: AB consistently achieved higher Signal-to-Interference-plus-Noise Ratios (SINRs) than MVDR-GSC. Gains ranged from 12dB to 27dB in synthetic cases and around 12dB in more practical scenarios.
  • Reduced Distortion: The AB beamformer also resulted in lower Log Spectral Distance (LSD) values, indicating less distortion in the separated speech signal.
  • Consistent Performance: AB demonstrated more stable performance across various scenarios, including different speaker movement patterns and overlapping speech conditions.
  • Practical Training Impact: When using a practical training scheme (OVL2), the SINR improvement dropped from ~26dB to ~12dB compared to ideal training (OVL1), underscoring the challenge of adaptation with limited data.

Limitations and Future Work

  • Training Data Dependency: The paper acknowledges that AB, like other adaptive beamformers, is challenged by dynamic environments and the availability of training data. Performance degrades when adaptation is limited for extended periods.
  • Future Research Directions: The authors suggest exploring adaptive training strategies that can better handle varying speaker locations and overlapping speech. They also note that the top path of the AB beamformer can accommodate various unit-gain beamformers, including adaptive ones, which could be exploited for further adaptation to speaker movement.

Summary updated

BibTeX
@inproceedings{tran2011adaptive,
  author = {Ngoc Thuy Tran and William Cowley and André Pollok},
  title = {{Adaptive blocking beamformer for speech separation}},
  booktitle = {Interspeech},
  year = {2011},
}
Figure