Multi-Channel Speech Enhancement using a Minimum Variance Distortionless Response Beamformer based on Graph Convolutional Network

Nguyen Huu Binh, Hai Duong, Tien-Dat Bui, Ngoc Chau Hoang, Nguyễn Quốc CườngInternational Journal of Advanced Computer Science and Applications 2022 · DOI

Abstract

The Minimum Variance Distortionless Response (MVDR) beamforming algorithm is frequently utilized to extract speech and noise from noisy signals captured from multiple microphones. A frequency-time mask should be employed to compute the Power Spectral Density (PSD) matrices of the noise and the speech signal of interest to obtain the optimal weights for the beamformer. Deep Neural Networks (DNNs) are widely used for estimating time-frequency masks. This paper adopts a novel method using Graph Convolutional Networks (GCNs) to learn spatial correlations among the different channels. GCNs are integrated into the embedding space of a U-Net architecture to estimate a Complex Ideal Ratio Mask (cIRM). We use the cIRM in an MVDR beamformer to further improve the enhancement system. We simulate room acoustics data to experiment extensively with our approach using different types of the microphone array. Results indicate the superiority of our approach when compared to current state-of-the-art methods. The metrics obtained by the proposed method are significantly improved, except the Scale-Invariant Source-to-Distortion Ratio (SI-SDR) score. The Perceptual Evaluation of Speech Quality (PESQ) score shows a noticeable improvement over the baseline models (i.e., 2.207 vs. 2.104 and 2.076).

Introduction

Speech enhancement in multi-microphone environments is a critical task for applications ranging from teleconferencing to hearing aids. Traditional spatial filtering techniques, such as the Minimum Variance Distortionless Response (MVDR) beamformer, aim to suppress noise while maintaining a distortionless response in the direction of the target speaker. However, the performance of MVDR is heavily dependent on the accurate estimation of speech and noise statistics, specifically their Power Spectral Density (PSD) matrices.

While Deep Neural Networks (DNNs) have become the standard for estimating these statistics via time-frequency masks, common architectures like CNNs or LSTMs often fail to explicitly capture the complex spatial correlations inherent in multi-channel audio. This paper introduces MVDR-GCN, a pipeline that leverages Graph Convolutional Networks (GCNs) within a U-Net framework to better model these spatial relationships and drive a neural MVDR beamformer.

Proposed Method: MVDR-GCN

The core of the proposed system is a mask estimator that combines the hierarchical feature extraction of a U-Net with the relational modeling of a GCN.

Mask Estimation with GCN-Bottleneck

The architecture, illustrated in Figure 1, processes multi-channel Short-Time Fourier Transform (STFT) features through a U-Net encoder. The innovation lies in the bottleneck: instead of a standard fully connected layer, the authors insert a GCN. Unlike previous approaches that tied GCN nodes to physical microphones, this method sets the number of nodes equal to the number of feature maps from the final encoder layer. This allows the graph to represent and propagate information across a high-level latent spatial-feature space.

The MVDR-GCN architecture, featuring a U-Net with a GCN bottleneck and an MVDR beamformer stage.

Complex Masking and Beamforming

The network is trained to output a Complex Ideal Ratio Mask (cIRM), which accounts for both amplitude and phase. These masks are then used to calculate the PSD matrices for speech and noise. These matrices are plugged into the MVDR formula to compute optimal beamforming weights on-the-fly. The final enhanced signal is reconstructed using an inverse STFT (iSTFT).

Experimental Setup

The model was evaluated using a large-scale dataset derived from the ConferencingSpeech 2021 Challenge, comprising approximately 550 hours of clean speech and 120 hours of noise. To test the system's robustness across different hardware configurations, over 10,000 Room Impulse Responses (RIRs) were simulated for three microphone array (MA) geometries: uniform linear arrays, circular arrays, and non-uniform linear arrays.

Results and Discussion

The MVDR-GCN system was compared against two strong baselines: an LSTM-based model and a standard GNN-based model.

Performance Across Array Geometries

As shown in Figure 2, the proposed MVDR-GCN consistently outperformed the baselines in perceptual quality (PESQ) and intelligibility (STOI/E-STOI) across all array types. Notably, the performance gains were most pronounced for circular and non-uniform arrays. This suggests that the GCN is particularly effective at "triangulating" sources when provided with the diverse spatial information inherent in non-linear or non-uniform layouts.

Comparison of performance metrics (PESQ, STOI, E-STOI, SI-SDR) across different microphone array geometries.

The SI-SDR Trade-off

One interesting finding is that while MVDR-GCN achieved the highest PESQ (2.207 vs. 2.104 for LSTM), it lagged behind the LSTM baseline in Scale-Invariant Source-to-Distortion Ratio (SI-SDR). This indicates a trade-off: while the MVDR-GCN produces speech that sounds cleaner and more natural to human listeners, the LSTM may be more effective at preserving the exact mathematical scale of the original signal.

Conclusion

The integration of GCNs into the latent space of a U-Net provides a powerful mechanism for learning spatial correlations in multi-channel audio. By using these learned features to drive a traditional MVDR beamformer, the proposed system achieves state-of-the-art perceptual quality. Future work will focus on closing the SI-SDR gap and validating the model's performance on real-world recordings.

Summary updated

Abstract

The Minimum Variance Distortionless Response (MVDR) beamforming algorithm is frequently utilized to extract speech and noise from noisy signals captured from multiple microphones. A frequency-time mask should be employed to compute the Power Spectral Density (PSD) matrices of the noise and the speech signal of interest to obtain the optimal weights for the beamformer. Deep Neural Networks (DNNs) are widely used for estimating time-frequency masks. This paper adopts a novel method using Graph Convolutional Networks (GCNs) to learn spatial correlations among the different channels. GCNs are integrated into the embedding space of a U-Net architecture to estimate a Complex Ideal Ratio Mask (cIRM). We use the cIRM in an MVDR beamformer to further improve the enhancement system. We simulate room acoustics data to experiment extensively with our approach using different types of the microphone array. Results indicate the superiority of our approach when compared to current state-of-the-art methods. The metrics obtained by the proposed method are significantly improved, except the Scale-Invariant Source-to-Distortion Ratio (SI-SDR) score. The Perceptual Evaluation of Speech Quality (PESQ) score shows a noticeable improvement over the baseline models (i.e., 2.207 vs. 2.104 and 2.076). Our implementation of the proposed method can be found in the following link: https://github.com/3i-hust-asr/gnn-mvdr-final.
BibTeX
@article{binh2022multi,
  author = {Nguyen Huu Binh and Hai Duong and Tien-Dat Bui and Ngoc Chau Hoang and Nguyễn Quốc Cường},
  title = {{Multi-Channel Speech Enhancement using a Minimum Variance Distortionless Response Beamformer based on Graph Convolutional Network}},
  journal = {International Journal of Advanced Computer Science and Applications},
  year = {2022},
  doi = {10.14569/ijacsa.2022.0131088},
}
Figure