Abstract
Multi-channel speech enhancement extracts speech using multiple microphones that capture spatial cues. Effectively utilizing directional information is key for multi-channel enhancement. Deep learning shows great potential on multi-channel speech enhancement and often takes short-time Fourier Transform (STFT) as inputs directly. To fully leverage the spatial information, we introduce a method using spherical harmonics transform (SHT) coefficients as auxiliary model inputs. These coefficients concisely represent spatial distributions. Specifically, our model has two encoders, one for the STFT and another for the SHT. By fusing both encoders in the decoder to estimate the enhanced STFT, we effectively incorporate spatial context. Evaluations on TIMIT under varying noise and reverberation show our model outperforms established benchmarks. Remarkably, this is achieved with fewer computations and parameters. By leveraging spherical harmonics to incorporate directional cues, our model efficiently improves the performance of the multi-channel speech enhancement.
Introduction
Multi-channel speech enhancement (MCSE) aims to isolate a target speech signal from complex acoustic environments containing noise and reverberation by leveraging multiple microphone inputs. The primary advantage of multi-channel systems lies in their ability to exploit spatial cues—the inter-channel differences in time and intensity that characterize the soundfield's geometry.
While deep learning has significantly advanced MCSE, many existing architectures simply concatenate raw Short-Time Fourier Transform (STFT) features from all channels. This approach often fails to explicitly model the spatial structure of the soundfield and can be computationally burdensome as the number of microphones increases. There is a clear need for a more compact and mathematically grounded representation of spatial information to guide neural networks.
Methodology
The proposed approach introduces Spherical Harmonics Injection to provide the model with a concise representation of the spatial environment. By transforming microphone signals into the spherical harmonic domain, the soundfield is decomposed into a set of basis functions on a sphere. Lower-order harmonics capture the general spatial distribution, while higher-order harmonics provide finer directional detail.
Dual-Encoder Architecture
The model utilizes a dual-encoder framework to process spectral and spatial information independently before fusion:
- STFT Encoder: Extracts spectral-temporal features from the primary audio signals.
- SHT Encoder: Processes the spherical harmonics coefficients to capture directional cues.
The outputs of these specialized encoders are concatenated and passed through a Bidirectional Long Short-Term Memory (Bi-LSTM) layer, which models the temporal dependencies across the fused spatial-spectral features. A decoder then reconstructs the enhanced STFT. To maintain computational efficiency, the model employs "inplace" gated linear units (GLUs), reducing memory overhead without sacrificing performance.
Experimental Setup
The model was evaluated using clean speech from the TIMIT database, corrupted by noise from the DNS Challenge, NOISEX-92, and CHiME3 datasets. Simulations involved a 9-microphone uniform circular array in rooms with reverberation times (RT60) ranging from 0.2s to 1.0s and Signal-to-Noise Ratios (SNR) from -6dB to 6dB.
Results and Discussion
The proposed "Proposed-parallel" model was compared against standard baselines, including GCRN and IGCRN. Performance was measured using the Perceptual Evaluation of Speech Quality (PESQ) and Short-Time Objective Intelligibility (STOI).
Performance Gains
The parallel dual-encoder architecture consistently outperformed the baselines across all noise and reverberation conditions. For instance, at a challenging -5dB SNR, the proposed model achieved a PESQ score of 1.89, compared to 1.74 for the IGCRN baseline.
Similar improvements were observed in intelligibility. In the -5dB SNR scenario, the STOI score increased from 65.97% (IGCRN) to 71.41% (Proposed-parallel). These results indicate that SHT injection effectively helps the model distinguish target speech from interference even in highly reverberant environments.
Computational Efficiency
Despite the performance gains, the proposed model is more efficient than the state-of-the-art IGCRN. It requires only 1.82M parameters (a reduction from 1.89M) and 19.52 GFLOPs (down from 21.01 GFLOPs). This efficiency stems from the compact nature of SHT coefficients and the optimized dual-encoder design.
Conclusion
This work demonstrates that injecting spatial information via the Spherical Harmonics Transform significantly enhances multi-channel speech enhancement performance. By using a dual-encoder architecture to process spectral and spatial cues in parallel, the model achieves superior speech quality and intelligibility while remaining computationally efficient. Future research will explore the model's robustness to irregular microphone geometries and its performance on real-world recordings.
Summary updated