Introduction
Isolating a target voice in noisy environments—such as a crowded cafe or a moving bus—remains a significant challenge for hearing aids and voice assistants. While multichannel systems leverage spatial cues (timing and volume differences between microphones), traditional beamforming often struggles in reverberant spaces. Modern deep learning models like U-Nets offer robustness but typically process channels independently at first, delaying "fusion" until later layers. RelUNet addresses this by introducing an "early fusion" strategy that captures crucial spatial relationships from the very first layer.
The RelUNet Architecture
The core innovation of RelUNet (Relative Channel Fusion U-Net) lies in its input representation. Rather than processing channels in isolation, the model stacks the real and imaginary components of each input channel's Short-Time Fourier Transform (STFT) with those of a designated "reference" channel.
This stacking allows the initial convolutional layers to immediately extract spatial features, effectively acting as an adaptive beamformer. The architecture follows a standard U-Net structure with an encoder, decoder, and skip connections. The model generates a complex Time-Frequency (T-F) mask, which is then applied to the reference channel to produce the enhanced audio. The authors also explored using Graph Convolutional Networks (GCN) or Graph Attention Networks (GAT) in the bottleneck to further model inter-microphone relationships.
Experimental Setup
The model was evaluated using the CHiME-3 dataset, which includes speech recorded in four noisy environments: Buses (BUS), Cafes (CAF), Pedestrian areas (PED), and Streets (STR). Training was conducted on simulated data, while evaluation spanned both simulated and real-world recordings to test generalization across "unseen" noise conditions.
Key Results and Performance
RelUNet consistently outperformed standard U-Net architectures across all metrics and noise types. Notably, the model achieved these improvements with a negligible 0.07% increase in parameters, making it highly efficient for real-time deployment.
On simulated data, RelUNet reached an average PESQ (Perceptual Evaluation of Speech Quality) of 1.809, compared to 1.576 for the baseline U-Net. More importantly, on real-world recordings, RelUNet maintained a PESQ of 1.404, significantly outperforming the standard U-Net (1.322) and the noisy baseline. Intelligibility, measured via STOI, also saw substantial gains, particularly in challenging environments like buses and streets.
Interestingly, the "early fusion" approach even benefited single-channel scenarios by stacking a channel with itself, likely mimicking autocorrelation techniques to identify periodic speech patterns.
Conclusion and Future Work
RelUNet demonstrates that early spatial feature extraction is vital for high-quality multichannel speech enhancement. While the current study focused on the fixed microphone geometry of CHiME-3, future work will investigate the model's performance with variable geometries—such as ad-hoc microphone arrays from multiple smartphones—and more sophisticated reference channel selection strategies.
Summary updated