Abstract
Most existing deep learning based binaural speaker separation systems focus on producing a monaural estimate for each of the target speakers, and thus do not preserve the interaural cues, which are crucial for human listeners to perform sound localization and lateralization. In this study, we address talker-independent binaural speaker separation with interaural cues preserved in the estimated binaural signals. Specifically, we extend a newly-developed gated recurrent neural network for monaural separation by additionally incorporating self-attention mechanisms and dense connectivity. We develop an end-to-end multiple-input multiple-output system, which directly maps from the binaural waveform of the mixture to those of the speech signals. The experimental results show that our proposed approach achieves significantly better separation performance than a recent binaural separation approach. In addition, our approach effectively preserves the interaural cues, which improves the accuracy of sound localization.
Introduction
The "cocktail party problem" highlights the human auditory system's remarkable ability to isolate specific voices in complex acoustic environments. This capability relies heavily on binaural cues, primarily Interaural Time Differences (ITD) and Interaural Level Differences (ILD), which allow for spatial localization. However, many current deep learning separation models are monaural-centric; even when processing two-channel inputs, they often output single-channel estimates, effectively stripping away spatial information. For applications such as hearing aids or spatial audio in virtual reality, preserving these cues is as vital as the separation itself. This paper introduces SAGRNN, a model designed to maintain the "where" alongside the "what" of speech signals.
Methods
The proposed Self-Attentive Gated RNN (SAGRNN) utilizes a Multiple-Input Multiple-Output (MIMO) framework to map binaural mixtures directly to binaural estimates for each target speaker. The core of the architecture consists of SA-MULCAT blocks, which integrate self-attention to capture long-range temporal dependencies and dense connectivity to facilitate feature reuse across layers.
Operating directly on raw waveforms, the system avoids the phase loss associated with spectrogram-based methods. A key design choice is the use of a standard Signal-to-Noise Ratio (SNR) loss function. Unlike scale-invariant alternatives (SI-SNR), standard SNR preserves the relative volume differences between ears, which are essential for maintaining accurate ILD cues.
Results
The model was evaluated using the WSJ0-2mix dataset convolved with Head-Related Transfer Functions (HRTFs) from the CIPIC database. SAGRNN demonstrated a substantial leap in performance over existing baselines. As detailed in Figure 6, the model achieved an SDR improvement of 27.19 dB, significantly higher than the 21.14 dB reached by the MIMO TasNet.
Crucially, the system excelled in spatial cue preservation. Localization accuracy tests revealed an average azimuth error of just 5.88°, a significant reduction from the 26.03° error found in the unprocessed mixture. Ablation studies further highlighted that the self-attention mechanism was the most critical component for maintaining high separation quality, as it allows the model to leverage global context to distinguish between overlapping speakers.
Conclusion
SAGRNN represents a significant advancement in binaural speaker separation by successfully integrating high-fidelity signal separation with the preservation of critical interaural cues. By employing a MIMO architecture with self-attention and dense connections, the model provides a robust solution for spatial audio applications. Future research will explore real-time implementation strategies and performance in more complex, reverberant acoustic environments.
Summary updated