Abstract
Speech separation in reverberant environments remains a significant challenge due to the overlap of direct sound and late reflections. We present the Time-Domain Deep Attractor Network (TD-DAN), a two-stream architecture designed to simultaneously perform speaker separation and dereverberation. By decoupling speaker identification from waveform reconstruction and targeting early reflections rather than anechoic signals, TD-DAN achieves state-of-the-art performance on the SMS-WSJ dataset, outperforming established models like Conv-TasNet.
Introduction
Real-world speech processing must contend with multiple concurrent speakers and acoustic reverberation. While the Deep Attractor Network (DAN) effectively separates speakers in clean conditions by mapping signals to an embedding space, its performance degrades in reverberant settings where the assumption of time-frequency bin sparsity fails. Existing methods often treat separation and dereverberation as sequential tasks. This paper introduces TD-DAN, a unified framework that addresses both challenges within a single time-domain architecture.
Methodology: The Two-Stream Architecture
The core of TD-DAN is its dual-stream design, which separates the tasks of identifying speakers and extracting their specific waveforms.
- Speaker Encoding Stream (SES): This stream acts as the "identity finder." It processes the mixture to create a high-dimensional embedding space where "attractors" (centroids representing individual speakers) are identified. The SES can utilize various encoders, such as STFT or learnable convolutional kernels, allowing for multi-domain learning.
- Speech Decoding Stream (SDS): Functioning as the "extractor," this stream uses the attractors generated by the SES to filter the original mixture and reconstruct the time-domain waveforms for each speaker.
Key Innovations and Loss Functions
To handle the "smearing" effect of reverberation, TD-DAN incorporates specialized loss functions and learning targets.
- Auxiliary Clustering Losses: The model employs a Concentration Loss to pull T-F bin embeddings closer to their respective attractors and a Discrimination Loss to maximize the distance between different speakers' attractors. This counters the overlap caused by echoes.
- Early Reflection Target: Rather than aiming for a perfectly dry (anechoic) signal, TD-DAN targets "early reflections." These initial sound bounces preserve intelligibility while avoiding the chaotic complexity of late reflections, making the learning objective more attainable and effective for downstream tasks like speech recognition.
Results and Discussion
Evaluations on the SMS-WSJ dataset demonstrate the superiority of the TD-DAN approach. In 2-speaker mixtures, TD-DAN achieved a Scale-Invariant Signal-to-Distortion Ratio (SI-SDR) gain of 9.79 dB, surpassing Conv-TasNet by 0.91 dB and the original DAN by 1.92 dB. Performance remained robust in 3-speaker scenarios, maintaining a 0.47 dB lead over Conv-TasNet. Crucially, the model also yielded the lowest Word Error Rates (WER), indicating that the dereverberation process significantly enhances speech intelligibility for automated systems.
Conclusion
TD-DAN represents a significant advancement in monaural speech separation for reverberant environments. By leveraging a two-stream architecture and targeting early reflections, it effectively mitigates the impact of late reflections. Future work will explore the integration of multi-channel spatial information to further improve performance in complex acoustic scenes.
Summary updated