Abstract
Simultaneous utterances pose a significant challenge for both hearing-impaired individuals and automatic speech recognition (ASR) systems. While deep neural networks (DNNs) have substantially improved speech separation performance, most existing methods focus solely on estimating spectral magnitude while relying on the original mixture phase for signal reconstruction. This reliance on the "mixture phase" creates a performance bottleneck. This study proposes a two-stage phase-aware approach that integrally recovers both magnitude and phase. By utilizing the Multiple Input Spectrogram Inversion (MISI) algorithm with an Ideal Amplitude Mask (IAM) for phase recovery, and a novel Phase-Recovered Mask (PRM) for magnitude estimation, the proposed framework significantly minimizes signal distortion. Experimental results on the WSJ0-2mix and TIMIT datasets demonstrate the effectiveness and generalization capabilities of the approach.
Introduction
The "cocktail party problem"—isolating individual voices from a single-channel (monaural) recording—remains a central challenge in audio processing. Traditional deep learning approaches typically estimate the spectral magnitude (the loudness at various frequencies) but reuse the phase (the timing of the waves) from the noisy mixture. Because the mixture phase is a complex combination of all active speakers, using it for reconstruction leads to audible artifacts and limited separation quality.
Estimating phase is notoriously difficult due to its seemingly unstructured nature. This research aims to break through this "phase ceiling" by introducing a framework that treats phase recovery as a primary objective rather than an afterthought.
Proposed Two-Stage Framework
The core of the proposed system is a cascaded architecture that separates the problem into two distinct but linked stages: phase recovery and magnitude estimation.
Stage 1: Phase Recovery
In the first stage, a Bi-directional Long Short-Term Memory (BiLSTM-I) network estimates an Ideal Amplitude Mask (IAM). This mask serves as the input for the Multiple Input Spectrogram Inversion (MISI) algorithm. MISI is an iterative process that alternates between the time and frequency domains to find a phase that is mathematically consistent with the estimated magnitude.
A key finding of this study is that the IAM is the optimal starting point for MISI-based phase recovery. While previous literature often suggested the Phase-Sensitive Mask (PSM), the authors demonstrate that the IAM provides a cleaner target, resulting in lower phase distortion during the iterative refinement process.
Stage 2: Magnitude Estimation
Since phase recovery is rarely perfect, the second stage is designed to compensate for residual errors. The authors introduce the Phase-Recovered Mask (PRM), a novel mask trained to minimize the Mean Square Error (MSE) between the separated signal and the clean ground truth, specifically accounting for the timing errors left by Stage 1.
The PRM acts as a mathematical bridge: if phase recovery is perfect, the PRM converges to the IAM; if phase is unrecovered (mixture phase), it behaves like the PSM. This flexibility allows the network to "mop up" distortions and fine-tune the volume levels to match the reconstructed phase waves.
Architectural Variants
The study evaluates two neural network configurations for the second stage to test flexibility and performance:
- BiLSTM-II: Employs parallel, independent networks to estimate the PRM for each speaker.
- BiLSTM-III: Utilizes a single, unified network to handle all speakers simultaneously. This architecture incorporates a Phase Alignment step to ensure that the estimated magnitudes are correctly mapped to their corresponding recovered phases.
Experimental Evaluation
The models were trained on the WSJ0-2mix dataset (20,000 utterances) and tested for generalization on the TIMIT dataset.
Key Results
- Superior Performance: The two-stage approach achieved a Signal-to-Distortion Ratio (SDR) of 11.81 dB on WSJ0-2mix, outperforming standard baselines like uPIT-ST (10.0 dB) and Chimera++ (11.5 dB).
- Phase Impact: Incorporating the phase recovery stage (using IAM with 6 iterations) provided a significant 1.59 dB boost in SDR compared to using the noisy mixture phase.
- Generalization: The model maintained high performance on the TIMIT dataset, indicating that it learns the underlying physics of speech rather than simply memorizing training patterns.
Conclusion
This research demonstrates that integrating phase recovery into the speech separation pipeline is essential for high-fidelity results. By cascading magnitude estimation after phase recovery, the network can adapt to the specific timing of reconstructed signals. Future work will focus on reducing the computational complexity of the iterative MISI algorithm to enable real-time applications and exploring end-to-end optimization by unfolding iterations directly into the neural architecture.
Summary updated