TL;DR
This paper introduces STRAUSS, a novel blind source separation method for reverberant environments that tackles the challenging scenario of having more sources than microphones. By leveraging sparsity and low-rank assumptions on the time-frequency representation of audio signals, STRAUSS achieves state-of-the-art performance while being significantly faster than existing methods, and remarkably, it only relies on amplitude information.
What Problem Does This Paper Address?
Blind Source Separation (BSS) aims to disentangle individual audio signals from a mixture of sounds without prior knowledge of the sources or the mixing process. A particularly difficult scenario is "underdetermined BSS," where the number of sources (N) exceeds the number of microphones (M). This is common in real-world applications like separating instruments in a musical piece or voices in a crowded room. Existing methods often struggle with reverberation (echoes) and computational cost. This paper focuses on convolutive mixtures (where echoes are present) in an underdetermined setting, aiming for a more efficient and robust solution.
Methods
The core of the STRAUSS method lies in combining two key assumptions about the audio signals in the time-frequency domain:
- Sparsity: At any given time-frequency bin, only one source is considered dominant. This is a common assumption in BSS, simplifying the problem by assuming that not all sources are active simultaneously in every tiny segment of the audio.
- Low-Rank Amplitude: The magnitude of the spectrogram of each source can be represented by a low-rank matrix. This means that the spectral patterns of a source can be approximated by a small number of fundamental "motifs" and their activations over time.
The proposed method, STRAUSS (SparsiTy and low-Rank AmplitUde based Source Separation), uses these assumptions to develop a novel multichannel Non-negative Matrix Factorization (NMF) approach. Instead of using the full complex STFT coefficients, STRAUSS operates solely on the amplitude of these coefficients. This is a significant departure from many existing methods that rely on phase information.
The process involves:
- Joint NMF: Performing a joint NMF on the amplitude spectrograms of the observed mixtures. This factorization aims to find shared activation patterns (H) across different mixtures, while also learning source-specific spectral patterns (V).
- Clustering: Using the learned spectral patterns (V) to cluster them, effectively grouping the patterns that belong to the same source. The paper proposes using ratios of these pattern matrices to overcome instabilities with small values.
- Source Reconstruction: Reconstructing the separated source signals using a Wiener filtering approach based on the estimated spectral patterns and activations.
Datasets
The paper evaluates its method using synthetic music mixtures. The room impulse responses (simulating reverberation) were generated using a toolbox [19]. The audio sources were drawn from publicly available datasets:
- MTG MASS database [20]
- The 2011 signal separation evaluation campaign (SiSEC2011) dataset [21]
The experiments were conducted on stereo music mixtures (M=2) with three sources (N=3).
Key Insights and Ideas
- Amplitude-Only Separation is Viable: A major insight is that effective source separation in reverberant, underdetermined scenarios can be achieved using only the amplitude information of the STFT coefficients, without needing the phase. This simplifies the problem and potentially reduces computational complexity.
- Synergy of Sparsity and Low-Rank: The paper demonstrates that the combination of sparsity and low-rank assumptions is powerful for underdetermined BSS, especially when dealing with realistic convolutive mixtures.
- Computational Efficiency: The proposed STRAUSS method is significantly faster than existing multichannel NMF approaches, making it more practical for real-time or large-scale applications.
- Robustness to Reverberation: STRAUSS shows good performance across a range of reverberation times, outperforming other methods in higher reverberation conditions.
Key Results
- Performance: STRAUSS (both IS and KL divergence versions) outperforms existing methods like MNMF and "Full rank" in terms of Signal-to-Distorsion Ratio (SDR) and Signal-to-Interference Ratio (SIR), especially in reverberant conditions.
- Speed: The STRAUSS algorithms are considerably faster, being up to 20 times quicker than MNMF and "Full rank" methods. For example, STRAUSS-KL took 36.7 seconds, while MNMF took 2381.4 seconds for a specific experimental setup.
- Amplitude vs. Oracle: Even with only amplitude information, the performance is close to oracle results (where ground truth is used for clustering), suggesting that amplitude carries a significant portion of the necessary information for separation. The improvement from oracle results using only amplitude was around 3 dB.
- Robustness: The method shows robustness to microphone distance, with performance degrading only slightly at larger distances.
Limitations and Future Work
- Phase Information: While the paper successfully demonstrates amplitude-only separation, it acknowledges that incorporating phase information could further improve performance, as suggested by previous work [12]. The authors believe there is significant potential to enhance separation techniques by better utilizing phase.
- Initialization Sensitivity: Although the paper uses deterministic initialization (complex SVD) for some experiments, the general NMF approach can be sensitive to initialization. The authors mention using multiple random initializations to mitigate this.
- Rank Selection: The choice of the rank (Kj) for the NMF decomposition is a parameter that might require careful tuning for optimal performance.
The authors conclude that their work highlights the potential of amplitude-based BSS and suggests that future research should focus on effectively integrating phase information to push the boundaries of separation performance.
Summary updated