Introduction
The "Cocktail Party Problem"—isolating a single voice from a noisy mixture—remains a classic challenge in signal processing. While many algorithms exist for Blind Source Separation (BSS), they often struggle with convolutive mixtures, where signals reach microphones at different times due to room acoustics and echoes.
Mathematically, if sources are treated as stationary white noise, the problem suffers from a "rotational invariance" flaw: one cannot uniquely identify the original sources using only second-order statistics. To break this ambiguity, researchers must exploit the unique properties of speech. This paper addresses the gap by leveraging two specific traits: non-stationarity (speech statistics change over time) and harmonicity (voiced speech has a periodic structure). By explicitly modeling speech as a combination of harmonic tones and colored noise, the authors demonstrate that it is possible to perform blind source separation and pitch tracking simultaneously.
The Harmonic plus Noise (H+N) Model
The authors formulate a block-based generative model where each source is represented by a Harmonic plus Noise (H+N) model:
- Harmonic Part: A sum of sinusoids representing the "voiced" components of speech.
- Noise Part: An Auto-Regressive (AR) process representing "unvoiced" sounds or colored noise.
This representation is embedded into a linear state-space framework. To learn the parameters—such as the mixing filters, noise statistics, and pitch—the authors employ the Expectation-Maximization (EM) algorithm:
- E-step: Uses a Kalman smoother to estimate the most likely hidden source signals given the observed mixture.
- M-step: Updates the model parameters (like the mixing matrix and fundamental frequency) to maximize the likelihood of the data.
Because pitch estimation is non-linear and prone to local maxima (e.g., getting stuck on sub-harmonics), the authors use a constrained grid search within the typical human vocal range (50–400 Hz) to track the fundamental frequency.
Experimental Setup
The approach was evaluated using two primary setups:
- Artificial Mixtures: Speech signals from the TIMIT corpus were mixed using known FIR filters and degraded with additive white noise (20dB SNR). This allowed the authors to compare their pitch estimates against "ground truth" Electro-laryngograph (EGG) recordings, which measure vocal fold vibration directly.
- Real Room Recordings: A standard benchmark dataset featuring two male speakers synchronously counting in English and Spanish, recorded in a real acoustic environment.
Key Results and Insights
Simultaneous Estimation and Identifiability
Instead of treating pitch tracking and source separation as two separate steps, the state-space model allows them to inform one another. Better separation leads to better pitch tracking, and vice versa. Furthermore, because speech is non-stationary, the "rotational" ambiguity of BSS is resolved. The model uses the changing statistics across different time frames to lock onto the correct mixing filters.
Pitch Accuracy
In the artificial mixture experiment, the estimated pitches closely followed the EGG ground truth. The estimates were "frame-local" yet remarkably smooth, indicating the model's robustness.
Separation Quality and Re-synthesis
Spectrogram analysis showed that the algorithm successfully recovered the distinct harmonic structures of two overlapping male voices.
Beyond just separating the signals, the model allows for the re-synthesis of speech from the estimated H+N parameters. This produces intelligible speech that is often cleaner than the raw Maximum A Posteriori (MAP) estimates. In the real-world English/Spanish counting experiment, listening tests confirmed successful separation and high-quality synthesis, even with 20dB of noise.
Limitations and Future Work
While highly effective, the authors acknowledge that the approach is computationally intensive, making it currently unsuitable for real-time applications without further optimization or approximate inference techniques. Additionally, while the model handles convolutive mixtures well, more realistic models of long-term reverberation are needed for complex environments like large halls. Future work could involve incorporating more sophisticated linguistic or phonetic priors to further improve separation in extremely noisy conditions.
Summary updated