Online Gaussian process for nonstationary speech separation

Hsin-Lung Hsieh, Jen-Tzung ChienInterspeech 2010

TL;DR

This paper introduces an "Online Gaussian Process Independent Component Analysis" (OLGP-ICA) method to separate speech signals in complex, real-world scenarios where multiple sound sources (speakers, music, noise) are constantly appearing, disappearing, and moving. By adaptively learning the changing relationships between sound sources and microphones, OLGP-ICA offers a more robust and efficient solution for nonstationary speech separation compared to existing methods.

What Problem Does This Paper Address?

The "cocktail party problem" – separating individual speech signals from a mixture of sounds – becomes significantly more challenging when the sound sources are nonstationary. This means speakers might move, enter or leave the scene, or their voices might overlap unpredictably. Traditional blind source separation (BSS) techniques often assume a fixed number of sources and a static mixing environment. However, in reality, the way sound waves travel from a source to a microphone (the mixing matrix) changes constantly due to factors like distance and speaker movement. Existing methods struggle to handle these dynamic changes, especially when the number of sources also varies. This paper aims to address this gap by developing a method that can adapt in real-time to these complex, nonstationary acoustic environments.

Methods

The core innovation is the Online Gaussian Process Independent Component Analysis (OLGP-ICA). This method combines two key ideas:

  • Gaussian Processes (GPs): GPs are powerful statistical tools for modeling functions with uncertainty. Here, they are used to model the temporal structure of the mixing matrix. Instead of assuming the mixing matrix is constant, GPs allow it to evolve smoothly over time, capturing the nonstationary nature of the acoustic environment.
  • Online Bayesian Learning: The algorithm learns incrementally, frame by frame, as new audio data arrives. This is crucial for real-time applications. It uses a variational Bayes (VB) approach to efficiently update its estimates of the mixing matrix and source signal distributions without needing to reprocess all past data. This is more computationally efficient than batch learning methods.

The OLGP-ICA model also incorporates a mechanism for automatically determining the number of active sources using Automatic Relevance Determination (ARD), which helps in scenarios where sources appear or disappear.

Datasets

The paper uses publicly available speech and music signals from http://www.kecl.ntt.co.jp/icl/signal/. The experiments simulate nonstationary scenarios by creating time-varying mixing matrices based on changing source-sensor distances. No new benchmark datasets are introduced; rather, existing audio data is used to create realistic nonstationary conditions.

Key Insights and Ideas

  • Modeling the Mixing Matrix with GPs is Key: The paper highlights that the nonstationary nature of the acoustic environment (i.e., the changing mixing matrix) is a critical factor to model, not just the source signals themselves.
  • Online Learning for Real-Time Adaptation: The incremental, frame-by-frame learning approach is essential for practical speech enhancement systems that need to operate in real-time.
  • ARD for Dynamic Source Count: The use of ARD within the GP framework allows the system to adapt to situations where sources appear or disappear, making it more robust than methods that assume a fixed number of sources.
  • Efficiency Gains: The OLGP-ICA method demonstrates significant computational savings compared to existing switching ICA methods.

Key Results

The OLGP-ICA method achieved the highest Signal-to-Interference Ratios (SIRs) for both demixed signals compared to several other established methods, including VB-ICA, Switching ICA, and BICA-HMM. Specifically, it achieved SIRs of 17.24 dB for demixed signal 1 and 9.96 dB for demixed signal 2. Furthermore, the proposed OLGP-ICA significantly reduced running time by 73% compared to switching ICA when implemented on a personal computer.

Limitations and Future Work

  • Computational Cost: While more efficient than some methods, online GP learning can still be computationally intensive. The authors suggest investigating sparse learning for GP models to further reduce computation and enhance model generalization.
  • Model Generalization: The authors aim to improve the model's generalization capabilities for speech separation tasks.
  • Real-World Complexity: The current experiments simulate nonstationary scenarios. Future work could explore more complex real-world acoustic environments with a wider variety of noise types and reverberation.

Summary updated

BibTeX
@inproceedings{hsieh2010online,
  author = {Hsin-Lung Hsieh and Jen-Tzung Chien},
  title = {{Online Gaussian process for nonstationary speech separation}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure