Speech activity detection for noisy data using adaptation techniques

Mohamed Kamal OmarInterspeech 2012

TL;DR

This paper tackles the challenge of accurately detecting speech in highly noisy audio by introducing a two-pass approach. By first performing an initial speech/non-speech segmentation and then using adaptation techniques (MAP and RFMLLR) to refine the models based on the specific audio characteristics, they achieve significant improvements over traditional single-pass methods, especially in degraded acoustic conditions.

What Problem Does This Paper Address?

Speech Activity Detection (SAD) is a crucial preprocessing step for many audio analysis tasks, including speech recognition, speaker recognition, and audio indexing. The core challenge is that real-world audio is often contaminated with significant noise and recorded under diverse, degraded channel conditions. Traditional SAD methods, whether heuristic, statistical, or pattern recognition-based, often struggle to generalize to these unseen acoustic environments. This paper specifically addresses the performance degradation of SAD systems when faced with highly noisy and varied channel conditions, aiming to improve robustness and accuracy in such scenarios.

Methods

The authors propose a two-pass SAD system built upon a modified Cumulative Sum (CUSUM) algorithm. The baseline system is a single-pass modified CUSUM that uses Gaussian Mixture Models (GMMs) for speech and non-speech classes to segment audio. The novel contribution lies in the two-pass systems:

  • Two-Pass MAP Adaptation: After an initial segmentation by the single-pass CUSUM, a confidence measure is used to select segments. These selected segments then inform a Maximum A Posteriori (MAP) adaptation of the GMM means for the speech and non-speech models. These adapted models are then used in a second pass of the CUSUM algorithm for the final segmentation.
  • Two-Pass RFMLLR Adaptation: Similar to the MAP approach, an initial segmentation is performed. However, instead of adapting GMM means, this system uses class-specific Regularized Feature-based Maximum Likelihood Linear Regression (RFMLLR) transforms to adapt the feature vectors themselves. These adapted features are then used in the second pass of the CUSUM algorithm.

The modified CUSUM algorithm itself is described as detecting segment boundaries by identifying extrema of the sum of log-likelihood ratios over a fixed duration, rather than a simple threshold comparison.

Datasets

The experiments were conducted using data from the Robust Automatic Transcription of Speech (RATS) program. This data consists of audio recordings from various sources transmitted through a multi radio-link channel collection system, simulating different modulation and frequency ranges. The training data comprised approximately 900 hours. Two test sets were used: RATS dev1 (approx. 11 hours from 8 channels) and RATS dev2 (approx. 17 hours from the same 8 channels). These datasets are characterized by their "highly degraded channel conditions."

Key Insights and Ideas

  • The Power of Two Passes: The core insight is that a single pass is insufficient for robust SAD in noisy conditions. A second pass, informed by the characteristics of the specific audio segment, is crucial for adaptation and improved performance.
  • MAP Adaptation Outperforms RFMLLR: The paper strongly suggests that adapting the GMM means (MAP) is more effective than adapting the feature vectors (RFMLLR) for this specific SAD task, especially when data for adaptation is limited.
  • Channel-Specific Models Matter: Using channel-specific GMM models for speech and non-speech, even in the baseline system, provides additional gains over channel-independent models. This highlights the significant impact of channel variability.
  • RFMLLR Limitations: The RFMLLR approach, while theoretically sound, appears to suffer from insufficient data for reliable transform estimation in this context, leading to less impressive results compared to MAP adaptation.

Key Results

  • The two-pass MAP adaptation system reduced the total error by 27%-54% relative to the single-pass baseline system on the RATS dev1 test set.
  • Using channel-specific GMM models for speech and non-speech (instead of channel-independent ones) provided additional gains of 3%-25% relative to the channel-independent two-pass MAP system.
  • Across most channels on both dev1 and dev2 test sets, the two-pass MAP system significantly outperformed the two-pass RFMLLR system.
  • Combining RFMLLR and MAP adaptation did not consistently improve performance over using MAP adaptation alone.

Limitations and Future Work

  • RFMLLR Data Requirements: The paper notes that the RFMLLR system's performance might be limited by the amount of data available for adaptation, suggesting that more data could potentially improve its effectiveness.
  • Channel-Specific Model Gains: While channel-specific models showed gains, the improvement varied across channels, indicating that further research into how to best leverage channel information might be beneficial.
  • Future Directions: The authors implicitly suggest exploring more sophisticated adaptation techniques or ways to better utilize the limited adaptation data for methods like RFMLLR. The consistent superiority of MAP adaptation also points towards further investigation into its application in SAD for noisy environments.

Summary updated

BibTeX
@inproceedings{omar2012speech,
  author = {Mohamed Kamal Omar},
  title = {{Speech activity detection for noisy data using adaptation techniques}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure