Adaptive high accuracy approaches to speech activity detection in noisy and hostile audio environments

Mark Huggins, Brett Smolenski, Aaron LawsonInterspeech 2010

TL;DR

This paper introduces two novel, unsupervised Speech Activity Detection (SAD) algorithms, SERB and HPS, designed to perform accurately in challenging noisy and hostile audio environments. Unlike traditional energy-based methods that struggle with varying background noise, these new approaches focus on inherent speech characteristics, demonstrating superior performance in real-world scenarios like air traffic control and low-SNR phone calls.

What Problem Does This Paper Address?

Speech Activity Detection (SAD) is a fundamental component in many audio processing applications, such as speaker recognition and speech recognition. The core challenge lies in accurately identifying segments of speech within an audio stream, especially when the audio is corrupted by significant background noise or distorted by poor transmission channels. Traditional SAD methods often rely on simple energy thresholds, which work well in clean environments but fail dramatically when the background noise level fluctuates or exceeds speech energy. This paper tackles the problem of SAD in two particularly difficult, real-world scenarios: AM push-to-talk air traffic control communications and international telephone conversations with very low Signal-to-Noise Ratios (SNRs). The goal is to develop SAD algorithms that are inherently adaptive and robust to these hostile conditions without requiring extensive supervised training for each new environment.

Methods

The authors propose two novel, unsupervised SAD algorithms that focus on identifying intrinsic speech characteristics rather than trying to separate speech from noise.

  • Speech Energy Resonance Band (SERB) Detection: This approach identifies contiguous bands of energy that correspond to the formant resonances of speech. It utilizes the Adjustable Bandwidth Concept (ABC) to detect short-term stable medium-band energy. SERB employs a hysteresis mechanism in both time and frequency domains to make decisions, requiring a sufficient number of detected speech-like energy bands over contiguous frames. This method is designed to be tolerant to noise and varying background conditions.
  • Harmonic Product Spectrum (HPS) SAD: This algorithm measures the local harmonic content of the signal. It extracts a feature derived from the Harmonic Product Spectrum (HPS) within a specific frequency range (50Hz to 500Hz) from Hamming windowed frames. This feature is then clustered into two distinct classes using a k-medians algorithm: one for voiced speech and another for unvoiced speech and background noise. Temporal hysteresis is applied to refine the segmentation and correct for short spurious detections.

Both methods are designed to be unsupervised and adaptive, meaning they don't require prior knowledge of the specific noise or channel conditions.

Datasets

The evaluation was conducted on four datasets:

  • LDC Call Home database (original): Used for clean speech evaluation.
  • LDC Call Home database with added pink noise: Two versions were used, one with an average SNR of 10 dB and another with an average SNR of 0 dB.
  • Air Traffic Control (ATC) AM push-to-talk audio transmissions: Collected with squelch off, this dataset represents a challenging, naturally noisy environment where speech energy can be lower than background energy.

Key Insights and Ideas

  • Speech Characteristics vs. Noise Separation: The fundamental insight is that focusing on the consistent, predictable spectral characteristics of speech itself (like formants and harmonic structure) is more robust than attempting to filter out highly variable background noise.
  • Unsupervised Adaptability: The proposed algorithms are unsupervised, meaning they can be applied to new audio environments without retraining, which is a significant advantage for real-world applications.
  • Robustness in Hostile Environments: The SERB and HPS approaches demonstrate a clear advantage over traditional energy-based methods (like Xtalk and TEO) when dealing with significant noise and challenging channel conditions.
  • Tunability of SERB: The SERB algorithm offers a degree of "tunability" by adjusting the number of required medium band hits, allowing for a trade-off between precision and recall to suit specific application needs.

Key Results

  • Clean Speech: All approaches performed well, with HPS generally scoring highest in F1-score.
  • Noisy Speech (10 dB SNR): F1-scores dropped significantly for all methods. SERB emerged as the top performer, achieving an F1-score of 0.86. TEO saw a substantial decline in recall and precision.
  • Very Noisy Speech (0 dB SNR): Energy-based methods (TEO, Xtalk) were severely impacted. SERB and Xtalk maintained high precision (around 0.99) but lost recall dramatically. HPS showed some resilience.
  • Air Traffic Control (AM Push-to-Talk): This "naturally" noisy environment proved very challenging for energy-based approaches (TEO, Xtalk), which performed poorly. SERB consistently achieved high F1-scores and very high precision (0.99) across its tested configurations.

Limitations and Future Work

  • HPS Algorithm Performance in Noise: The paper notes that the HPS algorithm had a tendency to identify certain noise regions with tone-like structures as speech in the ATC dataset, making it less robust than SERB in that specific scenario.
  • Recall vs. Precision Trade-off: While SERB offers tunability, the paper shows that increasing precision by requiring more medium band hits leads to a predictable decrease in recall.
  • Future Work: The paper implicitly suggests that further refinement of these adaptive, speech-characteristic-based approaches could lead to even more robust SAD systems for a wider range of challenging audio environments. The authors also highlight the potential for SERB's tunability to be leveraged for specific application requirements.

Summary updated

BibTeX
@inproceedings{huggins2010adaptive,
  author = {Mark Huggins and Brett Smolenski and Aaron Lawson},
  title = {{Adaptive high accuracy approaches to speech activity detection in noisy and hostile audio environments}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure