Spectral entropy-based voice activity detector for videoconferencing systems

Bowon Lee, Debargha MuhkerjeeInterspeech 2010

TL;DR

This paper introduces a novel Voice Activity Detector (VAD) for videoconferencing that leverages spectral entropy and energy to detect higher-level speech activities like sentences, rather than just short speech segments. This approach aims to improve the efficiency and context awareness of videoconferencing systems by providing a more nuanced understanding of participant engagement.

What Problem Does This Paper Address?

Traditional Voice Activity Detectors (VADs) are primarily designed for speech coding (detecting very short silences for bit-rate optimization) or automatic speech recognition (ASR) (detecting short pauses within speech). However, for applications like videoconferencing, these granular detections are insufficient. Videoconferencing systems require a VAD that can identify higher-level speech activities, such as entire sentences or even groups of sentences, to better understand participant engagement, manage background noise, and facilitate acoustic echo cancellation. Existing methods struggle to provide this higher-level detection, often leading to undesirable transitions or false positives for short pauses.

Methods

The proposed method, termed Energy-Entropy VAD (EHVAD), utilizes two distinct features derived from the decorrelated domain of the speech signal:

  • Signal Energy: Modeled as a chi-square distribution.
  • Spectral Entropy: Modeled as a Gaussian distribution.

The core idea is that speech signals, due to their harmonic and formant structure, tend to have lower spectral entropy than background noise. Additionally, speech signals generally exhibit higher energy than noise. By combining these two features, the authors formulate a statistical model that calculates a likelihood ratio for speech presence. This likelihood ratio is then smoothed using a Hidden Markov Model (HMM) to produce a soft measure of voice activity (a posteriori probability).

The paper also details a parameter estimation strategy. Initially, parameters are set to empirically chosen values. Then, during an initialization phase, these parameters are updated recursively based on the VAD's own output. This adaptation continues until a sufficient number of frames (hundreds) have been classified for both speech-absent (Ho) and speech-present (H1) hypotheses, after which the parameters are fixed.

Datasets

The paper describes experiments conducted on a "22.5 second noisy speech signal in an office environment at 16 kHz sampling rate." No specific dataset name or public availability is mentioned.

Key Insights and Ideas

  • Entropy as a Speech Indicator: The paper highlights that spectral entropy can serve as a valuable feature for distinguishing speech from noise, with speech generally exhibiting lower entropy.
  • Higher-Level Detection: The combination of energy and entropy allows for the detection of more substantial speech segments (sentences) rather than just brief utterances, which is crucial for videoconferencing context.
  • Robustness to Short Pauses: The proposed HMM-based smoothing helps in avoiding unnecessary transitions during short silences within speech, making it more suitable for applications requiring a stable understanding of ongoing speech.
  • Parameter Adaptation: The recursive parameter estimation method allows the VAD to adapt to specific acoustic conditions without requiring pre-labeled data for training.

Key Results

The experimental results, comparing EHVAD with a spectrum-based VAD (SVAD) and a Laplacian-Gaussian VAD (LGVAD), indicate that:

  • EHVAD provides higher-level detection of speech activities compared to SVAD, which is better suited for speech coding.
  • EHVAD demonstrates fewer false positives following speech offsets compared to LGVAD.
  • EHVAD exhibits more stable detection and avoids unnecessary transitions during short pauses, unlike LGVAD, making it more suitable for videoconferencing.

Specific quantitative metrics like False Acceptance Rate (FAR) or False Rejection Rate (FRR) are not explicitly detailed in the provided text, but the qualitative comparisons suggest superior performance for EHVAD in the target application.

Limitations and Future Work

  • Initialization Dependency: The performance of the VAD relies on an initialization phase to estimate parameters. While the paper describes an adaptive approach, the initial choice of parameters and the duration of the initialization phase could influence performance.
  • Dataset Specificity: The experiments were conducted on a single recorded noisy speech signal. Further evaluation on a wider range of noisy conditions and diverse speech content would be beneficial.
  • Computational Complexity: While not explicitly detailed, the use of DCT and statistical modeling might introduce computational overhead compared to simpler VAD methods.

The paper implicitly suggests that the proposed method's suitability for videoconferencing implies potential for further integration into multimedia context analysis and other applications requiring higher-level speech understanding.

Summary updated

BibTeX
@inproceedings{lee2010spectral,
  author = {Bowon Lee and Debargha Muhkerjee},
  title = {{Spectral entropy-based voice activity detector for videoconferencing systems}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure