Speaker-dependent voice activity detection robust to background speech noise

Shigeki Matsuda, Naoya Ito, Kosuke Tsujino, Hideki Kashioka, Shigeki SagayamaInterspeech 2012

TL;DR

This paper introduces a speaker-dependent Voice Activity Detection (VAD) system that significantly improves speech detection accuracy in noisy environments with background speech. By incorporating a speaker-adapted Gaussian Mixture Model (GMM) alongside traditional noise and speaker-independent GMMs, the system can better distinguish target speech from interfering speakers, leading to improved performance in downstream Automatic Speech Recognition (ASR) tasks.

What Problem Does This Paper Address?

Voice Activity Detection (VAD) is a fundamental component in many speech processing systems, responsible for identifying segments of speech within an audio stream. Traditional VAD systems perform well in clean environments but struggle significantly in noisy conditions, particularly when the noise source is other human speech. This background speech interference is a common problem in real-world applications like smartphone-based translation services, leading to errors like word insertions in Automatic Speech Recognition (ASR) systems. The challenge lies in the fact that background speech often shares similar acoustic characteristics with the target speaker's speech, making it difficult to differentiate.

Methods

The core innovation of this paper is a speaker-dependent VAD algorithm that leverages three Gaussian Mixture Models (GMMs):

  • Noise GMM: Models general background noise.
  • Speaker-Independent GMM (dubm): Models general speech characteristics, independent of any specific speaker.
  • Speaker-Adapted GMM (duser): This is the novel addition. It's adapted to the specific target user's voice.

The VAD decision is made by comparing acoustic probability scores from these GMMs. Specifically, equation (2) shows that the system favors frames where the combined scores from the noise and speaker-independent GMMs are less than the score from the speaker-adapted GMM. This means that if the speaker-adapted GMM strongly identifies the speech as belonging to the target user, it's classified as speech, even if the other GMMs indicate some noise or general speech.

The speaker-adapted GMM is trained using adaptation techniques like Maximum A Posteriori (MAP) adaptation and Multi-Class Maximum Likelihood Linear Regression (MLLR). The paper also incorporates a "hangover processing" mechanism (illustrated in Figure 1) to refine the detected speech segments by removing very short fragments, filling short silences between speech segments, and ensuring a minimum duration for detected speech.

Datasets

  • Training Data for Speaker-Independent GMM:
    • ATR travel arrangement task database (TRA) (5 hours of dialogue speech)
    • ATR phonetically balanced sentences (TRA-BLA) (25 hours of read speech)
    • Total: ~400 speakers, ~26k sentences. This data was contaminated with 15 types of noises.
  • Adaptation Data:
    • 200 sentences per speaker from the BTEC corpus (20 male, 20 female speakers). This data was also contaminated with noises and SNRs similar to the training data.
  • Testing Data:
    • Remaining 100 sentences per speaker from the BTEC corpus.
  • Contaminated with 5 types of noises from Table 1 at four SNRs (30, 25, 20, 15 dB).
  • Background speakers' speech was added at -12 dB relative to the target speaker.
  • ASR Evaluation Data:
    • ATR travel arrangement task database (TRA)
    • ATR Spontaneous Speech Database (APP)
    • Read speech of phonetically balanced sentences (TRA-BLA and APP-BLA)
    • Total: ~170k utterances.
  • Language models trained on spontaneous speech database (SDB), language database (LDB), and spoken language database (SLDB).

Key Insights and Ideas

  • Background Speech is a Distinct Challenge: The paper highlights that background speech is a particularly difficult type of noise for VAD because it shares statistical properties with target speech, unlike stationary noise.
  • Speaker Adaptation is Key: The success of the proposed VAD hinges on the ability to adapt a speech model to the specific target user. This allows the system to "listen" for a particular voice, effectively filtering out other voices.
  • MAP vs. MLLR Adaptation: The study found that MLLR adaptation generally yielded better VAD performance than MAP adaptation when adaptation data was limited. The authors hypothesize this is because MAP adaptation might not adapt all Gaussian distributions, leaving some susceptible to background speech influence.
  • Direct Impact on ASR: The improvements in VAD directly translate to better ASR performance, particularly in reducing word insertion errors, demonstrating the practical utility of the proposed VAD.

Key Results

  • VAD Performance: The proposed speaker-dependent VAD consistently outperformed the conventional GMM-based VAD in the presence of background speakers' speech across all tested SNRs (30, 25, 20, 15 dB). The performance gap widened at lower SNRs.
  • ASR Performance: When integrated into an ASR system, the proposed VAD (specifically using MLLR adaptation) achieved comparable performance to the baseline VAD in clean conditions. However, in the presence of background speech, the proposed VAD significantly reduced ASR errors, most notably word insertion errors.

Limitations and Future Work

  • Limitations:
    • The paper notes that MAP adaptation might be less effective than MLLR when adaptation data is limited, as it may not adapt all Gaussian components.
  • The study focuses on specific types of background speech noise (from the ATR Spontaneous Speech Database).
  • Future Work:
    • The authors plan to evaluate the performance of their proposed VAD in real-world speech data collected from the "VoiceTra" service.

Summary updated

BibTeX
@inproceedings{matsuda2012speaker,
  author = {Shigeki Matsuda and Naoya Ito and Kosuke Tsujino and Hideki Kashioka and Shigeki Sagayama},
  title = {{Speaker-dependent voice activity detection robust to background speech noise}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure