System request detection in human conversation based on multi-resolution Gabor wavelet features

Tomoyuki Yamagata, Tetsuya Takiguchi, Yasuo ArikiInterspeech 2009

TL;DR

This paper introduces a novel method for accurately detecting system requests within human conversations, outperforming previous approaches by leveraging multi-resolution Gabor wavelet features. This advancement is crucial for building truly hands-free and intuitive voice interfaces that can distinguish commands from casual chat.

What Problem Does This Paper Address?

Modern voice interfaces, like those in car navigation systems or robots, often rely on voice activity detection (VAD) to distinguish speech from noise. However, a significant challenge remains: differentiating between a user's command directed at the system and their spontaneous conversation with another person. Current systems often require a manual button press to activate, defeating the purpose of hands-free operation. While some methods attempt to distinguish these utterances using acoustic and prosodic features, they struggle with the nuances of natural human speech and can be confused by explanations that use similar keywords to commands. This paper aims to bridge this gap by developing a more robust system request detection method.

Methods

The core innovation lies in the use of multi-resolution analysis with Gabor wavelets applied to Log-scale Mel-frequency Filter-bank (FBANK) features.

Here's a breakdown:

  • FBANK Features: These are standard acoustic features that represent the spectral envelope of speech, adjusted to mimic human auditory perception. They capture short-term spectral characteristics.
  • Gabor Wavelet Transform: This technique is applied to each dimension of the FBANK features. Gabor wavelets are known for their ability to provide both time and frequency localization, meaning they can analyze signals at different scales and resolutions simultaneously.
  • Multi-Resolution Analysis: By varying the parameters of the Gabor wavelets (specifically, the Gaussian window length and the 1/f period), the system can analyze fluctuations in both long periods (like prosody) and short periods (like phonemes) within the speech signal.
  • Real and Imaginary Parts: The real and imaginary components of the Gabor wavelet transform are used independently as features. The imaginary part, being an odd function, acts similarly to a differential calculus, highlighting changes and fluctuations.
  • Support Vector Machines (SVM): These are used as the classifier to distinguish between system requests and spontaneous utterances based on the extracted features.

This approach differs from previous methods by going beyond simple power and pitch features and by analyzing the signal at multiple time-frequency resolutions.

Datasets

The authors used a custom-recorded robot dialog corpus. This corpus was collected from two speakers interacting with a mobile robot. The utterances were manually labeled as system requests or spontaneous conversation. The dataset consists of 1,024 total utterances, with 110 identified as system requests. The experiments were conducted using 10-fold cross-validation.

Key Insights and Ideas

  • Fluctuations are Key: The paper highlights that differences between system requests and spontaneous speech are not just in the overall energy or pitch, but in the patterns of fluctuation over different time scales.
  • Gabor Wavelets Excel at Multi-Resolution: The Gabor wavelet transform is particularly well-suited for capturing these multi-resolution characteristics, allowing the system to analyze both broad prosodic changes and fine-grained phonemic articulations.
  • Imaginary Part's Importance: The imaginary part of the Gabor wavelet transform proved to be a strong indicator, suggesting that the rate of change and fine details within the speech signal are crucial for distinguishing commands.
  • FBANK as a Foundation: While Gabor wavelets are the novel component, they are applied to FBANK features, which themselves provide a significant improvement over basic prosodic features, indicating the importance of spectrally rich representations.

Key Results

The proposed multi-resolution Gabor wavelet feature extraction method achieved a significant improvement in system request detection accuracy:

  • Baseline (Prosody-based): 66.7% F-measure.
  • FBANK Features: 80.3% F-measure.
  • Proposed Method (Gabor Wavelet Features): 92.6% F-measure.

This represents a substantial leap in performance, demonstrating the effectiveness of the multi-resolution analysis. The paper also notes that their new method improves upon their previous work [10] from 85.1% to 92.6%.

Limitations and Future Work

  • Noisy Conditions: The authors acknowledge that future work should include evaluation under noisy conditions, as the current experiments were conducted in a relatively controlled environment.
  • Linguistic Features: While the current method is purely acoustic, combining it with linguistic features (like keyword spotting or intent recognition) is suggested as a promising avenue for further improvement.
  • Broader Command Sets: The current system was tested with a limited set of commands, and future work will involve evaluating performance with a wider variety of commands and more flexible expressions.
  • Real-World Integration: The paper hints at the need for improved utterance section detection and the integration of other features for more robust real-world applications.

Summary updated

BibTeX
@inproceedings{yamagata2009system,
  author = {Tomoyuki Yamagata and Tetsuya Takiguchi and Yasuo Ariki},
  title = {{System request detection in human conversation based on multi-resolution Gabor wavelet features}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure