A fast one-pass-training feature selection technique for GMM-based acoustic event detection with audio-visual data

Taras Butko, Climent NadeuInterspeech 2010

TL;DR

This paper introduces a novel, fast one-pass-training feature selection method for acoustic event detection (AED) that significantly improves accuracy by intelligently selecting the most relevant audio-visual features for each specific sound event. This approach drastically reduces computational cost compared to traditional methods by avoiding repeated model retraining, making it more practical for complex, real-world scenarios.

What Problem Does This Paper Address?

Acoustic event detection (AED) aims to identify specific sounds and their timing within audio signals. This task is challenging due to the spontaneous and often overlapping nature of sounds, and the vast number of potential audio features that can be extracted. While many features exist, selecting the optimal subset for each distinct acoustic event is crucial for performance. Traditional feature selection methods can be computationally expensive, requiring retraining of acoustic models for each feature subset evaluation. Furthermore, when incorporating multi-modal data like audio and video, the dimensionality of features increases, exacerbating the selection problem. This paper addresses the need for an efficient and effective feature selection technique that can handle multi-modal data and improve AED accuracy without prohibitive computational overhead.

Methods

The core contribution is a fast one-pass-training feature selection technique. This method leverages a one-against-all detection approach, where each acoustic event class is trained against all other classes. Instead of retraining Gaussian Mixture Models (GMMs) for every feature subset evaluation, as in conventional methods, this new approach performs an initial GMM training with all available features.

The key innovation lies in how feature contributions are evaluated. The paper proposes a new, more precise metric called Soft Detection Accuracy (SDA). SDA goes beyond simply counting correct/incorrect detections by incorporating the confidence of the model's decision. It uses a normalized score based on the Log-Likelihood Ratio (LLR) of the "Class" and "non-Class" models. A higher LLR indicates greater confidence that a segment belongs to the target event.

The one-pass-training algorithm works by:

  1. Initial GMM Training: Training GMMs using all available features.
  2. Local LLR Computation: Calculating the local LLR for each frame and each feature. This is a crucial step that avoids retraining.
  3. Hill-Climbing Search: Employing a standard hill-climbing strategy to iteratively add features that maximize the SDA score. The LLRs are computed using the pre-trained models and the local LLR contributions from individual features.

This approach significantly reduces computational load because the GMMs are not retrained during the feature selection process.

Datasets

The experiments were conducted using a subset of isolated acoustic events from a recently recorded multimodal database collected at the UPC multimodal room. This database includes synchronized audio (from 6 T-shaped 4-microphone clusters) and video (from 5 calibrated cameras). The dataset was divided into three parts: training, development (for feature selection), and testing. The paper mentions that around 100 instances for every AE were collected, totaling approximately 2 hours of data.

Key Insights and Ideas

  • Feature Selection is Crucial for Multi-modal AED: The paper strongly emphasizes that not all features are equally useful for every acoustic event. Intelligent selection is key to improving performance, especially when dealing with diverse data sources like audio and video.
  • Computational Efficiency is Paramount: Traditional feature selection methods are computationally prohibitive due to repeated GMM retraining. The proposed one-pass-training approach offers a significant speedup by decoupling feature evaluation from model retraining.
  • Confidence Matters in Detection: The SDA metric is a more nuanced evaluation than simple accuracy. It accounts for the model's confidence in its predictions, which is vital for robust AED, especially in scenarios with ambiguous sounds.
  • Audio-Visual Fusion Benefits Specific Events: The results show that video features are particularly beneficial for detecting certain events, while acoustic localization features are useful for others, highlighting the importance of tailored feature selection for each event class.

Key Results

The proposed one-pass-training feature selection technique demonstrated an improvement in average detection rate compared to using the entire set of features. Specifically, the average mean relative improvement across acoustic events was 5.0% for the one-pass-training approach, slightly outperforming the conventional approach's 4.5%. This was achieved with a feature compression ratio of 1.67, meaning the number of selected features was significantly reduced. The paper also shows that the new technique works similarly to the conventional approach in terms of detection rate improvement, but with substantially less computational cost.

Limitations and Future Work

The authors acknowledge that the approximation of the local log-likelihood using the dominant component in the mixture (Equation 10) introduces a small average error, depicted in Figure 4. While this approximation is necessary for the efficiency of the one-pass-training method, it's a trade-off.

Future work directions include:

  • Exploring a larger initial set of features.
  • Comparing the proposed method with other dimensionality reduction techniques like Linear Discriminant Analysis (LDA) or Principal Component Analysis (PCA).
  • Investigating the method's performance against existing filter methods in terms of speed and efficiency.

Summary updated

BibTeX
@inproceedings{butko2010fast,
  author = {Taras Butko and Climent Nadeu},
  title = {{A fast one-pass-training feature selection technique for GMM-based acoustic event detection with audio-visual data}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure