Ensemble of Convolutional Neural Networks for Weakly-supervised Sound Event Detection Using Multiple Scale Input

Donmoon Lee, Subin Lee, Yoonchang Han, Kyogu LeeDCASE 2017 · OpenReview

Abstract

This paper presents a winning system for the DCASE 2017 Challenge Task 4, focusing on detecting automotive sound events using an ensemble of Convolutional Neural Networks (CNNs). The approach addresses the challenges of weak supervision and varying temporal scales by utilizing a multi-scale input strategy. By combining a global-input model for audio tagging with multiple separated-input models for localization, the system effectively bridges the gap between identifying sound categories and pinpointing their occurrences. The proposed ensemble achieved 1st place in Sound Event Detection and 2nd place in Audio Tagging.

Introduction

Sound Event Detection (SED) is a vital component for autonomous driving systems, serving as an acoustic safety net when visual sensors are compromised. However, SED faces two primary difficulties: weak supervision, where training data lacks precise timestamps, and the "scale problem," where different sounds require different temporal windows for optimal detection. Short windows excel at localizing transient sounds like horn honks, while longer windows provide the context necessary for sustained sounds like sirens.

Methods

The core of the proposed system is a multi-scale ensemble of CNNs. The architecture processes audio through two main pathways:

  1. Global-Input Model: Processes the entire 10-second audio clip to perform audio tagging.
  2. Separated-Input Models: Utilize sliding windows of varying lengths (1, 2, 3, 4, and 5 seconds) to capture local temporal features and facilitate localization.
Overview of the multi-scale input model architecture, showing global and separated input pathways.

Both model types employ a VGG-style fully convolutional structure with "double_conv" blocks. To enhance performance, a background noise removal step subtracts the median Mel-spectrogram value along the time axis. The final output is generated through an automated ensemble selection algorithm that weights the models to maximize validation performance.

The ensemble process combining clip-wise and segment-wise probabilities to produce event labels and precise timestamps.

Datasets

The study utilized the DCASE 2017 Challenge dataset (Task 4), which is a subset of Google’s AudioSet. It contains 17 categories of warning and vehicle sounds, such as ambulance sirens and car alarms. The training set consists of 51,172 clips with weak labels (presence/absence), while the evaluation sets are strongly labeled with exact timestamps to test localization accuracy.

Results and Discussion

The multi-scale approach proved highly effective, securing top rankings in the DCASE 2017 competition. The ensemble achieved a segment-based Error Rate (ER) of 0.66 and an event-based F1-score of 0.536 for localization, alongside an F1-score of 0.526 for tagging.

Key findings include:

  • Scale Specialization: The 10-second global model was superior for tagging, whereas the 3-second separated model was most effective for localization. Combining them allows the system to benefit from both context and precision.
  • Background Subtraction Trade-off: This technique significantly improved performance for short-window models but hindered 5-second and 10-second models, suggesting that long-duration context includes useful contextual information that shouldn't be filtered out.
  • Overcoming Weak Labels: The results demonstrate that an ensemble of segment-based classifiers can achieve high localization accuracy without timestamped training data by "voting" to find the most likely timestamps.

Conclusion

The research highlights the power of multi-scale temporal modeling in sound event detection. While successful, the authors noted that limited strongly labeled evaluation data makes models vulnerable to overfitting. Future work may involve exploring more complex temporal modeling through Recurrent Neural Networks or Transformers and tailoring internal CNN architectures to specific input scales.

Summary updated

BibTeX
@inproceedings{lee2017ensemble,
  author = {Donmoon Lee and Subin Lee and Yoonchang Han and Kyogu Lee},
  title = {{Ensemble of Convolutional Neural Networks for Weakly-supervised Sound Event Detection Using Multiple Scale Input}},
  booktitle = {DCASE},
  year = {2017},
  url = {https://openreview.net/forum?id=swYPjLWsCK},
}
Figure