Acoustic Scene Classification by Implicitly Identifying Distinct Sound Events

Hongwei Song, Jiqing Han, Shiwen Deng, Zhihao DuInterspeech 2019

TL;DR

This paper proposes a novel approach to acoustic scene classification (ASC) by focusing on identifying distinct sound events within a scene, rather than relying solely on global acoustic features. They introduce a Multi-Instance Learning (MIL) framework that implicitly learns to recognize these sound events, leading to improved classification accuracy and a deeper understanding of scene composition.

What Problem Does This Paper Address?

Acoustic scene classification (ASC) aims to categorize audio recordings into environmental classes like "office" or "park." Traditional methods often analyze the overall acoustic distribution or temporal evolution of audio features. While some approaches attempted a two-stage process of first detecting sound events and then classifying the scene, this requires extensive manual annotation of sound events (onset and offset times), which is labor-intensive and often impractical for real-world scenarios with overlapping sounds and ambiguous boundaries. This paper addresses the need for an ASC method that can learn from weakly labeled data (only the scene label is provided) and implicitly identify the crucial sound events that define a scene, mimicking how humans perceive their environment.

Methods

The core of this work lies in framing ASC as a Multi-Instance Learning (MIL) problem. Here's a breakdown of their approach:

  • Multi-Instance Learning (MIL) Framework: Each audio recording is treated as a "bag" of "instances." These instances are high-level representations derived from segments of the audio, intended to capture distinct sound events. The MIL framework allows learning from bag-level labels without explicit instance labels.
  • CNN-MIL Model: They propose a deep learning model that first uses a VGG-like Convolutional Neural Network (CNN) as an "instance generator" to transform log mel-spectrograms into these instance vectors.
  • Instance Detectors: For each scene class, a dedicated detector is trained to identify if an instance vector corresponds to a distinct sound event for that scene. This is achieved through an affine transformation followed by a sigmoid activation.
  • Prediction Aggregator: A "max pooling" function aggregates the predictions from individual instances to produce a bag-level prediction for the scene. This aligns with the standard MIL hypothesis where a positive bag contains at least one positive instance.
  • Multi-Temporal Scale (MTS) Module: To capture both short-lived (e.g., bird chirps) and long-lasting (e.g., engine hum) sound events, they employ dilated convolutions. This module expands the temporal receptive field of the instance representations.
  • Multi-Detector (MD) Module: Recognizing that scenes can be composed of multiple distinct sound events, they extend the single detector per class to multiple detectors (up to K). This allows the model to learn different "templates" for various sound events within the same scene, addressing the multi-modal nature of sound events.

Datasets

The experiments were conducted on the development set of the DCASE2018 Task1 Subtask B. This dataset contains single-channel audio recordings with a sampling rate of 44.1 kHz, featuring ten acoustic scene classes. Each scene has 864 segments of 10 seconds each, totaling 24 hours of audio. The official training and testing partitions were used.

Key Insights and Ideas

  • Implicit Sound Event Identification: The most significant insight is that by framing ASC within an MIL framework, the model can implicitly learn to identify distinct sound events that characterize a scene, without requiring manual annotations. This is a more scalable and practical approach.
  • Mimicking Human Perception: The paper draws inspiration from human listening tests, which suggest that humans identify scenes by recognizing prominent sound events. Their method aims to replicate this cognitive process computationally.
  • Multi-Modality of Sound Events: The introduction of the Multi-Detector (MD) module highlights the crucial idea that a single scene is often defined by a combination of multiple distinct sound events, not just one. This module allows for a richer representation of scene characteristics.
  • Temporal Scale Matters: The Multi-Temporal Scale (MTS) module demonstrates the importance of capturing sound events across different durations, from transient to sustained sounds, for accurate scene classification.

Key Results

  • The proposed CNN-MIL model, incorporating the MTS and MD modules, achieved an accuracy of 68.3% on the DCASE2018 Task1 Subtask B development set.
  • This represents a significant improvement of 9.4% over the official baseline (58.9%).
  • Ablation studies showed that both the MTS and MD modules individually contributed to performance improvements, with the MD module showing a more substantial impact.
  • The hyperparameter K (number of detectors in the MD module) was found to be influential, with K=4 yielding the best results. Increasing K beyond this point did not improve performance, suggesting that the model might learn redundant detectors.
  • The confusion matrix revealed confusions between scenes with similar acoustic characteristics (e.g., "metro," "tram," "bus"), suggesting potential for further improvements by combining evidence from different strategies.

Limitations and Future Work

  • Performance Gap: While the proposed models significantly outperform the baseline, they have not yet reached the state-of-the-art performance reported by other methods (e.g., Modified Xception [16] at 76.9%).
  • Prediction Aggregation: The authors acknowledge that the choice of prediction aggregator (currently max pooling) could be further investigated, as it has been shown to impact performance in related tasks like sound event detection.
  • Pre-trained Embeddings: They suggest that leveraging CNN embeddings pre-trained on large-scale sound event datasets could potentially guide the instance generator and improve performance.
  • Instance-Level Predictions: The instance-level predictions generated by the model are a valuable byproduct that could be further exploited for tasks like inferring bag labels or providing more detailed scene descriptions.
  • Generalizability: The authors believe their proposed MTS and MD modules are not limited to ASC and could be beneficial for other related tasks such as sound event detection and bird sound detection.

Summary updated

BibTeX
@inproceedings{song2019acoustic,
  author = {Hongwei Song and Jiqing Han and Shiwen Deng and Zhihao Du},
  title = {{Acoustic Scene Classification by Implicitly Identifying Distinct Sound Events}},
  booktitle = {Interspeech},
  year = {2019},
}
Figure