DCASE 2018 Challenge Surrey cross-task convolutional neural network baseline

Qiuqiang Kong, Turab Iqbal, Yong Xu, Wenwu Wang, Mark D. Plumbley

Abstract

The Detection and Classification of Acoustic Scenes and Events (DCASE) 2018 challenge encompasses a wide array of machine listening tasks, from identifying general environments to pinpointing specific sound events. This paper introduces a unified cross-task baseline system using convolutional neural networks (CNNs) to provide a standardized performance floor across all five challenge tasks. By comparing a 4-layer (CNN4) and an 8-layer (CNN8) architecture, we investigate the impact of model depth on diverse audio problems. Our findings show that while deeper architectures generally improve performance, the magnitude of gain is task-dependent, with acoustic scene classification showing less sensitivity to depth than sound event tagging.

Introduction

Audio analysis tasks are inherently diverse. Acoustic Scene Classification (ASC) requires capturing global environmental characteristics, while Sound Event Detection (SED) necessitates identifying overlapping, transient sounds in noisy streams. Historically, these problems were addressed with bespoke, task-specific models.

To provide a common starting point for the research community, this study proposes a standardized CNN framework applied to the full suite of DCASE 2018 datasets:

  1. Task 1: Acoustic Scene Classification (10 scenes like "airport" or "park").
  2. Task 2: General-purpose audio tagging (41 classes from Freesound).
  3. Task 3: Bird audio detection (binary presence/absence).
  4. Task 4: Weakly-labeled semi-supervised sound event detection (domestic environments).
  5. Task 5: Multi-channel audio classification (identifying domestic activities).

Methods

The baseline system evaluates two primary architectures: CNN4, a 4-layer network with $5 \times 5$ filters inspired by AlexNet, and CNN8, an 8-layer network with $3 \times 3$ filters modeled after the VGG architecture.

The processing pipeline begins with converting all audio to mono and generating log mel spectrograms with 64 frequency bins. Both models utilize Batch Normalization (BN) and ReLU activations to stabilize training. A key architectural feature is the use of Global Max Pooling (GMP) on the final feature maps before the classification layer. GMP ensures the network remains invariant to the temporal or spectral positioning of sounds within the input clip.

Comparison of CNN4 and CNN8 architectures, detailing layer configurations and parameter counts.

Results and Discussion

Experimental results demonstrate a clear "depth dividend" across most tasks. The CNN8 model consistently outperformed CNN4, suggesting that hierarchical feature learning—a staple of computer vision—is highly effective for complex audio pattern recognition. For instance, in Task 2 (audio tagging), CNN8 achieved 0.895 accuracy compared to 0.851 for CNN4.

However, Task 1 (ASC) emerged as a notable outlier. As shown in the detailed breakdown below, the performance gap between CNN4 and CNN8 was minimal for many scene labels. This suggests that acoustic scenes may be characterized by stationary, global statistics that do not necessarily require the high-level abstraction provided by deeper networks.

Performance comparison of CNN architectures across different scene labels in Task 1.

Task 4 (Weakly-labeled SED) proved to be the most significant challenge, with the CNN8 model reaching only a 20.8% F1 score. This difficulty stems from the lack of precise temporal labels in the training data and the inherent complexity of domestic soundscapes.

Conclusion

The Surrey cross-task CNN baseline provides a robust foundation for DCASE 2018 participants. While the 8-layer VGG-style network serves as a strong general-purpose model, the results highlight specific areas for improvement. Future research should explore architectures that better capture temporal dynamics, such as Convolutional Recurrent Neural Networks (CRNNs) or attention mechanisms, particularly for weakly-labeled tasks like Sound Event Detection. The source code for these baselines has been released under the MIT license to support further community development.

Summary updated

Abstract

The Detection and Classification of Acoustic Scenes and Events (DCASE) consists of five audio classification and sound event detection tasks: 1) Acoustic scene classification, 2) General-purpose audio tagging of Freesound, 3) Bird audio detection, 4) Weakly-labeled semi-supervised sound event detection and 5) Multi-channel audio classification. In this paper, we create a cross-task baseline system for all five tasks based on a convlutional neural network (CNN): a "CNN Baseline" system. We implemented CNNs with 4 layers and 8 layers originating from AlexNet and VGG from computer vision. We investigated how the performance varies from task to task with the same configuration of neural networks. Experiments show that deeper CNN with 8 layers performs better than CNN with 4 layers on all tasks except Task 1. Using CNN with 8 layers, we achieve an accuracy of 0.680 on Task 1, an accuracy of 0.895 and a mean average precision (MAP) of 0.928 on Task 2, an accuracy of 0.751 and an area under the curve (AUC) of 0.854 on Task 3, a sound event detection F1 score of 20.8% on Task 4, and an F1 score of 87.75% on Task 5. We released the Python source code of the baseline systems under the MIT license for further research.
BibTeX
@inproceedings{kong2018dcase,
  author = {Qiuqiang Kong and Turab Iqbal and Yong Xu and Wenwu Wang and Mark D. Plumbley},
  title = {{DCASE 2018 Challenge Surrey cross-task convolutional neural network baseline}},
  booktitle = {DCASE},
  year = {2018},
  url = {https://openreview.net/forum?id=6n9q9NUKuL},
}
Figure