DCASE-Models: A Python Library for Computational Environmental Sound Analysis using Deep-Learning Models

Pablo Zinemanas, Ignacio Hounie, Pablo Cancela, Frederic Font, Martín Rocamora, Xavier SerraDCASE 2020 · OpenReview

Abstract

DCASE-Models is an open-source Python library designed to streamline the development and reproduction of environmental sound analysis systems. By providing a standardized, modular framework for everything from data augmentation to model evaluation, it allows researchers to build complex deep-learning pipelines for acoustic scene classification and sound event detection with minimal boilerplate code.

Introduction

In the field of Computational Environmental Sound Analysis—specifically within the DCASE (Detection and Classification of Acoustic Scenes and Events) community—reproducibility is a major hurdle. While many researchers release their code, a lack of standardized coding conventions means that comparing a new model against an existing baseline often requires time-consuming re-implementation. Deep learning models are sensitive to "hidden" implementation details like specific feature extraction parameters, data augmentation techniques, and scaling methods. Without a unified framework, the "barrier to entry" for students and researchers is high, and the "technical debt" of maintaining new models often slows down the overall progress of the research community.

Library Architecture and Methods

The authors introduce a modular architecture built on several core base classes that handle the typical research pipeline. This structure is designed to be extensible and backend-agnostic.

The modular architecture of DCASE-Models, showing the relationships between Dataset, FeatureExtractor, and ModelContainer classes.
  • Dataset Management: A Dataset class handles downloading, path management, and resampling. It includes predefined "folds" to ensure evaluations match published benchmarks.
  • AugmentedDataset: Simplifies data augmentation (e.g., pitch-shifting, time-stretching, white noise) using tools like pysox.
  • FeatureExtractor: Provides a unified interface for classic time-frequency representations (Spectrograms, Mel-Spectrograms) via librosa, as well as pre-trained embeddings like VGGish and OpenL3.
  • ModelContainer: This is the core abstraction layer. It standardizes how models interact with the rest of the system. While the initial release focuses on Keras, the design allows for easy integration of PyTorch or other frameworks.

Supported Datasets

The library comes with built-in support for a wide array of standard environmental audio datasets, facilitating rapid experimentation across different tasks:

  • Classification: ESC-10/50, UrbanSound8k, and TAU Urban Acoustic Scenes (2019/2020).
  • Sound Event Detection (SED): URBAN-SED, TUT Sound Events 2017, and MAVD-traffic.
  • Audio Tagging: FSDKaggle2018 and SONYC-UST.

Visualization and Model Analysis

Beyond the API, the library includes a web application (built with Dash) for visual debugging. The inclusion of a web-based proof-of-concept for visualization suggests that the authors view "understanding the model" as just as important as "training the model."

The DCASE-Models web application showing a 2D PCA projection of model embeddings alongside a Mel-spectrogram of a selected audio sample.

The ability to see a 2D PCA projection of intermediate model outputs helps identify why certain classes are being confused. This visual debugging tool allows for inspecting misclassified samples and listening to audio files directly within the browser.

Key Results and Impact

As a software-focused project, the primary results are demonstrated through the library's utility:

  • Baseline Replications: The authors successfully used the library to replicate experiments from several key papers in the field, such as SB-CNN and A-CRNN architectures.
  • Code Efficiency: A full pipeline—from dataset initialization to model evaluation—can be achieved in just a few dozen lines of readable Python code.
  • Pre-trained Models: The library provides ready-to-use implementations of several baselines, including MLP, SB-CNN, and VGGish-based models, significantly lowering the effort required for comparative studies.

Conclusion and Future Work

While currently focused on Keras/TensorFlow, future work includes expanding PyTorch support within the ModelContainer. The authors also aim to continually update the library with state-of-the-art models from annual DCASE challenges and evolve the tool to improve the explainability of how deep learning models "hear" environmental sounds.

Summary updated

BibTeX
@inproceedings{zinemanas2020dcase,
  author = {Pablo Zinemanas and Ignacio Hounie and Pablo Cancela and Frederic Font and Martín Rocamora and Xavier Serra},
  title = {{DCASE-Models: A Python Library for Computational Environmental Sound Analysis using Deep-Learning Models}},
  booktitle = {DCASE},
  year = {2020},
  url = {https://openreview.net/forum?id=QohjI8kZ0y},
}
Figure