Soundata: Reproducible use of audio datasets

Magdalena Fuentes, Genís Plaja-Roglans, Guillem Cortès-Sebastià, Tanmay Khandelwal, Marius Miron, Xavier Serra, Juan Pablo Bello, Justin Salamon, Guillem CortèsJ. Open Source Softw 2024 · DOI · DBLP

Introduction

Soundata is an open-source Python library designed to standardize and simplify the way researchers interact with audio datasets. In fields such as bioacoustics, urban noise monitoring, and environmental soundscapes, data handling is a significant bottleneck. Researchers often spend considerable time writing custom scripts to download, parse, and load specific datasets. This "DIY" approach leads to inefficiency and a reproducibility crisis, where small discrepancies in data loading or versioning make it nearly impossible to fairly compare different algorithms.

While tools like mirdata exist for Music Information Research (MIR), the authors recognized that other audio communities (e.g., those involved in DCASE challenges) required a specialized tool focusing on non-music audio and its unique annotation requirements.

Modular Architecture

Soundata is built on a modular architecture designed for ease of use and scalability. Its design is centered around three main components:

  • Core Module: Implements generic functions used across all datasets, such as the base Dataset class.
  • Utils Module: Handles heavy lifting like downloading files, validating data integrity via MD5 hashes, and converting annotations to the JAMS (JSON Annotated Music Specification) format.
  • Loaders: Dataset-specific scripts containing the logic for parsing unique folder structures and metadata.
The modular architecture of Soundata, comprising Core, Utils, and Loaders.

Standardized Data Handling

The library follows a "plug-and-play" philosophy. A user can initialize a dataset, download it, and validate that their local copy matches the "canonical" version in just a few lines of code. One of Soundata's most critical features is the dataset.validate() function, which checks local files against a remote index to prevent "silent errors" caused by missing or corrupted files.

To ensure compatibility with common evaluation libraries like sed_eval, Soundata introduces standardized annotation types to handle the diverse needs of the audio research community.

Standardized annotation types supported by Soundata: Tags, Events, and Spatial Events.

Key Insights and Results

Soundata provides a standardized API regardless of the dataset, ensuring that commands to initialize, download, and load data remain consistent. It is framework-agnostic, integrating easily into modern deep learning pipelines like TensorFlow or PyTorch. By supporting conversion to the JAMS format, Soundata ensures that audio annotations are stored in a rich, human-readable, and standardized JSON structure.

The library is explicitly designed to support datasets used by the DCASE (Detection and Classification of Acoustic Scenes and Events) community, covering bioacoustics, environmental sounds, and spatial audio.

Future Directions

While Soundata currently supports primary annotation types like Tags, Events, and Spatial Events, the library is designed to evolve. Future work includes adding support for more complex types like audio captioning and expanding dataset coverage through community-driven contributions. Tighter integration with ecosystems like Hugging Face's datasets is also a likely path forward.

Summary updated

BibTeX
@article{fuentes2024soundata,
  author = {Magdalena Fuentes and Genís Plaja-Roglans and Guillem Cortès-Sebastià and Tanmay Khandelwal and Marius Miron and Xavier Serra and Juan Pablo Bello and Justin Salamon and Guillem Cortès},
  title = {{Soundata: Reproducible use of audio datasets}},
  journal = {J. Open Source Softw.},
  year = {2024},
  doi = {10.21105/joss.06634},
}
Figure