Using the SDIF Sound Description Interchange Format for Audio Features

Axel Roebel, Juan José Burred, Carmine Emanuele Cella, Geoffroy Peeters, Axel Röbel, Diemo Schwarz

Abstract

This paper proposes a standardized methodology for storing and exchanging audio features—such as spectral centroid or Mel-Frequency Cepstral Coefficients (MFCCs)—using the Sound Description Interchange Format (SDIF). By extending this established binary format, the authors aim to eliminate data silos in Music Information Retrieval (MIR), enabling researchers to move data seamlessly between disparate extraction, training, and classification environments.

Introduction

In audio analysis, "descriptors" or "features" are mathematical summaries extracted from sound signals to represent characteristics like loudness, pitch, or timbre. Historically, the MIR community has relied on ad-hoc, headerless ASCII files or proprietary binary formats. This lack of standardization creates significant hurdles for interoperability, requiring custom "glue code" to bridge different tools. Furthermore, text-based formats are computationally inefficient and often lack the metadata necessary to interpret the data correctly (e.g., sampling rates or windowing parameters).

The SDIF Framework

To address these issues, the authors leverage the Sound Description Interchange Format (SDIF), a binary "meta-format" developed at IRCAM. SDIF is inherently flexible, organizing data into a hierarchy of Streams (representing different entities like audio channels), Frames (time-tagged snapshots), and Matrices (the actual numerical data).

Standardized Extensions for Descriptors

The paper introduces specific conventions to adapt SDIF for modern audio descriptors:

  • Dimensionality: A convention is established where matrix rows represent dimensions (e.g., frequency bands) and columns represent variations (e.g., linear vs. logarithmic scales).
  • Temporal Modeling: The format distinguishes between short-term descriptors (calculated on 40–80ms windows) and "macro-features" (e.g., mean or variance) calculated over longer "texture windows" (500ms+).
  • Semantic Metadata: To ensure long-term interpretability, the authors propose "information matrices" (prefixed with I). These store labels, thresholds, and other parameters alongside raw numerical data.
An SDIF frame (1WMN) illustrating the inclusion of both a data matrix (1SCN) and a semantic information matrix (IWMN).

Stream Relationships and Hierarchy

A critical addition is the 2IDS frame, which explicitly defines relationships between data streams. This allows for a machine-readable "paper trail," formally stating, for instance, that a specific statistical stream is derived from a particular spectral analysis stream.

A sequence of SDIF frames showing how descriptors are organized across time and multiple streams.

The authors also recommend a "One-Data-Matrix-Per-Frame" rule to simplify the logic for reading software, ensuring a clean 1-to-1 mapping between data streams and their respective descriptors.

Adoption and Impact

The primary result of this work is a robust, open-source framework supported by a mature ecosystem of libraries in C, C++, Java, Matlab, and Python. The format has seen successful integration into major audio platforms such as AudioSculpt, CLAM, Csound, and Max/MSP. This adoption demonstrates the format's utility in both high-level research and real-time creative applications.

Conclusion and Future Work

While the proposed standard provides a robust container, it does not mandate specific mathematical implementations for features, leaving that to external definitions. The authors invite the community to refine these standards via the SDIF Wiki. Future work aims to extend the format to encompass more complex pattern recognition data, including statistical models and feature transformation matrices.

Summary updated

BibTeX
@inproceedings{roebel2008using,
  author = {Axel Roebel and Juan José Burred and Carmine Emanuele Cella and Geoffroy Peeters and Axel Röbel and Diemo Schwarz},
  title = {{Using the SDIF Sound Description Interchange Format for Audio Features}},
  booktitle = {Ismir},
  year = {2008},
}
Figure