An extensible cluster-graph taxonomy for open set sound scene analysis

Helen L. Bear, Emmanouil Benetos

Abstract

Sound scene analysis—the task of identifying acoustic environments and the events within them—has traditionally relied on rigid, closed-set taxonomies. These fixed label sets often lead to data fragmentation and naming inconsistencies across different research efforts. This paper introduces an extensible cluster-graph taxonomy designed for open-set sound scene analysis. By organizing labels into a graph structure supported by external lexical databases like WordNet, the proposed framework allows for the integration of disparate datasets, the dismantling of complex labels into atomic components, and a more nuanced description of acoustic scenes through three distinct pillars: Environment, Event, and Context.

Introduction: The Problem with Closed-Set Taxonomies

Most current research in sound scene analysis treats the task as a "closed-set" problem. Models are typically trained to recognize a fixed list of labels, such as "office," "bus," or "park." However, real-world acoustic environments are inherently "open" and multi-layered. A single audio clip might contain a "car horn" (event) in a "busy street" (environment) during a "traffic jam" (context).

The lack of a standardized taxonomy has led to a "taxonomy of everything," where researchers frequently invent new labels for each dataset. This results in significant overlap and redundancy—for example, "car" versus "vehicle"—making it difficult to reuse data across studies. To address this, we propose a labeling mechanism that is extensible, non-redundant, and capable of handling multi-perspective descriptions.

Proposed Methodology

The core of our approach is an extensible cluster-graph taxonomy. Unlike traditional hierarchical trees where a label belongs to exactly one parent, a graph structure allows for complex relationships where labels can be shared across different subsets.

The Three Pillars

The taxonomy categorizes labels into three distinct types to capture the full complexity of a scene:

  1. Environment: Tangible, physical descriptions (e.g., indoors, building).
  2. Event: Specific sound-emitting actions or objects (e.g., speech, keyboard).
  3. Context: Human perceptual or social descriptions (e.g., a meeting, a party).
Conceptual framework of the cluster-graph taxonomy.

Label Processing and Integrity

To maintain a clean and usable superset of labels ($T$), the framework incorporates several key processes:

  • Thesaurus Integration: We utilize WordNet to manage synonyms and homographs. This ensures that variations like "cough" and "coughing" are mapped to the same leaf node, preventing duplication.
  • Compound Label Dismantling: Complex labels are broken down into atomic parts. For instance, "washing dishes" is dismantled into "washing" (action) and "dishes" (object), allowing models to learn these components independently.
  • Graph Cuts: Researchers can take specific "cuts" or subsets of the graph for specialized tasks while maintaining the integrity of the global label set.
Workflow for label extension and integrity maintenance.

Results and Unified Datasets

To demonstrate the framework's utility, we applied it to the label sets from the DCASE (Detection and Classification of Acoustic Scenes and Events) challenges from 2013 to 2018. Historically, these datasets have used inconsistent labeling schemes.

By applying our framework, we successfully merged these fragmented labels into a unified system:

  • Unified Event Set ($T_{events}$): Hundreds of inconsistent labels were reduced to 78 unique, atomic events.
  • Unified Scene Set ($T_{scenes}$): Scene labels were consolidated into a streamlined set of 26 environments and contexts.

This consolidation proves that years of fragmented audio data can be reorganized into a single, coherent system, significantly boosting the potential for large-scale deep learning training.

Discussion and Future Work

The transition from rigid trees to flexible graphs acknowledges that sounds do not exist in silos. By standardizing labels, we enable the "amalgamation" of datasets, turning small, incompatible data collections into massive, unified resources.

While the framework requires a small overhead for initial annotation, the long-term benefits for data reusability are substantial. Future work will focus on mapping this taxonomy to Google’s AudioSet and implementing "weighted relations" within the graph—for example, increasing the probability of detecting "seagulls" if the environment is identified as a "beach." The ultimate success of this initiative depends on community adoption and the establishment of a central repository for dataset links and label definitions.

Summary updated

Abstract

We present a new extensible and divisible taxonomy for open set sound scene analysis. This new model allows complex scene analysis with tangible descriptors and perception labels. Its novel structure is a cluster graph such that each cluster (or subset) can stand alone for targeted analyses such as office sound event detection, whilst maintaining integrity over the whole graph (superset) of labels. The key design benefit is its extensibility as new labels are needed during new data capture. Furthermore, datasets which use the same taxonomy are easily augmented, saving future data collection effort. We balance the details needed for complex scene analysis with avoiding 'the taxonomy of everything' with our framework to ensure no duplicity in the superset of labels and demonstrate this with DCASE challenge classifications.
BibTeX
@inproceedings{bear2018extensible,
  author = {Helen L. Bear and Emmanouil Benetos},
  title = {{An extensible cluster-graph taxonomy for open set sound scene analysis}},
  booktitle = {DCASE},
  year = {2018},
  url = {https://openreview.net/forum?id=j6zNICmvBM},
}
Figure