An extensible cluster-graph taxonomy for open set sound scene analysis

Helen L. Bear, Emmanouil Benetos

Abstract

We present a new extensible and divisible taxonomy for open set sound scene analysis. This new model allows complex scene analysis with tangible descriptors and perception labels. Its novel structure is a cluster graph such that each cluster (or subset) can stand alone for targeted analyses such as office sound event detection, whilst maintaining integrity over the whole graph (superset) of labels. The key design benefit is its extensibility as new labels are needed during new data capture. Furthermore, datasets which use the same taxonomy are easily augmented, saving future data collection effort. We balance the details needed for complex scene analysis with avoiding 'the taxonomy of everything' with our framework to ensure no duplicity in the superset of labels and demonstrate this with DCASE challenge classifications.

Introduction

In computer audition, identifying a "sound scene" (e.g., a busy street or a quiet office) has traditionally been treated as a closed-set problem. Models are typically trained on a fixed number of labels, failing to account for the "open-set" nature of the real world. Current sound scene datasets are often siloed; inconsistent labeling (e.g., "urban park" vs. "outdoor playground") makes data merging difficult.

The authors identify four major gaps in current labeling practices:

  1. Rigidity: Taxonomies do not grow easily as new scenes are discovered.
  2. Duplication: Synonyms and homographs cause label bloat and data fragmentation.
  3. Complexity: Single labels fail to capture polyphonic events and human perceptual context.
  4. Hierarchy Limitations: Rigid tree structures cannot represent sounds belonging to multiple categories simultaneously.

The Cluster-Graph Taxonomy

The core innovation of this research is a cluster-graph structure where labels are nodes. This moves away from strict hierarchies to allow for multi-perspective descriptors, acknowledging that a human perceives a scene differently than a physical sensor might.

The three-tier approach to scene definition, intersecting Environments, Events, and Context.

The framework employs a three-tier approach to categorize sounds:

  • Environments: Tangible physical descriptions (e.g., indoors, office).
  • Events: Specific sound-emitting objects or actions (e.g., keyboard clicks, speech).
  • Context: Human perceptual descriptions (e.g., a meeting, a party).

By modeling these as overlapping subsets, the taxonomy can "slice" out specific clusters for niche tasks while remaining compatible with a global superset of sounds.

Label Processing and Extensibility

To maintain a clean superset of labels ($T$), the framework integrates lexical databases like WordNet. This ensures that new data capture does not lead to "the taxonomy of everything" through redundant labeling.

The logic flow for integrating new labels into the taxonomy, including synonym and compound label handling.

The process involves several key steps to ensure data integrity:

  • Thesaurus Integration: Checking if a proposed label is a synonym of an existing one using WordNet.
  • Dismantling Compounds: Breaking complex labels into atomic parts (e.g., "washing dishes" becomes "washing" and "dishes"), allowing labels to be reused across different scenes.
  • Homograph Resolution: Distinguishing between identical words with different meanings using integer descriptors.

Validation and Results

The framework was validated by "initializing" it with label sets from the DCASE (Detection and Classification of Acoustic Scenes and Events) challenges spanning 2013 to 2018. This process consolidated hundreds of fragmented labels into a unified, extensible superset.

Key results include:

  • $T_{events}$: A cleaned list of over 80 unique event labels ranging from "door slam" to "electric shaver."
  • $T_{scenes}$: A consolidated list of 25+ scene labels (e.g., "busy street," "library").
  • Contextual Subsets: The identification of minimal "context" subsets that prove even small amounts of perceptual data can be structured within the graph.

Conclusion

By moving from trees to graphs and adopting atomic labeling, this framework significantly increases data reusability and enables machines to "cross-classify" scenes based on human-like expectations. Future work will focus on scaling the taxonomy to massive datasets like AudioSet and incorporating weighted edges to represent the statistical likelihood of specific events occurring within certain environments.

Summary updated

Abstract

We present a new extensible and divisible taxonomy for open set sound scene analysis. This new model allows complex scene analysis with tangible descriptors and perception labels. Its novel structure is a cluster graph such that each cluster (or subset) can stand alone for targeted analyses such as office sound event detection, whilst maintaining integrity over the whole graph (superset) of labels. The key design benefit is its extensibility as new labels are needed during new data capture. Furthermore, datasets which use the same taxonomy are easily augmented, saving future data collection effort. We balance the details needed for complex scene analysis with avoiding 'the taxonomy of everything' with our framework to ensure no duplicity in the superset of labels and demonstrate this with DCASE challenge classifications.
BibTeX
@inproceedings{bear2018extensible,
  author = {Helen L. Bear and Emmanouil Benetos},
  title = {{An extensible cluster-graph taxonomy for open set sound scene analysis}},
  booktitle = {DCASE},
  year = {2018},
  url = {https://openreview.net/forum?id=j6zNICmvBM},
}
Figure