Abstract
Acoustic Scene Classification (ASC) and Sound Event Detection (SED) are two separate tasks in the field of computational sound scene analysis. In this work, we present a new dataset with both sound scene and sound event labels and use this to demonstrate a novel method for jointly classifying sound scenes and recognizing sound events. We show that by taking a joint approach, learning is more efficient and whilst improvements are still needed for sound event detection, SED results are robust in a dataset where the sample distribution is skewed towards sound scenes.
Introduction
In computational sound analysis, researchers typically distinguish between "Where are we?" (Acoustic Scene Classification, or ASC) and "What just happened?" (Sound Event Detection, or SED). While humans naturally use environmental context to identify sounds—expecting a "checkout beep" in a supermarket rather than a forest—AI models have traditionally treated these as separate silos. This separation is inefficient and ignores the natural synergy between a scene and its constituent events. This paper proposes a holistic approach, training a single model to perform both tasks simultaneously to create more efficient and biologically inspired AI systems.
Methods
The researchers developed a joint recognition model based on a Convolutional Recurrent Neural Network (CRNN).
Architecture
The model processes 128-band log-mel spectrograms using two stacked channels: a standard spectrogram for event detection and a temporally smoothed version for scene classification. The architecture consists of three convolutional layers for spatial feature extraction, followed by an LSTM layer to capture temporal dependencies. The final layer uses a Sigmoid activation to output a 43-element vector representing 10 scene classes and 32 event classes.
Dataset Creation
As no standard dataset existed for joint tasks, the authors synthesized a new one using the Scaper library. It includes 3,000 polyphonic recordings (30 seconds each) with 10 scene classes from DCASE 2013 and 32 event classes from DCASE 2016 and Freesound.org. Events were grouped logically (e.g., "birdsong" in "park") to maintain realism.
Results and Analysis
The joint model was compared against standalone "specialist" models. While ASC performance remained robust, SED performance declined significantly in the joint setup. This is largely attributed to data imbalance: scene labels are active for the entire 30-second clip, whereas events may last only a fraction of a second.
| Model | ASC Accuracy (Majority Vote) | SED F1 Score | SED Error Rate |
|---|---|---|---|
| Separate Models | 99% | 28.86% | 0.86 |
| Joint Model | 98% | 13.73% | 1.00 |
The joint model reached peak ASC performance in roughly half the training epochs required by a standalone model, suggesting that event-level information accelerates scene understanding. However, the model struggled with short-duration sounds or those mimicking background noise, likely due to the "skew" where scene labels dominate the training signal.
Conclusion
This work serves as a proof-of-concept for joint sound analysis. While joint recognition is more efficient and biologically plausible, the "forest" (the scene) currently tends to overshadow the "trees" (the events) in unified models. Future work will focus on loss function weighting and architectural tweaks to better capture short-duration features and counteract the data imbalance between scenes and events.
Summary updated