Abstract
Acoustic Scene Classification (ASC) and Sound Event Detection (SED) are two separate tasks in the field of computational sound scene analysis. In this work, we present a new dataset with both sound scene and sound event labels and use this to demonstrate a novel method for jointly classifying sound scenes and recognizing sound events. We show that by taking a joint approach, learning is more efficient and whilst improvements are still needed for sound event detection, SED results are robust in a dataset where the sample distribution is skewed towards sound scenes.
Introduction
In computational audio analysis, researchers typically treat Acoustic Scene Classification (ASC)—identifying environments like a "park" or "office"—and Sound Event Detection (SED)—spotting specific sounds like "birdsong" or "keyboard typing"—as separate silos. However, these tasks are inherently linked: a "supermarket" scene increases the likelihood of "checkout beeps," and hearing those beeps confirms the scene. This paper explores whether a single "joint" model can learn both tasks concurrently, potentially leading to more efficient and context-aware AI.
Methods
The proposed joint recognition system utilizes a Convolutional Recurrent Neural Network (CRNN) architecture designed for multi-label classification. Unlike standard one-hot classification, the model employs a Sigmoid activation layer and binary cross-entropy loss, allowing it to predict a scene and multiple overlapping (polyphonic) sound events simultaneously.
The model processes dual-channel features consisting of two versions of log-mel spectrograms. One channel is standard, optimized for the precise temporal resolution required for SED, while the second is temporally smoothed to capture the broader acoustic characteristics needed for ASC. The model is trained to predict all 43 labels (10 scenes and 32 events) in a single pass.
Dataset Construction
Since no public dataset provided labels for both scenes and specific events, the authors synthesized a new one using the Scaper library. Backgrounds were sourced from the DCASE 2013 ASC challenge, and foreground events were taken from DCASE 2016 and FreeSound.org. The synthesis process ensured realistic soundscapes, such as restricting "birdsong" to "park" scenes. The final dataset includes 3,000 30-second audio files.
Results and Discussion
The joint model demonstrated significant efficiency gains, reaching peak performance in roughly half the training epochs required by a standalone ASC model. This suggests that event-level data provides useful cues that help the model understand the broader scene more quickly.
In terms of accuracy, the joint model achieved 98% for ASC, nearly matching the 99% of specialized models. However, a performance trade-off was observed in SED. The standalone SED model achieved an F1 score of 28.86%, which dropped to 13.73% in the joint model.
This drop is largely attributed to "data skew." In a 30-second clip, the scene label is present for all 1,292 frames, whereas a sound event like a "door slam" might only span 5 frames. This imbalance biases the model toward the persistent scene labels, making it harder to detect sporadic, short-duration events.
Conclusion
While the joint approach offers computational efficiency and high scene classification accuracy, balancing the performance of sound event detection remains a challenge. Future work will focus on optimizing training schedules to prevent the ASC task from overpowering SED, refining loss functions to handle class imbalance, and exploring feature inputs that better highlight distinct sound events against background noise.
Summary updated