Abstract
The goal of acoustic (or sound) events detection (AED or SED) is to predict the temporal position of target events in given audio segments. This task plays a significant role in safety monitoring, acoustic early warning and other scenarios. However, the deficiency of data and diversity of acoustic event sources make the AED task a tough issue, especially for prevalent data-driven methods. In this paper, we start by analyzing acoustic events according to their time-frequency domain properties, showing that different acoustic events have different time-frequency scale characteristics. Inspired by the analysis, we propose an adaptive multi-scale detection (AdaMD) method. By taking advantage of the hourglass neural network and gated recurrent unit (GRU) module, our AdaMD produces multiple predictions at different temporal and frequency resolutions. An adaptive training algorithm is subsequently adopted to combine multi-scale predictions to enhance its overall capability. Experimental results on Detection and Classification of Acoustic Scenes and Events 2017 (DCASE 2017) Task 2, DCASE 2016 Task 3 and DCASE 2017 Task 3 demonstrate that the AdaMD outperforms published state-of-the-art competitors in terms of the metrics of event error rate (ER) and F1-score. The verification experiment on our collected factory mechanical dataset also proves the noise-resistant capability of the AdaMD, providing the possibility for it to be deployed in the complex environment.
Introduction
Acoustic Event Detection (AED) is a cornerstone of modern safety monitoring, smart home technology, and industrial diagnostics. The primary objective is to pinpoint exactly when a specific sound occurs within an audio stream. However, real-world audio environments present significant challenges. Target events, such as a window breaking or a gunshot, are often rare compared to hours of background noise. Furthermore, acoustic events exhibit extreme diversity in their time-frequency characteristics; a gunshot is a momentary impulse, whereas a baby’s cry or a scream can linger for several seconds with complex harmonic structures.
Existing methods often rely on fixed-window analysis, which fails to capture this variety. A window long enough to provide context for a scream may be too coarse to detect a short click. This paper introduces the Adaptive Multi-scale Detection (AdaMD) framework, which treats audio detection as a multi-scale problem analogous to keypoint detection in computer vision.
Methods
The AdaMD architecture integrates techniques from computer vision and sequence modeling to handle varying event scales. The core components include:
The Hourglass Module
Borrowed from human pose estimation, the hourglass structure uses a series of convolutional layers to downsample the input feature map into a low-dimensional representation and then upsamples it back to the original resolution. This symmetric structure allows the network to extract and preserve features at multiple resolutions simultaneously.
Bi-directional GRUs
At each scale of the hourglass module, the framework attaches a Bi-directional Gated Recurrent Unit (GRU). This allows the model to process temporal information both forwards and backwards, effectively capturing the temporal evolution and "tails" of sounds—whether they vanish slowly like a doorbell or end abruptly like a machine failure.
Adaptive Fusion Algorithm
Rather than simple averaging, AdaMD employs an adaptive training algorithm inspired by AdaBoost. The model learns to weight the predictions from different scales. During training, it dynamically adjusts these weights to favor the branches that are most effective for a specific sound type, allowing different scales to specialize in different acoustic characteristics.
Experimental Results
The model was evaluated on several benchmark datasets, including DCASE 2017 Task 2 (rare events), DCASE 2016/2017 Task 3 (polyphonic events), and a custom Factory Mechanical Dataset.
- Rare Event Detection: On DCASE 2017 Task 2, AdaMD achieved an Error Rate (ER) of 0.08 and an F1-score of 94.7%, significantly outperforming the competition's first-place winner (ER 0.13, F1 93.1%).
- Polyphonic Events: In the DCASE 2016 Task 3 office environment challenge, the model achieved an ER of 0.7821, surpassing the previous state-of-the-art (SOTA) of 0.8051.
- Industrial Robustness: In experiments with the factory dataset, AdaMD demonstrated superior resistance to additive Gaussian noise compared to standard CRNN baselines, highlighting its potential for deployment in high-noise industrial environments.
Conclusion
The AdaMD framework successfully addresses the challenges of acoustic event diversity by utilizing a multi-scale hourglass architecture and adaptive fusion. While the model shows state-of-the-art performance, its increased parameter count makes it susceptible to overfitting on clean data. Future work will focus on reducing model complexity and improving generalization across mismatched acoustic conditions. This multi-scale approach holds promise for broader audio tasks, including source separation and speech enhancement.
Summary updated