Abstract
This paper presents a high-performance hybrid system for Sound Event Detection (SED) developed for the DCASE 2022 Task 4 challenge. The system integrates an event-wise Sound Event Detection Transformer (SEDT) with a frame-wise CNN model (MLFL-CNN). By leveraging self-supervised pre-training, teacher-student learning with asymmetric augmentation, and a class-specific fusion strategy, the proposed approach effectively utilizes weakly labeled and unlabeled data. Experimental results demonstrate significant performance gains, achieving high Polyphonic Sound Detection Scores (PSDS) across different evaluation scenarios.
Introduction
Sound Event Detection (SED) involves identifying both the category and the precise temporal boundaries (onset and offset) of sound events in audio clips. The DCASE 2022 Task 4 challenge is particularly demanding due to its reliance on weakly labeled and unlabeled real-world data alongside perfectly labeled synthetic data. Traditional frame-wise models often struggle to capture the global temporal structure of sound events. This work addresses these challenges by proposing a hybrid system that fuses event-wise and frame-wise modeling philosophies to capture both local precision and global context.
Methodology
Sound Event Detection Transformer (SEDT)
The SEDT is an event-wise model that treats sound events as distinct objects. It utilizes a CNN backbone to extract features from spectrograms, which are then processed by a Transformer encoder-decoder to directly predict a set of candidate events.
To enhance the SEDT's performance, several novel techniques were implemented:
- Self-supervised Pre-training (SP-SEDT): The model learns to predict the location of random spectrogram crops, improving feature representation before supervised training.
- Teacher-Student Learning: An "online teacher" model, updated via Exponential Moving Average (EMA), generates pseudo-labels for unlabeled data. The system employs asymmetric augmentation, where the teacher processes "easy" audio while the student is challenged with "hard" augmented versions.
MLFL-CNN and Fusion Strategy
The second component is the MLFL-CNN, a frame-wise model based on Metric Learning and Focal Loss. Metric learning is used to bridge the domain gap between synthetic and real audio, while focal loss addresses class imbalance issues common in environmental sound datasets.
The final output is generated through a class-specific fusion method. Rather than simple averaging, weights are assigned to each model based on their performance for specific sound categories (e.g., "Speech" vs. "Vacuum cleaner") on the development set. This allows the system to leverage the specific architectural strengths of each model for different sound types.
Experiments and Results
The system was evaluated using the DCASE 2022 Task 4 framework, incorporating external data from AudioSet and the SINS dataset. Performance was measured using the Polyphonic Sound Detection Score (PSDS), where PSDS1 rewards temporal precision and PSDS2 rewards class macro-discovery.
The hybrid system outperformed individual models significantly. System 1 (optimized for precision) achieved a PSDS1 of 0.449 and PSDS2 of 0.645. System 2 (optimized for detection) achieved a massive PSDS2 of 0.816.
Ablation Study
An ablation study on the SEDT model highlights the contribution of various semi-supervised learning techniques to the overall performance.
As shown in the table, the combination of Mixup, Focal Loss, and Asymmetric Augmentation significantly boosts performance. For instance, the inclusion of Mixup (comparing the first and last rows) raised the PSDS1 from 0.372 to 0.388, validating the effectiveness of the proposed training pipeline.
Conclusion
The proposed hybrid system successfully combines the complementary strengths of event-wise Transformers and frame-wise CNNs. The integration of advanced semi-supervised learning techniques and class-specific fusion allows the model to excel in both temporal precision and event discovery. Future work will focus on improving the computational efficiency of the ensemble and reducing its reliance on synthetic data quality.
Summary updated