Abstract
This paper presents a sophisticated system for Sound Event Detection (SED) in domestic environments, developed for the DCASE 2020 Task 4 challenge. Building on a teacher-student "guided learning" framework, the authors introduce Multi-Branch Learning (MBL) to improve feature representation by utilizing diverse pooling strategies. The system effectively integrates weakly labeled, unlabeled, and synthetic data, while also incorporating Sound Separation (SS) to enhance performance in noisy acoustic scenes. Experimental results demonstrate that MBL reduces overfitting and that fusing sound-separated models into an ensemble provides a robust performance boost.
Introduction: Addressing Data Scarcity and Signal Interference
Sound Event Detection (SED) involves identifying the presence and temporal boundaries of specific sounds. This task is inherently difficult due to the high cost of obtaining "strongly labeled" data (precise timestamps). Most available datasets consist of "weakly labeled" clips (tags only) or large volumes of unlabeled audio. Additionally, domestic environments are characterized by overlapping sounds and background noise, which can significantly degrade detection accuracy.
The authors aim to overcome these hurdles by:
- Maximizing Data Utility: Effectively combining small amounts of synthetic and weakly labeled data with massive unlabeled datasets.
- Mitigating Interference: Using Sound Separation (SS) to "clean" audio signals before detection, thereby improving performance in complex acoustic environments.
Methodology: Guided and Multi-Branch Learning
The system architecture is based on a "guided learning" approach, where a Professional Teacher (PT-model) with a large receptive field generates pseudo-labels for unlabeled data. These labels then guide the training of a Promising Student (PS-model), which is optimized for frame-level detection.
Multi-Branch Learning (MBL)
A core innovation is the Multi-Branch Learning structure within the PS-model. Multiple branches share a single feature encoder but employ different pooling strategies to capture varied signal characteristics:
- Attention-based Pooling (ATP): Learns to focus on the most relevant parts of the audio signal.
- Global Max/Average Pooling (GMP/GAP): Provides statistical summaries of features.
- Sound Event Detection Branch (SEDB): Leverages strongly labeled synthetic data to predict instance-level probabilities.
By training the encoder to satisfy multiple pooling objectives, the model learns a more generalized feature space, which helps prevent overfitting to specific data subsets.
Sound Separation Integration
To further improve robustness, the authors integrated a Sound Separation (SS) module. Audio is first separated into distinct tracks (e.g., isolating speech from background noise), and dedicated SED models are trained on this processed audio. While these SS-SED models are less accurate on their own, they provide a unique perspective that enhances the overall performance when fused into a final ensemble.
Key Results and Insights
The system was evaluated on the DCASE 2020 Task 4 dataset using the event-based macro F1 score.
Performance Gains
- Pooling Synergy: Adding the MBL structure (specifically the I-GAP branch) improved the validation F1 score from 0.444 to 0.451. This confirms that diverse pooling methods capture complementary information.
- Ensemble Performance: The best ensemble achieved an F1 score of 44.6% for standard SED and 44.7% when integrated with sound separation. The inclusion of sound-separated models helped the system generalize better across different acoustic events.
- Batch Balancing: The authors found that MBL is sensitive to the ratio of unlabeled data. Carefully balancing labeled and unlabeled data in each mini-batch was crucial to prevent noisy pseudo-labels from degrading performance.
Conclusion and Future Directions
The combination of multi-branch learning and sound separation offers a powerful framework for SED in complex, real-world environments. While current sound separation systems can introduce artifacts that limit the performance of individual SS-SED models, their contribution to ensemble diversity is significant. Future research will focus on joint, end-to-end training of separation and detection modules to further refine these systems.
Summary updated