Introduction
Polyphonic Sound Event Detection (PSED) involves identifying the presence and temporal boundaries—onset and offset—of multiple overlapping sound events in audio recordings. A pervasive challenge in PSED is the extreme class imbalance between "active" frames (containing target sounds) and "inactive" frames (silence or background noise). In typical domestic datasets, inactive frames vastly outnumber active ones.
Standard training using Binary Cross-Entropy (BCE) often results in models that over-optimize for the majority background class. This leads to high confidence in background detection at the expense of precise temporal localization for target events. While alternatives like Asymmetric Focal Loss (AFL) address imbalance by weighting classes, they can sometimes "blur" impulsive sounds, failing to distinguish between rapid, repetitive events. This paper introduces Confidence Regularized Entropy (CRE) to address these limitations.
Methodology
The authors propose a novel loss function, Confidence Regularized Entropy (CRE), designed to mitigate the negative effects of class imbalance by preventing over-optimization on "easy" samples.
Confidence Regularized Entropy (CRE)
The core principle of CRE is to stop the gradient flow for samples once the model achieves a predefined level of confidence. By setting a lower boundary (e.g., 0.01) for inactive frames and an upper boundary (e.g., 0.99) for active frames, the model is discouraged from further refining predictions that are already "good enough." This forces the training process to focus on ambiguous frames, particularly those at the boundaries of sound events, leading to sharper onset and offset detection.
Axis-Wise Attention Module (AWAM)
To complement the CRE loss, the researchers developed the Axis-Wise Attention Module (AWAM). This module enhances the feature extraction process by calculating attention scores independently across the frequency (F) and time (T) axes. These scores are then used to weight the feature maps, allowing the model to better focus on relevant acoustic patterns in complex polyphonic environments.
Semi-Supervised Learning Framework
The proposed system is implemented within a Mean Teacher framework. This approach allows the model to leverage a combination of a small amount of strongly labeled data, weakly labeled data, and a large pool of unlabeled data, which is common in real-world sound detection scenarios.
Experimental Setup
The methodology was evaluated using the DESED (Domestic Environment Sound Event Detection) dataset, a benchmark for the DCASE challenges. The dataset includes:
- Synthetic Data: 10,000 clips with precise "strong" annotations.
- Real Data: A mixture of strongly labeled, weakly labeled, and unlabeled recordings across 10 domestic categories such as "Alarm/bell ringing," "Cat," and "Speech."
Results and Discussion
The performance of CRE was compared against standard BCE and AFL across several key metrics:
- Event-F1 Score: CRE consistently outperformed both baselines. The best-performing configuration (CRNN + AWAM) achieved an Event-F1 of 51.11, significantly higher than BCE (49.17) and AFL (45.16).
- Localization (PSDS1): CRE showed a clear advantage in the Polyphonic Sound Detection Score (PSDS) scenario 1, which prioritizes temporal accuracy. This confirms that CRE is highly effective for tasks requiring precise timing.
- Detection (PSDS2): In contrast, AFL performed better in PSDS scenario 2, which emphasizes overall detection sensitivity over timing. This suggests a trade-off: AFL is better at ensuring a sound is not missed, while CRE is superior at determining exactly when it occurs.
Conclusion
Confidence Regularized Entropy (CRE) provides a targeted solution to the class imbalance problem in PSED. By regularizing the model's confidence, it prevents overfitting on background noise and improves the precision of sound event boundaries. While AFL remains a strong choice for general detection sensitivity, CRE is the preferred objective for applications requiring high temporal resolution and precise localization.
Summary updated