Abstract
Target Sound Detection (TSD) is often hindered by the high cost of frame-level annotations. This paper proposes a mixed supervised learning framework that bridges the gap between expensive strong labels and abundant weak labels. Utilizing a "two-student" iterative learning strategy involving frame-level knowledge distillation and pseudo-supervised training, the framework achieves an 8% improvement in event-based F-scores over weakly supervised baselines, approaching the performance of fully supervised models while significantly reducing annotation labor.
Introduction
Target Sound Detection (TSD) involves identifying and localizing specific sound events within complex audio environments, guided by a reference clip. A major challenge in this field is the "supervision gap." Strongly supervised models require precise frame-level labels (start and end times), which are labor-intensive to produce. Conversely, weakly supervised models use clip-level labels, which are easier to obtain but often result in "boundary blindness," where the model fails to identify the exact onset and offset of a sound.
The Mixed Supervised Learning Framework
The proposed framework employs two "student" models that iteratively teach each other to leverage the strengths of different data types:
- The Strong Student ($f\_student$): Initially trained on a small, fully-annotated source dataset, this model excels at identifying temporal boundaries but lacks knowledge of the target domain's specific sounds.
- The Weak Student ($w\_student$): Trained on a large, weakly-annotated target dataset, it recognizes target sounds but lacks temporal precision.
The models interact through a mutual helping mechanism:
- Knowledge Distillation (KD): The Strong Student transfers temporal boundary information to the Weak Student to improve its localization.
- Pseudo-Supervised Training: The Weak Student generates "pseudo-labels" for the target domain, which the Strong Student uses for fine-tuning.
- Adversarial Training: A discriminator ensures the feature extractor learns domain-invariant representations, mitigating domain mismatch between source and target datasets.
Key Insights
Boundary Blindness and Knowledge Distillation
Weakly supervised models typically identify the most distinct "peak" of an audio event but struggle with the edges. Knowledge distillation from a strongly-trained model helps "sharpen" these boundaries, allowing the model to capture the full duration of the event.
Robustness to Label Noise
A critical finding of this research is that TSD models are remarkably resilient to errors in frame-level labels. Performance remains stable even with error rates as high as 35%, justifying the use of noisy pseudo-labels in the iterative training process.
Results and Conclusion
The Mixed Supervision (MS) method achieved an event-based F-score of 47.56% on the L-TSD test set, a significant increase from the 39.07% achieved by standard weak supervision. This performance closely rivals the 50.4% achieved by full strong supervision, demonstrating that the framework effectively leverages weak labels to reduce annotation costs without sacrificing significant accuracy. Future work will explore applying this mixed-supervision logic to general Sound Event Detection (SED) and computer vision tasks.
Summary updated