Improved Student Model Training for Acoustic Event Detection Models

Anthea Cheung, Qingming Tang, Chieh-Chi Kao, Ming Sun, Chao WangDCASE 2021 · OpenReview

Abstract

Acoustic Event Detection (AED) involves identifying the type and temporal boundaries of sound events. While state-of-the-art performance is often achieved by large ensemble models, these are impractical for real-time deployment on resource-constrained devices. This paper introduces a training pipeline for a compact three-layer LSTM student model that matches the performance of massive ensembles. By integrating curriculum learning, custom SpecAugment, event-based loss masking, and strong/weak pseudolabels, the researchers achieved a state-of-the-art F1-score of 42.7% on the DCASE 2019 Task 4 benchmark, representing an 8% absolute improvement over standard knowledge distillation baselines.

Introduction

The primary challenge in Acoustic Event Detection (AED) is balancing model accuracy with computational efficiency. Top-tier models typically rely on complex Convolutional Recurrent Neural Networks (CRNNs) or large ensembles, which are unsuitable for mobile or smart home applications. To address this, researchers from Amazon Alexa employed Knowledge Distillation (KD) to train a "shallow" student model. The goal was to minimize the performance gap between a large teacher model and a tiny, three-layer LSTM student.

Methodology

The proposed training recipe for the student model incorporates four specialized techniques designed to enhance learning from unlabeled data:

  1. Curriculum Learning: Unlabeled samples are ranked by difficulty based on the discrepancy between teacher and student predictions. Training begins with "easier" samples, where predictions are most similar, and gradually introduces more complex data.
  2. Custom SpecAugment: Unlike standard SpecAugment, which uses fixed masking blocks, this custom approach scales time masks according to the median duration of specific sound events. This ensures that short events (e.g., a plate clink) are not entirely obliterated by the augmentation.
  3. Event Masking: To combat the sparsity of AED datasets, the loss function is modified to focus on frames where sound events occur, preventing the model from over-optimizing for background silence.
  4. Strong and Weak Pseudolabels: The student model is supervised by both clip-level (weak) and frame-level (strong) labels generated by the teacher, providing a dense training signal.
Ablation study of SpecAugment and Masking strategies.

Experimental Setup

The methodology was evaluated using the DCASE 2019 Task 4 dataset, a standard benchmark for domestic acoustic environments. The dataset contains a mix of strongly labeled synthetic clips, weakly labeled real recordings, and a large pool of unlabeled real recordings. It encompasses 10 categories, ranging from transient sounds like "Dishes" to sustained sounds like "Vacuum cleaner."

Results and Analysis

The optimized student model achieved an event-based F1-score of 42.7%, matching the performance of the leading ensemble model from the DCASE 2019 challenge. This represents a significant jump from the 34.7% F1-score achieved by a baseline student model using standard distillation.

Comparison of curriculum learning strategies and the baseline performance.

Key Insights

  • Temporal Scaling: Tailoring data augmentation to event duration proved critical. Standard masking can inadvertently remove entire short-duration events, whereas Custom SpecAugment preserves the essential characteristics of the sound.
  • Curriculum Benefits: Starting with "easier" samples stabilized early training. Conversely, starting with "harder" samples yielded no significant benefit, as shown in the comparison of curriculum strategies.
  • Loss Focus: Event masking significantly outperformed segment masking (which includes a temporal buffer) and no masking, confirming that focusing the loss on active event regions is superior for training shallow models.
  • Pseudolabel Density: The inclusion of strong (frame-level) pseudolabels was vital; using only weak labels resulted in a drop in F1-score to 28.6%.

Conclusion

This research demonstrates that with a carefully designed training recipe, shallow LSTM models can achieve parity with much larger architectures in AED tasks. The combination of duration-aware augmentation and focused loss masking allows for high-performance sound detection on edge devices. Future work may explore "online" distillation techniques and further refinement of time-masking strategies for even more diverse acoustic environments.

Summary updated

BibTeX
@inproceedings{cheung2021improved,
  author = {Anthea Cheung and Qingming Tang and Chieh-Chi Kao and Ming Sun and Chao Wang},
  title = {{Improved Student Model Training for Acoustic Event Detection Models}},
  booktitle = {DCASE},
  year = {2021},
  url = {https://openreview.net/forum?id=HRlbLadMTb5},
}
Figure