Improving Sound Event Detection with Foreground-Background Classification and Domain Adaptation

Michel OlveraDCASE 2021 · OpenReview

Abstract

Sound Event Detection (SED) in domestic environments faces significant challenges due to the scarcity of strongly labeled real-world data and the resulting domain gap between synthetic and real audio. This paper proposes two novel techniques to address these issues: an auxiliary foreground-background (FB) classification task and an optimal transport-based domain adaptation strategy. The FB task leverages spectro-temporal structures to regularize the model, while the domain adaptation explicitly aligns feature distributions between synthetic and real domains. Experimental results on the DESED dataset demonstrate a substantial improvement, with the proposed system achieving a ~14-15% absolute increase in F1-score over the DCASE 2020 baseline.

Introduction

Sound Event Detection (SED) aims to identify the identity and temporal boundaries (onset and offset) of sound events within an audio recording. This technology is critical for applications such as smart home surveillance and assisted living. However, training robust SED models is hampered by the lack of strongly labeled real-world data. Researchers often employ semi-supervised learning using a combination of synthetic soundscapes (with strong labels) and real recordings (with weak or no labels). This approach frequently leads to a "domain gap," where models overfit to synthetic features and fail to generalize to complex real-world acoustic environments. This research addresses this mismatch by improving feature representations through auxiliary tasks and geometric distribution alignment.

Methods

The proposed system builds upon a standard Mean Teacher Convolutional Recurrent Neural Network (CRNN) architecture, enhanced by two primary components:

Foreground-Background (FB) Auxiliary Task

Domestic sounds are categorized into two broad groups based on their spectro-temporal characteristics: foreground (short, transient sounds like speech or dog barks) and background (long, stationary sounds like vacuum cleaners). An auxiliary branch is added to the network to classify these categories. Furthermore, a fused SEDFB branch integrates representations from both the SED and FB classifiers to refine the final output, effectively modeling the time-varying spectra of sound events.

The proposed CRNN architecture featuring the sound event detection (SED) branch, the foreground-background (FB) classifier, and the fused SEDFB branch.

Domain Adaptation via Optimal Transport

To bridge the gap between synthetic (source) and real (target) data, the authors utilize the Deep Joint Distribution Optimal Transport (DeepJDOT) framework. Unlike standard adversarial training, this method explicitly aligns the empirical distributions of latent feature representations for "active" and "inactive" audio frames across both domains. Pseudo-labels for real data are generated and refined using available weak clip-level labels to ensure high-quality alignment.

Experimental Setup and Results

Experiments were conducted on the DESED dataset, the benchmark for DCASE Challenge Task 4, comprising synthetic audio clips (source) and real soundscapes (target). The authors compared two post-processing techniques: standard median filtering (MF) and Hidden Markov Model (HMM) decoding.

Quantitative Performance

The proposed methods yielded drastic improvements over the DCASE 2020 baseline. As shown in the results table, the integration of the FB branch and Domain Adaptation (DA) significantly boosted performance across both validation and evaluation sets.

Comparison of event-based macro F1-scores for various model configurations, demonstrating the additive benefits of the FB branch and Domain Adaptation.

Key findings include:

  • Baseline: Achieved a macro F1-score of 34.8% (validation) and 38.1% (evaluation).
  • FB Branch Impact: Adding the FB branch alone increased the validation F1-score to 43.12%.
  • Combined System: The full model (SEDFB + DA) reached 47.61% on validation and 52.12% on evaluation using median filtering. With HMM smoothing, the evaluation F1-score reached 53.30%.

Discussion and Conclusion

The results suggest that spectro-temporal grouping acts as a powerful regularizer, helping the model understand the fundamental physics of audio scenes. Furthermore, Optimal Transport proved highly effective for audio domain adaptation, overcoming the synthetic-to-real performance drop without the instability typical of adversarial methods. The architectural fusion in the SEDFB branch provided benefits similar to complex post-processing, making simple median filtering nearly as effective as HMM decoding.

While highly effective, the system's reliance on pseudo-labeling suggests that performance is partially dependent on the quality of initial predictions. Future work could explore extending this optimal transport framework to other domains, such as bioacoustics, or investigating additional physics-inspired auxiliary tasks to further enhance robustness.

Summary updated

BibTeX
@inproceedings{olvera2021improving,
  author = {Michel Olvera},
  title = {{Improving Sound Event Detection with Foreground-Background Classification and Domain Adaptation}},
  booktitle = {DCASE},
  year = {2021},
  url = {https://openreview.net/forum?id=zCqGf9hGPX},
}
Figure