DNN and CNN with Weighted and Multi-task Loss Functions for Audio Event Detection

Huy Phan, Martin Krawczyk-Becker, Timo Gerkmann, Alfred Mertins

Abstract

Audio Event Detection (AED) involves identifying specific sound events within continuous, noisy audio streams. Traditional approaches often struggle with data imbalance and the neglect of temporal structures. This paper proposes a robust AED system that leverages specialized loss functions rather than just architectural modifications. By employing a weighted loss to mitigate background noise dominance and a multi-task loss to model event boundaries, the system achieved an 88.3% F1-score in the DCASE 2017 challenge, a substantial improvement over the 64.1% baseline.

Introduction

The primary goal of AED is to determine the "what" and "when" of sounds like baby cries or gunshots. Current research faces three main challenges: an over-reliance on architectures borrowed from other domains, severe data imbalance where background noise outweighs rare events, and the failure to account for the temporal onset and offset of sounds. This work addresses these by focusing on the mathematical formulation of the learning objectives.

Methodology

The proposed system follows a two-stage pipeline consisting of background rejection and event classification.

Overview of the proposed audio event detection pipeline, including signal enhancement, feature extraction, and two-stage neural network processing.

Phase-Aware Signal Enhancement

Audio signals are first cleaned using an enhancement technique that considers both magnitude and phase. By estimating the clean phase of harmonic sounds, the system better reconstructs signals from background noise compared to magnitude-only methods.

Weighted Loss for Background Rejection

To handle the "needle in a haystack" problem of rare events, a binary classifier is trained with a weighted cross-entropy loss. False negatives are penalized ten times more heavily than false positives, ensuring the model remains sensitive to rare occurrences amidst dominant background noise.

Multi-task Loss for Event Classification

For the classification stage, the network is trained on three concurrent tasks:

  1. Classification: Identifying the sound category.
  2. Distance Estimation: Predicting the temporal distance from the current frame to the event's onset and offset.
  3. Confidence: A term based on the Intersection over Union (IoU) of predicted versus ground truth boundaries.
The multi-task output layer structure, showing the parallel prediction of classification probabilities (via softmax) and temporal distance estimates (via sigmoid).

This multi-task approach acts as an implicit regularizer, forcing the network to learn the "shape" of the sound event.

Experimental Results

The system was evaluated on the DCASE 2017 Challenge Task 2 dataset, featuring rare events (baby cry, glass break, gun shot) mixed with TUT Acoustic Scenes 2016 background noise. The proposed approach significantly outperformed the DCASE baseline:

  • F1-Score: Improved from 64.1% to 88.3% on the evaluation set.
  • Error Rate: Reduced from 0.64 to 0.22.
  • Challenge Ranking: The system ranked 3rd out of 13 participating teams.

Discussion: CNNs vs. DNNs

The study revealed distinct advantages for different architectures. CNNs excelled at detecting baby cries, likely due to their ability to model vocal-tract variations. In contrast, DNNs performed better on glass breaks and gunshots, where preserving high-frequency details without the "blurring" effect of convolutional pooling proved beneficial.

Conclusion and Future Work

By shifting focus to loss function design, this work demonstrates that significant gains in AED performance are possible without complex architectural changes. Future work will explore using temporal distance estimates for real-time early event detection and extending the system to handle multi-event environments.

Summary updated

Abstract

This report presents our audio event detection system submitted for Task 2, "Detection of rare sound events", of DCASE 2017 challenge. The proposed system is based on convolutional neural networks (CNNs) and deep neural networks (DNNs) coupled with novel weighted and multi-task loss functions and state-of-the-art phase-aware signal enhancement. The loss functions are tailored for audio event detection in audio streams. The weighted loss is designed to tackle the common issue of imbalanced data in background/foreground classification while the multi-task loss enables the networks to simultaneously model the class distribution and the temporal structures of the target events for recognition. Our proposed systems significantly outperform the challenge baseline, improving F-score from 72.7% to 90.0% and reducing detection error rate from 0.53 to 0.18 on average on the development data. On the evaluation data, our submission obtains an average F1-score of 88.3% and an error rate of 0.22 which are significantly better than those obtained by the DCASE baseline (i.e. an F1-score of 64.1% and an error rate of 0.64).
BibTeX
@misc{phan2017dnn,
  author = {Huy Phan and Martin Krawczyk-Becker and Timo Gerkmann and Alfred Mertins},
  title = {{DNN and CNN with Weighted and Multi-task Loss Functions for Audio Event Detection}},
  year = {2017},
  eprint = {1708.03211},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1708.03211},
}
Figure