A Study of Improved Two-Stage Dual-Conv Coordinate Attention Model for Sound Event Detection and Localization

Guorong Chen, Yuan Yu, Yuan Qiao, Junliang Yang, Chongling Du, Zhang Qian, Xiao Huang

TL;DR

This paper introduces the TDCAM, a novel two-stage model for Sound Event Detection and Localization (SELD) that enhances feature extraction with a Dual-conv Coordinate Attention Module (DCAM) and improves temporal modeling with a two-layer Bi-GRU. The TDCAM achieves state-of-the-art performance on the TAU Spatial Sound Events 2019 dataset, demonstrating its effectiveness in complex acoustic scenarios with overlapping sounds.

What Problem Does This Paper Address?

Sound Event Detection and Localization (SELD) is a challenging task that requires simultaneously identifying what sound is occurring (Sound Event Detection - SED) and where it is coming from (Sound Source Localization - SSL). Real-world audio is often complex, with multiple sound events overlapping in both time and space. This overlap makes it difficult for models to accurately distinguish between events and pinpoint their locations. Existing methods, while improving, still struggle with effectively capturing both fine-grained local features and long-term temporal dependencies, especially in these complex, overlapping scenarios.

Methods

The core of this work is the TDCAM (Two-Stage Dual-conv Coordinate Attention Model). It builds upon a two-stage SELD architecture, separating SED and SSL into distinct branches that share an initial feature extraction layer.

Key innovations include:

  • Dual-conv Coordinate Attention Module (DCAM): This module replaces traditional convolutional layers. It combines:
  • Dual Convolutional Blocks: Inspired by VGG networks, these use multiple small convolutional kernels to capture local features.
  • Coordinate Attention: This mechanism focuses on the spatial information within feature maps, allowing the model to better understand feature importance based on their location and relationships across channels. This enhances feature selection and representation.
  • Two-Layer Bi-GRU: To address the limitation of single-layer Bi-GRUs in capturing temporal dependencies in the original two-stage network, the TDCAM employs two stacked Bi-GRU layers. This allows for a more comprehensive understanding of sequential audio data.
  • Data Augmentation: The researchers incorporate frequency and time masking techniques (similar to SpecAugment) applied to the spectrograms. This enriches the training data, improving the model's robustness and generalization capabilities, especially in diverse audio environments.

Datasets

The primary dataset used for training and evaluation is the TAU Spatial Sound Events 2019 development dataset. This dataset is part of the DCASE 2019 challenge and includes audio data in both First-Order Ambisonics (FOA) and Microphone Array (MIC) formats. It features 11 spatialized sound event categories, with up to two overlapping events per audio clip, and provides detailed metadata for sound event locations (azimuth, elevation, distance).

Key Insights and Ideas

  • Coordinate Attention is Crucial for Spatial Understanding: The paper highlights that simply using channel attention isn't enough. Coordinate Attention's ability to encode positional information is key to better understanding spatial structures and improving localization accuracy.
  • Depth Matters for Temporal Modeling: The improvement seen by simply stacking a second Bi-GRU layer suggests that capturing longer-term temporal context is vital for accurate SELD, especially when dealing with complex event sequences.
  • Data Augmentation is Not Just for Robustness, but for Generalization: The significant performance gains from frequency and time masking underscore their importance in making models adaptable to unseen audio conditions and improving overall generalization.
  • Synergistic Effects of Components: The ablation studies clearly show that the combination of the DCAM and the two-layer Bi-GRU, along with data augmentation, yields the best results, indicating a strong synergistic effect between these improvements.

Key Results

The proposed TDCAM model demonstrates significant improvements over the baseline two-stage network and other comparative models on the TAU Spatial Sound Events 2019 dataset.

  • SED Performance: The TDCAM achieves competitive or superior SED performance, particularly in terms of the F-score, indicating better event detection accuracy.
  • DOA Performance: While not always the absolute best in DOA metrics, the TDCAM significantly reduces the DOA error rate. For example, it lowers the DOA error rate by 0.6 and 1.4 compared to the two-stage model on FOA and MIC datasets, respectively. Frame recall also shows slight but consistent improvements.
  • Data Augmentation Impact: Incorporating data augmentation further boosts performance across all metrics, highlighting its critical role.
  • Ablation Studies: These experiments confirm the individual contributions of the two-layer Bi-GRU (improving DOA accuracy) and the DCAM (improving SED F-score), with their combination leading to the best overall results.

Limitations and Future Work

  • Limitations: The paper acknowledges that while the TDCAM performs well, it may not always be the top performer in every single metric compared to all other models, particularly in DOA. The effectiveness of the model is validated on a specific dataset, and its performance on more diverse or challenging datasets could be further explored.
  • Future Work: The authors plan to investigate alternative time-series modeling methods beyond Bi-GRU and explore other multi-task learning strategies. The goal is to further enhance both the accuracy and efficiency of SELD systems.

Summary updated

Abstract

Sound Event Detection and Localization (SELD) is a comprehensive task that aims to solve the subtasks of Sound Event Detection (SED) and Sound Source Localization (SSL) simultaneously. The task of SELD lies in the need to solve both sound recognition and spatial localization problems, and different categories of sound events may overlap in time and space, making it more difficult for the model to distinguish between different events occurring at the same time and to locate the sound source. In this study, the Dual-conv Coordinate Attention Module (DCAM) combines dual convolutional blocks and Coordinate Attention, and based on this, the network architecture based on the two-stage strategy is improved to form the SELD-oriented Two-Stage Dual-conv Coordinate Attention Model (TDCAM) for SELD. TDCAM draws on the concepts of Visual Geometry Group (VGG) networks and Coordinate Attention to effectively capture critical local information by focusing on the coordinate space information of the feature map and dealing with the relationship between the feature map channels to enhance the feature selection capability of the model. To address the limitation of a single-layer Bi-directional Gated Recurrent Unit (Bi-GRU) in the two-stage network in terms of timing processing, we add to the structure of the two-layer Bi-GRU and introduce the data enhancement techniques of the frequency mask and time mask to improve the modeling and generalization ability of the model for timing features. Through experimental validation on the TAU Spatial Sound Events 2019 development dataset, our approach significantly improves the performance of SELD compared to the two-stage network baseline model. Furthermore, the effectiveness of DCAM and the two-layer Bi-GRU structure is confirmed by performing ablation experiments.
BibTeX
@inproceedings{chen2024study,
  author = {Guorong Chen and Yuan Yu and Yuan Qiao and Junliang Yang and Chongling Du and Zhang Qian and Xiao Huang},
  title = {{A Study of Improved Two-Stage Dual-Conv Coordinate Attention Model for Sound Event Detection and Localization}},
  booktitle = {Sensors},
  year = {2024},
  url = {https://openreview.net/forum?id=6qo6dVMjE1},
}
Figure