Graph-Enhanced Dual-Stream Feature Fusion with Pre-Trained Model for Acoustic Traffic Monitoring

Shitong Fan, Feiyang Xiao, Wenbo Wang, Shuhan Qi, Qiaoxi Zhu, Wenwu Wang, Jian GuanICASSP 2025 · DOI

TL;DR

This paper introduces GEDF-Net, a novel deep learning model for acoustic traffic monitoring that simultaneously identifies vehicle types and their directions. By combining a pre-trained model for feature enhancement with a graph attention mechanism to capture temporal relationships, GEDF-Net significantly improves detection accuracy, achieving first place in the DCASE 2024 Challenge Task 10.

What Problem Does This Paper Address?

Accurate traffic monitoring is crucial for efficient urban planning and management. While acoustic sensors offer a promising, non-intrusive method for this, existing approaches face significant hurdles. These include the scarcity of labeled real-world traffic audio data and the complexity of distinguishing between different vehicle types and their movement directions (e.g., left-to-right vs. right-to-left). Traditional single-sensor setups struggle with direction detection due to limited spatial resolution, while microphone arrays, though better, are still hampered by data limitations. The DCASE 2024 Challenge Task 10 specifically targets this problem, requiring the detection of vehicle types and directions using multi-channel audio.

Methods

The proposed GEDF-Net employs a dual-stream architecture for feature extraction, followed by a fusion module and a predictor.

  • Vehicle Type Feature Extraction (VTFE) Branch: This branch leverages a pre-trained model (PANNs) trained on a large audio dataset (AudioSet). This transfer learning approach helps mitigate data scarcity and enhances the vehicle type features. Crucially, a graph attention mechanism is then applied to these features. This mechanism treats feature frames as nodes in a graph and learns to weigh their relationships (edges) based on temporal proximity and semantic similarity, effectively highlighting important audio events related to vehicle travel.
  • Vehicle Direction Feature Extraction (VDFE) Branch: This branch utilizes the Generalized Cross-Correlation with Phase Transform (GCC-PHAT) to estimate time delays between audio channels. These time delays are then processed through a convolutional encoder and an MLP to extract direction-specific features. An average pooling operation is also applied to capture important directional information.
  • Frame-level Feature Fusion Module: The features extracted from both the VTFE and VDFE branches are concatenated and fed into a Gated Recurrent Unit (GRU). The GRU processes these fused features over time, allowing it to capture temporal dependencies and produce a fine-grained representation that accounts for both vehicle type and direction.
  • Category Count Predictor: A simple linear layer with a ReLU activation function takes the fused features and predicts the counts for each specific category (e.g., car moving left-to-right, commercial vehicle moving right-to-left).

Datasets

The experiments were conducted on the DCASE 2024 Challenge Task 10 development dataset. This dataset includes real-world audio recordings from six different locations, along with synthetic data generated by the Pyroadacoustics simulator. The metadata for each location, including the number of audio samples, traffic density, and vehicle audio events, is provided in Table I. The paper also references the use of AudioSet for pre-training the PANNs model.

Key Insights and Ideas

  • The Power of Pre-training for Data Scarcity: Using a pre-trained model (PANNs) significantly boosts performance, especially in scenarios with limited real-world data. This highlights the effectiveness of transfer learning in overcoming data limitations in specialized audio tasks.
  • Graph Attention for Temporal Context: Applying graph attention to audio features allows the model to explicitly learn and leverage temporal relationships between audio events. This is crucial for understanding complex acoustic scenes where events related to the same vehicle might be spread across multiple frames.
  • Dual-Stream Fusion for Comprehensive Understanding: Combining vehicle type and direction features through a dedicated fusion module leads to a more robust and fine-grained representation, ultimately improving detection accuracy.
  • Visualizing Attention: The visualization of learned adjacency graphs (Figure 2) provides compelling evidence that the graph attention mechanism successfully highlights relevant vehicle travel events within the audio, offering interpretability to the model's decision-making process.

Key Results

The proposed GEDF-Net achieved 1st place in the DCASE 2024 Challenge Task 10. Ablation studies demonstrated that both the pre-trained model and the graph attention mechanism contribute significantly to the overall performance. The model outperformed the baseline significantly, particularly in locations with fewer training samples, underscoring the benefit of pre-training. The paper also shows that GEDF-Net surpasses other top-ranking systems, including a Transformer-based approach, on the official evaluation set.

Limitations and Future Work

  • Limited Open-Source Reproducibility: The authors note that the official evaluation set for DCASE 2024 Task 10 was not publicly released, and most top-ranking systems were not open-sourced, making direct reproduction of all comparisons challenging.
  • Performance at Very Low Data Locations: While generally effective, the graph attention mechanism showed slightly less improvement at location loc4, which had a very limited number of audio events (63). This suggests that the graph attention model might require a minimum amount of data to effectively learn robust representations.
  • Future Work: The paper implicitly suggests that further exploration of graph-based methods for acoustic event analysis and the integration of more sophisticated fusion techniques could lead to even greater improvements. The success of GEDF-Net also points towards the continued importance of leveraging large-scale pre-trained models and advanced attention mechanisms in audio processing tasks.

Summary updated

BibTeX
@inproceedings{fan2025graph,
  author = {Shitong Fan and Feiyang Xiao and Wenbo Wang and Shuhan Qi and Qiaoxi Zhu and Wenwu Wang and Jian Guan},
  title = {{Graph-Enhanced Dual-Stream Feature Fusion with Pre-Trained Model for Acoustic Traffic Monitoring}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2025},
  doi = {10.1109/icassp49660.2025.10889208},
}
Figure