Learning Region-Based Attention Network for Traffic Sign Recognition

Ke Zhou, Yufei Zhan, Dongmei Fu2021 · DOI

TL;DR

This paper introduces a new benchmark for traffic sign recognition in challenging ice and snow environments (ITSRB and ITSDB) and proposes PFANet, a novel attention-based network that achieves state-of-the-art accuracy on this benchmark and the German Traffic Sign Recognition Benchmark (GTSRB). The work highlights the difficulties of TSR in adverse weather and proposes a robust solution.

What Problem Does This Paper Address?

Traffic sign recognition (TSR) is crucial for autonomous driving and advanced driver-assistance systems. While significant progress has been made, TSR in harsh environments like those with ice and snow remains a significant challenge. Factors such as reduced visibility, glare from snow, and altered sign appearance due to weather conditions make it difficult for current systems to accurately detect and classify traffic signs. Existing datasets often lack the complexity and diversity of these adverse conditions, and robust networks capable of handling them are still needed. This paper aims to address this gap by creating a more challenging dataset and developing a network that performs well under these difficult circumstances.

Methods

The core of this paper lies in two main contributions: a new dataset and a novel network architecture.

  • Datasets: The authors introduce two new datasets:
  • ITSDB (Ice Traffic Sign Detection Benchmark): For traffic sign detection, featuring images with diverse lighting, time, environmental factors, and occlusions, all within ice and snow conditions. It includes over 43,000 traffic sign instances.
  • ITSRB (Ice Traffic Sign Recognition Benchmark): For traffic sign classification, containing cropped images of traffic signs from ITSDB, preserving the challenging environmental factors. This dataset has 34,577 training images and 8,709 test images, categorized into nine classes.
  • PFANet (Parallel Fusion Attention Network): This network is designed to tackle the challenges posed by the ice and snow environment. It comprises:
  • Parallel Feature Extraction: Utilizes HRNet, a high-resolution network known for maintaining spatial information across different scales.
  • Parallel Fusion Attention Modules: Two novel attention modules, PFAN-A and PFAN-B, are introduced.
  • PFAN-A: A self-attention module that enhances feature extraction by highlighting relevant information within a single feature map.
  • PFAN-B: A cross-attention module that fuses information from different resolution feature maps, allowing for better understanding of inter-map correlations.
  • These modules are integrated into a parallel structure to effectively combine multi-resolution features and improve classification accuracy.

Datasets

  • Newly Introduced:
    • ITSDB (Ice Traffic Sign Detection Benchmark): 5806 images with 43,290 traffic sign instances, collected under ice and snow environments with various challenging conditions.
  • ITSRB (Ice Traffic Sign Recognition Benchmark): Cropped traffic sign images from ITSDB, focusing on classification. It contains 34,577 training images and 8,709 test images, divided into nine categories.
  • Existing Datasets Used for Comparison:
    • GTSRB (German Traffic Sign Recognition Benchmark): A widely-used public dataset for traffic sign classification, used to validate the robustness of the proposed PFANet.

Key Insights and Ideas

  • Adverse Weather is a Major Hurdle: The paper strongly emphasizes that ice and snow environments significantly degrade the performance of existing TSR systems, highlighting the need for specialized datasets and models.
  • Dataset Diversity is Key: The newly proposed ITSDB and ITSRB datasets are designed to be more challenging than existing ones by incorporating a wider range of environmental factors, occlusions, and varying sign sizes, making them suitable for robust model evaluation.
  • Attention for Feature Fusion: The PFANet's parallel fusion attention mechanism is crucial. By combining self-attention (PFAN-A) for intra-map feature enhancement and cross-attention (PFAN-B) for inter-map feature fusion, the network can better capture complex relationships and subtle cues in the challenging visual data.
  • High-Resolution Networks are Beneficial: The use of HRNet as the backbone for feature extraction demonstrates the advantage of maintaining high-resolution representations throughout the network, which is important for preserving fine-grained details of traffic signs.
  • Robustness Across Datasets: PFANet not only excels on the challenging new ITSRB dataset but also achieves competitive results on the standard GTSRB, indicating its generalizability and robustness.

Key Results

  • ITSRB Performance: PFANet achieved an accuracy of 93.57% on the proposed ITSRB dataset, outperforming other advanced methods like HRNet-w18 and EfficientNet.
  • GTSRB Performance: PFANet achieved an accuracy of 97.21% on the GTSRB dataset, demonstrating its effectiveness even on a standard benchmark.
  • Detection Benchmark Evaluation: The paper evaluated existing detection models (Libra-RCNN, HRNetv2p, Faster-RCNN) on the ITSDB, showing that Libra-RCNN performed best, indicating the dataset's challenging nature.
  • Ablation Study: The ablation study confirmed that both PFAN-A and PFAN-B modules individually contribute to performance improvements, and their combination in PFANet leads to the best results.

Limitations and Future Work

  • Oscillation in Early Training: The authors note that PFANet exhibited more oscillation in loss and accuracy during the early stages of training compared to some baselines. They suggest this might be due to the extra computation introduced by the attention modules.
  • Parameter and GFLOPs Increase: While PFANet achieves higher accuracy, the attention modules do increase the number of parameters and computational cost (GFLOPs) compared to the baseline.
  • Future Work:
    • Investigate the reasons behind the early training oscillation and explore methods to mitigate it.
  • Focus on reducing the network's parameters to create a smaller, more efficient model.
  • Extend the research to handle other special environments beyond ice and snow, and explore advanced mechanisms for improved recognition.
  • Apply these findings to real-world autonomous driving systems to enhance intelligence.

Summary updated

BibTeX
@misc{zhou2021learning,
  author = {Ke Zhou and Yufei Zhan and Dongmei Fu},
  title = {{Learning Region-Based Attention Network for Traffic Sign Recognition}},
  year = {2021},
  doi = {10.3390/s21030686},
}
Figure