Traffic Sign Detection Based on Lightweight Multiscale Feature Fusion Network

Shan Lin, Zicheng Zhang, Jie Tao, Fan Zhang, Xing Fan, Qing-Chang LuSustainability 2022 · DOI

TL;DR

This paper introduces a lightweight yet effective network for traffic sign detection, addressing the challenges of varying scales, complex backgrounds, and the need for real-time performance in advanced driver-assistance systems. By combining a MobileNetV3 backbone with a novel multiscale feature fusion module and a hybrid attention mechanism, the proposed model achieves high accuracy and speed, outperforming existing methods, especially for small and challenging traffic signs.

What Problem Does This Paper Address?

Advanced Driver-Assistance Systems (ADAS) rely heavily on accurately identifying traffic signs to ensure road safety. However, existing detection methods often struggle with several key issues:

  • Scale Variation: Traffic signs can appear at vastly different sizes in images due to distance and camera perspective.
  • Complex Environments: Factors like varying lighting conditions, shadows, and occlusions (e.g., signs partially hidden by trees) make detection difficult.
  • Real-time Performance: For ADAS to be effective, detection must happen quickly, often in milliseconds. Many high-accuracy methods are too computationally expensive for real-time deployment.
  • Information Fusion: Effectively combining information from different feature levels (low-level details vs. high-level semantics) is crucial for robust detection.

This paper aims to tackle these challenges by developing a model that is both accurate and efficient, particularly for small and difficult-to-detect traffic signs.

Methods

The proposed approach, named MFHA-TSDR (Traffic Sign Detection and Recognition Network Combining Multiscale Features and Hybrid Attention Mechanism), builds upon several key components:

  • Lightweight Backbone (C-MobileNetV3): The authors adapt MobileNetV3, a known efficient architecture, to serve as the feature extractor. This choice is driven by its simplicity and fewer parameters, crucial for achieving high detection speed.
  • Hybrid Attention Module: This module is designed to enhance feature representation by incorporating both spatial and channel attention.
  • Spatial Attention: Focuses on identifying the most relevant spatial locations within a feature map.
  • Channel Attention: Assigns different weights to different feature channels, highlighting important features.
  • These two branches are combined to create a "hybrid" attention mechanism that adaptively re-weights features.
  • Multiscale Feature Fusion Network (Feature-Interleaving Module): Inspired by BiFPN, this module addresses the scale variation problem. It takes feature maps from different layers of the backbone and fuses them using a top-down and bottom-up approach. This allows the network to combine high-level semantic information with low-level spatial details, creating richer multiscale representations.
  • Key-Point Detection Network: Instead of traditional anchor-based methods, the paper adopts a key-point detection approach similar to CenterNet. Traffic signs are treated as points (their center), and the network predicts the center offset, dimensions (width and height), and classification probability for each detected point. This simplifies the detection process and can be more efficient.

Datasets

The primary dataset used for training, validation, and testing is TT100K. This dataset contains traffic signs encountered in real-world driving scenarios. The paper mentions that the model was trained on 36 common categories and analyzed for 221 types of traffic signs.

Key Insights and Ideas

  • Lightweight + Multiscale + Attention = Synergy: The core idea is that combining a lightweight backbone with sophisticated feature fusion and attention mechanisms can achieve a superior balance of speed and accuracy, overcoming the limitations of either approach alone.
  • Feature-Interleaving for Richer Representations: The proposed feature-interleaving module effectively merges features from different scales, providing a more comprehensive understanding of the traffic sign's context and appearance. This is particularly beneficial for small signs where fine details are critical.
  • Hybrid Attention for Adaptive Feature Weighting: By jointly considering spatial and channel importance, the hybrid attention module allows the network to focus on the most discriminative parts of the feature maps, improving robustness against occlusions and complex backgrounds.
  • Key-Point Detection for Efficiency: Shifting from anchor-based methods to key-point detection simplifies the network architecture and can contribute to faster inference times.

Key Results

  • High Accuracy: The proposed model achieved over 85% detection accuracy for 36 common traffic sign categories on the TT100K dataset, with five categories exceeding 95% accuracy.
  • Improved Small Target Detection: The method significantly improved detection performance for small traffic signs, a known challenge in the field.
  • Real-time Performance: The lightweight nature of the network, combined with the efficient detection approach, resulted in good real-time performance. While the exact FPS is reported as 10.8 (compared to 4.3 for CornerNet and 7.9 for CenterNet), it's competitive and achieved with higher accuracy.
  • Superiority over Baselines: Compared to established methods like Faster R-CNN, CornerNet, and CenterNet, the proposed MFHA-TSDR demonstrated clear advantages in both speed and accuracy, particularly in challenging scenarios like small, dense, or partially obscured signs.
  • Ablation Studies: Detailed ablation studies confirmed the effectiveness of each component. The full model (Model 8) consistently outperformed models with only parts of the proposed architecture, highlighting the synergistic benefits of the multiscale fusion and hybrid attention.

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • Generalization to Unseen Categories: The current network is trained on specific categories. Future work should focus on improving its migration capability to recognize traffic signs not present in the training set, addressing variations across different countries or regions.
  • Challenging Weather Conditions: Experiments were primarily conducted under normal conditions. Future research should investigate performance in adverse weather (fog, haze, rain, snow) which significantly impacts visibility and detection.
  • Driver Attention Mimicry: The paper suggests exploring how to make the network mimic human driver attention, focusing on relevant signs and ignoring less critical ones. This could further reduce information redundancy and improve efficiency.
  • Complex Traffic Environments: While the model performs well, applying it to highly complex, real-world automated driving systems requires further integration and testing.

Summary updated

Abstract

Traffic sign detection is a research hotspot in advanced assisted driving systems, given the complex background, light transformation, and scale changes of traffic sign targets, as well as the problems of slow result acquisition and low accuracy of existing detection methods. To solve the above problems, this paper proposes a traffic sign detection method based on a lightweight multiscale feature fusion network. Since a lightweight network model is simple and has fewer parameters, it can greatly improve the detection speed of a target. To learn more target features and improve the generalization ability of the model, a multiscale feature fusion method can be used to improve recognition accuracy during training. Firstly, MobileNetV3 was selected as the backbone network, a new spatial attention mechanism was introduced, and a spatial attention branch and a channel attention branch were constructed to obtain a mixed attention weight map. Secondly, a feature-interleaving module was constructed to convert the single-scale feature map of the specified layer into a multiscale feature fusion map to realize the combined encoding of high-level semantic information and low-level semantic information. Then, a feature extraction base network for lightweight multiscale feature fusion with an attention mechanism based on the above steps was constructed. Finally, a key-point detection network was constructed to output the location information, bias information, and category probability of the center points of traffic signs to achieve the detection and recognition of traffic signs. The model was trained, validated, and tested using TT100K datasets, and the detection accuracy of 36 common categories of traffic signs reached more than 85%, among which the detection accuracy of five categories exceeded 95%. The results showed that, compared with the traditional methods of Faster R-CNN, CornerNet, and CenterNet, traffic sign detection based on a lightweight multiscale feature fusion network had obvious advantages in the speed and accuracy of recognition, significantly improved the detection performance for small targets, and achieved a better real-time performance.
BibTeX
@inproceedings{lin2022traffic,
  author = {Shan Lin and Zicheng Zhang and Jie Tao and Fan Zhang and Xing Fan and Qing-Chang Lu},
  title = {{Traffic Sign Detection Based on Lightweight Multiscale Feature Fusion Network}},
  booktitle = {Sustainability},
  year = {2022},
  doi = {10.3390/su142114019},
}
Figure