A Model for Infrastructure Detection along Highways Based on Remote Sensing Images from UAVs

Xian Jiang, Qing Cui, Chongguo Wang, Fan Wang, Yingxiang Zhao, Yongjie Hou, Rujun Zhuang, Yunfei Mei, Gang Shi2023 · DOI

TL;DR

This paper introduces an improved object detection model, MSA-CenterNet, specifically designed for identifying highway infrastructure in high-resolution UAV imagery. By incorporating multi-scale feature fusion and an attention mechanism into a CenterNet backbone, the model significantly enhances the detection of small and varied-sized infrastructure elements, achieving a mean Average Precision (mAP) of 86.7%.

What Problem Does This Paper Address?

The increasing adoption of intelligent road systems necessitates accurate and efficient monitoring of highway infrastructure. Traditional manual surveys are time-consuming, resource-intensive, and prone to errors. While Unmanned Aerial Vehicles (UAVs) offer a promising solution for data collection, their high-resolution imagery presents unique challenges for object detection: significant variations in object scale, complex backgrounds, and a high proportion of small targets (e.g., warning stakes, traffic signs). Existing object detection models often struggle with these characteristics, leading to missed detections and inaccuracies. Furthermore, there's a scarcity of comprehensive datasets for highway infrastructure captured by UAVs.

Methods

The authors propose an enhanced object detection model named MSA-CenterNet, built upon the anchor-free CenterNet framework. The core improvements are:

  • Modified Backbone: ResNet50 is used as the backbone network for feature extraction, leveraging its ability to learn deep semantic features.
  • Multi-Scale Feature Fusion Module: To address the scale variation problem, a novel module inspired by Res2Net is introduced. This module enhances feature extraction by creating a hierarchy of receptive fields within a single residual block, allowing for the generation of fine-grained feature maps without losing coarse-grained information. This module effectively expands the perceptual field and improves the representation of features across different scales.
  • Inverted Residual Attention Module: To further refine feature representation and focus on salient regions, an attention mechanism is integrated. This module uses an inverted residual structure to learn attention weights, enabling the model to selectively emphasize important features and suppress irrelevant ones. This helps in overcoming the issue of information loss after masking and allows for deeper network stacking.

The MSA-CenterNet model predicts the center point of an object, its size, and other attributes directly from the feature map, avoiding the need for predefined anchor boxes, which simplifies the detection process and improves efficiency.

Datasets

The paper introduces a new highway infrastructure dataset specifically for this research. This dataset was created by capturing images using a DJI ZenmuseP1 drone over a highway section in Xinjiang, China, resulting in high-resolution images (8192 × 5460 pixels) with a spatial resolution of 1 cm/pixel. The dataset consists of 1500 representative images, manually annotated with bounding boxes for eight types of infrastructure: culverts, alarm posts, traffic signs, milestones, isolated gates, kerb signs, portal frames, and monitors. The dataset is divided into training (80%), validation (10%), and testing (10%) sets.

Key Insights and Ideas

  • The "small target" problem is critical: The paper highlights that the high proportion of small targets in UAV highway imagery is a major hurdle for standard object detectors.
  • Hierarchical feature fusion is key: The multi-scale feature fusion module, by creating a hierarchy of receptive fields within blocks, proves more effective than traditional image or feature pyramids for handling scale variations in this context.
  • Attention mechanisms are vital for focus: The inverted residual attention module helps the model to intelligently focus on relevant parts of the image, improving the detection of objects that might otherwise be obscured or lost in complex backgrounds.
  • Anchor-free is suitable: The adoption of an anchor-free approach like CenterNet simplifies the pipeline and is shown to be effective for this task.

Key Results

The proposed MSA-CenterNet model achieved a mean Average Precision (mAP) of 86.7% on the custom highway infrastructure dataset. This represents a significant improvement of 3.1 percentage points over the baseline CenterNet model. Ablation studies demonstrated the individual contributions of the proposed modules:

  • Using ResNet50 as a backbone improved mAP by 0.6% over the baseline.
  • Introducing the multi-scale feature fusion module increased mAP by 1.5% over the baseline.
  • The attention mechanism alone improved mAP by 3.1% over the baseline.
  • The combined MSA-CenterNet model outperformed other mainstream detection algorithms like Faster R-CNN, SSD, RetinaNet, and YOLOv5, particularly in detecting small targets like alarm posts and milestones.

Limitations and Future Work

  • Limitations: While the proposed method shows strong performance, the paper notes that YOLOv5 achieved better detection precision for larger targets. The authors also mention that their method is not as accurate as YOLOv5 for barrier and gantry detection.
  • Future Work: The authors suggest improving the algorithm to enhance the detection precision and speed for UAV aerial images, potentially by further optimizing the model to match or surpass YOLOv5's performance on larger objects and exploring ways to further boost overall detection speed.

Summary updated

BibTeX
@misc{jiang2023model,
  author = {Xian Jiang and Qing Cui and Chongguo Wang and Fan Wang and Yingxiang Zhao and Yongjie Hou and Rujun Zhuang and Yunfei Mei and Gang Shi},
  title = {{A Model for Infrastructure Detection along Highways Based on Remote Sensing Images from UAVs}},
  year = {2023},
  doi = {10.3390/s23083847},
}
Figure