DV-DETR: Improved UAV Aerial Small Target Detection Algorithm Based on RT-DETR

Xiaolong Wei, Ling Yin, Liangliang Zhang, Fei WuSensors 2024 · DOI

TL;DR

This paper introduces DV-DETR, an enhanced real-time object detection model for UAVs, specifically tackling the challenge of small target detection in dense aerial imagery. By integrating a lighter backbone, advanced attention mechanisms, and a specialized loss function, DV-DETR achieves improved accuracy and speed, making it more practical for real-world drone applications.

What Problem Does This Paper Address?

Drones are increasingly used for surveillance and monitoring, but accurately detecting small objects (like people or vehicles) in their aerial imagery is a significant challenge. This is due to factors like the small size of the targets, their potential for occlusion, and the high-density nature of urban scenes captured from above. Existing real-time object detection models, while fast, often struggle with the precision required for these small targets. The paper aims to bridge this gap by developing a model that is both accurate for small objects and efficient enough for real-time drone operation.

Methods

The proposed DV-DETR model builds upon the RT-DETR architecture and introduces several key improvements:

  • Lightweight Backbone (ResNet18): The authors replace the original backbone with ResNet18. This reduces model complexity and computational cost, making it more suitable for resource-constrained drone platforms, while still aiming to maintain sufficient feature extraction capabilities.
  • Enhanced Neck Network:
    • Recalibration Attention Units (RAU): Integrated into the SBA module, RAUs help fuse features from different hierarchical levels more effectively. They adaptively extract complementary information from deep semantic features and shallow boundary details, crucial for preserving fine object contours and location accuracy.
  • Deformable Attention Mechanism (DAttention): This replaces the baseline AIFI module. DAttention allows the model to dynamically focus on relevant regions in the feature map by using learned offset networks, improving its ability to capture relationships between objects in complex scenes.
  • Focaler-IoU Loss Function: To address the issue of imbalanced target scales and the difficulty in detecting small objects, this specialized loss function is used for bounding box regression. It adaptively adjusts weights to focus more on challenging samples and improve the regression accuracy for small targets.

Datasets

The primary dataset used for training and evaluation is VisDrone2019. This dataset is noted for its comprehensiveness, containing 288 video clips and 10,209 static images captured across 14 cities, covering diverse environments and lighting conditions. It specifically includes a wide variety of small and densely packed objects, making it ideal for small object detection research. The dataset defines 10 target categories, including pedestrians, vehicles, and bicycles, with many annotations featuring very small, densely clustered targets.

Key Insights and Ideas

  • The trade-off between speed and accuracy for small targets is critical for UAVs: While many real-time detectors are fast, they sacrifice accuracy on small objects. DV-DETR demonstrates that targeted architectural improvements can achieve both.
  • Attention mechanisms are key to handling multi-scale features: The integration of RAU and DAttention highlights how adaptive attention can better fuse information from different feature levels and dynamically focus on relevant image regions, which is vital for small object detection.
  • Loss functions matter for imbalanced data: The use of Focaler-IoU shows that tailoring the loss function to better handle the distribution of difficult and easy samples can significantly improve bounding box regression accuracy, especially for small or challenging targets.
  • Lightweight backbones can be effective with proper enhancements: Choosing ResNet18 demonstrates that a simpler backbone can be viable if the subsequent feature fusion and attention mechanisms are sufficiently powerful.

Key Results

The DV-DETR model achieved the following on the VisDrone2019 dataset:

  • Improved Accuracy: An [email protected] of 50.2%, which is a 1.7% improvement over the baseline RT-DETR model. Significant improvements were noted in categories like pedestrians, bicycles, and motorcycles.
  • Increased Speed: Detection speed increased from 75 FPS to 90 FPS, meeting real-time processing requirements.
  • Reduced Complexity: The use of ResNet18 as a backbone significantly reduced the model's parameter count and computational complexity (GFLOPS).

Limitations and Future Work

  • Limitations: While DV-DETR shows significant improvements, the paper acknowledges that further refinement and extensive field testing are required. The authors suggest that future testing in a wider variety of environments, including different weather conditions and longer observation periods, would facilitate comprehensive validation.
  • Future Work: The authors propose continued refinement and extensive field testing to validate the model's robustness across diverse operational conditions. This includes exploring different weather conditions, longer observation periods, and more diverse datasets to ensure comprehensive validation and adaptation for practical applications.

Summary updated

Abstract

For drone-based detection tasks, accurately identifying small-scale targets like people, bicycles, and pedestrians remains a key challenge. In this paper, we propose DV-DETR, an improved detection model based on the Real-Time Detection Transformer (RT-DETR), specifically optimized for small target detection in high-density scenes. To achieve this, we introduce three main enhancements: (1) ResNet18 as the backbone network to improve feature extraction and reduce model complexity; (2) the integration of recalibration attention units and deformable attention mechanisms in the neck network to enhance multi-scale feature fusion and improve localization accuracy; and (3) the use of the Focaler-IoU loss function to better handle the imbalanced distribution of target scales and focus on challenging samples. Experimental results on the VisDrone2019 dataset show that DV-DETR achieves an [email protected] of 50.1%, a 1.7% improvement over the baseline model, while increasing detection speed from 75 FPS to 90 FPS, meeting real-time processing requirements. These improvements not only enhance the model's accuracy and efficiency but also provide practical significance in complex, high-density urban environments, supporting real-world applications in UAV-based surveillance and monitoring tasks.
BibTeX
@inproceedings{wei2024dv,
  author = {Xiaolong Wei and Ling Yin and Liangliang Zhang and Fei Wu},
  title = {{DV-DETR: Improved UAV Aerial Small Target Detection Algorithm Based on RT-DETR}},
  booktitle = {Sensors},
  year = {2024},
  doi = {10.3390/s24227376},
}
Figure