TL;DR
This paper introduces a novel lightweight, anchor-free object detection network designed for resource-constrained UAV platforms. By combining a modified MobileNetV2 backbone with a simultaneous sub-pixel convolution-based upsampling and detection module, and an objectness branch, the method achieves significant reductions in computational cost and parameter size while improving detection accuracy on UAV imagery.
What Problem Does This Paper Address?
Unmanned Aerial Vehicles (UAVs) are increasingly used for various applications, including intelligent transportation systems, due to their mobility and broad spatial coverage. A crucial component for these applications is real-time object detection from UAV-captured images. However, UAVs have limited onboard storage and computational resources, making it challenging to deploy complex, high-accuracy object detection models. Existing methods often achieve good detection results on challenging UAV imagery (complex scenes, small objects, dense distributions) by increasing model complexity, which is not suitable for onboard deployment. This paper aims to bridge this gap by developing a lightweight and efficient object detection framework that can perform effectively on UAVs without sacrificing too much accuracy.
Methods
The proposed framework is an anchor-free, single-stage object detector. Key methodological contributions include:
- Lightweight Backbone: The authors adapt the MobileNetV2 network by removing its last 1x1 convolutional layer. This modification is found to reduce computational cost and parameter size without negatively impacting detection accuracy, and in some cases, even improving it.
- Simultaneous Upsampling and Detection Module: Instead of using computationally expensive transpose convolutions for upsampling feature maps, the paper employs sub-pixel convolution. This module integrates upsampling and detection into a single layer, sharing convolutional structures to further reduce computational burden and parameter count. This approach also avoids checkerboard artifacts often associated with transpose convolutions.
- Objectness Branch: To enhance detection accuracy, especially in challenging UAV scenarios, an objectness branch is added to the detection head. This branch predicts a heatmap indicating the likelihood of a pixel being the center of any object, providing an additional supervision signal that helps the model learn more robust feature representations and reduce false positives. The final bounding box score is a combination of classification and objectness scores.
- Anchor-Free Approach: The model predicts object centers and scales directly, eliminating the need for predefined anchor boxes, which simplifies the detection process and can improve robustness.
- Post-processing: The paper utilizes a simplified non-maximum suppression (NMS) method based on maxpooling for deduplication, which is computationally less demanding than traditional NMS.
Datasets
The proposed method was evaluated on two publicly available UAV image datasets:
- Visdrone-2018-VID: This dataset contains video clips captured by drones with varying resolutions and annotated objects across ten categories (pedestrian, person, car, van, bus, truck, motor, bicycle, awning-tricycle, and tricycle). The authors used the training and validation sets for training and testing.
- UAVDT-DET: This dataset consists of video clips from a UAV platform in urban environments, with annotations for three categories (car, truck, and bus).
Key Insights and Ideas
- Removing the last 1x1 conv in MobileNetV2 is beneficial: Contrary to typical classification network design, removing the final 1x1 convolutional layer of MobileNetV2 improved detection accuracy and significantly reduced computational load for subsequent layers. This suggests that for detection tasks, this layer might discard useful information or introduce unnecessary complexity.
- Sub-pixel convolution is a viable, efficient alternative to transpose convolution: The integration of sub-pixel convolution for upsampling and detection in a shared module offers a substantial reduction in computational cost and parameters compared to traditional transpose convolution, without compromising accuracy.
- Objectness branch significantly aids in reducing false positives: The addition of the objectness branch, even with a minimal increase in computational cost, demonstrably improves detection accuracy and reduces false positives, particularly in complex scenarios where distinguishing objects from background clutter is difficult.
Key Results
- Computational Efficiency: The proposed method achieves a computational cost that is 92.78% lower than CenterNet with a ResNet18 backbone.
- Parameter Reduction: The model has 86.73% fewer parameters compared to CenterNet with ResNet18.
- Accuracy Improvement: On the Visdrone-2018-VID dataset, the proposed method achieves 2.8 points higher mAP compared to CenterNet with ResNet18.
- Frame Rate: The model achieves a frame rate of approximately 220 FPS on a GPU, indicating its suitability for real-time applications.
- Comparison with Baselines: The method outperforms Tiny YOLOv3, Pelee, and SSD on the Visdrone-2018-VID dataset in terms of mAP, while maintaining significantly lower computational costs and parameter sizes. For instance, it achieves 6.4 mAP higher than Tiny YOLOv3 with only 23.2% of its FLOPs and 48.6% of its parameters.
Limitations and Future Work
Limitations:
- The paper primarily focuses on improving model structure for lightweight detection. While it achieves good results, further optimization might be needed for extremely resource-constrained edge devices.
Future Work:
- Explore novel data augmentation techniques to further boost performance.
- Investigate quantization and pruning methods to achieve even higher speeds and more accurate lightweight detection networks for UAVs.
- Deploy the proposed method on-board UAVs and integrate its detection results with Intelligent Transportation System (ITS) applications.
Summary updated