EdgeYOLO: An Edge-Real-Time Object Detector

Shihan Liu, Junlin Zha, Jian Sun, Zhuo Li, Gang WangarXiv 2023 · OpenReview

TL;DR

This paper introduces EdgeYOLO, an anchor-free, real-time object detector designed for edge computing devices. It achieves a strong balance between accuracy and speed by employing an enhanced data augmentation strategy, a novel hybrid random loss function for improved small object detection, and a lightweight decoupled head. EdgeYOLO demonstrates competitive performance on standard benchmarks while meeting real-time requirements on resource-constrained hardware.

What Problem Does This Paper Address?

The paper tackles the challenge of deploying high-performance object detection models on edge computing devices, which have limited computational resources. While state-of-the-art detectors like YOLO achieve impressive accuracy on powerful GPUs, their inference speed on edge devices is often too slow for real-time applications. Existing solutions that prioritize speed on edge devices often sacrifice significant accuracy. The authors aim to bridge this gap by developing a detector that is both accurate and efficient enough to run in real-time on platforms like the NVIDIA Jetson AGX Xavier, with a particular focus on improving the detection of small objects, which are notoriously difficult to detect accurately.

Methods

EdgeYOLO builds upon the YOLO framework but introduces several key innovations:

  • Anchor-Free Design: By adopting an anchor-free approach (inspired by FCOS), the model eliminates the need for predefined anchor boxes, significantly reducing the computational cost of post-processing and improving inference speed, especially on edge devices.
  • Enhanced Data Augmentation: The paper proposes a more flexible and powerful combined data augmentation method, termed "Enhanced-Mosaic & Mixup." This strategy aims to increase the richness and validity of training data, effectively mitigating overfitting. It involves applying Mosaic augmentation to groups of images and then mixing a simply processed image with these groups, ensuring that the final output image contains sufficient effective information.
  • Lite-Decoupled Head: To further optimize for speed without sacrificing too much precision, a lighter version of the decoupled head (originally from FCOS) is designed. This involves fewer channels and convolutional layers, and incorporates implicit representation layers for better regression performance. Re-parameterization techniques are used to integrate these layers into convolutional layers, reducing inference costs.
  • Hybrid Random Loss (HRL): A novel loss function is introduced to specifically improve the detection accuracy of small objects. This loss function is applied in stages during training, with a hybrid random loss function used in later epochs to better balance the precision of small objects with overall detection accuracy.
  • Model Reduction: The paper explores different model sizes (shown in Figure 1) to cater to edge devices with varying computing power, leveraging both lossy and lossless reduction methods.

Datasets

The models were trained and evaluated on the following datasets:

  • MS COCO2017: A common and widely used dataset for object detection, serving as the primary training and evaluation benchmark.
  • VisDrone2019-DET: A dataset specifically chosen for its focus on small object detection, particularly relevant for evaluating the effectiveness of the proposed small object optimization techniques.

Key Insights and Ideas

  • Anchor-free is key for edge speed: The paper strongly advocates for anchor-free detectors as a crucial step towards achieving real-time performance on edge devices, primarily due to the significant reduction in post-processing overhead.
  • Data augmentation is more than just mixing: The proposed "Enhanced-Mosaic & Mixup" highlights that thoughtful data augmentation can be as critical as architectural changes for improving robustness and preventing overfitting, especially when dealing with diverse datasets.
  • Small objects need special attention: The significant improvement in small object detection accuracy (2.2% AP increase on COCO) underscores the importance of tailored loss functions and augmentation strategies for this challenging aspect of object detection.
  • Lightweight components matter: The success of the "Lite-Decoupled Head" demonstrates that even within established architectures, careful design of specific modules can yield substantial performance gains on resource-constrained hardware.

Key Results

  • COCO2017 Performance: EdgeYOLO achieves 50.6% AP50:95 and 69.8% AP50 on the MS COCO2017 validation set.
  • VisDrone2019-DET Performance: The model achieves 26.4% AP50:95 and 44.8% AP50 on the VisDrone2019-DET dataset.
  • Real-time on Edge: EdgeYOLO meets real-time requirements (FPS ≥ 30) on the NVIDIA Jetson AGX Xavier edge computing device.
  • Improved Small Object Detection: The paper reports a 2.2% AP increase specifically for small objects on the COCO dataset.
  • Competitive Speed-Accuracy Trade-off: EdgeYOLO demonstrates a better balance between precision and inference speed compared to other popular real-time object detectors, as illustrated in Figure 1.

Limitations and Future Work

  • Limitations: The paper acknowledges that while the decoupled head improves precision, it does incur some extra inference costs. The effectiveness of the hybrid random loss function is primarily demonstrated in later training stages.
  • Future Work: The authors suggest further improving the detection accuracy for small objects and exploring more efficient optimizations for the framework. They also believe the framework can be extended to other pixel-level recognition tasks like instance segmentation.

Summary updated

Abstract

This paper proposes an efficient, low-complexity and anchor-free object detector based on the state-of-the-art YOLO framework, which can be implemented in real time on edge computing platforms. We develop an enhanced data augmentation method to effectively suppress overfitting during training, and design a hybrid random loss function to improve the detection accuracy of small objects. Inspired by FCOS, a lighter and more efficient decoupled head is proposed, and its inference speed can be improved with little loss of precision. Our baseline model can reach the accuracy of 50.6% AP50:95 and 69.8% AP50 in MS COCO2017 dataset, 26.4% AP50:95 and 44.8% AP50 in VisDrone2019-DET dataset, and it meets real-time requirements (FPS>=30) on edge-computing device Nvidia Jetson AGX Xavier. We also designed lighter models with less parameters for edge computing devices with lower computing power, which also show better performances. Our source code, hyper-parameters and model weights are all available at https://github.com/LSH9832/edgeyolo.
BibTeX
@misc{liu2023edgeyolo,
  author = {Shihan Liu and Junlin Zha and Jian Sun and Zhuo Li and Gang Wang},
  title = {{EdgeYOLO: An Edge-Real-Time Object Detector}},
  year = {2023},
  eprint = {2302.07483},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2302.07483},
}
Figure