TL;DR
This paper introduces Edge YOLO, a lightweight object detection system designed for autonomous vehicles that leverages edge-cloud cooperation. By optimizing YOLOv4 for edge devices and integrating it with cloud-based training, Edge YOLO achieves a compelling balance of high accuracy and real-time performance with significantly reduced computational resources.
What Problem Does This Paper Address?
Autonomous vehicles (AVs) and intelligent transportation systems (ITS) increasingly rely on deep learning-based object detection (DL-OD) for tasks like traffic monitoring and driving assistance. However, existing DL-OD models, while accurate, are often computationally intensive, leading to high energy consumption and latency issues, especially on resource-constrained edge devices common in AVs. This creates a dilemma: powerful cloud-based solutions offer accuracy but suffer from communication delays and bandwidth limitations, while on-device solutions are often too slow or inaccurate. The paper addresses this by proposing an edge-cloud cooperation (E-CC) framework that offloads heavy training to the cloud while enabling efficient real-time inference on the edge.
Methods
The core of the proposed system is Edge YOLO, a modified version of the YOLOv4 object detection network. The authors implemented several key optimizations:
- Lightweight Backbone: The CSPDarknet53 backbone from YOLOv4 was trimmed by reducing the number of layers and convolutional operations, significantly decreasing model size and computational load.
- Optimized Neck: The PANet in YOLOv4's neck was replaced with a combination of SPP (Spatial Pyramid Pooling) and FPN (Feature Pyramid Network) modules. This aims to retain multi-scale feature fusion capabilities while reducing redundancy and computational cost.
- Activation Function: The Mish activation function, known for its complexity and high RAM usage, was replaced with Leaky ReLU. This choice offers a better balance for edge devices by preventing "dead neurons" and reducing computational overhead.
- Edge-Cloud Cooperation (E-CC): The system employs a novel E-CC strategy. Edge devices (like NVIDIA Jetson) perform real-time object detection inference. When idle, they upload collected data to the cloud. The cloud then filters, labels, and retrains the model, sending updated weights back to the edge devices. This differs from federated learning by having the cloud perform the primary training and model updates.
- Anchor Box Optimization: The authors utilized a K-Means clustering approach to optimize anchor box sizes for better detection of objects at various scales.
- Soft-NMS: To improve detection robustness and reduce false positives/negatives, they adopted the Soft-NMS algorithm.
Datasets
The paper uses two primary datasets for training and evaluation:
- COCO2017: A widely used, large-scale dataset for object detection, segmentation, and captioning, containing 80 object categories. It's used for training and evaluating the general object detection performance.
- KITTI: A benchmark dataset specifically for autonomous driving scenarios, focusing on tasks like stereo vision, optical flow, and object detection. It's used to evaluate the system's performance in a more domain-specific context.
The authors also mention creating a custom dataset by collecting data from edge devices, which is regularly updated for mask detection.
Key Insights and Ideas
- Edge-Cloud Synergy: The paper demonstrates that a well-defined edge-cloud cooperation model can effectively balance the strengths of edge (real-time inference, low latency) and cloud (heavy training, model updates) for resource-constrained applications like AVs.
- Targeted Optimization: Simply using a lightweight network isn't enough; the authors show that specific architectural modifications (backbone trimming, neck redesign, activation function choice) are crucial for achieving high performance on edge devices.
- Practicality over State-of-the-Art: While YOLOv4 is a powerful model, Edge YOLO prioritizes practicality for edge deployment by sacrificing some of its complexity for significant gains in speed and reduced resource usage, achieving a better trade-off for the target application.
- Leaky ReLU's Role: The choice of Leaky ReLU over Mish is a pragmatic decision for edge devices, highlighting how seemingly minor algorithmic choices can have a substantial impact on performance and efficiency.
Key Results
- Size Reduction: Edge YOLO's model size is significantly reduced to 25.27 MB, which is only about 10.2% of the original YOLOv4.
- Accuracy: Achieved an mAP (mean Average Precision) of 47.3% on the COCO2017 dataset.
- Speed: Reached 26.6 FPS on the COCO2017 dataset, demonstrating real-time capabilities.
- KITTI Performance: On the KITTI dataset, Edge YOLO achieved an mAP of 0.8212 with 40.6 FPS on the Jetson Xavier platform.
- Power Efficiency: The average power consumption for the edge computing device was measured at 16.8W.
- Comparison: Edge YOLO outperformed MobileNetV3 SSD in terms of mAP while maintaining competitive FPS, and offered a better speed-accuracy trade-off compared to YOLOv3-Tiny.
Limitations and Future Work
- Slightly Inferior to YOLOv4 in Extreme Scenarios: The paper acknowledges that in scenarios with "super multi-target detection," Edge YOLO might be slightly inferior to the full YOLOv4 due to its reduced network layers and parameters.
- TensorRT Impact: While TensorRT acceleration is used, the paper notes that converting to FP16 can lead to a slight drop in mAP for large datasets like COCO2017.
- Future Work: The authors intend to deploy Edge YOLO on a wider range of edge computing devices and integrate it into more ITS platforms and scenarios, further exploring its applicability.
Summary updated