TL;DR
This paper introduces RE-YOLOv5, a novel algorithm designed to improve the detection of occluded objects in road scenes, a critical challenge for autonomous driving. By enhancing feature extraction with deformable convolutions and a receptive field augmentation module, and optimizing post-processing with EIOU-NMS, RE-YOLOv5 significantly boosts detection accuracy, especially in complex, occluded environments.
What Problem Does This Paper Address?
Intelligent assisted driving systems rely heavily on accurate road object detection. However, real-world driving scenarios are fraught with challenges, particularly the frequent occlusion of objects by other vehicles, pedestrians, or environmental elements. This occlusion leads to truncated or deformed object features, making them difficult for standard object detectors to identify. Existing methods often struggle with these scenarios, leading to missed detections or incorrect classifications, which can have serious safety implications. This paper aims to tackle this specific problem by developing a more robust object detection system capable of handling dense occlusions.
Methods
The proposed RE-YOLOv5 builds upon the YOLOv5 architecture and introduces several key enhancements:
- Deformable Convolutional Layers (CSPD Module): Standard convolutional layers have fixed kernel shapes. To better adapt to the irregular shapes and deformations of occluded objects, the authors replace some standard convolutional layers within YOLOv5's CSPLayer with deformable convolutions (specifically, DCNv2). This allows the network to dynamically adjust its receptive field to better capture features from occluded objects.
- Receptive Field Atrous Enhancement (RFAE) Module: To capture richer contextual information and understand the relationships between visible object parts and their surroundings, an RFAE module is designed. This module utilizes atrous convolution with varying dilation rates to expand the receptive field without increasing computational cost or losing feature resolution. It also incorporates an Efficient Channel Attention (ECA) mechanism to refine channel correlations.
- EIOU-Non-Maximum Suppression (EIOU-NMS): Traditional Non-Maximum Suppression (NMS) can incorrectly suppress valid bounding boxes of occluded objects in dense scenes. The paper proposes an optimized NMS method using EIOU (Efficient IOU). EIOU considers not only the overlap area but also the distance between box centers and the differences in width and height, leading to more accurate suppression of redundant boxes and better preservation of occluded object detections.
Datasets
The proposed RE-YOLOv5 was evaluated on two widely used benchmark datasets:
- KITTI: This dataset contains nine categories of labeled information, including cars, pedestrians, and cyclists. For this study, the dataset was reclassified, and the training, validation, and test sets were split in an 8:1:1 ratio. The KITTI dataset also provides difficulty grading (easy, moderate, hard) based on object height, visibility, and truncation.
- CityPersons: This dataset focuses specifically on pedestrian detection in autonomous driving scenarios. It categorizes pedestrians based on occlusion levels: "Bare" (no occlusion), "Reasonable" (largely visible), "Partial occlusion" (partially obscured), and "Heavy occlusion" (severely obscured). The authors used the training set for training and the validation set for quantitative analysis.
Key Insights and Ideas
- Deformable Convolutions are Crucial for Occlusion: The paper strongly demonstrates that adapting the convolutional kernel's shape to object deformation is key to improving detection in occluded scenarios.
- Context is King (Especially for Occlusion): The RFAE module highlights the importance of incorporating broader contextual information to infer the presence and location of occluded objects.
- NMS Needs to Be Smarter for Dense Scenes: Standard NMS is a bottleneck in crowded, occluded environments. EIOU-NMS offers a more nuanced approach to box suppression, significantly reducing false negatives.
- YOLOv5 as a Strong Baseline: The paper shows that by strategically enhancing an existing powerful architecture like YOLOv5, significant improvements can be achieved for a specific challenging task.
Key Results
The RE-YOLOv5 model demonstrated significant improvements over the baseline YOLOv5 and other state-of-the-art methods:
- KITTI Dataset:
- Achieved a mean Average Precision (mAP) of 82.04%, an improvement of 2.34% over the baseline.
- Outperformed Faster-RCNN by 19.94% mAP and state-of-the-art models like PSNS-SSD and PointRCNN by 7.63% and 11.93%, respectively.
- CityPersons Dataset:
- For heavily occluded objects, the Log Average Miss Rate (MR-2) was reduced to 40.31%, a decrease of 9.65% compared to the baseline.
- Showed consistent improvements across all occlusion levels (Bare, Reasonable, Partial, Heavy).
- Qualitative Results: Visual comparisons on both KITTI and CityPersons datasets clearly show RE-YOLOv5 detecting objects missed by the baseline YOLOv5, particularly in challenging intra- and inter-category occlusion scenarios.
- Real-world Video Data: Practical testing on dashcam video data confirmed RE-YOLOv5's effectiveness in real driving scenarios, despite a slight reduction in FPS due to deformable convolutions.
Limitations and Future Work
- Inference Speed: The introduction of deformable convolutions and the RFAE module increases the model's parameter count and inference time, leading to a slight decrease in FPS compared to the original YOLOv5. While the authors state that 43 FPS is still sufficient for practical applications, further optimization for speed might be beneficial.
- Computational Complexity: Deformable convolutions, while effective, can be computationally more intensive.
- Future Work: The paper does not explicitly detail future work directions, but the success of the proposed modules suggests potential for further exploration in combining these techniques with other advanced object detection architectures or applying them to different domains with similar occlusion challenges. The authors also mention that EIOU-NMS can be easily added to different models without additional training, hinting at its broad applicability.
Summary updated