Synchronous End-to-End Vehicle Pedestrian Detection Algorithm Based on Improved YOLOv8 in Complex Scenarios

Shi Lei, Yi He, Jeffrey S. SarmientoSensors 2024 · DOI

TL;DR

This paper tackles the challenge of accurately detecting vehicles and pedestrians in complex urban traffic scenarios, which often involve occlusions and varying scales. They achieve this by enhancing the YOLOv8 architecture with deformable convolutions and a novel attention mechanism, resulting in improved detection accuracy and speed for real-time applications.

What Problem Does This Paper Address?

Modern urban environments are characterized by dynamic and often chaotic traffic. Accurately identifying and tracking vehicles and pedestrians is crucial for applications like autonomous driving, traffic management, and public safety. However, existing detection methods struggle with common real-world issues such as:

  • Varying Scales: Objects can appear at vastly different sizes depending on their distance from the camera.
  • Occlusions: Vehicles and pedestrians frequently block each other, making them difficult to detect.
  • High Computational Costs: Many advanced detection models are too computationally intensive for real-time deployment on edge devices or in large-scale surveillance systems.

These limitations lead to lower detection accuracy and slower performance, hindering the effectiveness of intelligent transportation systems.

Methods

The core of this work is an improved YOLOv8 architecture designed to address the aforementioned challenges. Key modifications include:

  • Improved Backbone Network:
    • Deformable Convolution: Instead of fixed convolutional kernels, deformable convolutions are used. These kernels can adapt their shape and sampling locations to better match the geometry of objects, especially useful for irregular shapes like pedestrians and in scenarios with occlusions. This effectively expands the receptive field and improves robustness.
  • C2f DCN Module: This module integrates deformable convolutions within the YOLOv8 backbone, enhancing the model's ability to capture multi-scale features and improve pedestrian recognition.
  • Occlusion Perceptual Attention Mechanism: A novel attention mechanism is introduced that focuses on "occluded" areas. It uses global average and maximum pooling to extract multi-scale features, helping to better identify partially hidden pedestrians. This mechanism is designed to be more robust and efficient than existing attention methods for occlusion scenarios.
  • Improved Dynamic Decoupling Head: The detection head, responsible for final recognition and localization, is enhanced. This involves a dynamic head structure that incorporates task-aware and scale-aware attention to improve spatial cognitive performance and reduce model complexity.
  • Optimized Loss Function: The paper proposes an optimized loss function that combines the advantages of the Direct Feature Learning (DFL) loss for precise localization and the Wise-IoU loss for measuring overlap between predicted and actual bounding boxes. This aims to improve target positioning accuracy.

Datasets

The primary dataset used for evaluation is the Cityscapes dataset. This dataset is widely adopted for urban scene understanding and contains 5000 finely annotated street images with complex traffic scenes, including varied lighting, occlusions, and dense object distributions. The dataset was split into 2975 training, 500 validation, and 1525 testing samples.

Key Insights and Ideas

  • Deformable Convolutions are Key for Occlusions: The adaptive nature of deformable convolutions significantly helps in capturing features of partially occluded objects, a common problem in dense traffic.
  • Targeted Attention for Occlusion: A specialized attention mechanism that prioritizes occluded regions offers a more effective solution for challenging detection scenarios compared to general-purpose attention mechanisms.
  • End-to-End Improvement: By enhancing both the backbone (feature extraction) and the head (detection and localization) of YOLOv8, the researchers achieve synergistic improvements in overall performance.
  • Loss Function Matters for Precision: Optimizing the loss function, particularly by combining DFL and Wise-IoU, directly translates to better bounding box accuracy and more reliable target positioning.

Key Results

The proposed algorithm demonstrates significant improvements over state-of-the-art models:

  • Accuracy:
    • Achieves a vehicle recall of 97.19%, surpassing YOLOv8 (94.31%) and Faster R-CNN (92.52%).
  • Achieves a pedestrian precision of 96.02%, significantly improving upon YOLOv8 (94.89%) and Faster R-CNN (93.04%).
  • Obtains a mean average precision (mAP) of 95.23% on the Cityscapes dataset, outperforming YOLOv8 (89.61%) and Faster R-CNN (90.89%).
  • An 11.76% increase in precision and a 6.27% boost in mAP are reported compared to the baseline YOLOv8.
  • Speed:
    • Maintains a real-time detection speed of 41.46 FPS, ensuring its applicability for live monitoring and autonomous systems.
  • Loss Function Optimization: The optimized loss function led to improved AP on both common (94.86% to 96.02%) and intensive (71.44% to 90.3%) test sets, while maintaining a similar detection speed.

Limitations and Future Work

The authors acknowledge that the algorithm still faces challenges under:

  • Extreme Lighting Variations: Performance may degrade in very dark or very bright conditions.
  • Highly Complex Occlusions: While improved, extremely severe occlusions can still pose difficulties.

Future work will focus on:

  • Refining Attention Mechanisms: Further enhancing the attention mechanisms to better handle diverse environmental factors.
  • Reducing Computational Overhead: Optimizing the model for deployment on edge devices with limited computational resources.
  • Expanding Environmental Robustness: Improving the model's ability to handle a wider range of environmental conditions.

Summary updated

Abstract

In modern urban traffic, vehicles and pedestrians are fundamental elements in the study of traffic dynamics. Vehicle and pedestrian detection have significant practical value in fields like autonomous driving, traffic management, and public security. However, traditional detection methods struggle in complex environments due to challenges such as varying scales, target occlusion, and high computational costs, leading to lower detection accuracy and slower performance. To address these challenges, this paper proposes an improved vehicle and pedestrian detection algorithm based on YOLOv8, with the aim of enhancing detection in complex traffic scenes. The motivation behind our design is twofold: first, to address the limitations of traditional methods in handling targets of different scales and severe occlusions, and second, to improve the efficiency and accuracy of real-time detection. The new generation of dense pedestrian detection technology requires higher accuracy, less computing overhead, faster detection speed, and more convenient deployment. Based on the above background, this paper proposes a synchronous end-to-end vehicle pedestrian detection algorithm based on improved YOLOv8, aiming to solve the detection problem in complex scenes. First of all, we have improved YOLOv8 by designing a deformable convolutional improved backbone network and attention mechanism, optimized the network structure, and improved the detection accuracy and speed. Secondly, we introduced an end-to-end target search algorithm to make the algorithm more stable and accurate in vehicle and pedestrian detection. The experimental results show that, using the algorithm designed in this paper, our model achieves an 11.76% increase in precision and a 6.27% boost in mAP. In addition, the model maintains a real-time detection speed of 41.46 FPS, ensuring robust performance even in complex scenarios. These optimizations significantly enhance both the efficiency and robustness of vehicle and pedestrian detection, particularly in crowded urban environments. We further apply our improved YOLOv8 model for real-time detection in intelligent transportation systems and achieve exceptional performance with a mAP of 95.23%, outperforming state-of-the-art models like YOLOv5, YOLOv7, and Faster R-CNN.
BibTeX
@inproceedings{lei2024synchronous,
  author = {Shi Lei and Yi He and Jeffrey S. Sarmiento},
  title = {{Synchronous End-to-End Vehicle Pedestrian Detection Algorithm Based on Improved YOLOv8 in Complex Scenarios}},
  booktitle = {Sensors},
  year = {2024},
  doi = {10.3390/s24186116},
}
Figure