Pedestrian Detection Based on Feature Enhancement in Complex Scenes

Jiao Su, Yi An, Jialin Wu, Kai ZhangAlgorithms 2024 · DOI

TL;DR

This paper introduces PT-YOLO, a novel pedestrian detection network designed to overcome challenges in complex scenes like occlusion and scale variations. By integrating feature enhancement modules (SE, BiFPN, CoordConv) and a new loss function (WIoU) into the YOLOv5 framework, PT-YOLO achieves state-of-the-art performance with improved accuracy and robustness.

What Problem Does This Paper Address?

Pedestrian detection is a critical task in computer vision, vital for applications like intelligent transportation and security. However, detecting pedestrians in complex real-world scenarios presents significant challenges. These include:

  • Low detection accuracy: Especially for small or partially occluded pedestrians.
  • Misdetection and false detection: Leading to unreliable system performance.
  • Scale variations: Pedestrians can appear at vastly different sizes in an image.
  • Complex backgrounds and occlusions: Making it difficult to distinguish pedestrians from their surroundings.

Traditional methods struggle with adaptability, while existing deep learning approaches, though advanced, still face limitations in these challenging conditions. This paper aims to bridge this gap by proposing a more robust and accurate pedestrian detection system.

Methods

The proposed PT-YOLO network builds upon the YOLOv5 architecture, incorporating several key enhancements:

  • Squeeze-and-Excitation (SE) Module: Added to the backbone, this module allows the network to focus on the most important pedestrian features and suppress irrelevant ones, improving feature extraction and reducing missed detections.
  • Bi-directional Feature Pyramid Network (BiFPN) Module: Integrated into the neck, BiFPN enhances the fusion of multi-scale features from different layers. This improves information transfer and addresses challenges posed by scale variations, leading to higher accuracy.
  • Wise Intersection over Union (WIoU) Loss Function: Used in the head, WIOU is a novel bounding box regression loss that dynamically focuses on important samples, effectively balancing difficult and easy cases. This reduces regression errors and improves the overall detection accuracy.
  • Coordinate Convolution (CoordConv) Module: Introduced into the neck, CoordConv explicitly incorporates spatial coordinate information into the convolutional layers. This helps the network better perceive location information and enhance boundary detection, further improving accuracy.

Datasets

The primary dataset used for training and evaluation in this paper is the JRDB dataset. This dataset is specifically designed for pedestrian detection and tracking and contains large-scale pedestrian scenes. The authors preprocessed the dataset to convert its .json files into the YOLO format and split it into an 8:2 ratio for training and validation.

Key Insights and Ideas

  • Modular Enhancement is Key: The paper demonstrates that combining specific, well-chosen modules (SE, BiFPN, CoordConv) and a novel loss function (WIoU) can significantly boost the performance of an existing strong baseline like YOLOv5.
  • Feature Enhancement Matters: Explicitly guiding the network to focus on important features (SE) and better fuse multi-scale information (BiFPN) directly translates to improved detection accuracy, especially in challenging scenarios.
  • Spatial Awareness is Crucial: The introduction of CoordConv highlights the importance of explicitly encoding spatial location information, which is vital for precise localization of pedestrians.
  • Loss Function Design Impacts Robustness: The WIoU loss function's ability to handle sample imbalance is shown to be critical for improving bounding box regression accuracy and overall detection performance.

Key Results

The PT-YOLO network demonstrated superior performance compared to the baseline YOLOv5 and other mainstream detection networks:

  • Precision: Improved from 93.8% (YOLOv5) to 95.2%.
  • Recall: Improved from 81.2% (YOLOv5) to 85.2%.
  • [email protected]: Increased from 90.9% (YOLOv5) to 93.4%.
  • [email protected]: Increased from 67.4% (YOLOv5) to 72.8%.
  • Loss: The total loss was 0.59% lower than YOLOv5, with a box loss of 0.0328 and confidence loss of 0.0259.
  • Comparison: PT-YOLO achieved the highest MAP (93.4%) and lowest loss (0.054) among the compared networks (Faster R-CNN, SSD, YOLOv3, YOLOv7, YOLOv8), while maintaining competitive parameters and FLOPs.
  • Qualitative Results: Visualizations showed PT-YOLO effectively detecting small objects, reducing false detections in complex scenes, and accurately identifying pedestrians in crowded environments.

Limitations and Future Work

  • Limitations: The paper notes that while PT-YOLO shows superior performance, the SSD network achieved slightly higher detection accuracy, potentially due to its use of multiple feature maps at different scales. The authors suggest that PT-YOLO's single-scale feature mapping might limit its performance for very small or very large objects.
  • Future Work: The authors propose further studying the PT-YOLO network for pedestrian tracking, indicating a natural extension of their detection capabilities.

Summary updated

BibTeX
@article{su2024pedestrian,
  author = {Jiao Su and Yi An and Jialin Wu and Kai Zhang},
  title = {{Pedestrian Detection Based on Feature Enhancement in Complex Scenes}},
  journal = {Algorithms},
  year = {2024},
  doi = {10.3390/a17010039},
}
Figure