Attention-Guided Sample-Based Feature Enhancement Network for Crowded Pedestrian Detection Using Vision Sensors

Shuyuan Tang, Yiqing Zhou, Jintao Li, Chang Liu, Jinglin ShiSensors 2024 · DOI

TL;DR

This paper introduces AGFEN, a novel deep learning architecture for pedestrian detection in crowded scenes. By employing an attention-guided mechanism to enhance feature details and a modified non-maximum suppression strategy, AGFEN significantly improves detection accuracy, particularly in challenging occlusion scenarios.

What Problem Does This Paper Address?

Detecting pedestrians in crowded urban environments is a significant challenge for computer vision systems. This difficulty stems from two main types of occlusion: inter-class occlusion (objects obscuring pedestrians) and intra-class occlusion (pedestrians overlapping each other). These occlusions lead to fragmented or incomplete pedestrian features, making it hard for existing detection models to accurately identify and localize individuals. While many methods struggle with heavily occluded pedestrians, this paper aims to improve detection performance in such complex scenarios.

Methods

The proposed AGFEN architecture builds upon the Faster R-CNN framework and incorporates several key innovations:

  • Attention-Guided Feature Pyramid Network (AGFPN): This module enhances feature maps by using high-level semantic information to guide the sampling of low-level feature details. It effectively maps semantic context onto fine-grained visual features, improving the representation of occluded pedestrians without requiring additional annotation branches. This is achieved through a top-down enhancement process that emphasizes local features.
  • RoI Aggregation (RoI-A): This technique reuses the weights of the R-CNN classifier to aggregate local features within each Region of Interest (RoI). This helps to mitigate confusion between similar-looking objects and improves the robustness of feature representation.
  • Instance Set Prediction and Set NMS: Instead of a one-to-one correspondence between proposals and ground truths, the paper adopts a one-to-multiple approach. This is coupled with a modified Non-Maximum Suppression (NMS) strategy called "Set NMS." Set NMS treats sets of overlapping predictions as a single unit, allowing for the retention of true positives that might otherwise be suppressed by traditional NMS, especially in crowded scenarios.

Datasets

The proposed AGFEN model was evaluated on three widely used pedestrian detection datasets:

  • CrowdHuman: Known for its high density of pedestrians and significant occlusion.
  • CityPersons: A dataset focusing on pedestrian detection in urban driving scenarios.
  • Caltech: A dataset with varying occlusion levels, often used for evaluating pedestrian detection performance.

Key Insights and Ideas

  • Semantic guidance for fine details: The core idea of using high-level semantic features to guide the enhancement of low-level visual details is crucial for improving detection in occluded scenarios. This approach effectively "fills in the gaps" left by occlusion.
  • Beyond one-to-one matching: Moving from a strict one-to-one proposal-to-instance matching to a one-to-multiple approach, combined with Set NMS, is a clever way to handle overlapping predictions in crowded scenes without losing potentially correct detections.
  • Leveraging existing knowledge: Reusing classifier weights for RoI aggregation is an efficient way to improve feature representation without introducing entirely new trainable components.
  • AGFEN as a general enhancement: The AGFPN module is designed to be a general enhancement that can be embedded into existing FPN-based detectors, offering flexibility and potential for broader application.

Key Results

The AGFEN model demonstrated strong performance across the evaluated datasets:

  • On the CrowdHuman dataset, AGFEN achieved a significant improvement of 2.38% in AP compared to the baseline detector. It also achieved optimal scores on the MR-2 and JI metrics.
  • Comparisons with other state-of-the-art methods on CrowdHuman showed AGFEN achieving the optimal results across AP, MR-2, and JI metrics.
  • On the CityPersons dataset, AGFEN achieved comparable performance to top methods on the R and HO sets and was optimal on the partial set, highlighting its effectiveness in various occlusion levels.
  • On the Caltech dataset, AGFEN also showed strong competitiveness, achieving competitive MR-2 scores without requiring specialized occlusion handling modules like visible region branches.

Limitations and Future Work

  • Computational Complexity: While AGFEN improves accuracy, the paper notes that the RoI-A module's computational complexity increases exponentially with the number of RoI features. This suggests a trade-off between performance and computational cost that needs to be managed for real-time applications.
  • Generalizability of AGFPN: The paper suggests that AGFPN can be easily embedded into other backbones, implying future work could explore its integration with different network architectures.
  • Further NMS Exploration: While Set NMS is effective, the paper's ablation studies show that RoI-A can further improve performance when combined with various NMS strategies, suggesting ongoing potential for optimization in post-processing.

Summary updated

BibTeX
@article{tang2024attention,
  author = {Shuyuan Tang and Yiqing Zhou and Jintao Li and Chang Liu and Jinglin Shi},
  title = {{Attention-Guided Sample-Based Feature Enhancement Network for Crowded Pedestrian Detection Using Vision Sensors}},
  journal = {Sensors},
  year = {2024},
  doi = {10.3390/s24196350},
}
Figure