Occluded Pedestrian Detection with Visible IoU and Box Sign Predictor

Ruiqi Lu, Huimin Ma

Introduction

Detecting pedestrians in crowded urban environments is a cornerstone of autonomous driving and smart surveillance. However, heavy occlusion remains a significant hurdle. Standard detectors often struggle when pedestrians are partially hidden by vehicles or other pedestrians. This paper proposes two lightweight, plug-and-play modules: Visible IoU and the Box Sign Predictor. These additions improve detection performance on the CityPersons benchmark without the need for complex part-based feature extraction.

The Challenges of Occlusion

The authors identify two primary bottlenecks in standard CNN-based detectors:

  1. Suboptimal Positive Sample Selection: Detectors use Intersection over Union (IoU) to label training samples. In occluded scenes, a Region of Interest (RoI) might have a high IoU with a ground-truth (GT) box even if the visible part of the pedestrian is minimal. Training on such samples introduces noise, as the network learns features from the occluder rather than the pedestrian.
  2. Imprecise Bounding Box Regression: In dense crowds, accurate localization is critical. Standard regressors attempt to predict both the magnitude and direction of box shifts simultaneously, which often leads to errors that cause valid detections to be suppressed during Non-Maximum Suppression (NMS).

Proposed Methodology

Visible IoU

To address the issue of poor-quality training samples, the authors introduce "Visible IoU" ($IoU_{vis}$). This metric adjusts the standard IoU by a decay function $f$ based on the visible ratio ($vis\_ratio$) of the RoI.

Calculation of Visible IoU based on the visible ratio of the pedestrian within the RoI.

The $vis\_ratio$ is defined as the ratio of the visible area of the pedestrian within the RoI to the total visible area of the pedestrian. By applying a decay function (typically a Sigmoid), $IoU_{vis}$ effectively filters out samples where the pedestrian is mostly occluded, even if the overall box overlap is high.

Distribution of positive samples (vis_ratio vs. max IoU_ori) without decay (a) and with Sigmoid decay (b). The decay function removes low-quality, highly occluded samples.

Box Sign Predictor

To enhance localization, a "Box Sign Predictor" branch is added in parallel to the standard classifier and regressor. This branch treats the direction of the bounding box shift as a classification task. It predicts whether a box should move left/right or expand/shrink along its axes. By decoupling the direction (sign) from the magnitude, the model achieves more stable and accurate box refinement.

Experimental Results

The proposed methods were evaluated on the CityPersons dataset using the log-average Miss Rate ($\text{MR}^{-2}$), where lower values indicate better performance.

  • State-of-the-Art: The model achieved a miss rate of 10.8% on the "Reasonable" subset and 54.3% on the "Heavy Occlusion" subset, outperforming previous benchmarks like OR-CNN and Repulsion Loss.
  • Ablation Study: Visible IoU alone provided a 0.57% improvement in $\text{MR}^{-2}$, while the Box Sign Predictor added a further 0.60% gain.

Conclusion

This research demonstrates that significant gains in occluded pedestrian detection can be achieved through simple, effective modifications to the training and regression logic. By prioritizing high-quality training samples via Visible IoU and decoupling direction from magnitude in box regression, the proposed approach sets a new standard for performance on challenging datasets like CityPersons.

Summary updated

Abstract

Training a robust classifier and an accurate box regressor are difficult for occluded pedestrian detection. Traditionally adopted Intersection over Union (IoU) measurement does not consider the occluded region of the object and leads to improper training samples. To address such issue, a modification called visible IoU is proposed in this paper to explicitly incorporate the visible ratio in selecting samples. Then a newly designed box sign predictor is placed in parallel with box regressor to separately predict the moving direction of training samples. It leads to higher localization accuracy by introducing sign prediction loss during training and sign refining in testing. Following these novelties, we obtain state-of-the-art performance on CityPersons benchmark for occluded pedestrian detection.
BibTeX
@inproceedings{lu2019occluded,
  author = {Ruiqi Lu and Huimin Ma},
  title = {{Occluded Pedestrian Detection with Visible IoU and Box Sign Predictor}},
  booktitle = {ICIP},
  year = {2019},
  url = {https://openreview.net/forum?id=g0XkfwNur2-},
}
Figure