A one-and-half stage pedestrian detector

Ujjwal Ujjwal, Aziz Dziri, Bertrand Leroy, Francois BremondWACV 2020

TL;DR

This paper introduces a novel "one-and-a-half stage" pedestrian detector that achieves state-of-the-art accuracy and impressive inference speeds (~32 fps). By cleverly combining semantic segmentation for efficient anchor selection with a modified feature pooling strategy, the authors bridge the gap between slow but accurate two-stage detectors and fast but less precise one-stage detectors.

What Problem Does This Paper Address?

Pedestrian detection is crucial for applications like autonomous driving and surveillance. The core challenge lies in balancing detection accuracy with inference speed. Traditional two-stage detectors (e.g., Faster R-CNN) offer high accuracy by using a computationally intensive "feature pooling" step on proposed regions. Conversely, one-stage detectors (e.g., SSD, YOLO) are much faster because they skip this pooling step, but at the cost of reduced accuracy. This trade-off has been a persistent hurdle, as real-time applications demand both speed and reliability.

Methods

The proposed approach introduces a novel "0.5-stage" for anchor selection, preceding the standard detection stage, leading to a "1.5-stage" detector.

  • Pseudo-Semantic Segmentation: This is the cornerstone of the 0.5-stage. Instead of relying on full semantic segmentation, the authors generate "pseudo-segmentation masks" from ground truth bounding boxes during training. This mask helps the network focus on relevant pedestrian regions, effectively reducing the number of anchors that need to be processed. This is a key departure from prior work that used semantic segmentation solely for accuracy improvements.
  • Anchor Classification: Following the pseudo-semantic segmentation, an anchor classification layer further refines the selection of anchors. It classifies anchors into "positive" and "negative" categories, ensuring that only the most promising anchors are passed to the detection stage.
  • Modified Feature Pooling: By drastically reducing the number of anchors through the 0.5-stage, the authors can afford to use feature pooling (a technique known for improving accuracy) without incurring the significant speed penalty typically associated with it in two-stage detectors.
  • Visible and Full Body Bounding Boxes: The approach leverages both full-body and visible-body bounding boxes during training to create more robust anchor selection criteria, especially for occluded pedestrians.

Datasets

The proposed method was evaluated on two public datasets:

  • Caltech-Reasonable: A widely used benchmark for pedestrian detection.
  • CityPersons: Another significant dataset, known for its diversity and challenges with occlusions.

The authors also mention using the CityPersons dataset for pre-training before fine-tuning on Caltech-Reasonable.

Key Insights and Ideas

  • Semantic Segmentation for Speed: The most significant insight is using semantic segmentation not just for accuracy, but as a powerful mechanism to reduce the computational load by filtering anchors early on.
  • "One-and-a-Half Stage" Sweet Spot: The paper demonstrates that a hybrid approach, incorporating elements of both one-stage and two-stage detectors, can achieve a superior balance of speed and accuracy.
  • Anchor Selection is Key: The efficiency of the proposed method hinges on its ability to select a sparse, highly relevant subset of anchors, making subsequent processing steps much faster.
  • Leveraging Both Bounding Box Types: Using both visible and full-body bounding boxes during training provides a more comprehensive understanding of pedestrians, particularly in challenging occluded scenarios.

Key Results

  • State-of-the-Art Performance: The proposed detector achieves state-of-the-art results on both the Caltech-Reasonable and CityPersons datasets, with a Log Averaged Miss Rate (LAMR) of 8.12 on Caltech-Reasonable and 4.76 on CityPersons (when pre-trained on CityPersons).
  • Impressive Inference Speed: The method achieves an inference speed of approximately 32 frames per second (fps) on 1024x1024 images. This is significantly faster than other methods that achieve comparable accuracy. For instance, it's about 1.5 times faster than the next best speed of ~20 fps reported by [16].
  • Reduced Parameters: The 0.5-stage module has significantly fewer trainable parameters (7048) compared to a standard Region Proposal Network (RPN) (18432), contributing to faster training and inference.

Limitations and Future Work

  • Input Image Size: The paper notes that the higher distortion in Caltech-Reasonable images (640x480) compared to CityPersons (2048x1024) might explain the larger performance gains on the latter. This suggests that performance might be sensitive to input resolution and aspect ratio.
  • Anchor Imbalance: The ablation study highlights that removing the anchor classification layer leads to a significant drop in performance (LAMR of 16.43% on CityPersons validation set). This indicates that class imbalance among anchors can be a problem if not properly addressed.
  • Future Work: While not explicitly detailed, the success of this approach suggests further exploration into more sophisticated semantic segmentation techniques for anchor selection and potentially applying this "1.5-stage" paradigm to other object detection tasks.

Summary updated

BibTeX
@inproceedings{ujjwal2020one,
  author = {Ujjwal Ujjwal and Aziz Dziri and Bertrand Leroy and Francois Bremond},
  title = {{A one-and-half stage pedestrian detector}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2020},
}
Figure