Pedestrian Detection With Autoregressive Network Phases

Garrick Brazil, Xiaoming Liu

Abstract

We present an autoregressive pedestrian detection framework with cascaded phases designed to progressively improve precision. The proposed framework utilizes a novel lightweight stackable decoder-encoder module which uses convolutional re-sampling layers to improve features while maintaining efficient memory and runtime cost. Unlike previous cascaded detection systems, our proposed framework is designed within a region proposal network and thus retains greater context of nearby detections compared to independently processed RoI systems. We explicitly encourage increasing levels of precision by assigning strict labeling policies to each consecutive phase such that early phases develop features primarily focused on achieving high recall and later on accurate precision. In consequence, the final feature maps form more peaky radial gradients emulating from the centroids of unique pedestrians. Using our proposed autoregressive framework leads to new state-of-the-art performance on the reasonable and occlusion settings of the Caltech pedestrian dataset, and achieves competitive state-of-the-art performance on the KITTI dataset.

Introduction

Detecting pedestrians in complex urban environments remains a significant challenge for autonomous systems. While deep learning has advanced the field, current state-of-the-art methods often face a trade-off between robustness and efficiency. Ensemble networks are robust but computationally expensive, while recurrent networks are efficient but limited by weight sharing. This paper introduces AR-Ped, an autoregressive pedestrian detection framework that bridges this gap by iteratively refining detections through a series of specialized phases.

Methods: The Autoregressive Region Proposal Network

The core innovation of AR-Ped is the Autoregressive Region Proposal Network (AR-RPN). Unlike traditional detectors that produce a single set of proposals, the AR-RPN operates in multiple cascaded phases ($P_1, P_2, P_3$).

Overview of the AR-Ped architecture showing the autoregressive flow and incremental supervision policies.

Autoregressive Flow and Specialized Weights

Each phase in the AR-RPN takes the feature maps and predictions from the preceding phase as input. Crucially, each phase utilizes its own unique set of weights, allowing the model to specialize at different stages of the refinement process. This architecture provides the feature diversity of an ensemble with the iterative refinement of a recurrent network.

The De-encoder Module and Convolutional Re-sampling

To maintain efficiency, the authors developed a lightweight, stackable De-encoder Module. This module features integrated top-down (decoder) and bottom-up (encoder) pathways. A key optimization is the use of "convolutional re-sampling" layers, which combine up/down-sampling and feature generation into a single step. This approach reduces GPU memory usage by 41% compared to standard two-step methods (e.g., bilinear interpolation followed by convolution).

Incremental Supervision

The framework employs a strategy of incremental supervision to guide the refinement process. The first phase is trained with a lenient labeling policy (e.g., IoU > 0.4) to prioritize high recall. Subsequent phases are subjected to increasingly strict rules (e.g., IoU > 0.5, then 0.6), forcing the network to progressively sharpen its focus and improve precision.

Key Insights: "Peaky" Gradients and Context

A significant finding of this work is that stricter labeling policies in later phases lead to the development of "peaky" radial gradients in the final feature maps. Instead of broad regions of high activation, the network learns to produce sharp peaks centered on individual pedestrians.

Qualitative results demonstrating the refinement of feature maps into peaky gradients that accurately localize pedestrians.

By performing this refinement within the RPN rather than on isolated Regions of Interest (RoIs), the model retains global context. This allows it to better distinguish pedestrians from similar-looking background objects, such as vertical poles, thereby reducing false positives.

Experimental Results

The researchers evaluated AR-Ped on the Caltech Pedestrian Dataset and KITTI benchmarks.

  • State-of-the-Art Performance: On the Caltech "Reasonable" setting, AR-Ped achieved a 6.45% miss rate, setting a new benchmark at the time of publication.
  • Efficiency: The system maintains a competitive runtime of 91ms per image on a Titan 1080 Ti, making it suitable for real-time applications.
  • False Positive Reduction: Qualitative analysis confirms a significant reduction in false positives (red boxes) as the phases progress, while maintaining high-quality detections for true pedestrians (green boxes).

Conclusion and Future Work

AR-Ped demonstrates that an autoregressive approach with incremental supervision can significantly enhance pedestrian detection precision without sacrificing efficiency. While the model excels in standard and partially occluded scenarios, future work will focus on improving performance under heavy occlusion, perhaps by integrating specialized occlusion-handling modules or denser anchor sampling strategies.

Summary updated

Abstract

We present an autoregressive pedestrian detection framework with cascaded phases designed to progressively improve precision. The proposed framework utilizes a novel lightweight stackable decoder-encoder module which uses convolutional re-sampling layers to improve features while maintaining efficient memory and runtime cost. Unlike previous cascaded detection systems, our proposed framework is designed within a region proposal network and thus retains greater context of nearby detections compared to independently processed RoI systems. We explicitly encourage increasing levels of precision by assigning strict labeling policies to each consecutive phase such that early phases develop features primarily focused on achieving high recall and later on accurate precision. In consequence, the final feature maps form more peaky radial gradients emulating from the centroids of unique pedestrians. Using our proposed autoregressive framework leads to new state-of-the-art performance on the reasonable and occlusion settings of the Caltech pedestrian dataset, and achieves competitive state-of-the-art performance on the KITTI dataset.
BibTeX
@inproceedings{brazil2019pedestrian,
  author = {Garrick Brazil and Xiaoming Liu},
  title = {{Pedestrian Detection With Autoregressive Network Phases}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2019},
  url = {https://openreview.net/forum?id=-XgvxFGK7KP},
}
Figure