Abstract
We present an autoregressive pedestrian detection framework with cascaded phases designed to progressively improve precision. The proposed framework utilizes a novel lightweight stackable decoder-encoder module which uses convolutional re-sampling layers to improve features while maintaining efficient memory and runtime cost. Unlike previous cascaded detection systems, our proposed framework is designed within a region proposal network and thus retains greater context of nearby detections compared to independently processed RoI systems. We explicitly encourage increasing levels of precision by assigning strict labeling policies to each consecutive phase such that early phases develop features primarily focused on achieving high recall and later on accurate precision. In consequence, the final feature maps form more peaky radial gradients emulating from the centroids of unique pedestrians. Using our proposed autoregressive framework leads to new state-of-the-art performance on the reasonable and occlusion settings of the Caltech pedestrian dataset, and achieves competitive state-of-the-art performance on the KITTI dataset.
Introduction
Detecting pedestrians in urban environments remains a significant challenge for computer vision systems due to heavy occlusion, extreme scale variations, and crowded scenes. Traditional detection architectures typically fall into two categories: ensembles and recurrent networks. Ensembles utilize multiple independent networks to learn diverse features but suffer from high computational and memory costs. Recurrent networks are more efficient but often employ shared weights across iterations, which can restrict the network's ability to learn specialized features for different stages of refinement.
AR-Ped introduces a "best of both worlds" approach through an autoregressive framework. It iteratively refines detections like a recurrent network but utilizes non-shared weights to allow for specialized feature learning at each phase, similar to an ensemble, while maintaining the efficiency required for real-world applications.
The AR-Ped Framework
The core of the system is the AR-RPN (Autoregressive Region Proposal Network). Unlike standard RPNs that perform a single inference pass, AR-RPN consists of a sequence of phases that progressively sharpen the detection focus.
Autoregressive Flow and Specialized Features
In the proposed autoregressive flow, each phase is conditioned on both the feature maps and the classification predictions of the preceding phase. Crucially, weights are not shared between these phases. This architectural choice allows early phases to focus on high recall—ensuring no pedestrians are missed—while later phases specialize in high precision, filtering out false positives and refining bounding box accuracy.
Efficient De-encoder Module
To facilitate the connection between phases without incurring massive overhead, the authors developed a stackable "de-encoder" module. Traditional architectures often use generic two-step re-sampling (e.g., bilinear upsampling followed by a convolution). AR-Ped replaces these with convolutional re-sampling layers (strided and fractional-strided convolutions) that perform feature generation and re-sampling simultaneously. This integration results in a 41% reduction in GPU memory usage and a 16% speedup during inference compared to standard approaches.
Incremental Supervision
A key innovation is the application of increasingly strict labeling policies across consecutive phases. For instance, Phase 1 may define a positive detection using a 0.4 Intersection over Union (IoU) threshold, while Phase 3 requires a 0.6 IoU. This incremental supervision forces the network to produce classification maps that are increasingly "peaky," where high-confidence scores are tightly clustered around the pedestrian's centroid. This characteristic significantly improves the effectiveness of Non-Maximum Suppression (NMS) in removing duplicate detections.
Experimental Results
AR-Ped was evaluated on the Caltech Pedestrian Dataset (using both original and "New" annotations) and the KITTI Benchmark to test generalizability across different urban environments.
Performance Benchmarks
The full AR-Ped system achieved state-of-the-art results on the Caltech "Reasonable" setting with a miss rate of 6.45%, outperforming previous leading models such as SDS-RCNN (7.36%). The model also demonstrated superior robustness in handling partial and heavy occlusions, which are common failure points for general-purpose detectors.
Efficiency and Runtime
Despite its multi-phase nature, AR-Ped remains computationally efficient. It maintains a runtime of 91ms per image on a Titan X GPU, making it competitive with other high-performance frameworks while delivering superior precision.
Conclusion and Future Work
AR-Ped demonstrates that an autoregressive approach with non-shared weights and incremental supervision can significantly advance pedestrian detection. While the model excels in most scenarios, performance on "Heavy Occlusion" still trails behind models specifically engineered with complex occlusion-handling components. Furthermore, the authors noted diminishing returns when adding a fourth phase, suggesting that extremely high IoU thresholds may require denser anchor sampling to provide sufficient positive training examples. Future research will focus on integrating explicit occlusion modeling and exploring denser sampling strategies to further push the boundaries of detection precision.
Summary updated