Abstract: Pedestrian detection is an important but challenging problem in computer vision, especially in human-centric tasks. Over the past decade, significant improvement has been witnessed with the help of handcrafted features and deep features. Here we present a comprehensive survey on recent advances in pedestrian detection. First, we provide a detailed review of single-spectral pedestrian detection that includes handcrafted features based methods and deep features based approaches. For handcrafted features based methods, we present an extensive review of approaches and find that handcrafted features with large freedom degrees in shape and space have better performance. In the case of deep features based approaches, we split them into pure CNN based methods and those employing both handcrafted and CNN based features. We give the statistical analysis and tendency of these methods, where feature enhanced, part-aware, and post-processing methods have attracted main attention. In addition to single-spectral pedestrian detection, we also review multi-spectral pedestrian detection, which provides more robust features for illumination variance. Furthermore, we introduce some related datasets and evaluation metrics, and a deep experimental analysis. We conclude this survey by emphasizing open problems that need to be addressed and highlighting various future directions.
Introduction
Pedestrian detection is a cornerstone of computer vision, essential for autonomous driving, robotics, and intelligent surveillance. Unlike generic object detection, it faces unique hurdles such as extreme scale variation, heavy occlusion in crowded urban environments, and the highly deformable nature of the human body. This survey by Jiale Cao and colleagues provides a structured roadmap of the evolution of pedestrian detection over the last two decades, tracing the field's trajectory from manual feature engineering to the current dominance of deep convolutional neural networks (CNNs).
Methodological Evolution
The authors categorize the literature into three main eras:
- Handcrafted Features: Early methods relied on "Channel Features" (like HOG and LUV color channels) and Deformable Part Models (DPM). A key insight from this era is the "freedom degree" in shape and space; methods that allowed for more spatial flexibility generally performed better.
- Deep Features (Hybrid vs. Pure CNN):
- Hybrid: These models use CNNs for feature extraction but retain "shallow" classifiers like boosted decision forests or SVMs.
- Pure CNN: End-to-end architectures (e.g., Faster R-CNN, SSD) learn proposals, classification, and bounding-box regression simultaneously.
- Specialized Deep Architectures: Modern research focuses on scale-aware models (using Feature Pyramid Networks), part-based models for occlusion handling, and multispectral fusion (RGB + Thermal) for low-light robustness.
Benchmarking and Datasets
The survey covers the transition from small, early datasets to massive, diverse modern benchmarks:
- Early Benchmarks: MIT, INRIA, and Caltech (the long-standing gold standard).
- Modern Benchmarks: CityPersons (high diversity), KITTI (autonomous driving focus), and CrowdHuman (dense, occluded scenes).
- Multispectral: KAIST and CVC-14, which provide aligned visible and thermal image pairs.
Performance Analysis and the Occlusion Bottleneck
A major finding of the survey is the performance gap between ideal and challenging conditions. While performance on "Reasonable" (mostly unoccluded) test sets is nearing saturation with log-average miss rates under 5%, performance on "Heavy Occlusion" (HO) sets remains a significant bottleneck.
Key Insights
- The "Freedom Degree" Principle: Successful handcrafted models moved away from rigid templates toward features that could adapt to different human shapes.
- Pre-training Specificity: Pre-training on a person-related dataset (like CrowdHuman) yields significantly better results than generic pre-training on COCO.
- Modality Imbalance: In multispectral detection, simply concatenating RGB and Thermal data is insufficient; models must explicitly handle the imbalance where one sensor may be noisier than the other.
- Backbone Impact: HRNet is identified as a strong backbone because it maintains high-resolution representations, critical for detecting small-scale pedestrians.
Future Directions
Despite the leap from ~60% miss rates to under 3% on standard benchmarks, several open problems remain:
- Real-time Constraints: Many high-performing models are too slow for real-time deployment on edge devices.
- Domain Adaptation: Models still struggle to generalize across different cities or weather conditions.
- NMS-Free Detection: Moving toward end-to-end "one proposal, one prediction" models (like DETR) is a promising direction for handling crowded scenes without traditional Non-Maximum Suppression (NMS).
Summary updated