Abstract
It remains very challenging to build a pedestrian detection system for real world applications, which demand for both accuracy and speed. This work presents a novel hierarchical knowledge distillation framework to learn a lightweight pedestrian detector, which significantly reduces the computational cost and still holds the high accuracy at the same time. Following the 'teacher--student' diagram that a stronger, deeper neural network can teach a lightweight network to learn better representations, we explore multiple knowledge distillation architectures and reframe this approach as a unified, hierarchical distillation framework. In particular, the proposed distillation is performed at multiple hierarchies, multiple stages in a modern detector, which empowers the student detector to learn both low-level details and high-level abstractions simultaneously. Experiment result shows that a student model trained by our framework, with 6 times compression in number of parameters, still achieves competitive performance as the teacher model on the widely used pedestrian detection benchmark.
Introduction
Pedestrian detection is a cornerstone of autonomous driving and smart surveillance, but it faces a classic "speed vs. accuracy" dilemma. State-of-the-art detectors rely on massive deep neural networks that are too computationally expensive for real-time use on edge devices.
While "Knowledge Distillation" (KD)—a process where a small student network learns to mimic a large teacher network—is a common solution, existing methods for object detection often fall short. Most previous KD approaches only focus on the final output or high-level features. This causes the student to lose critical spatial information, making it struggle to detect pedestrians of varying sizes or those partially obscured.
Proposed Hierarchical Distillation Framework
The authors propose a unified hierarchical knowledge distillation framework based on a Faster R-CNN architecture with a Feature Pyramid Network (FPN). They use a ResNet50 as the "Teacher" and a much smaller ResNet18 as the "Student."
The core innovation lies in three distinct layers of distillation:
- Pyramid Distillation (PD): Instead of just looking at the final feature map, the student mimics the teacher across all levels of the feature pyramid ($P_2$ through $P_5$). This ensures the student learns both low-level textures (edges, contours) and high-level abstractions (body parts).
- Region Distillation (RD) with Pyramid RoIAlign: Standard detectors pick one level of a pyramid to extract features for a specific object. Here, the authors introduce "Pyramid RoIAlign," which extracts and concatenates features from all levels of the pyramid for every region of interest. The student is then trained to match the teacher’s combined multi-level region features.
- Logit Distillation (LD): The student mimics the teacher’s final predictions (the "logits") just before the classification and bounding box output.
Key Insights and Ideas
- Spatial Detail Matters: For pedestrians, high-level semantics aren't enough. Because pedestrians appear at many different scales, the student needs explicit supervision on low-level spatial features to handle small or distant targets.
- The Information Bottleneck: The authors found that region features (RD) provide much more useful "knowledge" than final logits (LD). This is because logits are heavily compressed through fully connected layers, losing the rich structural information present in the intermediate feature maps.
- Wider Receptive Fields: By using Pyramid RoIAlign to pull from all levels of the feature pyramid simultaneously, the student gains a wider receptive field, allowing it to "see" the context around a pedestrian more effectively.
Experimental Results
The framework was evaluated on the Caltech Pedestrian Dataset, using the "Caltech10×" set for training and reporting Log-Average Miss Rate (MR) on the "Reasonable" and "Small" subsets.
- Compression: The student model (ResNet18) has only 11 million parameters, compared to the teacher’s 68 million—a roughly 6x reduction in size.
- Accuracy: The student achieved a 10.03% Miss Rate (MR) on the Caltech "Reasonable" set. This is remarkably close to the teacher’s 9.29% and significantly better than the baseline student (12.52%) without distillation.
- Competitive Performance: Despite its small size, the distilled ResNet18 outperformed several much larger state-of-the-art models (like CompACT-Deep and UDN+SS) that use VGG-16 backbones with over 138 million parameters.
Conclusion
Hierarchical knowledge distillation effectively bridges the performance gap between heavy-duty models and mobile-ready detectors. By forcing a small student network to mimic a large teacher network across multiple levels—from raw spatial details to complex semantic concepts—the proposed framework achieves significant compression without sacrificing accuracy. Future work includes evaluating specific frames-per-second (FPS) benchmarks on mobile hardware and adapting this hierarchical approach to one-stage detectors like YOLO or SSD.
Summary updated