A Real-Time Deep Learning Pedestrian Detector for Robot Navigation

David Ribeiro, André Mateus, Jacinto C. Nascimento, Pedro Miraldo

Abstract

A real-time Deep Learning based method for Pedestrian Detection (PD) is applied to the Human-Aware robot navigation problem. The pedestrian detector combines the Aggregate Channel Features (ACF) detector with a deep Convolutional Neural Network (CNN) in order to obtain fast and accurate performance. Our solution is firstly evaluated using a set of real images taken from onboard and offboard cameras and, then, it is validated in a typical robot navigation environment with pedestrians. The results show that our pedestrian detector is robust and fast enough to be used on robot navigation applications.

Introduction

For mobile robots to navigate safely and socially in human-populated environments—a field known as Human-Aware Navigation (HAN)—accurate and timely pedestrian detection is critical. Beyond simple collision avoidance, robots must understand social norms, such as maintaining appropriate personal space or overtaking pedestrians on the correct side.

Historically, the technical bottleneck has been the trade-off between speed and accuracy. Deep learning models, specifically Convolutional Neural Networks (CNNs), offer high accuracy but are computationally intensive, often failing to reach real-time speeds on standard mobile robot hardware. Conversely, traditional "handcrafted" feature methods are fast but prone to false positives. This paper introduces a hybrid system that bridges this gap, achieving a reliable 10 FPS detection rate on a standard CPU.

Methodology: The Hybrid Cascade

The proposed architecture utilizes a two-stage "hybrid" approach to optimize both speed and precision:

  1. Proposal Generation (ACF): The system first employs the Aggregate Channel Features (ACF) detector. As a non-deep learning method, ACF is exceptionally fast at scanning entire image frames to identify "proposals"—regions of interest likely to contain a person.
  2. CNN Refinement: To minimize computational load, the CNN (based on the VGG-VD16 architecture) only evaluates the specific proposals generated by the ACF. The input size is downscaled to 64x64 pixels to further enhance processing speed.
  3. Score Rejection Threshold: A pivotal optimization involves a score threshold. If the initial ACF scan produces a low-confidence score (below 40), the proposal is discarded immediately, sparing the CNN from processing obvious false positives.
  4. Temporal Tracking: A Kalman Filter maintains the state of detected pedestrians across frames, allowing the robot to estimate velocity and direction for smoother navigation planning.
The hybrid pedestrian detection pipeline: ACF generates proposals which are then refined by a fine-tuned CNN to produce the final result.

Experimental Setup and Datasets

The CNN was initialized via transfer learning, pre-trained on the ImageNet dataset, and subsequently fine-tuned using the INRIA Pedestrian Dataset. To validate the system in real-world robotics contexts, the authors utilized two custom datasets:

  • Corridor Dataset: Consisting of 5,556 images captured from a ceiling-mounted perspective.
  • Mbot Dataset: Consisting of 3,966 images captured from a camera mounted directly on a mobile robot platform.

Results and Discussion

The implementation of the score rejection threshold was the key to achieving real-time performance. While a standard hybrid model (evaluating all proposals) ran between 4.8 and 7.8 FPS, the optimized version reached 10.41 FPS on a CPU. This frame rate is sufficient for a robot to react to human movement in indoor settings.

The system successfully detecting pedestrians in an indoor environment, providing the high-confidence scores necessary for path planning.

Accuracy was significantly improved by the CNN stage, which effectively filtered out the false positives common in standalone ACF detectors. In live navigation tests using an MBOT platform and an A* path planner, the robot successfully identified pedestrians and replanned its trajectory to overtake them according to social navigation rules.

Conclusion

This research demonstrates that deep learning can be effectively deployed for real-time robotics by using a cascaded architecture. By using a "shallow" method to prune the search space, the "deep" model can focus its computational power on high-probability regions. Future work will look toward integrating GPU acceleration to further increase frame rates and exploring sensor fusion with LIDAR to improve robustness in challenging lighting conditions.

Summary updated

Abstract

A real-time Deep Learning based method for Pedestrian Detection (PD) is applied to the Human-Aware robot navigation problem. The pedestrian detector combines the Aggregate Channel Features (ACF) detector with a deep Convolutional Neural Network (CNN) in order to obtain fast and accurate performance. Our solution is firstly evaluated using a set of real images taken from onboard and offboard cameras and, then, it is validated in a typical robot navigation environment with pedestrians (two distinct experiments are conducted). The results on both tests show that our pedestrian detector is robust and fast enough to be used on robot navigation applications.
BibTeX
@inproceedings{ribeiro2017real,
  author = {David Ribeiro and André Mateus and Jacinto C. Nascimento and Pedro Miraldo},
  title = {{A Real-Time Deep Learning Pedestrian Detector for Robot Navigation}},
  booktitle = {ICARSC},
  year = {2017},
  url = {https://openreview.net/forum?id=N-N228ki0V},
}
Figure