Combination of Fixed and Mobile Cameras for Automatic Pedestrian Detection

Alexandre Alahi, Michel Bierlaire, M. KuntInfoscience 2008

Abstract

Pedestrian detection in the surroundings of a vehicle is highly desirable to avoid dangerous traffic situations. Typical vision-based pedestrian detection algorithms on mobile cameras suffer from the lack of a-priori knowledge on the object to be detected. The variability in the shape, pose, color distribution, and behavior affect the robustness of the detection process. A novel vision-based system is proposed to detect pedestrians with a single mobile camera collaborating with a fixed camera observing the same scene. Nowadays, a large number of fixed cameras are installed in major cities. This work presents how features extracted from those fixed cameras can be used to detect pedestrians with mobile cameras present in the same scene. The proposed system outperforms state-of-the-art single frame pedestrian detectors using a feature-based classification framework. In addition, the system can be generalized to any object of interest. Any object detected by a fixed camera, can be detected with a mobile camera.

Introduction

Pedestrian detection is a cornerstone of modern driver assistance systems, yet it remains a significant challenge for mobile camera setups. The inherent noise of a moving environment, coupled with the extreme variability in human appearance (pose, clothing, and shape), often leads to high false-positive rates or missed detections. Traditional single-frame detectors typically rely on massive offline training datasets to generalize across these variations. However, they often lack specific a-priori knowledge of the immediate scene.

This research identifies a unique opportunity: the ubiquity of fixed surveillance cameras in urban environments. By leveraging these static, high-quality viewpoints as "live" priors, the proposed system guides the noisier sensors on moving vehicles, effectively turning urban infrastructure into a collaborative detection network.

Methodology

The core of the proposed approach is a "Master-Slave" architecture. Unlike traditional systems that require an extensive offline training phase, this collaborative system uses real-time detections from a fixed camera (the Master) to inform the search space of a mobile camera (the Slave).

Comparison between traditional feature-based detection and the proposed collaborative system.

Technical Components

The system utilizes several key technical innovations to ensure robust matching between disparate camera views:

  1. Region Matching Operator ($\Phi$): This algorithm identifies objects in the fixed camera's view and searches for the most similar regions within the mobile camera's frame.
  2. Covariance Matrices as Descriptors: To handle variations in lighting and viewpoint, the system employs covariance matrices. These fuse multiple features—such as intensity, spatial coordinates, and first-order derivatives—into a single descriptor that captures how features vary together, avoiding the need for complex normalization.
  3. The Dual Problem (Validation): To minimize false positives, a "back-check" validation is performed. The system takes a candidate region from the mobile camera and attempts to map it back to the original object in the fixed camera. A detection is only confirmed if these mappings coincide.
Visualization of the Region Matching Operator (a) and the Dual Problem validation process (b).
  1. Coarse-to-Fine Cascade: To maintain real-time performance, the system uses a multi-stage search. It quickly eliminates non-matching regions using coarse grids before applying more computationally intensive descriptors to high-probability candidates.

Experimental Setup

The system was evaluated using video sequences (320 x 240 pixels at 25 fps) captured simultaneously by fixed cameras mounted at first-floor height and mobile cameras (handheld or vehicle-mounted). These datasets present significant challenges, including drastic changes in viewpoint, lighting conditions, and color distribution between the two sensors.

Results and Discussion

The collaborative system demonstrated superior performance compared to contemporary state-of-the-art detectors, such as Haar-feature-based Adaboost and standard Covariance-feature detectors.

  • Detection Accuracy: The proposed system achieved a 69.7% True Positive (TP) rate, outperforming the standard single-frame detector's 57.4%.
  • False Positive Reduction: The collaborative approach significantly cleaned up the detection results. While standard detectors produced 4.32 false positives per frame, the proposed system reduced this to 0.63 FP/frame.
  • Validation Impact: The dual-problem validation operator alone was found to reduce the false positive rate of standard detectors by 83%.

A key insight is that this infrastructure-based approach provides "live" training data. Because fixed cameras can use background subtraction to reliably detect movement, they provide highly accurate references that allow mobile systems to detect pedestrians even in low-resolution or low-contrast scenarios where they would otherwise be invisible to standard algorithms.

Conclusion

This work demonstrates that collaborating with existing urban infrastructure can significantly enhance the robustness of mobile pedestrian detection. By shifting from offline training to real-time collaborative priors, the system achieves higher accuracy and fewer false alarms. Furthermore, the method is inherently generic; it can detect any object of interest—strollers, bicycles, or animals—as long as the fixed camera identifies them first.

Future work will explore bidirectional collaboration, where mobile viewpoints help resolve occlusions in fixed cameras, and the integration of behavioral modeling to further refine the matching process.

Summary updated

Abstract

Pedestrian detection in the surroundings of a vehicle is highly desirable to avoid dangerous traffic situations. Typical vision-based pedestrian detection algorithms on mobile cameras suffer from the lack of a-priori knowledge on the object to be detected. The variability in the shape, pose, color distribution, and behavior affect the robustness of the detection process. A novel vision-based system is proposed to detect pedestrians with a single mobile camera collaborating with a fixed camera observing the same scene. Nowadays, a large number of fixed cameras are installed in major cities. This work presents how features extracted from those fixed cameras can be used to detect pedestrians with mobile cameras present in the same scene. The proposed system outperforms state-of-the-art single frame pedestrian detectors using a feature-based classification framework. In addition, the system can be generalized to any object of interest. Any object detected by a fixed camera, can be detected with a mobile camera. 1
BibTeX
@inproceedings{alahi2008combination,
  author = {Alexandre Alahi and Michel Bierlaire and M. Kunt},
  title = {{Combination of Fixed and Mobile Cameras for Automatic Pedestrian Detection}},
  booktitle = {Infoscience},
  year = {2008},
}
Figure