TL;DR
This paper presents a robust human detection system for mobile robots operating in large outdoor environments by fusing data from laser range finders and cameras. They introduce a novel adaptive projection technique to improve image region selection and a new laser feature set that generalizes better to varying conditions, leading to more reliable detection and improved robustness.
What Problem Does This Paper Address?
Surveillance of large outdoor infrastructures is a critical task, but current systems often rely on static sensors and human supervision, which can be overwhelming and prone to fatigue. Mobile robots offer a promising solution for autonomous surveillance, but detecting humans from a moving platform presents significant challenges. These include noisy outdoor environments, varying illumination, weather conditions, and the computational constraints of real-time processing. Existing methods often struggle with generalization across different scenes and are susceptible to platform vibrations and terrain irregularities, leading to misalignments and false detections. This paper aims to address these challenges by developing a robust and computationally efficient human detection system for unmanned ground vehicles (UGVs) that leverages the complementary strengths of laser and vision sensors.
Methods
The proposed system employs a sensor fusion approach, combining data from a laser range finder and a monocular camera.
- Laser Detection Module:
- Novel Feature Set: A new feature set is introduced that aims to better encapsulate variations due to noise, distance, and human pose. This set is designed to be scene-independent and includes normalization techniques to improve generalization.
- Segmentation and Clustering: Laser data is segmented into clusters, and a second aggregation step is used to ensure a single cluster per person, even in outdoor conditions prone to oversegmentation.
- Classification: A Real AdaBoost classifier, using twenty two-split decision trees as weak classifiers, is employed for detection.
- Vision Detection Module:
- Histogram of Oriented Gradients (HoG): This standard descriptor is used for feature extraction from image regions.
- Adaptive ROI Projection: A novel technique is proposed to address misalignment issues caused by platform vibrations and terrain. Instead of a fixed-size window, it uses a pyramid of meaningful sizes and an overlapping sliding window to search for human presence, compensating for scale and improving the operational range.
- Classification: A linear Support Vector Machine (SVM) classifier is used to evaluate the HoG descriptors.
- Information Fusion:
- Calibrated Probabilities: Both the AdaBoost and SVM outputs are calibrated using Platt's sigmoid scaling method to obtain accurate probability distributions.
- Fusion Techniques: Three fusion techniques are evaluated: maximum, average, and Bayesian-average, to combine the calibrated probabilities from the laser and vision modules.
Datasets
The authors used a combination of proprietary and publicly available datasets:
- Proprietary Data: Collected by driving the Summit XL robot on the campus of the Polytechnic University of Madrid. Three datasets were gathered:
-
Gymnasium (static and moving scenarios)
- Outdoors (moving scenario) These datasets consist of time-synchronized laser scans and image frames.
-
Public Databases (for Vision Module):
- MIT pedestrian image database (924 frontal/rear view images)
- INRIA person database (2,416 positive, 1,218 negative training images; 1,126 positive, 453 negative test images)
Key Insights and Ideas
- Adaptive Projection is Crucial for Robustness: The adaptive ROI projection method significantly outperforms fixed-size projections, especially in challenging outdoor environments. It increases the operational range and robustness by evaluating more clusters, including those at greater distances.
- Distance-Independent Features Excel Outdoors: For outdoor environments, features that are independent of distance (like the proposed feature set) generalize better than those that rely on distance. In indoor settings, distance-based features can be highly discriminative due to confined surroundings.
- Fusion Significantly Improves Performance: Combining laser and vision data consistently leads to better detection rates than using either sensor individually, particularly in challenging scenarios like occlusions.
- Calibrated Probabilities are Essential for Fusion: Accurately converting classifier scores into probability distributions is vital for effective probabilistic fusion.
- Computational Efficiency is Maintained: Despite the increased complexity of the adaptive projection and the larger feature set, the system maintains real-time performance due to efficient processing and the ability of AdaBoost to select relevant features.
Key Results
- Laser Module: The proposed distance-independent feature set achieved better performance outdoors than distance-dependent features, showing an improvement of over 3.5% in recall and 2% in precision. In gymnasium experiments, the "Basis + DIST" set performed slightly better in recall and precision.
- Vision Module: The adaptive projection method showed a significant improvement of over 4.3% in performance compared to fixed-size projection.
- Fusion Results:
- Gymnasium (Fixed-size projection): Fusion methods achieved high recall (e.g., 98.15% for Maximum) and specificity (e.g., 99.01% for Mean/Bayes).
- Gymnasium (Adaptive projection): Fusion methods further improved recall (e.g., 99.27% for Maximum) and specificity (e.g., 99.76% for Mean/Bayes).
- Outdoors (Fixed-size projection): Fusion significantly boosted recall (e.g., 91.17% for Maximum) and precision (e.g., 99.63% for Mean/Bayes).
- Outdoors (Adaptive projection): Fusion provided the most substantial gains, with recall boosted by over 18.5% (13.5% precision, 3.2% specificity) with the adaptive method. Bayesian fusion consistently offered strong results, especially in the outdoor experiments.
- Qualitative Results: Figure 10 demonstrates the system's ability to detect humans in complex scenarios with multiple people, occlusions, and varying distances, highlighting cases where individual sensors failed but fusion succeeded.
Limitations and Future Work
- Detection Range: The detection range is ultimately limited by the vision sensor's resolution.
- Terrain and Acceleration: While the adaptive projection method addresses uneven terrain and rapid accelerations to some extent, it cannot fully compensate for extreme conditions.
- Future Work:
- Integrating the fusion detector within a tracking framework for human following.
- Exploiting vision data to distinguish between different tracked individuals.
- Extending the method for use with distributed multi-robot systems.
Summary updated