Abstract
This study presents a comprehensive cross-modal comparison of human detection algorithms using 2D laser, 3D lidar, and RGB-D sensors in industrial environments. Addressing the scarcity of annotated 3D data, the authors propose a multi-sensor transfer learning strategy. By utilizing an RGB-D detector trained on synthetic data to generate labels for unlabeled 3D lidar data, the research demonstrates a significant performance boost in target-domain lidar detection without manual annotation.
Introduction
Reliable human detection is a cornerstone of safe and efficient mobile robotics, particularly in industrial settings like warehouses. While autonomous driving research has advanced 3D lidar-based detection, these models often struggle when transitioned to indoor environments due to domain gaps in sensor placement, clutter, and resolution. Furthermore, the high cost of manual 3D annotation for specific industrial sites remains a major barrier. This paper evaluates various sensor-algorithm combinations and introduces a method to adapt high-performance models to new domains using cross-modal supervision.
Methodology
The evaluation spans three primary modalities: 2D laser, 3D lidar, and RGB-D. While classical methods and modern deep learning are tested for 2D and 3D sensors, the core innovation lies in the Multi-Sensor Transfer Learning approach.
Multi-Sensor Transfer Learning
To bridge the domain gap for 3D lidar detectors without manual labeling, the authors employ a "teacher-student" framework. An RGB-D YOLO model, trained exclusively on synthetic data, acts as the teacher. This model predicts oriented 3D bounding boxes from RGB-D pairs. When deployed on unlabeled real-world data from the target environment, it generates "weak" labels that are then used to fine-tune a 3D lidar detector (specifically SECOND with Dynamic Voxelization).
Experimental Setup
The study utilizes a custom industrial dataset recorded in a food factory warehouse. The mobile platform is equipped with a diverse sensor suite to capture the necessary modalities for comparison.
Datasets
- Evaluation: Two challenging sequences from a warehouse: NCFM Dynamic (high density) and NCFM Storage Room (narrow aisles).
- Training: KITTI for baseline lidar models, MS COCO combined with synthetic 3D data for the RGB-D teacher, and 27,000 unlabeled frames from the target warehouse for transfer learning.
Key Results and Insights
- Performance: The RGB-D YOLO model was the top performer, achieving a 93.5% Peak-F1 score.
- Transfer Learning Efficacy: Fine-tuning the 3D lidar model using the RGB-D teacher provided a substantial boost, increasing Average Precision (AP) by up to +11.5% in narrow storage environments compared to models trained only on the outdoor KITTI dataset.
- Domain Gap: Models optimized for autonomous driving (KITTI) frequently fail indoors where humans are in close proximity to walls or equipment, highlighting the need for domain-specific adaptation.
- Efficiency vs. Accuracy: While 3D methods offer better spatial awareness, 2D laser methods (like DR-SPAAM) remain highly efficient, running at ~48 Hz on a GPU, making them suitable for resource-constrained platforms.
Conclusion
This research underscores the power of synthetic data and cross-modal transfer learning in overcoming data scarcity for industrial robotics. While RGB-D sensors provide superior accuracy, their limited field of view remains a challenge. The proposed transfer learning method allows robots to leverage the strengths of camera-based detection to improve the robustness of 360° lidar systems in complex, cluttered indoor environments.
Summary updated