Abstract
Autonomous search-and-rescue in subterranean environments is hampered by "perceptually degraded" conditions, including darkness, dust, and smoke. This paper presents a multi-modal sensing strategy developed by researchers at ETH Zurich that leverages LiDAR intensity and depth data to generate object proposals. These proposals guide a Pan-Tilt-Zoom (PTZ) camera to perform high-resolution classification at distances significantly exceeding the range of static cameras. Experimental results from the DARPA Subterranean Challenge and underground mines demonstrate extended detection ranges and robust performance in challenging conditions.
Introduction
In search-and-rescue missions, robots must identify critical artifacts such as backpacks, fire extinguishers, or survivors. However, subterranean environments often present extreme visual challenges—pitch darkness or obscuring dust. While standard visual cameras offer high resolution, they are limited by low light and fixed fields of view. Conversely, LiDAR provides long-range spatial awareness in the dark but produces sparse data that makes object identification difficult.
To bridge this gap, the proposed system uses LiDAR to handle the "where" (spatial localization of potential objects) and a PTZ camera to handle the "what" (high-resolution identification). This directed search strategy allows the robot to focus its computational and optical resources on high-probability areas rather than scanning entire environments blindly.
Methodology
The system follows a multi-stage pipeline to transform raw sensor data into confirmed object classifications.
1. Point Cloud Processing
Because single LiDAR scans (e.g., from a Velodyne VLP-16) are sparse, the system aggregates multiple scans within a sliding window to create a denser 3D representation. This 3D data is then projected into 2D images representing range, intensity, and surface normals. A Gaussian kernel is applied to smooth noise and fill gaps in the projected data.
2. Object Segmentation and Filtering
The segmentation process begins by removing ground points. Remaining points are clustered based on depth disparity and LiDAR intensity. A key innovation is the use of intensity; materials like fabric reflect laser light differently than cave walls, providing a "pseudo-visual" signal. Clusters are further filtered based on volume and the standard deviation of surface normals, as man-made objects typically exhibit higher geometric complexity than flat tunnel walls.
3. Directed Search and Classification
Once a cluster is identified as a proposal, the system calculates a 3D waypoint. The PTZ camera orients itself toward this target and adjusts its zoom level based on the estimated distance. Finally, a YOLO-based CNN object detector processes the high-resolution image to confirm the artifact type.
Experimental Evaluation
The system was evaluated in three diverse environments: the Salt-Peter Cave (USA), the Seemuhle Underground Mine (Switzerland), and the DARPA SubT Challenge Finals (Louisville Mega Cavern).
Key Results
- Extended Detection Range: In the Seemuhle mine, the system detected backpacks at ranges of 5–10 meters, compared to just 3.5 meters for static cameras. In the DARPA finals, detection range for vents increased from 1.5m to 5m.
- Precision and Robustness: The system achieved 85.3% precision in the Seemuhle mine. While precision dropped to 53.4% in the complex DARPA finals due to non-target man-made objects (like junction boxes), the system remained effective at identifying true artifacts.
- Signal Utility: Ablation studies confirmed that incorporating LiDAR intensity and surface normal data significantly reduced false-positive proposals compared to using depth data alone.
Discussion and Future Work
The research highlights that LiDAR intensity is an underutilized signal for object segmentation in dark environments. However, challenges remain regarding intensity calibration across different sensors and the detection of low-reflectivity objects like ropes. Future work aims to integrate the proposal generator more tightly with the robot's path planner, allowing for autonomous "active investigation" of high-probability targets.
Summary updated