Detecting and Mapping Trees in Unstructured Environments with a Stereo Camera and Pseudo-Lidar

Brian H. Wang, Carlos Diaz-Ruiz, Jacopo Banfi, Mark E. CampbellICRA 2021 · DOI

TL;DR

This paper introduces a novel method for detecting and mapping trees in challenging outdoor environments using only stereo cameras. By leveraging a "pseudo-lidar" representation and a learned 3D object detector, the system achieves robust tree detection and tracking at ranges up to 7 meters, overcoming limitations of traditional stereo-based mapping.

What Problem Does This Paper Address?

Autonomous navigation in complex, unstructured environments like forests is a significant challenge for robots. While lidar sensors offer accurate 3D perception, they are often heavy, power-hungry, and expensive, making them unsuitable for many platforms like drones. Stereo cameras are a more accessible alternative, but their depth estimation accuracy degrades significantly with distance, limiting their effective range and thus the robot's ability to plan ahead and navigate safely at higher speeds. Existing methods often discard stereo data beyond a certain range or rely on lidar for accurate tree detection and mapping, leaving a gap for cost-effective, stereo-only solutions that can perceive distant obstacles.

Methods

The core of this work lies in a three-pronged approach:

  1. Automatic Training Data Generation: This is a key innovation. Instead of relying on expensive, manually labeled lidar data, the authors propose a pipeline to generate training data from stereo camera point clouds. This involves:
  2. Data Collection: Capturing stereo images and IMU data.
  3. Global Point Cloud Fusion: Using a visual-inertial SLAM system (Stereolabs ZED SDK) to fuse individual stereo point clouds into a single, consistent global point cloud map.
  4. Clustering: Applying DBSCAN to the global point cloud to identify individual tree clusters. This step is robust to noise in individual stereo frames because the fused global cloud has significantly reduced noise.
  5. Label Generation & Sparsification: Transforming the global tree clusters back into the local camera frame for each stereo frame and fitting 3D bounding boxes. Crucially, the stereo point clouds are then sparsified into a "pseudo-lidar" representation, mimicking lidar data for compatibility with existing 3D object detectors.
  6. 3D Pseudo-Lidar Object Detection: The generated pseudo-lidar point clouds are fed into a pre-trained PointRCNN 3D object detector. This detector, originally designed for lidar, is adapted to recognize trees based on the pseudo-lidar representation.
  7. Kalman Filter State Estimation: To improve robustness and track trees over time, a Kalman filter is employed. It takes the 3D bounding box detections from PointRCNN as measurements and estimates the consistent position and size of each tree. This helps to handle missed detections and refine estimates.

Datasets

  • Training Data: The authors collected a dataset consisting of 8680 stereo point clouds from forest-like environments near Cornell University. This data was processed using their novel automatic labeling pipeline.
  • Test Data: A separate test sequence of 2000 stereo point clouds was captured from a different forest-like environment to evaluate the system's performance outside the training domain.

Key Insights and Ideas

  • Pseudo-Lidar for Trees: The paper demonstrates that the pseudo-lidar representation, successful in autonomous driving for object detection, is also highly effective for detecting trees in unstructured environments, even with noisy stereo data.
  • Global Fusion for Robust Labeling: Fusing individual stereo point clouds into a global map is critical for robustly identifying and labeling trees. This global map acts as a "ground truth" for training, overcoming the inherent noise and distortion in individual stereo frames at longer ranges.
  • Stereo-Only Solution: The method successfully bypasses the need for lidar sensors, making it more accessible and cost-effective for platforms like drones.
  • Bridging the Range Gap: By enabling detection of trees at ranges up to 7 meters, the system significantly extends the usable perception range compared to traditional stereo methods that often discard data beyond 3-5 meters.

Key Results

  • The system achieves robust tree detection and mapping at ranges up to 7 meters, even when trained on data generated by a clustering method that is only accurate up to ~3-4 meters in noisy stereo point clouds.
  • The DBSCAN baseline (clustering directly on noisy stereo point clouds) shows significant degradation in accuracy and a rapid increase in false positives beyond 5 meters.
  • The proposed detector, in contrast, exhibits a low false positive rate and maintains accuracy at longer ranges.
  • The Kalman filter further reduces false negatives and refines estimates, especially at longer distances.
  • The system's bounding box estimates closely match ground truth positions and sizes, with significantly lower errors than the DBSCAN baseline at longer ranges.

Limitations and Future Work

  • Short-Range Detection: The detector tends to miss trees at very short ranges. The authors attribute this to the density of the point cloud and the detector's design, which was initially optimized for lidar. They suggest that integrating local grid-based maps could compensate for this.
  • Forest Environment Assumption: The current pipeline assumes all obstacles are trees. Future work could extend the labeling process to support multiple object classes, enabling multi-class 3D object detectors.
  • Extending Detector Range: The authors aim to further extend the range of learned 3D object detectors for even more distant obstacle awareness.

Summary updated

BibTeX
@inproceedings{wang2021detecting,
  author = {Brian H. Wang and Carlos Diaz-Ruiz and Jacopo Banfi and Mark E. Campbell},
  title = {{Detecting and Mapping Trees in Unstructured Environments with a Stereo Camera and Pseudo-Lidar}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2021},
  doi = {10.1109/icra48506.2021.9562056},
}
Figure