Investigation on Rock Segmentation and 3D Surface Evaluation Based on Binocular Vision

Hongwei Gao, Ben Niu, Bin Li, Yang YuInternational Journal of Intelligent Engineering and Systems 2010 · DOI

Abstract

Planetary exploration rovers must autonomously identify and evaluate rock surfaces for sampling. This paper investigates a suite of algorithms for rock segmentation, shadow elimination, stereo matching, and 3D reconstruction. By utilizing C-means clustering on surface normals derived from binocular vision, the system identifies optimal grinding planes. Experimental results using real-world images demonstrate the validity and precision of this approach, achieving sub-millimeter measurement accuracy.

Introduction

In planetary exploration, "sample and return" missions are critical. Rovers must navigate debris fields, identify target rocks, and find flat surfaces suitable for grinding or sampling tools. Harsh lighting and communication constraints (often limiting rovers to grayscale imagery) make this a significant challenge. This research presents an end-to-end computer vision pipeline to bridge the gap between raw binocular images and precise robotic arm guidance.

Methodology

1. Shadow Elimination and Rock Segmentation

To prevent mismatches in 3D reconstruction caused by textureless shadows, the system uses intensity thresholds derived from ground experiments to mask shadow pixels. The process often begins with a "human-computer interaction" phase where a region of interest is manually selected to focus the algorithm's resources.

2. Stereo Matching via Graph Cuts

Building a 3D model requires finding corresponding points in two camera views. The authors employ a Graph Cut algorithm, treating stereo matching as a global optimization problem. This approach is significantly more robust to noise than local correlation methods, as it considers the smoothness of the resulting disparity map.

Graph Cut optimization logic for stereo matching

3. 3D Reconstruction and Surface Evaluation

Using the Tsai two-step calibration and Radial Alignment Constraint (RAC) model, 2D pixel disparities are converted into a 3D point cloud. Delaunay triangulation then generates a continuous mesh of small triangles from these points.

The core innovation lies in the surface evaluation. Instead of clustering 3D coordinates (X, Y, Z), the algorithm calculates the normal vector for every triangle in the mesh. An advanced C-means clustering algorithm groups these vectors. Areas where large groups of triangles share nearly identical normals are identified as "flat planes" suitable for robotic interaction, regardless of their absolute position on the rock.

Geometric model for camera location and rock surface data points

Key Results and Discussion

Experimental validation was conducted using a Bumblebee2 stereo camera in a laboratory setting. The system achieved an average measuring error of less than 1mm across all three axes (X, Y, Z). In primary test cases, the algorithm successfully identified multiple distinct grinding planes on a single rock surface. By applying epipolar line rectification and edge constraints, the system efficiently filtered raw data, reducing a set of over 23,000 points down to approximately 21,000 high-confidence points for the final mesh.

Conclusion

The proposed pipeline demonstrates that clustering surface normals is a highly effective method for identifying flat surfaces on irregular rocks. While the current system benefits from manual initialization for rock selection, it provides a robust foundation for autonomous planetary sampling. Future work will focus on full automation and adapting to the specific atmospheric lighting conditions of the lunar or Martian surface.

Summary updated

Abstract

Planetary exploration is the embodiment of a nation's synthesis science and technology power. The first step is to land a rover to explore the unknown world. The science task always begins from rock and soil. Rock sample and return is one of the important tasks of planetary exploration, which should be accomplished by the rover's hardware and software. A series of algorithms related to main science exploring object-rock are investigated in this paper, such as a single rock's segmentation, shadow elimination, stereo matching and 3D reconstruction. C-means clustering and 3D surface evaluation realized by surrounding binocular vision are described. On the basis of the above algorithms, a virtual mechanism is developed by VC++ and OpenGL. The evaluated information for localization is transformed to the arm and leads the arm to locate the relative plane on the rock and shows the grinding process. The experiments and simulation results based on real image show the validity of these algorithms.
BibTeX
@article{gao2010investigation,
  author = {Hongwei Gao and Ben Niu and Bin Li and Yang Yu},
  title = {{Investigation on Rock Segmentation and 3D Surface Evaluation Based on Binocular Vision}},
  journal = {International Journal of Intelligent Engineering and Systems},
  year = {2010},
  doi = {10.22266/ijies2010.0930.01},
}
Figure