Three-Dimensional Point Cloud Segmentation Algorithm Based on Depth Camera for Large Size Model Point Cloud Unsupervised Class Segmentation

Kun Fang, Kaiming Xu, Zhigang Wu, Tengchao Huang, Yubang YangSensors 2023 · DOI

TL;DR

This paper introduces an unsupervised 3D point cloud segmentation algorithm that leverages depth camera data and a voxelization technique to efficiently process large-scale point clouds. The method achieves high accuracy and speed by segmenting voxels based on density and distance, outperforming existing state-of-the-art methods on benchmark datasets.

What Problem Does This Paper Address?

The increasing use of 3D point cloud data in fields like robotics and autonomous vehicles presents a significant challenge due to the sheer size of these datasets. Efficiently processing and segmenting these large point clouds into meaningful parts is crucial for downstream applications. While supervised methods exist, they require extensive labeled data. Unsupervised methods aim to overcome this limitation but often struggle with the complexity and scale of real-world 3D environments. This paper specifically tackles the problem of unsupervised segmentation of large-scale point clouds acquired from depth cameras, aiming for both high accuracy and computational efficiency.

Methods

The proposed algorithm employs a multi-stage approach:

  • Depth Information Extraction & Pre-processing: The core of the method relies on depth information captured by a depth camera. This raw data is pre-processed to extract depth information and construct a distance matrix between points. A key step here is voxelization, which reduces the point cloud's size by grouping points into cubic voxels. This significantly speeds up processing for large datasets.
  • Normalization: To ensure fair comparison and prevent bias in clustering, both Euclidean distances between points and voxel density are normalized to a standardized range.
  • GrowCut Segmentation: The algorithm adapts the GrowCut algorithm, traditionally used for 2D image segmentation, for 3D point clouds. It uses seed points (selected based on geometric properties) to iteratively "grow" segments by assigning neighboring points to the nearest seed point, with boundaries defined by significant changes in depth information.
  • Propagation: To refine the initial segmentation, a propagation step is introduced. This step further "grows" segments from seed points, considering both Euclidean distance and a depth similarity threshold to ensure consistency. This process continues until no point changes its segment assignment.
  • Optimization (Voxel-based): The voxelization technique is central to the optimization strategy. By operating on voxels instead of individual points, the computational complexity is dramatically reduced, making the algorithm suitable for large-scale data.
  • Post-processing: A Conditional Random Field (CRF) is applied to smooth the segmentation results, leveraging spatial coherence to improve accuracy and refine segment boundaries.

The novelty lies in the combination of depth camera data, voxelization for efficiency, and the adaptation of the GrowCut algorithm for unsupervised segmentation, specifically tailored for large-scale point clouds.

Datasets

The paper evaluates the proposed method on several widely-used 3D point cloud datasets:

  • ScanNet: Primarily indoor scenes.
  • S3DIS: Large-scale indoor scenes.
  • Semantic3D: Outdoor urban landscapes.

The authors also mention using their "own benchmark dataset" for which they report an average IoU of 90.2%.

Key Insights and Ideas

  • Depth is Key for Unsupervised Segmentation: The paper strongly emphasizes that leveraging depth information from cameras is a powerful, often underutilized, feature for unsupervised segmentation, especially for discerning object boundaries and spatial relationships.
  • Voxelization as a Scalability Enabler: For large-scale point clouds, voxelization isn't just a simplification; it's a fundamental strategy to make segmentation computationally feasible without sacrificing too much detail.
  • Adapting 2D Techniques to 3D: The successful adaptation of the GrowCut algorithm, a well-established 2D image segmentation technique, to the 3D point cloud domain highlights the potential for cross-domain algorithm transfer.
  • Balancing Accuracy and Speed: The core achievement is demonstrating that high segmentation accuracy doesn't necessarily come at the cost of prohibitive computation time, especially for large datasets.

Key Results

  • Superior Accuracy: The proposed method consistently outperforms state-of-the-art methods like PointNet, PointNet++, and SGPN across all tested datasets and metrics (OA, MCA, mIoU).
  • On ScanNet, the method achieved an mIoU of 69.3%, an improvement of 4.5 percentage points over SGPN.
  • On S3DIS, the mIoU was 75.9%, surpassing SGPN by 5.3 percentage points.
  • On Semantic3D, the mIoU reached 68.7%, an improvement of 6.6 percentage points over SGPN.
  • Fast Segmentation Speed: The algorithm processes a point cloud with 1 million points in approximately 1.85 seconds, a significant speedup compared to the typical 5-10 seconds required by other methods.
  • High Performance on Own Benchmark: An average Intersection over Union (IoU) of 90.2% was achieved on their own benchmark dataset.

Limitations and Future Work

  • Challenges with Clutter and Density: The authors acknowledge that segmenting highly cluttered and densely populated point cloud scenes was a primary challenge. Initially, the algorithm struggled to differentiate closely spaced objects with similar geometric and textural characteristics.
  • Refinements for Complex Scenarios: To address these challenges, they refined feature extraction techniques and depth resolution processing, which significantly improved segmentation accuracy in complex environments.
  • Future Work: Not explicitly detailed, but the success in handling large-scale data and achieving high accuracy suggests potential for further applications in robotics, AR/VR, and autonomous systems where real-time, accurate 3D scene understanding is critical. The paper's focus on unsupervised learning also points towards future work in reducing reliance on labeled data for even more complex segmentation tasks.

Summary updated

Abstract

This paper proposes a 3D point cloud segmentation algorithm based on a depth camera for large-scale model point cloud unsupervised class segmentation. The algorithm utilizes depth information obtained from a depth camera and a voxelization technique to reduce the size of the point cloud, and then uses clustering methods to segment the voxels based on their density and distance to the camera. Experimental results show that the proposed algorithm achieves high segmentation accuracy and fast segmentation speed on various large-scale model point clouds. Compared with recent similar works, the algorithm demonstrates superior performance in terms of accuracy metrics, with an average Intersection over Union (IoU) of 90.2% on our own benchmark dataset.
BibTeX
@inproceedings{fang2023three,
  author = {Kun Fang and Kaiming Xu and Zhigang Wu and Tengchao Huang and Yubang Yang},
  title = {{Three-Dimensional Point Cloud Segmentation Algorithm Based on Depth Camera for Large Size Model Point Cloud Unsupervised Class Segmentation}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s24010112},
}
Figure