Spherical Interpolated Convolutional Network With Distance-Feature Density for 3-D Semantic Segmentation of Point Clouds

Guangming Wang, Yehui Yang, Huixin Zhang, Zhe Liu, Hesheng WangIEEE Trans. Cybernetics 2022 · OpenReview

TL;DR

This paper introduces a novel "spherical interpolated convolution" operator designed to overcome the limitations of traditional 3D convolutions for semantic segmentation of unstructured point clouds. By incorporating a "distance-feature density" mechanism, their method achieves more accurate and robust feature extraction, leading to improved performance with fewer parameters.

What Problem Does This Paper Address?

Semantic segmentation of 3D point clouds is crucial for applications like robotics and autonomous driving, enabling machines to understand their environment. However, the irregular, unstructured nature of point clouds makes it challenging to directly apply standard 3D convolutional neural networks (CNNs), which are designed for grid-like data (e.g., images). Existing methods often struggle with varying point densities, leading to inaccurate feature extraction. For instance, traditional grid-based convolutions can be inefficient and require many parameters, while interpolation methods can be biased by dense regions or fail to capture features in sparse areas.

Methods

The core innovation is a Spherical Interpolated Convolutional Network (SIConv). Instead of using rigid, grid-based kernels, SIConv employs a spherical kernel that is more naturally suited to the 3D space. This kernel is constructed using a hierarchical, cubic close-packed arrangement of spheres, inspired by the dense packing of metal atoms.

Key methodological contributions include:

  • Spherical Interpolated Convolution Operator: This operator replaces traditional 3D convolutions. It interpolates features from neighboring points within a spherical region to the kernel's cell centers, allowing for a more spatially consistent feature extraction.
  • Distance-Feature Density: To address the issue of non-uniform point cloud density and feature distribution, the authors propose a "distance-feature density" mechanism. This method learns a density-aware weight that adjusts the influence of neighboring points during interpolation, making the feature extraction more robust to variations in point density. It combines spatial distance with feature correlation.
  • Network Architecture: The SIConv network follows a U-Net-like encoder-decoder structure. The encoder progressively downsamples the point cloud while extracting features, and the decoder upsamples to predict semantic labels for each point. Skip connections are used to preserve fine-grained details.

Datasets

The proposed method was evaluated on two benchmark datasets:

  • ScanNet: An indoor dataset featuring RGB-D camera data for 3D reconstructions.
  • Paris-Lille-3D: An outdoor dataset generated from a Mobile Laser System (MLS).

Key Insights and Ideas

  • Spherical Symmetry is Key: The paper argues that the inherent spherical symmetry of 3D space makes spherical kernels a more natural and effective choice for point cloud processing than cubic or grid-based kernels.
  • Density Matters (A Lot): The non-uniformity of point clouds is a significant challenge. The proposed "distance-feature density" effectively tackles this by adaptively weighting features based on both spatial proximity and feature similarity, leading to more reliable results.
  • Efficiency Gains: The spherical kernel design, by covering space more efficiently, leads to fewer trainable parameters and faster inference times compared to traditional cubic kernels, without sacrificing performance.

Key Results

The SIConv network demonstrated competitive performance on both ScanNet and Paris-Lille-3D datasets.

  • ScanNet: The method achieved state-of-the-art or near state-of-the-art results, outperforming several recent methods, particularly in terms of Mean Intersection over Union (mIoU).
  • Paris-Lille-3D: The proposed method also showed superior performance compared to existing graph-based methods, highlighting the effectiveness of the spherical kernel.
  • Parameter Efficiency: The spherical kernel requires significantly fewer parameters (9.06M) and achieves faster runtime (264ms/8192 points) compared to a cubic kernel (15.34M parameters, 397ms/8192 points) while maintaining or improving accuracy.

Limitations and Future Work

  • Limitations: While the paper highlights the robustness to uneven point distribution, the inherent challenges of extremely sparse regions or occlusions in real-world scenarios might still pose difficulties. The paper also mentions that the proposed method only uses point clouds, implying that incorporating other sensor modalities (like RGB images) could potentially further enhance performance, though they chose not to in this work.
  • Future Work: The authors suggest that their feature extraction method and distance-feature density are promising for practical applications such as autonomous driving and robot navigation, implying that further exploration in these domains is a natural next step. The code is also planned to be released, which will facilitate further research.

Summary updated

Abstract

The semantic segmentation of point clouds is an important part of the environment perception for robots. However, it is difficult to directly adopt the traditional 3-D convolution kernel to extract features from raw 3-D point clouds because of the unstructured property of point clouds. In this article, a spherical interpolated convolution operator is proposed to replace the traditional grid-shaped 3-D convolution operator. In addition, this article analyzes the defect of point cloud interpolation methods based on the distance as the interpolation weight and proposes the self-learned distance–feature density by combining the distance and the feature correlation. The proposed method makes the feature extraction of the spherical interpolated convolution network more rational and effective. The effectiveness of the proposed network is demonstrated on the 3-D semantic segmentation task of point clouds. Experiments show that the proposed method achieves good performance on the ScanNet dataset and Paris-Lille-3D dataset. The comparison experiments with the traditional grid-shaped 3-D convolution operator demonstrated that the newly proposed feature extraction operator improves the accuracy of the network and reduces the parameters of the network. The source codes will be released on https://github.com/IRMVLab/SIConv.
BibTeX
@article{wang2022spherical,
  author = {Guangming Wang and Yehui Yang and Huixin Zhang and Zhe Liu and Hesheng Wang},
  title = {{Spherical Interpolated Convolutional Network With Distance-Feature Density for 3-D Semantic Segmentation of Point Clouds}},
  journal = {IEEE Trans. Cybernetics},
  year = {2022},
  url = {https://openreview.net/forum?id=m2p0NGTrL8},
}
Figure