Abstract
Deep learning with 3D data has progressed significantly since the introduction of convolutional neural networks that can handle point order ambiguity in point cloud data. While being able to achieve good accuracies in various scene understanding tasks, previous methods often have low training speed and complex network architecture. In this paper, we address these problems by proposing an efficient end-to-end permutation invariant convolution for point cloud deep learning. Our simple yet effective convolution operator named ShellConv uses statistics from concentric spherical shells to define representative features and resolve the point order ambiguity, allowing traditional convolution to perform on such features. Based on ShellConv we further build an efficient neural network named ShellNet to directly consume the point clouds with larger receptive fields while maintaining less layers. We demonstrate the efficacy of ShellNet by producing state-of-the-art results on object classification, object part segmentation, and semantic scene segmentation while keeping the network very fast to train.
Introduction
Deep learning on 3D point clouds faces a fundamental challenge: point sets are inherently unordered. Unlike the regular grid of pixels in 2D images, a point cloud is a collection of $(x, y, z)$ coordinates. Any permutation of the list of points represents the same geometric shape, but standard neural networks are sensitive to this order. This is known as permutation ambiguity.
Previous approaches like PointNet addressed this using global symmetric functions, but often missed local geometric nuances. Later iterations like PointNet++ or PointCNN captured local details at the cost of increased complexity, such as learning transformation matrices or requiring deep, computationally expensive architectures. ShellNet introduces ShellConv, a mathematically simple and efficient operator designed to resolve permutation ambiguity while maintaining high performance and fast training speeds.
Methodology: The ShellConv Operator
The primary innovation of ShellNet is the ShellConv operator, which transforms unordered local neighborhoods into ordered sequences using geometric heuristics.
The operator functions in four distinct stages:
- Neighborhood Partitioning: For a representative point, nearest neighbors are identified and grouped into concentric spherical shells based on their Euclidean distance from the center.
- Feature Lifting: Local point features are mapped to a higher-dimensional space via a simple multi-layer perceptron (MLP).
- Intra-shell Statistics: A max-pooling operation is applied within each shell. This ensures that the features extracted from each shell are permutation-invariant—the internal order of points within a shell does not affect the output.
- Inter-shell Convolution: Since the shells have a natural radial order (from inner to outer), a standard 1D convolution can be applied across the shells to capture multi-scale spatial relationships.
By leveraging distance-based ordering, ShellNet avoids the need for complex learned transformations (like the X-transformation in PointCNN), significantly reducing computational overhead.
Experimental Evaluation and Results
ShellNet was evaluated on several standard benchmarks, including ModelNet40 (classification), ShapeNet (part segmentation), and large-scale indoor/outdoor datasets like ScanNet, S3DIS, and Semantic3D.
Performance and Efficiency
ShellNet achieved state-of-the-art results, reaching 93.1% overall accuracy on ModelNet40 and ranking first on ScanNet (85.2% OA) and S3DIS (66.8% mIoU) at the time of publication. Notably, it achieved these results with a highly efficient architecture of only 0.48M parameters, compared to 12.4M for PointNet++.
As shown in the figure above, ShellNet's most significant advantage is its speed of convergence. By using a fixed geometric heuristic for ordering rather than a learned one, the network reaches 90% accuracy on ModelNet40 in approximately 15 minutes. In contrast, competing models often require hours or even days of training to reach comparable performance levels.
Conclusion
ShellNet demonstrates that "order from geometry" is a powerful principle for point cloud processing. By organizing points into concentric shells, the network can utilize traditional 1D convolutions effectively. This approach results in a model that is not only highly accurate but also remarkably fast to train and parameter-efficient. Future work may explore more robust shell definitions to handle boundary sensitivities and extend the operator to tasks like 3D object detection and generative modeling.
Summary updated