EllipsoidNet: Ellipsoid Representation for Point Cloud Classification and Segmentation

Yecheng Lyu, Xinming Huang, Ziming Zhang

TL;DR

This paper introduces "EllipsoidNet," a novel 2D representation for 3D point clouds that projects them onto an ellipsoid surface, capturing local geometry more effectively than existing methods. This representation, combined with a new neural network architecture, achieves state-of-the-art performance in point cloud classification and segmentation while being more efficient.

What Problem Does This Paper Address?

3D point clouds, while rich in geometric information, are notoriously difficult for machine learning models to process due to their unordered nature and complex local structures. A popular approach to tackle this is by converting point clouds into 2D representations, which can then be processed by well-established 2D Convolutional Neural Networks (CNNs). However, existing 2D methods like spherical or multi-view projections often suffer from significant point loss, uneven distribution, or are computationally expensive. This leads to a trade-off between accuracy and efficiency, leaving a gap for a representation that is both effective at capturing fine-grained local geometry and computationally feasible for real-world applications.

Methods

The core innovation is the Ellipsoid Representation. Instead of projecting points onto a sphere, the authors first use Principal Component Analysis (PCA) to determine the principal axes of the point cloud. This allows them to construct an ellipsoid that better fits the object's shape. Points are then projected onto this ellipsoid surface.

Key aspects of their method include:

  • Hierarchical Decomposition: For large or complex point clouds, the method recursively decomposes the cloud into smaller partitions, generating ellipsoid representations for each. This allows for multi-scale feature extraction.
  • Feature Map Completion: To ensure dense feature maps, they employ a "feature-map-orientated projection." This means each pixel in the 2D latent space is filled by tracing back to the nearest point in the 3D cloud, ensuring no empty pixels and reducing point overlap issues.
  • EllipsoidNet Architecture: A novel CNN architecture is proposed to process these ellipsoid representations. It features:
  • Point-wise Feature Encoder: Extracts local features from the projected point-wise feature maps.
  • Ellipsoid-wise Feature Encoder: Fuses global features derived from the ellipsoid's properties (orientation, radii, center).
  • Hierarchical Processing: The network processes features from multiple decomposition levels (multi-scale).
  • Classification and Segmentation Heads: Dedicated modules for the final tasks.

Datasets

The proposed methods were evaluated on two standard benchmarks:

  • ModelNet40: For point cloud classification.
  • ShapeNet: For point cloud segmentation.

The paper does not introduce new datasets.

Key Insights and Ideas

  • Ellipsoid is a Better Fit: Using an ellipsoid, tailored to the point cloud's shape via PCA, provides a more accurate and less distorted representation than a generic sphere, especially for non-spherical objects.
  • Dense Feature Maps are Crucial: The feature-map-orientated projection is key to overcoming the sparsity and void pixel issues common in other 2D projection methods, leading to richer local feature extraction.
  • Hierarchy Matters for Complexity: Decomposing point clouds hierarchically and processing them at multiple scales allows the method to effectively handle complex shapes and capture both fine-grained and coarse-grained features.
  • Combined Features Boost Performance: Utilizing both point-wise (local) and ellipsoid-wise (global) features within the EllipsoidNet architecture leads to significantly improved performance compared to using either feature type alone.

Key Results

  • Classification: On ModelNet40, EllipsoidNet achieved near state-of-the-art overall accuracy (92.7%) and the best mean accuracy across shape classes (91.4%), with significantly fewer floating-point operations (FLOPs) than many competing methods (19 million).
  • Segmentation: On ShapeNet, the method achieved near state-of-the-art class mean IoU (86.9%) and instance mean IoU (86.5%), again with a substantially lower FLOP count.
  • Representation Metrics: The ellipsoid representation, particularly the multi-scale version, showed a significant improvement in "point usage rate" (72.9%) and "max seg. IoU" (95.3%) compared to spherical representations.
  • Qualitative Results: Visualizations showed that EllipsoidNet provided better segmentation of small parts compared to baseline methods like PointNet.

Limitations and Future Work

  • Limitations: The paper notes that while their method is efficient, applying it to point clouds scanned from real-world scenes (which can be noisy and incomplete) might require further investigation.
  • Future Work: The authors suggest exploring the application of their ellipsoid representation to point clouds acquired from real-world sensors, implying that robustness to noise and incompleteness will be a focus.

Summary updated

Abstract

Point cloud patterns are hard to learn because of the implicit local geometry features among the orderless points. In recent years, point cloud representation in 2D space has attracted increasing research interest since it exposes the local geometry features in a 2D space. By projecting those points to a 2D feature map, the relationship between points is inherited in the context between pixels, which are further extracted by a 2D convolutional neural network. However, existing 2D representing methods are either accuracy limited or time-consuming. In this paper, we propose a novel 2D representation method that projects a point cloud onto an ellipsoid surface space, where local patterns are well exposed in ellipsoid-level and point-level. Additionally, a novel convolutional neural network named EllipsoidNet is proposed to utilize those features for point cloud classification and segmentation applications. The proposed methods are evaluated in ModelNet40 and ShapeNet benchmarks, where the advantages are clearly shown over existing 2D representation methods.
BibTeX
@inproceedings{lyu2022ellipsoidnet,
  author = {Yecheng Lyu and Xinming Huang and Ziming Zhang},
  title = {{EllipsoidNet: Ellipsoid Representation for Point Cloud Classification and Segmentation}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
  url = {https://openreview.net/forum?id=nBj4B6Hk5D},
}
Figure