TL;DR
This paper introduces RIMeshGNN, a novel graph neural network architecture designed for 3D mesh classification that is inherently invariant to rotations, translations, and reflections. By developing equivariant GNN layers and a unique aggregation function, RIMeshGNN achieves state-of-the-art performance without requiring extensive data augmentation for rotations, a common bottleneck in existing methods.
What Problem Does This Paper Address?
Analyzing 3D shapes, especially those represented as meshes, is crucial for many applications like gaming, VR, and medical imaging. A significant challenge in this field is achieving robustness to geometric transformations, particularly rotations. Many existing 3D shape analysis methods, including those using deep learning, struggle when objects are presented in arbitrary orientations. This is because their learned features are sensitive to the object's pose, leading to a performance drop. While some methods use extensive data augmentation (rotating shapes during training) to mitigate this, it's computationally expensive and doesn't guarantee perfect invariance. The paper aims to solve this by creating a network that is intrinsically rotation-invariant, thus maintaining high accuracy regardless of the input mesh's orientation.
Methods
The core of RIMeshGNN lies in its novel Graph Neural Network (GNN) architecture, which is built upon three key components:
- E(3)-Equivariant GNN Layer: This layer is designed to process mesh data (vertices, edges, faces) in a way that its output features transform predictably with input rotations and translations. It achieves this by carefully handling both rotation-equivariant features (like vertex coordinates and normals) and rotation-invariant features (like curvature and area). The layer uses a shared transformation function to update edge features and then updates vertex features by considering relative differences and weights derived from these edge features.
- Permutation, Rotation, and Reflection Invariant Aggregation Function: To consolidate the learned features into a representation that is completely invariant to these transformations, the paper proposes a unique aggregation mechanism. It first "bucketizes" mesh nodes into spherical bins based on their distance from the origin. Then, it constructs a complete graph where each node represents a bin, and a Graph Attention Network (GAT) is applied to learn features for these bins. Finally, a global graph feature is computed by averaging these bin features, resulting in a permutation, rotation, and reflection invariant representation.
- Local Pooling Layer: To enable multi-resolution analysis, a local pooling layer is introduced. This layer performs mesh coarsening through edge contraction, similar to existing methods but with a learnable strategy. It prioritizes edges for contraction based on learned scores, ensuring that important mesh regions are preserved. This pooling mechanism is designed to be rotation-equivariant and not to disrupt the overall rotation-invariance of the network.
The overall network architecture stacks these GNN and pooling layers, concatenating features from intermediate layers to combat over-smoothing and enhance model capacity. An MLP sub-network is used as the final classifier.
Datasets
The paper evaluates RIMeshGNN on two standard datasets:
- SHREC11: This dataset contains 30 classes of 3D shapes. The authors use two common splits: Split 16 (16 training, 4 testing) and Split 10 (10 training, 10 testing).
- ModelNet40: A larger dataset with 40 classes and over 12,000 shapes. The authors preprocess ModelNet40 to create "Manifold40" by ensuring all meshes are watertight and then simplifying them to 750 faces, as many original models are not manifold.
Key Insights and Ideas
- Intrinsic Invariance is Key: The paper strongly advocates for building rotation invariance directly into the network architecture rather than relying solely on data augmentation. This leads to more efficient training and better generalization.
- Leveraging Both Equivariant and Invariant Features: By carefully distinguishing and processing both types of geometric features, the network can capture rich information while ensuring the final output is invariant.
- Spherical Binning for Aggregation: The novel aggregation function, which uses spherical bins to group nodes and then applies a GAT, is a clever way to achieve permutation and rotation invariance by abstracting local geometric information into a structured graph.
- Multi-Resolution Analysis: The inclusion of a local pooling layer allows the network to analyze shapes at different levels of detail, which is beneficial for capturing both fine-grained and coarse geometric structures.
Key Results
- State-of-the-Art Performance: RIMeshGNN achieves 100% accuracy on the SHREC11 dataset for both aligned (NR/NR) and arbitrarily rotated (NR/AR) test samples, outperforming all compared methods.
- Robustness to Rotation: On the Manifold40 (ModelNet40) dataset, RIMeshGNN maintains a consistent accuracy of 90.7% in the NR/AR setting, significantly outperforming other methods that experience a substantial performance drop when faced with rotated test samples.
- Efficiency: The proposed method does not require extensive rotation augmentation, saving significant computational resources during training.
Limitations and Future Work
- Data Preprocessing: The method assumes watertight manifold meshes. While they use ManifoldPlus to address this, it's an additional preprocessing step.
- Cross-Labeled Classes: The authors note that certain categories in datasets like SHREC11 have inherent labeling issues (e.g., desk/table), which can affect classification. They excluded these for some comparisons.
- Future Work: The paper suggests that the RIMeshGNN framework can be adapted for other mesh analysis tasks, such as mesh segmentation and retrieval, which are natural extensions of its representation learning capabilities.
Summary updated