TL;DR
This paper introduces MeshCaps, a novel deep learning framework for classifying 3D mesh models by adapting capsule networks to handle irregular mesh structures. By using polynomial fitting for local surface feature extraction and a modified capsule network architecture, MeshCaps achieves state-of-the-art accuracy on the SHREC15 dataset, outperforming existing methods while offering improved training efficiency.
What Problem Does This Paper Address?
The field of 3D object recognition has seen significant advancements with deep learning, particularly convolutional neural networks (CNNs). However, applying these methods to 3D data, especially complex mesh structures, presents challenges. Traditional CNNs struggle with the irregular topology of meshes, and common techniques like max-pooling can discard crucial spatial information. While voxel and point cloud representations have been explored, 3D meshes, which accurately capture surface geometry and neighborhood information, remain less explored in deep learning for classification. Existing mesh-based deep learning methods often rely on edge folding or specific grid structures, which may not fully exploit the rich geometric information present in mesh data. This paper aims to bridge this gap by developing a deep learning approach that effectively handles the complexity of 3D mesh data for classification.
Methods
The core of the proposed MeshCaps framework lies in two main innovations:
- Polynomial Convolution Template for Feature Extraction: Instead of traditional convolution kernels, MeshCaps employs a polynomial fitting approach. For each local surface patch (defined by a vertex and its neighbors), a high-order polynomial is fitted to approximate the surface shape. The parameters of this polynomial, along with the center point coordinates and normal vector of the surface, are used as local shape and pose features. This method is designed to capture fine geometric details and handle the irregular nature of mesh surfaces.
- Capsule Network Adaptation for 3D Meshes: To overcome the limitations of traditional CNNs (like information loss from pooling) and address the varying input sizes of 3D mesh models, the authors adapt capsule networks. A key modification is the introduction of a "capsule composition layer" and the use of shared weights for the pose transformation matrix. This allows the network to learn hierarchical representations of features and adapt to different input sizes without requiring explicit resizing or padding. The dynamic routing mechanism, characteristic of capsule networks, is retained to preserve spatial relationships between features.
Datasets
The primary dataset used for training and evaluation is:
- SHREC15: A standard benchmark dataset for 3D deformable mesh model retrieval and classification, containing 50 categories with 1200 3D mesh models (24 models per class). The dataset includes models with both rigid and non-rigid transformations.
Key Insights and Ideas
- Polynomials as Powerful Surface Descriptors: The paper highlights that high-order polynomial fitting can effectively represent complex local surface shapes in 3D meshes, providing a more expressive feature descriptor than traditional methods.
- Capsule Networks for Geometric Data: The successful adaptation of capsule networks to 3D mesh data demonstrates their potential beyond 2D image classification, particularly in preserving spatial relationships and handling irregular structures.
- Addressing Input Size Variability: The modified capsule network architecture with shared pose matrices effectively handles the inherent challenge of varying input sizes in 3D mesh datasets, a common issue in geometric deep learning.
- Feature Fusion is Crucial: Combining geometric features (polynomial parameters) with pose information (center point, normal vector) significantly boosts classification accuracy, underscoring the importance of a holistic feature representation.
Key Results
- State-of-the-Art Accuracy: MeshCaps achieved an average recognition accuracy of 93.8% on the SHREC15 dataset, outperforming previous methods like MeshNet (91.9%) and MeshCNN (91.4%).
- Improved Recognition: The proposed method improved the average accuracy by 3.4% compared to the original test set results of MeshCNN and 2.1% compared to MeshNet.
- Efficient Training: Despite its complexity, the MeshCaps algorithm demonstrated good convergence, reaching high accuracy in fewer training epochs compared to other methods. The total training time was approximately 2158 seconds on a GPU.
- Optimal Window Size: Experiments revealed that an optimal convolution window size of 152 points (around a vertex) yielded the highest classification accuracy.
Limitations and Future Work
- Misclassification of Similar Shapes: The paper notes that certain categories with similar curved surface units (e.g., glasses and pliers) led to higher misrecognition rates, suggesting that finer distinctions might be needed.
- Future Directions: The authors suggest that the MeshCaps framework can be further developed for 3D mesh segmentation or combined with grid generation algorithms for more advanced computer vision tasks. They also mention the potential for further research into optimizing the network for even faster convergence and improved performance on complex datasets.
Summary updated