Abstract
Fine-grained 3D shape classification is a critical yet challenging task in shape understanding, primarily due to the subtle geometric differences between subcategories. This paper addresses two major bottlenecks: the lack of high-quality fine-grained benchmarks and the inability of global feature extractors to capture local nuances. We introduce the FG3D dataset, a curated benchmark with 66 subcategories, and FG3D-Net, a novel framework that utilizes a Region Proposal Network (RPN) and hierarchical part-view attention to distinguish highly similar 3D models. Our approach achieves state-of-the-art performance by effectively aggregating local semantic details and capturing spatial relationships across multiple views.
Introduction
While 3D shape classification has advanced significantly with datasets like ModelNet and ShapeNet, most research focuses on "coarse-grained" tasks—distinguishing broad categories like chairs from cars. However, "fine-grained" classification—telling a Boeing 747 apart from a Tupolev Tu-154—is much harder because global silhouettes are often nearly identical. Standard methods tend to extract global features that "wash out" the tiny, local geometric differences (such as wingtip shapes or engine placements) that define a subcategory. To bridge this gap, we propose a method that focuses on "Generally Semantic Parts" (GSPs) and leverages hierarchical attention to prioritize the most discriminative features.
The FG3D Dataset
To facilitate research in this area, we introduced the FG3D dataset, a large-scale benchmark for fine-grained 3D shape classification. Unlike existing datasets that may have noisy or inconsistent labels, FG3D is a curated subset of ShapeNet and other repositories where each model is strictly assigned to one unique, verified subcategory label.
The dataset consists of three primary categories—Airplane, Car, and Chair—encompassing 66 subcategories and approximately 25,000 3D models. This clean benchmark allows for a rigorous evaluation of a model's ability to handle small inter-class variance.
FG3D-Net Methodology
Our proposed architecture, FG3D-Net, processes 3D objects through a multi-view pipeline designed to preserve and highlight local details.
1. Generally Semantic Part (GSP) Detection
Instead of analyzing a whole rendered view, FG3D-Net uses a Region Proposal Network (RPN) to identify semantic parts such as wheels, wings, or armrests. By focusing on these specific regions, the model can ignore common features shared by all members of a category and focus on the parts that actually vary between subcategories.
2. Hierarchical Part-View Attention
This core module applies attention at two distinct levels:
- Part-level Attention: Within a single view, the model learns which detected parts (e.g., the specific curve of a wing) are most important for classification.
- View-level Attention: Across all rendered views, it identifies which camera angles provide the most discriminative information, effectively filtering out redundant or uninformative perspectives.
3. Spatial Correlation with RNNs
Because the views are rendered in a sequence around the object, we integrate a Gated Recurrent Unit (GRU) to capture the spatial relationships between consecutive views. This ensures the model understands the object's continuous 3D structure rather than treating the views as an unordered collection of 2D images.
Experimental Results
FG3D-Net was evaluated against a wide array of competitors, including point-based (PointNet++), voxel-based (VoxNet), and view-based (MVCNN) models. Our method achieved superior instance accuracy across all three categories:
- Airplanes: 93.99%
- Chairs: 83.94%
- Cars: 79.47%
Ablation studies confirmed that both the hierarchical attention mechanism and the RNN are vital; removing either component led to a significant degradation in performance. This proves that both local detail and spatial context are necessary for successful fine-grained recognition.
Conclusion
FG3D-Net demonstrates that focusing on semantic parts and their spatial relationships is key to mastering fine-grained 3D shape classification. While the model currently relies on pre-trained part detectors, future work could explore unsupervised part discovery to expand the system's applicability to more diverse object categories. The FG3D dataset remains a valuable resource for the community to push the boundaries of detailed 3D shape understanding.
Summary updated