Introduction
3D object classification is a fundamental task in computer vision, with critical applications in autonomous driving and robotics. While direct processing of 3D data (e.g., voxels or point clouds) is intuitive, it is often computationally expensive and hindered by data sparsity. View-based methods, which project 3D objects into multiple 2D images, offer a more efficient alternative. However, traditional approaches like Multi-View CNN (MVCNN) often treat these views as independent or rely on simple pooling, neglecting the intrinsic spatial and content relationships between different camera angles.
This paper proposes a novel framework that explicitly models "inter-view" relationships and employs an adaptive weighting strategy to prioritize informative views while discarding redundant ones.
Methodology
The proposed architecture integrates three primary components to capture both local and global object features:
- Intra-view Feature Extraction: The system renders 12 2D views of a 3D object at 30-degree intervals. Each view is processed through a shared ResNet-18 backbone to extract high-level individual feature descriptors.
- Inter-view Information Calculation: To capture structural correlations, the model calculates the outer product between features of different view pairs. These correlation matrices are then processed via view pair pooling and 1D convolutions to generate a comprehensive inter-view descriptor for each perspective.
- Selective and Adaptive Loss Fusion: Recognizing that not all views are equally discriminative, the authors formulate a constrained optimization problem using an $\ell_0$ norm constraint. This mechanism adaptively assigns weights to views, effectively "zeroing out" uninformative or noisy perspectives during the final classification.
Experimental Setup
The model was evaluated on the ModelNet40 benchmark, which contains 12,311 CAD models across 40 categories. For each model, 12 virtual camera views were rendered at 30-degree increments around the vertical axis. The primary metric for evaluation is classification accuracy.
Results and Discussion
The proposed method, denoted as Ours (ResNet-18), achieves a state-of-the-art accuracy of 94.16% on ModelNet40. As shown in the comparison below, this approach outperforms several high-profile baselines, including standard MVCNN and GVCNN implementations.
Ablation studies indicate that the inclusion of inter-view information and the selective weighting strategy are both critical to the performance gains. Specifically, the full model outperforms a baseline using only feature concatenation (87.92%) by a significant margin, demonstrating the value of modeling view correlations. Visualization of learned weights further confirms that the model successfully identifies and prioritizes the most descriptive views of an object, such as the side profiles of a chair versus its bottom.
Conclusion
By treating 3D classification as a joint 2D image integration task, this research demonstrates that modeling inter-view correlations and implementing selective view fusion can significantly enhance performance. The approach leverages the maturity of 2D-CNN architectures while introducing a sophisticated logic for multi-view synthesis, achieving 94.16% accuracy on the ModelNet40 benchmark. Future work may explore extending this framework to arbitrary viewpoints and more complex tasks like 3D object retrieval.
Summary updated