Abstract
Efficient 3D model retrieval and classification are essential for managing the vast amounts of data generated in smart cities and digital manufacturing. Traditional view-based methods often suffer from information loss and order sensitivity. This paper introduces the Multi-View Tree Structure (MVTS) framework, which organizes 2D views into a content-aware tree structure using a Maximum Spanning Tree (MST) approach. By employing a bidirectional Tree-LSTM and an attention mechanism, MVTS captures complex spatial relationships and prioritizes informative views. Experimental results on ModelNet40 and ShapeNetCore55 demonstrate that MVTS achieves state-of-the-art performance, significantly outperforming existing sequence-based and pooling-based methods.
Introduction
In the era of smart cities and digital manufacturing, the volume of 3D data is expanding at an unprecedented rate. Efficiently searching for and categorizing these 3D models is critical for various applications, from urban planning to industrial design. Current 3D model processing techniques generally fall into two categories: model-based approaches, which process 3D voxels or point clouds directly, and view-based approaches, which utilize 2D images of the model. While model-based methods are computationally intensive, view-based methods offer efficiency but face three major hurdles: information loss from simple pooling, sensitivity to view order, and the varying importance of different viewing angles.
The Multi-View Tree Structure (MVTS) Framework
To address these challenges, the Multi-View Tree Structure (MVTS) learning method organizes 2D views into a content-aware tree rather than a simple sequence. This approach captures the complex spatial relationships between views without requiring fixed camera positions.
Methodology
The MVTS framework operates in three distinct stages:
- Visual Feature Learning: The system extracts features from 12 views of a 3D model using a pre-trained AlexNet backbone, providing a robust initial representation for each angle.
- Tree Model Construction: Rather than assuming a linear sequence, the method calculates a score matrix based on visual similarity between all pairs of views. Using Prim’s algorithm, a Maximum Spanning Tree (MST) is constructed to connect the most visually related views, effectively mapping the object's spatial flow. This MST is then converted into a Left-Child Right-Sibling (LCRS) tree for efficient computation.
- Tree Context Encoding:
- Bidirectional Tree-LSTM: The model processes the tree structure in both top-down and bottom-up directions, allowing each view node to incorporate context from the entire model.
- Tree Attention: An attention module assigns weights to different views, prioritizing those that contribute most to the unique identity of the shape.
Experimental Results
The MVTS method was rigorously evaluated on two primary benchmarks: ModelNet40 (12,311 CAD models) and ShapeNetCore55 (51,300 models using the official SHREC’17 split).
On the ModelNet40 dataset, MVTS achieved a 93.4% classification accuracy and an 87.3% retrieval mean Average Precision (mAP), outperforming established models such as MVCNN (90.1% accuracy) and GVCNN (90.7% accuracy). In the ShapeNetCore55 challenge, the method showed even more significant gains, particularly in macro metrics, where it outperformed existing state-of-the-art methods by margins ranging from 6.4% to 41.5%.
Key Insights and Discussion
- Camera-Agnostic Learning: By building the tree structure based on visual content rather than a fixed camera path, the model remains robust regardless of camera placement.
- Hierarchical Feature Representation: The Tree-LSTM architecture better reflects human perception of complex objects as a hierarchy of features rather than a flat sequence.
- Robustness: Sensitivity analysis indicates that the choice of the root node in the tree construction does not significantly impact performance, validating the stability of the MST approach.
Future Directions
While MVTS sets new benchmarks, future work will explore the use of multiple trees with diverse visual features and more modern backbone architectures like transformers. Additionally, testing the framework on noisy, real-world 3D scans from smart city environments will be a key area for further empirical research.
Summary updated