Abstract
Traditional 3D model retrieval methods often rely on global structural descriptors, which can overlook local details and struggle with cross-domain performance. This paper presents a novel framework that treats 3D shapes as a collection of "geometric words." By decomposing models into these constituent parts and organizing them within a heterogeneous Knowledge Graph (KG), the system achieves state-of-the-art retrieval performance and demonstrates significant robustness in cross-domain tasks.
Introduction
Most contemporary 3D retrieval systems attempt to summarize an entire object into a single mathematical signature using deep learning. While effective for within-domain tasks, this approach has two primary drawbacks: it ignores the relationships between local structural parts and is highly sensitive to the "style" of specific datasets. When a model trained on one dataset is applied to another, performance often drops significantly.
This research proposes viewing 3D models like documents. Just as a book is composed of recurring words, a 3D model is composed of recurring geometric components. By focusing on these "geometric words," the system can better understand the underlying structure of objects regardless of their global rendering style.
Methodology
The proposed framework follows a three-step pipeline to transform raw 3D data into a searchable Knowledge Graph.
1. Geometric Word Construction
Using PointNet++, the system segments 3D models into local shapes. These segments are then clustered using K-means to create a "dictionary" of standard geometric words, such as specific types of cylinders, cones, or arcs. This allows the system to identify shared components across different object classes.
2. Knowledge Graph Construction
A heterogeneous graph is built where nodes represent either 3D models or geometric words. The graph utilizes "lash edges" to link a model to its constituent words and "category edges" to link models belonging to the same class. This structure captures both the composition of individual models and the broader relationships between categories.
3. Retrieval via Graph Embedding
To enable efficient searching, the graph nodes are converted into vector embeddings using a modified struct2vec algorithm. This version is specifically adapted to handle the multi-modal nature of the graph, ensuring that models with similar "vocabularies" of geometric words are positioned closely in the embedding space. Retrieval is then performed by calculating similarity scores based on these embeddings.
Experimental Results
The framework was evaluated on the ModelNet40 and ShapeNetCore55 benchmarks, outperforming several state-of-the-art (SOTA) methods including MVCNN and PointNet++.
- ModelNet40 Performance: The method achieved a Nearest Neighbor (NN) score of 0.9124 and a Mean Average Precision (mAP) of 0.8357.
- Cross-Domain Retrieval: In the challenging task of using ShapeNet queries to find ModelNet40 matches, the framework maintained a high NN score of 0.8903, whereas traditional deep learning models saw significant performance degradation.
- Data Efficiency: The system proved remarkably stable even when training data was reduced by 50%, highlighting its utility for specialized domains where large labeled datasets are unavailable.
Discussion and Conclusion
The "Bag of Words" analogy for 3D geometry provides a powerful alternative to global descriptors. By focusing on structural relationships rather than statistical rendering, the model becomes less sensitive to domain shifts.
A key challenge identified is the "Goldilocks" problem regarding dictionary size: too few geometric words lead to ambiguity, while too many lead to data sparsity. The researchers found 40 words to be the optimal balance for current benchmarks. Future work will focus on automatically optimizing this dictionary size and exploring more complex spatial relationships within the Knowledge Graph.
Summary updated