Abstract
For 3D object retrieval, traditional multi-view methods usually rely on supervised classification for feature learning, which requires significant manual effort to annotate data. We present an unsupervised scheme to address this problem using loop view data. First, we perform unsupervised deep learning of 3D objects for multiple loop view data and leverage a random data mixture strategy to learn latent relations between different shapes. Second, we introduce an optimal matching method for similarity matching, where the optimal value is calculated by averaging minimized loop view distances. Finally, we propose a filtering algorithm for loop view features to reduce data redundancy, significantly saving computational costs while preserving retrieval accuracy. Experiments on ModelNet40 and SHREC15 show that our algorithm achieves excellent performance for unsupervised 3D object retrieval without requiring data annotation.
Introduction
3D object retrieval—the process of finding a specific 3D model within a massive database—is a critical component in fields such as autonomous driving, VR/AR, and industrial design. Currently, the most effective methods rely on "multi-view" deep learning, where a 3D object is rendered into multiple 2D images for analysis.
However, these methods typically require supervised learning, necessitating the manual labeling of thousands of models. This process is labor-intensive and does not scale well to the ever-growing volume of 3D data. Existing unsupervised methods often struggle to capture the fine-grained details needed to distinguish similar shapes. This research aims to bridge this gap by creating a system that learns discriminative features directly from raw geometry without human-provided labels.
Methodology
The proposed framework utilizes a three-stage pipeline to extract and match features without supervision, as illustrated in the overview below.
1. Unsupervised Feature Learning with Loop Views
Instead of category labels, the system assigns a unique "ID" to every individual 3D model in the training set. The model is rendered into multiple "loop views"—circular sequences of 12 images captured from a 360° path around the object. The neural network is then trained to classify these views into their respective unique IDs, forcing it to learn the specific, distinguishing features of every individual shape.
2. Random Mixture (RM)
To prevent the model from simply memorizing IDs and to help it understand the relationships between different shapes, a "random mixture" strategy is employed. During training, the system blends images and labels from two different objects. This forces the latent space of the network to become more continuous and robust, helping it recognize shared geometric features between different but related objects (e.g., a stool and a chair).
3. Similarity Matching and Filtering
- Optimal Matching (OP): When comparing two objects, the system calculates the distance between all their respective loop views and finds the best alignment, providing a more accurate similarity score than simple averaging.
- Attention-based Filtering (AF) & Aggregation (AA): To address the computational expense of multi-view matching, an attention mechanism identifies the most "informative" views. This allows the system to discard redundant data, drastically speeding up the retrieval process.
Experimental Results
The researchers evaluated their approach on two major benchmarks: ModelNet40 (12,311 CAD models) and SHREC15 (1,200 non-rigid shapes).
- Accuracy: On ModelNet40, the method achieved a Mean Average Precision (mAP) of 41.2%. On SHREC15, it reached 54.5%.
- Efficiency: The attention-based filtering reduced the matching time on ModelNet40 from 28.4 hours down to 1.4 hours—a 95% reduction in computational cost.
- Ablation Study: The "Random Mixture" (RM) module was shown to be vital; its inclusion improved the mAP on SHREC15 from 47.5% to 54.5%.
Conclusion
This framework demonstrates that treating every object as its own class (instance discrimination) combined with loop-view continuity is a powerful way to learn 3D features without human supervision. While a performance gap remains compared to fully supervised models, the significant gains in efficiency and the elimination of manual labeling represent a major step forward for scalable 3D object retrieval.
Summary updated