Abstract
The trend of employing training-free methods for point cloud recognition is becoming increasingly popular due to its significant reduction in computational resources and time costs. However, existing approaches are limited as they typically extract either geometric or semantic features. To address this limitation, we are the first to propose a novel training-free method that integrates both geometric and semantic features. For the geometric branch, we adopt a non-parametric strategy to extract geometric features. In the semantic branch, we leverage a model aligned with text features to obtain semantic features. Additionally, we introduce the GFE module to complement the geometric information of point clouds and the MFF module to improve performance in few-shot settings. Experimental results demonstrate that our method outperforms existing state-of-the-art training-free approaches on mainstream benchmark datasets, including ModelNet and ScanObjectNN.
Introduction
Deep learning for 3D point clouds typically requires massive computational resources and significant time for model training. While "training-free" methods have emerged to mitigate these costs—using either fixed geometric rules or pre-trained models that align 3D data with text—they are usually one-sided. Existing methods typically focus on raw geometric shapes or semantic meanings derived from text-image alignment, but rarely both. This creates a representation gap: a model might recognize the shape of a chair but miss the semantic context that distinguishes it from a stool, or vice versa. This research bridges this gap by fusing these two information streams in a zero-training environment.
Methodology
The core of the proposed framework is the Information Fusion Encoder (IF-Encoder), which processes data through two parallel branches to capture a holistic representation of 3D objects.
1. Geometric Branch
This branch is entirely non-parametric, utilizing standard operations like Farthest Point Sampling (FPS) and k-Nearest Neighbors (k-NN). To enhance robustness, the Geometric Feature Enhancement (GFE) module is introduced, which supplements raw XYZ data with spherical coordinates, edge vectors, and surface normals. This allows for rich descriptors without any learnable weights.
2. Semantic Branch
This branch leverages ULIP, a pre-trained model designed to align 3D point clouds with natural language descriptions. By utilizing these pre-aligned features, the model gains a global "meaning" context that complements local geometric structures.
3. Fusion and Memory Management
Features from both branches are combined via weighted summation. For classification, the model compares test features against a Memory Bank (MEM) using cosine similarity. To optimize performance in few-shot scenarios, the Memory Feature Filtering (MFF) module uses K-Means++ clustering to select the most representative centroids for each category, rather than relying on random samples.
Experimental Results
The approach was evaluated on three mainstream 3D benchmarks: ModelNet10, ModelNet40, and the real-world ScanObjectNN (including its challenging OBJ_T50RS variant).
As shown in the results table, the proposed method sets new records for training-free 3D recognition. On ModelNet40, it achieved 92.10% accuracy in the full-shot setting, outperforming the previous leader, PointTFA (90.88%). More impressively, on the hardest real-world variant of ScanObjectNN (OBJ_T50RS), the model reached 77.38%, a significant leap over PointTFA’s 68.22%. Beyond accuracy, the framework is highly efficient; the entire classification process for a dataset takes only minutes on a standard machine, compared to the hours or days required for traditional deep learning training.
Conclusion and Future Work
This work demonstrates that geometry and semantics are highly complementary in 3D recognition. Geometry provides local structural detail ("how it looks"), while semantics provides global context ("what it is"). Combining them yields a performance jump that neither can achieve alone. Furthermore, the MFF module proves that selecting representative samples for the memory bank is more critical than simply increasing memory size.
Despite these successes, the method currently relies on manual tuning of the fusion weighting parameter ($\alpha$) and is capped by the quality of the pre-trained semantic backbone. Future research will explore more sophisticated, automated fusion strategies and the application of this training-free paradigm to 3D segmentation and detection tasks.
Summary updated