Abstract
Self-supervised learning on point clouds has gained significant attention as a solution to label-efficiency and domain-gap challenges. This paper introduces a novel framework to learn informative representations from partial point clouds by disentangling content and pose attributes. Our architecture incorporates a completion network for holistic semantics, a pose regression network for viewpoint understanding, and a partial reconstruction network to enforce feature separation. We demonstrate the robustness of these representations across classification, part segmentation, and registration tasks, outperforming state-of-the-art self-supervised methods and showing superior generalizability from synthetic to real-world datasets.
Introduction
Deep learning on 3D point clouds typically requires massive labeled datasets, which are costly to produce. Self-supervised learning (SSL) mitigates this by using pretext tasks where the data provides its own supervision. However, real-world sensors like LiDAR often capture only partial scans, where the data is inherently "entangled" with both the object's identity (content) and the sensor's viewing angle (pose). If these factors are not explicitly separated, the resulting features may fail to generalize across different viewpoints or transfer effectively to new tasks.
Proposed Framework
The core of our approach is a multi-branch architecture designed to force the separation of content and pose features through a "disentangler" mechanism.
As shown in the architecture diagram, the framework consists of three primary components:
- Completion Branch: A content encoder ($E_{content}$) extracts a content feature from a partial scan, which a decoder then uses to reconstruct the complete 3D object. This forces the model to capture global semantic information.
- Pose Regression Branch: A separate pose encoder ($E_{pose}$) extracts a pose feature to predict the specific camera coordinates (azimuth and polar angles) of the partial scan.
- Partial Reconstruction Branch: This branch acts as the disentangler. By taking the content feature from one view and the pose feature from a different view of the same object, the model attempts to reconstruct the partial point cloud corresponding to the second view. Success in this "swapping" task proves that the features are truly disentangled.
Experimental Results and Analysis
We evaluated our model on several benchmarks, including ModelNet40 for synthetic classification and ScanObjectNN for real-world robustness. Our method achieved a state-of-the-art 92.0% accuracy on ModelNet40 (using a DGCNN backbone) and 74.8% on the challenging ScanObjectNN (PB_T50_RS) variant, outperforming existing SSL competitors.
The effectiveness of this disentanglement is visualized through t-SNE plots. As seen in the figure above, content features cluster distinctly by object category regardless of the viewing angle, while pose features cluster by viewpoint regardless of the object's identity. This separation is a "force multiplier" for downstream tasks; while completion helps with semantic tasks like classification, the ability to reconstruct partial views gives the model a significant edge in geometric tasks like point cloud registration.
Conclusion
By explicitly disentangling content and pose from partial point clouds, our framework learns more robust and generalizable 3D representations. The model's ability to transfer knowledge from synthetic pre-training (ShapeNet) to noisy real-world scans (ScanObjectNN) highlights its practical utility. Future work will explore extending this disentanglement strategy to complex scene-level data and addressing variations in sensor distance.
Summary updated