Self-Supervised Feature Learning from Partial Point Clouds via Pose Disentanglement

Meng-Shiun Tsai, Pei-Ze Chiang, Yi-Hsuan Tsai, Wei-Chen Chiu

Abstract

Self-supervised learning on point clouds has gained significant attention as a solution to label-efficiency and domain-gap challenges. This paper introduces a novel framework to learn informative representations from partial point clouds by disentangling content and pose attributes. Our architecture incorporates a completion network for holistic semantics, a pose regression network for viewpoint understanding, and a partial reconstruction network to enforce feature separation. We demonstrate the robustness of these representations across classification, part segmentation, and registration tasks, outperforming state-of-the-art self-supervised methods and showing superior generalizability from synthetic to real-world datasets.

Introduction

Deep learning on 3D point clouds typically requires massive labeled datasets, which are costly to produce. Self-supervised learning (SSL) mitigates this by using pretext tasks where the data provides its own supervision. However, real-world sensors like LiDAR often capture only partial scans, where the data is inherently "entangled" with both the object's identity (content) and the sensor's viewing angle (pose). If these factors are not explicitly separated, the resulting features may fail to generalize across different viewpoints or transfer effectively to new tasks.

Proposed Framework

The core of our approach is a multi-branch architecture designed to force the separation of content and pose features through a "disentangler" mechanism.

The proposed multi-branch architecture for disentangling content and pose features.

As shown in the architecture diagram, the framework consists of three primary components:

  1. Completion Branch: A content encoder ($E_{content}$) extracts a content feature from a partial scan, which a decoder then uses to reconstruct the complete 3D object. This forces the model to capture global semantic information.
  2. Pose Regression Branch: A separate pose encoder ($E_{pose}$) extracts a pose feature to predict the specific camera coordinates (azimuth and polar angles) of the partial scan.
  3. Partial Reconstruction Branch: This branch acts as the disentangler. By taking the content feature from one view and the pose feature from a different view of the same object, the model attempts to reconstruct the partial point cloud corresponding to the second view. Success in this "swapping" task proves that the features are truly disentangled.

Experimental Results and Analysis

We evaluated our model on several benchmarks, including ModelNet40 for synthetic classification and ScanObjectNN for real-world robustness. Our method achieved a state-of-the-art 92.0% accuracy on ModelNet40 (using a DGCNN backbone) and 74.8% on the challenging ScanObjectNN (PB_T50_RS) variant, outperforming existing SSL competitors.

t-SNE visualizations demonstrating the successful disentanglement of content and pose features.

The effectiveness of this disentanglement is visualized through t-SNE plots. As seen in the figure above, content features cluster distinctly by object category regardless of the viewing angle, while pose features cluster by viewpoint regardless of the object's identity. This separation is a "force multiplier" for downstream tasks; while completion helps with semantic tasks like classification, the ability to reconstruct partial views gives the model a significant edge in geometric tasks like point cloud registration.

Conclusion

By explicitly disentangling content and pose from partial point clouds, our framework learns more robust and generalizable 3D representations. The model's ability to transfer knowledge from synthetic pre-training (ShapeNet) to noisy real-world scans (ScanObjectNN) highlights its practical utility. Future work will explore extending this disentanglement strategy to complex scene-level data and addressing variations in sensor distance.

Summary updated

Abstract

Self-supervised learning on point clouds has gained a lot of attention recently, since it addresses the label-efficiency and domain-gap problems on point cloud tasks. In this paper, we propose a novel self-supervised framework to learn informative representations from partial point clouds. We leverage partial point clouds scanned by LiDAR that contain both content and pose attributes, and we show that disentangling such two factors from partial point clouds enhances feature representation learning. To this end, our framework consists of three main parts: 1) a completion network to capture holistic semantics of point clouds; 2) a pose regression network to understand the viewing angle where partial data is scanned from; 3) a partial reconstruction network to encourage the model to learn content and pose features. To demonstrate the robustness of the learnt feature representations, we conduct several downstream tasks including classification, part segmentation, and registration, with comparisons against state-of-the-art methods. Our method not only outperforms existing self-supervised methods, but also shows a better generalizability across synthetic and real-world datasets.
BibTeX
@inproceedings{tsai2022self,
  author = {Meng-Shiun Tsai and Pei-Ze Chiang and Yi-Hsuan Tsai and Wei-Chen Chiu},
  title = {{Self-Supervised Feature Learning from Partial Point Clouds via Pose Disentanglement}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=iyA23mZPkli},
}
Figure