Abstract
We present Occlusion Completion (OcCo), a simple yet effective unsupervised pre-training approach for 3D point clouds. OcCo leverages the natural phenomenon of occlusion to learn robust geometric features. The process involves three steps: masking occluded points from a simulated camera view, training an encoder-decoder to reconstruct the full geometry, and transferring the encoder weights to downstream tasks. Our experiments demonstrate that OcCo significantly improves performance across various datasets and architectures, particularly in few-shot scenarios and real-world noisy environments.
Introduction
Deep learning's success in 3D vision is often hampered by the high cost and complexity of labeling point cloud data. Unlike 2D images, 3D scans can be sparse, massive, and frequently occluded, making manual annotation labor-intensive. While unlabeled 3D data is abundant from LiDAR and depth sensors, effective self-supervised tasks are needed to harness it. Previous methods like orientation prediction or 3D jigsaw puzzles often fail to preserve geometric integrity or generalize to real-world noise. OcCo addresses this by using occlusion—a fundamental aspect of 3D perception—as a supervisory signal.
Methodology: The OcCo Pipeline
The OcCo framework consists of a straightforward three-step pipeline designed to force the model to understand both global and local 3D structures.
- Generate Occluded Dataset: For a given point cloud (e.g., from ModelNet40), a camera viewpoint is simulated. Using a pinhole camera model and z-buffering via Delaunay triangulation, the method identifies visible points and masks those that would be occluded from that specific perspective.
- Train Encoder-Decoder Completion Model: An encoder (such as PointNet, DGCNN, or PCN) processes the partial, occluded cloud into a latent representation. A decoder then attempts to reconstruct the original full point cloud. The training objective is to minimize the Chamfer Distance (CD) between the reconstructed and ground-truth point clouds.
- Use Encoded Weights as Initialization: After pre-training, the decoder is discarded. The encoder weights, having learned to "fill in" missing geometry, serve as a powerful initialization for downstream tasks like classification or segmentation.
Experimental Evaluation
The models were pre-trained on the synthetic ModelNet40 dataset. Evaluation spanned a diverse range of downstream tasks and datasets:
- Object Classification: Tested on ModelNet40, ScanNet10, and ScanObjectNN (real-world scans).
- Segmentation: Evaluated on ShapeNetPart (part segmentation), S3DIS (indoor semantic segmentation), and SensatUrban (large-scale city scans).
Key Insights
Beyond raw performance, the authors investigated why OcCo works:
- Architecture Agnostic: OcCo consistently improved performance across different backbones, including PointNet and DGCNN.
- Loss Landscape Analysis: Visualization revealed that OcCo pre-training leads to "flatter" local minima during fine-tuning. In deep learning, flatter minima are typically associated with superior generalization.
- Emergent Semantics: Interestingly, the pre-trained features showed high correlation with semantic parts. Specific neurons learned to activate for distinct components like chair legs or airplane wings, despite never seeing a label during pre-training.
Results and Performance
OcCo demonstrates substantial gains over training from scratch and previous self-supervised methods like Jigsaw and cTree.
- Few-Shot Learning: In a "10-way 10-shot" setup on ModelNet40, OcCo-pre-trained PointNet reached 83.9% accuracy, a massive leap from the 46.6% achieved with random initialization.
- Real-World Robustness: On the challenging ScanObjectNN dataset, OcCo improved DGCNN's accuracy to 83.9%, outperforming the Jigsaw baseline.
- Large-Scale Segmentation: On SensatUrban, OcCo improved mIoU for categories like "cars," showing that object-level pre-training scales to city-level scene understanding.
Conclusion
Occlusion Completion provides a natural and effective self-supervised task for 3D point clouds. By learning to reconstruct missing geometry, models develop a robust spatial understanding that generalizes across architectures and datasets. OcCo's ability to significantly boost performance in data-scarce and real-world scenarios makes it a valuable tool for the 3D computer vision community.
Summary updated