Introduction
Unsupervised representation learning for 3D point clouds has traditionally been dominated by contrastive learning paradigms. These methods rely heavily on complex data augmentations—such as random cropping, scaling, or rotation—to generate multiple views of the same object. However, 3D data poses unique challenges; for instance, a random crop might inadvertently remove the defining geometric features of an object, leading to semantic inconsistency during training. Additionally, contrastive methods often necessitate large batch sizes and sophisticated negative sampling to prevent representation collapse.
To address these issues, the authors introduce SoftClu, a novel framework that eliminates the need for data augmentation. SoftClu reinterprets unsupervised learning as a soft-clustering problem, leveraging Optimal Transport to learn robust, point-level features without the overhead of complex data transformations.
The SoftClu Framework
SoftClu operates through an iterative process reminiscent of Expectation-Maximization (EM), focusing on aligning point-level features with learned prototypes.
1. Prototype Computation
The model maintains two sets of prototypes (cluster centroids): one in the geometric space (actual 3D coordinates) and another in the feature space (latent vectors learned by the encoder). This dual-space approach ensures that learned clusters are both geometrically coherent and semantically meaningful.
2. Soft-label Assignment (E-step)
Points are assigned to clusters based on their proximity to these prototypes. To prevent the model from collapsing into a single cluster (the "dead neuron" problem), the assignment is formulated as an Optimal Transport problem. By applying the Sinkhorn-Knopp algorithm, the framework enforces an equipartition constraint, ensuring that points are distributed across all available clusters roughly equally.
3. Optimization (M-step)
The resulting cluster assignments serve as "soft pseudo-labels." The network is then optimized by minimizing the cross-entropy loss between its predicted scores and these pseudo-labels. An additional orthogonal regularization loss is applied to the prototypes to ensure they stay distinct from one another.
Experimental Results
The effectiveness of SoftClu was validated across several standard benchmarks, including ShapeNet and ScanNet for pre-training, and ModelNet40, ShapeNetPart, and S3DIS for downstream tasks.
Performance Highlights
- Object Classification: SoftClu achieved 91.9% accuracy on ModelNet40 using a DGCNN backbone, outperforming established unsupervised methods like OcCo (89.2%) and STRL (90.9%).
- Part Segmentation: The model reached 85.7% mIoU on ShapeNetPart, surpassing previous unsupervised state-of-the-art methods.
- Semantic Segmentation: On the S3DIS dataset, it reached 59.2% mIoU, a significant improvement over Jigsaw3D (55.6%).
- Efficiency: Unlike contrastive methods that degrade with smaller batch sizes, SoftClu remains stable and adds negligible computational overhead (approx. 0.014ms per iteration).
Key Insights
A critical finding in the study is the importance of combining geometric and feature-space prototypes. Ablation studies revealed that feature-space clustering alone is insufficient; incorporating spatial coordinates forces the model to learn that proximity in 3D space often correlates with semantic similarity.
Furthermore, the use of Optimal Transport acts as a powerful functional regularizer. It prevents the network from mapping all inputs to the same output, ensuring the model utilizes its full capacity to describe the variety in the point cloud.
Discussion and Conclusion
SoftClu represents a significant shift away from augmentation-dependent unsupervised learning in 3D vision. By framing representation learning as a constrained clustering problem, it achieves state-of-the-art results while being more robust and easier to train than contrastive alternatives.
While highly effective, the framework currently relies on a fixed cluster count ($J=64$), and future work could explore dynamic ways to determine the natural number of clusters in a scene. Additionally, while it performs well across different datasets, the transition from synthetic pre-training to real-world testing still offers room for improvement in domain adaptation. Overall, SoftClu provides a scalable and efficient path forward for 3D point cloud understanding.
Summary updated