Abstract
The Perturbed Self-Distillation (PSD) framework addresses the significant "annotation bottleneck" in 3D deep learning. It enables high-quality semantic segmentation of large-scale point clouds using as little as 1% of the data labeled, or even just a single point per class. By enforcing consistency between original and perturbed versions of a point cloud and explicitly modeling the relationships between labeled "anchor" points, PSD achieves results comparable to fully supervised methods while drastically reducing manual labeling effort.
Introduction: The Annotation Bottleneck
Semantic segmentation—the process of assigning a category (e.g., chair, floor, wall) to every point in a 3D scene—is a foundational task for autonomous systems and robotics. However, modern large-scale point clouds contain millions of points, making manual annotation prohibitively expensive. For instance, labeling a single scene in the ScanNet dataset can take over 22 minutes.
While weakly supervised methods aim to reduce this burden, they often struggle with large-scale data. These methods frequently fail to extract discriminative features for the vast majority of unlabeled points and ignore the underlying structural relationships (topology) of the point cloud. This results in "bleeding" errors, where the model fails to distinguish between adjacent objects with similar structures, such as a door and the surrounding wall.
The PSD Framework
The Perturbed Self-Distillation (PSD) framework is designed to be "backbone-agnostic," allowing it to enhance existing efficient architectures like RandLA-Net. It relies on two core innovations:
1. Perturbed Branch and Self-Distillation
PSD processes two versions of the same point cloud: the original and a perturbed version. Perturbations include standard geometric transformations (rotation, jittering) and a novel Attribute Attention layer. This layer learnably weights input features—such as color versus spatial coordinates—allowing the model to prioritize the most reliable information for a given context. By minimizing the Jensen-Shannon divergence between the predictions of both branches, the network learns to propagate labels from the few annotated points to their unlabeled neighbors.
2. Context-Aware Module (CAM)
To leverage the limited labels effectively, PSD treats them as "anchors." The CAM calculates a Point Affinity Matrix to model the relationships between these anchor points. By ensuring that labeled points of the same category have similar feature representations, the module regularizes the feature space and refines the global graph topology of the entire scene.
Key Results
The PSD framework was evaluated on three major benchmarks: S3DIS (indoor), ScanNet-v2 (RGB-D scans), and Semantic3D (outdoor urban environments).
- Efficiency with 1% Labels: On S3DIS (Area-5), PSD achieved a mean Intersection over Union (mIoU) of 63.5% with only 1% labels, outperforming previous weakly supervised methods and matching the performance of the fully supervised RandLA-Net (63.0%).
- The "1pt" Challenge: In an extreme scenario with only one labeled point per category, PSD reached 48.2% mIoU, a 7.5% improvement over the baseline.
- Outdoor Scalability: On the Semantic3D dataset, PSD showed a 6.9% gain over the baseline, demonstrating its effectiveness in complex, noisy outdoor environments.
Discussion and Insights
The success of PSD highlights several key principles for weakly supervised learning in 3D:
- Consistency as Supervision: Enforcing prediction consistency across perturbations allows the model to discover the underlying data structure without human intervention.
- Dynamic Feature Weighting: The Attribute Attention mechanism is crucial for handling noisy data, allowing the model to ignore deceptive features like inconsistent lighting or color.
- Relational Anchoring: Treating labeled points as relational anchors rather than isolated signals significantly improves the model's holistic understanding of the scene.
Conclusion
While PSD significantly reduces the labeling requirement, it introduces computational complexity due to the memory-intensive nature of the affinity matrix. To mitigate this, the Context-Aware Module is typically applied after the encoder stage. Future research will focus on optimizing these relational modules to further bridge the gap between weak and full supervision in even larger datasets.
Summary updated