3D shape segmentation—the process of partitioning a 3D object into semantic parts like legs, seats, or arms—is a cornerstone of 3D scene understanding. However, the field faces a major bottleneck: the high cost of supervision. Deep learning models typically require massive, high-quality labeled datasets, and manually annotating 3D meshes is significantly more labor-intensive than labeling 2D images.
This paper presents ROSS (Robust Learning of One-Shot 3D Shape Segmentation), a framework designed to tackle the "data hunger" of 3D computer vision. ROSS is a semi-supervised framework that performs 3D shape segmentation using only a single labeled exemplar. By combining a novel "reference weighted convolution" with a shape-morphing and label-transferring pipeline, it matches the performance of many fully supervised models and remains robust even when dealing with noisy or incomplete 3D meshes.
The ROSS Architecture
The ROSS architecture is built around three primary modules that transform raw mesh data into semantic segmentations through geometric warping and feature extraction.
- Global Shape Descriptor Generator: The authors introduce a Reference Weighted Convolution layer. Unlike standard convolutions that might lose spatial associations in irregular 3D grids, this layer uses an attention mechanism to weight the influence of neighboring points. It processes raw mesh points directly, avoiding the need for expensive pre-computations like surface normals or curvature.
- Part-Aware Shape Descriptor Constructor: To make the segmentation "aware" of different parts, the global descriptor is split into $k$ segments (where $k$ is the number of semantic parts). Each segment passes through a part-specific MLP (Multi-Layer Perceptron) to create a signature that highlights specific regions of the shape.
- Shape Morphing and Label Transfer: Instead of classifying every point on a new mesh from scratch, the model takes the labeled exemplar and "morphs" (warps) it to match the geometry of the target input mesh. The morphing is guided by a loss function combining Chamfer loss (for overall shape similarity), Edge loss, and Laplacian loss (to prevent mesh distortion). Once the exemplar is morphed to fit the target, the labels are transferred to the target mesh using a nearest-neighbor mapping.
Key Results and Robustness
The authors evaluated ROSS on the Princeton Segmentation Benchmark (PSB) and the SCAPE human dataset. On the PSB dataset, ROSS achieved an average accuracy of 90.11%. For context, this is only about 4.6% lower than state-of-the-art fully supervised models, and it actually outperformed several older supervised architectures.
A standout feature of ROSS is its robustness to incompleteness. On the SCAPE human dataset, ROSS achieved 94.02% accuracy on complete meshes. When tested on incomplete meshes with significant "holes" to simulate real-world sensor data, the accuracy only dropped to 92.82%, demonstrating high resilience to poor data quality.
Conclusion
ROSS demonstrates that accurate 3D segmentation can be achieved with minimal supervision by reframing the problem as a shape-morphing task. By preserving topological consistency through label transfer and utilizing attention-based local features, the framework provides a robust solution for 3D vision tasks where labeled data is scarce. Future work aims to address accuracy dips at part boundaries and improve performance under extreme non-rigid deformations.
Summary updated