Annotating 3D data is a notorious bottleneck in computer vision. While a 2D image can be labeled in seconds, a single 3D LiDAR scan or medical volume can take over 20 minutes to annotate point-by-point. This "annotation tax" has historically limited the scale of 3D datasets.
However, a recurring insight across recent research is that 3D scenes are highly redundant. Because 3D space is governed by geometric continuity—points that are close together and share similar normals usually belong to the same object—we do not need to label every voxel or point to train high-performance models. By leveraging sparse supervision, researchers are achieving upwards of 97% of the performance of fully supervised models using as little as 0.01% of the original labels.
The Redundancy of Density
The core intuition behind sparse supervision is that the informational value of 3D points is non-uniform. In a dense point cloud of a room, millions of points represent flat surfaces like walls or floors. Labeling every single one of these points provides diminishing returns.
Xun Xu demonstrated that semantic segmentation models can maintain high accuracy even when 90% of point-wise annotations are removed. This suggests that the underlying geometric features of a scene can be captured by a sparse skeleton of labels. Taking this to the extreme, Zhiyi Pan and Ge Li showed that by strategically recommending which points to label—focusing on high-information regions rather than random sampling—models can reach 90% of full-supervision performance with only 0.01% of the data labeled.
One Thing One Click
If dense labeling is redundant, what is the minimum viable signal? For many researchers, the answer is a single interaction per object.
The "One Thing One Click" (1T1C) framework, pioneered by XIAOJUAN QI and Zhengzhe Liu, operates on the principle that a single click on an object provides two critical pieces of information: the object's semantic class and its rough location. By combining this "seed" with a 3D U-Net and a relation network, the model can propagate that single label to all geometrically similar points in the vicinity.
This propagation works because 3D data provides a stronger inductive bias than 2D images. In 2D, a "chair" might be split by an occluding table leg; in 3D, the chair's points form a distinct, continuous cluster in space. Shichao Dong and Guosheng Lin extended this by using "competition" between multiple instance graphs, allowing the model to autonomously figure out object boundaries from these single-point seeds.
Propagation Through Consistency
To turn sparse clicks into dense masks, models must learn how to "fill in the blanks." This is typically achieved through three types of consistency:
- Geometric Consistency: Points that are spatially close and share similar local geometry (like surface normals) should share labels. Kangcheng Liu used boundary-aware energy losses to ensure that label propagation stops at sharp geometric edges.
- Multi-View Consistency: If a scene is captured from multiple angles, the labels must be consistent across all views. Shuaifeng Zhi and Tristan Laidlow demonstrated this with Semantic-NeRF, where a neural radiance field jointly encodes geometry and semantics. By labeling just two keyframes in a video, the multi-view consistency of the NeRF propagates those labels to the entire 3D volume.
- Cross-Modal Consistency: 2D images often contain textures that 3D points lack. Xiaoyu Zhu and Jeffrey Chen showed that image-level class labels can guide 3D segmentation by identifying which regions of a point cloud correspond to specific visual textures.
Bounding Boxes as Weak Proxies
When point-level clicks are unavailable, 3D bounding boxes serve as an effective middle ground. While a box is "noisy" (it contains both the object and some empty space), it provides a clear spatial constraint.
Francis Engelmann and Julian Chibane developed Box2Mask, which uses a Hough voting scheme to infer dense instance masks from these boxes. Similarly, Xiuwei Xu showed that even "center-point" clicks—taking only 5 seconds to produce—can be used to retrieve synthetic 3D shapes that act as "perfectly labeled" proxies for real-world objects.
The Future of Label Efficiency
The shift toward sparse supervision suggests a future where 3D models are trained not on exhaustive manual labor, but on high-level intent. Whether through a single click per object, a handful of 2D slices in a medical volume (Yuting Ji), or scene-level checklists (Xuexun Liu), the goal is the same: leveraging the inherent structure of the 3D world to minimize human effort. As Ozan Unal and Dengxin Dai demonstrated with scribble-based LiDAR segmentation, we can now achieve nearly 96% of full-supervision performance while ignoring 92% of the points. In 3D vision, it appears that less truly is more.