Abstract
Identifying individual objects within 3D point clouds is a fundamental task for autonomous systems, yet it traditionally relies on extensive human-annotated datasets. In this paper, we present OGC (Object Geometry Consistency), the first unsupervised framework designed to segment multiple 3D objects from raw point clouds in a single forward pass. By leveraging the rigid motion patterns inherent in sequential data as a natural supervisory signal, OGC learns to discover object geometries without any human labels. Our approach integrates a multi-object segmentation network with a self-supervised scene flow estimator and a novel geometry consistency component. Through extensive evaluation across five diverse datasets, we demonstrate that OGC effectively identifies object instances in both indoor and complex outdoor scenarios, significantly narrowing the performance gap between unsupervised and supervised methods.
Introduction
3D object segmentation—the process of partitioning a point cloud into distinct object instances—is critical for robotics and autonomous driving. While supervised deep learning has achieved remarkable success in this domain, the "data hunger" of these models presents a significant bottleneck. Labeling 3D point clouds is an arduous, expensive process that does not scale easily to the infinite variety of real-world environments.
Existing unsupervised methods generally fall into two categories, both with significant limitations. Motion-based segmentation can identify moving objects but often fails at inference time if the objects are stationary (the "parked car" problem). Conversely, spatial clustering (like DBSCAN) relies on physical gaps between objects and fails when objects are in close proximity, such as a chair tucked under a table. OGC overcomes these hurdles by using motion as a teacher during training to learn geometric representations that persist even when objects are static during testing.
The OGC Framework
The core philosophy of OGC is that rigid objects move as a single unit. By observing how points move together over time, a model can infer the underlying structure of an object. The framework consists of three primary pillars:
1. Architecture: Transformer-Based Segmentation
The model utilizes a PointNet++ backbone to extract local geometric features from a single point cloud frame. These features are processed by a Transformer decoder that outputs multiple object masks in parallel. Unlike previous methods that require temporal sequences during inference, OGC’s architecture is designed to predict masks from a single static frame once trained.
2. Supervision: Self-Supervised Scene Flow
During the training phase, OGC employs an auxiliary scene flow estimator. This component calculates the displacement of every point between two consecutive frames without requiring ground-truth labels. This flow provides the "natural" supervision needed to understand which points belong to the same rigid body.
3. The Geometry Consistency Component
To translate motion signals into robust geometric understanding, OGC employs three specific loss functions:
- Dynamic Rigid Consistency: This forces predicted masks to align with the estimated motion. If a group of points is identified as a single object, their collective motion must be mathematically explainable by a single rigid transformation (rotation and translation).
- Object Shape Smoothness: This constraint assumes spatial continuity, suggesting that neighboring points in 3D space are likely to share the same object identity.
- Object Shape Invariance: This is the key to generalization. It mandates that the model produce consistent object masks even if the input point cloud undergoes spatial transformations like rotation or scaling. This teaches the model that "object-ness" is a property of shape, not just position or motion.
Finally, an Iterative Optimization strategy is used to refine the results. The predicted masks are used to improve motion estimation via an object-aware Iterative Closest Point (ICP) algorithm, which in turn provides higher-quality supervision for the segmentation network.
Experimental Evaluation
OGC was evaluated across a spectrum of synthetic and real-world datasets, including SAPIEN (articulated objects), OGC-DR/DRSV (indoor scenes), and KITTI (outdoor traffic).
Performance Benchmarks
- SAPIEN Dataset: OGC achieved a 55.6 AP (Average Precision), vastly outperforming classical clustering (DBSCAN: 6.3 AP) and existing motion-based methods (SSC: 9.5 AP).
- KITTI-SF: In real-world outdoor scenarios, OGC reached 54.4 AP. This is particularly impressive as it approaches the performance of its fully supervised counterpart, OGC-sup, which scored 62.4 AP.
- Flow Estimation: The iterative refinement process also benefited the auxiliary task. OGC reduced the End-point Error (EPE3D) on KITTI-SF to 6.72, surpassing the state-of-the-art unsupervised flow estimator, FlowStep3D (10.21).
Discussion and Limitations
While OGC represents a significant step forward, it faces specific challenges. Because it relies on motion for supervision, the model may struggle to identify object classes that never move during the training phase (e.g., buildings or trees), as there is no dynamic signal to define their boundaries. Additionally, the current framework assumes rigidity; while it handles articulated parts well, highly deformable objects like pedestrians remain a challenge. Finally, OGC is an instance segmenter; it identifies that an object exists without assigning a semantic label (e.g., "car" or "pedestrian").
Conclusion
OGC demonstrates that the physical laws of motion can serve as a powerful, free substitute for human labels in 3D vision. By bridging the gap between dynamic training and static inference, OGC provides a scalable path toward truly autonomous perception in complex, unlabeled environments.
Summary updated