Abstract
Scene flow estimation is a fundamental task for autonomous driving and robotics, providing a 3D motion vector for every point in a scene. However, the scarcity of large-scale, human-labeled 3D motion datasets limits the performance of supervised models in real-world scenarios. We present RigidFlow, a self-supervised framework that leverages a local rigidity prior to generate high-quality pseudo-labels. By decomposing scenes into supervoxels and assuming piecewise rigid motion, RigidFlow produces structured supervision that significantly reduces noise compared to point-wise similarity metrics. Our method achieves state-of-the-art results on standard benchmarks, even outperforming supervised models in cross-domain generalization.
The Challenge of Scene Flow Estimation
Scene flow is the 3D analog of optical flow, essential for understanding dynamic environments. While synthetic datasets like FlyingThings3D provide perfect ground truth, models trained on them often fail to generalize to the noise and sparsity of real-world LiDAR data.
Existing self-supervised approaches typically rely on point-wise matching (e.g., Chamfer loss or nearest neighbor search). These methods often produce jittery, physically inconsistent flow because they lack an understanding of object structure. As shown in the comparison below, standard nearest-neighbor searches result in noisy pseudo-labels that do not respect the underlying geometry of objects like vehicles.
The RigidFlow Framework
RigidFlow addresses these limitations by enforcing a local rigidity prior. The core assumption is that most real-world scenes consist of objects that move rigidly or can be approximated as piecewise rigid (e.g., a walking person).
The pipeline, illustrated in the figure below, consists of four main stages:
- Over-segmentation: The source point cloud is partitioned into small, geometrically consistent clusters called supervoxels.
- Piecewise Rigid Motion Estimation: For each supervoxel, the algorithm estimates a single rigid transformation (rotation and translation) rather than independent point-wise vectors.
- Iterative Refinement: A modified Iterative Closest Point (ICP) approach is used to find the optimal transformation. Crucially, this process is initialized using the current predictions of the neural network. This creates a "virtuous cycle" where the network and pseudo-labels mutually improve during training.
- Pseudo-Label Training: The refined rigid transformations serve as pseudo-labels to train a standard scene flow backbone (e.g., FLOT or FlowNet3D).
Key Insights: Structure and Bootstrapping
The success of RigidFlow stems from two primary innovations:
- Structure over Similarity: By forcing motion to be spatially consistent within a supervoxel, the model ignores spurious point matches that are physically impossible. This "structural" constraint acts as a powerful regularizer.
- Self-Bootstrapping: Using the network's own output to initialize the pseudo-label generation solves the "chicken and egg" problem of self-supervision. As the model learns the general motion of the scene, it provides better initializations for the rigid registration, leading to increasingly accurate pseudo-labels.
Experimental Results and Benchmarks
RigidFlow was evaluated on the FlyingThings3D (FT3D) and KITTI datasets, considering both occluded and non-occluded scenarios.
- State-of-the-Art Accuracy: On FT3D, RigidFlow achieved an End Point Error (EPE) of 0.069m, a significant improvement over previous self-supervised methods (~0.085m).
- Superior Generalization: When trained on real-world KITTI data, RigidFlow outperformed several supervised models that were trained on synthetic data. This highlights the value of self-supervision on domain-specific, real-world data over supervised learning on synthetic proxies.
- Computational Efficiency: The pseudo-label generation process is highly efficient, averaging 172ms per sample, which facilitates large-scale training.
Conclusion and Future Work
RigidFlow demonstrates that a simple physical prior—local rigidity—can bridge the gap between self-supervised and supervised scene flow learning. While highly effective for most urban scenes, the method faces challenges with strongly non-rigid motions (e.g., fluid dynamics) and extreme occlusions. Future research may explore more complex motion models and enhanced occlusion handling to further improve the robustness of self-supervised 3D perception.
Summary updated