RigidFlow: Self-Supervised Scene Flow Learning on Point Clouds by Local Rigidity Prior

Ruibo Li, Chi Zhang, Guosheng Lin, Zhe Wang, Chunhua Shen

Abstract

Scene flow estimation is a fundamental task for autonomous driving and robotics, providing a 3D motion vector for every point in a scene. However, the scarcity of large-scale, human-labeled 3D motion datasets limits the performance of supervised models in real-world scenarios. We present RigidFlow, a self-supervised framework that leverages a local rigidity prior to generate high-quality pseudo-labels. By decomposing scenes into supervoxels and assuming piecewise rigid motion, RigidFlow produces structured supervision that significantly reduces noise compared to point-wise similarity metrics. Our method achieves state-of-the-art results on standard benchmarks, even outperforming supervised models in cross-domain generalization.

The Challenge of Scene Flow Estimation

Scene flow is the 3D analog of optical flow, essential for understanding dynamic environments. While synthetic datasets like FlyingThings3D provide perfect ground truth, models trained on them often fail to generalize to the noise and sparsity of real-world LiDAR data.

Existing self-supervised approaches typically rely on point-wise matching (e.g., Chamfer loss or nearest neighbor search). These methods often produce jittery, physically inconsistent flow because they lack an understanding of object structure. As shown in the comparison below, standard nearest-neighbor searches result in noisy pseudo-labels that do not respect the underlying geometry of objects like vehicles.

Comparison of pseudo-label generation methods: (a) Input point clouds, (b) Nearest search results showing significant noise, and (c) RigidFlow's structured pseudo-labels.

The RigidFlow Framework

RigidFlow addresses these limitations by enforcing a local rigidity prior. The core assumption is that most real-world scenes consist of objects that move rigidly or can be approximated as piecewise rigid (e.g., a walking person).

The pipeline, illustrated in the figure below, consists of four main stages:

  1. Over-segmentation: The source point cloud is partitioned into small, geometrically consistent clusters called supervoxels.
  2. Piecewise Rigid Motion Estimation: For each supervoxel, the algorithm estimates a single rigid transformation (rotation and translation) rather than independent point-wise vectors.
  3. Iterative Refinement: A modified Iterative Closest Point (ICP) approach is used to find the optimal transformation. Crucially, this process is initialized using the current predictions of the neural network. This creates a "virtuous cycle" where the network and pseudo-labels mutually improve during training.
  4. Pseudo-Label Training: The refined rigid transformations serve as pseudo-labels to train a standard scene flow backbone (e.g., FLOT or FlowNet3D).
The RigidFlow pipeline, illustrating the over-segmentation and iterative piecewise rigid motion estimation process.

Key Insights: Structure and Bootstrapping

The success of RigidFlow stems from two primary innovations:

  • Structure over Similarity: By forcing motion to be spatially consistent within a supervoxel, the model ignores spurious point matches that are physically impossible. This "structural" constraint acts as a powerful regularizer.
  • Self-Bootstrapping: Using the network's own output to initialize the pseudo-label generation solves the "chicken and egg" problem of self-supervision. As the model learns the general motion of the scene, it provides better initializations for the rigid registration, leading to increasingly accurate pseudo-labels.

Experimental Results and Benchmarks

RigidFlow was evaluated on the FlyingThings3D (FT3D) and KITTI datasets, considering both occluded and non-occluded scenarios.

  • State-of-the-Art Accuracy: On FT3D, RigidFlow achieved an End Point Error (EPE) of 0.069m, a significant improvement over previous self-supervised methods (~0.085m).
  • Superior Generalization: When trained on real-world KITTI data, RigidFlow outperformed several supervised models that were trained on synthetic data. This highlights the value of self-supervision on domain-specific, real-world data over supervised learning on synthetic proxies.
  • Computational Efficiency: The pseudo-label generation process is highly efficient, averaging 172ms per sample, which facilitates large-scale training.

Conclusion and Future Work

RigidFlow demonstrates that a simple physical prior—local rigidity—can bridge the gap between self-supervised and supervised scene flow learning. While highly effective for most urban scenes, the method faces challenges with strongly non-rigid motions (e.g., fluid dynamics) and extreme occlusions. Future research may explore more complex motion models and enhanced occlusion handling to further improve the robustness of self-supervised 3D perception.

Summary updated

Abstract

In this work, we focus on scene flow learning on point clouds in a self-supervised manner. A real-world scene can be well modeled as a collection of rigidly moving parts, therefore its scene flow can be represented as a combination of rigid motion of each part. Inspired by this observation, we propose to generate pseudo scene flow for self-supervised learning based on piecewise rigid motion estimation, in which the source point cloud is decomposed into a set of local regions and each region is treated as rigid. By rigidly aligning each region with its potential counterpart in the target point cloud, we obtain a region-specific rigid transformation to represent the flow, which together constitutes the pseudo scene flow labels of the entire scene to enable network training. Compared with most existing approaches relying on point-wise similarities for point matching, our method explicitly enforces region-wise rigid alignments, yielding locally rigid pseudo scene flow labels. We demonstrate the effectiveness of our self-supervised learning method on FlyingThings3D and KITTI datasets. Comprehensive experiments show that our method achieves new state-of-the-art performance in self-supervised scene flow learning, without any ground truth scene flow for supervision, even outperforming some supervised counterparts.
BibTeX
@inproceedings{li2022rigidflow,
  author = {Ruibo Li and Chi Zhang and Guosheng Lin and Zhe Wang and Chunhua Shen},
  title = {{RigidFlow: Self-Supervised Scene Flow Learning on Point Clouds by Local Rigidity Prior}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2022},
  doi = {10.1109/cvpr52688.2022.01645},
}
Figure