Abstract
Estimating 3D scene flow from LiDAR data is crucial for autonomous navigation but suffers from a lack of ground-truth labels. This paper introduces a self-supervised framework that jointly optimizes point motion and object clustering. By integrating overlapping "soft" clusters with evolving "hard" clusters, the method achieves state-of-the-art accuracy on LiDAR benchmarks—particularly for small, dynamic objects—while significantly outperforming previous optimization-based methods in speed.
Introduction
3D scene flow estimation—calculating the motion vector for every point in a LiDAR scan—is a fundamental task for collision avoidance and tracking in autonomous driving. However, the high cost of manual labeling makes self-supervised learning the preferred approach. Existing methods often rely on rigid-body assumptions and two-step pipelines: clustering points into objects and then optimizing their motion. This frequently leads to under-segmentation (merging distinct moving objects) or over-segmentation (splitting a single object), both of which degrade flow accuracy.
Furthermore, many state-of-the-art methods rely on "Neural Priors" (using a neural network as a regularizer). While popular, these can over-smooth motion and struggle to distinguish multiple independent objects moving close to one another. This paper proposes a more expressive, optimization-based alternative.
Methodology: Joint Optimization of Flow and Clustering
The proposed "Neural-Prior-free" approach avoids the over-smoothing common in neural-network-based regularizers. Instead, it uses a mechanical "spring" analogy to balance local rigidity with global scene structure through three primary loss functions:
- Hard Rigid Clusters: These are small, non-overlapping groups assumed to be part of a single rigid body. During optimization, these clusters can grow and merge if their estimated motions align.
- Soft Rigid Clusters: These fixed-size, overlapping neighborhoods allow for outlier rejection. Using spectral clustering, the system identifies which points within a neighborhood share a common rigid motion.
- Loss Functions: A distance loss pulls points toward their predicted future positions, while rigidity losses (with varying stiffness) maintain the structural integrity of the clusters.
By optimizing these components simultaneously, the flow field and object segmentation refine each other iteratively. The use of overlapping soft clusters is particularly powerful, as it allows the algorithm to "bridge" the gap between different rigid parts and resolve motion correctly even when a point is an outlier in one cluster but an inlier in another.
Experimental Results
The method was evaluated on major autonomous driving benchmarks including Argoverse 1 & 2, Waymo Open Dataset, and StereoKITTI.
State-of-the-Art Accuracy
The framework achieved a 38% improvement in End-Point Error (EPE) on StereoKITTI compared to previous leaders like SCOOP. Notably, the method excels in the "long-tail" of object classes—pedestrians and cyclists—which are often mismanaged by traditional clustering due to their small size and proximity to static backgrounds.
On Argoverse 2, the method significantly outperformed competitors on dynamic objects. For pedestrians, it achieved an EPE of 0.031m, compared to 0.062m for the popular NSFP method.
Efficiency and Expressiveness
By moving beyond Neural Priors, the optimization becomes more expressive in dynamic scenes. Furthermore, the use of fixed-size soft clusters allows for high GPU parallelization, resulting in speeds up to 10x faster than competing optimization-based methods like MBNSFP while maintaining lower error rates.
Conclusion and Future Work
This work demonstrates that simultaneous optimization of flow and clustering can overcome the limitations of rigid-body heuristics and neural regularizers. While currently an inference-time optimization, the method is significantly more efficient than its predecessors. Future work aims to integrate these structural constraints into feed-forward neural networks to combine the speed of deep learning with the robustness of this optimization framework, potentially enabling real-time performance for vehicle control.
Summary updated