Regularizing Self-supervised 3D Scene Flows with Surface Awareness and Cyclic Consistency

Patrik Vacek, David Hurych, Karel Zimmermann, Patrick Pérez, Tomás SvobodaarXiv 2023 · DOI · arXiv

Abstract

Self-supervised 3D scene flow estimation is critical for autonomous driving but often suffers from degenerate solutions due to a lack of strong geometric constraints. We present two novel, model-agnostic regularization terms: Surface-aware Smoothness ($L_{surf}$) and Cyclic Consistency ($L_{cyc}$). By incorporating surface normals and temporal forward-backward loops, our method better defines rigid clusters, preventing motion "bleeding" at object boundaries and enlarging consistent regions. Our approach achieves state-of-the-art performance across four major datasets (KITTI, Argoverse, nuScenes, Waymo), reducing End-Point-Error by up to 35%.

Introduction

Scene flow estimation—predicting the 3D motion of every point in a point cloud between consecutive frames—is a fundamental task for autonomous systems to perceive dynamic environments. While supervised methods perform well, the scarcity of ground-truth 3D motion labels in real-world LiDAR data has shifted focus toward self-supervised learning.

However, self-supervised models often produce "degenerate" solutions where rigid objects appear to warp or fragment. Existing regularization techniques typically rely on spatial proximity to enforce smoothness, but these fail at object boundaries (e.g., a car moving past a static wall) or produce clusters that are too small to provide a robust training signal. This paper addresses these limitations by introducing geometric and temporal constraints that better capture the underlying rigidity of 3D scenes.

Proposed Methodology

The core of our contribution is a set of two "plug-and-play" loss functions that can be integrated into various scene flow architectures. These losses refine the definition of "rigid clusters"—groups of points expected to move with the same transformation.

1. Surface-aware Smoothness ($L_{surf}$)

Standard smoothness losses assume that spatially close points should move together. We enhance this by adding a geometric constraint: points must be close and have similar surface orientations (normals). This prevents the motion of a dynamic object from incorrectly "bleeding" into adjacent static surfaces, such as the road or a nearby building.

2. Cyclic Consistency ($L_{cyc}$)

To overcome the limitation of small, localized clusters, we enforce a forward-backward temporal loop. By projecting points into the target frame and ensuring their backward flow returns them to a consistent neighborhood in the source frame, we can identify and enforce larger rigid clusters. This provides a significantly stronger regularization signal than looking at the source cloud in isolation.

Figure 2: Visual comparison of baseline smoothness versus the proposed surface-aware and cyclic consistency constraints.

Experimental Setup and Datasets

We evaluated our framework across four diverse autonomous driving datasets to ensure robustness across different sensor types and sampling patterns:

  • stereoKITTI: Generated from stereo imagery.
  • Argoverse & nuScenes: Large-scale LiDAR datasets with complex dynamic scenes.
  • Waymo Open Dataset: High-resolution LiDAR data.

Performance was measured using End-Point-Error (EPE), comparing the predicted flow against pseudo-ground-truth derived from object annotations.

Results and Discussion

Our proposed losses consistently improved the performance of baseline architectures like SCOOP and Neural Prior. On the KITTI dataset, adding our regularization to the SCOOP model resulted in a 35% reduction in EPE. Similar gains were observed on the nuScenes dataset, where the Neural Prior baseline saw a 32% improvement.

Figure 1: Overview of the proposed regularization framework and quantitative End-Point-Error (EPE) results across major datasets.

Qualitative results demonstrate that our method effectively "corrects" the rigidity of moving objects. While baseline models often struggle to move a vehicle as a single unit, our surface-aware and cyclic constraints ensure that the entire body of the car moves consistently, respecting its physical boundaries.

Conclusion

By moving beyond simple spatial proximity and incorporating surface geometry and temporal consistency, we provide a more robust foundation for self-supervised scene flow. Our model-independent losses achieve state-of-the-art results across multiple LiDAR and stereo datasets. Future work will explore extending cyclic consistency across multiple frames to further stabilize long-term motion prediction and addressing the challenges of extremely sparse point clouds where surface normal estimation is less reliable.

Summary updated

Abstract

Learning without supervision how to predict 3D scene flows from point clouds is essential to many perception systems. We propose a novel learning framework for this task which improves the necessary regularization. Relying on the assumption that scene elements are mostly rigid, current smoothness losses are built on the definition of "rigid clusters" in the input point clouds. The definition of these clusters is challenging and has a significant impact on the quality of predicted flows. We introduce two new consistency losses that enlarge clusters while preventing them from spreading over distinct objects. In particular, we enforce \emph{temporal} consistency with a forward-backward cyclic loss and \emph{spatial} consistency by considering surface orientation similarity in addition to spatial proximity. The proposed losses are model-independent and can thus be used in a plug-and-play fashion to significantly improve the performance of existing models, as demonstrated on two most widely used architectures. We also showcase the effectiveness and generalization capability of our framework on four standard sensor-unique driving datasets, achieving state-of-the-art performance in 3D scene flow estimation. Our codes are available on https://github.com/ctu-vras/sac-flow.
BibTeX
@misc{vacek2023regularizing,
  author = {Patrik Vacek and David Hurych and Karel Zimmermann and Patrick Pérez and Tomás Svoboda},
  title = {{Regularizing Self-supervised 3D Scene Flows with Surface Awareness and Cyclic Consistency}},
  year = {2023},
  eprint = {2312.08879},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2312.08879},
}
Figure