NccFlow: Unsupervised Learning of Optical Flow With Non-occlusion from Geometry

Guangming Wang, Shuaiqi Ren, Hesheng Wang

Abstract

NccFlow introduces a novel framework for unsupervised optical flow estimation by leveraging geometric constraints inherent in physical motion. By enforcing two new "laws"—non-intersection and non-blocking—within visible regions, the model significantly improves its ability to generalize from synthetic training data to complex real-world environments. This approach addresses the limitations of traditional photometric consistency, which often fails in the presence of occlusions or lighting changes.

Introduction

Optical flow estimation, the process of calculating pixel-wise motion between consecutive video frames, is critical for autonomous systems like self-driving cars and robotics. While supervised deep learning models have achieved high accuracy, they often struggle with the "domain gap" when moving from synthetic datasets to messy real-world footage. Unsupervised learning offers a more scalable alternative by using "warping" techniques to match frames based on color. However, these methods typically ignore the strict geometric rules that must apply to non-occluded (visible) areas. NccFlow bridges this gap by introducing geometric loss functions that act as a "physicality check" for predicted motion.

Methodology

Built upon the PWC-Net architecture, NccFlow integrates two novel geometric loss functions into a standard unsupervised pipeline alongside Census and Smoothness losses.

Optical Flow Non-intersection Loss ($L_{non-inter}$)

In a continuous physical world, neighboring points on a surface should not cross each other's paths as they move. The non-intersection loss evaluates a $3 \times 3$ sliding window to check the trajectories of a center pixel and its eight neighbors. If the model proposes motion paths that intersect, it is penalized. This ensures that the resulting flow field remains topologically consistent.

The non-intersection principle ensures that pixel trajectories do not cross in visible regions, maintaining geometric consistency during deformation.

Optical Flow Non-blocking Loss ($L_{non-block}$)

This constraint ensures that a cluster of pixels moves as a coherent unit without "swallowing" or overlapping with its neighbors. The model defines a quadrilateral formed by four adjacent pixels and monitors whether surrounding pixels "flow into" this area in the subsequent frame. Such events suggest unphysical blocking or overlapping, which should not occur in visible regions.

The non-blocking constraint prevents unphysical pixel overlaps during large motions of multiple objects.

Datasets and Results

The authors evaluated NccFlow on industry-standard benchmarks, including Flying Chairs, MPI Sintel, and KITTI 2015.

  • Superior Generalization: When trained on synthetic data and tested on the real-world KITTI dataset, NccFlow achieved an Outlier Rate (ER%) of 18.63%, outperforming the supervised version of PWC-Net.
  • Sintel Performance: On the Sintel "Clean" test set, NccFlow achieved an Endpoint Error (EPE) of 4.26, a notable improvement over the baseline UFlow model (5.21).
  • Qualitative Sharpness: Visually, the model produces much cleaner boundaries around moving objects, correctly estimating uniform motion for complex shapes like walking pedestrians or swinging tools where other models produce fragmented results.

Conclusion

NccFlow demonstrates that geometry is a universal language that can bridge the gap between synthetic and real-world data. By forcing neural networks to obey physical laws like non-intersection and non-blocking, the researchers have created a more robust and accurate tool for motion estimation. Future work may extend these 2D geometric principles to 3D scene flow and depth estimation, where the relationship between geometry and motion is even more direct.

Summary updated

BibTeX
@article{wang2022nccflow,
  author = {Guangming Wang and Shuaiqi Ren and Hesheng Wang},
  title = {{NccFlow: Unsupervised Learning of Optical Flow With Non-occlusion from Geometry}},
  journal = {IEEE Trans. Intell. Transp. Syst.},
  year = {2022},
  url = {https://openreview.net/forum?id=YmdzLuhhX9},
}
Figure