OAMaskFlow: Occlusion-Aware Motion Mask for Scene Flow

Xiongfeng Peng, Zhihua Liu, Weiming Li, Yamin Mao, Qiang WangAAAI 2025 · DOI

TL;DR

This paper introduces OAMaskFlow, a novel scene flow estimation method that tackles the challenge of occlusions and inconsistent motion cues by introducing an "occlusion-aware motion" (OAM) mask. This mask helps to better supervise motion embeddings and a subsequent propagation module, leading to more accurate 3D motion estimation, especially in challenging regions.

What Problem Does This Paper Address?

Estimating scene flow, which involves determining the 3D motion and structure of objects in a scene between consecutive frames, is crucial for applications like robotics and autonomous driving. Existing methods often struggle with occluded regions where visual information is missing or inconsistent. While some approaches try to group pixels into rigid objects, their implicitly learned motion embeddings are insufficient for handling these challenging areas. This leads to inaccurate motion predictions, particularly when objects are partially or fully occluded, or when there are inconsistencies in geometric properties across views. The paper aims to improve the robustness and accuracy of scene flow estimation by explicitly addressing these occlusion and motion ambiguity issues.

Methods

The core novelty of OAMaskFlow lies in its Occlusion-Aware Motion (OAM) mask. This mask is generated using ground truth scene flow and camera poses, and it distinguishes between:

  • Occluded pixels: Marked as black, these are regions where information is missing due to occlusion or being out of view.
  • Static pixels: Marked as dark red, these pixels move with the camera's ego-motion.
  • Dynamic object pixels: All other pixels, representing objects with their own motion.

This OAM mask is then used to supervise the motion embedding learned by the network. This explicit supervision helps the network learn more informative and reliable motion representations, enabling better grouping of pixels with similar motion.

Furthermore, OAMaskFlow incorporates a 3D motion propagation module. This module leverages an attention mechanism to propagate accurate 3D motion from reliable, non-occluded regions to the challenging occluded areas, improving overall scene flow accuracy. The method builds upon the RAFT-3D architecture, enhancing its end-to-end differentiable framework.

Datasets

The paper uses the following datasets:

  • FlyingThings3D: Used for training and evaluation. This dataset provides dense, accurate, and multi-task ground truth, including instance labels and camera poses, which are crucial for generating the OAM mask.
  • KITTI Scene Flow Benchmark: Used for evaluation. This dataset consists of stereo images with sparse ground truth. The authors note that depth information is not directly available and is generated using GA-Net.
  • TartanAir: Used for evaluating the application of OAM mask in SLAM. This is a synthetic dataset with realistic simulation environments, providing stereo RGB images, depth, ground truth segmentation, optical flow, and camera poses.

Key Insights and Ideas

  • Explicitly supervising motion embeddings with an OAM mask is more effective than implicit learning: The paper demonstrates that by creating a specific mask that highlights occlusions and static vs. dynamic regions, the network can learn more robust motion representations, especially in challenging scenarios.
  • OAM mask is a versatile tool: The authors show that the OAM mask concept can be applied beyond scene flow estimation, proving effective in improving Simultaneous Localization and Mapping (SLAM) performance.
  • Motion propagation is crucial for occluded regions: The proposed attention-based propagation module effectively transfers reliable motion information to areas where direct estimation is difficult, significantly boosting accuracy.
  • Reduced model complexity: By leveraging the OAM mask and motion propagation, OAMaskFlow achieves state-of-the-art results with significantly fewer parameters compared to some previous methods.

Key Results

  • Scene Flow Performance:
    • On the FlyingThings3D dataset, OAMaskFlow reduced the EPE3D metric by 21.0% and the SF-all metric by 24.3% compared to the baseline RAFT-3D.
  • On the KITTI Scene Flow benchmark, the method achieved the lowest scene flow error on both all pixels and background regions, with a reduction in erroneous pixels from 5.77% to 4.37% compared to the baseline.
  • SLAM Performance:
    • When integrated into the DROID-SLAM framework (named OAM-DROID-SLAM), the TartanAir monocular dataset saw a decrease in the ATE metric by 65.7%.
  • On the TartanAir stereo dataset, the ATE metric decreased by 58.3%.
  • Parameter Efficiency: OAMaskFlow achieved these improvements with a significant reduction in parameters (83.2% fewer than RAFT-3D).

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that if an entire object is occluded, accurately estimating its scene flow remains challenging due to insufficient measurements.
  • The current approach simply concatenates RGB and depth images, similar to RAFT-3D, which may limit the full utilization of 3D structural information from depth.
  • Future Work:
    • The authors suggest exploring scene flow estimation from image and point cloud data using the OAM mask.
  • Another promising direction is scene flow estimation with 3D Gaussian Splatting.

Summary updated

BibTeX
@inproceedings{peng2025oamaskflow,
  author = {Xiongfeng Peng and Zhihua Liu and Weiming Li and Yamin Mao and Qiang Wang},
  title = {{OAMaskFlow: Occlusion-Aware Motion Mask for Scene Flow}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i6.32696},
}
Figure