Abstract
Estimating optical flow between distant video frames remains a significant challenge due to increasing occlusions and compounding accumulation errors. We present AccFlow, a novel framework that utilizes a "backward accumulation" strategy to estimate long-range motion. By recursively fusing local flows in reverse order and employing a deformable AccPlus module alongside an adaptive blending mechanism, the framework effectively manages compounding errors and increasing occlusions in distant frames. Our approach demonstrates substantial improvements on the CVO and HS-Sintel benchmarks, particularly in complex, fast-moving scenes.
Introduction
Optical flow—the task of tracking pixel motion between frames—is well-solved for consecutive frames (local flow). However, estimating long-range optical flow (e.g., between frame 1 and frame 10) remains a major challenge. As the temporal gap between frames increases, two primary issues emerge:
- Occlusions increase: Objects move behind one another, making it impossible to find direct correspondences.
- Accumulation errors compound: Traditional "forward accumulation" (chaining local flows from the reference frame) causes small errors in each step to snowball. Furthermore, the area considered "occluded" relative to the first frame grows progressively larger, eventually overwhelming the estimator.
Methods
The AccFlow framework is built on three primary technical pillars designed to address the limitations of long-range flow estimation.
Backward Accumulation
Unlike traditional methods that accumulate flow from the reference frame forward ($1 \to 2, 1 \to 3, \dots$), AccFlow reverses this process. It starts from the target frame and works backward. The mathematical insight is that by aligning the large accumulated flow to the smaller local flow at each step, the "occlusion proportion" remains stable and minimal, rather than growing uncontrollably.
AccPlus and Adaptive Blending
The AccPlus module serves as the core engine for motion accumulation, utilizing deformable convolutions in the feature domain to align motion features accurately. It includes an internal "occlusion solver" that leverages spatial context to fill in gaps. To prevent recursive drift, the Adaptive Blending module incorporates a "direct" (but potentially noisy) estimate of the long-range flow as a prior, using an attention-based confidence mask to blend the accumulated flow with this direct estimate.
Key Insights: The Geometry of Accumulation
The most significant contribution of this work is the demonstration that backward accumulation is superior to forward accumulation. In forward accumulation, the occlusion mask grows monotonically with time. In contrast, with backward accumulation, the reference frame effectively "resets" at each step of the recursion. This keeps the occlusion area small and manageable, making the task significantly easier for the neural network to learn and execute.
Experimental Evaluation
The authors introduced two new datasets to evaluate long-range flow: CVO (Cross-frame Video Optical flow), a large-scale synthetic dataset, and HS-Sintel, a high-speed version of the classic Sintel dataset.
AccFlow was evaluated by integrating standard estimators like RAFT and GMA as its backbone. On the CVO (Final) benchmark, the Acc+GMA configuration achieved an Average End-Point Error (EPE) of 2.808, a massive improvement over the 6.265 EPE of the vanilla GMA model. The method particularly excels in occluded regions, reducing EPE in the HS-Sintel dataset by nearly 30% compared to previous state-of-the-art methods. Qualitative results confirm that AccFlow maintains sharp object boundaries and successfully recovers small objects with large motions that other methods often lose.
Conclusion and Future Work
AccFlow represents a significant step forward in long-range motion estimation. While highly effective, the recursive nature of the framework introduces computational overhead; for a 7-frame sequence, inference takes between 0.8s and 1.5s on an NVIDIA GTX 3090. Future work will focus on developing more efficient architectures to reduce inference time and testing the framework's limits on even longer sequences spanning dozens or hundreds of frames.
Summary updated