Introduction
Optical flow estimation—the task of calculating pixel-wise motion between consecutive video frames—requires a delicate balance between two competing objectives: discrimination and smoothness. A model must be discriminative enough to distinguish a pixel from its neighbors for accurate matching, yet smooth enough to ensure that neighboring pixels belonging to the same object move consistently.
While state-of-the-art architectures like RAFT and FlowFormer have advanced the field using Transformers and 4D correlation volumes, they often lack a strong local inductive bias. Global attention mechanisms can introduce noise from unrelated image regions, while rigid local windows fail to capture complex deformations. GAFlow addresses this "neighborhood priority" gap by incorporating learnable Gaussian Attention into both feature encoding and motion refinement.
Proposed Architecture
The Gaussian Attention Flow (GAFlow) network introduces two primary modules designed to be "plug-and-play" within existing flow architectures.
1. Gaussian-Constrained Layer (GCL)
Integrated into the feature encoder, the GCL modifies standard Transformer blocks by applying a learnable Gaussian kernel as a constraint mask on the attention matrix. This forces the model to prioritize fine-grained structural details in a pixel's immediate neighborhood, resulting in more distinct features that are easier to match across frames.
2. Gaussian-Guided Attention Module (GGAM)
Used during the motion estimation (decoding) phase, the GGAM provides a learnable smoothing mechanism. Its most advanced iteration, the Gaussian-Guided Attention with Deformation (GGAD), learns dynamic offsets to warp the Gaussian kernel and amplitudes to scale it. This allows the "attention bubble" to adapt its shape and size to match deforming objects or large displacements, ensuring smoothing does not cross object boundaries.
Local Inductive Bias and Feature Peakiness
A key insight of GAFlow is that re-introducing locality as a prior prevents the model from being distracted by global similarities that are irrelevant to motion. This results in significantly more "peaky" feature representations.
As shown in the visualization above, standard global attention models often produce blurry affinity maps with multiple potential matches. In contrast, GAFlow’s Gaussian-constrained approach produces sharp, localized peaks, leading to more accurate and robust pixel matching.
Experimental Results
GAFlow was evaluated on the Sintel and KITTI-2015 benchmarks, setting new performance records while maintaining high computational efficiency.
- Sintel Benchmark: GAFlow-FF (based on FlowFormer) achieved an End-Point Error (EPE) of 1.15 on Clean and 2.05 on Final, outperforming previous state-of-the-art models.
- KITTI-2015: The model achieved a top-tier 4.42% F1-all score on the online leaderboard.
- Efficiency: GAFlow is significantly faster and lighter than its predecessors. On KITTI frames, GAFlow inference takes 0.36s (vs. FlowFormer’s 1.77s) and uses approximately 40% fewer parameters (11.1M vs. 18.2M).
Conclusion
GAFlow demonstrates that incorporating learnable local priors via Gaussian Attention can significantly enhance the performance and efficiency of optical flow models. By making feature matching more discriminative and motion smoothing more adaptive, GAFlow provides a robust framework for dense correspondence tasks. Future work may extend this approach to related domains such as stereo matching and depth estimation.
Summary updated