Introduction
Optical flow estimation—the process of determining pixel-level motion between consecutive frames—is foundational for autonomous navigation and robotics. However, recent studies have exposed a critical vulnerability: small adversarial patches can induce catastrophic, global errors in flow predictions. This paper investigates the root cause of this fragility, identifying the "aperture problem" and limited receptive fields as the primary culprits.
The Aperture Problem and Architectural Vulnerability
The authors hypothesize that adversarial patches exploit the classic "aperture problem" in computer vision. When a motion-estimating network views a scene through a small "aperture" (its receptive field), self-similar or ambiguous patterns can lead to incorrect motion vectors. If the network's receptive field is smaller than the adversarial patch, it lacks the context needed to distinguish the patch's artificial motion from the true scene motion.
This vulnerability is most pronounced at the correlation layer. If the encoder fails to provide sufficient spatial context, the correlation layer generates ambiguous signals that the decoder subsequently amplifies, resulting in the widespread flow distortions seen in the heatmaps above.
Methodology: Building Robustness
To validate their theory, the researchers developed Robust FlowNetC. While the original FlowNetC utilizes a shallow encoder with large 5x5 kernels, the robust variant employs a deeper encoder with multiple 3x3 kernels. This architectural shift significantly expands the network's receptive field before the correlation layer, allowing it to "see" the stable environment surrounding an adversarial patch.
The team also utilized t-SNE visualizations and Maximum Mean Discrepancy (MMD) to track how feature distributions change under attack, confirming that increased context prevents the "poisoning" of features entering the correlation bottleneck.
Key Results
The results demonstrate that architectural context is a powerful defense. Under a standard adversarial patch attack, the original FlowNetC’s End-Point Error (EPE) increased from 11.50 to 52.66. In contrast, Robust FlowNetC maintained an EPE of 9.60, effectively ignoring the patch's influence on the rest of the scene.
Further experiments confirmed this finding: when the robust RAFT architecture was modified to use the shallow FlowNetC encoder, it immediately became vulnerable to the same patch attacks. This confirms that the receptive field, rather than the specific decoder type, is the primary line of defense.
Conclusion
This research shifts the focus from general "encoder-decoder" weaknesses to specific architectural bottlenecks. By addressing the aperture problem through expanded receptive fields, optical flow networks can be made significantly more resilient to physical-world adversarial attacks, a crucial step for the safety of autonomous systems. Future work will explore the remaining challenges of rotational errors and the theoretical limits of universal adversarial perturbations in motion estimation.
Summary updated