FAVR: Flow-Aware Visual Residual Alignment for Vision-Language-Action Policies

Qixuan Wu, Xiaole Yang, Yuguang Yang, Canyu Chen, Linlin Yang, Haixia Pan, Guodong Guo, Baochang ZhangarXiv 2026 · arXiv

TL;DR

FAVR (Flow-Aware Visual Residual Alignment) is a lightweight framework that integrates pretrained video representations into frozen Vision-Language-Action (VLA) policies. By adapting visual conditioning at every step of the action generation process—a technique called "flow-aware gating"—FAVR significantly boosts robotic manipulation performance with only ~2 million trainable parameters.

What Problem Does This Paper Address?

Modern robotic policies, like OpenVLA or $\pi_0$, often use "flow-matching" or diffusion-based methods to generate actions. These models produce continuous action "chunks" by iteratively refining a noisy initial state over several "flow steps." While these models are powerful, they often struggle to incorporate rich temporal information from video, which is crucial for understanding scene dynamics and object interactions.

The challenge is two-fold:

  1. Integration: How do you add video features to a massive, frozen VLA model without breaking it or requiring expensive full-model fine-tuning?
  2. Dynamic Alignment: Even if the visual scene is static during a single action chunk, the model's internal representation of the action changes at every flow step. A visual feature that helps at the start of the generation process might be less relevant (or even harmful) by the end.

Methods

FAVR introduces a lightweight "visual interface" that sits alongside a frozen VLA policy. Its architecture consists of three main components:

  1. Temporal Visual State: It uses a frozen V-JEPA 2.1 video encoder to extract features. These are compressed into a compact state using linear Test-Time Training (TTT). This allows the model to maintain a "memory" of recent history that is updated as the robot moves.
  2. Action-Conditioned Retrieval: At each flow step $\tau$, the model queries the TTT state using the current features from the VLA's action expert. This ensures the retrieved visual information is relevant to the specific action being generated.
  3. Flow-Aware Gating: This is the "secret sauce." FAVR predicts a velocity residual—essentially, what the base policy is missing to reach the target action. It then solves an optimization problem to find a channel-wise gate $g_\tau$ that aligns the injected visual features with this residual.
Figure 2. Overview of FAVR. Observed video features update a linear TTT state queried by action features at each flow step. A predicted velocity residual guides FISTA-based joint or approximate closed-form gating before the frozen affine action head. Executed actions provide subsequent observations for state updates.

The authors exploit the affine (linear) structure of the frozen VLA action head. Because the final layer is linear, they can mathematically characterize exactly how changing a visual feature channel will affect the final predicted velocity. They offer two ways to solve for this gate: a joint optimizer (FISTA) and a super-fast closed-form approximation (FAVR-CF).

Datasets

The authors evaluated FAVR across a diverse set of benchmarks:

  • LIBERO: A standard suite for language-conditioned manipulation (Spatial, Object, Goal, and Long tasks).
  • LIBERO-Plus: A harder version of LIBERO designed to test robustness against perturbations like lighting changes, camera shifts, and sensor noise.
  • RoboTwin 2.0: A benchmark featuring 50 bimanual (two-armed) manipulation tasks, tested in both "clean" and "randomized" (C2R) environments.
  • Real-World (Cobot Magic): Experiments on a physical dual-arm platform for tasks like block pick-and-place and paper-cup grasping.

Key Insights and Ideas

  • Flow-Awareness is Crucial: The most significant finding is that the "best" visual conditioning changes throughout the action generation process. As shown in Figure 1, using a different gate at each flow step ($g_1$ through $g_5$) performs much better than using one "shared" gate for the whole chunk. Comparison of shared vs. flow-aware gating.

  • Action-Space Alignment: Instead of just "dumping" visual features into the model and hoping for the best, FAVR explicitly aligns them in the velocity space. By targeting the residual (the error of the base policy), the visual features act as a corrective signal.

  • Efficiency via Linear Algebra: By injecting features right before the final linear layer, the complex optimization of a neural network is reduced to a much simpler box-constrained optimization problem that can be solved very efficiently.
  • Video > Images: The experiments confirmed that features from video-pretrained models (V-JEPA) provide better temporal context for manipulation than image-pretrained models (I-JEPA), particularly for long-horizon tasks.
Figure 1. Shared versus flow-aware gating under the same video context. The heatmaps show recorded channel weights at repre- sentative flow steps. On five RoboTwin 2.0 tasks, average success increases from 72.0% with a shared gate to 82.0%.

Key Results

  • RoboTwin 2.0: FAVR improved the LingBot-VLA baseline from 87.12% to 90.04% in clean settings and from 86.88% to 89.38% under randomized conditions.
  • LIBERO: On the standard benchmark, success rose from 97.4% to 98.6%. On the more challenging LIBERO-Plus, it improved from 84.22% to 85.05%.
  • Real-World: In physical robot tests, FAVR outperformed continued fine-tuning. For example, in block pick-and-place, it achieved 15/20 successes compared to 12/20 for standard fine-tuning.
  • Low Overhead: The FAVR-CF (closed-form) variant adds only 1.5% inference latency over the frozen baseline while retaining most of the performance gains.

Limitations and Future Work

  • Base Policy Dependency: FAVR is an adapter; it improves an existing policy but cannot teach a robot basic skills from scratch if the base policy is completely incapable.
  • Memory Depth: While TTT provides a good temporal state, the authors noted that for tasks requiring very specific long-term memory (like remembering which cup hides a block), the improvement was modest, suggesting room for more specialized memory modules.
  • Generalization: Future work could explore extending this velocity-residual alignment to non-linear action heads or more complex action mappings beyond simple flow-matching.

Summary updated

Abstract

Pretrained video representations provide rich temporal information for robotic manipulation, yet effectively integrating them into existing vision-language-action (VLA) policies remains challenging. We present Flow-Aware Visual Residual Alignment (FAVR), a lightweight framework for flow-aware visual conditioning of frozen VLA policies. FAVR maintains a compact temporal visual state through linear Test-Time Training (TTT) and retrieves action-conditioned visual features throughout flow-based action generation. By exploiting the affine structure of the frozen action head, we explicitly characterize how injected visual features influence predicted action velocities and formulate flow-aware channel selection as a box-constrained optimization problem. We develop a FISTA-based solver for joint gate optimization and derive an efficient closed-form approximation through diagonalization and rescaling. With only 1.74-2.19M trainable parameters and 5,000 adaptation steps, FAVR improves pretrained VLA policies across benchmarks. On RoboTwin 2.0, FAVR improves LingBot-VLA from 87.12% to 90.04% under clean conditions and from 86.88% to 89.38% under randomized conditions. A controlled ablation on five RoboTwin tasks further shows that flow-aware gating increases average success from 72.0% with a gate shared across flow steps to 82.0%, demonstrating the benefit of adapting visual conditioning throughout action generation.
BibTeX
@misc{wu2026favr,
  author = {Qixuan Wu and Xiaole Yang and Yuguang Yang and Canyu Chen and Linlin Yang and Haixia Pan and Guodong Guo and Baochang Zhang},
  title = {{FAVR: Flow-Aware Visual Residual Alignment for Vision-Language-Action Policies}},
  year = {2026},
  eprint = {2611.20971},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20971},
}
Figure