Robot manipulation policies built on vision-language models see the scene through a low-resolution bottleneck. Their image encoders shrink every view to a small fixed grid, so the millimetre-scale detail that decides whether a peg enters a hole or a gripper closes on a thin handle is often gone before the policy reasons about it. Recent systems fix this by letting the model "zoom": they pick a region and re-render it at higher effective resolution. The choice of where and how to look, however, is made separately from the choice of how to move. This proposal couples the two. The policy first drafts a coarse motor action. That draft sets the pose of a virtual zoomed camera, placed at the intended contact point and pointed along the intended approach direction. The policy then revises the action from the zoomed view. The draft action primes perception, and the sharper perception rewrites the action.
The idea comes from common coding in cognitive science, the claim that perception and action share one representational format so that a planned movement biases what is seen. Walking observers, for example, are more likely to see an ambiguous figure as walking toward them (A 2012 study on action orientation). A related line of work on the efference copy, the internal copy of a motor command used to predict its sensory consequences, has already improved self-supervised image representations (Self-Supervised Learning Through Efference Copies). We turn the efference copy into a camera command. The practical bet is simple: the pixels that matter most for the next action are the ones around where that action will land, viewed from the direction it will arrive.
Core Claim
In a coarse-to-fine manipulation policy, a zoomed view whose pose is computed from the policy's own draft action (target position plus approach orientation) is worth more than a zoomed view chosen by task-relevance heatmaps and generic geometric heuristics. The gain should show up in two places. First, success should rise on precision-limited tasks. Second, the latency should stay close to that of a single extra forward pass, because the loop needs no candidate-view search. We also claim that the benefit depends on two things the draft supplies and heuristics do not: the camera orientation taken from the approach axis, and a revision head trained to correct perturbed drafts. It does not depend only on centering the zoom well.
Positioning Against Prior Work
The closest work is ActiveVLA. It predicts 2D heatmaps of "crucial areas" from three orthographic renders of a point cloud through PaliGemma. It then scores candidate virtual cameras around that area by visibility, distance and angular diversity, re-renders with a narrowed field of view, and decodes a 6-DoF end-effector pose. ActiveVLA motivates this proposal: its own ablations show that viewpoint selection and zoom each matter on RLBench (91.8% overall, 89.4% without active viewpoint selection, 87.6% without zoom). It also constrains it, because its viewpoint choice never consults the action being planned and it costs 0.53 s per step against 0.26 s for a fixed view. We keep its rendering machinery and replace the heuristic viewpoint scorer with a camera pose computed directly from the draft action. We also add an explicit draft-conditioned revision step.
We should be honest that ActiveVLA's crucial-area centre and our draft translation target may often coincide. The new ingredients are therefore camera orientation and revision, and the ablations below are built to isolate exactly those.
Three lines of work on the resolution bottleneck supply the zoom-as-action idea but act only on static 2D images. ACTIVE-o3 trains a sensing module with group relative policy optimization to propose up to three crops for a downstream task head. AdaptVision fetches high-resolution crops only when needed, keeping 97.9% of full-resolution accuracy with 33% of the visual tokens. FOVEA casts crop selection as sequential experimental design and admits a "cold start" failure when the first proposal misses. In all three, sensing is an upstream filter with no motor plan feeding back into it. We borrow their lesson that selective zoom beats uniform high resolution, and we inherit FOVEA's cold-start risk, which we treat directly.
On the common-coding side, SUGARL trains separate motor and camera policies, rewarding the camera policy when an inverse-dynamics model can recover the motor action from consecutive views. It does this on 50×50 foveal Atari and Robosuite, and it assumes the two kinds of action are independent. AdaptiveNN learns sequential 2D fixations with adaptive stopping, including on CALVIN, but its fixations are not driven by the manipulation plan. The active efficient coding models, in which vergence control and sensory tuning co-develop from scratch (Active Binocular Vision), suggest that perception and action should shape each other. BridgeVLA warns against doing this by altering the pretrained backbone: adding 3D convolutions dropped RLBench success from 88.2% to 56.2%. Our design respects that warning. The coupling acts only through which image is rendered, and every input stays an ordinary 2D render that the frozen-distribution backbone already understands.
Research Program
The base policy follows the BridgeVLA/ActiveVLA recipe. A point cloud is rendered to three orthographic views and passed through a PaliGemma backbone. The backbone produces a translation heatmap per view, plus rotation and gripper outputs.
The first pass runs on coarse views and yields a draft action consisting of a target point $p$ and an approach rotation $R$. From that draft we place a virtual camera on the approach axis at a fixed standoff behind $p$, looking along the gripper's approach direction, with a narrowed field of view. We add one or two orthogonal side views, which matter for tasks like insertion where depth along the axis is critical. This pose is a deterministic function of the draft, so selecting it costs nothing beyond rendering.
The second pass sees the zoomed renders together with a small token encoding of the draft. It outputs the revised action as a residual on the draft.
To make revision meaningful, we train on drafts perturbed with noise matched to the first pass's empirical error distribution, which we collect from held-out rollouts. The revision head therefore learns to correct realistic mistakes rather than copy the draft. An optional efference-copy loss asks a small head to predict the draft's residual error from the pair of coarse and zoomed features. This mirrors the inverse-model loss in S-TEC, but here the "action" is a real motor draft rather than a data augmentation.
The draft can be wrong enough that the zoom misses the relevant region, which is FOVEA's cold-start problem in motor form. We handle this by rendering zooms for the top-$k$ heatmap modes when the first-pass heatmap is multimodal, letting the revision pass score them. We then iterate draft→zoom→revise only while the revision moves the target by more than a threshold, a convergence-based stop in the spirit of AdaptiveNN's termination. We will report the distribution of loop counts and the resulting wall-clock time.
A physical extension tests the eye-in-hand coupling that SUGARL leaves open. On the real Franka arm, the wrist camera is driven partway along the draft approach before the final revision, so the motor draft literally moves the sensor. The analysis will ask whether this physical version matches the virtual one.
Baselines
The naive reference is the same backbone with the fixed three-view renders and no zoom. This is essentially BridgeVLA, which reports 88.2% average success on RLBench and 64.0% on THE COLOSSEUM. A second reference doubles the render resolution of the fixed views at matched token cost. It tells us whether resolution alone explains any gain.
The strongest existing method is ActiveVLA, which reports 91.8% average success on RLBench, 65.9% on COLOSSEUM, and 91.8% on its 13-task real-world Franka suite. We will rerun it from its released configuration on the same demonstrations, seeds and training budget as our model, so the comparison does not rely only on reported numbers. For GemBench we found no reported result on Lacuna for ActiveVLA, so both methods will be run fresh there.
Beating ActiveVLA means one of two outcomes. Either our model achieves higher mean success across seeds, with non-overlapping confidence intervals on the precision-heavy task subset (insertion, peg, screw and stacking tasks), while overall RLBench success is no lower than 91.8%. Or our model matches ActiveVLA's success at clearly lower latency than its 0.53 s per step.
Three ablations isolate the key ingredient, and all share one training budget. The first centres the zoom on the draft target but takes camera orientation from ActiveVLA's visibility/distance/diversity scorer. This separates the value of the approach-aligned orientation. The second keeps our camera pose but removes draft conditioning and perturbation training from the second pass, so it re-predicts from scratch. This separates the value of revision. The third centres the zoom on ActiveVLA's crucial-area heatmap instead of the draft. This tests whether "where I will act" differs usefully from "what is task-relevant."
Why not just…?
Why not just use the wrist camera, which already looks where the gripper is? This is the most serious threat, because it is common coding that comes free with the hardware. The wrist camera shows where the gripper is now, not where the next waypoint will put it. In keyframe control that target is often far away and seen obliquely. We will test a variant that zooms the existing wrist and shoulder renders around the draft target without re-posing the camera. If it matches our model, the contribution shrinks to "centre on the draft," and we will report that plainly.
Why not just raise the resolution of the fixed views or add more cameras? On RLBench, static multi-view BridgeVLA (88.2%) and RVT-2 (81.4%) trail ActiveVLA's 91.8%. COLOSSEUM reports 30–50% drops for leading models under camera and distractor perturbations. Both results suggest that fixed views overfit to geometry rather than lacking pixels. Our matched-token high-resolution baseline checks this claim in our own setting.
Why not just keep ActiveVLA's heuristic viewpoint scorer and tune it? It may already pick near-optimal centres. What it cannot do is condition on the planned rotation, and it pays for candidate search in latency. The orientation-swap ablation settles whether orientation from the draft matters. If that ablation equals the full model, the heuristic scorer is sufficient, and our remaining claim is the latency saving.
Why not just learn the camera policy with reinforcement learning, as SUGARL does? SUGARL shows that an inverse-dynamics reward is needed even in small environments: a single policy without it scored 8.42 against 108.6 on NutAssemblySquare. Learning camera placement over 6-DoF poses on top of a large VLA is far costlier than computing the pose from the draft. We treat a learned pose offset on top of the draft pose as a later extension, not a starting point.
Proposed Experiments
The main study trains all variants on RLBench and evaluates them on RLBench, COLOSSEUM and GemBench. We use at least three seeds and report success rate, per-step latency, and loop count.
The decisive check is a stratified analysis. If the claim is right, our gains over ActiveVLA should concentrate on episodes where two things are true: the draft approach axis differs substantially from the axis ActiveVLA's scorer chooses, and success depends on rotation accuracy. Gains should be near zero on coarse pick-and-place tasks. A uniform small gain would instead point to generic extra compute.
A second check plots final success against first-pass draft error. This measures how far revision can pull back a bad draft, and whether top-$k$ multi-mode zooms remove the cold-start collapse.
On COLOSSEUM we specifically examine the camera-pose and distractor perturbation axes. Because our zoom pose is tied to the action rather than to the scene cameras, we predict smaller drops there than for ActiveVLA.
Finally, the real Franka suite compares virtual draft-zoom with the physical wrist-driven version on the occlusion-heavy tasks from ActiveVLA's evaluation. This tells us whether the coupling survives when the motor draft physically moves the sensor.
References
- Are You Approaching Me? Motor Execution Influences Perceived Action Orientation
- Self-Supervised Learning Through Efference Copies
- ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulation
- ACTIVE-o3 : Empowering MLLMs with Active Perception via Pure Reinforcement Learning
- AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisition
- The Perceptual Bandwidth Bottleneck in Vision-Language Models: Active Visual Reasoning via Sequential Experimental Design
- Active Vision Reinforcement Learning under Limited Visual Observability
- Emulating human-like adaptive vision for efficient and flexible machine visual perception
- The development of active binocular vision under normal and alternate rearing conditions
- BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Models
- RLBench: The Robot Learning Benchmark & Learning Environment.
- THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation