Robot arms that learn manipulation from camera images often fail when something irrelevant changes: a new tablecloth, a different light color, a mug placed in the background. Today the only reliable way to find these weaknesses is to run the policy under each kind of change and count failures. That is expensive in simulation and nearly impossible to do exhaustively on real hardware. We propose a cheaper test that uses only ordinary, unperturbed camera frames. It asks how strongly the policy's commanded motion responds to pixels that lie outside the objects being manipulated. If a policy's actions would shift when the tablecloth pixels shift, it is already depending on the tablecloth, whether or not anyone has changed the tablecloth yet.
The tool for this is gradient-based saliency. Saliency maps are heatmaps built by backpropagating a model's output to its input pixels, which shows how sensitive the output is to each pixel. They were developed for image classifiers, which produce a single score per class. Manipulation policies instead output a chunk of continuous future actions. We define a sensitivity measure for whole action chunks, measure how much of it falls on pixels outside the task-relevant objects, and test one main claim. The claim is that this off-task sensitivity, measured on clean training-distribution frames, ranks policies by how much success they will lose under background and lighting perturbations. We also test whether that ranking reflects a causal dependence on the background, or only a correlation.
Core Claim
Consider a policy $f$ that maps an image $I$ to an action chunk $f(I) \in \mathbb{R}^{H_a \times D_a}$, meaning $H_a$ future steps of $D_a$ action dimensions, whitened by the expert action standard deviation. Let $\bar M$ be a binary mask over pixels outside the manipulated object, the receiver object and the gripper. We define off-task action sensitivity as
$$E_{\text{out}}(I) = \mathbb{E}_{v \sim \mathcal{N}(0, \mathbf{I})} \left\| \bar M \odot \nabla_I \big(v^\top f(I)\big) \right\|_2^2 ,$$
where $v$ is a random direction in action-chunk space. In expectation this equals the squared Frobenius norm of the action Jacobian restricted to off-task pixels. It can therefore be estimated with a handful of backward passes, which answers the "no scalar logit" problem without averaging the trajectory into a single number.
The reasoning behind the measure is first-order. If a perturbation $\delta$ touches only off-task pixels, the resulting change in actions is approximately $J\delta$, which is controlled by this restricted Jacobian. We also compute a variant integrated along a path from a generic reference image, such as a blurred version or one with random texture outside the mask, in the style of Integrated or Expected Gradients. This variant addresses gradient saturation, the case where a strongly activated unit has near-zero local gradient. Because the measure is computed offline, the real-time latency objection to path methods does not apply.
The claim is falsifiable at the level of each perturbation axis. $E_{\text{out}}$ should predict drops on axes that change off-task pixels, such as table texture, background distractors and lighting. It should predict little about axes that change the objects themselves or the physics, such as object color, mass and friction. Those latter axes serve as built-in negative controls.
Relation to Prior Work
Several papers motivate the gap. THE COLOSSEUM stress-tests policies on 14 perturbation axes. It reports success drops of 30–50% under single perturbations and failure rates above 75% under combined ones, with a sim-to-real failure correlation of $R^2 = 0.614$. It detects failures only by running rollouts. Rethinking Latent Redundancy in Behavior Cloning shows qualitatively that vanilla behavior cloning attends to background clutter while its information-bottleneck variant does not. That work uses the heatmaps only as illustrations, never as a quantitative predictor.
Two papers supply our methods and also constrain the design. RobustVLA penalizes the whole-image gradient of log-action probability during online reinforcement-learning post-training. Its quantity is our measure without the mask, so we use the unmasked version as the ablation that isolates spatial selectivity. DIGR regularizes a student policy's input gradients only in regions that a teacher's perturbation maps mark as irrelevant. It does this in MiniGrid, Atari and CARLA, and it shows gradient maps cost about 0.002 s versus 0.97 s for perturbation maps. DIGR uses gradients as a training target for interpretability and adversarial robustness. It does not test whether gradient maps predict brittleness to natural scene changes, and it needs a converged teacher.
Grad-CAM and the Expected Gradients analysis in predictive imaging biomarker discovery supply the attribution machinery. The latter is the closest conceptual precedent. It shows that attribution can separate causal features from confounders in static images. We carry this question into closed-loop control, where the ground truth is a measured drop in success rate rather than an expert's judgment.
ADP ranks visual tokens by forward Layer-0 attention and reports that this ranking degrades on long-horizon tasks. This shapes our choice to measure action-conditioned gradients over time rather than static attention.
Research Program
The first component is a population of policies with naturally varying brittleness, trained on the Colosseum's RLBench tasks using its standard 100 demonstrations per task. The population spans several sources of variation. It includes a ResNet behavior-cloning policy, an Action Chunking Transformer (the architecture RoCoDA builds on), and the information-bottleneck variant. Each is trained with and without standard color and crop augmentation, at three demonstration counts and several seeds, with intermediate checkpoints kept. The result is roughly a hundred policies per task family. Because simulation provides segmentation masks for free, $\bar M$ is exact there.
The second component computes the sensitivity measures on held-out clean frames sampled along expert trajectories. Everything is reported per manipulation phase, using gripper state and end-effector speed to separate reaching from contact, because which pixels matter changes between phases. The third component runs every policy on every Colosseum perturbation axis to obtain the ground-truth success drops.
The fourth component is the causal test. If off-task sensitivity is merely correlated with brittleness, lowering it directly should not help. We therefore add $\lambda E_{\text{out}}$ as a training penalty in ordinary offline behavior cloning, with no reward and no teacher. This differs from RobustVLA, which requires online rewards, and from DIGR, which requires a teacher. We then check whether background-axis drops shrink while object and physics axes stay the same.
Finally, we test the measure where masks are unavailable, which is the regime that matters for real robots. There we replace exact masks with masks from an off-the-shelf open-vocabulary segmenter and measure how much predictive power survives.
Baselines
The naive reference point is clean success rate and clean held-out action-prediction loss. The hypothesis "policies that are better in-distribution are also more robust" needs no saliency at all. Our measure has to add rank correlation with perturbation drop beyond what these two give.
The ablation that isolates the key ingredient is the unmasked Jacobian energy, the quantity RobustVLA regularizes, computed with the same probes, frames and compute. If it predicts as well as $E_{\text{out}}$, the spatial mask contributes nothing. RobustVLA reports 82.5% average success on LIBERO under five visual perturbations, versus 47.9% for OpenVLA. No reported result was found on Lacuna for using Jacobian norms as a brittleness predictor.
The forward-attention competitor is the fraction of Layer-0 attention mass on off-task tokens, in the style of ADP, computed for the transformer policies. It is cheaper than gradients and is the natural "no backward pass" alternative. The faithful but slow competitor is occlusion saliency restricted to off-task regions. It is computed offline here, so its 0.97 s per frame cost (reported by DIGR in CARLA) is affordable, but it is still about two orders of magnitude more expensive.
The strongest competitor is offline perturbed-frame deviation. Here we render clean frames with generic off-task perturbations, such as random textures and color shifts outside the mask, and measure the action change. It is given the same rendering budget as our path-integrated variant. We also report an oracle version that uses the exact Colosseum perturbation for each axis, as a ceiling. No reported result was found on Lacuna for any method predicting Colosseum drops from clean data.
Beating these baselines means a higher Spearman correlation between predicted brittleness and measured drop across the policy population. This is evaluated on held-out tasks, with bootstrap confidence intervals that exclude the naive and unmasked baselines. Ideally the measure also matches the generic perturbed-deviation baseline while needing no renderer.
For the intervention, the strongest existing method is causality-aware counterfactual augmentation. RoCoDA reports 99.3% versus 22.3% for vanilla behavior cloning on LIBERO-Object with unseen distractors. We compare against it, and against simple mask-based background randomization, on that same split.
Why not just…?
Why not just augment heavily and run the benchmark? Augmentation is a training fix, not a diagnostic, and it does not remove the need for perturbation rollouts. The Colosseum reports 30–50% drops for policies trained with standard regimens, so augmented policies remain brittle in ways that must still be measured. Augmented policies sit inside our population. If augmentation alone explained the variation in brittleness, a single augmentation indicator would predict as well as $E_{\text{out}}$. We report that comparison directly.
Why not just measure action change on rendered perturbed frames? This is the real threat. Where a renderer for the relevant perturbation exists, the oracle version will probably beat any gradient measure, which is why we report it as a ceiling rather than argue around it. The gradient measure is meant for shifts nobody has enumerated and for real robots without a renderer. The decisive comparison is therefore against the generic-perturbation version under equal compute, plus the segmenter-mask experiment. If the generic rendered baseline matches us everywhere, we will report that gradients add only speed.
Why not just use the global Jacobian norm, or a global Lipschitz bound? A global quantity cannot tell sensitivity that is necessary, such as attention to an object's edge during a grasp, from sensitivity that is spurious, such as attention to the tablecloth. LipsNet itself notes that its global bound trades robustness against fine control. The unmasked-Jacobian ablation and the object/physics negative-control axes test this directly. A global measure should track drops on all axes indiscriminately, while $E_{\text{out}}$ should track only the background axes.
Why not just randomize the background using the masks during training? For the intervention part, this may work as well as the gradient penalty, and we treat that as a live outcome. The penalty differs because it constrains first-order sensitivity over all off-task pixel changes, not only the sampled textures. Its advantage should therefore appear on perturbation types absent from the randomization set, such as lighting color or distractor objects when training randomizes only textures. If it does not, the intervention result reduces to "masks plus augmentation suffice." The diagnostic claim would stand on its own.
Decisive Checks
Three outcomes would settle the proposal. The first is that $E_{\text{out}}$ beats clean loss and the unmasked Jacobian in rank correlation with background-axis drops on held-out Colosseum tasks, while showing near-zero correlation on the mass and friction axes. Failing either part falsifies the claim. The second is that penalizing $E_{\text{out}}$ in offline behavior cloning reduces background-axis drops more than mask-based randomization does on held-out perturbation types, without lowering clean success or contact-phase precision. The third is that predictive power computed with segmenter masks stays within a stated margin of the exact-mask result. That would make the measure usable as a pre-deployment audit on real hardware, where the Colosseum's 3D-printed Franka replica offers a first check.
References
- THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation
- Rethinking Latent Redundancy in Behavior Cloning: An Information Bottleneck Approach for Robot Manipulation
- RobustVLA: Robustness-Aware Reinforcement Post-Training for Vision-Language-Action Models
- Policy Distillation with Selective Input Gradient Regularization for Efficient Interpretability
- Grad-CAM: Why did you say that?
- Enhancing Predictive Imaging Biomarker Discovery Through Treatment Effect Analysis
- Action-aware Dynamic Pruning for Efficient Vision-Language-Action Manipulation
- RoCoDA: Counterfactual Data Augmentation for Data-Efficient Robot Learning from Demonstrations
- Robust Behavior Cloning Via Global Lipschitz Regularization