Counterfactual Instruction Sensitivity as a Shift-Robust Confidence Signal for Discrete-Action Robot Policies

Many robot policies now work like language models. A vision-language-action model (VLA), a large network that reads a camera image and a text instruction and outputs motor commands, cuts each continuous motion dimension into 256 ordered bins and predicts one bin per dimension as if it were a word. Because each prediction is a probability distribution over bins, it looks like a ready-made confidence score. Recent work shows the score can be fixed up within familiar tasks. It also shows that learned failure detectors stop working when the robot meets a new family of tasks. That case is exactly where a confidence score is needed, because the policy lacks the relevant knowledge: it recognizes the objects but does not understand the spatial relation it was asked to produce.

We propose a confidence signal that does not have to be learned for each task family. The idea is to ask whether the policy's action distribution would change if the instruction meant something different. We rewrite the instruction to a contrasting one, for example "left of the plate" becomes "right of the plate," or one goal is replaced by another goal available in the same scene. We then measure how far the predicted bin distributions move, using a distance that respects bin order. A policy that sends the arm to the same place regardless of the instruction is confident because it is ignoring the part it does not understand. That insensitivity should predict failure under task-family shift better than entropy, calibrated softmax probabilities, or probes trained on hidden states.

Core Claim

Under a task-family shift, ordinary overconfidence in discrete-action policies is mostly grounding overconfidence. The action distribution is sharp, but it depends on the image and the object nouns, not on the relational or goal part of the instruction. The low sensitivity that results can be measured with a second forward pass that uses a contrasting instruction. It needs no task-specific training, so a simple two- or three-parameter calibration map fitted on source task families should carry over to held-out families. Learned readouts do not carry over in this way.

Discretization matters twice in this claim. First, it makes the comparison cheap and exact. Each action dimension is a categorical distribution over ordered bins, so the distance between two predictions is a one-dimensional transport distance computed from cumulative sums. Second, it corrects a known flaw of categorical uncertainty. Entropy treats adjacent bins as just as different as bins at opposite ends of the range, so it cannot tell harmless fine-grained ambiguity from a genuine left-versus-right indecision. For bin distributions $p_d$ and $p'_d$ on action dimension $d$ under the original and contrast instructions, with cumulative distributions $F_d, F'_d$ and bin width $\Delta$, we define

$$ S_t = \frac{1}{|C|}\sum_{l' \in C}\;\sum_{d \in D} \Delta \sum_{k=1}^{K} \big|F_d(k) - F'_d(k)\big| , $$

where $C$ is the set of contrast instructions, $D$ is the set of translation dimensions, and $K=256$ is the number of bins. We pair this with an order-aware spread $U_t = \sum_{d \in D} \mathbb{E}_{p_d}|k - \mathrm{median}(p_d)|\,\Delta$, which captures execution-level uncertainty that sensitivity cannot see. The episode confidence is a logistic function of $S$ and $U$, aggregated over the first $T_0$ control steps. The rewritten instruction and first-step contrasts are defined by fixed templates. They are not learned.

Where This Sits in Prior Work

The motivating evidence comes from three papers. Sparse Progress and Risk Features in OpenVLA decodes progress from sparse-autoencoder features of OpenVLA. Its readout reaches 0.932 AUROC within task families but falls to 0.537 AUROC when trained on LIBERO-Goal and LIBERO-Object and tested on LIBERO-Spatial and LIBERO-Long. That result is the reason we avoid learned readouts over representations. Temporal Difference Calibration in Sequential Tasks learns a value head (TDQC) whose output is calibrated task-success probability. It reports that the head fails across embodiments without retraining, and its search procedure needs a one-step simulator. Nouns, Not Modifiers shows that OpenVLA parses object nouns but fails at spatial composition. That is the specific knowledge gap our contrast instructions are built to expose.

Two further papers supply methods we reuse or contrast against. Confidence Calibration in Vision-Language-Action Models averages mean token probability over paraphrases of the instruction and fits Platt scaling per action dimension, reducing ECE by over 20% on LIBERO. We keep its per-dimension Platt scaling as a baseline. Our perturbation runs the other way: paraphrases should leave the action unchanged, while contrasts should change it. Their method rewards agreement, and ours penalizes agreement where the model should disagree. The paper also finds that calibration is worst early in an episode. That is where we expect contrast sensitivity to help most, because early steps decide which object and region the arm heads toward.

SCALE and UAOR use uncertainty to steer decoding. SCALE compares the distribution to one-hot and uniform references. UAOR monitors entropy across layers. Both rely on order-blind uncertainty with hand-tuned thresholds, and neither checks against empirical success. We treat them as intervention consumers of a better signal, not as competitors on calibration.

Among non-robot work, Uncertainty Calibration for Tool-Using Language Agents shows that hidden-state MLP probes beat logits for multi-step agents (TabMWP ECE from 0.690 to 0.035). It evaluates in distribution, so it motivates a probe baseline but not a transfer claim. How (not) to ensemble LVLMs for VQA constrains our design: temperature scaling and verbal self-reflection did not produce reliable confidence on knowledge-heavy VQA, and majority voting across models fell below the best single model because the models shared errors. For that reason we do not use self-reports or voting across checkpoints.

Research Program

The base policy is OpenVLA, which uses 256 uniform bins per dimension on a Llama-2-based backbone. We use it as released and in LIBERO-fine-tuned checkpoints. We will first build contrast generators for the LIBERO suites. For Spatial, the generator swaps relational words and reference objects. For Goal, it substitutes other goals that are valid in the same scene. For Object, it substitutes other objects present in the scene. For Long, it perturbs the first subgoal.

Every contrast must be physically achievable in the current scene. This keeps sensitivity meaningful: a competent policy should move differently for each contrast. The forward pass for the contrast instruction is batched with the original, so the added wall-clock cost is one batch element. We will report control frequency against OpenVLA's roughly 6 Hz.

The analysis has three parts. First, we measure how $S$, $U$, entropy, and max-probability behave on successful and failed episodes under the family-held split. The split is taken from the SAE paper so that numbers are directly comparable. Second, we fit the calibration map on the Goal and Object families only and evaluate it on Spatial and Long. We report failure-prediction AUROC, ECE of episode success probability, and the risk-coverage curve when low-confidence episodes are halted. Third, we use the signal to gate SCALE-style resampling, with higher temperature only when confidence is low. This tests whether a calibrated gate beats SCALE's uncalibrated gate at equal compute. A further variant chooses, among sampled first actions, the candidate with the highest instruction sensitivity. This variant needs no simulator.

Baselines

The naive reference point is OpenVLA's raw mean token probability over the action dimensions at each step, averaged over the episode prefix. No recalibration is applied. Its failure-prediction AUROC and ECE under the family-held split set the floor.

The first strong baseline is the method of Confidence Calibration in Vision-Language-Action Models: paraphrase ensembles plus per-dimension Platt scaling. It is the most direct existing calibration method for binned VLAs and is cheap at test time. We give it the same number of extra forward passes as our method, fit its Platt parameters on Goal and Object only, and evaluate on Spatial and Long. Its reported in-distribution gain is an ECE reduction of over 20%. We found no reported result on Lacuna for its failure prediction under task-family shift, so we will measure it.

The second strong baseline is TDQC from Temporal Difference Calibration in Sequential Tasks. It is the most principled sequential calibrator, and its search raised LIBERO-10 success from 40% to 55%. We train it on source-family rollouts with the same rollout budget we use to fit our calibration map. We found no reported result on Lacuna for its calibration under task-family shift. The third learned baseline is a hidden-state probe in the style of PROBECAL, plus a linear readout on sparse-autoencoder features. These represent the white-box camp. The SAE progress readout fell to 0.537 AUROC on this split, though for progress and not for failure.

Beating these baselines means higher held-out-family failure AUROC and lower ECE than all three at matched compute. The margin must exceed seed-to-seed variation across three fine-tuning seeds and at least 50 rollouts per task.

Three ablations isolate the key ingredient. In the first, contrast instructions are replaced by paraphrases, keeping the order-aware distance. This separates "contrast" from "multiple prompts." In the second, the order-aware transport distance is replaced by KL divergence and the spread $U$ by entropy. This tests whether respecting bin order matters. In the third, $U$ is dropped, which shows how much of the signal is grounding and how much is execution. For the downstream gating experiment, the baseline is SCALE itself, which raised OpenVLA success on LIBERO-Long from 52.7% to 63.3%. We run it with the same temperature schedule and the same number of resampled steps, changing only the gate.

Why not just…?

Why not just use Platt or temperature scaling on the softmax? It is the cheapest option, and within a task family it may be good enough. Our hypothesis is that it mainly corrects the overall level of confidence. It cannot rank a sharp but instruction-blind prediction below a sharp and correct one, because both have the same probability mass. How (not) to ensemble LVLMs for VQA found that temperature scaling left confidence a poor predictor of correctness on knowledge-heavy tasks. The Platt baseline and the family-held split settle this directly for robots. If Platt scaling matches our AUROC on held-out families, the proposal's central claim fails.

Why not just train a better probe or value head on more diverse data? This is the real threat, because adding source diversity might repair transfer. The evidence against it is the collapse of learned readouts across families (Sparse Progress and Risk Features in OpenVLA) and across embodiments (Temporal Difference Calibration in Sequential Tasks). To test this fairly, we vary the number of source families from one to three and plot held-out AUROC for probes, TDQC, and our score. If the learned methods close the gap as diversity grows, our advantage is limited to data-poor settings, and we will say so.

Why not just average over paraphrased prompts? Paraphrase ensembles reduce noise from wording, but a policy that ignores a spatial modifier ignores it in every paraphrase. It then agrees with itself confidently. The paraphrase-for-contrast ablation measures exactly this difference.

Why not just use entropy, as UAOR and SCALE do? Entropy over 256 unordered classes counts spread across adjacent bins the same as a split between distant modes. Instruction insensitivity cannot show up in entropy at all. The KL-and-entropy ablation checks whether order-awareness adds anything once contrast is present.

Proposed Experiments and Decisive Checks

The main experiment uses LIBERO with the family-held split: fit on Goal and Object, test on Spatial and Long. Metrics are episode-failure AUROC, ECE, and risk-coverage under halting, reported per step so we can see how early the signal separates successes from failures. A secondary transfer test fits on LIBERO and evaluates on SIMPLER-WidowX, the embodiment shift where TDQC is reported to fail. We will also run a subset of BridgeData V2 tasks, where OpenVLA reports 70.6% success, to check the signal on real-data-trained checkpoints.

The decisive check has two parts. The first is whether $S$ alone, with calibration fitted only on source families, keeps held-out failure AUROC well above both the Platt baseline and the collapsed 0.537 regime seen for learned readouts. The second is whether its advantage is concentrated on relational tasks in LIBERO-Spatial, as the spatial-composition account predicts. If sensitivity helps equally on tasks with no relational content, the grounding explanation is wrong even if the score works. If it does not help on Spatial, the idea fails. For the downstream check, the calibrated gate should match SCALE's LIBERO-Long success at fewer resampled steps, or beat it at equal steps.