Correcting the Robot Where Layout Shift Hurts Most: Minimax Allocation of Expert Corrections under Unknown Displacement Costs

Robot policies trained by copying demonstrations often look competent until an object sits somewhere new. On LIBERO-Gen, a benchmark that moves objects to new tabletop locations, OpenVLA falls from 98% success to 0.0% (Dismantling the Illusion of Vision-Language-Action Models Competence via Explicit Distributional Shifts). The standard repair for a policy that drifts into unfamiliar states is interactive imitation learning: the learner acts, and an expert labels the states it actually reaches with corrective actions. That repair has to assume where the learner will be when it fails. DART (DART: Noise Injection for Robust Imitation Learning) perturbs the expert with noise fitted to the learner's training-time error, and DAgger-style methods label whatever the learner visits in the training layouts. Neither addresses a more basic uncertainty: which layout changes the robot will meet at deployment, and which ones are cheap enough to happen often.

We propose to borrow a tool from strategic classification, the study of classifiers facing people who change their features at a private cost. In that field, One-Shot Strategic Classification Under Unknown Costs proves that committing to one guessed cost of change can drive accuracy down to chance, and that robustness requires a min-max optimization over a set of plausible cost functions. Our claim is that the same structure fits spatial generalization. The deployment environment "moves" objects at an unknown cost, meaning some displacement directions and sizes are common and others rare, and we do not know which. Under a fixed budget of expert corrections, the robot should therefore collect those corrections against the worst plausible displacement cost rather than a single assumed one. We expect this to raise worst-case success across a set of layout shifts at equal expert effort, compared with uniform layout randomization, DART-style noise, and staged curricula.

Core Claim

Spatial layout shift can be modeled as displacement of task-relevant objects, $\Delta$, drawn according to an unknown cost function from an uncertainty set $\mathcal{C} = \{c_1, \dots, c_K\}$. Each $c_k(\Delta) = \|A_k \Delta\|$ is a weighted norm that makes some displacements (for example, along the table's long axis, or toward the robot) cheap and others expensive. Each $c_k$ with budget $B$ induces a distribution $P_k$ over layouts. The learner solves

$$\min_{\pi} \max_{w \in \Delta_K} \sum_{k=1}^{K} w_k \, \mathbb{E}_{\Delta \sim P_k}\big[ L(\pi, \Delta) \big],$$

where $\pi$ is the policy, $w$ is a mixture weight over cost hypotheses on the simplex $\Delta_K$, and $L(\pi,\Delta)$ is the policy's imitation loss on the states it visits in the layout displaced by $\Delta$. The contribution is the interactive version of this problem. The inner maximization does not reweight a fixed dataset. It decides where the next expert corrections are collected, and the expert labels the learner's own trajectories in those layouts. Expert labels are the scarce resource; simulated rollouts used to estimate failure are cheap.

This differs from the closest work in specific ways. DART fits one Gaussian to the learner's training error and never asks about deployment shift. Stair sampling (from the LIBERO-Gen paper) cycles training through a sliding window of environmental factors to stop skills binding to backgrounds, but it is a fixed schedule and does not respond to the learner. Student-forcing in Vision-and-Language Navigation exposes the agent to its own mistakes, but only in training buildings. The strategic-classification papers (Strategic Classification, its unknown-cost one-shot treatment, and the online Strategic Classification from Revealed Preferences) handle only static classifiers with no sequential control or perception. We take their min-max-over-costs structure and drop the dual-norm regularizer they derive. A smoothness penalty would be wrong here. "Action binding collapse" means the policy ignores object displacement, so the fix is to make the policy respond to displacement, not to make it less sensitive.

Research Program

The algorithm alternates two players, following the stochastic mirror descent-ascent scheme analyzed in the one-shot paper. The adversary keeps weights $w$ over the $K$ cost hypotheses. Each round it estimates the current policy's failure rate under each $P_k$ from unlabeled rollouts scored by the simulator's success predicate, then updates $w$ by exponentiated gradient. This parallels the "strategic poking" in Learning to Steer Markovian Agents under Model Uncertainty, which found that targeted probes identify hidden structure far faster than random ones (30 versus more than 300 steps). The learner then samples layouts from the weighted mixture, rolls out, has the expert label the visited states, aggregates the data, and fine-tunes.

In simulation the expert is a privileged motion planner with access to object poses. A labeled state counts against a fixed budget so that the comparison reflects costly human corrections. Expert resets to demonstration states, in the style of FILTER from Inverse Reinforcement Learning without Reinforcement Learning, are used only in an optional variant, because physical resets are often infeasible.

The analysis has three parts. First, we test whether the hardness phenomenon from the one-shot paper appears in embodied control: does correcting against a single wrong cost hypothesis collapse success under the true one? Second, we measure how the size of $\mathcal{C}$ trades nominal success against worst-case success. The one-shot paper flags oversized uncertainty sets as a source of heavy utility loss and leaves graceful degradation outside $\mathcal{C}$ unproven, so we test it directly. Third, an action-binding diagnostic checks whether the policy's commanded end-effector target moves with the displaced object. This separates "the policy sees the object but moves wrongly" from perception failure, the two failure modes the LIBERO-Gen paper distinguishes.

Baselines

The naive reference is behavior cloning on the 50 original LIBERO demonstrations per task (LIBERO), plus extra static expert demonstrations up to the same label budget at nominal layouts. This shows how much any shift-aware collection matters. The literature anchor is OpenVLA at 0.0% under spatial shift.

The strongest existing method is Pi0.5 trained with Stair sampling, which reports 64.0% success on LIBERO-Gen Spatial-CG, against 52.3% with random sampling (Dismantling the Illusion…). For a fair comparison we hold the backbone fixed: Pi0.5 if its weights and training recipe can be reproduced, otherwise a smaller flow-matching policy used for every method. Each baseline gets the same number of expert-labeled states, the same fine-tuning compute, and the same hyperparameter search. Stair sampling's windows are applied over the same displacement set $\mathcal{C}$ that our adversary uses. Beating it means higher worst-case success over the held-out displacement regions with no more than a few points' loss in average and nominal success, across at least three seeds with non-overlapping confidence intervals.

DART is run with noise covariance fitted to training error as published, which tests whether recovery data near nominal layouts already transfers. DAgger at nominal layouts tests whether interactivity alone suffices.

Three ablations isolate the key ingredient. The first keeps everything but replaces the adversary with uniform sampling over $\mathcal{C}$ (interactive domain randomization); this isolates the min-max weighting. The second fixes $w$ on one cost hypothesis, which isolates uncertainty-set robustness. The third uses the adversary to choose layouts for fresh expert demonstrations rather than labeling learner rollouts, which isolates the interactive component.

Why not just…?

Why not just randomize object positions uniformly during data collection? This is the most serious threat. If the label budget is large enough to cover $\mathcal{C}$ densely, uniform coverage should match min-max allocation, because the adversary's advantage comes only from concentrating labels where the policy is weakest. The uniform-over-$\mathcal{C}$ ablation tests exactly this, and we sweep the label budget. We predict a gap at small budgets that shrinks at large ones. If no gap appears at any budget, the min-max ingredient is not useful, and we would report that.

Why not just use Stair sampling or collect more static demonstrations? Stair sampling is already strong (64.0% Spatial-CG), but it never observes where the learner fails, and the LIBERO paper finds that static supervised pretraining can impede later transfer. Static demonstrations also never contain the off-path states the policy reaches after a misplaced grasp. The DART physical-grasping result illustrates the cost: behavior cloning reached 49% against DART's 79%. Our equal-budget comparison against Stair sampling settles whether the interactive, adaptive allocation adds anything beyond a good schedule.

Why not just run DAgger or DART in the training layouts? Both fix compounding drift near demonstrated layouts. However, the R2R result shows a large seen-to-unseen drop even with student-forcing (38.6% versus 21.8%), so on-policy correction alone does not close a layout gap. These baselines run at equal budget. If they match our method on held-out displacements, the cost-uncertainty framing adds nothing.

Proposed Experiments

The primary testbed is the LIBERO-Gen spatial-configuration axis with its 10 discrete locations. We build $\mathcal{C}$ from 6 to 8 anisotropic displacement norms over a subset of locations. Two locations, and one cost hypothesis outside $\mathcal{C}$, are held out to measure degradation beyond the set. Metrics are worst-case and average success over the $\mathcal{C}$-induced layouts, success at held-out locations, nominal success, expert labels consumed, and the action-binding diagnostic.

A secondary check on R2R uses a shortest-path expert, with $\mathcal{C}$ defined over start-pose displacements and path-length costs. Success within 3 m is measured on Val Unseen against student-forcing's 21.8%. The aim is only to show the method does not hurt in a domain where layout shift is coarser.

The decisive checks come from three comparisons. Min-max allocation must beat uniform allocation over $\mathcal{C}$ at small label budgets, or the core claim fails. Correcting against a single wrong cost must produce a measurable collapse under the true cost, or the borrowed hardness intuition does not apply to embodied control. Performance at held-out locations must degrade smoothly as $\mathcal{C}$ grows, rather than nominal success being traded away. The proposal is supported only if all three hold.