Overlap as Supervision: EMA-Teacher Test-Time Adaptation for Action-Chunked Robot Policies

Robot policies that predict a short sequence of future actions at once, called action chunking, now underlie most strong imitation-learning systems. These policies have a property no image classifier has. Every time they replan, the new chunk covers several future timesteps that the previous chunk already predicted. When the robot's camera view drifts away from the training distribution (new lighting, a new table texture, a distractor object), those overlapping predictions start to disagree, and the robot stutters, flips between strategies, or stalls. Today's systems treat that disagreement only as something to hide at execution time: they average it away or pick the most coherent candidate. We propose to treat it as a free, label-free training signal that can be used while the robot is running.

The idea is to borrow the stabilizer that test-time adaptation uses for image models: an exponential-moving-average (EMA) teacher, a slowly updated copy of the network whose weights are a running average of the learner's weights. The teacher's earlier chunk becomes the target for the learner's current chunk on the overlapping steps. The learner updates its weights online. The teacher's slow averaging and occasional resets of weights to their pretrained values keep self-training from collapsing. In short, overlapping chunks tell the robot when its perception has drifted, and weight averaging makes it safe to correct that drift by learning, not merely by smoothing the output.

Core Claim

Under gradual or cycling visual distribution shift, a chunked visuomotor policy adapted online with overlap-consistency targets from an EMA teacher will recover task success that output-space methods cannot recover, at a fraction of their per-step sampling cost. Output-space methods here means temporal ensembling and guided chunk selection. They make actions smoother, but they cannot repair features that the visual encoder no longer extracts correctly. The two ingredients are each necessary. Consistency targets without weight averaging should drift into confirmation bias. Weight averaging without the temporal-overlap signal has no policy-appropriate target to average toward.

This is clearly different from the closest work. Bidirectional Decoding samples many candidate chunks at test time and selects the one most coherent with the previous chunk, contrasted against a weak policy. It combines this with EMA smoothing of actions, but its weights stay frozen and its "EMA" lives in action space. ACT introduced temporal ensembling, an exponentially weighted average of overlapping chunk outputs, again with no learning. On the other side, CoTTA and SwapPrompt use EMA teachers to adapt image classifiers and vision-language prompts online. Their targets come from augmented views of a single input, and they have never been posed for continuous, temporally extended action outputs. Our contribution is a temporal pseudo-label that exists only because the policy chunks, used inside the CoTTA-style stabilization recipe.

Research Program

We start from a pretrained chunked policy. We use Diffusion Policy, a generator that produces an action chunk by iteratively denoising random noise, and ACT. Both are available in LeRobot. During deployment the policy predicts $k$ actions and executes $m$ of them before replanning, so consecutive chunks overlap on $k-m$ steps. At replanning time $t+m$, the student network $\theta_S$ sees the new observation. The teacher $\theta_T$'s chunk from time $t$ supplies targets for the overlapping steps. For a diffusion or flow-matching policy, the student is trained with its ordinary denoising loss, conditioned on the current observation, using the teacher's overlapping actions as the clean sample. For ACT the loss is a direct regression. After each student step the teacher updates as $\theta_T \leftarrow \alpha\,\theta_T + (1-\alpha)\,\theta_S$, where $\alpha$ is the momentum. Following CoTTA, a small fraction of student weights is reset to the pretrained source weights at each step.

A naive overlap loss has two failure modes, and we design against both. The first is legitimate disagreement. At decision points the demonstrations are multimodal, and after a real external disturbance the earlier chunk is simply wrong. Forcing agreement in either case teaches the wrong thing. Borrowing the gating idea from AdaSemiCD and the multi-sample agreement measure from UA2C, we draw a few teacher samples and use an overlap step as a target only when those samples agree. We also skip updates when proprioceptive tracking error indicates the world has diverged from what the teacher expected.

The second failure mode is trivial consistency, where the policy ignores its observations and outputs the same smooth motion every time. Source-weight restoration and the slow teacher are the main defense. We will measure observation sensitivity directly to check that it holds.

The analysis question running through the work is how the teacher momentum $\alpha$ should scale with chunk geometry. Each replanning step yields $k-m$ targets, and the teacher's half-life should plausibly cover a few replanning cycles. We will map success against $\alpha$ for several $(k, m)$ pairs to test whether a single rule, half-life measured in replans rather than in gradient steps, transfers across tasks.

Baselines

The naive reference point is the frozen pretrained policy executed with standard receding-horizon control: predict 16 actions, execute 8, as in Diffusion Policy. It is run on exactly the same shifted episode streams. Any adaptation method must beat it on mean success over the shift sequence to matter at all.

The strongest existing competitor is Bidirectional Decoding combined with action-space EMA smoothing. It is the only method in the file aimed at the same symptom, overlapping-chunk incoherence at test time, and it already reports an average relative improvement of 46% over vanilla baselines across Push-T, RoboMimic, and Franka Kitchen, versus 32% for decoding alone (Bidirectional Decoding). We will run it with its published candidate count and weak-policy construction, using the same base checkpoint. We will report its wall-clock cost per replan beside ours. Beating it means higher success averaged over shift cycles at equal or lower per-step compute. If we only match it in success while using fewer samples, we report that as a compute result, not a success result. We note that the file reports no number for any method under the injected visual shifts we propose. The 46% figure was measured in-distribution, so the decisive comparison is our own head-to-head.

ACT-style temporal ensembling of outputs is run on ACT checkpoints as the cheapest output-space competitor (ACT). We adapt CoTTA directly, with an EMA teacher, targets averaged over augmented views of the current observation only, and stochastic restoration. Running it with the same update budget isolates whether the temporal-overlap target, rather than the EMA machinery, carries the gain.

Three ablations each remove one ingredient. In the first, the student targets its own previous chunk, so there is no weight averaging. The second removes the agreement gate. The third adapts only normalization layers with the overlap loss. All methods get the same hyperparameter search budget on a held-out shift type that is never used for reporting.

Why not just…?

Why not just use temporal ensembling or guided chunk selection, which need no training? This is the main threat on mild shifts, and we expect output-space methods to match us there. The case against them on severe shifts is that they recombine what the frozen encoder already produces. If the encoder misreads a relit object's pose, every candidate chunk inherits the error. Bidirectional Decoding itself notes that plain ensembling lags under sudden perturbations. The experiment that settles it is success as a function of shift severity. If the curves for ensembling and decoding never separate from ours, the proposal fails.

Why not just train with heavy visual augmentation or domain randomization? This is a real alternative, and it may close much of the gap for shifts that augmentation anticipates. We therefore include an augmentation-trained checkpoint as a second starting point and adapt it too. The claim survives only if adaptation adds on top of augmentation for shifts outside the augmentation family, such as novel distractor objects. We will report it honestly if it does not.

Why not just adapt normalization layers, which is cheaper? Evidence from classification is mixed. CoTTA reports normalization-statistic adaptation at 20.4% error on CIFAR10-C versus 16.2% for its EMA-teacher method, and reports degradation of simple baselines over repeated shift cycles. That is not robot evidence, however. Our normalization-only ablation, which uses the same overlap loss, answers the question directly. If it matches full adaptation, the paper's recommendation becomes the cheaper variant.

Why not just fine-tune with reinforcement learning or a residual edit policy? EXPO-FT reaches 30/30 success on real tasks by training a small correction policy, but it needs rewards, human interventions, and resets. Our setting assumes none of these. The comparison is out of scope rather than a competition: the two methods could compose, with our adaptation running between reward-supervised sessions.

Proposed Experiments

We build shifted versions of RoboMimic (Lift, Can, Square, Transport), Push-T, and LIBERO. Lighting, texture, camera-color, and distractor shifts are applied as a continual stream that cycles through shift types ten times, mirroring the Cityscapes-to-ACDC protocol of CoTTA. The metrics are success rate per cycle and its trend across cycles (to expose collapse), cumulative jerk, overlap discrepancy between consecutive chunks, a sensitivity score (the change in the predicted chunk when the goal object is moved), and milliseconds per replan.

Three checks are decisive. First, on the strongest shifts, adapted Diffusion Policy must beat Bidirectional Decoding with EMA smoothing in mean success while costing less per step. Second, the self-target ablation without weight averaging must degrade across cycles while the full method does not. This is the evidence that weight averaging, not just consistency, is doing the work. Third, the sensitivity score must stay near its pretrained value, which shows the consistency loss has not taught the robot to ignore its camera.

A dynamic-perturbation test, such as a Push-T block nudged mid-episode, checks that the agreement gate prevents the method from learning to resist legitimate corrections. A short real-robot study on an SO-100 arm under changed room lighting follows only if the simulation checks pass.