Dune: A Zeroth-Order Optimizer with Time-Conditioned Historical Error Regression for VLA Test-Time Adaptation

Chengyang Dong, Ze Chen, Kai Ye, Chutian Wang, Heng Xu, Wei Mo, Maomao Li, Jiayan Xu, Jiayuan Wu, Yihao Xiao, Linhua Ye, Zhonghua Peng, Ruize Han, Jialong SunarXiv 2026 · arXiv

TL;DR

Dune is a memory-efficient zeroth-order optimizer that allows Vision-Language-Action (VLA) models to adapt to new environments directly on robotic hardware. By using historical error regression to filter out gradient noise, Dune overcomes the "performance plateau" typical of derivative-free methods, enabling stable, long-term learning without the massive memory overhead of backpropagation.

What Problem Does This Paper Address?

Vision-Language-Action (VLA) models are the "brains" of modern robots, translating visual inputs and text instructions into physical movements. While these models are powerful, they often struggle when they encounter "out-of-distribution" scenarios—tasks or environments not covered in their massive pretraining datasets.

To fix this, robots need Test-Time Adaptation (TTA): the ability to fine-tune their parameters while they work. However, standard first-order (FO) optimization—the backpropagation we use to train almost all AI—is incredibly memory-intensive. Most edge-computing hardware on robots simply doesn't have the RAM to store the intermediate "activation graphs" required for backprop.

Zeroth-order (ZO) optimization is a tempting alternative. It estimates gradients by slightly perturbing model weights (or activations) and observing how the loss changes, requiring only forward passes and minimal memory. The catch? In high-dimensional spaces like VLA models, these estimates are incredibly noisy. As shown in Figure 1, standard ZO methods like "Dust" start strong but quickly hit a performance plateau or even degrade as noise overwhelms the learning signal over time.

Figure 1. Loss and task success rate in a preliminary compari- son of Dust and standard first-order optimizers for VLA test-time adaptation (VLA-TTA). Dust uses a perturbation population size of K = 64 and converges faster early in adaptation. After 6,000 steps, its task success rate plateaus and then declines, while the first-order optimizers continue to improve steadily.

Methods

The authors introduce Dune (Dust Using Nodewise Error-feedback), which stabilizes ZO learning by treating gradient estimation as a regression problem over time.

  1. Activation Perturbation: Building on the "Dust" optimizer, Dune injects Gaussian noise into the model's internal activations rather than its weights. This allows for more efficient local error estimation.
  2. Time-Conditioned Error Prediction: Recognizing that a robot's error dynamics change throughout a task (e.g., the beginning of a reach vs. the final grasp), Dune uses "time buckets" based on flow-matching timesteps to categorize optimization states.
  3. Online Historical Regression: This is the core innovation. Dune maintains a running history of past perturbations and their resulting errors. It uses online ridge regression to learn a mapping from the current model's output residuals to the expected local error signals.
  4. Weighted Fusion: Instead of relying solely on the noisy "instant" ZO estimate, Dune fuses it with the prediction from its historical regression model. This acts as a sophisticated low-pass filter, preserving the dominant direction of the error signal while suppressing high-frequency noise.
Figure 2. Overview of Dune. The left panel shows the target layers in the Transformer, and the right panel illustrates activation perturbation, conditional historical regression, and error fusion. The upper inset depicts the residual between the current zeroth-order perturbation observation and the historical regression prediction, and shows how weighted fusion corrects the prediction.

Datasets

The researchers evaluated Dune using:

  • LIBERO Benchmark: A suite of 40 simulated manipulation tasks (e.g., "bowl to plate," "mug into microwave") designed to test knowledge transfer and lifelong learning in robots.
  • Real-World Bimanual Task: A physical setup using an AgileX PiPER-X bimanual robot. The task involved a complex box transfer between two arms, requiring precise coordination and adaptation to physical disturbances.

Key Insights and Ideas

  • The Dimensionality Trap: The paper provides a theoretical characterization (Theorem 3.2) showing that ZO variance scales linearly with the number of parameters. In VLAs with millions of parameters, the signal-to-noise ratio (SNR) inevitably collapses, explaining why previous methods plateaued.
  • History as a Denoiser: By leveraging the fact that robotic observations in a video stream are temporally correlated, Dune proves that "looking back" at previous optimization steps is a viable way to recover the true gradient signal without needing backpropagation.
  • Curvature Preservation: One of the most striking findings is that Dune preserves the "block-diagonal" curvature structure of the model's Hessian matrix, similar to advanced first-order optimizers like AdamW. Standard ZO methods lose this structure quickly, leading to instability.

Key Results

Dune achieves a "best of both worlds" result: the memory efficiency of zeroth-order methods with the stability of first-order ones.

  • Success Rates: On the LIBERO benchmark, Dune achieved a 61.4% mean success rate, significantly outperforming other ZO baselines like MeZO (0%) and Dust (47.5%). It even beat first-order SGD (24.2%).
  • Memory Footprint: Dune reduced peak GPU memory consumption by 41.3% compared to AdamW (10.9 GB vs 18.6 GB). This allows full VLA fine-tuning on consumer-grade hardware like an NVIDIA GTX 1080 Ti.
  • Robustness: In disturbance tests (e.g., manually resetting a stove knob while the robot is working), Dune showed a 9/10 recovery rate, whereas the version without historical regression stalled at 4/8.
Figure 6. Trajectory visualizations showing that Dune (top) produces much more stable and successful paths than the version without historical regression (bottom).

Limitations and Future Work

The primary trade-off in Dune is wall-clock time. Because zeroth-order methods require multiple forward passes to estimate a single gradient, Dune takes several times longer to complete the same number of training steps as a backprop-based method.

The authors suggest that future work should focus on accelerating ZO gradient estimation, perhaps through better parallelization or more aggressive dimensionality reduction, to bridge this runtime gap for real-time, high-frequency robotic control.

Summary updated

Abstract

Test-Time Adaptation (TTA) enables Vision-Language-Action (VLA) models to handle distribution shifts during deployment without source-domain retraining. However, backpropagation-based adaptation incurs heavy memory overhead on resource-constrained robots, whereas zeroth-order (ZO) methods suffer from high estimation variance under limited perturbation budgets. We observe that while ZO estimation enables low-memory adaptation with rapid initial gains, it often stalls or degrades over long-horizon updates due to noisy gradient estimates. To address these limitations, we propose Dune, a zeroth-order optimizer that stabilizes long-horizon adaptation by coupling online historical error prediction with current perturbation observations. Specifically, Dune learns a lightweight, time-conditioned error predictor from past optimization dynamics and fuses its predictions with instantaneous ZO estimates, achieving effective variance reduction without backpropagation or extra forward passes. Additionally, we theoretically characterize how estimation variance restricts convergence in high-dimensional, small-population regimes, motivating the use of historical statistics. Extensive experiments on LIBERO and real-world robots demonstrate that Dune achieves superior success rates under a low memory footprint, with ablations confirming improved closed-loop stability and post-disturbance recovery.
BibTeX
@misc{dong2026dune,
  author = {Chengyang Dong and Ze Chen and Kai Ye and Chutian Wang and Heng Xu and Wei Mo and Maomao Li and Jiayan Xu and Jiayuan Wu and Yihao Xiao and Linhua Ye and Zhonghua Peng and Ruize Han and Jialong Sun},
  title = {{Dune: A Zeroth-Order Optimizer with Time-Conditioned Historical Error Regression for VLA Test-Time Adaptation}},
  year = {2026},
  eprint = {2611.25400},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.25400},
}
Figure