AdaPower: Specializing World Foundation Models for Predictive Manipulation

Yuhang Huang, SHilong Zou, Jiazhao Zhang, Xinwang Liu, Ruizhen Hu, Kai XuarXiv 2025 · DOI · arXiv

Abstract

World Foundation Models (WFMs) have demonstrated remarkable capabilities in generating realistic video content, yet they often lack the physical precision and action-conditioning necessary for robotic manipulation. We introduce AdaPower, a framework that transforms general-purpose WFMs into action-conditioned "specialist" world models. By employing Temporal-Spatial Test-Time Training (TS-TTT) and Memory Persistence (MP), AdaPower enables high-fidelity dynamics prediction in novel environments. When integrated into a Model Predictive Control (MPC) framework, AdaPower empowers existing Vision-Language-Action (VLA) models, increasing success rates on unseen tasks by over 41% without requiring additional policy training.

Introduction

The emergence of World Foundation Models (WFMs) like Sora and Cosmos has revolutionized video generation. However, for robotics, "visual realism" is insufficient; models must be physically accurate and responsive to specific robot actions. Current WFMs primarily focus on text-to-video generation, whereas robots require action-to-video capabilities to predict the outcomes of their movements. Furthermore, standard models often struggle with generalization to new environments and long-horizon consistency.

Overview of the AdaPower framework, illustrating the adaptation of generalist models into specialists for MPC-based robotic control.

AdaPower addresses these challenges by adapting a pre-trained WFM (specifically Cosmos-Predict-2B) into a high-fidelity "mental simulator." This Specialist World Model (SWM) is then used within a Model Predictive Control (MPC) loop to evaluate and select the best action sequences proposed by a pre-trained VLA policy, such as CogACT or $\pi_0$.

Methodology

AdaPower introduces several architectural innovations to bridge the gap between general video generation and precise robotic simulation.

Action-Conditioned Adaptation

The framework replaces the standard text encoder with a multi-layer perceptron (MLP) action encoder. This allows the model to process robot action sequences as tokens, enabling precise action-to-video prediction rather than relying on vague text descriptions.

Temporal-Spatial Test-Time Training (TS-TTT)

To handle distribution shifts in new environments, AdaPower performs self-supervised updates during inference. It leverages spatial and temporal redundancies in video data—such as static backgrounds and local pixel similarities—to reconstruct the current scene accurately. This "test-time training" allows the model to adapt to unfamiliar lighting or objects on the fly.

Memory Persistence (MP)

To maintain object consistency over long sequences, AdaPower utilizes a DINOv2 encoder to extract object-centric features from historical frames. These features are integrated via cross-attention, preventing the model from "forgetting" object states or locations during long-horizon tasks.

Detailed architecture of AdaPower, highlighting the Action Encoder, Memory Persistence, and Temporal-Spatial Test-Time Training (TS-TTT) modules.

Results

The framework was evaluated on the LIBERO benchmark and in real-world settings, demonstrating significant improvements over base VLA models.

  • Success Rate Boost: On LIBERO-LONG unseen tasks, the base CogACT model's success rate improved from near-zero to 41.5% when empowered by AdaPower.
  • Efficiency: AdaPower is highly efficient, adding only approximately 150M parameters (<10% of the base model size), making it a lightweight alternative to full policy fine-tuning.
  • Real-World Transfer: In physical experiments with a Franka Research 3 arm, AdaPower increased zero-shot success rates by over 30% on tasks involving complex manipulations like pouring and stacking bowls.

Conclusion

AdaPower demonstrates that WFMs can serve as powerful dynamics engines for robotic planning rather than just data generators. By enabling on-the-fly adaptation and long-term consistency, it provides a scalable path for improving robotic manipulation in diverse, unseen environments. Future work will focus on reducing inference latency and extending the framework to more complex physical dynamics, such as deformable objects.

Summary updated

Abstract

World Foundation Models (WFMs) offer remarkable visual dynamics simulation capabilities, yet their application to precise robotic control remains limited by the gap between generative realism and control-oriented precision. While existing approaches use WFMs as synthetic data generators, they suffer from high computational costs and underutilization of pre-trained VLA policies. We introduce \textbf{AdaPower} (\textbf{Ada}pt and Em\textbf{power}), a lightweight adaptation framework that transforms general-purpose WFMs into specialist world models through two novel components: Temporal-Spatial Test-Time Training (TS-TTT) for inference-time adaptation and Memory Persistence (MP) for long-horizon consistency. Integrated within a Model Predictive Control framework, our adapted world model empowers pre-trained VLAs, achieving over 41\% improvement in task success rates on LIBERO benchmarks without policy retraining, while preserving computational efficiency and generalist capabilities.
BibTeX
@misc{huang2025adapower,
  author = {Yuhang Huang and SHilong Zou and Jiazhao Zhang and Xinwang Liu and Ruizhen Hu and Kai Xu},
  title = {{AdaPower: Specializing World Foundation Models for Predictive Manipulation}},
  year = {2025},
  eprint = {2512.03538},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.03538},
}
Figure