AdaWM: Adaptive World Model based Planning for Autonomous Driving

Abhirup Mallik

Abstract

World model-based reinforcement learning (RL) has emerged as a promising approach for autonomous driving, utilizing a learned latent dynamics model to train planning policies. To accelerate learning, researchers often employ a pretrain-finetune paradigm, where online RL is initialized with a model and policy learned from offline data. However, transitioning to new tasks often triggers significant performance degradation due to distribution shifts. We identify two primary causes for this "performance dip": mismatches in the planning policy and the dynamics model. To address this, we introduce AdaWM (Adaptive World Model), a planning method featuring mismatch identification and alignment-driven finetuning. By selectively updating components using efficient low-rank updates, AdaWM ensures robust and efficient adaptation. Extensive experiments in the CARLA simulator demonstrate that AdaWM significantly outperforms existing baselines in complex driving scenarios.

Introduction

In the quest for safe and efficient autonomous driving, the ability to adapt to new environments—such as different weather conditions, traffic densities, or road layouts—is critical. While training from scratch is computationally expensive and potentially dangerous, the standard alternative—finetuning a pretrained model—often suffers from a "catastrophic" performance drop during the initial stages of online interaction. This phenomenon occurs because the agent's internal representation of the world (the world model) and its decision-making strategy (the policy) are no longer aligned with the new environment's dynamics.

The Challenge of Distribution Shift

The core difficulty in the pretrain-finetune paradigm is distribution shift. When an agent trained on right-hand turns is suddenly tasked with navigating a dense roundabout, it faces two distinct types of errors:

  1. Dynamics Model Mismatch: The agent's internal "physics engine" fails to predict how the environment will evolve, leading to inaccurate simulations of future states.
  2. Policy Mismatch: Even if the model were perfect, the agent's learned maneuvers may be inappropriate for the new task's specific requirements.

Naive finetuning strategies typically update all parameters simultaneously. However, this often leads to noisy updates that overwrite useful pretrained knowledge, resulting in the observed performance crash.

The AdaWM Framework

AdaWM introduces a surgical approach to finetuning through a two-step process: Mismatch Identification and Alignment-driven Finetuning.

1. Mismatch Identification

Rather than assuming both the model and policy are equally flawed, AdaWM quantifies the specific source of error at each step. It utilizes Total Variation (TV) distance to measure the shift in both the dynamics model and the policy. This allows the system to determine whether the performance bottleneck is a misunderstanding of the world's "physics" or a failure in decision-making logic.

2. Alignment-driven Finetuning

Once the dominant mismatch is identified, AdaWM selectively updates the corresponding component. This prevents the policy from trying to learn based on a broken world model, or vice versa. To maintain efficiency and stability, the method employs:

  • Low-Rank Adaptation (LoRA): For the World Model, AdaWM updates only a small set of low-dimensional vectors, preserving the core knowledge of the massive neural network while allowing for rapid adaptation.
  • Weighted Convex Ensembles: The policy is treated as a collection of sub-units, where finetuning focuses on updating the weights of these units to streamline the learning process.

Experimental Evaluation

We evaluated AdaWM using the CARLA simulator, a high-fidelity 3D environment. The testing suite included the Bench2Drive benchmark, encompassing 2 million frames across 44 scenarios and 23 weather conditions. We specifically focused on challenging transitions, such as moving from simple right turns in light traffic to dense-traffic roundabouts (ROM03) and complex left turns (LTD03).

Key Results

AdaWM demonstrated substantial improvements over state-of-the-art baselines, including DreamerV3, VAD, and UniAD:

  • Success Rate (SR): In the highly demanding LTD03 task (Left Turn, Dense Traffic), AdaWM achieved a 70% success rate, doubling the performance of the strongest baseline, DreamerV3 (35%). Supervised methods like UniAD and VAD struggled to exceed a 10% success rate in these interactive settings.
  • Safety Metrics: Safety is quantified by Time-to-Collision (TTC). In the ROM03 task, AdaWM maintained a TTC of 2.05, significantly higher than the baseline average of approximately 0.95, indicating much safer driving behavior.
  • Stability: By identifying the "Model-First" priority—fixing the world model before the policy—AdaWM successfully mitigated the performance crash typically seen at the start of online finetuning.

Discussion and Future Work

AdaWM proves that selective, alignment-driven updates are more effective than global finetuning for autonomous agents. However, several avenues for future research remain. Currently, the method relies on a threshold parameter ($C$) to switch between model and policy updates; automating the selection of this parameter would improve generalizability. Additionally, while AdaWM handles dense traffic well, explicitly modeling the social negotiations inherent in multi-agent interactions could further enhance performance in complex urban environments.

By addressing the root causes of performance degradation during finetuning, AdaWM provides a robust framework for deploying autonomous driving systems that can safely and efficiently adapt to the ever-changing real world.

Summary updated

Abstract

World model based reinforcement learning (RL) has emerged as a promising approach for autonomous driving, which learns a latent dynamics model and use it to train a planning policy. To speed up the learning process, the pretrain-finetune paradigm is often used, where online RL is initialized by a pre-trained model and a policy learned offline. However, naively performing such initialization in RL may result in dramatic performance degradation during the online interactions in the new task. To tackle this challenge, we first analyze the performance degradation and identify two primary root causes therein: the mismatch of the action policyand the mismatch of the dynamics model. due to distribution shift. We further analyze the impact of these factors on performance degradation during finetuning, and our findings reveal that the choice of finetuning strategy plays a pivotal role in mitigating these effects. We then introduce AdaWM, an Adaptive World Model based planning method, featuring two key steps: (a) mismatch identification, which quantifies the mismatches and informs the finetuning strategy, and (b) alignment-driven finetuning, which selectively updates either the policy or the model as needed using efficient low-rank updates. Extensive experiments on the challenging CARLA driving tasks demonstrate that AdaWM significantly improves the finetuning process, resulting in more robust and efficient performance in autonomous driving systems.
BibTeX
@inproceedings{mallik2025adawm,
  author = {Abhirup Mallik},
  title = {{AdaWM: Adaptive World Model based Planning for Autonomous Driving}},
  booktitle = {International Conference on Learning Representations},
  year = {2025},
  url = {https://openreview.net/forum?id=NEu8wgPctU},
}
Figure