Autoregressive End-to-End Planning with Time-Invariant Spatial Alignment and Multi-Objective Policy Refinement

Jianbo Zhao, Taiyu Ban, Xiangjie Li, Xingtai Gui, Hangning Zhou, Lei Liu, Hongwei Zhao, Bin LiarXiv 2025 · DOI · arXiv

Introduction

Autoregressive models have emerged as a powerful paradigm for autonomous driving planning, capable of capturing the multi-modal nature of human driving behaviors. Despite their promise, these models encounter a fundamental challenge: the "stale worldview" problem. In standard end-to-end configurations, a model processes sensor data at time $t$ to predict a sequence of future actions. However, as the vehicle moves, the spatial relationship between the initial sensor "snapshot" and the environment changes, leading to spatio-temporal misalignment.

Spatio-temporal misalignment in end-to-end planning vs. structured data approaches.

Methodology

To address these limitations, this work proposes a three-pronged architectural improvement:

1. Time-Invariant Spatial Alignment (TISA)

Predicting future world states in raw pixel or voxel space is computationally prohibitive for real-time planning. TISA sidesteps this by performing geometric transformations directly in the latent space. By utilizing learnable key-value pairs, TISA "realigns" environmental features from the current timestamp to the ego-vehicle's predicted future poses. This ensures that each step of the autoregressive plan is conditioned on a geometrically consistent representation of the world.

2. Kinematic Action Head

To ensure physical feasibility, the model avoids predicting raw spatial waypoints. Instead, it outputs discretized values for acceleration and yaw rate. These control inputs are processed through a kinematic bicycle model to generate smooth, driveable trajectories, preventing "teleportation" artifacts common in simpler models.

3. Multi-Objective Policy Refinement (MOPT)

While imitation learning provides a strong baseline, it often fails to capture the nuances of safety and comfort. The authors employ Direct Preference Optimization (DPO) to refine the policy. Unlike standard DPO which uses a single scalar reward, this multi-objective approach generates preference pairs based on specific metrics such as Time-to-Collision (TTC) and Drivable Area (DA) compliance. This allows for targeted improvements in safety without sacrificing general driving performance.

The complete model pipeline, highlighting the TISA module and the DPO-based training/inference flow.

Experimental Results

The model was evaluated on the NAVSIM benchmark, which utilizes the OpenScene dataset to provide a comprehensive testing ground with 360-degree camera and LiDAR data.

Performance Highlights

  • State-of-the-Art Performance: The model achieved a 89.8 PDMS score, the highest recorded for autoregressive planners on NAVSIM.
  • Efficiency over Scale: Despite using a lightweight ResNet-34 backbone, the model outperformed massive Vision-Language-Action (VLA) models like AutoVLA (3B parameters), highlighting the importance of architectural alignment over raw parameter count.
  • Ablation Gains: The TISA module contributed a 1.2-point improvement in PDMS, while the Multi-Objective DPO stage provided a significant 3.0-point boost over pure imitation learning.

Conclusion

By addressing the core issue of spatio-temporal misalignment through geometric latent alignment and refining driving policies with multi-objective preferences, this work sets a new standard for end-to-end autonomous planning. The results demonstrate that specialized architectural components like TISA can achieve superior performance compared to general-purpose large-scale models, offering a more efficient path toward safe and reliable autonomous navigation.

Summary updated

BibTeX
@misc{zhao2025autoregressive,
  author = {Jianbo Zhao and Taiyu Ban and Xiangjie Li and Xingtai Gui and Hangning Zhou and Lei Liu and Hongwei Zhao and Bin Li},
  title = {{Autoregressive End-to-End Planning with Time-Invariant Spatial Alignment and Multi-Objective Policy Refinement}},
  year = {2025},
  eprint = {2509.20938},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.20938},
}
Figure