OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models

Yiwei Zhang, Xuesong Chen, Fudong Ge, Hanshi Wang, Jin Gao, Weiming Hu, Shaoshuai Shi, Zhipeng ZhangOpenReview 2027 · Submission to ICLR 2027 · Withdrawn · OpenReview

Its authors withdrew this paper, so there is no full text to read or summarize.

Abstract

Vision-Language Models (VLMs) excel at autoregressive text generation, yet end-to-end autonomous driving requires multi-task learning with structured outputs and heterogeneous decoding behaviors, such as autoregressive language generation, parallel object detection and trajectory regression. To accommodate these differences, existing systems typically introduce separate or cascaded decoders, resulting in architectural fragmentation and limited backbone reuse. In this work, we present a unified autonomous driving framework built upon a pretrained VLM, where heterogeneous decoding behaviors are reconciled within a single transformer decoder. We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling. By organizing visual and structured query tokens within a single causal decoder, structured queries can naturally condition on visual context through the original attention mechanism. Textual and structured outputs share a common attention backbone, enabling \textit{stable joint optimization across heterogeneous tasks}. Trajectory planning is realized within the same causal LLM decoder by introducing structured trajectory queries. This unified formulation enables planning to share the pretrained attention backbone with images and perception tokens. Extensive experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance, including 0.28 L2 and 0.18 collision rate on nuScenes open-loop evaluation and competitive results (87.3 PDMS) on NAVSIM closed-loop evaluation. The full model preserves multi-modal generation capability, while an efficient inference mode achieves approximately 40\% lower latency.
BibTeX
@misc{zhang2027onedrive,
  author = {Yiwei Zhang and Xuesong Chen and Fudong Ge and Hanshi Wang and Jin Gao and Weiming Hu and Shaoshuai Shi and Zhipeng Zhang},
  title = {{OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models}},
  year = {2027},
  url = {https://openreview.net/forum?id=9rDXTPy3SJ},
  howpublished = {OpenReview},
  note = {Submitted to ICLR 2027},
}

Loading related content...

Figure