Unveiling the Surprising Efficacy of Navigation Understanding in End-to-End Autonomous Driving

Zhihua Hua, Junli Wang, Pengfei Li, Qihao Jin, Bo Zhang, Kehua Sheng, Yilun Chen, Zhongxue Gan, Wenchao DingarXiv 2026 · arXiv

TL;DR

Current end-to-end autonomous driving systems suffer from a surprising flaw: they largely ignore global navigation commands, over-relying instead on local scene perception. To fix this, the authors introduce Sequential Navigation Guidance (SNG), a framework that combines noisy navigation paths with rich, turn-by-turn textual prompts, delivering state-of-the-art planning performance without requiring auxiliary perception tasks.

What Problem Does This Paper Address?

Most state-of-the-art end-to-end (E2E) autonomous driving models ingest simple, one-hot encoded driving commands like "Turn Left" or "Go Forward" to guide their trajectory planning. However, this paper uncovers a startling reality: corrupting or completely removing these navigation commands barely impacts the performance of popular models like Transfuser.

The authors argue that standard driving commands are fundamentally flawed due to two main issues:

  1. Annotation Ambiguity: Commands are often tied to fixed spatial or temporal intervals. For example, a vehicle navigating a roundabout might experience significant lateral displacement, causing a "Go Forward" intent to be erroneously labeled as a "Turn Left".
  2. Causal Confusion: In beyond-visual-range (BVR) scenarios, a vehicle needs to change lanes well in advance of a turn. If the global command remains "Go Forward" during the lane change, the model becomes confused by the expert trajectory's behavior.

As a result, current E2E planners simply learn to ignore global navigation inputs, leading to brittle behavior in complex, unfamiliar environments.

Fig. 2: We demonstrate erroneous examples in the annotation process of driving commands. (a) Incorrect annotation occurs in roundabouts due to longitudinal displacement. (b) The annotation information fails to comprehend lane-changing behavior, resulting in causal confusion.

Methods

To bridge the gap between global routing and local planning, the authors propose the Sequential Navigation Guidance (SNG) framework, which redesigns how navigation information is represented and ingested:

  • Sequential Navigation Guidance (SNG): Instead of discrete commands, SNG provides two synchronized inputs:
  • Navigation Path: A reference line sampled from road centerlines up to 40 meters ahead. Crucially, the authors inject substantial noise into this path to simulate real-world localization errors and prevent the model from cheating using privileged simulator data.
  • Turn-by-Turn (TBT) Information: Rich textual descriptions detailing current actions, durations, future actions, and supplementary context (e.g., "Enter the roundabout and take the third exit to the left"). These are generated automatically using a powerful offline Vision-Language Model (Qwen2.5-VL-72B).
  • SNG-VLA Model Architecture: The authors build a Vision-Language-Action (VLA) model using a Qwen2.5-0.5B backbone and a SigLIP vision encoder. It processes multi-view images, TBT text, the noisy navigation path, and ego-states. To prevent the model from overfitting to ego-velocity, they implement an attention-based State Dropout Encoder (SDE). The model autoregressively outputs textual reasoning followed by trajectory waypoints via a cross-attention decoder.
Overview of the SNG pipeline, SNG-QA dataset generation, and the SNG-VLA model architecture.

Datasets

The paper utilizes and introduces the following datasets:

  • NAVSIM: A real-world driving dataset used as the base for open-loop and closed-loop evaluations.
  • SNG-QA Dataset (New): The authors developed an automated pipeline to generate 100,000 high-quality QA samples based on NAVSIM. The dataset breaks down driving logic into hierarchical steps: global navigation summarization, local scene planning, and trajectory waypoint generation.
  • Bench2Drive: A closed-loop, multi-ability benchmark based on the CARLA simulator used to evaluate the model across various driving scenarios (merging, overtaking, traffic signs, etc.).

Key Insights and Ideas

  • The Navigation Illusion: Standard E2E models are effectively "blind" to navigation commands. As shown in the command ablation experiments, feeding a model random or fixed incorrect commands results in virtually the same performance as giving it the correct command.
  • Geometry + Semantics Synergy: A sparse navigation path provides excellent long-term geometric constraints, while TBT text provides immediate decision-making logic. Together, they prevent the planner from drifting or making dangerous counter-flow maneuvers.
  • Less is More in Perception Supervision: Unlike many contemporary E2E models (like UniAD) that rely heavily on multi-task auxiliary losses (bounding boxes, semantic segmentation masks), SNG-VLA achieves SOTA results purely through precise navigation modeling and trajectory imitation.
Fig. 1: We demonstrate the impact of introducing pertur- bations to the driving command and the effectiveness of the SNG. The command ablation experiments comprise five distinct experimental settings, with Transfuser-command- ablation representing the statistical results across all five experiments. Detailed data are presented in Table IV. The utilization of random or erroneous navigation information demonstrates minimal impact on model performance. SNG exhibits significant performance improvement over the base- line approach.

Key Results

  • NAVSIM Benchmark: SNG-VLA achieved a state-of-the-art PDMS (Planner Demonstration Metric Score) of 88.24, outperforming competitive baselines like DiffusionDrive (88.1) and Hydra-MDP (86.5). Notably, adding SNG to the baseline Transfuser model boosted its PDMS from 84.0 to 85.5.
  • Bench2Drive Benchmark: SNG-VLA achieved a Driving Score of 67.17 and a Success Rate of 35.90%, outperforming UniAD-Base by 46.6% and 119.4% respectively.
  • Efficiency: SNG-VLA operates with a highly competitive latency of 159.6 ms on an NVIDIA A6000 GPU, making it significantly faster than other large-backbone planners like ThinkTwice (762 ms) and DriveAdapter (931 ms).

Limitations and Future Work

  • CARLA Evaluation Constraints: Due to real-time constraints and the lack of rich textual annotations within the CARLA simulator, the authors had to rely on rule-based methods to generate SNG inputs during Bench2Drive testing, rather than using the full SNG-QA language generation pipeline.
  • Offline VLM Dependency: Constructing the SNG-QA dataset relies heavily on massive offline VLMs (Qwen2.5-VL-72B) for data curation and filtering, which is computationally expensive. Future work could explore distilling this capability into smaller, online modules.

Summary updated

Abstract

Global navigation information and local scene understanding are two crucial components of autonomous driving systems. However, our experimental results indicate that many end-to-end autonomous driving systems tend to over-rely on local scene understanding while failing to utilize global navigation information. These systems exhibit weak correlation between their planning capabilities and navigation input, and struggle to perform navigation-following in complex scenarios. To overcome this limitation, we propose the Sequential Navigation Guidance (SNG) framework, an efficient representation of global navigation information based on real-world navigation patterns. The SNG encompasses both navigation paths for constraining long-term trajectories and turn-by-turn (TBT) information for real-time decision-making logic. We constructed the SNG-QA dataset, a visual question answering (VQA) dataset based on SNG that aligns global and local planning. Additionally, we introduce an efficient model SNG-VLA that fuses local planning with global planning. The SNG-VLA achieves state-of-the-art performance through precise navigation information modeling without requiring auxiliary loss functions from perception tasks. Project page: SNG-VLA
BibTeX
@misc{hua2026unveiling,
  author = {Zhihua Hua and Junli Wang and Pengfei LI and Qihao Jin and Bo Zhang and Kehua Sheng and Yilun Chen and Zhongxue Gan and Wenchao Ding},
  title = {{Unveiling the Surprising Efficacy of Navigation Understanding in End-to-End Autonomous Driving}},
  year = {2026},
  eprint = {2604.12208},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.12208},
}
Figure