Do VLMs Really Look at the Road? A Diagnostic Evaluation for Autonomous Driving

Shankar Gangisetty, Muskan Singh, C V JawahararXiv 2026 · arXiv

TL;DR

This paper introduces a diagnostic benchmark to reveal that high accuracy in Vision-Language Models (VLMs) for autonomous driving is often a mirage. By testing models on counterfactual scenes and negated questions, the researchers show that current VLMs frequently rely on language biases and "safety priors" rather than actually looking at the road to make decisions.

What Problem Does This Paper Address?

In the world of autonomous driving, "why" a model makes a decision is just as important as the decision itself. Recent Vision-Language Models (VLMs) have shown impressive performance on driving benchmarks, leading some to believe they are ready to act as interpretable reasoning modules for self-driving cars.

However, the authors argue that aggregate accuracy is a deceptive metric. A model might correctly answer "Yes" to "Should the driver turn on the headlights?" simply because it has learned a linguistic bias that "headlights" usually go with "Yes" in a driving context, not because it actually perceives darkness or rain. This "Clever Hans" effect—where a model performs well by exploiting dataset regularities rather than reasoning—poses a massive safety risk. If a model doesn't truly ground its decisions in visual evidence, it cannot be trusted in edge cases or adverse conditions.

Methods

The authors propose a diagnostic evaluation framework that moves beyond simple accuracy. Their approach uses four distinct "stress tests":

  1. Image vs. No-Image Evaluation: They feed the model a blank input instead of a road scene. If the model still answers driving questions with high confidence and accuracy, it’s relying on language priors, not vision.
  2. Counterfactual Weather Sensitivity: Using physics-based degradation and AI synthesis, they transform clear-weather scenes into foggy, rainy, or snowy versions of the exact same scene. They measure Scene-Change Sensitivity (SCS)—how often the model actually updates its decision based on these visual changes.
  3. Linguistic Robustness: They test models using paraphrases (asking the same thing in different words) and negation (e.g., "Is there NO pedestrian?"). This checks if the model understands the semantic logic or is just triggered by keywords.
  4. Targeted Reasoning: For failed cases, they prompt the model to provide an explanation. This qualitative step reveals whether the model is hallucinating evidence or using generic safety heuristics (e.g., "I should slow down because it might be slippery," even if the road is bone dry).

Datasets

The benchmark, totaling 520 images and nearly 27,000 question-answer pairs, is built using:

  • BDD100K: Used for the "Adverse Weather" and "Junctions & Intersections" categories.
  • DriveLM / nuScenes: Used for a spatial reasoning extension that requires the model to understand the relative positions of objects (left, right, ahead).
  • Programmatic Ground Truth: Crucially, all labels were derived programmatically from existing metadata, ensuring the benchmark is reproducible and free from subjective manual labeling bias.

Key Insights and Ideas

  • The Affirmation Bias: Most models suffer from a strong "Yes" bias. For example, Moondream 2B was nearly 30 percentage points more likely to answer "Yes" than "No," regardless of the image.
  • The "Night" Paradox: Models actually performed better on night scenes than clear day scenes. Why? Because at night, the "safe" answer is usually "Yes" (turn on lights, slow down), which aligns with the models' internal language priors.
  • Negation Blindness: While models are consistent when you paraphrase a question, their performance collapses to near-chance (50/50) when a question is negated. They struggle to understand the difference between "Is a pedestrian crossing?" and "Is a pedestrian NOT crossing?"
  • Perception vs. Reasoning Gap: Models are surprisingly good at seeing (e.g., "Is there a traffic light?"), but fail at reasoning (e.g., "Is it safe to proceed?"). This suggests that the bottleneck isn't the "Vision" part of the VLM, but the "Language" part's ability to integrate visual cues into a logical driving plan.

Key Results

The researchers evaluated six open-source models, including Qwen2.5-VL, InternVL3, and LLaVA-OV.

  • Accuracy vs. Grounding: While top models achieved ~78% overall accuracy, their Scene-Change Sensitivity (SCS) was dismal, averaging only 29.2%. This means that in 70% of cases, changing the weather from clear to a blinding fog didn't change the model's driving advice.
  • No-Image Performance: Models continued to produce plausible-sounding driving justifications even when the input image was completely removed.
  • Negation: Only one model (Qwen2.5-VL 7B) performed significantly above chance on negated questions (73.4%), while others hovered around 40-50%.

Limitations and Future Work

  • Model Scope: The study focused on open-source models. While these are the backbone of much current research, proprietary models (like GPT-4o or Gemini) were not included in this specific diagnostic.
  • Zero-Shot Focus: The evaluation was strictly zero-shot. The authors note that fine-tuning or in-context learning might mitigate some biases, but the core goal was to test the "out-of-the-box" reasoning capabilities.
  • Future Directions: The authors call for the community to move away from "aggregate accuracy" and toward "diagnostic metrics" that explicitly measure visual grounding and linguistic robustness in safety-critical AI.

Summary updated

Abstract

Vision-language models (VLMs) are increasingly being evaluated for autonomous driving through question-answering benchmarks, yet high benchmark accuracy does not necessarily reflect visually grounded reasoning. We present a diagnostic evaluation framework that systematically probes whether VLMs base safety-critical driving decisions on visual evidence or instead rely on language priors, answer biases, and dataset regularities. Our benchmark comprises two task categories, Adverse Weather and Junctions \& Intersections, with a nuScenes spatial reasoning extension, and evaluates open-source (general, driving, and spatial-reasoning) VLMs along four diagnostic protocols: image versus no-image evaluation, counterfactual weather sensitivity, linguistic robustness under paraphrasing and negation, and spatial reasoning. All ground-truth labels are derived programmatically from BDD100K and DriveLM annotations, enabling reproducible evaluation without manual labeling. Across all models, we find that competitive overall accuracy conceals systematic failure modes, including strong affirmation bias, limited sensitivity to counterfactual scene changes, near-chance performance on negated questions despite high paraphrase consistency, and confident driving decisions even without visual input. These results demonstrate that current open-source VLMs remain insufficiently grounded for safety-critical autonomous driving and underscore the need for diagnostic evaluation beyond aggregate accuracy.
BibTeX
@misc{gangisetty2026do,
  author = {Shankar Gangisetty and Muskan Singh and C V Jawahar},
  title = {{Do VLMs Really Look at the Road? A Diagnostic Evaluation for Autonomous Driving}},
  year = {2026},
  eprint = {2611.19184},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.19184},
}
Figure