Competent Autonomous Driving Through Fine Tuning Vision Language Action Models with Symbolic Traffic Rules

Barkin Dagda, Vinal Asodia, Iman Sharifi, Peng Wei, Saber FallaharXiv 2026 · arXiv

This paper, recently published by researchers from the University of Surrey and George Washington University, tackles one of the most persistent "black box" problems in autonomous driving: how to ensure that end-to-end neural networks actually follow the rules of the road.

TL;DR

The authors propose a rule-guided Vision-Language-Action (VLA) framework that uses symbolic logic to "teach" autonomous vehicles the UK Highway Code. By combining supervised imitation learning with reinforcement learning guided by executable first-order logic rules, they increased traffic rule compliance from 92.9% to 97.6%, achieving zero violations in critical road-user priority scenarios.

What Problem Does This Paper Address?

Modern autonomous driving is shifting toward Vision-Language-Action (VLA) models. These models are powerful because they can perceive a scene, reason about it in natural language ("there is a pedestrian on the left"), and output control actions (steering, throttle) all within a single architecture.

However, these models are typically trained via Imitation Learning (IL)—they try to copy what a human driver does. The problem? Human demonstrations don't explicitly contain the "why" of traffic laws. A model might learn to drive smoothly but fail to yield at a specific type of crossing because that rule wasn't triggered enough in the training data. There is currently a gap between the high-level reasoning of VLAs and the strict, symbolic requirements of traffic regulations.

Methods

The researchers developed a two-stage learning pipeline to bridge this gap:

  1. Supervised Fine-Tuning (SFT): They first trained a Qwen3-VL-2B backbone to predict driving actions and generate reasoning text based on expert demonstrations.
  2. Reinforcement Learning Fine-Tuning (RLFT): This is the core innovation. They refined the model using a reward function that combines an imitation penalty (don't stray too far from the expert) with a symbolic rule reward.

The Symbolic Evaluator: To calculate the rule reward, they used a frozen VLM to extract "predicates" (Boolean flags like is_bending or ped_crossing) from the camera feed. These predicates, along with vehicle telemetry, are fed into PyReason, a symbolic reasoning engine. PyReason evaluates the vehicle's projected path against formalised First-Order Logic (FOL) versions of the UK Highway Code, specifically:

  • Rule 125 (Safe Speed): Adjusting speed for weather, visibility, and road layout.
  • Rule 126 (Following Distance): Maintaining a safe gap behind the lead vehicle.
  • Rule H (Hierarchy of Road Users): Prioritizing pedestrians and cyclists at junctions.

The authors compared three group-based RL algorithms: GRPO, GDPO, and a variant they call Dr. GRPO (GRPO Done Right), which omits certain normalization terms to provide more stable updates for this specific task.

Datasets

  • CARLA Simulator: All data was collected in the CARLA environment across multiple "towns" featuring urban, residential, and highway settings.
  • Training Set: ~20,000 synchronized frames including RGB images, kinematic states, and expert control actions.
  • Test Set: ~3,600 held-out samples specifically chosen to trigger the three active Highway Code rules (e.g., rainy weather for Rule 125, pedestrian crossings for Rule H).

Key Insights and Ideas

  • The "Priority" Failure of Imitation: One of the most striking findings was that the SFT (imitation-only) model failed the road-user priority check (Rule H) nearly 100% of the time in specific scenarios. This suggests that "yielding" is a nuanced behavior that is difficult to capture through pure imitation without explicit logical grounding.
  • Dr. GRPO is the Winner: The authors found that removing the KL-divergence regularization (common in RLHF for LLMs) allowed the model more flexibility to prioritize safety rules over exact imitation, leading to the best performance.
  • Hard vs. Easy Rules: Learning to follow a speed limit (Rule 125) is relatively easy for these models. However, maintaining a safe following distance (Rule 126) remains "considerably challenging" because it requires continuous, long-horizon reasoning about the relative motion of other dynamic agents.

Key Results

  • Rule Pass Rate: The best-performing model (Dr. GRPO) achieved an overall pass rate of 97.59%, up from 92.93% for the SFT baseline.
  • Zero Violations: In the "Hierarchy of Road Users" (Rule H) tests, the RL-refined model achieved zero observed violations, a massive improvement over the baseline.
  • Control Accuracy: Surprisingly, adding rule-based rewards didn't just make the car safer; it made it more accurate. The Mean Absolute Error (MAE) for throttle dropped from 47.38% (SFT) to 19.16% (Dr. GRPO).
  • Conservative Behavior: Qualitative analysis showed that the rule-guided models were more "conservative," choosing to brake or coast more frequently in straight-line driving to ensure they never breached safety thresholds.

Limitations and Future Work

  • Offline Evaluation: The study was conducted on a pre-collected dataset (offline RL). The authors acknowledge that closed-loop evaluation (letting the model drive live in the simulator) is the necessary next step to see how these rules affect long-term trajectory planning.
  • Rule Breadth: The system currently only encodes three specific rules. Expanding this to the hundreds of provisions in the full Highway Code is a significant scaling challenge.
  • Sim-to-Real: As with all CARLA-based research, the "reality gap" remains. How well these symbolic predicates can be extracted from noisy, real-world sensor data is yet to be determined.

Paper Citation: Dagda, B., et al. "Competent Autonomous Driving Through Fine-Tuning Vision–Language–Action Models with Symbolic Traffic Rules." arXiv:2611.20648v1, Nov 2026.

Summary updated

Abstract

Vision-language-action (VLA) models offer a unified approach to perception, reasoning, and control for autonomous driving, but accurate action prediction alone does not establish compliance with traffic regulations. This paper presents a rule-guided VLA framework that incorporates symbolic traffic-rule feedback into policy optimisation. The framework combines supervised fine-tuning on expert demonstrations with reinforcement learning (RL) refinement using a reward that encourages encoded-rule compliance while penalising deviations from expert actions. Selected provisions of the United Kingdom Highway Code are formalised as executable first-order logic rules using PyReason, covering speed adaptation, following distance, and road-user priority. A frozen vision-language model extracts scene predicates, which are combined with vehicle-state information to assess the projected consequences of candidate control actions. Offline evaluation on held-out data collected in the CARLA simulator compares three group-based RL methods. The results show improvements in both action prediction accuracy and encoded-rule compliance over the supervised baseline. The best-performing variant increases the overall rule pass rate from 92.9% to 97.6%, with zero observed violations of the encoded road-user-priority check in the evaluated samples. These findings support symbolic rule feedback as a means of improving VLA policy alignment with explicit traffic-rule criteria.
BibTeX
@misc{dagda2026competent,
  author = {Barkin Dagda and Vinal Asodia and Iman Sharifi and Peng Wei and Saber Fallah},
  title = {{Competent Autonomous Driving Through Fine Tuning Vision Language Action Models with Symbolic Traffic Rules}},
  year = {2026},
  eprint = {2611.20648},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20648},
}
Figure