This paper, recently published by researchers from the University of Surrey and George Washington University, tackles one of the most persistent "black box" problems in autonomous driving: how to ensure that end-to-end neural networks actually follow the rules of the road.
TL;DR
The authors propose a rule-guided Vision-Language-Action (VLA) framework that uses symbolic logic to "teach" autonomous vehicles the UK Highway Code. By combining supervised imitation learning with reinforcement learning guided by executable first-order logic rules, they increased traffic rule compliance from 92.9% to 97.6%, achieving zero violations in critical road-user priority scenarios.
What Problem Does This Paper Address?
Modern autonomous driving is shifting toward Vision-Language-Action (VLA) models. These models are powerful because they can perceive a scene, reason about it in natural language ("there is a pedestrian on the left"), and output control actions (steering, throttle) all within a single architecture.
However, these models are typically trained via Imitation Learning (IL)—they try to copy what a human driver does. The problem? Human demonstrations don't explicitly contain the "why" of traffic laws. A model might learn to drive smoothly but fail to yield at a specific type of crossing because that rule wasn't triggered enough in the training data. There is currently a gap between the high-level reasoning of VLAs and the strict, symbolic requirements of traffic regulations.
Methods
The researchers developed a two-stage learning pipeline to bridge this gap:
- Supervised Fine-Tuning (SFT): They first trained a Qwen3-VL-2B backbone to predict driving actions and generate reasoning text based on expert demonstrations.
- Reinforcement Learning Fine-Tuning (RLFT): This is the core innovation. They refined the model using a reward function that combines an imitation penalty (don't stray too far from the expert) with a symbolic rule reward.
The Symbolic Evaluator:
To calculate the rule reward, they used a frozen VLM to extract "predicates" (Boolean flags like is_bending or ped_crossing) from the camera feed. These predicates, along with vehicle telemetry, are fed into PyReason, a symbolic reasoning engine. PyReason evaluates the vehicle's projected path against formalised First-Order Logic (FOL) versions of the UK Highway Code, specifically:
- Rule 125 (Safe Speed): Adjusting speed for weather, visibility, and road layout.
- Rule 126 (Following Distance): Maintaining a safe gap behind the lead vehicle.
- Rule H (Hierarchy of Road Users): Prioritizing pedestrians and cyclists at junctions.
The authors compared three group-based RL algorithms: GRPO, GDPO, and a variant they call Dr. GRPO (GRPO Done Right), which omits certain normalization terms to provide more stable updates for this specific task.
Datasets
- CARLA Simulator: All data was collected in the CARLA environment across multiple "towns" featuring urban, residential, and highway settings.
- Training Set: ~20,000 synchronized frames including RGB images, kinematic states, and expert control actions.
- Test Set: ~3,600 held-out samples specifically chosen to trigger the three active Highway Code rules (e.g., rainy weather for Rule 125, pedestrian crossings for Rule H).
Key Insights and Ideas
- The "Priority" Failure of Imitation: One of the most striking findings was that the SFT (imitation-only) model failed the road-user priority check (Rule H) nearly 100% of the time in specific scenarios. This suggests that "yielding" is a nuanced behavior that is difficult to capture through pure imitation without explicit logical grounding.
- Dr. GRPO is the Winner: The authors found that removing the KL-divergence regularization (common in RLHF for LLMs) allowed the model more flexibility to prioritize safety rules over exact imitation, leading to the best performance.
- Hard vs. Easy Rules: Learning to follow a speed limit (Rule 125) is relatively easy for these models. However, maintaining a safe following distance (Rule 126) remains "considerably challenging" because it requires continuous, long-horizon reasoning about the relative motion of other dynamic agents.
Key Results
- Rule Pass Rate: The best-performing model (Dr. GRPO) achieved an overall pass rate of 97.59%, up from 92.93% for the SFT baseline.
- Zero Violations: In the "Hierarchy of Road Users" (Rule H) tests, the RL-refined model achieved zero observed violations, a massive improvement over the baseline.
- Control Accuracy: Surprisingly, adding rule-based rewards didn't just make the car safer; it made it more accurate. The Mean Absolute Error (MAE) for throttle dropped from 47.38% (SFT) to 19.16% (Dr. GRPO).
- Conservative Behavior: Qualitative analysis showed that the rule-guided models were more "conservative," choosing to brake or coast more frequently in straight-line driving to ensure they never breached safety thresholds.
Limitations and Future Work
- Offline Evaluation: The study was conducted on a pre-collected dataset (offline RL). The authors acknowledge that closed-loop evaluation (letting the model drive live in the simulator) is the necessary next step to see how these rules affect long-term trajectory planning.
- Rule Breadth: The system currently only encodes three specific rules. Expanding this to the hundreds of provisions in the full Highway Code is a significant scaling challenge.
- Sim-to-Real: As with all CARLA-based research, the "reality gap" remains. How well these symbolic predicates can be extracted from noisy, real-world sensor data is yet to be determined.
Paper Citation: Dagda, B., et al. "Competent Autonomous Driving Through Fine-Tuning Vision–Language–Action Models with Symbolic Traffic Rules." arXiv:2611.20648v1, Nov 2026.
Summary updated