Large vision-language models, which read camera images and answer in text, can reason about traffic in ways that end-to-end driving networks cannot. They are also too slow to sit in a 10 Hz control loop. Even after heavy optimization, the emergency-braking model in Dual-AEB takes 1.21 seconds per decision on an embedded GPU. At highway speed the car covers more than a hundred feet in that time. Current systems handle this in one of two ways. They make the model fire faster on every frame, or they call it only when a hand-written alarm such as time-to-collision goes off. In the second case the delay lands exactly when the car can least afford it. We propose a third option, borrowed from Belief-Desire-Intention (BDI) agents. BDI is a decades-old agent design in which an agent commits to a plan (an "intention") and reconsiders it only when new facts contradict it.
The idea, in two sentences: each time the slow model reasons, it outputs a committed maneuver together with a short list of checkable conditions under which that maneuver stops being valid, and a ready-made fallback for each one. A fast monitor checks those conditions against tracked object state at every control tick, switches to the matching fallback the instant one fails, and wakes the slow model in the background to replan. The slow model is never on the critical path of a reaction. It is called only when its own stated assumptions break, so its compute goes where the situation has actually changed.
Core Claim
A vision-language planner that emits guarded intentions, meaning a maneuver plus explicit invalidation predicates plus precomputed contingencies, should beat three alternatives in closed-loop driving: a fixed-rate asynchronous planner, a threshold-triggered planner, and an accelerated every-frame planner. We expect the advantage under realistic latency and a matched budget of model calls. It should be largest in interactive scenarios, where the world changes between scheduled reasoning steps. The new ingredient is narrow and testable. The trigger for reasoning is written by the reasoner itself, in a symbolic vocabulary a fast checker can evaluate, and the response to a trigger is decided before the trigger fires.
Where This Sits in Prior Work
The BDI literature supplies the mechanism. Reactive Reasoning and Planning introduced the Procedural Reasoning System, in which committed intentions act as a filter: beliefs are updated continuously, but deliberation happens only when a belief conflicts with an active plan. We reuse this filter, but the conditions under which a plan conflicts are generated by a learned model per situation rather than hand-coded. Intention Guided Belief Revision argues that belief updates should prioritize what active plans depend on. We make this concrete: the guard predicates decide which tracked objects and quantities the fast loop must keep precise. Real-Time BDI Agents adds CPU and deadline constraints to symbolic BDI reasoning but never confronts neural decoding latency. That is our setting. The survey by Agiollo and Omicini motivates the work directly. Across 98 papers it finds none that use machine learning for option generation, the step that proposes which goals are viable. It also notes that neural components erode BDI's verifiability. Our slow model is exactly a learned option generator. Because its guards are explicit predicates, every commitment it makes can be checked against perception rather than trusted.
The latency literature supplies the competitors. Dual-AEB pairs a rule-based time-to-collision reflex with a multimodal model triggered when the reflex detects a threat. It reasons about each trigger in isolation and outputs only a braking scalar. Our system differs in three ways: it keeps a persistent intention, its trigger is model-written, and its response to the trigger is prepared in advance. VLMLight uses a fast reinforcement-learning policy steered by a slower vision-language meta-controller for traffic signals. It is stationary and lacks persistent commitment tracking. Accelerating Structured Chain-of-Thought in Autonomous Vehicles (FastDriveCoT) restructures reasoning into a dependency graph whose independent fields decode in parallel, reporting a 3.1× to 4.1× speedup in reasoning generation. It still runs the full graph every cycle. We treat its decoder as complementary and reuse it inside our slow branch. VLP uses a language model only at training time and reports zero added deployment latency, but it has no reasoning at test time.
The multimodal-reasoning literature constrains what the guards may be written over. STSBench reports that text-only GPT-4o given ground-truth 3D coordinates scores 57.08% on spatio-temporal driving questions. Pixel-fed InternVL 2.5 8B scores 46.07%, DriveMM 39.51%, and OmniDrive 29.33%. GeoSQA and MATH-Vision find that models fail deduction even after perceiving correctly. Together these say two things. Guards should be predicates over tracked symbolic state, such as gaps, lanes and relative velocities, not over pixels. And the reasoning that produces them should be kept short and decomposed. Divide-and-conquer prompting gives the rationale for evaluating each candidate option in a clean context rather than one long chain.
Research Program
The system has three parts. The belief store is the output of a conventional fast perception-and-tracking stack: tracked agents, lanes and ego state, updated every tick. The slow reasoner is a vision-language model of FastDriveCoT's size class (Qwen2.5-VL-3B). It receives the current images plus a compact text rendering of the belief store. It proposes a few candidate options (for example "hold lane at 25 m/s," "yield to merging vehicle," "overtake left") and commits to one. For the committed option it emits a fixed grammar of guard predicates, such as "lead gap stays above 15 m" or "no agent enters left lane within 30 m behind." Each guard is paired with a contingency maneuver drawn from a small library executable by the low-level planner. The fast monitor evaluates every guard at control rate. It executes the committed maneuver through an existing trajectory planner (VAD), switches to the paired contingency the moment a guard fails, and queues an asynchronous replan. A guard that is malformed or references an object not in the belief store counts as a failed guard. This turns hallucinated commitments into conservative behavior rather than silent errors.
Training data comes from the Bench2Drive expert logs, which are 2 million frames from the Think2Drive policy. We segment them into meta-action episodes. Guards are labeled by asking, at each expert maneuver switch, which tracked quantity crossed which threshold just before the switch. A large teacher model drafts candidate guards and we keep only those that are verified against the logged state. This follows FastDriveCoT's practice of auto-labeling with a 72B model, but adds a programmatic verification step. The reasoner is fine-tuned to produce options, choice, guards and contingencies in one structured output, decoded with FastDriveCoT-style parallel fields.
To make latency real in simulation, we run CARLA synchronously but deliver each slow-branch output only after a delay of $d$ simulation ticks. We sweep $d$ from 0 to 20 ticks at 10 Hz, with 12 ticks matching Dual-AEB's reported 1.21 s. Every method is charged the same delay.
Baselines
The naive reference is the fast branch alone: VAD driving Bench2Drive with no language model. It shows how much any reasoning adds.
The strongest published system on our exact question is Dual-AEB layered on VAD. On Bench2Drive it reports a 14.74% Driving Score increase and a VAD collision-rate drop from 56.43 to 50.86, at 1.21 s latency. Its MLLM recall is 93.14% versus 67.34% for YOLO-based emergency braking. We reimplement its time-to-collision trigger with our reasoner rather than their checkpoint, so the backbone, data, fine-tuning budget and injected delay are identical, and only the trigger logic differs. For reference on the benchmark itself, Bench2Drive reports DriveAdapter as the top system with a 42.08% mean ability score, and only 26.38% success on overtaking.
The second strong competitor is a fixed-rate asynchronous planner. The same reasoner is called every $k$ ticks, and its latest maneuver is held between calls with VAD executing. We set $k$ so its total number of calls equals ours on each route. This is the comparison the claim must survive.
The third competitor is an every-frame accelerated planner using FastDriveCoT parallel decoding. That paper reports its numbers on an internal dataset; no reported result was found on Lacuna for FastDriveCoT on Bench2Drive, so we run it ourselves with the same backbone.
We also include the training-time-only route of VLP. VLP reports a 60.5% collision reduction on nuScenes but no closed-loop Bench2Drive number on Lacuna, so we report it as a reference.
Two ablations isolate the key ingredient. Guards without contingencies: a guard failure triggers replanning but the car holds the old maneuver until the reasoner answers. Contingencies with generic guards: the same fallback library, triggered by fixed hand-written conditions rather than model-written ones. Beating the baselines means a higher Driving Score and lower collision rate than both Dual-AEB-style triggering and fixed-rate calling at matched call budget, at delays of 12 ticks and above. It also means hazard-to-reaction time that does not grow with $d$, which no triggered or periodic baseline can achieve.
Why not just…?
Why not just call the model on a fixed schedule in the background? This is the most serious threat. If the world rarely changes faster than the schedule, periodic calling should match us at no extra complexity. The matched-budget comparison settles it directly. We also split Bench2Drive's 44 scenarios by how abruptly agents change behavior. Our prediction is a gap that is small in cruising scenarios and large in cut-ins, merges and overtakes. If no such gap appears, the guard mechanism is not earning its place, and we would report that.
Why not just trigger on time-to-collision, as Dual-AEB does? A threshold fires only once danger is already close, and the answer then arrives 1.21 s later. Our contingency ablation tests exactly this: if precomputed fallbacks, not model-written guards, account for most of the gain, then a hand-triggered system with fallbacks would suffice. The generic-guard ablation measures that.
Why not just make the model fast enough to run every frame? FastDriveCoT's 1.9× to 3.1× end-to-end reduction does not by itself bring a 1.21 s decision under 100 ms on embedded hardware. It also spends full reasoning on frames where nothing changed. We use its decoder inside our slow branch, so the two are complementary. The every-frame baseline tells us how much delay remains once both are applied.
Why not just distill the language model away, like VLP? Distillation removes reasoning when it is most needed. STSBench's driving-specialized models trail a generalist on interaction questions, which suggests narrow distillation loses multi-agent reasoning. We include VLP-style training as a baseline rather than assume it loses.
Decisive Checks
The first check is closed-loop Bench2Drive Driving Score, success rate and collision rate across the delay sweep, against every baseline above.
The second check is guard faithfulness. Offline, we run the reasoner on every frame of held-out routes and mark the ticks where its chosen maneuver changes. We then measure what fraction of those changes were preceded by a guard failure (recall) and what fraction of guard failures led to a changed decision (precision). High recall with modest precision is acceptable. Low recall means the model's stated commitments do not describe its real decision boundary, and the approach fails at its core.
The third check uses STSBench, via the STSBench resource, to test the belief interface on its own. We feed our reasoner images alone versus images plus the text-rendered belief store, and expect the gap to resemble the reported 57.08% versus 46.07% difference, especially on Agent-to-Agent questions. If symbolic beliefs do not help there, guards written over them are unlikely to help in driving.
Finally, we report model calls per kilometer and the fraction of driving time spent executing a maneuver whose guards are currently violated with no contingency in force. That fraction directly measures stale-plan risk, which neither periodic nor triggered baselines can drive to zero.
References
- Dual-AEB: Synergizing Rule-Based and Multimodal Large Language Models for Effective Emergency Braking
- Reactive Reasoning and Planning
- Intention Guided Belief Revision
- Real-Time BDI Agents: A Model and Its Implementation
- Integrating Machine Learning into Belief-Desire-Intention Agents: Current Advances and Open Challenges
- VLMLight: Safety-Critical Traffic Signal Control via Vision-Language Meta-Control and Dual-Branch Reasoning Architecture
- Accelerating Structured Chain-of-Thought in Autonomous Vehicles
- VLP: Vision Language Planning for Autonomous Driving
- STSBench: A Spatio-temporal Scenario Benchmark for Multi-modal Large Language Models in Autonomous Driving
- GeoSQA: A Benchmark for Scenario-based Question Answering in the Geography Domain at High School Level
- Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset
- When Splitting Makes Stronger: A Theoretical and Empirical Analysis of Divide-and-Conquer Prompting in LLMs