Does the Wording Change the Crash? Prompt Sensitivity in Language-Driven Edits of NGSIM Traffic

Autonomous-vehicle teams increasingly describe test scenarios in plain English, such as "make the car in the next lane cut in aggressively." A large language model translates the sentence into a structured scenario file, and a simulator turns that file into vehicle motion. Two failure modes are already well understood. The model can emit broken files, and grammar-constrained decoding fixes that. It can also emit physically impossible motion, and a kinematic vehicle model can filter or repair that. This proposal targets a third failure that neither fix touches. Two prompts that mean the same thing can produce two well-formed, physically feasible scenarios with very different danger levels. One has a cut-in with eight meters of gap and the other has thirty. A test engineer who believes they asked for "an aggressive cut-in" may be measuring something else entirely, and safety conclusions about the vehicle under test can change with the phrasing.

We propose to measure this effect on real highway traffic from the Next Generation Simulation (NGSIM) dataset, a set of camera-tracked vehicle trajectories from the US-101 and I-80 freeways. We hold syntax and physics fixed by construction, so whatever variation remains is attributable to wording alone. We then test whether that variation can be put to use. Following evidence that answers which change under rephrasing are more often wrong, we test whether disagreement across paraphrases flags scenario edits that miss the user's intent, so that a scenario generator can refuse or escalate them.

Core Claim

Our claim is that once a scenario pipeline guarantees valid syntax and kinematic feasibility, a substantial share of prompt sensitivity survives. It shows up in the safety-relevant quantities of the generated scenario: minimum time-to-collision, cut-in gap, and the deceleration the test vehicle must apply. We further claim that the spread of these quantities across semantically equivalent prompts predicts whether the scenario matches the stated intent better than single-prompt confidence signals do. The novelty is not "LLMs are brittle," which is well documented. It lies in isolating brittleness in continuous physical outcomes after the obvious engineering fixes are applied, and in testing whether that brittleness changes downstream conclusions about which controller is safer.

Where This Sits in Prior Work

The motivating systems are the language-conditioned traffic generators. Language Conditioned Traffic Generation (LCTGen) prompts GPT-4 zero-shot with chain-of-thought reasoning to write a YAML-like scenario description, which a learned generator then turns into trajectories. Language-Guided Traffic Simulation via Scene-Level Diffusion (CTG++) instead has GPT-4 write a differentiable loss that steers a diffusion model, and it reports a 0.000 rule-violation rate. Both systems evaluate a fixed prompt template. Neither asks what happens when the user phrases the same request differently, and neither constrains decoding. LANGTRAJ removes the test-time language model by training a text-conditioned diffusion model. That avoids prompt brittleness but requires annotated data and new training. Our setting is the one LCTGen and CTG++ occupy: a frozen, general-purpose model editing real replay data. We study the reliability question they left open.

The method for measuring sensitivity is borrowed from the prompt-robustness literature. ProSA scores sensitivity across 12 templates. It finds that sensitivity tracks low decoding confidence, that "output requirement" prompts are especially fragile, and that moving from zero to one demonstration is the largest single stabilizer. On the Relation between Sensitivity and Accuracy in In-Context Learning (SENSEL) shows that predictions which flip under paraphrase or demonstration reordering are more often wrong (Pearson correlation up to −0.40). It uses this for abstention and gains +4.1 accuracy–coverage AUC over maximum-probability and entropy baselines on GPT-J-6B. Rethinking Hallucinations: Correctness, Consistency, and Prompt Multiplicity defines prompt multiplicity, the share of questions whose answer changes across perturbations, and reports ambiguity above 50% on Med-HALT. That paper also constrains us. It finds that standard uncertainty detectors separate consistent from inconsistent answers much better than correct from incorrect ones, and that consistent-but-wrong errors dominate some benchmarks. Paraphrase consistency may therefore fail to detect intent errors that the model makes reliably, and our experiments must separately measure how often such errors occur. All three papers study discrete text outputs. None touches continuous kinematics.

The structural guarantees come from grammar-constrained decoding, a technique that masks out tokens which would break a formal grammar at each generation step. SynCode eliminates JSON syntax errors entirely. Flexible and Efficient Grammar-Constrained Decoding (GREATGRAMMA) makes grammar preprocessing 17.71x faster than SynCode, with 5–32 ms per-token overhead. SynCode explicitly disclaims semantic correctness. We take that disclaimer as our starting point and push the grammar as far as it can go, including bounded numeric ranges. That lets us ask what is left.

The data regime comes from NGSIM practice. Inertial Single Vehicle Trajectory Prediction Baselines and Applications with the NGSIM Dataset supplies the 5 Hz relative-coordinate preprocessing and a Kalman-filtered constant-velocity model, which reaches a mean negative log-likelihood of 4.44 at five seconds. We use that model as the physical prior for smoothing the replayed background traffic. OpenCDA shows NGSIM replay running inside a CARLA–SUMO co-simulation for merging tests, and we reuse that kind of harness.

Research Program

The task is language-driven editing of a real NGSIM highway segment. A user instruction asks for a maneuver by a background vehicle, such as a cut-in, a hard brake ahead of the test vehicle, or a slow merge. The language model outputs a small JSON scenario edit. The edit names the target vehicle by its NGSIM identifier and gives the onset time, lane-change duration, the gap to the test vehicle at lane crossing, and relative speed. A kinematic bicycle model then renders the edited vehicle's path and splices it into the replay. We deliberately keep the language model's job small and parametric. This makes syntax and physics easy to guarantee, which in turn makes any remaining variation easy to attribute to wording.

Ground truth comes from NGSIM itself, which avoids the need for subjective labels. We extract a few thousand real lane changes and hard-braking events and compute each one's criticality: minimum time-to-collision and the required deceleration of the follower. We bin these into bands such as benign, tight, and critical. Each event is then described in natural language at the level a test engineer would use ("a tight cut-in about one car length ahead, closing slowly"). The descriptions come from templates for scale, plus a human-written subset. Each instruction therefore has a known target band and a known recorded realization. We also write purely synthetic instructions for maneuvers NGSIM rarely contains, scored only by band.

For every instruction we build a paraphrase family. It includes human rewrites, which SENSEL found most informative. It includes surface perturbations of the kind shown to flip classifications in The Butterfly Effect of Altering Prompts: trailing whitespace, politeness phrases, reordered clauses. It also varies how the surrounding traffic state is serialized, through coordinate precision, coordinate frame, and field order, a factor STSBench left untested even though prompting GPT-4o with raw coordinates beat vision models in that benchmark. In few-shot conditions we additionally permute the order of the demonstrations.

The central quantity is kinematic ambiguity. For instruction $i$ with paraphrase family $P_i$, let $b(s)$ be the criticality band of the rendered scenario $s$. We define

$$A_i = 1 - \max_{b} \frac{|\{p \in P_i : b(s_p) = b\}|}{|P_i|},$$

the fraction of paraphrases that disagree with the most common band. We pair it with the continuous spread $\sigma_i$, the standard deviation across $P_i$ of minimum time-to-collision. Selective acceptance proceeds as follows. The pipeline returns the majority-band scenario only when $A_i$ falls below a threshold and otherwise abstains. We trace accuracy against coverage as the threshold varies.

Baselines

The naive reference is the LCTGen-style interpreter. A single zero-shot chain-of-thought prompt is decoded without constraints, and a parse-failure retry loop is allowed up to the same call budget as our method. It shows what current practice delivers. We report its syntax-validity rate, its feasibility rate, and its band accuracy.

The strongest structural competitor stacks every engineering fix: GREATGRAMMA-constrained decoding with a typed schema and bounded numeric ranges, followed by projection of the rendered path onto bicycle-model limits for acceleration, jerk, and steering. If this pipeline has negligible kinematic ambiguity, our core claim fails. All later conditions are built on top of it, so they are compared with syntax and feasibility held equal by construction.

The strongest prompting competitor is ProSA's one-shot anchoring. It uses one NGSIM demonstration, alongside three-shot variants with permuted order. It runs on the same constrained pipeline and is evaluated on the same paraphrase families.

For selective acceptance, the competitors receive an equal number of model calls per instruction. The first is temperature-sampled self-consistency on a single fixed prompt. The second is token-level confidence: the mean log-probability of the numeric fields, playing the role of maximum-probability scoring in SENSEL. The third is SENSEL's demonstration-order-only variant. The only cited figure for this family is SENSEL's +4.1 AUC over maximum-probability scoring on GPT-J-6B text classification. No reported result was found on Lacuna for prompt-sensitivity-based selection on trajectory or scenario generation. Beating these baselines means a statistically significant gain in band accuracy–coverage AUC over the best equal-budget competitor on held-out NGSIM events, for at least two of the three model families.

One ablation isolates the key ingredient. It compares paraphrase variation against temperature sampling at the same total sample count. This tells us whether disagreement induced by wording carries information beyond disagreement induced by sampling.

Models will be Llama-3-70B-Instruct, which ProSA found unusually stable; a small open model such as Gemma2-2B, which SynCode tested; and GPT-4o through its API. Constrained decoding is unavailable through the API, so the GPT-4o arm uses JSON mode. All models use identical prompts, paraphrase sets, and tuning budgets, and thresholds are selected only on a validation split.

Why not just…?

Why not just clamp the output to physical limits? Clamping and projection are already in our strongest baseline. They make a 450 m/s value impossible, but they cannot tell an 8 m cut-in from a 30 m one, because both are feasible. The first experiment measures how much band disagreement remains after projection. If it is near zero, we report that the simple fix suffices and the proposal stops there.

Why not just add one demonstration, since ProSA says it fixes brittleness? This is a real threat. We expect one-shot anchoring to lower ambiguity, but possibly by copying the demonstration's gap and speed, which would collapse the variety of generated scenarios toward the example. We therefore measure both ambiguity and diversity: the coverage of the time-to-collision and gap space, and the distance of each output from its demonstration's parameters. If anchoring removes ambiguity without collapse, selective acceptance becomes unnecessary for that model, and we will say so.

Why not just sample several times at nonzero temperature and vote? This is cheaper, needs no paraphrase writing, and is run as an equal-budget baseline. The Prompt Multiplicity results suggest that both signals mostly track consistency rather than correctness. The decisive comparison is therefore band accuracy among accepted scenarios, not agreement rates. If sampling matches paraphrasing, the practical recommendation is simply to sample.

Why not fine-tune a small model on NGSIM-derived pairs? Template-generated pairs are cheap here. However, LCTGen points to the scarcity of real paired language–trajectory data, and STSBench found driving-specialized fine-tuned models (DriveMM 39.51%, OmniDrive 29.33%) behind zero-shot GPT-4o (57.08%) on spatiotemporal questions. We include a LoRA-tuned small model as a secondary arm, evaluated on the human-written paraphrases it never saw, to test whether its robustness holds outside the template language.

Proposed Experiments

The first decisive check is the existence result. We measure kinematic ambiguity and time-to-collision spread for each model under the fully constrained, projected pipeline on NGSIM US-101 and I-80 events, broken down by perturbation type. Serialization perturbations get special attention, because they are invisible to the user.

The second check is predictive value. Accuracy–coverage AUC for paraphrase-based acceptance is compared against the equal-budget baselines above. We also report the share of consistent-but-wrong instructions, which bounds what any consistency method can catch.

The third check measures the trade-off between anchoring and diversity, across zero-, one-, and three-shot prompts.

The fourth check asks whether wording changes safety conclusions. We run three test-vehicle controllers in the OpenCDA-style replay harness on each paraphrase family's scenarios: the Intelligent Driver Model, a constant-time-gap adaptive cruise controller, and a PPO policy trained on the unedited replay. We then compute Kendall's rank correlation between controller safety rankings, by collision rate and required braking, across paraphrases. This is the physical analogue of the leaderboard reshuffling reported in When Benchmarks are Targets. If filtering by ambiguity restores stable rankings while the unfiltered pipeline lets them flip, the method's value for safety testing is demonstrated directly rather than argued.