CaFe: Causally Faithful Conditional Traffic Scenario Generation

Steven Yu Tung, Shubham Vaijanath Phoolari, Christoph Lauer, Abhinav Valada, Steven PetersarXiv 2026 · arXiv

In the world of autonomous vehicle (AV) testing, we often use simulators to create "what-if" scenarios. We might tell a simulated car, "Cut in aggressively in five seconds." The goal is to see how our AV reacts. But there’s a hidden bug in many current simulators: intent leakage. The other cars in the simulation "cheat"—they react to the instruction before the maneuver even begins, ruining the test.

A new paper from researchers at CARIAD, University of Freiburg, and TU Darmstadt introduces CaFe (Causally Faithful), a framework designed to stop this cheating and ensure simulations remain realistic and "causally faithful."

TL;DR

CaFe is a language-conditioned traffic simulator that eliminates "intent leakage"—a failure mode where agents react to private instructions they shouldn't be able to see. By using a dual-stream architecture that separates private intent from public observations, CaFe generates realistic, safety-critical scenarios where agents react only to what they can actually see.

What Problem Does This Paper Address?

To validate AVs, we need rare, safety-critical interactions like sudden cut-ins. Modern simulators allow us to "steer" agents using natural language. However, the authors identified a major flaw: intent leakage.

In many existing models, an instruction given to Agent A (e.g., "cut in late") is visible to Agent B through shared neural representations or future trajectory plans. Consequently, Agent B might start braking seconds before Agent A even moves its steering wheel. This "preemptive reaction" defuses the intended interaction, making the simulation useless for testing real-world safety. The paper formalizes this requirement as causal faithfulness: an agent's behavior should only be influenced by its own instructions and the observable history of others.

Figure 1: Intent leakage in existing simulators vs. CaFe. In existing models, Agent B slows down early because it "knows" A's plan. In CaFe, B only reacts once A's motion becomes visible.

Methods

The authors tackle intent leakage through three main contributions:

  1. Paired Intervention Test: To measure leakage, they run a scene twice with the same random seed. In the first run, Agent A follows an instruction. In the second, Agent A's exact motion from the first run is replayed, but the instruction is removed. If Agent B behaves differently in the second run, it proves Agent B was "reading" the instruction, not just reacting to the motion.
  2. CaFe Architecture: A dual-stream diffusion-forcing simulator.
  3. Private Stream: Each agent is conditioned on its own private language instruction.
  4. Public Stream: A language-blind pass that predicts "public beliefs"—what an outside observer would expect agents to do based only on their past motion.
  5. Belief Attention: Crucially, when agents "look" at each other in the neural network (via inter-agent attention), they are restricted to reading these public beliefs, not the private intent or future plans of others.
  6. Closed-Loop Fine-Tuning (CLFT-ES): The model is trained in a closed loop using an "energy score" to ensure that early actions are held responsible for their long-term consequences, improving the overall realism of the rollout.
Figure 3: The CaFe architecture. The private stream handles intent, while the public stream generates beliefs. Inter-agent attention is restricted to these public beliefs to prevent leakage.

Datasets

The researchers introduced WOMD-PACT (Private, Verifiable Traffic Instructions), a massive dataset built on top of the Waymo Open Motion Dataset (WOMD).

  • Scale: 531,099 scenes.
  • Features: It includes timed behavior tags (e.g., "accelerate," "lane shift") and an interaction graph derived from a "counterfactual ghost test" (simulating what would happen if a vehicle hadn't reacted to identify true causal parents).
  • Privacy: Unlike previous datasets that used scene-level descriptions, WOMD-PACT provides per-agent, fact-grounded instructions that are private to each vehicle.

Key Insights and Ideas

  • Leakage is a Shortcut: Models naturally want to use any available information to minimize prediction error. If a future plan is available in the feature space, the model will use it, even if it violates the laws of physics/causality.
  • Architectural Bottlenecks are Necessary: You cannot simply "ask" a model to ignore private info. You must physically block the information flow in the architecture.
  • Anticipation vs. Leakage: There is a fine line between "legitimate anticipation" (braking because a car is slowing down) and "leakage" (braking because a car will slow down in 3 seconds). CaFe preserves the former by allowing agents to attend to public beliefs derived from motion.

Key Results

The authors compared CaFe against several baselines, including broadcast-instruction models (like ProSim) and single-stream models (like LangTraj).

  • Leakage Elimination: Existing models displaced 23% to 83% of nearby uninstructed agents through intent leakage. CaFe recorded 0.0% measurable leakage.
  • Realism: Despite the strict information bottleneck, CaFe maintained high realism, achieving a WOSAC score of 0.763, nearly identical to "leaky" state-of-the-art models.
  • Safety: CaFe actually reduced collision rates (from ~5.8% in single-stream models to 4.8%), likely because agents were no longer reacting to "hallucinated" future plans that might change.
Figure 2: The process for measuring intent leakage using paired rollouts.

Limitations and Future Work

  • Conditioning Modalities: The current work focuses on natural language. The authors suggest that the same "causal faithfulness" principles should be applied to other steering methods, like goal points or route sketches.
  • Belief Modeling: CaFe uses a relatively simple frozen backbone to generate public beliefs. More sophisticated "theory of mind" or explicit belief representations could further improve how agents anticipate one another without cheating.
  • Evaluation Scope: The evaluation is centered on language-conditioned scenarios; broader traffic simulation realism across all possible driving behaviors remains an ongoing challenge.

Summary updated

Abstract

Language-conditioned traffic simulation enables scalable test scenarios steered by natural language. The goal is to control selected agents while surrounding traffic reacts to their observed behaviour. For example, a vehicle instructed to cut in late should cause the following vehicle to brake only when the manoeuvre becomes observable. Yet simulators can leak private instructions across agents, causing surrounding vehicles to react before the instructed behaviour becomes observable. Such premature reactions defuse desired events and alter the ensuing closed-loop scenario. Existing benchmarks measure what agents do, not what they could have known, leaving this failure unmeasured. We formalise it as intent leakage and address it through evaluation, data and modelling. Our paired intervention test holds an instructed agent's executed trajectory fixed while varying its private instruction, isolating leakage-induced changes in surrounding agents. We also construct WOMD-PACT, an automatically generated, verifiable dataset of 531,099 WOMD scenes with timed behaviour tags with reaction episodes identified through a counterfactual ghost test and grounded private instructions. Finally, we introduce CaFe, a dual-stream diffusion-forcing simulator fine-tuned in closed loop that restricts inter-agent interactions to public beliefs, preserving legitimate anticipation without leaking private intent. Our evaluation reveals that existing conditioning designs displace 23-83% of nearby uninstructed agents through private-intent leakage, even when the instructed agent's executed motion is held fixed. CaFe eliminates measurable leakage while maintaining comparable WOSAC realism (0.763), demonstrating that causal faithfulness can be enforced without sacrificing measured simulation realism.
BibTeX
@misc{tung2026cafe,
  author = {Steven Yu Tung and Shubham Vaijanath Phoolari and Christoph Lauer and Abhinav Valada and Steven Peters},
  title = {{CaFe: Causally Faithful Conditional Traffic Scenario Generation}},
  year = {2026},
  eprint = {2611.22855},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.22855},
}
Figure