Some distributed protocols decide what to do by asking "what if" questions. A sender keeps transmitting only if, had it stopped, the receiver would not have gotten the message. A robot keeps yielding only if, had it gone ahead, it would have collided. Counterfactual belief-based programs are a formal language for writing such rules. To answer the "what if" part, they rely on a designer-chosen ranking of how surprising each possible execution is. Nobody checks that ranking against measurements. A simulator or digital twin (a high-fidelity software replica of a physical system) removes the excuse, because it can do what the real world cannot. It can rewind to the moment of choice, substitute the other action, and run forward. Every counterfactual test in such a program then becomes an experiment with a measurable answer.
This note proposes treating the plausibility ranking of a counterfactual belief-based program as a model parameter that must be validated the way finite-element models are validated against physical specimens. The program's counterfactual tests are compiled into rewind-and-substitute queries on a fault-injected twin. The ranking is fitted as an order-of-magnitude summary of measured run frequencies. We report a single number, a separation margin, that predicts when the qualitative program will agree with a full probabilistic decision rule and when it will fail. The broader claim is that the formal side supplies verifiable, finite decision rules and the experimental side supplies the evidence those rules silently assume. Neither currently does the other's job.
Core Claim
Using counterfactuals in knowledge-based programming shows why the counterfactual is needed. With ordinary logical implication, "if I stop, the bit is lost" is vacuously true whenever the sender does not stop. The rule then becomes circular. Halpern and Moses replace it with a counterfactual evaluated over Spohn ranking functions, which assign each run a natural-number "surprise" score. Runs that follow the protocol get rank 0, runs where an agent deviates get rank 1, and hardware failures get higher ranks. From knowledge-based programs to graded belief-based programs, part II: off-line reasoning extends this to rankings over belief states themselves.
Both papers leave the actual rank values to the designer. The research file lists this arbitrariness as the direction's first open problem. Choosing between the "rewind" and "independence" readings of a counterfactual in Choice-Driven Counterfactuals is similarly unconstrained. Under the rewind reading, other agents react to your deviation by their default rules. Under the independence reading, their actions stay fixed.
Our claim has three parts. First, in a twin that can rewind, each ingredient of such a program can be checked empirically: the truth of each counterfactual belief the program acts on, the ordering of ranks, and which counterfactual semantics matches how peers actually respond. Second, the standard link between Spohn ranks and infinitesimal probabilities gives a principled fitting rule and a falsifiable prediction of where the qualitative program breaks. Third, and most speculative, ranks fitted in simulation will transfer to hardware better than the underlying probabilities do. An order-of-magnitude summary tolerates the kind of moderate distribution mismatch that validation studies routinely report, such as the latency KL divergence of 0.1 between twin and nuScenes logs in ADDT.
The fitting rule reads ranks off measured frequencies. Let $\hat P(c)$ be the measured frequency of run class $c$ under fault injection, and let $\epsilon \in (0,1)$ be a base that sets how much less likely one rank level is than the one below it. The fitted rank is
$$\hat\kappa(c) = \mathrm{round}\!\left(\frac{\log \hat P(c)}{\log \epsilon}\right).$$
The rank-0 part of the program is a faithful abstraction only when classes at adjacent ranks are well separated in probability. We define the separation margin as
$$m = \min_{c,c' : \hat\kappa(c') = \hat\kappa(c)+1} \left(\log \hat P(c) - \log \hat P(c')\right) / |\log \epsilon|.$$
Here $c$ and $c'$ range over run classes one rank apart, so $m$ is the smallest measured gap between adjacent ranks, in units of rank levels. The prediction is that when $m$ is well above zero, the ranked program's decisions coincide with those of an expected-cost policy using the same $\hat P$. As $m$ approaches zero, disagreements appear, and they appear first at the class pair that sets the margin. This follows from the fact that Spohn ranks are the limit of probabilities as $\epsilon \to 0$. The approximation error is governed by the smallest gap.
Research Program
We would build a counterfactual query compiler for one task family: safe-to-stop coordination. This family covers the bit-transmission problem and its physical cousin, deciding when a robot may stop yielding at a crossing.
The compiler takes a counterfactual belief-based program whose tests have the form "had I done $a'$, would $\psi$ hold?" For each such test it emits a twin query. The query checkpoints the simulator at the choice point, substitutes $a'$, and rolls forward many times under a fault model. Peers follow either their default rules (rewind semantics) or a frozen copy of their recorded actions (independence semantics). The fault model covers message loss, latency spikes, frame drops, and calibration drift of the kind ADDT injects.
Rollouts are grouped into run classes by which protocol steps were followed and which faults fired. From those classes we produce three outputs:
- the fitted ranking $\hat\kappa$ and its margin $m$;
- a counterfactual belief accuracy score, the fraction of the program's acted-on counterfactual beliefs that are true under rewind in the twin;
- an empirical test of the characteristic formula
Expfrom Choice-Driven Counterfactuals. That formula holds in rewind models and fails in independence models, so its measured truth rate tells us which semantics the environment actually realizes.
Calibration is not the whole contribution. The loop is closed by deploying the calibrated program and measuring the outcomes it was designed to optimize: delivery and success, redundant transmissions or needless yields, and delay fairness.
The environments are chosen to span the gap from symbolic to physical:
- Discrete-event network twin. A twin of the bit-transmission problem, where ground truth is cheap and the theory's guarantees can be checked exactly.
- Multi-robot yielding. A reimplementation of the continuous UNIFORM and CORNER navigation maps from Counterfactual Fairness Filter for Fair-Delay Multi-Robot Navigation, with an added lossy communication channel so that yielding depends on beliefs about peers.
- Hardware link. A hardware-in-the-loop version of the messaging layer on embedded boards, in the style of ADDT's Jetson setup, used only for the transfer question.
For the formal side we reuse the SMT-based translation of Program Semantics and a Verification Technique for Knowledge-Based Multi-Agent Systems. It checks that the calibrated program, unrolled to a bounded horizon, satisfies its safety property. The bounded horizon respects that paper's loop-free restriction instead of claiming to lift it.
Baselines
Naive reference: a classical acknowledgment protocol with fixed timeouts. This is a knowledge-based program without counterfactuals. It retransmits or yields until an explicit acknowledgment arrives. Halpern and Moses show it is safe but wasteful. It sets the ceiling on redundant actions and the floor on risk. It runs on identical fault seeds. The calibrated program beats it by cutting redundant transmissions or yields while keeping delivery and collision rates within confidence intervals of the reference.
Hand-ranked counterfactual belief-based program. This isolates our key ingredient. It uses the default ranks of Using counterfactuals in knowledge-based programming: protocol runs at 0, deviations at 1, failures at 2. Program text and compute are identical to ours; only $\hat\kappa$ differs. If hand ranks already match the measured ordering in every environment, calibration adds nothing, and we will report that. The proposal needs calibrated ranks to fix measurable counterfactual belief errors in at least the fault regimes where hand ranks misorder classes. The prime suspects are latency-induced loss versus deliberate deviation, the kind of hardware effect behind ADDT's 40-object latency wall.
Full probabilistic expected-cost policy. This is the strongest competitor on the decision question. It uses exactly the same measured $\hat P$ and the same rewind rollouts, but decides by expected cost rather than by ranks. No paper in the file reports a number for such a policy on these tasks. Our success criterion is therefore not to beat it on cost. We aim to match it wherever $m$ is large, and to diverge from it exactly where $m$ predicts.
NCF2 on the navigation maps. On the CORNER-12-25 map, Counterfactual Fairness Filter for Fair-Delay Multi-Robot Navigation reports a variance of delays of 74.6, against 133.9 for its navigation-only ablation. NCF2 is the strongest existing learned counterfactual yielding method. We rerun it with its published setup plus our communication faults and the same seed count, and give our program's learned navigation module the same training budget. Beating it means lower variance of delays at equal or better success rate and makespan, and specifically under message loss, which NCF2's deterministic-observation assumption does not cover.
Safe reinforcement learning (PPO-Lagrangian). We use the SafePO implementation from Safety Gymnasium. That paper reports a 98% cut in violations with a 45% reward drop, but on velocity-constrained tasks, not ours. We tune its penalty multiplier with the same budget we spend on choosing $\epsilon$.
LLM counterfactual reasoner (belief accuracy only). Following Counterfactual World Models via Digital Twin-conditioned Video Diffusion, we give an 8B vision-language model the structured twin state and ask it the program's counterfactual questions. That paper reports 62.50% on CausalVQA for its full system. We compare counterfactual belief accuracy against rewind ground truth only.
Why not just…?
Why not just use the probabilistic policy and drop ranks entirely? This is the real threat, and the experiments are built around it. If the probabilistic policy is equally cheap to verify and transfers equally well, ranks are unnecessary. We expect two advantages for ranks, and both are testable. The ranked program has a small finite rank structure that the SMT pipeline can check. Its decisions also depend only on the ordering and margin of frequencies, not their exact values, so they should survive the twin-to-hardware shift that changes the frequencies. If on hardware the probabilistic policy's decisions stay as stable as the ranked program's, we will conclude that the ranked program's only remaining value is verifiability and scale the claim back accordingly.
Why not just train safe reinforcement learning in the twin? Safety Gymnasium documents the cost: a 45% reward drop and sensitivity to the penalty multiplier. More importantly, the resulting policy has no inspectable counterfactual beliefs, so there is nothing to validate or verify. The direct comparison in our yielding environment settles the performance question.
Why not just sweep faults exhaustively, as ADDT and AutoDRIVE do? We do run fault sweeps, but aimed at the run classes the program's tests mention. The sweeps in ADDT and Off-Road Autonomy Validation Using Scalable Digital Twin Simulations measure how sensitive a system is to faults. They do not produce a decision rule, and their authors note that the space of combinations explodes. We will count rollouts per decision against an untargeted sweep of equal size.
Why not just estimate counterfactuals offline from logs? The estimator in Safe Counterfactual Reinforcement Learning requires the logging policy to have tried every action being evaluated. Protocol deviations such as stopping early have zero logged probability by design. Rewinding the twin sidesteps that requirement.
Proposed Experiments and Decisive Checks
The first check tests the margin prediction. In the network twin we sweep injected loss and latency rates so that $m$ moves from large to near zero, and we record the rate at which the calibrated program disagrees with the probabilistic policy. The theory predicts almost no disagreement at large $m$ and a sharp rise concentrated on the margin-setting class pair. A flat or random disagreement pattern would falsify the approach.
The second check tests semantics. In the yielding maps we compare two peer populations: peers that react to others by default rules, and peers following fixed plans. Exp should hold empirically for reactive peers and fail for fixed-plan peers. A program run under the mismatched semantics should show lower counterfactual belief accuracy and more needless yields. This would be the first experimental adjudication between the two readings in Choice-Driven Counterfactuals.
The third check tests transfer. We fit both $\hat\kappa$ and $\hat P$ in the twin and refit both on the hardware link. We then compare decision agreement across the two. Ranks should transfer at higher agreement than probabilities. If they do not, the robustness argument for ranks fails.
Across all three checks, the task metrics are delivery or success rate, redundant actions, makespan, and variance of delays, reported over matched seeds against the baselines above. A 20-to-30-point drop on agent-to-agent questions, as seen on STSBench, is a reminder that multi-agent counterfactual beliefs are where heuristic reasoners fail. We therefore report counterfactual belief accuracy separately for self-directed and peer-directed tests.
References
- Using counterfactuals in knowledge-based programming
- From knowledge-based programs to graded belief-based programs, part II: off-line reasoning
- Choice-Driven Counterfactuals
- ADDT - A Digital Twin Framework for Proactive Safety Validation in Autonomous Driving Systems
- Counterfactual Fairness Filter for Fair-Delay Multi-Robot Navigation
- Program Semantics and a Verification Technique for Knowledge-Based Multi-Agent Systems
- Safety Gymnasium: A Unified Safe Reinforcement Learning Benchmark
- Counterfactual World Models via Digital Twin-conditioned Video Diffusion
- Off-Road Autonomy Validation Using Scalable Digital Twin Simulations Within High-Performance Computing Clusters
- Safe Counterfactual Reinforcement Learning
- STSBench: A Spatio-temporal Scenario Benchmark for Multi-modal Large Language Models in Autonomous Driving