Language models answer causal questions badly in a specific way. A causal question asks what would happen if we forced a variable to a value (an "intervention"), or what would have happened under different circumstances (a "counterfactual"). When models are asked to turn such a question, posed as a story, into a precise causal query and a formula computable from the stated data, they get the translation wrong. The final arithmetic is usually fine. The standard benchmark for this, CLadder, has binary yes/no answers, so a model that gets the translation wrong still lands on the right verdict about half the time. Any training method that rewards only the final answer will reward wrong reasoning on roughly half the cases where the reasoning is wrong.
We propose training a compact open model (around 8B parameters) to be a specialist at the step where models fail. Given a causal story, it writes down a formal specification: the causal graph, the type of query, and an "estimand," meaning a formula over observable probabilities that equals the causal quantity asked about. A deterministic engine then executes that specification. We grade the model's translation with an exact check that cannot be guessed. We sample many random causal models consistent with the true graph. For each one, we evaluate the model's formula on that model's observational distribution and compare it with the true interventional or counterfactual answer. A formula that matches across all samples is correct; one that matches only by coincidence on a single dataset is not. The claim is that this graded-translation signal yields a small model that beats frontier prompting on formal causal reasoning. It should also stay accurate when the causal arrows in a story contradict common sense. Answer-only reinforcement learning and supervised imitation on the same data should not achieve this.
Core Claim and Where It Comes From
Three findings motivate the proposal.
- Translation is the bottleneck. The CLadder error analysis (CLadder) finds that models mostly fail when identifying the query type and deriving the estimand, not in the arithmetic. GPT-4 reaches only 62.03% accuracy, and 70.40% with the six-step CausalCoT prompt.
- Graph extraction is not the problem. Counterfactual Causal Inference in Natural Language with Large Language Models shows that models extract graphs from text well, yet barely improve even when handed the true graph. The remaining errors come from reasoning over how variables combine.
- Right answers often come from wrong reasoning. CausalityCheck reports Claude-Sonnet-4.5 at 82.27% on judging whether a question is answerable but only 59.46% on judging whether a reasoning process is valid.
Together these say two things. The signal worth training on is the formal translation. Graph-retrieval and answer-level signals miss it.
The method comes from domain specialization work. AutoTriton shows that an 8B model trained with Group Relative Policy Optimization (GRPO) can outperform DeepSeek-R1-0528 and GPT-4o on GPU-kernel generation. GRPO is a reinforcement learning method that compares several sampled answers to the same prompt and pushes toward the better ones. AutoTriton's rewards come from automated execution checks: 39.16% versus 30.12% and 15.06% execution accuracy on TritonBench-T. We carry over the recipe of "small model plus automatic verifier," and change what is verified. AutoTriton checks whether output code runs and matches a reference. Causal queries need a check on whether a formula is the right formula, because a scalar yes/no answer is too easy to hit by chance.
Our contribution differs from the closest work as follows:
- Counterfactual distillation. Teaching Small Language Models Reasoning through Counterfactual Distillation perturbs nouns in questions. It never touches structural causal models or interventions.
- FARD. FARD regularizes attention along the internal dependency graph of a math solution. It does not reason over an external causal graph.
- Double Counterfactual Consistency. Better Think Thrice rewards answers that stay the same after a perturbation is applied and then reversed, but only on math and code. It also reports that consistency and accuracy diverge, which warns that consistency alone is not a substitute for correctness.
- Causal Cartographer. Causal Cartographer shows that an 8B model can do counterfactual reasoning when its context is restricted to a variable's causal parents. It never updates model weights.
- None of these grades a model's derived causal formula for correctness.
Research Program
Generator and data. We build a problem generator over structural causal models: small directed acyclic graphs of binary variables with explicit mechanisms. It has three deliberate differences from CLadder's generator:
- a wider family of graph shapes, including ones CLadder does not use;
- story templates written independently of CLadder's;
- a majority of training stories using nonsense variable names, so that real-world word associations carry no information.
Each problem comes with its true graph, its rung on Pearl's ladder (association, intervention, counterfactual), its true causal quantity, and the observational and conditional probabilities stated in the story. We keep CLadder itself entirely out of training.
Output format. The model emits a short reasoning trace followed by a machine-readable block. The block contains the graph as an edge list, a query label, and an estimand written in a small expression language over the stated probabilities. An executor evaluates the estimand on the story's numbers and returns the yes/no verdict.
Reward. The reward has staged terms, given in decreasing weight:
- Estimand equivalence (the dominant term), defined below.
- Graph agreement, scored as one minus the normalized number of edge edits needed to turn the model's graph into the true one.
- Query-type match.
- Final verdict.
Estimand equivalence is defined as follows. Let $\hat{E}$ be the model's formula, $Q$ the true causal quantity, $G$ the true graph, and $\theta_1,\dots,\theta_K$ random parameter settings of models consistent with $G$. Write $P_{\theta_k}$ for the observational distribution under $\theta_k$, and $Q(\theta_k)$ for the true quantity computed by direct intervention or abduction in that model. The term is awarded when
$$\max_{k \le K} \left| \hat{E}(P_{\theta_k}) - Q(\theta_k) \right| < \epsilon .$$
A formula that is wrong for the graph, such as one that adjusts for a variable that should not be adjusted for, will fail on some $\theta_k$ with high probability as $K$ grows. A guessed verdict earns nothing on this term.
Training stages. We train in two stages, mirroring AutoTriton. First, a brief supervised warm-start on generator traces teaches the output format. Then GRPO runs with the staged reward.
Specialization cost. We also measure the cost of specializing. Specializing Smaller Language Models towards Multi-Step Reasoning found that narrowing a small model onto one reasoning distribution degrades it on BigBench Hard. We test whether formal-translation training does the same.
Baselines
Naive references. The floor is chance on CLadder's binary answers, which the CLadder paper puts at about 50%. The "do nothing" reference is the same 8B base model with plain input-output prompting and with CausalCoT prompting.
Strongest prompting baseline. The strongest existing method with reported full-dataset numbers is GPT-4 with CausalCoT: 70.40% accuracy on CLadder (CLadder), against 62.03% for vanilla GPT-4. We also report DeepSeek R1 at 89.1% on CLadder's commonsensical subset with simple prompting (From Causal Parrots to Causal Prophets?). This is a high bar, but only on the subset where memorized associations help most. For CausalPoT on GPT-4o, the same paper reports only a gain of 8.8 points over input-output prompting, not an absolute score. We will rerun it rather than quote a figure.
What counts as winning against frontier prompting. Our 8B model must exceed 70.40% on the full CLadder dataset. It must also shrink the gap between commonsensical and anti-commonsensical stories below the gap GPT-4 shows under CausalCoT.
Strongest trained competitors. These isolate our key ingredient. All use the identical generator data, the same base model, the same GPU-hour budget, and the same hyperparameter-search budget:
- Supervised fine-tuning on gold CausalCoT-style traces with the formal block.
- Outcome-only GRPO, where the reward is the final verdict alone. This is AutoTriton's recipe transplanted.
- Ablation without estimand equivalence. Staged GRPO keeping the graph, query-type and verdict terms, with the estimand-equivalence term removed.
- Consistency-reward variant. A version adding the reverse-intervention consistency reward from Better Think Thrice, to test whether consistency adds anything once correctness is graded.
What counts as winning against trained competitors. Beating supervised fine-tuning and outcome-only GRPO on held-out benchmarks by a margin larger than seed-to-seed variance. The comparison that matters is held-out performance, not the generator's own test split.
Why not just…?
Why not just prompt a frontier model with CausalCoT or CausalPoT? Structured prompting tops out at 70.40% on CLadder for GPT-4, and models still do worse on anti-commonsensical stories (CLadder). Better Think Thrice also reports that few-shot formatting can hurt consistency for some model families. CausalPoT, which offloads computation to the DoWhy library, is the real threat. If frontier CausalPoT matches our 8B model, then translation errors are not the bottleneck at frontier scale, and the contribution reduces to cost and latency. We rerun it on all test sets to settle this.
Why not just give a small model the true graph or its parents' subgraph? Gendron found that providing the true graph barely helps (Counterfactual Causal Inference in Natural Language with Large Language Models). We include an oracle-graph condition for the base model to confirm that this holds at 8B.
Why not just supervised fine-tuning on gold traces from the generator? This is the most plausible competitor. Fine-tuning on synthetic templates will likely saturate in-distribution. Our bet is that imitation learns the surface form of estimands tied to template phrasing, while equivalence-graded reinforcement learning rewards any correct formula and penalizes plausible-looking wrong ones. If fine-tuning matches us on CLadder, CausalityCheck and CausalWorld-CR, we will report that the verifier adds nothing beyond data. That would itself be a useful negative result.
Why not just answer-only reinforcement learning, as in AutoTriton? With binary verdicts, a rollout with a wrong formula earns full reward about half the time. We will measure this directly: the fraction of rewarded rollouts whose estimand fails the equivalence check, tracked over training for outcome-only GRPO versus staged GRPO.
Proposed Experiments and Decisive Checks
Primary evaluation. CLadder in full, broken down by rung and by semantic regime: commonsensical, anti-commonsensical, and nonsensical.
Transfer evaluations. We probe whether the skill leaves our generator's distribution:
- CausalityCheck, especially Process Judgment and Virtual Causal Reasoning. Reported references are 59.46% Process Judgment for Claude-Sonnet-4.5 and 74.51% Virtual Causal Reasoning for GPT-5.1.
- CausalWorld-CR's counterfactual half, which uses news-derived graphs. No reported accuracy figure was found on Lacuna for CausalWorld-CR, so we establish baselines ourselves.
- Specialization cost on BigBench Hard and GSM8K.
Decisive checks. Four results would confirm the claim:
- On CLadder's anti-commonsensical and nonsensical regimes, the staged model loses no more than a few points relative to the commonsensical regime, while both trained baselines show a larger drop.
- Removing the estimand-equivalence term recovers most of the gap to outcome-only GRPO.
- Gains on CausalityCheck Process Judgment exceed gains on Answerable Judging, which is what improved translation rather than improved guessing predicts.
- Accuracy is reported per rung. If counterfactual questions show no gain, this would locate any remaining failure in abduction, where the formula language may need extension.
References
- CLadder: A Benchmark to Assess Causal Reasoning Capabilities of Language Models
- Counterfactual Causal Inference in Natural Language with Large Language Models
- CausalityCheck: A Framework for Evaluating Causal Reasoning in Large Language Models
- AUTOTRITON: Automatic Triton Programming with Reinforcement Learning in LLMs
- Teaching Small Language Models Reasoning through Counterfactual Distillation
- Towards Faithful Multi-step Reasoning through Fine-Grained Causal-aware Attribution Reasoning Distillation
- Better Think Thrice: Learning to Reason Causally with Double Counterfactual Consistency
- Causal Cartographer: From Mapping to Reasoning Over Counterfactual Worlds
- Specializing Smaller Language Models towards Multi-Step Reasoning
- From Causal Parrots to Causal Prophets? Towards Sound Causal Reasoning with Large Language Models