A causal graph often admits more than one valid formula for the same interventional quantity. Suppose we want $P(Y \mid do(X))$, the outcome distribution if we forced the treatment $X$ to a value. One graph might allow a back-door adjustment (averaging over confounders) and also a front-door formula (routing through a mediator). Different orderings of the variables, or different adjustment sets, give still more formulas. With infinite data, all of these formulas return the same number. With finite data, they do not. They disagree, they have different variances, and some amplify noise far more than others.
Current practice ignores this. The identification algorithm returns one arbitrary formula, and an estimator is then built for that formula alone. We propose to do the opposite. We enumerate the equivalent formulas, build a debiased estimator for each, and combine them with weights estimated from the data. The disagreements among them then serve as a test of the assumed graph that needs no counterfactual ground truth.
The same construction addresses two problems at once. The first is the gap between identification (a formula exists) and estimation (the formula can be computed well from a finite sample). The second is the lack of empirical validation: causal estimators are rarely checked against anything except prediction metrics on observed outcomes.
Core Claim
Two papers motivate this work, and both are purely theoretical. Stability of Causal Inference proves that formulas produced by the identification algorithm can amplify input errors with a condition number growing like $\exp(n^{0.49})$ on an adversarial "Gadget" graph. Here the condition number measures how much a small error in the input probabilities is magnified in the output, and $n$ is the number of nodes. Condition number bounds for causal inference adds the "Equivalent Expression Trap": formulas that agree on the true model can have very different condition numbers once the inputs are perturbed. Neither paper runs an experiment, and neither offers a way to choose among formulas.
On the estimation side, Estimating Identifiable Causal Effects through Double Machine Learning (DML-ID) gives the method we reuse. It breaks any identifiable formula into building blocks called modified sequential back-door operators and derives the influence function of each block. An influence function describes how much each data point moves the estimate. With it, DML-ID builds an estimator that is insensitive to first-order errors in the fitted helper ("nuisance") models, the property called Neyman orthogonality, and fits those models on separate data splits, called cross-fitting. DML-ID applies this to the single formula it is handed.
Our observation is that equivalent formulas are only equivalent under the conditional-independence constraints the graph implies. Treated as functionals of an unrestricted distribution, they are different objects with different influence functions and different finite-sample variances. Their differences are therefore informative in two ways.
The first is efficiency. Let $\hat\psi_1,\dots,\hat\psi_K$ be cross-fitted DML-ID estimates from $K$ equivalent formulas, with estimated influence functions $\hat\phi_1,\dots,\hat\phi_K$. Let $\hat\Sigma$ be the empirical covariance matrix of those influence functions and $\mathbf{1}$ a vector of ones. The pooled estimator is
$$\hat\psi_w=\sum_k w_k\hat\psi_k,\qquad w=\frac{\hat\Sigma^{-1}\mathbf{1}}{\mathbf{1}^\top\hat\Sigma^{-1}\mathbf{1}}.$$
This is the minimum-variance unbiased linear combination, by the same argument as generalized method of moments. It can never be asymptotically worse than the best single formula, and it is strictly better when the formulas carry partly independent information.
The second is falsification. Under a correct graph, the vector of pairwise differences $\hat d=(\hat\psi_2-\hat\psi_1,\dots,\hat\psi_K-\hat\psi_1)$ is asymptotically mean-zero. Let $\hat\Sigma_d$ be its covariance, implied by the same influence functions. Then
$$T=n\,\hat d^\top\hat\Sigma_d^{-1}\hat d$$
is approximately chi-squared with $K-1$ degrees of freedom, where $n$ is the sample size. A large $T$ signals that the graph is wrong. Observed-data diagnostics of this kind are exactly what IncomeSCM shows the field lacks. Its "Factual Fallacy" result is that models predicting observed outcomes well (treatment-prediction AUC ≈ 0.98, factual $R^2$ ≈ 0.80) still estimate individual-level effects poorly (CATE $R^2$ ≈ 0.30–0.33).
What is new, then, is the use of the multiplicity of identification formulas as a statistical resource. Prior work treats that multiplicity either as a numerical hazard to be bounded or as irrelevant to estimation. We use it to reduce variance and to test the graph.
Research Program
Enumerating equivalent formulas. The first component is an enumerator of distinct formulas for a given graph and query, restricted to discrete variables as in DML-ID. It generates candidates from three sources: alternative valid adjustment sets; alternative variable orderings and confounded-component factorizations inside the identification recursion; and, where both apply, back-door versus front-door style routes.
The minimal-positivity algorithm IDENTIFY+ from On Positivity Condition for Causal Inference plays a constraining role. It outputs, for each formula, the exact support conditions that formula needs. Formulas whose conditions fail on the observed support are discarded rather than estimated with exploding weights. This is the first empirical use of IDENTIFY+ on finite samples, which that paper leaves open.
Estimation. Each surviving formula is passed through the DML-ID building-block decomposition to obtain a cross-fitted orthogonal estimator and its influence function, using gradient-boosted trees as nuisance models as DML-ID does. All $K$ estimators share the same data folds, so their influence functions can be estimated jointly and $\hat\Sigma$ is consistent.
When $K$ is large relative to $n$, inverting $\hat\Sigma$ becomes noisy. This is the many-moments problem familiar from econometrics. We therefore shrink $\hat\Sigma$ toward its diagonal, with the shrinkage chosen by cross-fitted variance, and report the unshrunk version as an ablation.
Audit. The pipeline also computes, for every formula, the linear-program condition number from Condition number bounds for causal inference. This lets us ask an empirical question neither stability paper answers: does that worst-case number predict which formula has the lowest finite-sample error? Stability of Linear Structural Equation Models found a benign condition number of about $10^{-1}$ on one real linear sociology model, while the Gadget results are catastrophic. Our audit measures where realistic nonlinear, discrete, semi-Markovian graphs fall between those two extremes.
Baselines
Naive reference: DML-ID on the default formula. The reference point is DML-ID applied to whatever single formula the identification algorithm returns. This is current best practice for general graphs DML-ID. It shares our nuisance learner, folds, and tuning budget, so any gain is attributable to pooling. Beating it means lower root mean squared error and narrower confidence intervals with valid coverage, holding the error rate at the nominal 5%. Its predecessor, the plug-in estimator, is also included for context, since DML-ID reports it to be biased. We found no reported scalar error for DML-ID on the Alarm subgraphs on Lacuna; results there are shown only as plots. We will therefore rerun DML-ID ourselves rather than quote a number.
Strongest existing competitor: best single formula selected afterwards. The strongest competitor is an "oracle-ish" selector. It picks the single formula with the smallest cross-fitted influence-function variance, or alternatively the smallest bootstrap variance. This is the obvious simple fix and the closest relative of our method. It isolates whether weighting adds anything beyond selection. We beat it if pooling reduces mean squared error by a meaningful margin on graphs where the formulas are not nearly collinear, without inflating coverage error.
Condition-number selector. This baseline picks the formula with the smallest condition number from Gordon et al.. It is the theory-driven alternative, and it is computed from the same fitted probabilities.
Ablations. The first is uniform averaging of the $K$ estimators, which isolates the value of estimating the weights. The second is pooling without IDENTIFY+ screening, which isolates the positivity step. The third is pooling with unshrunk $\hat\Sigma$.
Graph-test reference. The falsification test needs a point of comparison. The natural one is a direct kernel or regression-based test of the graph's implied conditional independences, run on the same data. Our test probes only the constraints that matter for this particular effect, so it should have more power against graph errors that actually bias the estimate.
Why not just…?
Why not just pick the formula with the lowest estimated variance? This is a real threat. If the equivalent formulas are highly correlated, selection and pooling will perform almost the same, and selection is simpler. The method still needs the same machinery, however: the enumerator, per-formula influence functions, and shared folds. Selection also discards the falsification test.
The selection baseline above settles the question empirically. We will report the correlation structure of $\hat\Sigma$ across graphs, so readers can see when pooling helps. Our expectation is that it helps most for formulas using disjoint variables, such as front-door versus back-door routes. If pooling never beats selection, we will say so, and the contribution narrows to the audit and the test.
Why not just trust the condition number? The condition number is a worst-case, deterministic perturbation bound that ignores sampling variance and the orthogonalization step. Gordon et al. themselves note that they do not integrate their bound with statistical estimation. Our audit tests directly whether ranking formulas by condition number agrees with ranking them by realized error. A weak correlation would be a substantive negative finding about the practical reach of the stability theory.
Why not derive the efficient influence function under the full graph model? In principle that is the right object, and our pooled estimator is a finite-dimensional approximation to it, obtained by projecting onto the span of the formula differences. No general derivation exists for semi-Markovian graphs in the file. Even the equivalence-class extension DML-IDP works per formula and costs polynomially many operations per query. Where the efficient estimator is known, for simple graphs, we will compare against it to measure how much efficiency pooling leaves on the table.
Why not just coordinate the nuisance models, as C-DML does? Coordinated Double Machine Learning reduces finite-sample bias from correlated nuisance errors, but only in single back-door settings. It is complementary to our method, not a substitute: it reduces bias within one formula, while pooling reduces variance across formulas. On the back-door-plus-front-door benchmark we will run both alone and together to check for additivity.
Proposed Experiments
Synthetic graphs with known structure. We start with the two Alarm subgraphs used by DML-ID, the PAG examples from DML-IDP, and small instances of the Schulman–Srivastava Gadget. These have known ground truth and admit several formulas each. They span the stable regime (small confounded components, where Gordon et al.'s bound $n(2c)^{c+1}$ is small) and the adversarial one. We vary $n$ from 500 to 50,000 and report root mean squared error, interval coverage, and width, together with the per-formula condition numbers.
Realistic semi-synthetic benchmark. A realistic test needs both hidden confounding and non-trivial identification. IncomeSCM is a seven-year simulator built from US census income data, but it identifies effects only by back-door adjustment. We extend it by hiding a confounder of the studies-to-income effect and exposing a mediator, so that front-door and alternative adjustment formulas coexist. We then rerun T-learners, S-learners, and DML as the original paper does, so the Factual Fallacy numbers can be replicated before the hidden confounder is introduced.
Falsification under graph errors. We deliberately delete or add one edge in each benchmark graph and measure two things: the power of the test $T$ to detect the error, and the correlation between $T$ and the resulting bias of the default DML-ID estimate. This also gives a first quantitative look at how structural errors propagate into estimation, a problem DML-IDP leaves open.
Real data. On the US crash-fatality database FARS, using the front-door seat-belt setup and code from Debiased Front-Door Learners (FD-CATE), there is no ground truth. There we report whether the available formulas agree, using $T$, at the population level, without claiming accuracy.
Decisive checks. Three outcomes would decide the proposal. First, pooling should beat best-variance selection on mean squared error with valid coverage on the extended IncomeSCM and the front-door-plus-back-door Alarm variant. Second, the condition-number ranking either does or does not predict the realized-error ranking. Third, $T$ should reject at roughly the nominal rate under correct graphs and with high power under edge errors that move the estimate. If the first check fails everywhere, the project reduces to an empirical stability audit plus a graph test, which is still a contribution, but a smaller one.
References
- Stability of Causal Inference
- Condition number bounds for causal inference
- Estimating Identifiable Causal Effects through Double Machine Learning
- IncomeSCM: From tabular data set to time-series simulator and causal estimation benchmark
- On Positivity Condition for Causal Inference
- Stability of Linear Structural Equation Models of Causal Inference
- Estimating Identifiable Causal Effects on Markov Equivalence Class through Double Machine Learning
- Coordinated Double Machine Learning
- Debiased Front-Door Learners for Heterogeneous Effects