The Variance of Causal Effect Estimators for Binary V-structures

Jack Kuipers, Giusi Moffa

Abstract

Adjusting for covariates is a well established method to estimate the total causal effect of an exposure variable on an outcome of interest. Depending on the causal structure of the mechanism under study there may be different adjustment sets, equally valid from a theoretical perspective, leading to identical causal effects. However, in practice, with finite data, estimators built on different sets may display different precision. To investigate the extent of this variability we consider the simplest non-trivial non-linear model of a v-structure on three nodes for binary data. We explicitly compute and compare the variance of the two possible different causal estimators. Further, by going beyond leading order asymptotics we show that there are parameter regimes where the set with the asymptotically optimal variance does depend on the edge coefficients, a result which is not captured by the recent leading order developments for general causal models. As a practical consequence, the adjustment set selection needs to account for the relative magnitude of the relationships between variables with respect to the sample size, and cannot rely on purely graphical criteria.

Introduction

In causal inference, identifying the total causal effect of an exposure ($X$) on an outcome ($Y$) often involves adjusting for a set of covariates ($Z$). While graphical criteria like Pearl’s back-door criterion provide sets that are theoretically valid in the limit of infinite data, they do not account for the precision of these estimators in finite samples. Recent literature has suggested that an "optimal" adjustment set—one minimizing variance—can be determined solely from the structure of the Causal Directed Acyclic Graph (DAG). This paper challenges that notion by examining a simple v-structure (Figure 1) and demonstrating that the strength of the relationships (edge coefficients) significantly impacts estimator precision.

A causal v-structure where exposure X and covariate Z both influence outcome Y.

Methodology

The study focuses on a binary v-structure where $X$ and $Z$ are independent causes of $Y$. Two primary estimators for the causal effect of $X$ on $Y$ are compared:

  1. Raw Conditionals ($R$): This estimator ignores $Z$, calculating the effect based solely on the observed relationship between $X$ and $Y$.
  2. Marginalization ($M$): This estimator adjusts for $Z$ by calculating the effect of $X$ within each level of $Z$ and then averaging the results.

To derive exact variance expressions, the authors utilize a mathematical framework involving generating functions and generalized hypergeometric functions. This approach moves beyond "leading-order" asymptotics, which often obscure the influence of sub-leading terms that are critical in finite-sample regimes. The analytical results were validated using Monte Carlo simulations consisting of 40 million repetitions.

Key Insights: The Role of Edge Strength

A central finding is the existence of a "crossover" point. The precision of the estimators depends on the strength of the relationship between the covariate $Z$ and the outcome $Y$, denoted by a parameter $C$.

If the connection $Z \to Y$ is weak (below a specific threshold $C^*$), the "raw" estimator ($R$)—which theoretically "misspecifies" the model by ignoring a parent of $Y$—actually yields a lower variance than the "marginalized" estimator ($M$). Conversely, as the strength of $C$ increases, the marginalized estimator becomes more precise. This relationship is also sensitive to the sample size $N$ and the probability distribution of the exposure $p_X$.

Difference in variance (Δ) between estimators as a function of edge strength C and exposure probability pX for N=100 and N=400.

As shown in Figure 4, the region where the raw estimator is superior (where $\Delta < 0$) shrinks as the sample size $N$ increases, but it does not disappear. This highlights that for any finite $N$, there exists a regime where graphical optimality does not align with statistical precision.

Results and Discussion

The authors provide the first exact variance formulas for these binary estimators. A notable practical takeaway is the connection to the Akaike Information Criterion (AIC). If the edge $Z \to Y$ is sufficiently weak that it would not be selected by AIC, adjusting for $Z$ typically increases the variance of the causal estimate.

This suggests that researchers should not rely solely on graphical rules for adjustment set selection. Instead, they must consider the relative magnitudes of the relationships between variables. While marginalization is asymptotically optimal as $N \to \infty$, the sub-leading terms—which depend on edge coefficients—can dominate in real-world datasets with limited samples.

Conclusion

This work demonstrates that the selection of an optimal adjustment set is not a purely graphical problem. In finite samples, the precision of causal effect estimators is a function of both the graph topology and the underlying parameters. Future research will need to address the mathematical complexity of extending these exact results to larger DAGs and explore the implications for causal structure learning, where the existence of edges is itself uncertain.

Summary updated

Abstract

Adjusting for covariates is a well established method to estimate the total causal effect of an exposure variable on an outcome of interest. Depending on the causal structure of the mechanism under study there may be different adjustment sets, equally valid from a theoretical perspective, leading to identical causal effects. However, in practice, with finite data, estimators built on different sets may display different precision. To investigate the extent of this variability we consider the simplest non-trivial non-linear model of a v-structure on three nodes for binary data. We explicitly compute and compare the variance of the two possible different causal estimators. Further, by going beyond leading order asymptotics we show that there are parameter regimes where the set with the asymptotically optimal variance does depend on the edge coefficients, a result which is not captured by the recent leading order developments for general causal models. As a practical consequence, the adjustment set selection needs to account for the relative magnitude of the relationships between variables with respect to the sample size, and cannot rely on purely graphical criteria.
BibTeX
@article{kuipers2022variance,
  author = {Jack Kuipers and Giusi Moffa},
  title = {{The Variance of Causal Effect Estimators for Binary V-structures}},
  journal = {Journal of Causal Inference},
  year = {2022},
  doi = {10.1515/jci-2021-0025},
}
Figure