A common-cause principle for eliminating selection bias in causal estimands through covariate adjustment

Maya B. Mathur, Tyler J. VanderWeele, Ilya Shpitser2023 · DOI

This paper, published in the Annals of Statistics, provides a theoretical breakthrough for a common headache in data science: selection bias. By establishing a "common-cause principle" for selection, the authors show that we can often recover valid treatment effects from non-representative samples simply by adjusting for the right variables—even when data is "Missing Not At Random" (MNAR).

TL;DR

The authors propose a "common-cause principle" for selection bias: if a valid adjustment set exists, then adjusting for the common causes of selection and the outcome (excluding the exposure and its descendants) is sufficient to eliminate bias. Crucially, this allows for the identification of conditional treatment effects using only data from the selected sample, often outperforming complex imputation methods.

What Problem Does This Paper Address?

In an ideal world, we calculate the Average Treatment Effect (ATE) using a representative sample of the population. In reality, we often deal with "selected" samples—think of clinical trial dropouts, survey non-responders, or case-control studies. This leads to selection bias.

Standard fixes like Multiple Imputation (MI) usually rely on the "Missing At Random" (MAR) assumption, which posits that the reason data is missing only depends on observed variables. However, if selection depends on the unobserved value of the variable itself (Missing Not At Random, or MNAR), these methods often fail. Furthermore, many existing graphical methods for selection bias require the researcher to know the entire causal structure of the system, which is rarely feasible in practice.

Methods

The researchers bridge the gap between complex causal graphs and practical data analysis by introducing several key concepts:

  • The Common-Cause Principle for Selection: They prove that if any set of variables can eliminate selection bias, then the set $Z$—consisting of all common causes of the selection indicator ($R$) and the outcome ($Y$), excluding the exposure ($A$) and its descendants—is also a sufficient adjustment set.
  • Focus on CATEs: The paper highlights that while the marginal ATE (the effect on the whole population) often requires knowing covariate distributions for the entire population, the Conditional Average Treatment Effect (CATE) can be identified using only the data available in the selected sample.
  • Graphical "Insufficiency" Criteria: They provide two simple rules to identify when selection bias cannot be fixed via covariate adjustment:
  • Type-D Insufficiency: A direct causal path exists between the outcome and selection (e.g., people with worse outcomes are more likely to drop out).
  • Type-M Insufficiency: Selection is affected by a variable that mediates the relationship between the treatment and the outcome.

Datasets

The paper is primarily theoretical, but the authors validate their findings using synthetic datasets generated from six distinct Directed Acyclic Graphs (DAGs). These DAGs represent various missingness mechanisms, including MAR and MNAR scenarios, and structures where a sufficient adjustment set either does or does not exist.

Key Insights and Ideas

  • Adjusting the Sample is Enough: You don't necessarily need to know what the "missing" people look like to calculate a valid treatment effect for specific subgroups. If you measure the common causes of selection and outcome within your sample, you can block the bias.
  • CCA is More Powerful Than We Thought: Complete-Case Analysis (CCA)—often dismissed as a "naive" approach—can actually be more robust than Multiple Imputation when the data is MNAR, provided the researcher adjusts for the common-cause set $Z$.
  • The "Forbidden" Nodes: The principle mirrors the rules for confounding: never adjust for descendants of the exposure or the outcome. Doing so can inadvertently open "collider" paths that introduce new biases.

Key Results

Through extensive simulations, the authors found:

  • Bias Elimination: In scenarios where the common-cause set $Z$ was sufficient, adjusted CCA was unbiased, whereas unadjusted CCA showed significant negative bias.
  • Outperforming Imputation: Adjusted CCA frequently outperformed Multiple Imputation in MNAR settings.
  • Efficiency in Small Samples: Even in MAR settings, if the sample size was small ($N \le 500$), adjusted CCA often had a lower Root Mean Squared Error (RMSE) and better confidence interval coverage than Multiple Imputation, which can be sensitive to model misspecification in the imputation step.

Limitations and Future Work

  • CATE vs. ATE: The primary results apply to conditional effects. To translate these back to a population-wide marginal ATE, researchers still need to know the distribution of those covariates in the target population (unless the statistical model is "collapsible," like linear regression).
  • Identifying the Set: The method assumes the researcher can correctly identify all common causes of $R$ and $Y$. If a major common cause remains unmeasured, residual selection bias will persist.
  • Future Directions: The authors suggest further research into how these sufficiency results can be integrated with sensitivity analyses to account for potentially unmeasured common causes of selection.

Summary updated

Abstract

Average treatment effects (ATEs) may be subject to selection bias when they are estimated among only a non-representative subset of the target population. Selection bias can sometimes be eliminated by conditioning on a “sufficient adjustment set” of covariates, even for some forms of missingness not at random (MNAR). Without requiring full specification of the causal structure, we consider sufficient adjustment sets to allow nonparametric identification of conditional ATEs in the target population. Covariates in the sufficient set may be collected among only the selected sample. We establish that if a sufficient set exists, then the set consisting of common causes of the outcome and selection, excluding the exposure and its descendants, also suffices. We establish simple graphical criteria for when a sufficient set will not exist, which could help indicate whether this is plausible for a given study. Simulations considering selection due to missing data indicated that sufficiently-adjusted complete-case analysis (CCA) can considerably outperform multiple imputation under MNAR and, if the sample size is not large, sometimes even under missingness at random. Analogous to the common-cause principle for confounding, these sufficiency results clarify when and how selection bias can be eliminated through covariate adjustment.
BibTeX
@misc{mathur2023common,
  author = {Maya B. Mathur and Tyler J. VanderWeele and Ilya Shpitser},
  title = {{A common-cause principle for eliminating selection bias in causal estimands through covariate adjustment}},
  year = {2023},
  doi = {10.31219/osf.io/ths4e},
}
Figure