Towards a Holistic Understanding of Selection Bias for Causal Effect Identification

Yiwen Qiu, Filip Kovačević, Shimeng Huang, Peter Spirtes, Francesco Locatello

Introduction

Selection bias is a pervasive challenge in observational studies. In an ideal scenario, collected data perfectly mirrors the target population. In reality, researchers frequently encounter biased sub-samples, such as the "healthy volunteer bias" seen in large-scale biobanks where respondents are typically healthier and of higher socio-economic status than the general public. If researchers calculate the Average Treatment Effect (ATE) strictly on this sub-population, the resulting estimate for the broader population will be severely biased.

The fundamental challenge in causal inference under these conditions is identifiability: determining whether it is mathematically possible to recover the true causal effect from a biased sample. Existing solutions heavily rely on Causal Directed Acyclic Graphs (DAGs), which require precise knowledge of where the selection mechanism occurs within the causal system (e.g., whether income or health status directly causes survey participation). Pinpointing these mechanisms in practice is notoriously difficult. This paper introduces a more holistic framework that shifts the focus from rigid graphical models to flexible distributional assumptions, establishing necessary and sufficient conditions to identify causal effects in scenarios previously deemed unidentifiable.

Methodology

The authors propose a unified framework centered on a mathematical criterion—termed Condition 1—that dictates whether an ATE is identifiable under selection bias.

Key technical contributions of this approach include:

  • Distributional over Graphical Assumptions: Rather than relying exclusively on the structure of a DAG, the framework leverages "probability classes." The authors demonstrate that if the underlying data distributions are sufficiently "smooth" (e.g., Gaussian, Laplace, or Pareto distributions), researchers can use the mathematics of truncated statistics to extrapolate from the biased sample back to the full population.
  • Agnostic Selection Mechanisms: Unlike prior graphical methods, this framework is agnostic to where the selection occurs in the causal chain. It successfully handles both deterministic selection (where specific groups are entirely excluded) and non-deterministic selection (where certain groups are simply less likely to be included).
  • Novel Estimators: To operationalize these insights, the paper introduces two practical estimation methods:
  • MLE+$\beta$: A Maximum Likelihood Estimation approach that incorporates a regularized selection function ($\beta$) to account for the bias.
  • SM+$\beta$: A Score Matching approach that relies on the gradients of the data density. This method is often more robust because it bypasses the need to compute intractable normalization constants.

Experimental Validation

The researchers validated their framework using two primary data sources:

  • Synthetic Data: Datasets were generated with various noise profiles (Gaussian, Laplace, Pareto, Log-normal) and functional forms (additive and multiplicative) to rigorously test the boundaries of Condition 1.
  • Semi-synthetic "All of Us" (AoS) Dataset: To simulate real-world complexity, the authors utilized medical data from the NIH’s All of Us Research Program. They combined real covariates (such as BMI) and exposure data (Type 2 Diabetes) with a synthetic outcome and a non-deterministic selection mechanism. This provided a realistic testing ground where the "ground truth" ATE was still known for evaluation.

Key Insights and Results

The core insight of this work is the power of extrapolation. By assuming the data follows a specific family of distributions (such as the exponential family), selection bias can be treated as a form of statistical truncation. Much like reconstructing a full circle from a sufficiently large arc, a full population distribution can be reconstructed from a biased slice, provided the underlying distribution curves are smooth.

Crucially, the authors prove that their proposed condition is both necessary and sufficient. It represents the mathematical boundary for ATE recovery; if Condition 1 is not met, identifying the ATE is impossible without additional information.

In empirical evaluations, the proposed corrected estimators (MLE+$\beta$ and SM+$\beta$) consistently outperformed standard baselines:

  • Superiority over IPW: Standard Inverse Propensity Weighting (IPW) yielded significantly biased results because it fails to account for the unobserved selection mechanism.
  • Robustness to Complexity: In scenarios involving non-polynomial functions (e.g., sine or logarithmic functions), traditional polynomial regression failed, whereas the proposed methods maintained their robustness.
  • Real-World Efficacy: On the All of Us dataset, the new estimators drastically reduced bias compared to naive approaches, even in highly challenging "weak overlap" scenarios where underlying propensity scores were very low.

Limitations and Future Directions

While the distributional assumptions required by this framework are strictly weaker than the graphical assumptions of previous works, they remain assumptions. The framework relies on the data belonging to certain distribution classes to enable extrapolation. If the true population distribution is highly irregular or unpredictable, this extrapolation may fail.

The authors outline several promising directions for future research:

  1. Relaxing Assumptions: Developing methods to identify ATEs with even fewer restrictions on the underlying distribution shapes.
  2. Causal Discovery: Extending the framework beyond effect estimation to aid in discovering the underlying causal graph itself under biased conditions.
  3. Experimental Design: Applying these theoretical insights to "active sampling"—optimizing which data points to collect next to efficiently mitigate selection bias under a limited budget.

Summary updated

Abstract

Selection bias is pervasive in observational studies. For example, large scale biobanks data can exhibit ``healthy volunteer bias'' when respondents are healthier and of higher socio-economic status than the population they are meant to represent. Recovering causal effects from such sub-population is an important problem in causal inference, as estimating average treatment effects (ATE) from selected populations can result in a severely biased estimate of the ATE from the whole population. In this paper, we investigate the identifiability of the ATE under selection bias. We provide *necessary and sufficient conditions* for ATE identifiability, leveraging weak assumptions on probability classes to characterize propensity score and selection probability, which are weaker assumptions than those required by existing general identification frameworks. Compared to previous works, our results extend existing graphical identifiability criteria and offer a more comprehensive understanding of causal effect identification *with strictly weaker conditions* in the presence of selection bias.
BibTeX
@inproceedings{qiu2026towards,
  author = {Yiwen Qiu and Filip Kovačević and Shimeng Huang and Peter Spirtes and Francesco Locatello},
  title = {{Towards a Holistic Understanding of Selection Bias for Causal Effect Identification}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=39650lCGDP},
}
Figure