Bounds on the conditional and average treatment effect with unobserved confounding factors

Steve Yadlowsky, Hongseok Namkoong, Sanjay Basu, John C. Duchi, Lü Tian

TL;DR

This paper tackles the challenge of estimating treatment effects in observational studies when unobserved factors might influence both treatment assignment and outcomes. The authors introduce a novel method to derive bounds on these effects, rather than point estimates, by assuming a bounded influence of these unobserved confounders. This approach is flexible, scalable, and provides theoretically sound confidence intervals, even with complex machine learning models.

What Problem Does This Paper Address?

In causal inference, a core goal is to understand the effect of an intervention (like a drug or a policy) on an outcome. Ideally, we'd use randomized controlled trials (RCTs) where treatment is assigned randomly, ensuring that any observed differences in outcomes are due to the treatment itself. However, RCTs are often infeasible or unethical, forcing us to rely on observational data.

The main hurdle with observational data is confounding. This happens when unmeasured factors (confounders) influence both who receives the treatment and the outcome. For example, if sicker people are more likely to receive a new treatment and also have worse outcomes, it's hard to tell if the treatment is ineffective or if it's just the underlying sickness. The standard assumption to overcome this is that all relevant confounders are observed. However, this is often too strong a requirement in practice.

This paper addresses the scenario where there might be unobserved confounders. Instead of trying to estimate a precise treatment effect (which is impossible under unobserved confounding), the authors aim to provide bounds on the possible range of the treatment effect. They do this by assuming that the influence of these unobserved confounders on treatment selection is bounded in a specific way, inspired by Rosenbaum's work on sensitivity analysis. This allows them to quantify how much the unobserved confounders could be affecting the results.

Methods

The core of the paper's methodology is a loss minimization approach to estimate bounds on the conditional average treatment effect (CATE) and the average treatment effect (ATE).

  1. Bounding the CATE: The authors propose a novel loss function that penalizes negative residuals more heavily. Minimizing this loss function allows them to estimate a lower bound for the potential outcome under treatment, even in the presence of unobserved confounders with bounded influence. This approach is flexible and can incorporate various machine learning models (like random forests or gradient boosted trees) for estimation.
  2. Bounding the ATE: Building on the CATE bounds, they develop a semiparametric estimator for the ATE. This estimator is inspired by the well-known Augmented Inverse Propensity Weighted (AIPW) estimator but is adapted to handle unobserved confounding. A key technical innovation is the construction of a Neyman orthogonal score, which ensures that their estimator is root-n consistent (meaning it converges to the true value at a predictable rate) as long as the nuisance parameters (like propensity scores and outcome models) are estimated at a sufficiently fast rate.
  3. Sensitivity Analysis: The bounds themselves provide a form of sensitivity analysis. By varying the assumed bound on the unobserved confounders' influence (represented by the parameter $\Gamma$), they can see how much the estimated treatment effect range changes.
  4. Optimality Results: The paper also includes theoretical results demonstrating that their bounds are "tight" (meaning they are the best possible under the given assumptions) in certain specific scenarios.

Datasets

  • Simulated Data: The authors generate data from various distributions to test their method's performance under controlled conditions, varying sample size, covariate dimension, and the strength of unobserved confounding.
  • Real Observational Data: The method is applied to a dataset on fish consumption and blood mercury levels from the National Health and Nutrition Examination Survey (NHANES). This dataset is used to infer the effect of high fish consumption on blood mercury concentration.

Key Insights and Ideas

  • Bounds, Not Point Estimates: The fundamental insight is that when unobserved confounders are present, precise point estimates of treatment effects are impossible. The paper shifts the focus to providing bounds that quantify the uncertainty introduced by unobserved confounding.
  • Bounded Influence is Key: The assumption that unobserved confounders have a bounded influence on treatment selection is crucial. This assumption allows for a principled way to derive these bounds and provides a framework for sensitivity analysis.
  • Loss Minimization for Bounds: Using a specially designed loss function (that penalizes negative residuals more) is an elegant way to derive bounds on potential outcomes. This approach is flexible and can leverage powerful machine learning estimators.
  • Neyman Orthogonality for Robustness: The construction of a Neyman orthogonal score is a significant technical contribution, ensuring that the estimator for the ATE is robust to estimation errors in the nuisance parameters, a common challenge in semiparametric estimation.
  • Scalability with ML: The method's ability to incorporate black-box machine learning models makes it applicable to high-dimensional data, a common scenario in modern observational studies.

Key Results

  • Valid Confidence Intervals: The proposed method provides confidence intervals for the ATE that demonstrate accurate coverage in simulations, even with unobserved confounding.
  • Tighter Bounds: In real-data applications and simulations, the proposed method often produces tighter confidence intervals compared to existing matching-based sensitivity analysis methods, especially when the assumed level of confounding ($\Gamma$) is appropriately chosen.
  • Scalability Demonstrated: The use of machine learning (xgboost) for estimating nuisance parameters shows that the method can scale to high-dimensional datasets.
  • Tightness in Specific Cases: Theoretical results show that the derived bounds are optimal (tight) under certain symmetry conditions on the potential outcomes.

Limitations and Future Work

  • Choice of $\Gamma$: The practical application of the method requires choosing a value for $\Gamma$, which represents the assumed bound on unobserved confounding. The paper acknowledges that this choice can be challenging and suggests that providing results for a range of $\Gamma$ values can help contextualize the findings. Future work could explore methods for selecting or learning $\Gamma$.
  • Optimality: While the bounds are tight in specific symmetric cases, the paper notes that in general, optimal estimation might require considering the joint distribution of potential outcomes, which complicates the estimation procedure. This is identified as an important direction for future research.
  • Computational Cost: While the method is scalable, the nested cross-fitting procedure can be computationally intensive. The paper explores iterative cross-fitting to reduce this cost.
  • Assumptions: The method relies on the F-selection bias condition and other regularity assumptions. The sensitivity of the results to violations of these assumptions could be further explored.

Summary updated

Abstract

For observational studies, we study the sensitivity of causal inference when treatment assignments may depend on unobserved confounders. We develop a loss minimization approach for estimating bounds on the conditional average treatment effect (CATE) when unobserved confounders have a bounded effect on the odds ratio of treatment selection. Our approach is scalable and allows flexible use of model classes in estimation, including nonparametric and black-box machine learning methods. Based on these bounds for the CATE, we propose a sensitivity analysis for the average treatment effect (ATE). Our semiparametric estimator extends/bounds the augmented inverse propensity weighted (AIPW) estimator for the ATE under bounded unobserved confounding. By constructing a Neyman orthogonal score, our estimator of the bound for the ATE is a regular root-<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>n</mml:mi></mml:math> estimator so long as the nuisance parameters are estimated at the <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:msub><mml:mrow><mml:mi>o</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow></mml:msub><mml:mfenced><mml:mrow><mml:msup><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo><mml:mn>1</mml:mn><mml:mo>/</mml:mo><mml:mn>4</mml:mn></mml:mrow></mml:msup></mml:mrow></mml:mfenced></mml:math> rate. We complement our methodology with optimality results showing that our proposed bounds are tight in certain cases. We demonstrate our method on simulated and real data examples, and show accurate coverage of our confidence intervals in practical finite sample regimes with rich covariate information.
BibTeX
@article{yadlowsky2022bounds,
  author = {Steve Yadlowsky and Hongseok Namkoong and Sanjay Basu and John C. Duchi and Lü Tian},
  title = {{Bounds on the conditional and average treatment effect with unobserved confounding factors}},
  journal = {The Annals of Statistics},
  year = {2022},
  doi = {10.1214/22-aos2195},
}
Figure