Bounds on the conditional and average treatment effect with unobserved confounding factors

Steve Yadlowsky, Hongseok Namkoong, Sanjay Basu, John C. Duchi, Lü Tian

Abstract

In observational studies, the assumption of unconfoundedness—that all variables affecting both treatment and outcome are observed—is often untenable. This paper introduces a scalable sensitivity analysis framework to estimate bounds on the conditional average treatment effect (CATE) and the average treatment effect (ATE) when unobserved confounders are present. By reformulating the bounding problem as a convex loss minimization task, the authors enable the use of flexible machine learning models. Furthermore, they develop a semi-parametric estimator for ATE bounds that achieves root-$n$ consistency through Neyman orthogonality, providing a robust tool for causal inference in high-dimensional settings.

Introduction

Causal inference from observational data typically relies on the "ignorability" or "unconfoundedness" assumption. However, in practice, hidden factors often influence both treatment assignment and outcomes, leading to biased estimates. Sensitivity analysis seeks to quantify how robust a causal conclusion is to such hidden bias. Traditional methods, such as Rosenbaum’s matching-based sensitivity analysis, often struggle with high-dimensional covariates and do not easily integrate with modern non-parametric estimators.

Yadlowsky et al. address these limitations by proposing a framework based on the $\Gamma$-selection bias model. This model assumes that the odds ratio of treatment assignment between two individuals with the same observed covariates, but different unobserved factors, is bounded by a factor $\Gamma$. The authors provide a scalable approach to estimate the tightest possible bounds on treatment effects under this assumption.

Methodology

CATE Bounds via Loss Minimization

The core innovation for estimating CATE bounds is the transformation of the bounding problem into a convex loss minimization problem. The authors show that the lower and upper bounds of the conditional mean outcome can be estimated by minimizing a reweighted squared loss function. For a given $\Gamma$, the loss function is defined as:

$$\ell_\Gamma(\theta, y) := \frac{1}{2} \left[ (y - \theta)_+^2 + \Gamma (y - \theta)_-^2 \right]$$

This asymmetric loss function upweights residuals to account for the potential systematic bias introduced by unobserved confounding. Because this formulation is convex, it allows practitioners to utilize a wide array of "black-box" machine learning learners—such as Gradient Boosted Trees or Neural Networks—to estimate the bounds, making the method highly scalable to complex data.

Robust ATE Estimation

To estimate the Average Treatment Effect (ATE), the authors extend the Augmented Inverse Propensity Weighted (AIPW) estimator. A significant challenge in using flexible models for sensitivity analysis is that the bias of the underlying learners can propagate into the final causal estimate. To mitigate this, the authors construct a Neyman orthogonal score for the ATE bounds.

This property ensures that the estimator is first-order insensitive to errors in the estimation of nuisance parameters (the propensity score and outcome models). Consequently, the estimator achieves $\sqrt{n}$ convergence and remains statistically valid even when the nuisance parameters are estimated at slower, non-parametric rates ($o_p(n^{-1/4})$).

Empirical Results

Simulation Studies

The methodology was tested across various simulated scenarios with dimensions ranging from $d=1$ to $d=20$. The results confirmed that the 95% confidence intervals for the bounds maintained accurate coverage as sample sizes increased. The use of xgboost as a base learner demonstrated the framework's ability to handle high-dimensional covariate spaces where traditional matching methods fail.

NHANES Case Study

The authors applied their method to data from the National Health and Nutrition Examination Survey (NHANES) to examine the effect of high fish consumption on blood mercury levels. Using a sample of 2,512 adults and 8 key covariates, the proposed method was compared against traditional matching-based sensitivity analysis.

At a sensitivity level of $\Gamma = e^3$, the proposed method yielded a confidence interval of $[0.47, 3.29]$, maintaining a statistically significant positive effect. In contrast, the traditional matching method produced a wider interval of $[-0.24, 4.48]$, which included zero and failed to confirm the effect. This demonstrates that the proposed semi-parametric approach can provide significantly more power and tighter bounds in practical applications.

Conclusion

This work provides a rigorous and scalable bridge between sensitivity analysis and modern machine learning. By framing the search for causal bounds as an optimization problem and utilizing Neyman orthogonality for statistical efficiency, the authors enable robust causal inference in the presence of unobserved confounding. While the method relies on the choice of the sensitivity parameter $\Gamma$ and certain symmetry assumptions for ATE tightness, it represents a substantial advancement in making sensitivity analysis a standard part of the high-dimensional causal inference pipeline.

Summary updated

Abstract

For observational studies, we study the sensitivity of causal inference when treatment assignments may depend on unobserved confounders. We develop a loss minimization approach for estimating bounds on the conditional average treatment effect (CATE) when unobserved confounders have a bounded effect on the odds ratio of treatment selection. Our approach is scalable and allows flexible use of model classes in estimation, including nonparametric and black-box machine learning methods. Based on these bounds for the CATE, we propose a sensitivity analysis for the average treatment effect (ATE). Our semi-parametric estimator extends/bounds the augmented inverse propensity weighted (AIPW) estimator for the ATE under bounded unobserved confounding. By constructing a Neyman orthogonal score, our estimator of the bound for the ATE is a regular root-$n$ estimator so long as the nuisance parameters are estimated at the $o_p(n^{-1/4})$ rate. We complement our methodology with optimality results showing that our proposed bounds are tight in certain cases. We demonstrate our method on simulated and real data examples, and show accurate coverage of our confidence intervals in practical finite sample regimes with rich covariate information.
BibTeX
@article{yadlowsky2022bounds,
  author = {Steve Yadlowsky and Hongseok Namkoong and Sanjay Basu and John C. Duchi and Lü Tian},
  title = {{Bounds on the conditional and average treatment effect with unobserved confounding factors}},
  journal = {The Annals of Statistics},
  year = {2022},
  doi = {10.1214/22-aos2195},
}
Figure