On a Class of Bias-Amplifying Variables that Endanger Effect Estimates

Judea PearlUAI 2010

TL;DR

This paper identifies a class of variables, including instrumental variables (IVs), that can paradoxically increase confounding bias in causal effect estimation, even when used to adjust for it. The authors provide a theoretical framework and intuitive explanation for this phenomenon, demonstrating its implications for statistical modeling and covariate selection.

What Problem Does This Paper Address?

In causal inference, a primary goal is to estimate the true effect of a treatment or exposure (X) on an outcome (Y), free from the influence of confounding variables (U) that affect both X and Y. A common strategy is to adjust for a set of observed covariates that satisfy the "back-door criterion," ensuring they block all backdoor paths between X and Y. However, it's known that adjusting for certain variables, like colliders or intermediate variables, can introduce or amplify bias.

This paper tackles a more subtle problem: a specific class of variables, notably instrumental variables (IVs), that, while seemingly well-behaved, can exacerbate existing confounding bias. This is counterintuitive because IVs are typically used to reduce bias by isolating variation in X that is exogenous. The paper aims to explain why and when this bias amplification occurs, particularly in both linear and non-linear models, and how it contrasts with selection bias.

Methods

The authors employ a combination of theoretical derivation and intuitive explanation.

  • Structural Causal Models: The core of the analysis relies on linear structural causal models (SCMs) to represent the relationships between variables (treatment X, outcome Y, confounder U, instrumental variable Z). These models allow for precise mathematical formulation of causal pathways.
  • Mathematical Derivation: The paper derives expressions for the bias in effect estimates under different conditioning scenarios (unadjusted, adjusted for Z). This involves calculating conditional expectations and derivatives of these expectations with respect to the treatment variable.
  • Intuitive Explanation: Beyond the math, the authors provide intuitive explanations using simplified examples and thought experiments to illustrate how conditioning on certain variables can distort the estimation of the causal effect.
  • Extension to Non-linear Models: The analysis is extended from linear to non-linear models to explore whether the bias-amplifying phenomenon persists and how it might differ.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • The Paradox of Instrumental Variables: The most striking insight is that instrumental variables, often seen as a tool to reduce bias, can actually amplify confounding bias if not carefully considered. This happens when the IV has a strong association with the treatment but also indirectly influences the outcome through a confounder.
  • Bias Amplification Factor: The paper quantifies this amplification, showing that conditioning on an IV can increase bias by a factor related to the strength of the IV's association with the confounder.
  • Non-linear Models Complicate Things: While the bias amplification effect persists in non-linear models, it becomes more nuanced. Conditioning on an IV can sometimes attenuate bias, and in some cases, it can even introduce new bias where none existed before.
  • Distinction from Selection Bias: The paper clearly distinguishes this bias amplification from selection bias. While confounding bias is sensitive to conditioning on IVs, pure selection bias (bias arising from non-random sampling) is generally insensitive to IVs. This distinction offers a potential diagnostic tool.
  • Implications for Covariate Selection: The findings challenge the common practice of selecting covariates for propensity score models solely based on their predictive power for the treatment. Instead, the paper advocates for prioritizing covariates that are direct causes of the outcome, as they are less likely to amplify confounding bias.

Key Results

  • In linear models, conditioning on an instrumental variable (Z) amplifies confounding bias (Bo) by a factor of $1/(1-c_3)$, where $c_3$ is the strength of the association between Z and the confounder U. This means bias is always amplified if $c_3 > 0$.
  • In non-linear models, conditioning on Z can either amplify or attenuate bias, and can even introduce new bias where none existed.
  • Conditioning on an instrumental variable has no effect on selection bias, provided the selection mechanism does not involve confounding.

Limitations and Future Work

  • Linearity Assumption: While the paper extends to non-linear models, the primary theoretical derivations are based on linear structural models. The behavior in more complex non-linear systems might require further investigation.
  • "Pure" Selection Bias: The analysis of selection bias assumes it is "pure," meaning it doesn't involve confounding. In reality, selection mechanisms can be intertwined with confounding, which the paper briefly touches upon in Figure 4.
  • Diagnostic Tool Potential: The paper suggests using IV-sensitivity as a diagnostic tool to distinguish between confounding and selection bias. Further research could explore the practical implementation and robustness of this diagnostic.
  • Practical Guidance: The authors strongly advocate for a shift in covariate selection strategy, emphasizing outcome-oriented predictors. Further empirical studies could validate this recommendation across various domains.

Summary updated

BibTeX
@inproceedings{pearl2010class,
  author = {Judea Pearl},
  title = {{On a Class of Bias-Amplifying Variables that Endanger Effect Estimates}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2010},
}
Figure