Bounding causal effects with leaky instruments

David Watson, Gecia Bravo-Hermsdorff, Lee M. Gunderson, Jordan Penn, Afsaneh Mastouri, Ricardo Silva

Abstract

Traditional causal inference relies on "perfect" instruments that only affect outcomes through a specific treatment—an assumption that often fails in the real world. This paper introduces a framework for "leaky" instruments, providing provably sharp bounds on causal effects even when instruments have direct, unintended impacts on the outcome.

Introduction: The Problem of Leaky Instruments

Estimating the causal effect of a treatment ($X$) on an outcome ($Y$) is notoriously difficult when unobserved confounders ($U$) influence both. The gold standard for solving this in observational studies is the Instrumental Variable (IV) method.

However, the IV method rests on a fragile pillar called the exclusion criterion: the instrument ($Z$) must have zero direct effect on the outcome; its influence must be entirely mediated by the treatment. In fields like genetics (Mendelian randomization), this is rarely true due to "horizontal pleiotropy," where a gene affects multiple traits independently. When this assumption is violated, standard estimators like Two-Stage Least Squares (2SLS) produce wildly biased results.

The structural model for leaky instruments, where $\gamma$ represents the direct effect (leakage) of the instrument $Z$ on outcome $Y$.

The authors address this by moving away from the hunt for a single "point estimate" and instead focusing on partial identification—calculating a range (bounds) within which the true effect must lie, given a limited amount of "leakage."

Methodology

The researchers focus on Linear Structural Equation Models (SEMs) and introduce the concept of $\tau$-exclusion. Instead of assuming the direct effect of the instrument on the outcome is zero, they allow it to be bounded by a threshold, $\tau$.

Key technical contributions include:

  • Convex Optimization: They derive a convex objective function to find the minimum and maximum possible Average Treatment Effect (ATE) consistent with the data and the leakage threshold.
  • Closed-Form Solutions: For the $L_2$ norm, they provide an elegant closed-form solution for these bounds, making the method computationally efficient.
  • Falsification Test: They propose a Monte Carlo test based on "tetrad constraints" to determine if the classical exclusion criterion is actually violated.
  • Inference via Bootstrapping: To handle the uncertainty of finite data, they implement a bootstrapping procedure that provides confidence intervals for the estimated bounds.
Visualization of the partial identification strategy. The true treatment effect $\theta^*$ is bounded by the leakage threshold $\tau$.

Experimental Evaluation

The authors evaluated their method using extensive simulated datasets to benchmark against existing techniques. They generated 684 unique simulation configurations, varying the number of instruments, signal-to-noise ratios, and levels of confounding. To facilitate adoption, they released an accompanying R package, leakyIV, available on CRAN.

Key Insights

  • Sharpness: The bounds provided are "sharp," meaning they are the tightest possible intervals that can be derived without making further structural assumptions.
  • Dense vs. Sparse Leakage: Most existing "robust" IV methods assume that a subset of instruments is perfectly valid. This paper allows for a "dense" setting where all instruments might be slightly leaky, a more realistic assumption in many domains.
  • The Geometry of Confounding: The paper demonstrates that the causal effect is a bijective function of the unobserved confounding correlation ($\rho$). By mapping the problem into $\rho$-space, they turn a complex high-dimensional optimization into a manageable one-dimensional search.

Results

  • High Reliability: In simulations, the LeakyIV bounds contained the true causal effect in 99.85% of trials.
  • Superior to Baselines: While 2SLS and backdoor adjustments were consistently biased, the leaky IV bounds remained informative and captured the true direction of the effect in over half of the trials, even with weak signals.
  • Robustness: Unlike competing methods like sisVIVE or MBE, which showed erratic behavior under extreme confounding, the proposed bounds remained stable and valid.

Limitations and Future Work

The current framework is strictly for linear models. While linear SEMs are common, many real-world systems involve non-linear dependencies. Additionally, the method requires the user to specify the leakage threshold ($\tau$), making it ideal for sensitivity analysis. Future work aims to extend this to heterogeneous treatment effects, where the impact of $X$ on $Y$ varies based on individual characteristics.

Summary updated

Abstract

Instrumental variables (IVs) are a popular and powerful tool for estimating causal effects in the presence of unobserved confounding. However, classical approaches rely on strong assumptions such as the $\textit{exclusion criterion}$, which states that instrumental effects must be entirely mediated by treatments. This assumption often fails in practice. When IV methods are improperly applied to data that do not meet the exclusion criterion, estimated causal effects may be badly biased. In this work, we propose a novel solution that provides $\textit{partial}$ identification in linear systems given a set of $\textit{leaky instruments}$, which are allowed to violate the exclusion criterion to some limited degree. We derive a convex optimization objective that provides provably sharp bounds on the average treatment effect under some common forms of information leakage, and implement inference procedures to quantify the uncertainty of resulting estimates. We demonstrate our method in a set of experiments with simulated data, where it performs favorably against the state of the art. An accompanying $\texttt{R}$ package, $\texttt{leakyIV}$, is available from $\texttt{CRAN}$.
BibTeX
@inproceedings{watson2024bounding,
  author = {David Watson and Gecia Bravo-Hermsdorff and Lee M. Gunderson and Jordan Penn and Afsaneh Mastouri and Ricardo Silva},
  title = {{Bounding causal effects with leaky instruments}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=OaJLMx2nwS},
}
Figure