Sensitivity Analysis for Causal Inference with Hidden Confounding

This direction is split into other directions: sensitivity parameter selection. Its last version stays readable and citable here.

Version 1 of 157 papers67 conceptsfrozen 2026-10-04

The "gold standard" of causal inference is the assumption of unconfoundedness: the belief that we have measured every variable that influences both the treatment and the outcome. In the real world, this assumption is almost always a fiction. Whether in healthcare, economics, or reinforcement learning, there are nearly always "hidden" factors—a patient’s unrecorded lifestyle choice, a trader’s private intuition, or a robot’s unlogged sensor glitch—that bias our data.

If we cannot guarantee that we have measured everything, how can we trust our causal conclusions? Instead of providing a single, potentially biased "point estimate," researchers are developing frameworks to quantify our ignorance. By moving from single numbers to sharp bounds, this research direction allows us to ask: How much hidden bias would it take to overturn our conclusion?

The Budget of Hidden Bias

The core intuition behind sensitivity analysis is the creation of a "budget" for unobserved confounding. In a standard model, we assume the probability of receiving a treatment is entirely explained by observed features. In a sensitivity model, we relax this.

We introduce a sensitivity parameter—often denoted as $\Gamma$ or $\Lambda$—which represents the maximum factor by which an unobserved variable could change the odds of treatment. For example, if $\Gamma = 2$, we are entertaining a world where two people who look identical in our data might actually have a two-fold difference in their probability of being treated due to something we didn't measure.

As Nathan Kallus and Angela Zhou demonstrate in Confounding-Robust Policy Improvement, this budget allows us to construct an uncertainty set. Instead of one "true" causal effect, we find a range of possible effects. If the entire range stays positive even under a high budget for bias, our conclusion is "robust."

From Loose Bounds to Sharp Efficiency

Early methods for bounding causal effects often produced intervals that were too wide to be useful—what researchers call "conservative" bounds. Recent work has focused on making these bounds sharp, meaning they represent the tightest possible range consistent with the data and the sensitivity budget.

Dennis Frauen, Valentyn Melnychuk, and Stefan Feuerriegel have pushed this frontier in Efficient and Sharp Off-Policy Learning under Unobserved Confounding. They derive closed-form solutions for the worst-case scenarios, avoiding the numerical instability of previous optimization-based approaches. This allows practitioners to calculate exactly how much "ignorance" remains in their estimates with high statistical efficiency.

Robustness in Sequential Decisions

In Reinforcement Learning (RL), unobserved confounding is particularly dangerous because a single biased decision can cascade through time. If we evaluate a new medical policy using historical data, we might mistakenly think a treatment is effective simply because doctors previously gave it to patients they knew (via unrecorded "bedside information") were likely to recover anyway.

To solve this, David Bruns-Smith and Angela Zhou propose frameworks like Robust Fitted-Q-Evaluation. These methods treat the environment as a "Robust MDP," where the transitions and rewards are not fixed but exist within a set of possibilities defined by the sensitivity budget. This ensures that a learned policy is "safe"—it won't perform worse than the status quo even if the historical data was heavily confounded.

Quantifying Ignorance with Conformal Inference

While sensitivity parameters give us a range, they don't always tell us how confident we should be in that range. Mingzhang Yin, Yixin Wang, and David Blei addressed this by introducing Conformal Sensitivity Analysis.

By combining sensitivity models with conformal inference—a technique for generating uncertainty intervals with guaranteed coverage—they provide "distribution-free" intervals for individual treatment effects. This means that for any given patient, we can provide a range of outcomes that is guaranteed to contain the truth (given the bias budget) without making strong assumptions about the underlying noise in the data.

The Calibration Challenge

The most significant hurdle in this field remains calibration. As noted across dozens of papers, including Calibrating confounding strength in sensitivity models by Jean‐Baptiste Baitairian and colleagues, choosing the "right" value for the sensitivity parameter is inherently subjective.

If we set the budget too low, we risk false certainty; if we set it too high, the bounds become so wide (e.g., "the treatment might save you or kill you") that they are useless for decision-making. Current research is shifting toward using negative controls (variables known to have no causal link) or proxy variables to ground these parameters in reality, moving the field from "expert guesses" toward data-driven robustness.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_sensitivity_analysis_for_causal_inferenc_d73ca5f1,
  title = {Sensitivity Analysis for Causal Inference with Hidden Confounding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/sensitivity-analysis-for-causal-inference-with-hidden-confounding/rdr_d73ca5f175e44bdd97e0b8640524cbc2},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_sensitivity_analysis_for_causal_inferenc_d73ca5f1_v1,
  title = {Sensitivity Analysis for Causal Inference with Hidden Confounding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/sensitivity-analysis-for-causal-inference-with-hidden-confounding/rdr_d73ca5f175e44bdd97e0b8640524cbc2/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}