Introduction
Learning personalized treatment policies is a cornerstone of modern decision-making in domains like healthcare and public policy. However, practitioners often face a data dilemma: observational data is rich in covariates but plagued by unobserved confounders, while experimental data from Randomized Controlled Trials (RCTs) is unconfounded but often lacks the specific covariates needed for personalization. This paper addresses the challenge of "safe policy learning" under partial identifiability, where the true treatment effect cannot be uniquely determined due to these data limitations.
The Problem of Partial Identifiability
In many real-world scenarios, we rely on historical observational data where the "No Unmeasured Confounding" (NUC) assumption is violated. While RCTs are the gold standard for causal inference, they frequently suffer from partial observability—missing key demographic or biomarker data available in Electronic Health Records (EHRs) that are crucial for personalization. When these data sources are used in isolation, the causal effect of a personalized policy remains unidentifiable. The authors propose a framework to learn policies that are guaranteed to perform at least as well as the current baseline by deriving sharp bounds from these imperfect, heterogeneous data sources.
Methodology: Fusing Data for Robustness
The authors utilize Structural Causal Models (SCMs) to formalize the interaction between observational and experimental data, framing the problem as a robust policy improvement task.
The methodology consists of three primary components:
- Sharp Causal Bounds: The paper proves that while marginal experimental data alone provides no informative lower bounds, its fusion with confounded observational data yields novel, closed-form lower bounds on conditional treatment effects. These bounds are mathematically proven to be "sharp," meaning they are the tightest possible without further assumptions about the data-generating process.
- Maximin Optimization: Given the uncertainty of the exact reward, the authors adopt a pessimistic approach. They optimize for the "worst-case" scenario by maximizing the lower bound of the treatment effect relative to the baseline policy that generated the observational data.
- Safe Policy Learning Algorithm: A three-step procedure is introduced: estimating bounds via supervised learning (e.g., regression), computing worst-case weights for the observed covariates, and performing policy ascent on a differentiable weighted loss function.
Experimental Evaluation
The approach was validated using both synthetic and real-world clinical data to demonstrate the utility of the derived bounds.
- Synthetic Contextual Bandit: In a simulated environment with unobserved confounders and partially observed experimental data, the policy leveraging the full suite of combined bounds consistently outperformed random and baseline policies across various decision thresholds.
- International Stroke Trial (IST): Using data from a real-world trial on Aspirin for ischemic stroke, the authors simulated observational bias by hiding a key confounder (conscious state). The resulting "safe" policies dominated the baseline behavior policy, achieving higher mean survival outcomes while selecting a more targeted fraction of the patient population for treatment.
Conclusion and Future Work
The core insight of this work is that the "sum is greater than the parts": combining biased observational data with incomplete experimental data creates a synergy that tightens causal bounds significantly. By optimizing for the lower bound, the framework provides a mathematical safety net for deployment, ensuring that new policies do no harm relative to the status quo.
However, the method currently relies on parametric assumptions for estimating intermediate distributions. Misspecification of these models could compromise the safety guarantee. Future research should focus on non-parametric estimators and exploring the sample complexity required for robust bound estimation in finite-sample settings.
Summary updated