Abstract
We investigate the problem of bounding causal effects from experimental studies in which treatment assignment is randomized but the subject compliance is imperfect. It is well known that under such conditions, the actual causal effects are not point-identifiable due to uncontrollable unobserved confounding. In their seminal work, Balke and Pearl (1994) derived the tightest bounds over the causal effects in this settings by employing an algebra program to derive analytic expressions. However, Pearl's approach assumes the primary outcome to be discrete and finite. Solving such a program could be intractable when high-dimensional context variables are present. In this paper, we present novel non-parametric methods to bound causal effects on the continuous outcome from studies with imperfect compliance. These bounds could be generalized to settings with a high-dimensional context.
Introduction
Randomized controlled trials (RCTs) are the gold standard for determining treatment efficacy. However, human subjects often exhibit non-compliance, failing to adhere to their assigned treatment protocols. This introduces unobserved confounding bias, meaning the true causal effect of the treatment is no longer point-identifiable from observational data alone.
To address this, researchers utilize Instrumental Variable (IV) models to calculate bounds—minimum and maximum possible values—for the causal effect. While established methods exist for discrete outcomes, they suffer from computational intractability when applied to continuous outcomes or high-dimensional contexts (e.g., a patient's full medical history) due to an exponential blowup in the parameter space. This paper introduces a framework to overcome these bottlenecks.
Methodology
We overcome the intractability of continuous outcomes and high-dimensional contexts through two primary innovations:
- The Restricted IV (RIV) Model: We introduce a non-parametric assumption that potential outcomes (counterfactuals) are mutually independent given the latent counterfactual treatment. This structural assumption reduces the required probability table size from exponential to linear.
- Tractable Linear Programs (LPs): Leveraging the RIV model, we formulate an efficient Linear Program to determine optimal upper and lower bounds for causal effects on continuous outcomes. We also provide robust estimation procedures for finite samples in high-dimensional contexts.
- Causal-UCB (UCB$^c$): These causal bounds are applied to online reinforcement learning, specifically multi-armed and contextual bandits. Standard Upper Confidence Bound (UCB) algorithms explore actions based on statistical uncertainty. Our modified UCB$^c$ algorithms "clip" these statistical confidence intervals using the computed causal bounds. If an action's causal upper bound is lower than another's lower bound, the agent immediately ceases exploring the suboptimal action.
Experimental Evaluation
We validated our approach using real-world data from the International Stroke Trial (IST), examining the effect of aspirin allocation on a continuous recovery likelihood score. To simulate unobserved confounding, we filtered the dataset using hidden inclusion rules and masked certain patient attributes from the learning agent.
Key Results
Theoretical proofs indicate that causally-enhanced bandit algorithms (kl-UCB$^c$, lil'LUCB$^c$, D-UCB$^c$) possess strictly tighter regret and sample complexity bounds than standard versions. Empirical results on the IST dataset include:
- Multi-Armed Bandits: The causal kl-UCB$^c$ achieved a cumulative regret of 0.07, significantly outperforming the standard kl-UCB algorithm's regret of 38.63.
- Contextual Bandits: The causal D-UCB$^c$ achieved a cumulative regret of 0.07, compared to 111.8 for the standard D-UCB.
- Best Arm Identification: The causal lil'LUCB$^c$ algorithm identified the optimal treatment using approximately 4,400 samples, whereas the standard version required 6,500 samples.
Conclusion and Future Work
This research demonstrates that confounded observational data, often discarded in rigorous experimental setups, can generate powerful causal bounds that act as "guardrails" for online learning. By injecting these bounds, agents can definitively rule out suboptimal actions without exhaustive exploration, leading to massive reductions in regret.
While robust, this approach relies on the validity of Instrumental Variable assumptions. If the instrument is flawed, the resulting bounds may be invalid. Future work will explore applying this framework to more complex reinforcement learning settings, such as full Markov Decision Processes (MDPs).
Summary updated