A Clinician’s Tool for Analyzing Non-Compliance

AAAI 1996

TL;DR

This paper introduces a Bayesian approach using graphical models and Gibbs sampling to estimate the true efficacy of treatments in clinical trials, even when subjects don't fully comply with their assigned treatment. It provides clinicians with an "assumption-free" tool to assess average treatment effects, moving beyond traditional methods that can be misleading due to non-compliance.

What Problem Does This Paper Address?

Clinical trials often rely on randomized assignment to treatment groups to assess a drug's effectiveness. However, a significant challenge arises when subjects don't adhere to their assigned treatment (e.g., stop taking a drug due to side effects, or obtain it illicitly if in the control group). This "non-compliance" can severely bias the results. Simply comparing outcomes between groups (intent-to-treat analysis) can be misleading because it measures the effect of assignment, not the drug itself. Existing methods to correct for this, like using instrumental variables, have limitations, often requiring strong assumptions about subject behavior or only providing bounds on the effect, especially for smaller sample sizes. The core problem is how to accurately estimate the average treatment effect – what would happen if everyone uniformly received the treatment – when compliance is imperfect and data is limited.

Methods

The paper proposes a system that leverages Bayesian learning and Gibbs sampling within a causal graphical model framework.

  • Graphical Model: They represent the clinical trial using a directed acyclic graph (DAG) that explicitly models treatment assignment (Z), treatment received (D), and observed outcome (Y), along with latent factors (U) influencing compliance and response. A key innovation is the use of a "potential-response" model, which replaces the latent U with a 16-state variable (CR) representing all possible combinations of compliance and response behaviors. This allows for a more efficient representation.
  • Bayesian Inference: The system takes as input prior probabilities (from the clinician) about subject compliance and response behaviors, and the observed experimental data. It then uses Bayesian updating to compute the posterior distribution of the average treatment effect (ACE(D→Y)).
  • Gibbs Sampling: Because directly computing the posterior distribution is intractable, they employ Gibbs sampling. This Markov Chain Monte Carlo (MCMC) method iteratively samples from conditional distributions to approximate the desired posterior distribution. This allows for estimation even when compliance behaviors are unobserved.

Datasets

The paper evaluates their system on:

  • Simulated Data: Datasets generated from known distributions, including one where the causal effect is "identifiable" (meaning it can be precisely determined even with imperfect compliance under large-sample assumptions).
  • Real Data:
    • Lipid Research Clinics Coronary Primary Prevention trial data (cholestyramine for cholesterol reduction).
  • Vitamin A supplementation study data (impact on childhood mortality).

The authors note that their system takes observed data (counts of outcomes for different combinations of Z, D, Y) and a Dirichlet prior over the unknown fractions of compliance/response behaviors as input.

Key Insights and Ideas

  • Non-compliance is a fundamental problem: The paper clearly articulates why non-compliance makes treatment efficacy "non-identifiable" without additional assumptions or data.
  • Graphical models as a unifying framework: They demonstrate how graphical models can elegantly represent the causal assumptions of a clinical trial and facilitate the integration of prior knowledge with observed data.
  • Potential-response variables simplify complex latent structures: The introduction of the 16-state CR variable is a clever way to handle unobserved compliance and response behaviors, making the model computationally tractable.
  • Bayesian approach for uncertainty: The system outputs a distribution for the treatment effect, not just a point estimate. This is crucial for conveying the uncertainty inherent in clinical trial data, especially with non-compliance.
  • Counterfactual queries for individuals: Beyond average effects, the system can be adapted to answer "what if" questions about specific individuals, offering deeper clinical insights.

Key Results

  • The system successfully estimates the posterior distribution of the average treatment effect (ACE(D→Y)) for both simulated and real-world datasets.
  • For an identifiable causal effect scenario with simulated data, the posterior distributions (histograms) become increasingly concentrated around the true value as the sample size increases, regardless of whether a flat or skewed prior was used.
  • When applied to real data (cholestyramine and Vitamin A studies), the posterior distributions were found to be highly concentrated, even with relatively small sample sizes (e.g., 337 cases for the Lipid data), and often fell within the large-sample bounds derived by previous work.
  • The choice of prior distribution can influence the posterior, particularly when sample sizes are smaller, highlighting the importance of sensitivity analysis.
  • In a counterfactual query example, the system provided strong support for the conclusion that an individual would have benefited from treatment, despite non-compliance in the study.

Limitations and Future Work

  • Gibbs Sampling Art: The authors acknowledge that tuning Gibbs samplers for optimal results is more of an "art than a science," involving choices about burn-in periods, convergence diagnostics, and handling dependent samples.
  • Prior Sensitivity: While the system can handle priors, the choice of prior can significantly impact the posterior, especially with limited data. This necessitates sensitivity analysis.
  • Assumption Transparency: The authors use the term "assumption-transparent" to describe their approach, but acknowledge that the model still relies on fundamental assumptions (randomized assignment, no side effects) which are visually represented in the graph.
  • Future Work: The paper mentions modifying the system to answer counterfactual queries about specific individuals, which is demonstrated with an example. This suggests a direction for more personalized clinical decision support.

Summary updated

BibTeX
@inproceedings{1996clinicians,
  title = {{A Clinician’s Tool for Analyzing Non-Compliance}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1996},
}
Figure