Towards Regulatory-Confirmed Adaptive Clinical Trials: Machine Learning Opportunities and Solutions

Omer Noy Klein, Alihan Hüyük, Ron Shamir, Uri Shalit, Mihaela van der Schaar

Introduction: The Gap Between Approval and Utility

Standard Randomized Controlled Trials (RCTs) are the "gold standard" for proving a drug is safe and effective. However, they often suffer from a significant limitation: they are primarily designed for regulatory approval, not necessarily for clinical utility. Because trials frequently lack demographic diversity, they often result in a "one-size-fits-all" approval for treatments that may perform differently across various ethnic groups, ages, or sexes.

In practice, doctors often do not learn the optimal way to prescribe these drugs for specific subgroups until "Phase IV" (post-marketing surveillance). This gap between regulatory success and real-world clinical optimization can take years to bridge, potentially putting patients at risk. This paper introduces a novel framework to address this challenge, asking: How can we design a trial that satisfies regulators while simultaneously learning an optimal, fair treatment policy for everyone?

The RFAN Framework: Balancing Rigor and Learning

The authors propose Randomize First Augment Next (RFAN), a two-stage clinical trial framework that integrates traditional randomized testing with Bayesian active learning.

The two-stage RFAN framework: starting with a randomized stage to satisfy regulatory constraints, followed by an augmented stage using active learning to optimize treatment policies.

RFAN operates in two distinct phases:

  1. The Randomized (Passive) Stage: This stage functions like a traditional RCT. Patients are assigned to treatment or control groups at random. This ensures the trial meets the strict "Type I error" requirements (the probability of falsely claiming a drug works) demanded by health authorities like the FDA.
  2. The Augmented (Active) Stage: Once sufficient data is gathered to potentially prove efficacy, the trial switches to an active learning mode. Using Causal-BALD (Bayesian Active Learning by Disagreement), the system identifies patient profiles where the model is most "uncertain"—often those from under-served or heterogeneous subgroups—and prioritizes recruiting them to refine the treatment policy.

To guide this process, the framework optimizes for two key objectives: Post-Trial Mean Benefit (PTMB), which maximizes overall clinical benefit upon deployment, and Post-Trial Fairness (PTF), which ensures the treatment policy performs equitably across sensitive groups.

Evaluation and Key Results

The framework was evaluated using synthetic data, a semi-synthetic dataset for Warfarin dosing (PharmGKB), and real-world COVID-19 hospitalization data (SIVEP-Gripe).

Visual comparison of a standard RCT (a) versus the proposed RFAN method (b). RFAN actively samples across the covariate space to better estimate the treatment effect curve, especially in regions of high uncertainty.

Key Insights:

  • Fairness and Accuracy: On synthetic benchmarks, RFAN achieved a two-fold increase in the Post-Trial Fairness (PTF) objective compared to standard RCTs. In the Warfarin simulation, it showed a 24% improvement in policy quality, proving significantly better at identifying optimal dosages for diverse patient profiles.
  • Efficiency: RFAN reached higher policy accuracy with fewer patients. By using uncertainty as a compass, the model learns a more nuanced Conditional Average Treatment Effect (CATE) than a standard trial, decreasing the policy error rate across all sample sizes.
  • Regulatory Safety: Crucially, the "Randomize First" component ensured that the trial maintained a high success rate for regulatory approval (90-100% in simulations), matching the statistical rigor of standard RCTs.

Conclusion and Future Directions

RFAN demonstrates that regulatory success and clinical utility are not mutually exclusive. By incorporating machine learning into the trial design, researchers can create more equitable and effective treatment policies from the outset.

While the results are promising, the authors note that future work is needed to address challenges such as delayed outcomes (where results take months to manifest) and the logistics of "streaming" active learning in real-world clinical settings. Nonetheless, RFAN provides a vital blueprint for making clinical trials both fairer and smarter.

Summary updated

BibTeX
@inproceedings{klein2025towards,
  author = {Omer Noy Klein and Alihan Hüyük and Ron Shamir and Uri Shalit and Mihaela van der Schaar},
  title = {{Towards Regulatory-Confirmed Adaptive Clinical Trials: Machine Learning Opportunities and Solutions}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2025},
  url = {https://openreview.net/forum?id=fgEJ8iEqjk},
}
Figure