Data reuse enables cost-efficient randomized trials of medical AI models

Michael Nercessian, Wenxin Zhang, Alexander Schubert, Daphne Yang, Maggie Chung, Ahmed Alaa, Adam YalaarXiv 2025 · DOI · arXiv

Introduction

Randomized controlled trials (RCTs) are the gold standard for validating medical AI, but their high costs and slow timelines clash with the rapid iteration cycles of software development. Unlike a new pharmaceutical drug, which remains static once formulated, AI models are constantly updated—retrained on new data, upgraded with new architectures, or fine-tuned for local populations. Running a full-scale, multi-million-dollar RCT from scratch for every minor version update is financially and operationally impossible. This creates a massive bottleneck for clinical AI, where models may become obsolete by the time they are fully validated.

The BRIDGE Methodology

To address this gap, we introduce BRIDGE, a novel trial design that reuses patient data from previous trials when legacy and new AI models make concordant predictions. The core innovation relies on a unique property of AI: unlike a physical drug, an AI model's predictions can be retrospectively applied to historical patient data to see what it would have recommended.

When evaluating a new model, BRIDGE stratifies prospective patients into two groups:

  1. Concordant cases: Both the legacy model (from the old trial) and the new model agree on the patient's risk level.
  2. Discordant cases: The models disagree (e.g., the new model flags them as high-risk, but the old one did not).

Instead of recruiting a completely fresh cohort, BRIDGE reuses historical trial data for the concordant stratum. New patient enrollment is strictly focused on the discordant cases (and topping up the concordant stratum if needed). To ensure this doesn't introduce statistical bias, the design relies on the assumption of exchangeability—ensuring that historical patients would have experienced the exact same clinical decisions and outcomes had they been enrolled in the new trial.

Empirical Validation

We validated the real-world feasibility of BRIDGE across three diverse clinical domains and data modalities:

  • Breast Cancer (Imaging): 130,712 screening mammograms from UCSF. Evaluated models included AI-Density, ImgOnly DL, and Mirai.
  • Cardiovascular Disease (Time-series): 103,952 emergency department ECGs from the Nightingale Open Science dataset.
  • Sepsis (Structured EHR): ~6.4 million hourly measurements from the eICU Collaborative Research Database.

Key Findings and Results

Our analysis revealed that successive generations of AI models usually agree on the most obvious cases, creating a massive opportunity for data reuse. Across the three case studies, we observed substantial overlap in the top 5% high-risk cohorts identified by successive models: sepsis models showed up to 64.8% overlap, ECG models showed 49.6% overlap, and breast cancer models showed 46.6% overlap.

This concordance translates directly into cost savings. In a simulated breast cancer screening trial where an AI model allocates supplemental MRIs, using BRIDGE to reuse data from a legacy trial reduced the required prospective enrollment by 46.6% (saving 9,503 participants). At an estimated $1,500 per research MRI, this reduction translates to over **$2.85 million in savings**, all while successfully maintaining 80% statistical power.

Discussion and Limitations

While BRIDGE offers a path to more efficient validation, its validity hinges on the exchangeability assumption, which can be threatened by data drift. If the standard of care changes, population demographics shift, or hospital hardware is upgraded, historical data may no longer represent current clinical reality. Furthermore, reusing historical data risks propagating historical biases; if a legacy trial underrepresented minority populations, that bias must be addressed during data reuse.

Future work will explore how to integrate this statistical framework with regulatory pathways, such as the FDA's Predetermined Change Control Plans (PCCPs), to create a seamless pipeline for the continuous, rigorous post-market surveillance of evolving AI models.

Summary updated

Abstract

Randomized controlled trials (RCTs) are indispensable for establishing the clinical value of medical artificial-intelligence (AI) tools, yet their high cost and long timelines hinder timely validation as new models emerge rapidly. Here, we propose BRIDGE, a data-reuse RCT design for AI-based risk models. AI risk models support a broad range of interventions, including screening, treatment selection, and clinical alerts. BRIDGE trials recycle participant-level data from completed trials of AI models when legacy and updated models make concordant predictions, thereby reducing the enrollment requirement for subsequent trials. We provide a practical checklist for investigators to assess whether reusing data from previous trials allows for valid causal inference and preserves type I error. Using real-world datasets across breast cancer, cardiovascular disease, and sepsis, we demonstrate concordance between successive AI models, with up to 64.8% overlap in top 5% high-risk cohorts. We then simulate a series of breast cancer screening studies, where our design reduced required enrollment by 46.6%--saving over US$2.8 million--while maintaining 80% power. By transforming trials into adaptive, modular studies, our proposed design makes Level I evidence generation feasible for every model iteration, thereby accelerating cost-effective translation of AI into routine care.
BibTeX
@misc{nercessian2025data,
  author = {Michael Nercessian and Wenxin Zhang and Alexander Schubert and Daphne Yang and Maggie Chung and Ahmed Alaa and Adam Yala},
  title = {{Data reuse enables cost-efficient randomized trials of medical AI models}},
  year = {2025},
  eprint = {2511.08986},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.08986},
}
Figure