Reproducible workflow for online AI in digital health

Susobhan Ghosh, Bhanu T. Gulapalli, Daiqi Gao, Asim H. Gazi, Anna L. Trella, Ziping Xu, Kelly W. Zhang, Susan A. MurphyarXiv 2025 · DOI · arXiv

Introduction

Digital health interventions (DHIs), such as mobile applications for physical activity or substance use management, are increasingly utilizing online AI. Unlike "offline" models that are trained on static datasets, online AI systems learn and adapt in real-time based on continuous data streams. While this allows for personalized care, it introduces significant challenges for scientific reproducibility.

The adaptive feedback loop of online AI in digital health interventions.

The core issue lies in the adaptive feedback loop: the AI's decisions influence user behavior, which subsequently alters the data the AI receives. This circular dependency can lead to "divergence" where small variations in code or data latency result in vastly different system states.

The Reproducibility Crisis in Digital Health

The authors identify two primary sources of instability that hinder reproducibility:

  1. Engineering Instability: Discrepancies between development and production environments, inconsistent codebases, or silent system failures.
  2. Algorithmic Instability: Learning algorithms that lack convergence guarantees or react unpredictably to noisy, real-world data.

Without a reproducible framework, researchers cannot audit specific AI decisions, compromising both scientific discovery and patient safety.

A Three-Phase Reproducible Workflow

To address these challenges, a systems-engineering approach is proposed, dividing the AI lifecycle into three distinct phases.

The proposed three-phase reproducible workflow for online AI systems.

Phase 1: Design Reproducibility

Before deployment, candidate algorithms are stress-tested using Digital Twins. These are high-fidelity simulators constructed from historical data that allow researchers to evaluate how an algorithm handles user heterogeneity, data drift, and missing values in a controlled "sandbox" environment.

Phase 2: Deployment Reproducibility

This phase focuses on the technical infrastructure required for reliable real-time operation:

  • Software Isolation: Utilizing containerization (e.g., Docker) and orchestration (e.g., Kubernetes) ensures that the execution environment remains identical across all stages of the project.
  • Traceable Logging: Every input, model state, and random seed must be recorded. This creates an audit trail that allows for a post-hoc "replay" of the AI's logic to verify its decisions.

Phase 3: Inference Reproducibility

Post-deployment, the logged data is used for rigorous performance analysis. These insights are then fed back into Phase 1 to inform the next iteration of the algorithm design.

Lessons from the Field: The "Imputation Trap"

The framework is grounded in real-world interventions like HeartSteps (physical activity), Oralytics (oral care), and MiWaves (cannabis use reduction). A key insight from the HeartSteps study is the Imputation Trap. Due to network delays, the AI often had to use estimated (imputed) step counts to make immediate decisions. When the actual data arrived later, the system would overwrite the imputed values. This practice destroyed the record of what the AI actually "saw" at the moment of decision, making it impossible to reconstruct the logic behind specific interventions.

Recommendation: Never overwrite real-time decision data with delayed ground truth; maintain both for a complete audit trail.

Operational Recommendations for Robust AI

  • Strict Separation: Maintain a hard line between development and production to avoid "dependency hell."
  • Fallback Mechanisms: Define "safety net" behaviors (e.g., default or random interventions) to ensure patient care continues even if the AI system crashes or data is missing.
  • Validation via Replay: Researchers should be able to input logged data back into the algorithm to see if the resulting model states match the original logs.

Conclusion and Future Directions

While this workflow provides a roadmap for reproducibility, challenges remain. The reliance on proprietary hardware (e.g., smartwatches) often introduces "black box" data processing that researchers cannot control. Furthermore, resource constraints on wearable devices can limit the depth of data logging. Future research should focus on modular Digital Twins and stable autonomous learning rules to further bridge the gap between AI theory and clinical practice.

Summary updated

Abstract

Online artificial intelligence (AI) algorithms are an important component of digital health interventions. These online algorithms are designed to continually learn and improve their performance as streaming data is collected on individuals. Deploying online AI presents a key challenge: balancing adaptability of online AI with reproducibility. Online AI in digital interventions is a rapidly evolving area, driven by advances in algorithms, sensors, software, and devices. Digital health intervention development and deployment is a continuous process, where implementation - including the AI decision-making algorithm - is interspersed with cycles of re-development and optimization. Each deployment informs the next, making iterative deployment a defining characteristic of this field. This iterative nature underscores the importance of reproducibility: data collected across deployments must be accurately stored to have scientific utility, algorithm behavior must be auditable, and results must be comparable over time to facilitate scientific discovery and trustworthy refinement. This paper proposes a reproducible scientific workflow for developing, deploying, and analyzing online AI decision-making algorithms in digital health interventions. Grounded in practical experience from multiple real-world deployments, this workflow addresses key challenges to reproducibility across all phases of the online AI algorithm development life-cycle.
BibTeX
@misc{ghosh2025reproducible,
  author = {Susobhan Ghosh and Bhanu T. Gulapalli and Daiqi Gao and Asim H. Gazi and Anna L. Trella and Ziping Xu and Kelly W. Zhang and Susan A. Murphy},
  title = {{Reproducible workflow for online AI in digital health}},
  year = {2025},
  eprint = {2509.13499},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.13499},
}
Figure