RELEAP: Reinforcement-Enhanced Label-Efficient Active Phenotyping for Electronic Health Records

Yang Yang, Kathryn I. Pollak, Bibhas Chakraborty, Molei Liu, Doudou Zhou, Chuan HongarXiv 2025 · DOI · arXiv

Abstract

Electronic Health Records (EHR) offer vast potential for clinical research, yet identifying accurate patient phenotypes remains a challenge due to noisy proxy labels and the high cost of expert chart review. We present RELEAP (Reinforcement-Enhanced Label-Efficient Active Phenotyping), a framework that optimizes the selection of patient records for manual labeling using reinforcement learning. By adaptively combining multiple active learning strategies and using downstream model performance as a reward signal, RELEAP achieves high accuracy in clinical risk prediction with minimal labeling effort.

Introduction

The process of identifying a patient's true clinical state—phenotyping—is foundational to medical research. While EHRs provide large-scale data, they often rely on "noisy" proxies like billing codes, which can be inaccurate or incomplete. Manual "chart review" remains the gold standard but is prohibitively expensive. Traditional Active Learning (AL) attempts to mitigate this by selecting informative samples based on fixed heuristics (e.g., model uncertainty). However, these heuristics are often "task-blind," failing to account for the specific requirements of the final clinical prediction task.

Methods

RELEAP addresses these limitations by treating sample selection as a dynamic decision-making problem managed by a Reinforcement Learning (RL) agent.

Overview of the RELEAP framework, showing the RL-AL process loop and its connection to the downstream prediction task.

The framework consists of several key components:

  • RL Agent: Utilizing Proximal Policy Optimization (PPO), the agent learns to manage the labeling process.
  • Strategy Fusion: Instead of a single heuristic, the agent dynamically weights three strategies: Uncertainty (low confidence samples), Diversity (covering the feature space), and Query-by-Committee (disagreement between models).
  • Task-Aware Feedback: The agent receives a reward based on the improvement in a downstream prediction model (e.g., lung cancer risk). This ensures the selection process is optimized for the final clinical goal rather than just internal model metrics.
  • Automatic Reference Phenotype: To simulate expert labels during development, the authors utilized a high-fidelity reference combining structured data with Large Language Model (LLM) extractions from clinical notes.

Datasets

The framework was validated on two primary datasets:

  1. DUHS Cohort: A real-world dataset from Duke University Health System comprising 238,119 patients (2014–2024), focused on predicting incident lung cancer risk.
  2. Synthetic Data: A controlled simulation of 1,000 samples used to validate the mathematical framework.

Results

RELEAP demonstrated significant improvements over standard noisy baselines and fixed-strategy active learning:

  • Predictive Accuracy: In lung cancer risk prediction, RELEAP improved the Logistic Regression AUC from 0.774 (noisy baseline) to 0.805, nearly reaching the "Oracle" performance of 0.807.
  • Survival Analysis: For time-to-event prediction, the C-index improved from 0.718 to 0.752.
  • Labeling Efficiency: The system reached near-optimal performance with a significantly lower labeling budget compared to random sampling, particularly within complex patient subgroups.

Discussion and Conclusion

RELEAP highlights the importance of "task-awareness" in active learning for clinical applications. By learning a "curriculum" of selection strategies—for instance, prioritizing diversity early on and uncertainty later—the framework provides more stable and reliable performance gains than traditional methods. Future work will focus on multi-site validation and the integration of fairness and interpretability into the RL reward function to ensure unbiased model performance across diverse demographic groups.

Summary updated

Abstract

Objective: Electronic health record (EHR) phenotyping often relies on noisy proxy labels, which undermine the reliability of downstream risk prediction. Active learning can reduce annotation costs, but most rely on fixed heuristics and do not ensure that phenotype refinement improves prediction performance. Our goal was to develop a framework that directly uses downstream prediction performance as feedback to guide phenotype correction and sample selection under constrained labeling budgets. Materials and Methods: We propose Reinforcement-Enhanced Label-Efficient Active Phenotyping (RELEAP), a reinforcement learning-based active learning framework. RELEAP adaptively integrates multiple querying strategies and, unlike prior methods, updates its policy based on feedback from downstream models. We evaluated RELEAP on a de-identified Duke University Health System (DUHS) cohort (2014-2024) for incident lung cancer risk prediction, using logistic regression and penalized Cox survival models. Performance was benchmarked against noisy-label baselines and single-strategy active learning. Results: RELEAP consistently outperformed all baselines. Logistic AUC increased from 0.774 to 0.805 and survival C-index from 0.718 to 0.752. Using downstream performance as feedback, RELEAP produced smoother and more stable gains than heuristic methods under the same labeling budget. Discussion: By linking phenotype refinement to prediction outcomes, RELEAP learns which samples most improve downstream discrimination and calibration, offering a more principled alternative to fixed active learning rules. Conclusion: RELEAP optimizes phenotype correction through downstream feedback, offering a scalable, label-efficient paradigm that reduces manual chart review and enhances the reliability of EHR-based risk prediction.
BibTeX
@misc{yang2025releap,
  author = {Yang Yang and Kathryn I. Pollak and Bibhas Chakraborty and Molei Liu and Doudou Zhou and Chuan Hong},
  title = {{RELEAP: Reinforcement-Enhanced Label-Efficient Active Phenotyping for Electronic Health Records}},
  year = {2025},
  eprint = {2511.07473},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.07473},
}
Figure