Abstract
Electronic Health Records (EHR) offer vast potential for clinical research, yet identifying accurate patient phenotypes remains a challenge due to noisy proxy labels and the high cost of expert chart review. We present RELEAP (Reinforcement-Enhanced Label-Efficient Active Phenotyping), a framework that optimizes the selection of patient records for manual labeling using reinforcement learning. By adaptively combining multiple active learning strategies and using downstream model performance as a reward signal, RELEAP achieves high accuracy in clinical risk prediction with minimal labeling effort.
Introduction
The process of identifying a patient's true clinical state—phenotyping—is foundational to medical research. While EHRs provide large-scale data, they often rely on "noisy" proxies like billing codes, which can be inaccurate or incomplete. Manual "chart review" remains the gold standard but is prohibitively expensive. Traditional Active Learning (AL) attempts to mitigate this by selecting informative samples based on fixed heuristics (e.g., model uncertainty). However, these heuristics are often "task-blind," failing to account for the specific requirements of the final clinical prediction task.
Methods
RELEAP addresses these limitations by treating sample selection as a dynamic decision-making problem managed by a Reinforcement Learning (RL) agent.
The framework consists of several key components:
- RL Agent: Utilizing Proximal Policy Optimization (PPO), the agent learns to manage the labeling process.
- Strategy Fusion: Instead of a single heuristic, the agent dynamically weights three strategies: Uncertainty (low confidence samples), Diversity (covering the feature space), and Query-by-Committee (disagreement between models).
- Task-Aware Feedback: The agent receives a reward based on the improvement in a downstream prediction model (e.g., lung cancer risk). This ensures the selection process is optimized for the final clinical goal rather than just internal model metrics.
- Automatic Reference Phenotype: To simulate expert labels during development, the authors utilized a high-fidelity reference combining structured data with Large Language Model (LLM) extractions from clinical notes.
Datasets
The framework was validated on two primary datasets:
- DUHS Cohort: A real-world dataset from Duke University Health System comprising 238,119 patients (2014–2024), focused on predicting incident lung cancer risk.
- Synthetic Data: A controlled simulation of 1,000 samples used to validate the mathematical framework.
Results
RELEAP demonstrated significant improvements over standard noisy baselines and fixed-strategy active learning:
- Predictive Accuracy: In lung cancer risk prediction, RELEAP improved the Logistic Regression AUC from 0.774 (noisy baseline) to 0.805, nearly reaching the "Oracle" performance of 0.807.
- Survival Analysis: For time-to-event prediction, the C-index improved from 0.718 to 0.752.
- Labeling Efficiency: The system reached near-optimal performance with a significantly lower labeling budget compared to random sampling, particularly within complex patient subgroups.
Discussion and Conclusion
RELEAP highlights the importance of "task-awareness" in active learning for clinical applications. By learning a "curriculum" of selection strategies—for instance, prioritizing diversity early on and uncertainty later—the framework provides more stable and reliable performance gains than traditional methods. Future work will focus on multi-site validation and the integration of fairness and interpretability into the RL reward function to ensure unbiased model performance across diverse demographic groups.
Summary updated