Abstract
Objectives: This study set out to develop and validate a risk prediction tool for the early detection of heart failure (HF) onset using real-world electronic health records (EHRs). Background: While existing HF risk assessment models have shown promise in clinical settings, they are often tailored to specific medical conditions, limiting their generalizability. Moreover, most methods rely on hand-crafted features, making it difficult to capture the high-dimensional, sparse, and temporal nature of EHR data, thus reducing their predictive accuracy. Methods: A total of 2,561 HF and 5,493 matched control patients were identified from the OneFlorida Clinical Research Consortium. We employed a suite of natural language processing (NLP) models, including Bag of Words, Skip-gram, and ClinicalBERT, to generate EHR embeddings, which were used as inputs for five prediction models. Model calibration was assessed under three calibration scenarios: no recalibration, recalibration in the large, and logistic recalibration. Results: The XGBoost model demonstrated the best overall performance, achieving an AUROC of 0.7672, an F1 score of 0.5547, an AUPRC of 0.6382, and a Matthews correlation coefficient of 0.3993. The most impactful predictors included diagnoses, procedures, medications, lab tests, and patient age. Model performance varied across gender, race, and ethnicity subgroups. Logistic recalibration significantly improved model calibration in the overall cohort and demographic subgroups. Conclusions: Our NLP-based approach demonstrated strong predictive performance and clinical relevance, highlighting its potential for integration into real-world clinical applications to facilitate early detection and proactive management of individuals at risk for HF.
Introduction
Heart failure (HF) is a global health crisis that places an immense burden on both patients and healthcare systems. Early detection is critical for improving outcomes, yet traditional risk assessment models often struggle with generalizability and the inherent complexity of Electronic Health Records (EHRs). EHR data is high-dimensional, sparse, and temporal, making it difficult for standard statistical methods to capture the nuanced relationships between various clinical events over time. This study explores a Natural Language Processing (NLP)-based framework to "read" patient histories and predict HF onset before clinical diagnosis.
Methodology
To overcome the limitations of manual feature engineering, the researchers treated medical codes—including diagnoses, procedures, medications, and laboratory tests—as "words" within a patient's clinical "sentence." Three distinct NLP embedding techniques were evaluated:
- Bag-of-Words (BoW): Captures the frequency of clinical codes.
- Skip-gram (Word2Vec): Learns local patterns by predicting neighboring codes in a patient's record.
- ClinicalBERT: A transformer-based model that captures complex, contextual relationships across the entire clinical history.
These mathematical embeddings served as inputs for five machine learning classifiers: Logistic Regression, Lasso, Random Forest, XGBoost, and a Multilayer Perceptron (MLP). The study utilized a 12-month observation window for data collection and a 6-month prediction window to forecast HF onset.
Data and Cohort Selection
The study utilized data from the OneFlorida Clinical Research Consortium, encompassing records for approximately 16.8 million residents. Cases were defined by strict criteria, including multiple consecutive HF diagnoses and specific medication histories.
The final study population included 2,561 HF cases and 5,493 matched controls. Controls were matched based on age, gender, race, and encounter type to ensure a balanced comparison. Baseline characteristics for both groups, including vital signs and demographic distributions, are detailed below.
Results and Discussion
The tree-based XGBoost model emerged as the top performer, achieving an AUROC of 0.7672 and an AUPRC of 0.6382. Interestingly, XGBoost and Random Forest outperformed more complex neural network architectures like GRU and RETAIN, suggesting that hierarchical decision-making is highly effective for structured EHR data.
ClinicalBERT embeddings proved particularly powerful for contextualizing medications and lab tests. Furthermore, the study highlighted the importance of model calibration; applying logistic recalibration ensured that predicted risk scores accurately reflected actual clinical risk across diverse demographic subgroups. While the model performed exceptionally well in the Hispanic subgroup (AUROC 0.8300), the researchers noted performance variations across races, emphasizing the need for demographic-specific tuning in future iterations.
Conclusion
By treating EHR data as a language, this NLP-based approach successfully identifies patients at risk for heart failure with high accuracy. The integration of such tools into clinical workflows could enable proactive interventions, potentially reducing the long-term impact of heart failure through earlier detection and personalized management.
Summary updated