Pretrained Patient Trajectories for Adverse Drug Event Prediction Using Common Data Model-based Electronic Health Records

Junmo Kim, Joo Seong Kim, Ji‐Hyang Lee, Mingyu Kim, Taehyun Kim, Chaeeun Cho, Rae Woong Park, Kwangsoo KimmedRxiv 2024 · DOI

Abstract

Pretraining electronic health record (EHR) data using language models by treating patient trajectories as natural language sentences has enhanced performance across various medical tasks. However, EHR pretraining models have never been utilized in adverse drug event (ADE) prediction. A retrospective study was conducted on observational medical outcomes partnership (OMOP)-common data model (CDM) based EHR data from two separate tertiary hospitals. For pretraining, codes were randomly masked, and the model was trained to infer the masked tokens utilizing preceding and following history. In this process, we introduced domain embedding (DE) to provide information about the domain of the masked token, preventing the model from finding codes from irrelevant domains. EHR pretraining model with DE outperformed all the other baselines in all cohorts. For feature importance analysis, we demonstrated that the results were consistent with priorly reported background clinical knowledge. CDM-based EHR pretraining model with DE is a proper model for various ADE prediction tasks.

Introduction

Adverse Drug Events (ADEs)—injuries resulting from medical interventions related to a drug—are a leading cause of hospital morbidity and mortality. Traditional methods for detecting ADEs often rely on retrospective statistical analysis or simple machine learning models that struggle to capture the complex, temporal "story" of a patient’s health. This paper presents a novel application of transformer-based language models to the critical field of pharmacovigilance. By treating a patient’s medical history as a "sentence" and individual medical codes as "words," the researchers developed a system to predict ADEs with high accuracy across different hospital systems.

Methods: Domain-Aware Pretraining

The researchers adapted the BERT architecture to pretrain on patient trajectories from Electronic Health Records (EHR). The model (6 layers, 8 attention heads) was trained using Masked Language Modeling (MLM), where 30% of the medical codes in a patient’s history are hidden, and the model must predict them based on the surrounding context.

Figure 1: Input representation for the EHR pretraining model, showing the integration of medical codes, age, date, and domain embeddings within the CDM schema.

As illustrated in Figure 1, the key innovation is the introduction of Domain Embedding (DE). Medical codes are drawn from different "domains" (e.g., diagnosis, drug, measurement, or procedure). By adding a specific embedding that identifies the domain of a masked token, the researchers provided a "hint" that narrowed the search space. This prevented the model from attempting to predict a drug code when the masked slot was clearly a lab measurement, significantly improving learning efficiency.

Experimental Setup and Validation

The study utilized massive standardized datasets from two major tertiary hospitals in South Korea: Seoul National University Hospital (SNUH) and Ajou University Medical Center (AUMC). The researchers focused on three specific ADE cohorts: NSAID-induced Peptic Ulcers (NSAID-PU), anticoagulant-induced Intracranial Hemorrhage (AC-ICH), and chemotherapy-induced Neutropenic Fever (Chemo-NF).

Figure 3: Internal and external validation workflows, illustrating the transfer learning strategies (Type I, II, and III) used to mitigate data distribution gaps between hospitals.

To evaluate the model's robustness, the team implemented a rigorous validation framework (Figure 3). This included internal validation at Hospital A and external validation at Hospital B using three transfer strategies. The "Type II" approach—taking a model pretrained on Hospital A, performing a short "warm-up" pretraining on Hospital B, and then fine-tuning—yielded the best results, successfully mitigating the data distribution gap between institutions.

Key Results and Insights

The pretrained model with Domain Embedding consistently outperformed all baselines, including models initialized with randomized weights and pretrained models without DE.

  • Internal Validation (AUROC): The model achieved exceptional performance with scores of 0.977 for NSAID-PU, 0.908 for AC-ICH, and 0.989 for Chemo-NF.
  • Clinical Interpretability: Analysis of the model’s attention matrices revealed that it focused on clinically relevant features. For example, in the anticoagulant cohort, the model paid high attention to "skin lesions," a logical clinical precursor to internal bleeding.
  • Transferability: Despite low overlap in specific medical codes between hospitals (only ~7,400 shared out of 20,000+), the pretraining on standardized CDM data allowed for effective knowledge transfer.

Conclusion

CDM-based EHR pretraining with Domain Embedding is a powerful tool for ADE prediction. By providing categorical context during the learning phase, the model more effectively captures the underlying logic of medical trajectories. Future work aims to incorporate unstructured data, such as clinical notes, and integrate Large Language Models to provide natural-language explanations for high-risk flags, enhancing utility for bedside clinicians.

Summary updated

Abstract

Abstract Background Pretraining electronic health record (EHR) data using language models by treating patient trajectories as natural language sentences has enhanced performance across various medical tasks. However, EHR pretraining models have never been utilized in adverse drug event (ADE) prediction. Methods A retrospective study was conducted on observational medical outcomes partnership (OMOP)-common data model (CDM) based EHR data from two separate tertiary hospitals. The data included patient information in various domains such as diagnosis, prescription, measurement, and procedure. For pretraining, codes were randomly masked, and the model was trained to infer the masked tokens utilizing preceding and following history. In this process, we introduced domain embedding (DE) to provide information about the domain of the masked token, preventing the model from finding codes from irrelevant domains. For qualitative analysis, we identified important features using the attention matrix from each finetuned model. Results 510,879 and 419,505 adult inpatients from two separate tertiary hospitals were included in internal and external datasets. EHR pretraining model with DE outperformed all the other baselines in all cohorts. For feature importance analysis, we demonstrated that the results were consistent with priorly reported background clinical knowledge. In addition to cohort-level interpretation, patient-level interpretation was also available. Conclusions CDM-based EHR pretraining model with DE is a proper model for various ADE prediction tasks. The results of the qualitative analysis with feature importance were consistent with background clinical knowledge. Plain language summary Patient history is like natural language; each medical code corresponds to a word, and the sequence of medical codes corresponds to a sentence. Language models learn from sentences by inferring masked words using remaining unmasked words, and language model-based EHR pretraining models comprehend medical context similarly. As several studies that have utilized EHR pretraining models have achieved great success in various tasks, we applied EHR pretraining models for adverse drug event (ADE) prediction. For better inference of the EHR pretraining model, we introduced domain embedding (DE) to provide a hint of the domain for each masked code. The model pretrained with DE performed the best in various ADE tasks, and regarding background clinical knowledge was well-reflected in our feature importance-based qualitative analysis.
BibTeX
@misc{kim2024pretrained,
  author = {Junmo Kim and Joo Seong Kim and Ji‐Hyang Lee and Mingyu Kim and Taehyun Kim and Chaeeun Cho and Rae Woong Park and Kwangsoo Kim},
  title = {{Pretrained Patient Trajectories for Adverse Drug Event Prediction Using Common Data Model-based Electronic Health Records}},
  year = {2024},
  doi = {10.1101/2024.09.30.24314595},
  howpublished = {medRxiv},
}
Figure