From EHRs to Patient Pathways: Scalable Modeling of Longitudinal Health Trajectories with LLMs

Overview

Electronic Health Records (EHRs) are a goldmine of clinical information, yet their fragmented and high-volume nature makes them difficult to model holistically. EHR2Path is a novel foundation model designed to transform these disparate data points into structured "patient pathways." By employing a task-focused summarization bottleneck, the model can process extensive historical health data to simulate future patient trajectories—ranging from vital signs to lab results—with significantly higher accuracy and efficiency than previous methods.

The Challenge of Longitudinal EHR Modeling

Modern healthcare environments, particularly Intensive Care Units (ICUs) and Emergency Departments (EDs), generate a staggering density of data. This includes everything from high-frequency numerical vitals to unstructured physician notes and imaging reports. Traditional AI models in this domain are often "compartmentalized," optimized for narrow tasks like binary mortality prediction or specific diagnostic classification.

These specialized models often fail to capture the continuous, holistic journey of a patient. Furthermore, standard Large Language Models (LLMs) face significant scaling issues with EHR data; a single hospital admission can produce tens of thousands of data points, quickly exhausting the context window of even the most advanced models. EHR2Path addresses this by creating a system capable of ingesting a patient's entire history to simulate their future health pathway in a scalable way.

The EHR2Path Framework

The researchers developed a three-pronged approach to enable scalable longitudinal modeling:

1. Unified Structured Text Representation: To leverage the semantic reasoning of LLMs, heterogeneous EHR data (vitals, medications, labs, and reports) are converted into a unified, hierarchical text format. This allows the model to interpret medical terms naturally without needing rigid medical coding.
2. Masked Summarization Bottleneck: This core innovation addresses the context window limitation. The model compresses thousands of tokens of history into a small set of "summary tokens." Unlike standard auto-encoders, this bottleneck is task-focused, retaining only the information essential for predicting the patient's next clinical state.
3. Length-of-Stay (LOS) Integration: To ensure realistic simulations, a countdown timer (LOS indicator) was added during training. This helps the model learn the temporal rhythms of hospital stays, preventing simulations from running indefinitely and accurately modeling discharge or transfer events.

Expand

Source, p. 3

The framework is built upon the Qwen-0.5B architecture, choosing a compact LLM to balance high-level reasoning with the computational efficiency required for processing massive clinical datasets.

Data Sources and Representation

The study was conducted using the MIMIC-IV dataset, a comprehensive de-identified EHR database containing records for approximately 300,000 patients. The model was trained across 22 different data tables, encompassing:

• Emergency Department (ED) stays.
• General Hospital ward admissions.
• Intensive Care Unit (ICU) transitions.
• Diverse data types including medications, laboratory tests, vital signs, and imaging reports.

Performance and Key Findings

EHR2Path demonstrates that LLMs can act as "universal translators" for medical data, handling noise and missing values without extensive manual preprocessing.

Superior Context Handling

By utilizing the summarization bottleneck, EHR2Path managed an average of 9,823 tokens of context, with peaks exceeding 86,000 tokens. This represents a significant increase in effective context compared to text-only baselines like ETHOS, which are limited to 2,048 tokens.

Clinical Task Accuracy

The model's ability to simulate pathways translates directly into high performance on downstream clinical tasks. In 24-hour forecasts, EHR2Path consistently outperformed baselines in predicting ICU vital signs and hospital lab values.

Expand

Source, p. 8

As shown in the results, the summary-based model (E2P-S) and its variants achieved significant improvements in Mean Absolute Error (MAE) for numerical values. For instance, in ICU Vital Sign Development, it achieved a Value MAE of 0.08, a sharp improvement over the 0.21 baseline. Furthermore, when fine-tuned for specific outcomes like "ED Admission Prediction," EHR2Path outperformed specialized models like MEME, proving its versatility as a foundation model.

Future Directions

While EHR2Path represents a significant step forward, several areas for future research remain:

• Generalizability: Currently trained on the Boston-based MIMIC-IV dataset, the model needs validation across diverse international clinical settings where practices may vary.
• Long-term Trajectories: The current focus is on acute hospital stays. Future iterations aim to model "lifetime" trajectories, tracking chronic disease progression over years rather than days.
• Clinical Integration: Moving from technical modeling to clinical utility requires exploring how these simulations can be safely and effectively integrated into real-world decision-making workflows.

@misc{pellegrini2025from,
  author = {Chantal Pellegrini and Ege Özsoy and David Bani-Harouni and Matthias Keicher and Nassir Navab},
  title = {{From EHRs to Patient Pathways: Scalable Modeling of Longitudinal Health Trajectories with LLMs}},
  year = {2025},
  eprint = {2506.04831},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.04831},
}