Large language models forecast patient health trajectories enabling digital twins

Nikita Makarov, Maria Bordukova, Papichaya Quengdaeng, Daniel Garger, Raul Rodriguez-Esteban, Fabian Schmich, Michael P. MendenNpj Digit. Medicine 2025 · OpenReview

TL;DR

Researchers have developed DT-GPT, a novel framework that leverages large language models (LLMs) to forecast patient health trajectories, creating "digital twins" for personalized medicine. This approach excels at handling real-world electronic health record complexities and demonstrates state-of-the-art performance in predicting clinical outcomes across various diseases, even for variables it wasn't explicitly trained on.

What Problem Does This Paper Address?

Predicting a patient's future health trajectory is crucial for personalized medicine, treatment selection, and clinical trial design. Digital twins, which are virtual patient representations, offer a promising avenue for this, but their development has been hampered by challenges in handling complex, real-world electronic health record (EHR) data. These challenges include missing values, noisy data, and limited sample sizes. Existing machine learning models often struggle with these issues, requiring extensive data preprocessing like imputation and normalization, and can be limited in their ability to capture the intricate relationships between various clinical variables over time. Furthermore, many models focus on single-point predictions rather than comprehensive longitudinal trajectories.

Methods

The core innovation is the Digital Twin—Generative Pretrained Transformer (DT-GPT). This framework extends existing LLM-based forecasting by fine-tuning a pre-trained LLM (specifically, BioMistral 7B) on clinical EHR data. Key aspects of the method include:

  • Leveraging EHRs without Imputation/Normalization: DT-GPT directly processes EHR data, bypassing the need for complex preprocessing steps like imputing missing values or normalizing data. This is achieved by encoding patient histories into a text format that LLMs can understand.
  • Text-based Encoding: Patient data (history, demographics, forecast dates) is converted into a structured text format, allowing LLMs to process it naturally.
  • Fine-tuning for Clinical Trajectories: The pre-trained LLM is fine-tuned on this encoded clinical data to learn to predict future health trajectories.
  • Zero-Shot Forecasting: A significant capability is DT-GPT's ability to forecast variables it was not explicitly trained on, demonstrating its generalizability and adaptability.
  • Human-Interpretable Interface: The model provides a chatbot-like interface that allows users to inquire about the reasoning behind predictions, enhancing interpretability.

Datasets

The study evaluated DT-GPT on three distinct datasets:

  • Non-Small Cell Lung Cancer (NSCLC): A large, real-world EHR dataset from Flatiron Health, containing data from approximately 16,496 patients. This dataset was used for long-term forecasting (up to 13 weeks).
  • Intensive Care Unit (ICU): A publicly available dataset (MIMIC-IV) with data from 35,131 patients, used for short-term forecasting (next 24 hours).
  • Alzheimer's Disease: A dataset from the Alzheimer's Disease Neuroimaging Initiative (ADNI), comprising 1,140 patients, used for long-term forecasting (up to 24 months).

These datasets represent different time scales and patient populations, allowing for a comprehensive evaluation of DT-GPT's performance.

Key Insights and Ideas

  • LLMs as Digital Twins: The paper strongly advocates for LLMs as a powerful foundation for building digital twins, capable of mimicking individual patient trajectories.
  • EHR Data Robustness: DT-GPT's ability to handle missing data, noise, and lack of normalization directly from EHRs is a major advantage over traditional ML models.
  • Zero-Shot Learning for Clinical Insights: The zero-shot forecasting capability is groundbreaking, allowing the model to predict outcomes for variables it hasn't seen during training, opening doors for broader clinical applications.
  • Preservation of Inter-Variable Relationships: The model not only predicts individual variables but also maintains the crucial correlations between them, reflecting biological reality.
  • Interpretability through Conversation: The chatbot interface for explaining predictions is a significant step towards making complex AI models more trustworthy and understandable for clinicians.

Key Results

  • State-of-the-Art Performance: DT-GPT outperformed 14 baseline models across all three datasets, achieving a reduction in scaled mean absolute error (MAE) of 3.4% (NSCLC), 1.3% (ICU), and 1.8% (Alzheimer's).
  • Distributional Fidelity: DT-GPT closely reproduced the distribution of original data and maintained inter-variable relationships, assessed via Kolmogorov-Smirnov distance.
  • Zero-Shot Superiority: In zero-shot forecasting, DT-GPT outperformed a fully trained LightGBM model on 13 out of 69 non-target variables, particularly for those biologically related to the trained targets.
  • Robustness to Data Challenges: DT-GPT showed resilience to missingness (up to 20% additional masking) and misspellings in input data.
  • Conversational AI: The model successfully retained its conversational ability post-fine-tuning, providing explanations for predictions.

Limitations and Future Work

  • Forecasting Critical Events: DT-GPT struggled with forecasting rare, critical events (e.g., severe hemoglobin drops) due to their low prevalence, suggesting a need for methods to enhance sensitivity to high-risk events.
  • Limited Number of Forecasted Variables: Current LLM sequence length limits the number of variables that can be forecasted simultaneously. Future work aims to address this by using larger models or extending context length.
  • Trajectory Aggregation: The averaging of multiple generated trajectories can sometimes mask individual successful predictions. Developing improved aggregation methods or human-in-the-loop selection is suggested.
  • Hallucination and Bias: Like all LLMs, DT-GPT can potentially hallucinate or reflect biases present in the training data. Future work will focus on human-in-the-loop validation and bias mitigation strategies.
  • Few-Shot Learning: Transitioning from zero-shot to few-shot learning could further enhance performance and expand the range of predictable variables.
  • Disease-Specific Metrics: Developing forecasting metrics tailored to clinical utility is an important direction for future research.

Summary updated

Abstract

Generative artificial intelligence is revolutionizing digital twin development, enabling virtual patient representations that predict health trajectories, with large language models (LLMs) showcasing untapped clinical forecasting potential. We developed the Digital Twin—Generative Pretrained Transformer (DT-GPT), extending LLM-based forecasting solutions to clinical trajectory prediction. DT-GPT leverages electronic health records without requiring data imputation or normalization and overcomes real-world data challenges such as missingness, noise, and limited sample sizes. Benchmarking on non-small cell lung cancer, intensive care unit, and Alzheimer’s disease datasets, DT-GPT outperformed state-of-the-art machine learning models, reducing the scaled mean absolute error by 3.4%, 1.3% and 1.8%, respectively. It maintained distributions and cross-correlations of clinical variables, and demonstrated explainability through a human-interpretable interface. Additionally, DT-GPT’s ability to perform zero-shot forecasting highlights potential advantages of LLMs as clinical forecasting platforms, proposing a path towards digital twin applications in clinical trials, treatment selection, and adverse event mitigation.
BibTeX
@inproceedings{makarov2025large,
  author = {Nikita Makarov and Maria Bordukova and Papichaya Quengdaeng and Daniel Garger and Raul Rodriguez-Esteban and Fabian Schmich and Michael P. Menden},
  title = {{Large language models forecast patient health trajectories enabling digital twins}},
  booktitle = {Npj Digit. Medicine},
  year = {2025},
  url = {https://openreview.net/forum?id=iUZCGRzf2f},
}
Figure