Large language models as oncology world models

Julien Vibert, Jakob Nikolas KatherarXiv 2026 · arXiv

TL;DR

This paper evaluates whether Large Language Models (LLMs) can function as "world models" for oncology—internalizing the complex, dynamic logic of cancer progression and treatment. While LLMs excel at "textbook" biomarker logic, the authors reveal significant blind spots in how models handle patient history and physical frailty, alongside a persistent "optimism bias" regarding survival.

What Problem Does This Paper Address?

LLMs are increasingly deployed in clinical settings for tasks like tumor board assistance and patient-facing chatbots. However, most evaluations focus on static question-answering rather than the longitudinal trajectory of a patient’s journey.

The authors argue that to be truly useful, an LLM must possess a "world model": an internal representation of the "physics" of oncology—how disease burden, host condition, and treatments interact over time. The gap addressed here is the lack of a systematic audit of these internal models. Do LLMs understand that a frail patient responds differently than a fit one? Do they recognize how a previous treatment failure changes the stakes for the next one?

Methods

The researchers developed a model-agnostic pipeline to generate and analyze structured patient trajectories. Key components include:

  • Clinical State Space (CSS): A novel framework that uses Principal Component Analysis (PCA) to map a patient's status onto a 2D plane based on 20 variables (10 tumor-related, 10 host-related). This allows researchers to visualize a patient's journey as a path through a "map" of health and disease.
  • Three-Tiered Evaluation:
    1. Unconstrained Generation: Asking models to generate patients from scratch based only on cancer type to check for "archetypes" or mode collapse.
  • Clinical Probing: Using 12 "what-if" scenarios to test if models follow medical logic when specific variables (like biomarkers or frailty) are changed.
  • Real-Patient Forecasting: Masking real-world trajectories and asking the LLM to predict the "continuation" (future) of those patients.
  • Models Tested: A suite of high-performing models including Claude Sonnet 5, Gemini 3.1 Flash-Lite, Kimi K3, GLM-5.2, Qwen 3.6 Flash, and Gemma 4.

Datasets

  • Synthetic Atlas: 4,434 trajectories for 148 tumor types generated by the LLMs.
  • GENIE-BPC: A high-quality registry of real-world data for lung, breast, and colorectal cancer used for the forecasting experiments.
  • PRIME Trial: A dataset from a randomized clinical trial (panitumumab in colorectal cancer) used to validate the Clinical State Space framework against gold-standard clinical evidence.

Key Insights and Ideas

  • The "Archetype" Trap: When left to their own devices, LLMs suffer from a form of mode collapse. They tend to generate "textbook" patients with very little diversity; 81% to 93% of generated patients for a specific cancer shared the exact same baseline features.
  • Biomarker Logic vs. Clinical Common Sense: LLMs are excellent at "If-Then" logic for biomarkers (e.g., "If EGFR mutant, give osimertinib"). They succeeded in 29 of 36 biomarker-driven scenarios. However, they failed miserably (1 out of 12) when the correct decision depended on patient history or physical frailty.
  • The Optimism Bias: LLMs are "over-optimistic" about survival. When forecasting real patients, the models predicted a 12-month survival rate of ~90%, whereas the actual observed survival was 72.2%. This likely reflects the "success bias" of the medical literature (trials and case studies) used in their training data.
  • In-Context Learning Helps: Providing the model with "matched examples" (similar real-world patients) significantly improved the accuracy of survival forecasts, though it still didn't beat traditional statistical baselines like Cox models.

Key Results

  • Survival Discrepancy: In lung cancer, median survival predictions varied by 11.2 months between different LLMs, showing high inter-model disagreement.
  • Logic Testing: In a factorial experiment with Claude, the model correctly reversed treatment advantages when a biomarker was absent, but it almost entirely ignored the "pace of untreated growth" (history) when making decisions.
  • CSS Utility: The Clinical State Space successfully mapped treatment effects in the PRIME trial, showing it can organize dense clinical data into a format that describes "whole courses" rather than just single endpoints like survival.

Limitations and Future Work

  • Data Density: The CSS requires dense longitudinal data (frequent updates on symptoms, weight, and tumor sites) which is often missing from standard electronic health records.
  • Predictive Reliability: The authors explicitly state that current LLMs are not yet reliable enough to be used as predictive models for individual patients.
  • Future Directions: The authors suggest that "digital twins"—simulated versions of real patients used to test treatment strategies—could be built on improved world models. Future work should focus on training lightweight, domain-specific models on diverse, longitudinal clinical data rather than relying on general-purpose "off-the-shelf" LLMs.

Summary updated

Abstract

Large language models (LLMs) can synthesize cancer patient cases, but their internal representation of oncology, or world model, has not been explored systematically. By only specifying tumor type, we show that LLMs tend to generate similar patients per cancer, with on average 81.0% to 93.2% of a cancer's patients sharing the most frequent value of each baseline feature. By systematically varying patient features in 12 clinical scenarios, we show that they follow biomarker and treatment logic in 29 of 36 model-scenario assessments, against 1 of 12 when previous treatment or frailty matters. Continuing real patient trajectories, an LLM predicted 89.9% 12-month survival, and 79.2% with similar patients as examples, against 72.2% observed. To represent these trajectories, we developed the Clinical State Space, a dynamic representation framework that we also applied to real patients. Despite the limits we show, improved world models may support digital twins, which our framework can evaluate.
BibTeX
@misc{vibert2026large,
  author = {Julien Vibert and Jakob Nikolas Kather},
  title = {{Large language models as oncology world models}},
  year = {2026},
  eprint = {2611.18994},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.18994},
}
Figure