Curiosity-0826-4B: Generative Medical Event Sequence Modeling Across Pediatric and Adult Populations

Mani Nagarathnam, Luke Benham, Paul Blazek, John Cashy, Kevin Chung, John Muenning, Hank Voeller, Shane Waxler, Davis White, Alison Callahan, Kevin Dysart, Andrew Loza, Michal A. Mankowski, Mattheus Ramsis, Nigam H. Shah, Karen K. Wong, Rahul ShaharXiv 2026 · arXiv

TL;DR

Researchers have unveiled Curiosity-0826-4B, a 4.3-billion parameter generative foundation model designed to simulate longitudinal patient records. By training on a massive dataset of over 190 million patients, the model achieves state-of-the-art zero-shot performance in clinical tasks ranging from pediatric asthma onset to adult mortality, significantly outperforming its predecessor in laboratory value prediction.

What Problem Does This Paper Address?

Clinical predictive modeling has traditionally relied on task-specific models—one for readmission, another for sepsis, etc. This approach is fragmented and often fails to capture the holistic "patient journey." Furthermore, pediatric populations are frequently underrepresented in large-scale medical AI research.

The authors tackle these gaps by building a generative medical event foundation model. Instead of training for a specific outcome, the model learns to predict the "next event" in a patient's timeline (diagnoses, labs, vitals, medications). This allows a single model to perform a wide variety of clinical predictions "zero-shot" by simply simulating possible patient futures and averaging the results.

Methods

The model represents a patient's chart as a sequence of discrete tokens. Key methodological innovations include:

  • Scaling Law Optimization: The team performed a Chinchilla-style scaling analysis to determine the compute-optimal balance between model size and training data. They found an optimal point at roughly 75 tokens per parameter (TPP), a massive improvement in sample efficiency over their previous iteration (~1,000 TPP).
  • Two-Stage Training: Initial pretraining occurred at an 8,192-token context window, followed by a "context extension" phase that doubled the window to 16,384 tokens. This allows the model to "remember" much longer patient histories.
  • Refined Tokenization: To better handle numerical data, they moved from simple decile-based bucketing for labs to a z-score-based system. They also introduced specialized tokens for pediatric milestones (birth weight, gestational age) and explicit markers for death and "right-censoring" (when a patient simply stops appearing in the record).
The authors used scaling laws to find the compute-optimal model size and token count.

Datasets

The model was trained on Epic Cosmos, one of the world's largest de-identified Electronic Health Record (EHR) datasets.

  • Training/Test Set: 191 million unique patients.
  • Pediatric Focus: Includes 49 million patients under 18, with 14 million having documented birth encounters.
  • Scope: The dataset spans billions of encounters, diagnoses, and lab results, providing a high-fidelity map of real-world clinical practice.

Key Insights and Ideas

  • Generative Simulation as Prediction: By treating clinical prediction as a sequence generation problem, the model can estimate the probability of any future event without being explicitly trained for it.
  • Pediatric Continuity: Training a single model across all ages allows for better modeling of the transition from pediatric to adult care, rather than having disjointed age-specific models.
  • Sample Efficiency Matters: The jump from 1B to 4.3B parameters, combined with better hyperparameter selection, allowed the model to achieve superior performance while processing 56% fewer training tokens than its predecessor.
Comparison of training budgets between the 1B and 4.3B models.

Key Results

  • Laboratory Prediction: The new z-score tokenization led to a dramatic 18–46% reduction in Mean Absolute Error (MAE) for predicting values like HgbA1c and cholesterol.
  • Mortality Prediction: The model achieved high accuracy across settings, with AUROCs of 0.821 to 0.942 for predicting death in hospice, inpatient, and ambulatory cohorts.
  • Pediatric Performance: Zero-shot AUROCs for new asthma diagnoses (0.742–0.785) and exacerbations (0.829–0.852) in children were comparable to adult reference cohorts.
  • Readmission: For adult 30-day readmission, the 4.3B model reached an AUROC of 0.799, outperforming both the previous 1B model (0.772) and a supervised baseline (0.719).

Limitations and Future Work

Despite the impressive scale, the authors note several hurdles:

  • Documentation Bias: EHR data only captures what is recorded. Deaths occurring outside the health system (e.g., at home) may be missed, leading to "under-ascertainment."
  • Calibration: While the model is discriminatively accurate (high AUROC), further work is needed to ensure the predicted probabilities are well-calibrated for individual clinical risk in local settings.
  • Causality: The model predicts observed events. It does not currently estimate the causal effect of a specific treatment—it only predicts what is likely to happen given the current trajectory.
  • Inference Costs: Generating thousands of simulations per patient to reach a prediction is computationally expensive compared to traditional "one-pass" classifiers.

Summary updated

Abstract

We introduce Curiosity-0826-4B, an updated version of the generative medical event foundation model trained on Epic Cosmos that simulates patient timelines to produce zero-shot clinical predictions. Curiosity-0826-4B incorporates a broader patient population and richer clinical event representations, while generally matching or exceeding Curiosity-0226-1B, the largest model in the previous Curiosity-0226 series, on evaluated tasks supported by both models. Laboratory value prediction improved, with MAE 18--46% lower than that of Curiosity-0226-1B. We evaluated prediction of disease events across pediatric cohorts aged 2--17 years. AUROCs for new asthma diagnosis (0.742--0.785) and asthma exacerbation (0.829--0.852) were comparable to those in the adult reference cohort, while AUROCs for sickle cell crisis (0.820--0.847) were lower than in adults (0.915). Explicit representation of documented death enabled new mortality prediction tasks among adults across hospice, inpatient, and ambulatory settings (AUROC 0.821--0.942). Scaling-law analysis guided compute-optimal initial pretraining followed by continued pretraining with context extension. The resulting Curiosity-0826-4B model has more parameters (4.3B versus 1B) and a longer context window (16K versus 8K). It processes 56% fewer training tokens and requires 2.2 times the training floating point operations (FLOPs) compared with its predecessor. By extending a common zero-shot prediction strategy to pediatric patients and additional outcomes, Curiosity-0826-4B broadens opportunities to develop clinical and operational applications without training separate task-specific models. Further validation should assess local calibration, performance across patient subgroups and event frequencies, timeline completion rates, and generalization to new settings, alongside prospective evaluation of clinical and operational utility.
BibTeX
@misc{nagarathnam2026curiosity,
  author = {Mani Nagarathnam and Luke Benham and Paul Blazek and John Cashy and Kevin Chung and John Muenning and Hank Voeller and Shane Waxler and Davis White and Alison Callahan and Kevin Dysart and Andrew Loza and Michal A. Mankowski and Mattheus Ramsis and Nigam H. Shah and Karen K. Wong and Rahul Shah},
  title = {{Curiosity-0826-4B: Generative Medical Event Sequence Modeling Across Pediatric and Adult Populations}},
  year = {2026},
  eprint = {2611.15710},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.15710},
}
Figure