An optimal dynamic treatment regime estimator for indefinite-horizon survival outcomes

Junjun She, Matthew Egberg, Michael R. KosorokarXiv 2025 · DOI · arXiv

TL;DR

Researchers have developed a novel method for estimating optimal Dynamic Treatment Regimes (DTRs) for chronic diseases where the number of treatment stages is unpredictable (indefinite-horizon). By combining generalized survival random forests with a recursive model-refitting algorithm, the approach maximizes mean survival time while handling complex issues like dependent censoring and varying visit lengths.

What Problem Does This Paper Address?

In precision medicine, a Dynamic Treatment Regime (DTR) is a set of rules that tells a clinician which treatment to give a patient based on their evolving health history. While DTRs are well-studied for "finite horizons" (a fixed number of visits), they struggle with chronic conditions like Crohn’s disease or diabetes, where patients have a varying number of visits over an unpredictable timeframe—an indefinite-horizon setting.

Existing methods for survival data (time-to-event outcomes) often fall into two traps:

  1. They assume a fixed number of stages, making them inflexible for real-world clinical paths.
  2. they fit separate models for every visit, which causes errors to compound as the number of stages grows.

Furthermore, these models must account for censoring—when a patient drops out of a study or the study ends before an event occurs. If that dropout is related to the patient's health (dependent censoring), standard statistical tools often break down.

Methods

The authors propose a method called IHsurvrf (Indefinite Horizon Survival Random Forests). Its core innovation lies in how it structures the learning process:

  • Generalized Survival Random Forests: The method uses non-parametric random forests to estimate survival curves without assuming a specific underlying distribution (like the "bell curve").
  • Recursive Model Refitting: Instead of training a new model for every treatment stage, the algorithm uses a single model that is iteratively "refit." It starts at the final visit and works backward, using a technique called probabilistic augmentation to carry information about future survival back to earlier visits.
  • Data Pooling and Summarized History: To prevent the data from becoming "too wide" as a patient’s history grows, the authors use a summarized version of the patient's past. They pool data across all patients and all visits into a single training set, which stabilizes the estimator.
  • Temporal Strata: The algorithm allows for dividing patients into "strata" based on how long they have been in treatment, allowing the model to adapt to different patient trajectories (e.g., short-term vs. long-term survivors).

Datasets

The paper evaluates the method using two primary sources:

  1. Simulation Studies: Synthetic datasets designed to mimic clinical scenarios with 10 and 15 treatment visits and varying levels of censoring (moderate ~28% to high ~50%).
  2. Pediatric Crohn’s Disease Data: Real-world administrative insurance claims from the IQVIA Legacy PharMetric database. This included a cohort of 418 pediatric patients who underwent intestinal resection. The goal was to maximize "hospital-free survival" (time-to-second-hospitalization).

Key Insights and Ideas

  • The "Single Model" Advantage: By refitting a single model rather than training many, the authors significantly reduce the risk of compounding errors. This makes the approach much more robust for long-term chronic disease management.
  • Handling the "Indefinite" Nature: The use of summarized history ($H_{k,i}$) is a clever "dimensionality hack." It ensures that the model can process a patient at their 2nd visit or their 20th visit using the same mathematical framework.
  • Consistency Proofs: The paper provides rigorous mathematical proofs showing that as the sample size grows, the estimated treatment regime and the resulting survival values converge to the true optimal values.
  • Overfitting Risks with Strata: Interestingly, the simulations showed that a single-strata model sometimes performed slightly better than a two-strata model. This suggests that while stratification adds flexibility, it can lead to overfitting if the data is split too thinly.

Key Results

  • Superior Performance: In all simulation settings, the proposed IHsurvrf method outperformed the "observed policy" (the standard treatment path used to generate the data).
  • Clinical Impact: In the Crohn’s disease application, the estimated optimal regime extended the predicted time-to-second-hospitalization by approximately 30 days compared to what patients actually received in the 2-strata case.
  • Hyperparameter Sensitivity: The model's performance is sensitive to the number of trees in the forest. For the Crohn’s data, performance peaked at 100 trees for the 2-strata model and 600 trees for the 1-strata model.

Limitations and Future Work

  • Computational Cost: The recursive refitting process is computationally expensive. Training forests repeatedly over many iterations and large datasets requires significant time and processing power.
  • Value Function Estimation: The authors note that estimating the value of a regime in an indefinite-horizon setting is an open research challenge. Standard techniques like IPCW (Inverse Probability of Censoring Weighting) become unreliable when there are many treatment stages because it becomes mathematically unlikely for any observed patient to perfectly match the "optimal" path for a long duration.
  • Future Directions: The authors suggest exploring faster ways to perform survival probability augmentation (e.g., using single random draws) to speed up the algorithm. They also call for the development of new "proportion-based matching" estimators to better evaluate these regimes in the future.

Summary updated

Abstract

We propose a new method in indefinite-horizon settings for estimating optimal dynamic treatment regimes for time-to-event outcomes. This method allows patients to have different numbers of treatment stages and is constructed using generalized survival random forests to maximize mean survival time. We use summarized history and data pooling, preventing data from growing in dimension as a patient's decision points increase. The algorithm operates through model re-fitting, resulting in a single model optimized for all patients and all stages. We derive theoretical properties of the estimator such as consistency of the estimator and value function and characterize the number of refitting iterations needed. We also conduct a simulation study of patients with a flexible number of treatment stages to examine finite-sample performance of the estimator. Finally, we illustrate use of the algorithm using administrative insurance claims data for pediatric Crohn's disease patients.
BibTeX
@misc{she2025optimal,
  author = {Junjun She and Matthew Egberg and Michael R. Kosorok},
  title = {{An optimal dynamic treatment regime estimator for indefinite-horizon survival outcomes}},
  year = {2025},
  eprint = {2501.18070},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.18070},
  howpublished = {arXiv},
}
Figure