The super learner for time-to-event outcomes: A tutorial

Ruth H. Keogh, Karla Díaz-Ordaz, Nan van Geloven, Jon Michael Gran, Kamaryn TannerarXiv 2025 · DOI · arXiv

Abstract

Estimating risks or survival probabilities conditional on individual characteristics is a fundamental task in clinical research and engineering. However, the presence of censored data complicates this process, and researchers often struggle to identify the optimal model for a given dataset. The "Super Learner" (SL) is a powerful ensemble learning framework designed to address this by combining multiple candidate models through cross-validation. This paper provides a practical tutorial on implementing Super Learner methods for time-to-event outcomes, comparing discrete-time and continuous-time approaches, and illustrating their application using the Rotterdam breast cancer dataset.

Introduction

In survival analysis, the goal is to predict the time until an event occurs, such as a patient's relapse or a mechanical failure. This task is uniquely challenging due to censoring, where the event of interest is not observed for all individuals within the study period. While the Cox Proportional Hazards model has long been the default choice, the rise of machine learning has introduced alternatives like Random Survival Forests and Deep Learning.

Because no single model is universally superior across all datasets, researchers often face an ad-hoc selection process. The Super Learner framework—a "stacking" algorithm—systematizes this choice by mathematically blending models. Despite its utility, the literature on SL for survival data has historically been technical and fragmented. This tutorial aims to bridge that gap, offering a unified guide to the methodology and its practical implementation.

The Super Learner Framework

The Super Learner algorithm operates by creating a "library" of candidate learners, which can include both simple parametric models (e.g., Cox models) and complex non-parametric algorithms (e.g., XGBoost). The process follows several key steps:

  1. V-fold Cross-Validation: The training data is split into $V$ blocks. Each model in the library is trained on $V-1$ blocks and used to predict outcomes for the remaining block.
  2. Weight Assignment: The algorithm calculates a loss function (such as the Brier score) for each model's cross-validated predictions. It then determines the optimal combination of weights that minimizes the overall risk.
  3. The Oracle Property: A defining feature of the Super Learner is its mathematical guarantee. As the sample size increases, the SL is proven to perform as well as the best possible combination of models in the library, effectively shielding the researcher from the risk of choosing a sub-optimal model.

Specific Implementations for Survival Data

The paper focuses on three primary implementations adapted for time-to-event data:

  • Discrete-Time SL (Polley & van der Laan): This method discretizes the time scale into specific intervals, transforming the survival problem into a series of binary classification tasks. While flexible, it can become computationally intensive as the dataset grows "long" (one row per person per time interval).
  • Continuous-Time SL (Westling et al.): This more recent approach avoids discretization. It utilizes an iterative process to estimate both the survival distribution and the censoring distribution, optimizing a weighted integrated Brier score to handle the nuisance of censoring.
  • Joint Survival SL (Munch & Gerds): This method focuses on state occupation probabilities. In this tutorial, it is discussed primarily as a "selector" to identify the best-performing pair of event and censoring models.

Case Study: The Rotterdam Breast Cancer Dataset

To illustrate these methods, the authors utilized the Rotterdam breast cancer dataset, consisting of 2,982 individuals who underwent surgery. The goal was to predict conditional survival probability up to 10 years based on features such as age, tumor size, and hormonal treatment. The data was split into a 70% training set and a 30% external testing set.

Key Results

  • Model Performance: In individual testing, Random Survival Forests emerged as the strongest single learner (Brier score: 0.196, C-index: 71.7), while the standard Cox model performed the worst.
  • Ensemble Advantage: The continuous-time Super Learner matched the performance of the best individual model and slightly improved discrimination (C-index: 72.0). The SL consistently placed the highest weights on the most robust learners, such as Random Forests and Lasso-regularized GLMs.
  • The Pitfalls of Discretization: The study found that in discrete-time SL, using too many time intervals (e.g., 100 vs. 10) actually degraded the Brier score (0.212 vs. 0.204). This occurs because very small windows make it difficult to accurately model the baseline hazard.

Practical Insights

  1. Library Diversity is Essential: The Super Learner thrives on a "rich" library. Including a mix of simple and complex models ensures the ensemble can capture both linear and non-linear relationships. Including multiple versions of the same model (e.g., ten slightly different Cox models) provides little benefit.
  2. Weighting vs. Selection: The "Ensemble SL" (which weights multiple models) generally outperforms the "Discrete SL" (which simply picks the single best model). Weighting helps smooth out prediction errors that any single model might produce.
  3. Modeling the Nuisance: Censoring is a nuisance parameter that must be accounted for. Continuous-time SLs are particularly effective because they can use a separate Super Learner to model the censoring distribution itself, leading to more accurate survival weights.

Limitations and Future Directions

While powerful, the Super Learner for survival data faces several hurdles. Computational cost remains a concern, particularly for discrete-time methods on large cohorts. Additionally, current implementations often lack support for left-truncated data, where individuals enter a study after the initial time-zero. Finally, while R packages like SuperLearner and survSuperLearner are available, the software ecosystem is not yet as unified as general machine learning libraries like scikit-learn, often requiring custom wrappers for specific models.

Conclusion

The Super Learner provides a principled, data-driven solution to model selection in survival analysis. By leveraging cross-validation and a diverse library of learners, it offers a robust framework for risk prediction that frequently outperforms individual statistical or machine learning models.

Summary updated

Abstract

Estimating risks or survival probabilities conditional on individual characteristics based on censored time-to-event data is a commonly faced task. This may be for the purpose of developing a prediction model or may be part of a wider estimation procedure, such as in causal inference. A challenge is that it is impossible to know at the outset which of a set of candidate models will provide the best risk estimates. The super learner is a powerful approach for finding the best model or combination of models ('ensemble') among a pre-specified set of candidate models or 'learners', which can include both 'statistical' models (e.g. parametric, semi-parametric models) and 'machine learning' models. Super learners for time-to-event outcomes have been developed, but the literature is technical and the full details of how these methods work and can be implemented in practice have not previously been presented in an accessible format. In this paper we provide a practical tutorial on super learner methods for time-to-event outcomes. An overview of the general steps involved in the super learner is given, followed by details of three specific implementations for time-to-event outcomes. These include the originally proposed super learner, which involves using a discrete time scale, and two more recently proposed versions of the super learner for continuous-time data. We compare the properties of the methods and provide information on how they can be implemented in R. The methods are illustrated using an open access data set and R code is provided.
BibTeX
@misc{keogh2025super,
  author = {Ruth H. Keogh and Karla Díaz-Ordaz and Nan van Geloven and Jon Michael Gran and Kamaryn Tanner},
  title = {{The super learner for time-to-event outcomes: A tutorial}},
  year = {2025},
  eprint = {2509.03315},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.03315},
}
Figure