CleanSurvival: Automated data preprocessing for time-to-event models using reinforcement learning

Yousef Koka, David Antony Selby, Gerrit Großmann, Sebastian J. Vollmer

TL;DR

This paper introduces CleanSurvival, an automated data preprocessing framework for time-to-event (survival) analysis that leverages reinforcement learning (specifically Q-learning). It addresses the critical gap in AutoML for survival models by dynamically optimizing preprocessing pipelines (imputation, outlier detection, feature selection) to improve predictive performance and robustness, outperforming standard methods.

What Problem Does This Paper Address?

Data preprocessing is a crucial step in machine learning, significantly impacting model performance. While Automated Machine Learning (AutoML) has made strides in integrating preprocessing for classification and regression, it has largely overlooked specialized areas like survival analysis. Survival analysis, which deals with time-to-event data (e.g., patient survival, equipment failure), presents unique challenges like censored data. The lack of automated, tailored preprocessing solutions for survival models means researchers often rely on manual, time-consuming, and potentially suboptimal methods, hindering the reliability and efficiency of survival studies.

Methods

CleanSurvival builds upon the Learn2Clean framework and employs Q-learning, a model-free reinforcement learning algorithm. The core idea is to train an agent to learn an optimal sequence of data preprocessing steps. The agent interacts with the data and a survival model, receiving rewards based on the model's predictive performance (specifically, the C-index). This allows the agent to dynamically select and order preprocessing techniques like imputation (complete case analysis, mean/median imputation, multiple imputation, KNN imputation), outlier detection (Elliptic Envelope, Martingale residuals), and feature selection (Univariate Cox, LASSO, RFE, Information Gain) to maximize the downstream survival model's performance. The framework supports various survival models, including Cox Proportional Hazards, Random Survival Forests, and DeepHit neural networks.

Datasets

The authors evaluated CleanSurvival on three real-world datasets:

  • Rotterdam dataset: Derived from the Rotterdam Study, a large prospective cohort study.
  • Flchain dataset: Studies the relationship between serum free light chain levels and mortality. This dataset naturally contains missing values.
  • GBSG dataset: From the German Breast Cancer Study Group.

For benchmarking, synthetic missingness was introduced into the Rotterdam and GBSG datasets at various levels (10% to 50%) and mechanisms (MCAR, MAR, MNAR) to simulate different data quality scenarios.

Key Insights and Ideas

  • Reinforcement Learning for Preprocessing: Applying RL, specifically Q-learning, to dynamically optimize preprocessing pipelines is a powerful approach for survival analysis, adapting to the unique challenges of censored data.
  • Model-Specific Optimization: Preprocessing pipelines that are optimal for one survival model (e.g., CoxPH) may differ from those optimal for another (e.g., Random Survival Forest), highlighting the need for tailored AutoML solutions.
  • Feature Selection is Key: Feature selection operators like LASSO and RFE were frequently selected by the Q-learning policy, indicating their importance in improving survival model performance.
  • Robustness to Missingness: CleanSurvival demonstrates effectiveness across different types and levels of missingness, suggesting its potential for real-world, imperfect data.

Key Results

  • Improved Predictive Performance: CleanSurvival consistently outperformed simple analyst baselines (Complete Case Analysis, Mean Imputation) and undirected random search in terms of predictive performance (C-index).
  • Efficiency: CleanSurvival found performant pipelines up to 10 times faster than undirected random grid search.
  • Stable Calibration: Integrated Brier Scores (IBS) indicated generally stable calibration across various missingness settings, with some exceptions at higher missingness levels.
  • Adaptability: The framework demonstrated its ability to learn effective pipelines for different survival models and data conditions.

Limitations and Future Work

  • Generalization to High-Dimensional/Multimodal Data: The current analysis used three datasets; generalization to higher-dimensional or multimodal clinical datasets remains unverified.
  • Computational Overhead: The current implementation can have significant computational overhead, especially when using KNN imputation and complex deep learning models repeatedly within the Q-learning loop.
  • Future Work: The authors suggest extending the framework to handle additional preprocessing tasks, incorporating advanced RL strategies, and improving scalability for larger datasets. Ensembling over differently cleaned datasets is also proposed as a way to potentially increase predictive performance. Ablation studies to isolate the benefits of specific search-space subsets are also mentioned as future work.

Summary updated

Abstract

Data preprocessing is a critical yet frequently neglected aspect of machine learning, often paid little attention despite its potentially significant impact on model performance. While automated machine learning pipelines are starting to recognize and integrate data preprocessing into their solutions for classification and regression tasks, this integration is lacking for more specialized tasks like survival or time-to-event models. As a result, survival analysis not only faces the general challenges of data preprocessing but also suffers from the lack of tailored, automated solutions in this area. To address this gap, this paper presents 'CleanSurvival', a reinforcement-learning-based solution for optimizing preprocessing pipelines, extended specifically for survival analysis. The framework can handle continuous and categorical variables, using Q-learning to select which combination of data imputation, outlier detection and feature extraction techniques achieves optimal performance for a Cox, random forest, neural network or user-supplied time-to-event model. The package is available on GitHub: https://github.com/datasciapps/CleanSurvival Experimental benchmarks on real-world datasets show that the Q-learning-based data preprocessing results in superior predictive performance to standard approaches, finding such a model up to 10 times faster than undirected random grid search. Furthermore, a simulation study demonstrates the effectiveness in different types and levels of missingness and noise in the data.
BibTeX
@misc{koka2025cleansurvival,
  author = {Yousef Koka and David Antony Selby and Gerrit Großmann and Sebastian J. Vollmer},
  title = {{CleanSurvival: Automated data preprocessing for time-to-event models using reinforcement learning}},
  year = {2025},
  eprint = {2502.03946},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.03946},
}
Figure