Kullback-Leibler-based discrete failure time models for integration of published prediction models with new time-to-event dataset

Di Wang, Wen Ye, Randall Sung, Hui Jiang, Jeremy M. G. Taylor, Lisa Ly, Kevin HeThe Annals of Applied Statistics 2021

Abstract

Predicting time-to-event outcomes in clinical settings often involves a trade-off between the breadth of large-scale registries and the depth of local clinical datasets. While local studies offer granular biomarkers, they frequently suffer from small sample sizes and low signal-to-noise ratios. This paper introduces a novel framework using discrete hazard-based Kullback-Leibler (KL) divergence to integrate external "wisdom" from published models into local survival analyses. By employing a penalized log-likelihood approach and coordinate-wise gradient boosting, the proposed method improves prediction stability and accuracy even in high-dimensional settings.

Introduction

In medical research, the "Goldilocks" problem of data is pervasive. National registries provide the "Big Data" necessary for statistical power but often lack the specific clinical nuances required for personalized medicine. Conversely, local hospital studies capture rich, high-dimensional data but are often too small to produce reliable predictive models. This discrepancy leads to models that are either too generic or too unstable to be clinically useful.

Wang et al. (2022) address this gap by proposing a mathematical bridge that "anchors" small-scale survival models to established, large-scale published models. Their research, published in Biometrics, introduces a framework to integrate these two worlds using information theory, specifically Kullback-Leibler (KL) divergence, to allow local models to learn from external evidence without being overwhelmed by it.

Methodology

The core of the proposed framework is a KL-based discrete failure time modeling procedure. This approach departs from traditional continuous-time survival analysis in three significant ways:

1. Discrete Hazard Focus

Rather than treating time as a continuous variable, the model partitions time into discrete intervals (e.g., months or years). By focusing on the discrete hazard function—the probability of an event occurring within a specific window given survival up to that point—the researchers can treat survival data as a sequence of longitudinal binary outcomes. This transformation allows for the application of information-theoretic tools that are typically difficult to use with censored survival data.

2. The Information Bridge: KL Divergence

The researchers introduce a penalized log-likelihood function that incorporates a KL divergence term. This term measures the discrepancy between the local model's hazard predictions and those of a pre-existing, published model. A tuning parameter, $\eta$, acts as a "trust" mechanism. If the external model is highly relevant to the local population, $\eta$ is increased; if the external model is a poor fit due to population heterogeneity, the system automatically reduces $\eta$ toward zero, ensuring the local data remains the primary driver of the final model.

3. High-Dimensional Scaling

To accommodate modern datasets with hundreds of potential risk factors, the method is extended via a coordinate-wise gradient boosting algorithm. This allows for automated variable selection, using the external model as a guide to distinguish true signals from random noise in small samples. This is particularly effective in high-dimensional settings where the number of predictors exceeds the number of events.

Empirical Evaluation

The framework was validated through both extensive simulations and a real-world application in transplant medicine.

Simulation Studies

The authors tested the model against synthetic datasets ranging from 10 to 400 predictors. These simulations demonstrated that the KL-based method consistently achieved lower predictive deviance and higher stability (narrower interquantile ranges) than models relying solely on local data, the external model alone, or simple "stacking" techniques.

Case Study: Kidney Graft Survival

The researchers applied their method to a dataset of 498 kidney transplant patients from the University of Michigan Medical Center. They integrated this local data with the Scientific Registry of Transplant Recipients (SRTR) model, a national benchmark utilizing 59 covariates to predict graft survival.

The results were striking:

  • Predictive Accuracy: The KL-integrated model achieved a predictive deviance of 13.56, vastly outperforming the standard survival model built on local data alone, which scored 44.40.
  • Clinical Insight: While the national SRTR model provides a broad overview, the integrated model identified liver disease as a significant risk factor for this specific local cohort—a nuance that was emphasized through the integration process but might have been lost in a purely national or purely local analysis.

Discussion and Limitations

The strength of this approach lies in its robustness. By using cross-validation to select the tuning parameter $\eta$, the model protects against "negative transfer"—where a poorly fitting external model degrades the performance of the local model. If a published model is irrelevant, the framework naturally defaults to the local evidence.

However, some limitations remain. The requirement for discrete time intervals may lead to a loss of precision in certain clinical contexts compared to continuous-time models. Furthermore, the method requires that the published external model provides sufficient detail (such as hazard functions or survival probabilities) to allow for the calculation of KL divergence, which is not always available in summary-level literature.

Conclusion

The KL-based discrete relative risk model represents a significant step forward in evidence synthesis for survival analysis. By providing a formal mechanism to integrate "big data" insights into "small data" clinical environments, this framework enables researchers to build more accurate, stable, and clinically relevant prediction models that respect both global trends and local realities.

Reference

Wang, D., Ye, W., Sung, R., Jiang, H., Taylor, J. M. G., Ly, L., & He, K. (2022). Kullback-Leibler-Based Discrete Failure Time Models for Integration of Published Prediction Models with New Time-To-Event Dataset. Biometrics. arXiv:2101.02354v2

Summary updated

Abstract

Prediction of time-to-event data often suffers from rare event rates, small sample sizes, high dimensionality, and low signal-to-noise ratios. Incorporating published prediction models from external large-scale studies is expected to improve the performance of prognosis prediction from internal individual-level data. However, existing integration approaches typically assume that the underlying distributions of the external and internal data sources are similar, which is often invalid. To account for challenges, including heterogeneity, data sharing, and privacy constraints, we propose a failure time integration procedure, which utilizes a discrete hazard-based Kullback-Leibler discriminatory information measuring the discrepancy between the external models and the internal dataset. The asymptotic properties and simulation results show the advantage of the proposed method compared to those solely based on internal data. We apply the proposed method to improve prediction performance on a kidney transplant dataset from a local hospital by integrating this small-sized dataset with a published survival model obtained from the national transplant registry.
BibTeX
@article{wang2021kullback,
  author = {Di Wang and Wen Ye and Randall Sung and Hui Jiang and Jeremy M. G. Taylor and Lisa Ly and Kevin He},
  title = {{Kullback-Leibler-based discrete failure time models for integration of published prediction models with new time-to-event dataset}},
  journal = {The Annals of Applied Statistics},
  year = {2021},
}
Figure