Optimal model averaging forecasting in high-dimensional survival analysis

Xiaodong Yan, Hongni Wang, Wei Wang, Jinhan Xie, Yanyan Ren, Xinjun WangInternational Journal of Forecasting 2016 · arXiv

TL;DR

This paper introduces a robust two-step framework for predicting survival outcomes in ultrahigh-dimensional settings (where features vastly outnumber samples). By combining a novel feature screening method that handles complex censoring with a "weight-relaxed" model averaging approach, the authors achieve significantly higher forecasting accuracy than traditional penalized regression methods like Lasso or SCAD.

What Problem Does This Paper Address?

In fields like genomics and finance, researchers often try to predict a "survival time" (e.g., time until a patient relapses or a loan defaults). This is challenging for two reasons:

  1. The Curse of Dimensionality: We often have thousands of predictors (like gene expressions) but only a few dozen or hundred observations.
  2. Censoring: In survival studies, the event of interest might not happen before the study ends. Crucially, most existing methods assume that this "censoring" is independent of the predictors—an assumption that rarely holds in the real world.

Existing "model selection" techniques try to find the single best model, but this often ignores the uncertainty inherent in high-dimensional data. This paper tackles these gaps by proposing a method that handles covariate-dependent censoring and uses model averaging to provide more stable, accurate predictions.

Methods

The authors propose a two-step procedure:

Step 1: The Marginal Buckley-James Filter Standard feature screening methods often fail when censoring depends on the covariates. The authors developed a new screening index based on the Buckley-James (BJ) estimator. This method ranks all $p$ predictors and selects the top $q$ "active" ones. These selected predictors are then partitioned into $K$ groups to form a set of candidate models.

Step 2: Optimal Model Averaging via Delete-One CV Instead of picking the "best" candidate model, the authors calculate a weighted average of all $K$ models.

  • The Innovation: Traditionally, model weights are constrained to sum to one. The authors remove this constraint, arguing that in high-dimensional settings where models might be uncorrelated, a relaxed weight structure allows for better recovery of the true conditional mean.
  • The Criterion: They adapt a "delete-one cross-validation" (CV) criterion specifically for survival data to find the optimal weights that minimize forecasting loss.

Datasets

The researchers evaluated their method using both synthetic and real-world data:

  • Simulated Data: A high-dimensional setup with $n=200$ subjects and $p=2000$ predictors. They tested two censoring proportions (45% and 65%) to see how the model handles missing information.
  • Mantle Cell Lymphoma (MCL) Microarray Data: A real-world dataset containing expression values for 6,312 genes across 92 patients. The goal was to identify genes influencing survival and predict health risks.

Key Insights and Ideas

  • Censoring is not "Noise": By using the Buckley-James index, the authors treat censoring as a structured part of the model rather than a random nuisance. This allows the screening process to remain consistent even when the "missingness" of data is tied to the features themselves.
  • The Power of Relaxation: The "aha moment" here is the removal of the $\sum w_k = 1$ constraint. In traditional low-dimensional stats, this constraint is a safety rail. In high-dimensional survival analysis, it's a bottleneck. Relaxing it allows the model to better account for noise variance and uncorrelated predictors.
  • Sure Screening Consistency: The authors provide mathematical proof that their screening method has a high probability of retaining all truly "active" predictors, even as the number of features grows exponentially relative to the sample size.

Key Results

  • Superior Screening: In simulations, the proposed BJS filter identified the correct predictors with a median model size of 3 (at 45% censoring), while competing methods like SIRScen or QAScen required much larger, noisier models (sizes of 5 to 25) to capture the same information.
  • Lower Prediction Error: In the MCL gene study, the proposed model averaging method (MCV3) achieved the lowest Averaged Squared Forecasting Error (ASPE) compared to heavyweights like Lasso, SCAD, and MCP.
  • Robustness to Censoring: Even when the censoring rate was bumped up to 65%, the proposed method maintained a significantly lower Mean Squared Error (MSE) than traditional penalized regression techniques.

Limitations and Future Work

  • Model Scope: The current work focuses on semiparametric linear regression (Accelerated Failure Time models). The authors note that extending this to the Cox proportional hazards model or additive hazards models is a necessary next step.
  • Hyperparameter Tuning: While the method is powerful, the rigorous determination of the optimal number of candidate groups ($K$) and the specific constants used in the screening threshold requires further investigation.
  • Computational Intensity: While screening helps, performing delete-one cross-validation on many candidate models can be computationally demanding as $n$ and $K$ grow.

Summary updated

Abstract

This article considers ultrahigh-dimensional forecasting problems with survival response variables. We propose a two-step model averaging procedure for improving the forecasting accuracy of the true conditional mean of a survival response variable. The first step is to construct a class of candidate models, each with low-dimensional covariates. For this, a feature screening procedure is developed to separate the active and inactive predictors through a marginal BuckleyCJames index, and to group covariates with a similar index size together to form regression models with survival response variables. The proposed screening method can select active predictors under covariate-dependent censoring, and enjoys sure screening consistency under mild regularity conditions. The second step is to find the optimal model weights for averaging by adapting a delete-one cross-validation criterion, without the standard constraint that the weights sum to one. The theoretical results show that the delete-one cross-validation criterion achieves the lowest possible forecasting loss asymptotically. Numerical studies demonstrate the superior performance of the proposed variable screening and model averaging procedures over existing methods.
BibTeX
@article{yan2016optimal,
  author = {Xiaodong Yan and Hongni Wang and Wei Wang and Jinhan Xie and Yanyan Ren and Xinjun Wang},
  title = {{Optimal model averaging forecasting in high-dimensional survival analysis}},
  journal = {International Journal of Forecasting},
  year = {2016},
}
Figure