Variational Inference for Gaussian Process Models for Survival Analysis

Minyoung Kim, Vladimir PavlovicUAI 2018 · OpenReview

Abstract

Survival analysis, the study of time-to-event data, is often hampered by censoring and the restrictive proportional hazards assumption of standard models like CoxPH. While Gaussian Process (GP) models offer a flexible non-parametric alternative, their traditional reliance on Markov Chain Monte Carlo (MCMC) sampling makes them computationally prohibitive for large datasets. This paper introduces scalable variational inference (VI) algorithms for GP Survival Analysis (GPSAM). By employing pseudo-inputs and random feature expansions, the proposed methods achieve an order-of-magnitude speedup while simultaneously improving predictive accuracy on clinical and social science datasets.

Introduction

Survival analysis aims to predict the time until a specific event occurs. Two primary challenges characterize this field: censoring, where the event is not observed for all subjects, and model rigidity. The industry-standard Cox Proportional Hazard (CoxPH) model assumes that risk factors are constant over time, an assumption frequently violated in real-world scenarios. Gaussian Process (GP) models provide the flexibility to capture complex, time-varying risks and offer inherent uncertainty estimates. However, traditional GP survival models utilize MCMC sampling, which scales poorly ($O(N^3)$) and converges slowly, limiting their practical application.

Methods

To address these scalability issues, two novel variational inference strategies are proposed:

  1. Pseudo-input Approximation ($VI_{PI}$): This method utilizes a small set of $M$ "inducing points" to represent the GP, reducing the computational complexity from $O(N^3)$ to $O(NM^2)$.
  2. Random Feature Expansion ($VI_{RF}$): This approach approximates the GP kernel using a finite-dimensional feature map, transforming the non-parametric problem into a manageable parametric regression in a high-dimensional space.

Several technical innovations facilitate these methods. The authors replace the traditional sigmoid link function with a square non-negative link function ($g(y) = y^2$), enabling more analytic derivations. Furthermore, the "reparameterization trick" is employed for efficient gradient-based optimization of variational parameters. Both "full" and "partial" likelihood formulations were developed for these schemes.

Results and Discussion

The proposed models were evaluated on synthetic data, clinical datasets (Veteran’s and Mayo Lung Cancer), and a large-scale social science dataset (Divorce).

Computational Efficiency

The VI approaches demonstrate a massive speedup over MCMC. On the Mayo Lung Cancer (MLC) dataset, $VI_{PI}$ required approximately 572 seconds, a 14x improvement over the 8,121 seconds required by MCMC-based GPSAM. As shown in Figure 1, the computational advantage of VI methods becomes more pronounced as model complexity increases.

Computational time comparison between MCMC and variational inference methods as model complexity increases.

Predictive Accuracy

Variational approximations not only improve speed but also enhance accuracy by allowing for more reliable convergence within practical timeframes. In synthetic tests involving non-proportional hazards, the $VI_{PI}$ model achieved a C-index of 83.04%, significantly outperforming both MCMC (77.20%) and standard CoxPH (73.43%).

Performance comparison of various survival analysis models on synthetic non-proportional hazard data.

The models also proved robust to high censoring rates, as seen in the Divorce dataset (70% censoring), effectively leveraging information from incomplete samples. The use of a square link function allowed for analytic solutions to parts of the expected log-likelihood, further stabilizing training.

Conclusion

This work demonstrates that variational inference makes Gaussian Process survival analysis both practical and superior to traditional methods. By breaking the proportional hazard assumption and providing scalable computation, these methods offer a powerful tool for complex time-to-event modeling. Future work includes refining inducing point selection and exploring more complex kernel structures to further enhance performance.

Summary updated

BibTeX
@inproceedings{kim2018variational,
  author = {Minyoung Kim and Vladimir Pavlovic},
  title = {{Variational Inference for Gaussian Process Models for Survival Analysis}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2018},
  url = {https://openreview.net/forum?id=l3PZZiLRWM},
}
Figure