Survival Regression with Proper Scoring Rules and Monotonic Neural Networks

David Rindt, Robert Hu, David Steinsaltz, Dino Sejdinović

Abstract

We consider frequently used scoring rules for right-censored survival regression models such as time-dependent concordance, survival-CRPS, integrated Brier score and integrated binomial log-likelihood, and prove that neither of them is a proper scoring rule. This means that the true survival distribution may be scored worse than incorrect distributions, leading to inaccurate estimation. We prove that, in contrast to these scores, the right-censored log-likelihood is a proper scoring rule, i.e., the highest expected score is achieved by the true distribution. Despite this, modern feed-forward neural-network-based survival regression models are unable to train and validate directly on the right-censored log-likelihood, due to its intractability, and resort to the aforementioned alternatives, i.e., non-proper scoring rules. We therefore propose a simple novel survival regression method capable of directly optimizing log-likelihood using a monotonic restriction on the time-dependent weights, coined SurvivalMonotonic-net (SuMo-net). SuMo-net achieves state-of-the-art log-likelihood scores across several datasets with 20–100$\times$ computational speedup on inference over existing state-of-the-art neural methods, and is readily applicable to datasets with several million observations.

Introduction

Survival analysis is fundamental to fields ranging from medicine to engineering, where the goal is to predict the time until an event of interest occurs. A defining characteristic of this domain is right-censoring, where the event is not observed for all subjects within the study period. While various metrics have been developed to evaluate survival models under censoring, this work identifies a critical theoretical flaw: many of the most popular metrics are not "proper scoring rules."

The Pitfalls of Improper Scoring Rules

A scoring rule is considered "proper" if the maximum expected score is uniquely obtained by the true probability distribution. The authors provide a theoretical proof that common metrics—including Time-Dependent Concordance ($C^{td}$), the Brier Score, and the Continuous Ranked Probability Score (CRPS)—fail this criterion for censored data.

As illustrated in Figure 1, an improper metric like $C^{td}$ can actually favor a "false" distribution over the "true" one. This discrepancy suggests that models optimized for these metrics may converge to inaccurate representations of the underlying survival process. Conversely, the authors demonstrate that the right-censored log-likelihood is a proper scoring rule, providing a theoretically sound objective for model training and evaluation.

Comparison of True vs. False distributions showing how improper metrics can favor incorrect models.

SuMo-net: Direct Likelihood Optimization

To address the challenge of optimizing the log-likelihood directly, the authors propose SuMo-net (Survival Monotonic-net). A valid survival function $S(t|x)$ must be non-increasing with respect to time $t$. SuMo-net enforces this property through its architecture rather than through external penalties.

As shown in Figure 2, the network utilizes a monotonic restriction where weights ($w_{ij} \in \mathbb{R}^+$) on paths originating from the time input $t$ are constrained to be non-negative. By leveraging automatic differentiation, SuMo-net can compute both the survival function $\hat{S}(t|x)$ and the probability density function $\hat{f}(t|x)$ simultaneously. This allows the model to be trained directly on the proper log-likelihood score without requiring time discretization or computationally expensive numerical integration.

SuMo-net architecture with monotonic weight constraints.

Experimental Results

The performance of SuMo-net was evaluated across several real-world datasets, including SUPPORT, METABRIC, and the large-scale KKBOX dataset (2.6 million observations).

Key findings include:

  • State-of-the-Art Accuracy: SuMo-net consistently achieved the highest log-likelihood scores, outperforming established neural methods like DeepHit and Cox-Time.
  • Efficiency and Scalability: SuMo-net offers a 20–100$\times$ speedup in inference compared to methods like Cox-Time, which require integrating a hazard function. Furthermore, its architecture allows it to scale to millions of observations where NeuralODE-based methods often fail.
  • Improved Calibration: The model demonstrated superior calibration, ensuring that predicted survival probabilities align closely with observed event frequencies.

Conclusion

By identifying the impropriety of standard survival metrics and introducing SuMo-net, this research provides both a theoretical correction and a practical, high-performance tool for survival analysis. The shift toward direct log-likelihood optimization via monotonic networks ensures more reliable predictions and significantly faster computational performance.

Summary updated

Abstract

We consider frequently used scoring rules for right-censored survival regression models such as time-dependent concordance, survival-CRPS, integrated Brier score and integrated binomial log-likelihood, and prove that neither of them is a proper scoring rule. This means that the true survival distribution may be scored worse than incorrect distributions, leading to inaccurate estimation. We prove that, in contrast to these scores, the right-censored log-likelihood is a proper scoring rule, i.e., the highest expected score is achieved by the true distribution. Despite this, modern feed-forward neural-network-based survival regression models are unable to train and validate directly on the right-censored log-likelihood, due to its intractability, and resort to the aforementioned alternatives, i.e., non-proper scoring rules. We therefore propose a simple novel survival regression method capable of directly optimizing log-likelihood using a monotonic restriction on the time-dependent weights, coined SurvivalMonotonic-net (SuMo-net). SuMo-net achieves state-of-the-art log-likelihood scores across several datasets with 20--100$\times$ computational speedup on inference over existing state-of-the-art neural methods, and is readily applicable to datasets with several million observations.
BibTeX
@inproceedings{rindt2022survival,
  author = {David Rindt and Robert Hu and David Steinsaltz and Dino Sejdinović},
  title = {{Survival Regression with Proper Scoring Rules and Monotonic Neural Networks}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2022},
  url = {https://openreview.net/forum?id=KBWUUhVfT4},
}
Figure