Abstract
This paper investigates the "double descent" phenomenon—where test error decreases again after an initial peak in overparametrized models—within the context of survival analysis. While well-documented in standard regression and classification, survival models present unique challenges due to censored data and likelihood-based loss functions. The authors prove that while common survival models can technically "interpolate" (reach zero training loss), they require infinite parameter norms to do so. This leads to generalization behaviors that range from a weak second descent to outright divergence, suggesting that overparametrization is not inherently benign in survival analysis.
Introduction
In modern machine learning, increasing model capacity often leads to improved performance. Classical statistical theory suggests a U-shaped risk curve based on the bias-variance tradeoff. However, recent research has identified a "double descent" pattern where, beyond the point of fitting training data perfectly (the interpolation threshold), test loss decreases again.
Survival analysis, however, deals with censored data—where the exact time of an event is unknown, only that it has not yet occurred. Survival models typically use likelihood-based loss functions to estimate latent structures like hazard rates. This study asks whether the double descent phenomenon persists in this specialized field and if overparametrization provides the same benefits seen in other domains.
Methods
The researchers established a rigorous theoretical framework by distinguishing between two types of interpolation:
- Interpolation: The minimum number of parameters required for training loss to reach its theoretical infimum.
- Finite-norm Interpolation: Whether that minimum can be achieved with bounded (finite) parameter values.
They analyzed four representative neural survival models: DeepSurv and PC-Hazard (continuous-time), and Nnet-Survival and N-MTLR (discrete-time). The study utilized synthetic datasets generated with a Weibull baseline hazard to maintain control over signal-to-noise ratios and ground truth hazard functions.
Key Insights and Results
The core finding of the paper is the "Infinite Norm" hurdle. The authors prove that while all four models admit interpolation, none admit finite-norm interpolation. Reaching the theoretical minimum loss requires model weights to diverge to infinity, a behavior analogous to logistic regression on perfectly separable classes.
The geometry of the loss function dictates the shape of the test loss curve in the overparametrized regime:
- Pattern A (Divergence): Discrete-time models (Nnet-Survival, N-MTLR) exhibit test loss that explodes after the interpolation threshold.
- Pattern B (Weak Descent): Continuous-time models (PC-Hazard, DeepSurv) show a "quasi-descent" where loss stabilizes or drops slightly, but remains significantly higher than the initial "sweet spot."
Specifically, PC-Hazard displayed the most canonical double-descent shape, though its second descent stabilized at a loss level much higher than the classical U-shaped minimum. DeepSurv showed a very weak second descent, likely due to "optimization stall" from flattening gradients rather than improved generalization. In contrast, discrete models failed to show any meaningful second descent, with test losses increasing sharply as capacity grew.
Conclusion
The study suggests that overparametrization is not "benign" in survival analysis. The "sweet spot" in the underparametrized regime is almost always superior to the overparametrized regime. The paper links the absence of finite-norm interpolation to a "margin-norm tradeoff," where pushing for zero training loss forces weight norms to blow up, destabilizing generalization. Future work is needed to determine if a survival model's second minimum could ever outperform the first and to validate these synthetic findings on large-scale real-world clinical datasets.
Summary updated