Introduction
In fields such as medicine (patient survival) or engineering (machine failure), researchers frequently encounter censored data. This occurs when only a lower bound for a variable is known—for instance, a clinical trial might end before a patient experiences an event, meaning we only know they survived at least a certain number of days.
Standard regression models often fail in these scenarios because they either ignore censored points or treat them as exact values, leading to significant bias. While Quantile Regression is a robust alternative for survival analysis because it avoids strict distributional assumptions, integrating it with deep learning has historically been computationally prohibitive. Previous methods were "sequential," requiring a separate neural network for every desired quantile, which ignores shared information and scales poorly.
The CQRNN Approach
The Censored Quantile Regression Neural Network (CQRNN) addresses these limitations by predicting an entire grid of conditional quantiles simultaneously. This multi-task learning approach allows the model to learn a shared representation (a "trunk") for all quantiles, improving both efficiency and generalization.
Core Innovations
- Portnoy’s Estimator Integration: CQRNN utilizes a re-weighting scheme where censored data points are split into two "pseudo-points": one at the censored value and one at a very large "pseudo-infinity" value ($y^*$).
- Simultaneous Grid Output: Instead of training $M$ independent networks, CQRNN uses a single network with multiple output heads. This provides a helpful inductive bias, as adjacent quantiles (e.g., the 50th and 60th percentiles) should naturally be similar.
- Bootstrapped Weights: During training, the network uses its own current predictions to estimate the "quantile of censoring" for each data point, updating loss function weights in real-time.
- Theoretical Grounding: The authors demonstrate that this approach is a form of Generalized Hard Expectation-Maximization (EM). They also identified a "self-correcting" property: if the model underestimates a weight, the resulting gradient push is stronger, forcing the model back toward the correct value.
Experimental Evaluation
The researchers evaluated CQRNN across three tiers of datasets:
- Synthetic: Fully simulated data (e.g., Weibull, LogNormal) to compare against known ground truths.
- Semi-Synthetic: Real-world regression datasets (e.g., Housing prices, MNIST) with artificial censoring added to test robustness.
- Real Survival Data: Seven classic biomedical datasets, including METABRIC (breast cancer) and SUPPORT.
Key Results
CQRNN demonstrated significant improvements over both traditional statistical methods and existing deep learning baselines like DeepQuantReg.
- Computational Efficiency: CQRNN achieved training speedups of 12x to 34x compared to sequential grid algorithms, as it only requires a single training pass for all quantiles.
- Improved Calibration: On real-world survival datasets, CQRNN produced quantiles that were better calibrated (measured by CensDCal) in 10 out of 12 instances.
- Distribution-Free Flexibility: Unlike models that assume a specific error distribution (like Cox Proportional Hazards), CQRNN remains flexible for complex, non-linear real-world data.
Limitations and Future Work
While CQRNN is highly effective, the authors noted a few areas for further development. Like most deep learning methods, there is no theoretical guarantee of converging to a global optimum. Additionally, while the model usually learns the correct order of quantiles, it does not strictly forbid "quantile crossing" (where a higher percentile prediction is lower than a lower one). Future work may involve architectural constraints to enforce strict monotonicity. Finally, in cases of extreme censoring, higher quantiles may be mathematically undefined, requiring careful selection of the "pseudo-infinity" value.
Summary updated