Whether a head-and-neck cancer will spread to distant organs is one of the most consequential predictions in radiotherapy planning, and the best published model makes it by drawing a graph from each patient's pre-treatment CT. Each node of the graph is the primary tumor or one involved lymph node, and edges are weighted by the distance between them. On the 1,570-patient RADCURE cohort, this model reaches a 2-year distant-metastasis AUROC of 0.757, against 0.668 for clinical variables alone (Lymph Node Graph Neural Networks for Cancer Metastasis Prediction). It also reports a striking biological reading: individual lymph nodes carry up to 60% more predictive weight than the primary tumor. That model is a single deterministic network trained on a cohort in which distant metastasis is a minority event. It cannot tell a clinician when a patient's nodal pattern is unlike anything it was trained on, and nobody has checked whether its biological conclusion survives retraining from a different random start.
We propose to train a deep ensemble of these graph survival models. A deep ensemble is several copies of the same network trained from independent random initializations, whose disagreement measures how much the model does not know. We use it for two things. The first is a patient-level warning signal: flag and defer cases where the members disagree. The second is a stability test for the claim that the lymph-node graph, and not a simpler summary of it, is what predicts spread. The proposal is deliberately narrow. One cohort, one model family, and a set of cheap competitors that the ensemble has to beat outright.
Core Claim
In small-cohort, rare-event metastasis prediction, disagreement among independently trained graph models is informative in a way that neither a single calibrated model nor a trivial "atypical patient" score captures. Concretely, we predict three things. The ensemble's epistemic uncertainty will identify patients the model gets wrong better than node count or distance in embedding space does. Deferring those patients will improve calibration and discrimination on the patients who are kept. The nodal importance pattern reported for the single model will be either confirmed or shown to vary across members, and the ensemble is what lets us decide which. If the first prediction fails, meaning uncertainty is just a proxy for having many nodes, we will report that as the main finding. It would still be useful, because it would tell clinicians that a one-line rule does the same job.
Relation to Prior Work
The closest existing work is a Bayesian version of the same survival head. Uncertainty Estimation in Cancer Survival Prediction places variational distributions over the weights of a neural multi-task logistic regression. Multi-task logistic regression (MTLR) predicts a probability for each discrete time bin. That paper applies it to 47 tabular features from TCGA-BRCA and METABRIC and reports C-index 0.71 against 0.65 for Cox regression, plus a 110% rise in uncertainty under an age shift. It motivates our work but differs in three ways. It treats nodal status as a single tabular covariate rather than a spatial graph. Its target is overall survival rather than distant metastasis. Its uncertainty comes from variational inference, which What Are Bayesian Neural Network Posteriors Really Like? argues explores only the neighbourhood of one solution, whereas deep ensembles approximate the Bayesian average better by visiting several distinct solutions. That argument supplies our method choice.
Evidence from medical imaging constrains the design. Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation found that ensembles of five or more members cut negative log-likelihood by 62–66% relative to single networks and beat Monte Carlo dropout on calibration. That result is for dense segmentation, not graph survival prediction, and whether it transfers is exactly what we test. DGRNet shows a cheaper route: lightweight adapters on a shared backbone reach an error-detection AUC of 0.910 on brain tumors. However, performance degraded beyond four views, a warning that shared-parameter ensembles lose diversity. We include adapters only as a compute-matched comparison.
For turning raw uncertainty into calibrated decisions we reuse ideas from JUCAL and CLEAR. Both fit separate scale factors for the data-noise and model-ignorance components on held-out data. CLEAR reports that standard conformal intervals fall to 40–70% coverage in extrapolation regions while its joint calibration holds 95%. Neither has been applied to survival outputs or to graphs.
Finally, the nomogram study on SEER data (Nomogram Development for Predicting Synchronous Lung Metastasis in Patients with T1 Colorectal Cancer) shows AUC inflating from 0.716 to 0.849 when concurrent metastatic sites leak into the inputs. We therefore restrict inputs to strictly pre-treatment CT and clinical variables.
Research Program
We reimplement the edge-gated graph convolutional network with an MTLR survival head. We use the published graph construction: primary tumor and involved nodes as vertices, 3D distance-weighted edges, and radiomic node features from the existing manual contours. We then train $M = 10$ members that differ only in random initialization and data order.
MTLR gives each member a discrete distribution over time-to-metastasis bins. That lets us split uncertainty in the standard way, following the ensemble decomposition used in Global canopy height regression and uncertainty estimation from GEDI LIDAR waveforms with deep ensembles and written here in its entropy form:
$$H\Big[\tfrac{1}{M}\sum_{m} p_m(t\mid x)\Big] = \underbrace{\tfrac{1}{M}\sum_{m} H[p_m(t\mid x)]}_{\text{data noise}} + \underbrace{I(t;m\mid x)}_{\text{model disagreement}}.$$
Here $p_m(t\mid x)$ is member $m$'s predicted distribution over time bins for patient graph $x$, $H$ is entropy, and $I(t;m\mid x)$ is the mutual information between outcome and member identity, which is zero exactly when all members agree. A patient with genuinely uncertain prognosis shows high average member entropy. A patient whose graph is unfamiliar shows high mutual information.
We fit two JUCAL-style scale factors on a validation fold to calibrate the 2-year risk. We then use the disagreement term alone as the deferral score.
The stability analysis reruns Integrated Gradients on every member. We ask two questions. Does the share of attribution on lymph nodes versus the primary tumor stay consistent? Do the same individual nodes rank highest within a patient? The published figure of up to 60% more weight on nodes is a single-model estimate. Here it becomes a distribution with an interval.
Baselines
The naive reference point is the clinical-variable model, for which the lymph-node GNN paper reports a 2-year distant-metastasis AUROC of 0.668. The tumor-volume model at 0.693 serves as a second simple anchor. Both are refit on our folds so that all methods share identical patients, splits and censoring.
The strongest existing method is the single edge-gated GCN with MTLR head, reported at AUROC 0.757 and lifetime C-index 0.725 (Lymph Node Graph Neural Networks for Cancer Metastasis Prediction). The paper reports no calibration numbers, so no reported result was found on Lacuna for its calibration error or Brier score. We will measure these ourselves. To be fair, the single model receives the same hyperparameter search budget as one ensemble member plus a post-hoc temperature fit. We also report a "best single member" chosen on validation. The ensemble counts as beating it if it matches its AUROC within confidence intervals while clearly lowering the integrated Brier score and calibration error, and if deferring the 10–20% most uncertain patients raises AUROC on the retained set more than deferring by any competing score does.
Uncertainty competitors are all run at matched inference cost where possible. Monte Carlo dropout uses ten passes of one network. The DGRNet-style shared-backbone adapter ensemble uses four heads. A bootstrap ensemble of a regularized logistic or Cox model on clinical variables plus node count and total nodal volume tests whether disagreement needs the graph network at all.
Two trivial atypicality scores are evaluated as deferral rules: the number of involved nodes, and Mahalanobis distance of the patient's graph embedding from the training set. Two ablations isolate the key ingredient. The first replaces edges with order-free set pooling over nodes, which tests whether graph structure contributes beyond the node collection. The second is a snapshot ensemble taken along a single training run, which tests whether independent initialization, and therefore distinct solutions, is what matters.
Why not just…?
Why not just use Monte Carlo dropout on the single network? It costs one training run, and for a small graph model the ensemble's cost is modest anyway, so the real question is quality. Prior comparisons favour ensembles on calibration and out-of-distribution detection in medical segmentation (Mehrtash et al.) and in speech enhancement (Integrating Uncertainty into Neural Network-based Speech Enhancement). Neither involves rare-event survival on graphs, so we run the comparison directly and will report dropout as sufficient if its risk-coverage curve matches the ensemble's.
Why not just temperature-scale the single model? Temperature scaling may fix average calibration, and on that metric alone it could match the ensemble. It yields one confidence number per patient, though, and cannot separate a hard prognosis from an unfamiliar patient. The deferral experiment settles this. If a temperature-scaled model's confidence defers errors as well as ensemble disagreement does, the decomposition adds nothing clinically, and we will say so.
Why not just flag patients with many nodes or unusual embeddings? This is the most serious threat. The graph model's signal comes from nodal burden, so disagreement may simply track node count. We treat node count and embedding distance as first-class deferral baselines. We also report the partial correlation of disagreement with error after controlling for both. Only if disagreement adds error-detection AUC beyond them do we claim it carries new information.
Why not just bootstrap a simple clinical model? If a bootstrapped regression on clinical variables plus nodal summaries matches the graph ensemble in discrimination and deferral, then the graph network's advantage over clinical variables is fragile. The set-pooling ablation and the attribution-stability analysis will show whether the edges do real work.
Proposed Experiments
All experiments use RADCURE with patient-level stratified five-fold cross-validation, repeated over seeds, and censoring-aware metrics. These are 2-year AUROC, C-index, integrated Brier score, expected calibration error of 2-year risk, error-detection AUC, and area under the risk-coverage curve.
Four checks are decisive. The first compares calibration of the ensemble against the temperature-scaled single model at equal discrimination. The second asks whether disagreement predicts errors beyond node count and embedding distance. The third is a constructed shift experiment: train on patients with few involved nodes, test on those with many, and, if primary tumor site is recorded, hold out one site. It asks whether disagreement rises on the shifted group and whether calibrated risk intervals keep their coverage where the single model's do not, mirroring CLEAR's extrapolation test. The fourth is the attribution-stability analysis, reporting the spread across members of the node-versus-primary attribution ratio.
A positive outcome would give radiation oncologists a principled "not sure, get more staging" signal and would put an error bar on a biological claim about how nodal spread predicts distant metastasis. A negative outcome on the second check would show that a simple nodal-burden rule serves as well, which is also worth publishing.
References
- Lymph Node Graph Neural Networks for Cancer Metastasis Prediction
- Uncertainty Estimation in Cancer Survival Prediction
- What Are Bayesian Neural Network Posteriors Really Like?
- Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation
- DGRNet: Disagreement-Guided Refinement for Uncertainty-Aware Brain Tumor Segmentation
- JUCAL: Jointly Calibrating Aleatoric and Epistemic Uncertainty in Classification Tasks
- CLEAR: Calibrated Learning for Epistemic and Aleatoric Risk
- Nomogram Development for Predicting Synchronous Lung Metastasis in Patients with T1 Colorectal Cancer: An SEER-Based Analysis
- Global canopy height regression and uncertainty estimation from GEDI LIDAR waveforms with deep ensembles
- Integrating Uncertainty Into Neural Network-Based Speech Enhancement