Many cancer prognosis systems run in two stages. A segmentation network outlines the tumor on CT or MRI. Shape and texture measurements taken from that outline then feed a survival model, which predicts when an event such as death or recurrence will occur. The survival model has to handle right-censoring, meaning some patients leave follow-up before the event is observed. Expert outlines are the expensive input. Active learning, the practice of choosing which unlabeled scans to send to an annotator, currently picks scans where the segmentation network is unsure about pixel labels. In low-contrast tissue, that uncertainty sits mostly at fuzzy tumor edges, and much of it is irreducible blur that even experts disagree on. We propose a different question for choosing what to annotate: would correcting this patient's outline change whether a clinician acts on the predicted risk?
The proposed method works as follows. It samples many plausible outlines for each scan and pushes each one through the survival model, which gives a spread of predicted risks for that patient. It then measures how often that spread crosses the risk thresholds at which clinicians would actually intervene. It also separates the part of the spread due to the model's lack of knowledge, which annotation can fix, from the part due to inherent blur, which it cannot. Batches are made diverse with respect to these decision flips rather than image appearance alone. Evaluation uses decision curve analysis, which scores a model by the net clinical benefit of acting on its predictions across a window of risk thresholds. We use this rather than Dice overlap or the concordance index.
Core Claim
Under a fixed budget of expert delineations, an acquisition rule that targets epistemic, threshold-crossing risk instability will yield higher censoring-adjusted net benefit in the clinically relevant threshold window than pixel-uncertainty or diversity-based active learning. It should do so even when it yields equal or lower Dice. The rule should also skip ambiguous boundaries whose ambiguity does not move any patient across a decision threshold.
The gap is real in the file. The 3D segmentation active learner ClaSP PE (Finally Outshining the Random Baseline) controls batch redundancy with scheduled noise but is judged only on Dice and area under the budget curve. The only survival active learner, ARC (Active Learning based Survival Regression for Censored Data), queries tabular patients one at a time by Cox gradient change and never touches images. Decision-targeted information gain (Targeted Active Learning for Bayesian Decision-Making) is the closest conceptual relative. It queries points that reduce entropy about the optimal decision, but it uses Gaussian processes, single queries, discrete actions and no censoring. The imaging-to-survival pipelines on BraTS (BRATS 2017 contribution; BRATS meta-analysis) treat masks as fixed inputs. They report survival accuracy stuck near 62% even as whole-tumor Dice reaches about 0.91. Our proposal differs from all of these in one specific way. The quantity being reduced is the uncertainty in a patient's treat/no-treat decision that is induced by uncertainty in that patient's tumor outline, measured over a threshold window and for a fixed time horizon.
Research Program
Pipeline. We fix a horizon $\tau$, for example one year. The survival model, a regularized Cox model or a random survival forest on radiomic features plus age, outputs a risk $r = P(T \le \tau \mid \text{features}(m))$ for a patient with mask $m$.
Sampling outlines. For each unlabeled scan we draw $K$ outlines from a deep ensemble of $E$ nnU-Net members. Within each member we add test-time perturbations confined to the low-confidence boundary band. The band is defined in the spirit of the soft centroid-to-edge supervision of Bounding Maps, and its width is calibrated on any cases that have multiple raters.
Acquisition score. Let $q_{e}(p)$ be the fraction of member $e$'s samples whose risk exceeds threshold $p$, and let $\bar q(p)$ be the average over members. Let $H(x) = -x\log x - (1-x)\log(1-x)$. The score for patient $i$ is
$$A_i = \int_{p_l}^{p_h} w(p)\Big[H(\bar q_i(p)) - \tfrac{1}{E}\textstyle\sum_e H(q_{i,e}(p))\Big]\,dp,$$
where $[p_l, p_h]$ is the clinician-specified threshold window and $w(p)$ is a weighting over that window.
The bracket is the mutual-information decomposition familiar from Bayesian active learning, applied to the binary decision at $p$ rather than to pixel labels. It is large only when ensemble members disagree about which side of the threshold the patient falls on. Blur that every member reproduces contributes to both terms and cancels. This directly addresses the tension in the file between uncertainty sampling and repeatedly querying aleatoric boundary noise.
The choice of $w$ follows A Consequentialist Critique of Binary Classification Evaluation Practices. That paper shows that Brier scores bounded to a threshold window equal integrated net benefit over the window. We therefore use the corresponding weight so that the acquisition score and the evaluation metric measure the same thing.
Batch selection. Patients often flip for the same reason: a whole cluster of similar-sized enhancing rims might sit near one threshold. We therefore build each batch in two steps, following FASS from Submodularity in Data Subset Selection and Active Learning. First we filter the pool to the top candidates by $A_i$. Then we greedily maximize a facility-location objective. Its similarity function combines encoder-embedding similarity with similarity of the patients' threshold-crossing profiles $\bar q_i(\cdot)$ over the window.
Retraining. After each round, the segmentation network is retrained on the delineations collected so far. Features are then recomputed for all patients, and the survival model is refit on the cohort's existing outcome labels. Outcomes are assumed known; the scarce resource is delineation, which matches how retrospective imaging cohorts are actually built.
Evaluation. Net benefit at horizon $\tau$ is estimated with inverse-probability-of-censoring weights derived from the Kaplan–Meier estimator of the censoring distribution. This avoids binarizing survival at a fixed cutoff, as Machine Learning-Based Prediction of Surgical Benefit did at eight months.
Baselines
Random sampling with strong augmentation. The naive reference is random selection of scans to delineate, with nnU-Net's full augmentation pipeline, at matched annotation budgets. Realistic Evaluation of Deep Active Learning shows that strong augmentation can shrink the active-learning advantage on CIFAR-10 from about 3.2% to 0.4%. We also include "treat all" and "treat none" net-benefit curves as the floor any model must clear. Our method beats this reference only if its integrated net benefit over the window exceeds random's with a paired bootstrap interval excluding zero at two or more budgets.
ClaSP PE. This is the strongest existing 3D segmentation active learner. It combines class-stratified patch queries with Gumbel power noise on predictive entropy under a decaying schedule. Its authors report that it is the only method to generally beat improved random baselines across 24 nnActive settings. The file gives no per-dataset numbers ("no reported result found on Lacuna for ClaSP PE Dice/AUBC values"). We run it in the same nnU-Net backbone with identical budgets, rounds, epochs and seeds, and give it the same hyperparameter search size as our method. Beating it means higher integrated net benefit at equal budget. We will also report whether this comes at a Dice cost.
Pixel entropy with core-set diversity. This is the standard uncertainty-plus-representativeness recipe. It tests whether the decision-based score adds anything over geometric diversity.
ARC-style query. This adapts ARC's hazard-weighted gradient-change criterion to radiomic features. It tests whether outcome-awareness without mask propagation suffices.
Downstream reference numbers. Survival-side performance from expert masks is reported as 52.6% three-class accuracy on the BraTS 2017 test set (BRATS 2017 contribution) and roughly 62% across challenge years (BRATS meta-analysis). No net-benefit result was found on Lacuna for BraTS survival, so our full-annotation pipeline defines the oracle ceiling.
Ablations. Three ablations isolate the key ingredient. Threshold-proximity only queries patients whose mean risk lies inside the window, ignoring mask uncertainty. Total decision entropy drops the epistemic subtraction. No batch diversity keeps the top-$B$ by $A_i$.
Why not just…?
Why not just use random batches with semi-supervised learning and heavy augmentation? It is a serious threat for segmentation quality, given Mittal et al.. However, ClaSP PE reports that tuned random sampling is beatable in 3D medical volumes. More importantly, our target is not Dice. We will give the random arm the same semi-supervised training as every other arm. If random still matches us on windowed net benefit, we will report that the decision-aware score buys nothing.
Why not just query patients whose predicted risk sits near the threshold? This is the most dangerous alternative because it is cheap and outcome-aware. It fails in principle when a patient is near the threshold but their outline is stable, since then annotation cannot move the decision. It also fails when the patient's mean risk is far from the threshold but the outline is unstable enough to cross it. The threshold-proximity ablation measures this directly. If the two scores rank patients nearly identically (rank correlation above 0.9) and give equal net benefit, we will adopt the simpler rule.
Why not just skip boundary refinement and make the survival model robust to mask noise, or use coarse box features? Leiming Zhang et al. suggest that patient biology can dominate anatomy. If risk is insensitive to the outline, no acquisition rule can matter. Our first experiment therefore measures how much risk shifts between predicted and expert masks before any active learning is run. We also include a mask-perturbation-augmented survival model trained on a random budget as a competing arm.
Why not just use the concordance index to choose and evaluate? Flores et al. show, on BCSC data, a model that looks worse globally being best within the relevant 1.66–3% window. Ranking quality over the whole cohort is the wrong target when only a few thresholds govern action. We report the concordance index anyway, so that any divergence between the two metrics is visible.
Proposed Experiments
Feasibility check. On the BraTS training cohort with survival labels, we compare one-year risk computed from expert masks with risk from masks of nnU-Nets trained on 10%, 25% and 50% of delineations. We report the fraction of patients whose decision flips somewhere in the window. If this is under a few percent, the proposal is moot for gliomas, and we say so.
Primary active-learning experiment. On BraTS, the clinically meaningful window is around the short-survival boundary. We run five rounds of batch acquisition with all baselines and three to five seeds. The metrics are integrated net benefit over the window with censoring weights, bounded Brier score computed with briertools, decision agreement with the full-annotation oracle, concordance index, and Dice/AUBC. BraTS survival labels may contain little censoring. If so, we impose administrative censoring at several cutoffs to test that the censoring-weighted estimator and the method behave correctly.
Semi-synthetic control. BraTS alone cannot reveal when the method should win, so we add a controlled test. On nnActive datasets with low-contrast lesions (KiTS, LiTS), we generate synthetic censored outcomes from features of the ground-truth masks. We vary how much outcome signal lives in boundary-sensitive features, such as rim volume and surface irregularity, versus boundary-insensitive ones. The prediction is that our advantage grows with boundary sensitivity, vanishes when it is zero, and holds against threshold-proximity sampling only when outline instability is heterogeneous across patients.
Aleatoric-noise test. We inject rater-like random boundary jitter into a subset of training labels. The epistemic-only score should avoid querying those cases, while the total-entropy ablation and pixel-entropy baselines should keep returning to them.
Deployment-time review triage. Using the same score at test time to rank which automated outlines a radiologist should check, we measure net benefit recovered per reviewed case against random review and pixel-uncertainty review.
References
- Finally Outshining the Random Baseline: A Simple and Effective Solution for Active Learning in 3D Biomedical Imaging
- Active Learning based Survival Regression for Censored Data
- Targeted Active Learning for Bayesian Decision-Making
- Brain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge
- Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge
- Bounding Maps for Universal Lesion Detection
- A Consequentialist Critique of Binary Classification Evaluation Practices
- Submodularity in Data Subset Selection and Active Learning
- Machine Learning-Based Prediction of Surgical Benefit in Borderline Resectable and Locally Advanced Pancreatic Cancer
- Realistic Evaluation of Deep Active Learning for Image Classification and Semantic Segmentation