Before a radical prostatectomy, a surgeon decides separately for each side of the prostate whether to preserve the neurovascular bundle, the sheet of nerves beside the gland that controls erections. Preserving it protects function. If cancer has already grown through the capsule on that side (extraprostatic extension, or EPE), preserving it risks leaving tumor behind. Today this decision rests on side-specific risk calculators built from a few clinical numbers and a radiologist's visual score. Deep networks trained on MRI have repeatedly failed to beat simple clinical variables in this setting, largely because the labeled cohorts are small. We propose to train an MRI model that predicts EPE risk at each patch of the capsule surface without any voxel-level annotation. It learns from two cheap sources: side-level pathology labels ("EPE present on the left"), which already exist for every prostatectomy, and self-supervised pretraining on unlabeled MRI. Self-supervised pretraining means the network first learns anatomy by predicting hidden parts of scans, with no human labels.
The model will not be judged mainly by AUC. We will judge it by the decision it supports: how many neurovascular bundles can be spared at a fixed rate of sparing a side that turned out to carry EPE. The central question is whether label-efficient imaging models can move this sparing-versus-safety tradeoff beyond the side-specific clinical calculator, and how many labeled patients that takes.
Core Claim
A 3D MRI encoder pretrained without labels can be fine-tuned with multiple-instance learning over capsule-surface sectors, using only side-level pathology outcomes. Multiple-instance learning here means the model scores many small regions, but only a label for the whole side is known. We claim this model will spare more bundles than the side-specific logistic calculator of Li et al. at the same unsafe-sparing rate. We also claim it will need a fraction of the labeled patients a scratch-trained network needs to reach the calculator's performance.
The ingredient we think matters is the surface-resolved pooling. EPE is a local event at the capsule, often sub-millimeter. Global pooling over a whole hemi-gland lets that signal be swamped by the rest of the gland. If global pooling does just as well, the claim shrinks to "pretraining plus side labels," and our ablations are designed to reveal that.
Why This Gap Is Real
The morbidity side of the problem is documented but unsolved. Li et al. showed that EPE on the two sides is statistically independent, which justifies side-specific decisions. Their calculator combines side-specific PI-RADS (the standard radiologist score for suspicious prostate lesions), Gleason grade, biopsy cores and PSA density. It reaches AUC 0.77 on the dominant side and 0.79 on the non-dominant side. The visual cue of capsular abutment had only 27% specificity and added nothing. This paper supplies both our strongest baseline and our decision unit: the side.
Lin et al. used a deep network only to grade T2 image quality. They found that radiologist EPE specificity falls from 72% to 63% on low-quality scans, and that subtle capsular bulging loses predictive value there (odds ratio 3.05 falling to 1.76, P=0.24). This motivates testing whether learned features survive image degradation where human reading does not.
The strongest constraint comes from the erectile-dysfunction study. It predicted 12-month function directly from MRI. Clinical variables reached AUC 0.663, a supervised hybrid CNN-transformer on mid-prostate slices reached 0.569, and fusion reached 0.586. We read this as a warning with two parts. Scratch-trained imaging models on a few hundred patients do not add value. Postoperative function is also a poor imaging target, because it depends on the surgeon's choice and on baseline potency. We therefore target the anatomical quantity MRI can actually see, capsular breach, and treat function as something the sparing decision protects.
The label-efficient side supplies the method. AR-SSL4M pretrains 3D CT/MRI encoders by predicting patch sequences across depth, across co-registered contrasts, and across patients. Trained on 23,287 volumes, it is competitive with 10% labels, but only on whole-organ segmentation. UniMiSS enforces agreement between 2D slices and 3D volumes and beats full supervision on BCV with 40% of the labels. Neither has been tested on a boundary-level surgical question.
LoopNet is the closest precedent for our weak-supervision idea. It localizes microvascular-invasion cells in histology from slide-level labels by feeding its own candidate cell masks back into the classifier. We borrow that feedback check as a faithfulness test, but apply it to preoperative 3D MRI rather than resected tissue. Song et al. is the closest in spirit. It linked an MRI prediction of liver microvascular invasion to a margin decision (wide margin above 5 mm), but used a patient-level output from a network trained from scratch.
Research Program
The capsule surface comes first. A zonal prostate segmentation network, such as the Bayesian attentive model of Liu et al., is trained on PROSTATEX. It reports Dice 0.80 for the peripheral zone and 0.89 for the transition zone, with similar external performance. From its output we extract the gland surface on T2 MRI and divide each side into $K$ sectors, indexed by base, mid or apex crossed with posterolateral, lateral or anterior position. Each sector yields a thin band of voxels straddling the capsule across T2, diffusion-weighted and contrast-enhanced sequences.
The encoder is initialized from AR-SSL4M. It then receives continued self-supervised pretraining on unlabeled prostate MRI from PROSTATEX and whatever unlabeled institutional archives the partner sites provide. We test one change to that pretraining. The research file notes that masking 70–85% of patches can erase sub-patch boundary texture. We therefore compare standard random masking against masking that keeps the capsule band at a lower masking ratio.
For fine-tuning, the model outputs a breach probability $p_{s,k}$ for sector $k$ on side $s$. These are combined into a side risk by a noisy-OR:
$$ P(\text{EPE}_s) = 1 - \prod_{k=1}^{K} (1 - p_{s,k}). $$
The noisy-OR encodes the assumption that a side is positive if any sector is breached. Training uses cross-entropy against the side-level pathology label. Clinical covariates from the Li et al. calculator enter as a side-level offset, so the imaging branch only has to explain what the clinical variables miss.
Many pathology reports also name a coarse EPE site, such as "left posterolateral base." Where they do, we add a weak sector-group term, but never a voxel mask.
The LoopNet-style check has two parts. If we remove the top-scoring sectors' bands from the input, the side risk should collapse. If we remove random bands, it should not. This is how we test whether the sector map is causally used rather than decorative.
The decision layer turns side risk into a sparing rule: spare side $s$ if $P(\text{EPE}_s) < t$. Sweeping the threshold $t$ traces a frontier. The horizontal axis is the fraction of all sides spared, a proxy for preserved function. The vertical axis is the fraction of spared sides with pathological EPE, a proxy for oncological risk. Pathological EPE is an imperfect stand-in for "sparing would have caused a positive margin," and we report it as such. Decision-curve net benefit is a secondary summary.
Baselines
The naive reference is guideline practice without a model. It includes two fixed policies: spare every side whose PI-RADS score is below 4, and a radiologist-graded EPE rule of the kind analyzed by Lin et al.. Each gives a single point on the frontier, and any learned method must dominate it.
The strongest existing method is the side-specific multivariable logistic calculator of Li et al., with reported AUC 0.77 (dominant side) and 0.79 (non-dominant side) on their 601-patient Northwestern cohort. The numbers in that paper come from a different cohort, so we refit the calculator on exactly our training split with the same covariates. Refitting gives it every advantage of in-distribution fitting, and it needs no tuning budget worth matching. Beating it means two things on the held-out and external test sets. First, the model must spare significantly more sides at the unsafe-sparing rate the calculator achieves at its clinically used threshold, with a paired bootstrap over patients. Second, it must not do worse at any point of the frontier within the clinically relevant range. An AUC gain alone does not count.
We also run a scratch-trained supervised 3D network with the same architecture, clinical offset and sector pooling. It gets the same labeled data and the same hyperparameter search budget as our model. This is the regime in which the erectile-dysfunction study saw imaging lose to clinical variables.
Three ablations isolate the key ingredient. The first swaps sector-level noisy-OR pooling for global average pooling over the hemi-gland, keeping the same pretrained encoder. The second removes the report-derived sector-group term. The third replaces boundary-aware continued pretraining with the off-the-shelf AR-SSL4M weights. All variants share splits, epochs and search budget.
Why not just…?
Why not just keep the clinical calculator? This is a serious threat. The erectile-dysfunction study found that clinical variables beat both supervised imaging (0.663 vs 0.569) and fusion (0.586). Our design concedes that point: clinical covariates are built in as an offset, so the model can at worst reproduce the calculator. The experiment settles the question directly. If the fused model's frontier is not separated from the refit calculator's, we will report that pretrained MRI adds nothing to this decision. That is a useful negative result for surgeons.
Why not just fine-tune a pretrained 3D encoder as a side-level classifier? This could well work as well as our model on side-level decisions. That is why the global-pooling ablation is a primary comparison, not an afterthought. Our bet is that EPE is too local for global pooling at a few hundred labeled patients. We expect the gap to be largest at small label fractions and on subtle breaches. If the gap is absent at every label fraction, we drop the claim that surface resolution matters for the side decision. We would then keep it only for the localization and faithfulness results, which a global classifier cannot provide.
Why not just predict erectile dysfunction directly? Function is the quantity patients care about. But it is confounded by the surgeon's actual sparing choice and by baseline potency, and imaging has already failed at it (AUC 0.569 in that study). Predicting the anatomical breach separates the question MRI can answer from the one it cannot, and the sparing frontier then expresses the functional stakes.
Why not just filter out low-quality scans and rely on radiologists for the rest? Lin et al. show that low-quality scans are where human reading loses its signal. Filtering them out leaves about a third of patients (36% in their cohort) with no imaging input at all. We will stratify every result by an image-quality grade. If the model also fails on low-quality scans, the right deployment is to defer to the calculator there, and we will report the frontier of that hybrid rule.
Proposed Experiments
The data requirement is multiparametric MRI paired with side-level prostatectomy pathology. The cohorts that fit are those of Li et al. (601 patients, 53% EPE prevalence) and Lin et al. (773 consecutive patients across 2007–2022). Both are institutional, so access requires collaboration. We would train on one site and test externally on the other. PROSTATEX serves only for capsule segmentation and unlabeled pretraining, because the file records no EPE labels for it.
The label-efficiency experiment fine-tunes every model on 10%, 25%, 50% and 100% of the labeled training patients, with five random draws each. It reports the smallest fraction at which each model's frontier matches the refit calculator. The prediction is that the pretrained sector model matches it at 25% or less, while the scratch-trained network does not match it even at 100%. The second half of that prediction repeats the erectile-dysfunction finding in a new target.
The decisive checks are three. The first is the paired frontier comparison against the refit calculator on the external site. The second is the global-pooling ablation at each label fraction, which says whether surface resolution is the source of any gain. The third is the loopback faithfulness test, together with agreement between top-scoring sectors and report-named EPE sites on patients whose reports record location. For secondary analysis, we compare the frontier on high- versus low-quality scans. We also compare boundary-aware masking with standard masking on the subset of focal, small-volume EPE cases, where erosion of fine boundary texture should matter most.
References
- Prostate MRI and clinicopathologic risk calculator to predict laterality of extraprostatic extension at radical prostatectomy
- Deep learning-based image quality assessment: impact on detection accuracy of prostate cancer extraprostatic extension on MRI
- Evaluating the Predictive Value of Preoperative MRI for Erectile Dysfunction Following Radical Prostatectomy
- Autoregressive Sequence Modeling for 3D Medical Image Representation
- UniMiSS: Universal Medical Self-supervised Learning via Breaking Dimensionality Barrier
- A Loopback Network for Explainable Microvascular Invasion Classification
- Using deep learning to predict microvascular invasion in hepatocellular carcinoma based on dynamic contrast-enhanced MRI combined with clinical parameters
- Exploring Uncertainty Measures in Bayesian Deep Attentive Neural Networks for Prostate Zonal Segmentation