When several radiologists read the same chest CT, they disagree in two different ways. Sometimes they agree a nodule is there but draw its edge differently. Sometimes one reader marks a nodule and the others see nothing, perhaps because they read it as a vessel seen end-on. Current uncertainty methods handle the first kind of disagreement reasonably well and nearly miss the second. Pixel-wise entropy detects "does this nodule exist?" disagreement on LIDC-IDRI at an AUROC of only 0.48, which is chance level (Beyond Boundary Noise). Radiologists settle exactly this question by changing viewpoint. A small sphere looks like a round blob from every direction. A vessel looks round in one plane and tubular in the others. We propose to turn that clinical habit into a model. For each candidate lesion, we render a handful of small 2D slices tilted at different angles through its center. A frozen pretrained 2D image model reads each slice, and we measure how much its reading changes across views. That view-inconsistency serves as a signal of presence ambiguity. Each region then gets its own two-part uncertainty model: a probability over how many raters would mark it, and a separate shape latent for how they would draw it.
The design also answers the cost problem that usually sinks multi-view methods on 3D data. Rendering whole-volume projections from many cameras and running a large 2D network on each is slow (Multi-View Vertebra Localization). Doing it once per rater hypothesis would multiply that cost again. We render only small patches, only for candidate regions, and only as many views as the presence question needs. In two sentences: disagreement about whether a lesion exists is largely a viewpoint-dependent appearance problem, so cheap region-centered multi-view rendering should expose it even without disagreement labels. Giving each region its own presence and boundary uncertainty then separates the two kinds of rater disagreement that global models blur together.
Core Claim
We make two claims that can each fail on their own. The first is label-free. On LIDC-IDRI and on the external LNDb cohort, the spread of a frozen 2D model's readings across a candidate's tilted views ranks presence-ambiguous nodules well above pixel-entropy aggregation. The reference points are 0.48 AUROC on LIDC and 0.54 on LNDb (Beyond Boundary Noise). The target is to approach that paper's supervised post-hoc Ambiguity Head (0.74 and 0.67) without using any annotator-disagreement labels.
The second claim is supervised. We add a per-region model with two parts, a presence part fit to rater counts and a boundary part fit only to the masks of raters who marked the region. It should match or exceed the Ambiguity Head on presence, and it should transfer better to LNDb. On boundaries, it should match multi-rater generative segmenters on distribution quality. The per-region split should matter most on scans where raters agree about one lesion and disagree about another.
Research Program
The pipeline has three stages.
Stage 1: candidate generation. A 3D candidate generator proposes regions, deliberately tuned for high recall. We train it on the union of all raters' marks and supervise it with the continuous center-weighted targets of Bounding Maps. That work improved low-contrast lesion sensitivity on DeepLesion from 82.43% to 86.28% at 4 false positives per image. Here we reuse its method, not its claim. Recall matters because the file names detector dependency as the main weakness of region-based pipelines: a lesion that is never proposed cannot be scored.
Stage 2: region-centric rendering. For each candidate $k$, we resample $V$ small oblique planes through its centroid, for example 64×64 at native spacing. We use planes spread over a sphere of orientations, not axial slices alone. A frozen 2D encoder, DINOv2 in the spirit of DINO in the Room, embeds each plane. A light 2D head predicts a per-view "lesion-like" probability and a per-view mask. The label-free ambiguity score $s_k$ combines two quantities: the dispersion of per-view probabilities, and the dispersion of per-view mask elongation after mapping the masks back to 3D. This parallels ViewAL's viewpoint entropy (ViewAL). ViewAL measures disagreement across camera rays in indoor scenes to choose superpixels for annotation. We apply the same idea to region-centered medical reformats, to predict rater disagreement rather than to choose annotations.
Two cost controls keep rendering cheap. Because rendering is restricted to candidate patches, its cost scales with the number of candidates, not with volume size. A cascade renders a second batch of views only for candidates whose first-batch score falls in an uncertain band.
Stage 3: the per-region uncertainty model. It reads the pooled view tokens plus a small 3D crop feature. For region $k$ read by $R_k$ raters, of whom $m_k$ marked it, the presence part outputs Beta parameters $(\alpha_k, \beta_k)$. The boundary part outputs a Gaussian latent $z_k$ that is decoded into a mask, trained with a conditional ELBO on the marking raters' masks only. The objective is
$$ \mathcal{L}=\sum_k \Big[-\log \mathrm{BetaBin}(m_k \mid R_k,\alpha_k,\beta_k) + \mathbb{1}[m_k>0]\,\mathcal{L}^{\text{ELBO}}_k\Big], $$
where BetaBin is the beta-binomial likelihood of $m_k$ marks out of $R_k$. The presence-ambiguity score is the predicted variance of the rater fraction, $\alpha_k\beta_k/((\alpha_k+\beta_k)^2(\alpha_k+\beta_k+1))$. That quantity is high exactly when the model expects a split vote.
The boundary part reuses the late-injection latent of the Probabilistic U-Net, which adds a low-dimensional latent to the final feature maps. It differs in scope: that paper samples one global code per image, and we sample one code per region. Scan-level hypotheses are then assembled by sampling each region's presence and shape independently. This directly addresses the global-compromise problem that single latents share, whether that latent represents a whole image or a rater (Pionono). The union and intersection bounds of D-Persona add an optional regularizer on the boundary part. Rater personalization is left out, because our question is presence versus boundary, not individual reader style.
Baselines
The naive reference is aggregated pixel-wise entropy from an MC-dropout or ensemble 3D U-Net, summed over each candidate. Beyond Boundary Noise reports 0.48 AUROC on LIDC-IDRI and 0.54 on LNDb for this approach. We rerun it on our own candidates so that every method scores the same set of regions.
The strongest presence competitor is that paper's post-hoc Ambiguity Head, a small network on 3D bottleneck features trained on disagreement labels. It reports AUROC 0.74 on LIDC-IDRI, 0.67 on LNDb zero-shot, and Spearman ρ≈0.63 against the degree of disagreement. We reimplement it on the same 3D backbone, splits and candidates, and give it the same hyperparameter search budget as our head. Our label-free score counts as a success if it clearly beats 0.48 and closes most of the gap to 0.74. The supervised model counts as beating the Ambiguity Head only if it exceeds 0.74 on LIDC and, more importantly, 0.67 on LNDb.
The boundary competitors are the Probabilistic U-Net and its normalizing-flow variant (Improving Aleatoric Uncertainty with Normalizing Flows). The flow variant reports GED and IoU gains of up to 14% and 13% relative to the Probabilistic U-Net on 128×128 LIDC crops. The file gives no absolute GED values for either, so we rerun both on identical crops. We also include D-Persona, which reports 89.17% mean Dice for personalized LIDC segmentation. We treat that number as context, since personalization is a different task.
The ablations isolate the key ingredient. The first removes the 2D views and feeds only the 3D crop feature into the same per-region model. The second uses axial slices only instead of oblique views. The third renders views densely, every candidate at maximum $V$, with no cascade. The fourth replaces per-region latents with one global latent.
Why not just…?
Why not just train the Ambiguity Head, or a rater-count head, on a native 3D network? A 3D crop already contains every orientation, so in principle a 3D network can learn the vessel-versus-sphere cue. This is the most serious threat to the proposal. The 3D-only ablation settles it on equal data and tuning budget. If the 3D-only ablation matches the full model on LIDC and LNDb, the multi-view part adds nothing to the supervised claim, and only the label-free result survives. That result does not depend on this comparison, because the 3D head needs disagreement labels and the view score does not. Our bet is that pretrained 2D appearance priors plus explicit view dispersion transfer across cohorts better than a head fit to about a thousand LIDC disagreement cases. The LNDb drop from 0.74 to 0.67 suggests the supervised head overfits to its cohort.
Why not just compute 3D sphericity or elongation of the segmented candidate? A hand-crafted shape measure may capture most of the vessel-versus-nodule signal for free. We include it as a label-free baseline. If it matches the view-dispersion score, we will report that plainly and reframe the contribution as the per-region presence and boundary model. The view score should beat it when ambiguity comes from appearance rather than shape, for example faint ground-glass nodules. We will check that by stratifying LIDC by the nodule texture attributes that come with the dataset.
Why not just use deep ensembles or MC dropout, aggregated per region instead of per pixel? Aggregating per region fixes part of the pixel-entropy failure, and the file reports this as the approach that scores 0.48. But ensembles tend to collapse toward the consensus mode (Probabilistic U-Net), and in 3D they multiply compute (QUBIQ). We run a five-member ensemble with region-level variance of predicted presence and report it with latency, so the comparison is explicit rather than assumed.
Proposed Experiments and Decisive Checks
LIDC-IDRI, with up to four readers, is the main testbed. We use patient-level splits and define presence ambiguity as in Beyond Boundary Noise: marked by one reader versus agreed by three or four. LNDb is used for zero-shot transfer. We report presence AUROC and Spearman ρ against the number of marking readers. For boundaries on commonly marked nodules, we report GED and per-rater Dice. We report candidate recall at fixed false-positive rates, because every region score is capped by recall. We report wall-clock time per scan, broken into rendering, 2D forward passes and 3D passes, as a function of $V$, with and without the cascade.
We sweep $V$ from 2 to 16. Multi-View Vertebra Localization found about 10 whole-volume views to be the best accuracy-latency trade-off for its task. Our expectation is that region-sized views saturate earlier. As a secondary check that the boundary part does not degrade where presence is not in question, we run QUBIQ kidney and pancreatic-lesion CT and report Q-Dice. The file's only QUBIQ figure is the challenge's summary range of 0.80–0.85 Dice for top ensembles, not a task-level number to beat.
The decisive checks come in three steps. First, if the label-free view-dispersion score does not beat both pixel entropy and 3D sphericity on LNDb, the central hypothesis that presence ambiguity is view-dependent is wrong. Second, if the 3D-only ablation matches the full supervised model on both cohorts, multi-view rendering is not worth its cost for this problem. Third, on the subset of scans that contain one agreed and one disputed nodule, per-region latents should give lower GED than a single global latent. If they do not, the per-region split buys nothing.
References
- Beyond Boundary Noise: Aggregated Aleatoric Uncertainty Fails to Capture Presence Ambiguity in 3D Lung Nodule Segmentation
- Multi-view Vertebra Localization and Identification from CT Images
- Bounding Maps for Universal Lesion Detection
- DINO in the Room: Leveraging 2D Foundation Models for 3D Segmentation
- ViewAL: Active Learning With Viewpoint Entropy for Semantic Segmentation
- A Probabilistic U-Net for Segmentation of Ambiguous Images
- Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation
- Diversified and personalized multi-rater medical image segmentation
- Improving Aleatoric Uncertainty Quantification in Multi-annotated Medical Image Segmentation with Normalizing Flows
- QUBIQ: Uncertainty Quantification for Biomedical Image Segmentation Challenge