When the Contour Decides the Operation: Reader-Uncertainty Propagation for Lung-Sparing Resection

Surgeons choose between removing a whole lung lobe and removing only a small wedge or segment, and for early lung cancers that look like faint haze on CT ("ground-glass" nodules), that choice increasingly rests on two numbers computed from the scan. The first is the lesion's average radiodensity, its mean brightness on the calibrated Hounsfield scale, where air is −1000 and water is 0. The second is the fraction of the lesion that is dense, solid tissue. A multi-center study of 321 patients found that an average density above −198 HU or a solid fraction above 54% marks invasive, higher-risk disease, and argued that patients below both cutoffs can safely receive the smaller, less traumatic operation (Radiologic Predictors for Clinical Stage IA Lung Adenocarcinoma with Ground Glass Components). Both numbers, however, depend on where someone drew the lesion's outline and on which bright voxels they counted as tumor rather than as a blood vessel of the same density. Radiologists disagree about exactly those things.

This proposal tests a simple claim. For a substantial, identifiable minority of patients, plausible differences between readers' outlines are large enough to move the density or solid-fraction value across the surgical cutoff. In those patients the operation is effectively decided by who drew the contour. We propose to sample many plausible outlines per lesion, push each one through the density calculation, and report a per-patient decision-flip probability: the chance that the recommended operation would change under reader-level boundary uncertainty. The decisive question is whether this lesion-specific quantity identifies unstable decisions better than the obvious shortcut of flagging every patient whose single measurement lies close to the cutoff.

Core Claim

Existing multi-rater segmentation work measures disagreement in pixels and Dice overlap. Existing radiodensity work treats the outline as fixed and reports a cutoff. Nobody propagates the first into the second, so nobody knows how often reader disagreement changes the surgical recommendation, or whether this can be predicted per patient. Our claim is that it can, and that the predictability comes from lesion-specific sources of ambiguity rather than from distance to the threshold alone. The two main sources are blurred boundaries, where finite voxel size averages tumor with surrounding air, and solid-looking regions that may be vessels. If the claim holds, a flip probability gives a principled "gray zone." Patients with high flip probability are the ones who should receive a hedged plan, such as anatomical segmentectomy or intraoperative frozen-section confirmation, while confident patients can proceed directly to the smaller operation or to lobectomy. Over-treatment (unnecessary lobectomy) and under-treatment (a sublobar resection that misses occult invasion) are the two failure modes of trauma-reducing surgery, and the gray zone targets exactly the cases where either is most likely.

Formally, for lesion $i$ we draw $K$ plausible segmentations $S_{i,1},\dots,S_{i,K}$. Each yields an average density $a_{i,k}$ and a solid fraction $r_{i,k}$. The cutoff rule $D(a,r)=\mathbb{1}[a>-198 \text{ or } r>0.54]$ maps each sample to a recommendation, and the flip probability is

$$ \phi_i = \min\Big(\tfrac{1}{K}\textstyle\sum_k D(a_{i,k},r_{i,k}),\; 1-\tfrac{1}{K}\textstyle\sum_k D(a_{i,k},r_{i,k})\Big), $$

which is the fraction of plausible outlines that disagree with the majority recommendation. It is 0 for a stable decision and 0.5 for a coin flip.

Research Program

The work has three parts.

Building the outline sampler. We follow the finding from the uncertainty-in-segmentation challenge (QUBIQ) that simple ensembles of ordinary networks, each trained to imitate different raters, capture human disagreement better than elaborate probabilistic models. We therefore train per-reader 3D U-Net ensembles rather than inventing a new generative architecture. This paper supplies the method; we do not claim novelty here.

We add a second stochastic stage for the solid component. Inside each sampled outline, the solid region is defined by a density threshold and a vessel-exclusion step. We vary both: the threshold within the range readers actually use, and vessel removal using the geometric vessel-subtraction approach of Solid component evaluation in mixed ground glass nodules, with its radius and elongation parameters perturbed. That paper showed tumor cores and adjacent vessels share identical density and need geometric modeling to separate. It evaluated only 15 real nodules with categorical quality grades and never asked how much the separation choice moves downstream numbers. The pGGO contrast-phase study likewise stripped vessels by hand, which is exactly the step whose variability we quantify.

Measuring how much density readings actually move. On lesions with several independent reader contours, we compute the spread of average density and solid fraction across real readers. We then check whether the sampler's spread matches it. This calibration is at the level of the clinical quantity, not the pixel.

Linking flip probability to surgical outcome proxies. On pathology-confirmed cohorts, we test whether high-$\phi$ patients are the ones the cutoff rule gets wrong. The reference standard is invasive adenocarcinoma versus pre-invasive lesions, or the high-risk pathology defined by Zhao et al. We also ask whether deferring them improves accuracy among the rest.

Data. LIDC-IDRI provides up to four radiologist contours per nodule and is our main source of genuine reader disagreement. We restrict it to the part-solid and ground-glass subset. Its subjective malignancy scores are biased toward over-calling (Re-thinking and Re-labeling LIDC-IDRI), so we use only its contours, never its malignancy labels.

For pathology, we need cohorts like the 321-patient Shanghai cohort or the 90-patient Shandong pure ground-glass cohort (42 pre-invasive, 48 invasive). The file gives no public release for either. The program therefore includes a re-reading study: at least three thoracic radiologists independently contour a pathology-confirmed ground-glass cohort of comparable size, so that real reader spread and pathology exist on the same lesions.

QUBIQ's kidney and pancreas CT tasks serve as an external check that the sampler reproduces reader spread on other CT anatomy.

Baselines

Naive reference: single deterministic outline. One automated segmentation, or one reader's contour, is fed into the fixed cutoffs exactly as Zhao et al. apply them. This shows what the rule achieves when uncertainty is ignored. The strongest reported number for the rule itself is Zhao et al.'s AUC of 0.921 for average CT value, alongside 0.907 for solid fraction, against pathologic risk. On the pure ground-glass task, Li et al. report AUC 0.906 for 60-second post-contrast mean density. We will not claim to beat these AUCs. Our target is different: among patients the method does not defer, accuracy should exceed the single-outline rule's accuracy at matched coverage.

Strongest competitor: distance-to-cutoff gray zone. We defer every patient whose single-outline density or solid fraction lies within a fixed margin of −198 HU or 54%. The margin is tuned on the same development data to give the same deferral rate as our method. This is the baseline most likely to make the idea unnecessary, because flip probability is partly a smooth function of distance to the cutoff. Beating it means two things at equal deferral rate on held-out pathology data: higher accuracy among retained patients, and better prediction of which patients real readers actually split on, measured by AUC for "readers disagree on recommendation."

Strongest multi-rater segmentation competitor. We train MRNet (Learning Calibrated Medical Image Segmentation via Multi-Rater Agreement Modeling), which models each rater's expertise and produces pixel-wise uncertainty maps, on the same LIDC contours with the same compute and hyperparameter budget as our ensemble. Its uncertainty map is converted to outline samples by thresholding at multiple levels. This tests whether a more elaborate disagreement model yields better-calibrated density spread than a plain ensemble. Its reported RIGA gain is 1.2% soft Dice on the optic cup; no reported result was found on Lacuna for MRNet's QUBIQ numbers. For context, QUBIQ's top participants reached average Dice of 0.80–0.85.

Ablations. We run the sampler with outline variation only, then with solid-component and vessel variation only. This isolates which source of ambiguity drives decision flips. It directly tests whether vessel mimicry, rather than outer-boundary blur, is the dominant hidden variable.

Why not just…?

Why not just use one automated segmenter, which removes reader variability by being perfectly reproducible? Reproducibility is not correctness. A deterministic model returns the same outline every time, but on an ambiguous lesion that outline is one arbitrary member of a plausible set, and the model gives no signal that the decision is fragile. Comp2Comp-style pipelines report high Dice on large, well-defined tissues (Comp2Comp), but faint ground-glass borders and vessel-adjacent solid cores are where partial-volume effects dominate. The naive baseline measures this directly. If a single automated outline already matches the multi-sample rule's accuracy and the flip probability adds nothing, the idea fails, and we will report that.

Why not just widen the cutoff into a fixed gray zone? This is the real threat, and it is our primary competitor rather than a straw man. If boundary ambiguity were roughly uniform across lesions, distance to the cutoff would capture nearly everything. Our hypothesis requires that ambiguity vary by lesion, through vessel contact, size relative to voxel size, and margin sharpness, so that two lesions equally close to −198 HU differ in stability. The matched-coverage comparison settles this. We would accept a null result showing that the simple margin suffices, which would itself be clinically useful because it is cheap.

Why not just train an end-to-end classifier for invasiveness and skip outlines and cutoffs? End-to-end and radiomic models can be strong. They also struggle on rare mimics, with oncocytoma AUC around 0.57–0.69 in the fat-poor renal tumor study, and they risk learning scanner artifacts. More importantly, surgical guidelines and the Zhao et al. evidence are stated as interpretable density cutoffs that surgeons use. We will train a 3D classifier on the same pathology data as an upper reference. If it clearly dominates, the honest conclusion is that the cutoff rule should be replaced, not patched. Even then, flip probability remains useful for auditing the current rule.

Why not just biopsy the ambiguous ones? Needle biopsy of sub-centimeter ground-glass lesions carries pneumothorax and bleeding risk and frequent sampling error, which conflicts with the goal of reducing trauma. Flip probability is designed to shrink the set of patients for whom any extra step, whether biopsy, segmentectomy, or frozen section, is needed.

Proposed Experiments

Experiment 1: does disagreement move the numbers? On the LIDC-IDRI ground-glass and part-solid subset, we compute the spread of average density and solid fraction across real readers, and the fraction of nodules whose recommendation under the Zhao cutoffs differs between at least two readers. A near-zero fraction falsifies the premise and ends the project early.

Experiment 2: sampler calibration. On held-out LIDC lesions and QUBIQ CT tasks, we check whether the sampler's predicted spread of each density quantity matches the observed reader spread. The measures are interval coverage and rank correlation of predicted versus observed spread. We compare the ensemble against MRNet-derived samples under equal budget.

Experiment 3: flip probability as a predictor of reader splits. We compute the AUC of $\phi$ for predicting whether real readers disagree on the recommendation, compared with the AUC of distance-to-cutoff and with a logistic combination of both. The key quantity is the gain of $\phi$ over distance alone.

Experiment 4, decisive: pathology. On the re-contoured, pathology-confirmed ground-glass cohort, we plot accuracy among retained patients against deferral rate. We compare the single-outline rule, the fixed-margin gray zone, $\phi$-based deferral, and the end-to-end classifier. We also report how many over-treatment cases (pre-invasive lesions sent to lobectomy) and under-treatment cases (invasive lesions sent to wedge resection) each strategy removes at 10% and 20% deferral.

The idea earns its place only if $\phi$-based deferral beats the fixed margin at matched deferral on both counts. The vessel-only versus outline-only ablation then tells us which source of ambiguity a better imaging protocol or reading rule should target first.