Locked Nomograms That Know Their Own Contour Noise

A radiomics nomogram is a simple risk calculator. A lesion is outlined on a scan, a few hundred texture and shape numbers ("radiomic features") are computed inside the outline, a sparse regression compresses them into one radiomics score, and that score is combined with clinical variables in a logistic or Cox model. Such calculators are validated at one or two hospitals and then frozen, because changing coefficients means re-certifying a medical device. When the frozen calculator moves to a new scanner, the automatic segmenter that draws the outline becomes less certain about where the lesion boundary lies. The features computed inside that wobblier boundary become noisier. A fixed coefficient applied to a noisier input produces predictions that are too extreme: the model becomes overconfident in exactly the place it knows least.

We propose to measure that extra noise at the new site without any labels, by resampling the segmenter's own plausible outlines. We then correct the locked nomogram with a single classical errors-in-variables rescaling, a technique from the statistics of measuring inputs with error, and leave every certified coefficient untouched. In two sentences: segmentation uncertainty under domain shift is mainly a measurement-error problem for the downstream nomogram. Estimating the target site's feature reliability from sampled masks lets a frozen nomogram recalibrate itself, with no labeled data, at the moment of deployment.

Core Claim

The research file documents each link of this chain separately but never joins them. Robust Radiomics Feature Quantification Using Semiautomatic Volumetric Segmentation shows that the choice of contour alone moves feature reproducibility from an intra-class correlation of 0.85 to 0.77. Intra-class correlation is the share of feature variance that comes from real between-patient differences rather than from the outline. This paper motivates the proposal: boundary noise is large relative to biological signal. Improved post-hoc probability calibration for out-of-domain MRI segmentation shows that segmenters become spatially miscalibrated under scanner and sequence shift, and repairs this with test-time perturbation and a shape prior on a frozen network. Its authors note that the shape prior fails on variable tumor boundaries, and the paper never looks downstream at features or risk scores. The two deployed nomograms in the file, the multiphase CT ccRCC grading nomogram and the spectral CT STAS nomogram, treat their contours as fixed truth and report only AUC and standard calibration curves at their external sites.

The closest methodological relative is DAFIT, from Optimal Statistical Incorporation of Independent Feature Stability Information into Radiomics Studies. At training time, DAFIT injects noise whose size is estimated from separate test-retest or multi-reader side studies, so that the model down-weights fragile features. It also shows that ignoring instability overstates cross-setting AUC by nearly 20%. Our proposal differs in where and how the noise is estimated. DAFIT fixes one noise level, learned once from a side study, at training time. We estimate a site-specific and patient-specific noise level at deployment, from the segmenter's own outline samples on the unlabeled target scans. We then apply it to a model whose coefficients cannot change.

The mechanism is regression calibration. Write the locked nomogram as $p = \sigma(\alpha + \beta x + \gamma^\top c)$. Here $\sigma$ is the logistic function, $x$ is the standardized radiomics score, $c$ the clinical variables, and $\alpha, \beta, \gamma$ the frozen coefficients. If the observed score is the true score plus independent outline-induced noise of variance $\tau^2$, the reliability at a site is

$$\lambda = \frac{\mathrm{Var}(x_{\text{obs}}) - \tau^2}{\mathrm{Var}(x_{\text{obs}})}.$$

The source-site coefficient was fitted on data with reliability $\lambda_s$. At a target site with reliability $\lambda_t$, the slope that is approximately correct for observed scores is

$$\beta_t \approx \beta \,\frac{\lambda_t}{\lambda_s}.$$

This follows from the standard attenuation result for a linear predictor with additive measurement error, which carries over approximately to logistic models when effects are moderate. Both $\tau^2$ terms are estimated without labels, as the average within-patient variance of the radiomics score across $K$ sampled outlines per lesion.

A second, patient-level step averages the prediction over that patient's own outline samples. A patient whose boundary is ambiguous is therefore pulled toward the clinical-only risk rather than receiving a confident radiomics-driven call. The certified coefficients $\alpha, \beta, \gamma$ never change. What gets added is a deterministic, auditable rescaling factor computed from unlabeled target images.

The domain-shift direction is where the method applies. Region-of-interest segmentation is the source of the noise. The clinical-radiomic nomogram is the object being protected. The cross-domain feature-mismatch direction enters as a secondary question: whether the correction transfers to deep features pooled inside the mask. In Lesion-Centric Latent Phenotypes from Segmentation Encoders for Breast Ultrasound Interpretability, pooling a segmentation encoder's features inside the predicted lesion beats hand-crafted radiomics (AUC 0.982 versus 0.774). We test whether such in-domain features are also more reliable under shift, meaning a higher $\lambda_t$, than ImageNet-initialized features of the kind used in the ccRCC nomogram.

Research Program

The pipeline has three pieces. The first is an automated lesion segmenter that produces a distribution of outlines rather than one outline. We will compare three ways of sampling outlines: a small deep ensemble, test-time augmentation, and the post-hoc temperature maps of the out-of-domain calibration method. The second is a standard locked nomogram, built by sparse (LASSO) selection, a radiomics score, and multivariable logistic regression. It will be built once from hand-crafted features and once from mask-weighted pooled encoder features. The third is the deployment-time correction above.

Its one dangerous assumption is that sampled outlines vary about as much as real outline error does. If the segmenter becomes overconfident at a new scanner, which is exactly the failure the out-of-domain calibration paper documents, then $\tau_t^2$ is underestimated and the correction is too weak. A central analysis therefore validates the estimated $\tau^2$ against real inter-reader variance, wherever multiple human outlines exist.

The main cohort is LIDC-IDRI, with lesions annotated by up to four radiologists across multiple scanner manufacturers. We will split it by manufacturer to create source and target sites, and use the reader-to-reader spread as ground truth for outline noise. Malignancy labels in LIDC are known to be biased toward calling nodules malignant. Re-thinking and Re-labeling LIDC-IDRI for Robust Pulmonary Cancer Prediction reports sensitivity 0.97 with specificity 0.23 on raw consensus labels. We will therefore report results on both raw consensus labels and that paper's re-labeled targets.

LIDC has little structured clinical data. The non-imaging arm of the nomogram will use radiologist semantic ratings, playing the role that radiological signs play in the STAS nomogram. This is a stand-in for real clinical covariates, and we state it plainly. Controlled acquisition shift will come from the virtual CT pipeline of Assessing radiomics feature stability with simulated CT acquisitions, which varies dose, slice thickness and reconstruction method on known regions of interest. This lets us dial in the amount of shift.

For the deep-feature question, the ultrasound triad from the latent-phenotype paper provides the setting: BUSI and BUS-UCLM as sources and BUS-BRA as an external site. If a collaborating center can share a two-site cohort like the ccRCC study, it becomes the clinical confirmation. That study's reported external AUC of 0.911 is the published reference for that setting.

Baselines

The do-nothing reference is the locked nomogram applied unchanged at the target site, with a single automatic outline per lesion. It is the deployment practice in both two-center nomogram studies. Every method uses the same segmenter, the same frozen coefficients and the same target scans.

The strongest label-free competitor is ComBat-style feature harmonization. ComBat estimates each site's feature mean and scale from unlabeled target data and maps them back to the source. It is widely used in radiomics and needs no labels, which makes it the right peer. We run it on exactly the same unlabeled target pool. Beating it means a lower calibration error and a calibration slope closer to 1 at equal or better AUC. We also run our correction stacked on top of ComBat, because the two address different errors: ComBat fixes shifts in location and scale, while our correction fixes reduced reliability.

The strongest label-using competitor is classical logistic recalibration, which refits the intercept and slope on $n$ labeled target patients. We give it label budgets of 20, 50 and 100 patients. Our claim counts as supported only if the label-free correction matches recalibration at a stated small budget. It should also stay within the bootstrap interval of recalibration at larger budgets.

DAFIT trained with LIDC multi-reader noise is the strongest training-time robustness baseline. The comparison isolates whether target-specific noise estimates beat a fixed, side-study estimate.

The ablation that isolates the key ingredient keeps the patient-level averaging over sampled outlines but removes the site-level $\lambda_t/\lambda_s$ rescaling, and then does the reverse. Sampling alone reduces noise but cannot correct a slope fitted at a different reliability, so the rescaling should account for most of the calibration gain.

For reference points from the file: DenseSharp+ in Probabilistic Radiomics reports AUC 0.9566 on the ambiguous LIDC subset. That figure is within a single repository, without a scanner split. The out-of-domain calibration network reports pixel-level calibration error of 15.37% versus 16.78% for local temperature scaling under the bSSFP-to-LGE MRI sequence shift. No reported result was found on Lacuna for nomogram-level calibration on LIDC under a scanner-split protocol, and none for ComBat on any benchmark in the file. We therefore establish both ourselves.

Why not just…?

Why not just harmonize features with ComBat and keep the nomogram? ComBat aligns the population mean and variance of each feature across sites. It cannot tell biological variance from outline-induced variance, so it rescales both together and leaves the reliability ratio unchanged. It also says nothing about which individual patients have ambiguous boundaries. It remains a real threat when site shift is dominated by intensity offsets rather than boundary noise. The manufacturer splits and the simulated dose sweep separate those two regimes. If ComBat alone matches the full method across both, our claim fails.

Why not just recalibrate the intercept and slope with a few labeled target cases? If a site can supply 50 to 100 labeled, pathology-confirmed cases, this simple method may well win, and we expect it to set the ceiling. The case for our approach rests on sites that cannot supply those labels at deployment time, and on recalibration changing the coefficients, which reopens certification. The label-budget curve shows exactly where the crossover lies, and we will report it whichever way it falls.

Why not just use DAFIT or a more reproducible segmenter so the noise is small to begin with? Semiautomatic outlining raises reproducibility, but it still leaves 30% of features below an intra-class correlation of 0.8 in the semiautomatic-segmentation study. The out-of-domain calibration paper shows that automatic segmenters degrade under shift. DAFIT's noise level is fixed in advance and cannot track a target site's larger noise. The DAFIT baseline directly tests whether that limitation costs calibration in practice.

Why not just use an end-to-end deep model or a domain generalization algorithm? An Empirical Framework for Domain Generalization in Clinical Settings finds that eight such algorithms fail to beat pooled training under realistic model selection: chest X-ray AUROC 0.862 for pooled training versus 0.860 for GroupDRO, the best of them. Separately, end-to-end models without a lesion mask learn shortcuts, as The Role of Pleura and Adipose in Lung Ultrasound AI documents. We include pooled training on the unmasked image as a further reference rather than rely on these citations alone.

Proposed Experiments

The first decisive check is whether label-free reliability estimates are accurate. On the LIDC manufacturer splits, we compare the estimated outline-noise variance from sampled outlines with the variance computed from the up-to-four reader outlines, for each feature and for the radiomics score. If the estimate falls well short of reader variance at target sites, even after the post-hoc temperature correction, the method's premise fails. We would report that as the main finding.

The second check is calibration at the nomogram level. We report calibration slope, expected calibration error, and the Bounded DCA Log Score from Aligning Evaluation with Clinical Priorities. That score is a proper scoring rule restricted to clinically realistic disease-prevalence ranges; it is used here purely as an evaluation measure. We report AUC alongside so that any gain in calibration is not bought with lost discrimination. We will assess the attenuation formula by checking that the uncorrected nomogram's calibration slope falls roughly in proportion to $\lambda_t/\lambda_s$ as the simulated dose and slice-thickness sweep increases outline noise.

The third check repeats the comparison with hand-crafted and with mask-pooled deep features on the BUSI and BUS-UCLM to BUS-BRA transfer. It asks whether in-domain encoder features keep higher reliability under shift, and whether the same correction applies to them unchanged.

All comparisons use patient-level bootstrap confidence intervals with a single tuning budget per method, fixed before any target data are inspected.