Segmentation networks trained at one hospital often fail on another hospital's scanner. A large share of that failure looks like low-level image statistics: a sharper CT reconstruction filter, a lower dose, or a different MRI coil changes the noise texture the network sees. We propose to give the segmentation network an explicit "noise dial," a small input that tells its early layers how noisy and how sharp the image is. This borrows the trick that lets a single image denoiser handle every noise level: the denoiser is told the noise level rather than asked to guess it. The network is trained on physically simulated acquisitions whose noise parameters are known exactly. At a new site, the network weights stay frozen, and only the dial's setting is chosen from unlabeled scans.
The opportunity matters for two reasons. First, the cheapest and strongest current remedy, heavy augmentation, forces the network to be invariant to acquisition differences, and benchmarks show it still leaves large gaps. Second, deployed medical software often cannot have its weights changed after certification. A model whose only test-time degree of freedom is a few physically meaningful numbers is far easier to audit than one that fine-tunes itself. The core claim is falsifiable and narrow: conditioning the encoder of a segmentation network on the acquisition's noise and sharpness parameters, then setting those parameters without labels at a new site, beats training the same network with the same physics augmentation but no conditioning. If it does not, the result is still informative, because it would show that segmentation networks infer noise implicitly well enough, as recent generative-model work suggests.
What exists and what is missing
Explicit noise conditioning is well established for restoration. Plug-and-Play Image Restoration With Deep Denoiser Prior feeds a per-pixel noise-level map into its denoiser, DRUNet, so that one model covers a continuous range of noise. A Noise-level-aware Framework for PET Image Denoising embeds a local coefficient-of-variation estimate to modulate attention blocks, but it does so for denoising on a single GE scanner. Both papers supply our mechanism. Neither uses conditioning for a downstream task, and neither applies it across sites.
On the domain-shift side, several papers motivate the design and constrain it. Anatomy of Domain Shift Impact on U-Net Layers in MRI Segmentation shows that encoder layers absorb most cross-scanner shift on CC359. This tells us where to inject the dial. However, their remedy, a learned policy that switches between frozen and fine-tuned blocks, needs target labels. Zero-Shot Domain Adaptation in CT Segmentation by Filtered Back Projection Augmentation simulates reconstruction kernels in sinogram space and raises cross-kernel prediction consistency from 0.54 to 0.92 Dice. We reuse its simulator. Our change is to keep the simulated kernel and noise parameters and feed them to the network, instead of discarding them as random augmentation.
Three works connect noise and domain shift more directly, and all three differ in kind from this proposal:
- Noise as Domain Shift translates noisy OCT scans to clean ones with a fixed unpaired mapping. It has no knob for intermediate noise levels.
- Robust Cross-domain CT Image Reconstruction via Bayesian Noise Uncertainty Alignment aligns domains implicitly, using Monte Carlo uncertainty and adversarial texture matching. It is expensive at inference and targets reconstruction, not segmentation.
- A generalizable diffusion framework for 3D low-dose and few-view cardiac SPECT imaging conditions sampling on scanner physics at test time, but it needs raw projection data, which routine image exports from hospitals (DICOM) do not contain.
Finally, Is Noise Conditioning Necessary for Denoising Generative Models? and The Geometry of Noise argue that networks can infer the noise level on their own. Those papers are the direct challenge to our hypothesis, and the ablations below are designed to answer them.
Research Program
The backbone is a standard nnU-Net. During training, each CT volume is passed through the filtered back-projection simulator. The simulator re-projects the volume, alters the ramp-filter parameters that control kernel sharpness, injects dose-dependent noise in projection space, and reconstructs. The physical parameters behind each simulated volume are recorded as a short vector $c$, holding kernel sharpness and a dose-equivalent noise level. A spatial noise map is computed from the known injected noise.
For MRI, the analogous step injects Rician noise, the noise distribution of MRI magnitude images, at known levels, together with bias fields. The vector $c$ then records the noise level and field strength.
The dial enters only the first two encoder stages, through feature-wise scaling and shifting: a small network maps $c$ to a per-channel multiplier and offset. The spatial noise map is concatenated as an extra input channel, following the DRUNet design. We also train a version that conditions the decoder instead, to test whether the encoder finding from SpotTUnet transfers.
At a new site, the weights are frozen. The dial is set in one of two ways. The first is a direct estimate from a small regressor trained on simulated volumes to predict $c$ from the image. The second is a label-free search that picks the setting minimizing prediction uncertainty while keeping the predicted anatomy stable:
$$\hat c = \arg\min_{c} \; \frac{1}{N}\sum_{i=1}^{N} \Big[ H\big(f(x_i, c)\big) + \lambda \, \lVert f(x_i, c) - f(\tilde x_i, c)\rVert_1 \Big],$$
The terms are defined as follows:
- $x_i$ are $N$ unlabeled target scans and $\tilde x_i$ is a flipped or slightly rescaled copy of $x_i$.
- $f(x, c)$ is the predicted label map.
- $H$ is the mean voxel entropy.
- $\lambda$ weights the consistency term.
The search runs over a low-dimensional, physically bounded space, so it cannot drift into degenerate solutions the way full-weight entropy minimization can.
Baselines
The naive reference point is nnU-Net with its default augmentation, trained on the source site and applied unchanged. The AMOS paper reports 89.04% mDice for a 3D U-Net in-domain on CT and a drop of more than 20% on unseen MRI scanners (AMOS). This baseline shows how large the gap is in our own runs.
The strongest competitor for kernel shift is filtered back-projection augmentation without conditioning. It is the closest method, and it already reports 0.92 cross-kernel consistency Dice (FBPAug). To make the comparison fair, it is trained with exactly the same simulator, the same parameter ranges, the same number of epochs, and the same hyperparameter search budget as our model. The only difference is whether $c$ and the noise map are given to the network.
The strongest unsupervised adaptation method is entropy minimization combined with heavy augmentation, which closes 62% of the source-to-oracle gap on average across M3DA and 96% on the CT-to-low-dose-CT task (M3DA). We run it on the same M3DA splits and give it the same unlabeled target scans our dial search sees. Because it updates weights, it is a deliberately stronger-than-permitted comparator in the frozen-model setting.
Histogram matching serves as a parameter-free preprocessing baseline. A two-stage pipeline serves as an input-space baseline: DRUNet denoising, with the noise map set by the same estimator, followed by the source-trained segmenter.
Ablations isolate the key ingredient by holding the training data and weights fixed and varying only what the dial receives at test time. We compare oracle $c$ (known on simulated targets), estimated $c$, searched $c$, shuffled $c$ taken from a different volume, and a constant mid-range $c$. We also compare encoder conditioning against decoder conditioning.
Beating the unconditioned augmentation baseline means two things. On cross-kernel consistency and target Dice, the improvement must hold across seeds with a paired test. On the hardest, out-of-range kernel and dose settings, the gap must widen rather than shrink.
Why not just…?
Why not just augment and let the network infer the noise? This is the real threat. The generative-model papers above argue that networks infer the noise level on their own, and M3DA shows that augmentation alone rivals years of adaptation research. Our ablation answers the question directly. If the oracle-$c$ model fails to beat the unconditioned model trained on identical simulated data, the dial adds nothing for segmentation, and we will report that. We expect an advantage specifically at the extremes and outside the training range. Invariance learned from augmentation averages over conditions, whereas a conditioned model can specialize per condition, as DRUNet does across noise levels. We test the extremes by holding out the sharpest kernels and lowest doses during training.
Why not just denoise or harmonize the image first? This route keeps the segmenter frozen and is attractive for certification. However, DRUNet is trained on additive Gaussian noise, which differs from CT and MRI noise. Fixed translators also need architectural repair to avoid erasing fine structure (Noise as Domain Shift), and CycleGAN-style adaptation trailed explicit noise alignment, 30.87 versus 32.68 dB on AAPM-A (Bayesian Noise Uncertainty Alignment). Denoising also removes texture the segmenter may need. We run the denoise-then-segment pipeline under the same estimated noise map, so the comparison is direct.
Why not just run entropy-minimization adaptation on the whole network? On CT-to-low-dose-CT it already closes 96% of the gap (M3DA). We therefore do not expect to win on that task, and we treat it as a check that the dial search is competitive there. Our case rests on two things. One is the frozen-weight constraint. The other is the larger shifts in kernel and MRI scanner, where adaptation leaves most of the gap. On the MRI-to-CT task, the best adapted score is still only 0.388 Dice. If full-weight adaptation also dominates on the kernel-shift and CC359 settings, the dial's value reduces to auditability alone, and we will say so.
Proposed Experiments
The first experiment is cross-kernel CT. We use the Covid-private and Mosmed-1110 protocol from the FBPAug paper and measure smooth-versus-sharp prediction consistency Dice plus in-domain Dice. The reference point is FBPAug's reported 0.92. If Covid-private is not accessible, we use in-silico kernel pairs generated on Mosmed-1110 and flag the result as weaker evidence.
The second experiment is dose shift, using M3DA's CT-to-low-dose-CT task and AMOS CT with simulated dose reduction. The third is cross-vendor MRI on CC359's 30 source-target pairs. No per-pair Dice values were reported on Lacuna for SpotTUnet, so all methods there are rerun by us.
Two checks will decide the outcome. The first is the shuffled-$c$ ablation: if randomizing the dial does not hurt, the network is ignoring it. The second is the gap between oracle and searched dial settings on simulated targets, which tells us whether the label-free search recovers the true physics. Secondary metrics are calibration error under shift, compared against the local temperature scaling result of 14.48% ECE in Improved post-hoc probability calibration, and the test-time cost of the dial search compared with weight-updating adaptation.
References
- Plug-and-Play Image Restoration with Deep Denoiser Prior
- A Noise-Level-Aware Framework for PET Image Denoising
- Anatomy of Domain Shift Impact on U-Net Layers in MRI Segmentation
- Zero-Shot Domain Adaptation in CT Segmentation by Filtered Back Projection Augmentation
- Noise as Domain Shift: Denoising Medical Images by Unpaired Image Translation
- Robust Cross-domain CT Image Reconstruction via Bayesian Noise Uncertainty Alignment
- A generalizable diffusion framework for 3D low-dose and few-view cardiac SPECT imaging
- Is Noise Conditioning Necessary for Denoising Generative Models?
- The Geometry of Noise: Why Diffusion Models Don't Need Noise Conditioning
- AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation
- M3DA: Benchmark for Unsupervised Domain Adaptation in 3D Medical Image Segmentation
- Improved Post-hoc Probability Calibration for Out-of-Domain MRI Segmentation