Center-Surround Normalization in Physical Units for Scanner-Robust Segmentation

Medical image segmentation models fail when a scan comes from a hospital, scanner, or imaging protocol they were not trained on. A large share of that failure comes from effects that vary slowly across the image: MRI coil sensitivity makes one side of the body brighter than the other, a different sequence changes tissue contrast, and endoscope lighting falls off toward the frame edges. These effects leave anatomy intact but change the raw numbers the network reads. Brain circuits face a similar problem and handle it with a particular wiring pattern. Each neuron is excited by its near neighbors and inhibited by a wider ring around them, with connection strength falling off with physical distance. This "Mexican-hat" profile responds to local contrast and discards slow changes in overall brightness.

We propose to build this wiring into the early layers of a segmentation network. The layer is a divisive center-surround stage whose excitatory and inhibitory extents are measured in millimeters, not pixels. Each channel learns its own extents under a wiring-cost penalty. The resulting network should be structurally blind to any brightness or contrast change that is roughly constant over a few millimeters. Data augmentation only teaches this property for the finite family of distortions it happens to sample. The main question is whether this built-in invariance improves robustness to real, unsimulated scanner shift beyond what strong augmentation already provides. If it does not, the idea is dead.

Core Claim

A center-surround normalization layer with distance-limited, learned lateral interactions, placed in the first encoder stages of a U-Net, will reduce the cross-scanner performance drop on MRI and endoscopy segmentation. The gain should hold even on top of bias-field and intensity augmentation. It should appear specifically for shifts dominated by slowly varying gain and contrast. It should not appear, and may reverse, for shifts dominated by high-frequency noise or ghosting.

That last prediction makes the claim falsifiable. It separates a mechanistic account from a vague "brain-inspired helps" story.

Why This Intersection, and What Is New

The spatial-connectivity literature supplies the mechanism, but not in segmentation. Two groups show that Mexican-hat lateral interaction acts as a local contrast enhancer that suppresses runaway excitation: Sajedin et al. and Toyoizumi et al.. We reuse their interaction profile as our layer. Miconi et al. show that fixing inhibition to a spatial range lets competition work across layers of different resolution without retuning. We borrow that idea directly, and extend it to a medical problem they did not address: voxel spacing differs across scanners, so a range fixed in pixels is a different physical range at each site. Jarman et al. show that a wiring-cost penalty makes networks self-organize toward local structure with sparse long-range links. We use a wiring cost to let each channel choose its own surround width while staying short-range by default.

The closest topographic models embed Mexican-hat kernels in a latent lattice. Grogan et al. do this in a VAE trained on limb kinematics, and the self-organized map of AlexNet features builds a similar lattice post hoc. Both organize an abstract feature space. Neither touches image-space robustness or domain shift. Multi-Barrel routing arranges modules on a physical lattice with local gating. It was tested only in-distribution on tactile data, and the file lists scaling to dense vision as open.

On the medical side, the work connecting to topology or structure does so at the output or post hoc. Deep Poincaré Map traces boundaries as a dynamical limit cycle, but with an unconstrained AlexNet. ErrorNet repairs broken vessel masks after prediction. Ouyang et al. recalibrate confidence on a frozen model. None changes how the network reads intensities in the first place. That is the specific point where acquisition differences enter, and it is what we target.

Two results motivate the proposal. AMOS reports that a plain 3D-UNet (89.04% CT mDice) beats Swin-UNetr (86.32%). This suggests local inductive bias matters more than global capacity. AMOS also reports that all models lose more than 20% on unseen MRI scanners. Ouyang et al. show that models stay badly miscalibrated under simulated bias fields and sequence change. Their local temperature scaling reaches 14.48% expected calibration error (ECE), which measures the gap between confidence and accuracy, on Torchio artifacts.

A negative result from Ouyang et al. also constrains the design. Their shape prior fails on thin and branching structures. That is why we act on intensities rather than learn shape, and why we report adrenal glands and vessels separately.

Mechanism

For channel $c$ at location $p$, the layer computes

$$ y_c(p) = \frac{(G_{\sigma_{E,c}} * x_c)(p) - \alpha_c\,(G_{\sigma_{I,c}} * x_c)(p)}{\epsilon + (G_{\sigma_{I,c}} * |x_c|)(p)} $$

The terms are defined as follows. $x_c$ is the input feature map. $G_\sigma$ is a Gaussian whose width $\sigma$ is given in millimeters and converted to voxels using each scan's recorded spacing. $\sigma_{E,c} < \sigma_{I,c}$ are the learned excitatory and inhibitory widths. $\alpha_c \in [0,1]$ is a learned inhibition strength. $\epsilon$ is a small constant.

Suppose the input is multiplied by a gain field $g(p)$ that changes little across a distance $\sigma_{I,c}$. Then the numerator and denominator both scale by approximately $g(p)$, so $y_c$ is unchanged to first order. The subtractive term additionally removes locally constant offsets. So the invariance has a stated range: fields smoother than the surround.

Training adds a wiring-cost term $\lambda \sum_c \sigma_{I,c}^2$ to the segmentation loss. This favors short surrounds unless longer ones reduce the task loss, following the rewiring-cost logic of Jarman et al.

Two limits of this layer serve as built-in ablations. As $\sigma_I \to \infty$ it approaches instance normalization, a single gain for the whole image. As $\alpha \to 0$ it approaches plain local contrast scaling.

Research Program

We insert the layer before the stem and after the first encoder stage of the 3D-UNet configuration that AMOS found strongest, and into the 2D U-Net and ResNet-50 encoders used for polyp, cardiac, and retinal tasks. The rest of each architecture stays untouched.

One known risk shapes the CT design. CT intensities are physically calibrated in Hounsfield units, and discarding absolute intensity could hurt there. For CT we therefore also test a variant that concatenates raw and normalized inputs. We predict the layer helps MRI, cardiac LGE, and endoscopy, where intensities are uncalibrated, and is neutral on CT. A second risk is that center-surround stages amplify high-frequency content. We predict losses under noise, ghosting, and k-space spikes, and we will report them.

Beyond accuracy, we analyze what the network self-organizes. We examine the distribution of learned $\sigma_{I,c}$ in millimeters per channel and per stage. We test whether surround widths track organ scale. We also run a controlled sweep in which synthetic bias fields of varying correlation length are applied to held-out MRI. The mechanism predicts that the robustness gain appears only once the field's correlation length exceeds the learned surround widths. That crossover curve is the most direct test that the layer works for the stated reason.

Baselines

The naive reference is the same U-Net trained with standard geometric augmentation and per-volume z-score normalization, evaluated on unseen domains. It sets the size of the drop we aim to close.

The strongest existing competitor on AMOS is the 3D-UNet from AMOS, which reports 89.04% CT mDice. AMOS reports only a relative drop of more than 20% for unseen MRI scanners. No absolute unseen-scanner mDice is reported on Lacuna, so we retrain this baseline on our split.

To make the comparison fair, the strong augmented baseline gets the same augmentation used for our model: random bias fields, gamma, contrast, and Torchio artifact simulation. It also gets the same epochs, the same hyperparameter search budget of equal trials, and the same parameter count. Our layer adds only a few scalars per channel. We count beating it as a reduction of at least one third in the unseen-scanner MRI drop, averaged over three seeds, with no loss beyond 0.5 mDice on in-distribution CT.

For polyps, the strongest numbers in the file are these. DPN68×2 from Galdrán et al. reaches 76.19% Dice on ETIS after Kvasir training. TransNetR reports a 2.81% mIoU gain over the next best model on PolypGen, but the file gives no absolute value. We compare against TransNetR retrained on our pipeline.

For retinal vessels, ErrorNet reports 71.0 average cross-dataset Dice from CHASE training. For cardiac calibration, the file reports ECE for bSSFP→LGE as 16.78% with local temperature scaling and 15.37% with Ouyang et al.'s method. We apply that post-hoc calibrator on top of both the baseline and our model, because it addresses a different stage of the pipeline.

The ablations isolate the key ingredient. The first uses unconstrained learned 5×5 convolutions at the same positions with matched parameters, which tests whether "extra early layers" explain any gain. The second uses instance normalization, the infinite-surround limit. The third defines $\sigma$ in pixels rather than millimeters, run with and without resampling to a common spacing. The fourth uses subtractive-only interaction without the divisive denominator. The fifth removes the wiring cost, which tests whether self-organized short surrounds matter.

Why not just…?

Why not just augment with random bias fields and intensity transforms? This is the real threat. Augmentation directly targets the distortions our layer is invariant to. The evidence against it being sufficient is indirect: AMOS models trained with extensive augmentation still drop more than 20% on unseen scanners. The decisive test is a two-by-two design crossing augmentation with our layer, evaluated only on real shifts that augmentation did not simulate: unseen AMOS MRI scanners, LGE, PolypGen centers, and ETIS. If our layer gives no gain on top of augmentation there, we will report that the invariance is learnable from data and the architecture adds nothing.

Why not just use instance normalization or bias-correct the input with N4 or histogram matching? Instance normalization removes one global gain and cannot cancel a spatially varying one. It is literally our layer with infinite surround, so the surround-width ablation measures this directly. N4 correction is a strong MRI-specific baseline, and we include it on AMOS MRI and MS-CMRSeg. It has no counterpart for endoscope illumination or LGE contrast change, and it runs as a separate, tunable preprocessing step per modality. If N4 plus a standard U-Net matches us on MRI, the method's value reduces to modalities without a physics-based corrector. We would state that explicitly.

Why not just use a transformer for global context? AMOS reports that Swin-UNetr trails a 3D-UNet (86.32% vs 89.04% mDice), and its global attention remained subject to the scanner drop. Global context does not address the problem of reading uncalibrated local intensities. We still include TransNetR on polyps so the question is settled on shared data.

Why not just repair or recalibrate outputs, as ErrorNet or Ouyang et al. do? These methods fix fragmented topology or overconfidence after the network has already misread the input. Ours acts on the input reading. We therefore test stacking, and expect gains to add. If the post-hoc methods alone close most of the gap, the input-side layer is not worth its place.

Decisive Experiments

On AMOS we train on CT plus MRI from a subset of scanners and hold out the remaining MRI scanners. We report per-organ Dice, with adrenal glands, pancreas, and vessels shown separately as the thin and irregular cases, alongside ECE. On MS-CMRSeg we train on bSSFP and test on LGE and on each Torchio artifact type separately. The key readout is the predicted sign pattern: gains on bias field and contrast shift, and no gain or a loss on noise, ghosting, and spikes. For polyps we train on Kvasir-SEG and test on PolypGen per center, ETIS, and CVC-ClinicDB. For retina we train on CHASE and test on DRIVE, STARE, ARIA, and HRF.

The idea is supported only if three conditions all hold across three seeds: our layer reduces the unseen-domain drop on top of matched augmentation; the pixel-unit and unconstrained-conv ablations do not reproduce the gain; and the synthetic sweep shows the crossover at the learned surround widths.