Tissue-Dependent Contrast Augmentation for Phase-, Sequence- and Scanner-Robust Medical Image Models

Medical imaging models break when image brightness changes for reasons that have nothing to do with anatomy. A contrast agent injected into the blood makes arteries bright in one scan and veins bright a minute later. Switching MRI sequences can invert which organ looks brighter. A different hospital's scanner rescales everything nonlinearly. The usual defenses are global. They normalize histograms, randomly remap intensities, or train on edge maps, and all of them treat appearance change as one function applied to every voxel's brightness. Contrast agents and sequence changes do not behave that way. Two tissues that look identical before injection can diverge after it, and two tissues can swap order between T1- and T2-weighted MRI. The new brightness of a voxel depends on which tissue it belongs to, not only on its old brightness.

We propose to train models with tissue-dependent appearance augmentation. Each training volume is split into soft, unsupervised tissue compartments, preferably derived from how voxels brighten over the phases of a dynamic contrast scan. Each compartment then receives its own random intensity curve, and a smooth scanner-like bias field is applied on top. A consistency loss requires the network's internal features to be the same across these variants. The claim is narrow and testable: much of the remaining failure under contrast reversal and phase changes comes from the global form of current augmentations, and relaxing that single assumption closes a substantial part of the gap. The headline case is liver MRI, where a strong label-efficient model trained on a contrast-enhanced phase drops to 0.7673 Dice on T2-weighted images.

Core Claim and What Is New

The motivating evidence comes from Label-Efficient Cross-Modality Generalization for Liver Segmentation in Multi-Phase MRI. That paper trains a pretrained STU-Net with cross pseudo supervision on 30 labeled scans from the late contrast phase (GED4) of the LiQA dataset. It reaches 0.9685 Dice on GED4 and transfers zero-shot to unenhanced T1-weighted images at 0.9465. It falls to 0.7673 on T2-weighted images, which the authors attribute to contrast reversal.

The closest method is CAR. CAR trains a registration network on single-contrast MRI. It augments the images with stacks of random 1×1 convolutions and LeakyReLUs, then penalizes differences between the latent codes of the original and augmented images. A 1×1 convolution acts on each voxel's intensity independently of its location. It can therefore invert or bend the global intensity scale, but it cannot make two equally bright tissues diverge, and it cannot flip liver against spleen while leaving liver against fat unchanged. CAR was also evaluated only on 2D static MRI contrasts (CamCAN brain, CMRxRecon T1 mapping), with no dynamic contrast and no multi-site shift.

Our proposal keeps CAR's latent-consistency idea, which supplies the method. It replaces the voxel-wise remapping with a spatially varying, compartment-wise one, extends the approach to 3D, and evaluates it where phase, sequence and scanner shifts co-occur.

A second ingredient addresses the contrast-kinetics direction. When several dynamic phases of the same patient are available, compartments are found by clustering each voxel's enhancement curve, so the augmentation perturbs groups of tissue that really do change together. This differs from label-driven synthesis such as SynthMorph, which CAR uses as a baseline. SynthMorph needs anatomical label maps and replaces the image with a synthetic one. We keep real anatomy and texture and need no labels.

Research Program

The augmentation has three parts. The first is a soft partition of the volume into $K$ compartments. We will compare two sources for it: kinetic clustering of multi-phase curves, used when phases exist, and supervoxels computed on a single volume. The second is an independent random smooth intensity curve per compartment. The curves may be monotone or non-monotone, and they are blended with soft memberships so that partition boundaries do not create artificial edges. The third is a low-frequency multiplicative bias field that mimics coil and vendor effects.

The soft blending matters. If compartment boundaries produced sharp intensity steps, the network would learn to segment along invented edges. One ablation therefore uses hard partitions to show that this failure is real and that the blending fixes it.

Training uses the task loss plus a consistency term,

$$\mathcal{L} = \mathcal{L}_{\text{task}} + \lambda \,\| Z(I) - Z(A_{\pi,f,b}(I)) \|_2^2 .$$

Here $Z$ is the encoder's projected latent, $I$ the input volume, $A$ the augmentation with partition $\pi$, per-compartment curves $f$ and bias field $b$, and $\lambda$ a weight. This is the CAR regularizer with a different augmentation.

We apply it to two tasks that share the encoder recipe. The first is 3D liver segmentation in the LiQA setting. The second is 3D deformable registration across contrast phases. For registration, a training pair is two differently augmented copies of the same volume, one of which is also warped by a known random deformation. This gives exact displacement supervision on real anatomy, in addition to an unsupervised local normalized cross-correlation loss on real pairs.

The domain-shift direction is used to stress-test the results rather than as a third mechanism. LiQA's four centers and three vendors let us separate phase or sequence shift from scanner shift, which Label-Efficient Cross-Modality Generalization for Liver Segmentation in Multi-Phase MRI did not do. We will report every result both pooled and as leave-one-vendor-out.

Baselines

The naive reference point is a 3D nnU-Net-style segmenter, using the same STU-Net backbone, trained on the 30 labeled GED4 scans with standard z-score normalization and default intensity augmentation. For registration, the naive reference is VoxelMorph-style training with local normalized cross-correlation. These show how much any robustness method buys.

The strongest published competitor on the central question is the label-efficient cross pseudo supervision pipeline of Label-Efficient Cross-Modality Generalization for Liver Segmentation in Multi-Phase MRI. It reports 0.9685 Dice on GED4, 0.9465 zero-shot on T1-weighted images and 0.7673 zero-shot on T2-weighted images. We will reproduce it with the same backbone initialization, the same 30 labeled cases, the same unlabeled pool and the same epochs. Our augmentation will be added to that pipeline, not substituted for it, so that any gain comes from the augmentation alone. Beating it means a T2-weighted Dice clearly above the reproduced value, which we expect near 0.77, across three seeds, with GED4 and T1-weighted Dice within noise of their reported values. We regard reaching about 0.90 on T2-weighted images as the bar for a strong result.

The key ablation compares against CAR's global random-convolution augmentation CAR, run inside the identical pipeline with the identical consistency loss and an equal number of augmented views per step. Only the spatial dependence of the remapping differs. A second ablation swaps the compartment source among kinetic clustering, single-volume supervoxels, oracle organ labels (a SynthMorph-like upper reference) and a single compartment, which reduces to the global case.

For the translation family, we run the crossMoDA recipe described in crossMoDA Challenge: Evolution of Cross-Modality Domain Adaptation Techniques for Vestibular Schwannoma and Cochlea Segmentation from 2021 to 2023: CycleGAN translation of GED4 into T2-weighted style, nnU-Net training, then pseudo-label self-training. It gets the same unlabeled T2-weighted scans and a matched GPU-hour budget. That recipe reaches median Dice above 87% for vestibular schwannoma on its own benchmark, but no reported result found on Lacuna for it on LiQA.

For gradient representations, we run Gradient-Map-Guided Adaptive Domain Generalization for Cross Modality MRI Segmentation. It reports raising cardiac bSSFP→T2 Dice from 4.9% to 85.4%, and we run it both with and without its test-time adaptation, with latency reported alongside accuracy.

For registration, the baselines are CAR and iterative NiftyReg. Structure-aware registration network for liver DCE-CT images reports NiftyReg taking over 400 seconds per volume pair and VoxelMorph reaching 98.39% liver Dice and 0.67 mm surface distance on its in-house DCE-CT data. That structure-aware method needs vessel centerlines we do not have, so we cite it as context rather than running it.

Why not just…?

Why not just use CAR's random convolutions, which can already invert contrast? This is the most serious threat. A random 1×1 convolution with negative weights does flip the global intensity order, and that may be enough to rescue T2-weighted performance. The global-versus-compartment ablation settles the question directly, because it holds the loss, the views and the compute fixed. If global random convolutions match our method on T2-weighted images and on cross-phase registration, the core claim is false and we will report it that way. We expect a difference specifically on tissue pairs whose relative order changes inconsistently between sequences, and we will measure this by error on organ-boundary voxels between liver and spleen versus between liver and fat.

Why not just translate T2-weighted images into the training contrast with CycleGAN, as crossMoDA winners do? The translation-plus-self-training recipe is strong. Its gains, however, have plateaued around 87–88% Dice against a 92.5% supervised upper bound on crossMoDA, it needs target-domain data before deployment, and it risks erasing or inventing small structures, a failure that AngioMoCo reports for image-to-image models on thin vessels. We run it under a matched budget. If it wins on T2-weighted images, our method still has to justify itself on unseen sequences and vendors, where translation had no target data to learn from, and the leave-one-vendor-out split tests exactly that.

Why not just train on gradient maps with test-time adaptation? Gradient-Map-Guided Adaptive Domain Generalization for Cross Modality MRI Segmentation shows that the large gain requires per-volume test-time optimization; source-only results are far worse. It also fixes a class prior that pathology can violate. Edges themselves change when contrast pools in tissue. We report its accuracy with and without adaptation, together with per-volume latency. A win that holds only with test-time weight updates would not be deployable in a frozen, regulated device.

Why not just synthesize images from label maps, SynthMorph-style? That approach needs organ labels for every training volume and discards real texture. Our oracle-label ablation is effectively this baseline. If it clearly beats unsupervised compartments, then the practical contribution shrinks to "label-driven synthesis with real texture," and we would say so.

Proposed Experiments

The decisive experiment is LiQA GED4→T2-weighted liver segmentation as described above. We report Dice and 95th-percentile Hausdorff distance pooled and per vendor, with three seeds. We also report GED1–GED3, T1-weighted and diffusion-weighted transfer to check that gains on T2-weighted images are not traded against other phases. A per-vendor breakdown tells us whether the T2-weighted gap is mostly sequence or mostly scanner. If gains appear only on in-distribution vendors, the scanner-shift part of the claim fails.

The second experiment is a contrast-enhanced generalization check on MS-CMRSeg2019, training on bSSFP and testing on late gadolinium enhancement and T2, where Gradient-Map-Guided Adaptive Domain Generalization for Cross Modality MRI Segmentation provides the published comparison of 85.4% for bSSFP→T2. No reported result was found on Lacuna for its late gadolinium enhancement number.

The third experiment covers 3D registration across LiQA phases within each patient. We will manually annotate liver and spleen in about 20 held-out patients on GED1 and T2-weighted images so that warped-label Dice can be computed. We will also report the fraction of folded voxels and error against known synthetic deformations. On CMRxRecon T1 mapping, the setting of CAR, CAR's numbers are not given in the file, so we will rerun it.

A final stress test uses breast DCE-MRI from MAMA-MIA, the multi-site dataset used by Acquisition Time-Informed Breast Tumor Segmentation from Dynamic Contrast-Enhanced MRI, which reports 0.762 external Dice against 0.736 for vanilla nnU-Net on the Yunnan cohort. We test whether kinetic compartments help tumor segmentation as a complement to that paper's time-conditioned layers. The Yunnan cohort is used only if it is accessible.

The project succeeds if the compartment-wise augmentation beats the global random-convolution ablation on T2-weighted Dice and on cross-phase registration under identical budgets, and if that advantage persists on held-out vendors.