Deformable image registration computes a dense warp that maps each point in one medical scan onto the matching anatomical point in another. Registration networks trained with organ masks, rather than raw image similarity alone, are the only deep models that clearly beat classical optimization in accuracy. They also break down when the scanner, protocol, or contrast phase changes, and then lose to the same classical tools they beat in-distribution. We propose a way to keep the label signal working on new domains. Pass the warp through the unlabeled new-domain scan: warp a labeled reference scan $i$ onto the new scan, then warp the new scan onto a second labeled reference $j$. The composed result must reproduce $j$'s known masks. Both ends of this cycle have ground truth, but the path runs through the shifted image. The network therefore gets a label-based training signal that it can satisfy reliably only by reading the new image's anatomy correctly.
The setting where this matters most is the abdomen. Organs such as the pancreas, duodenum and adrenal glands vary greatly in shape between patients and are barely separable by intensity from surrounding fat and bowel. Abdominal benchmarks also show large shifts across scanners and contrast phases. No registration work in the file studies label supervision under these combined conditions. In two sentences: label-supervised registration fails out-of-distribution because its use of labels stops at the training set. We extend the labels into the new domain by requiring that label-to-label cycles routed through each unlabeled target scan close correctly, and we test whether this keeps the accuracy edge on shape-variable organs where intensity-only methods cannot compete.
Core Claim
Deep Learning in Medical Image Registration: Magic or Mirage? is the motivating evidence. On brain MRI it shows that label-supervised networks reach Dice near 0.90 in-distribution on OASIS, against about 0.80 for carefully tuned classical tools. The same paper shows that unsupervised networks give no accuracy gain over classical tools, and that OASIS-trained supervised networks lose to FireANTs and Greedy on LPBA40, MGH10, IBSR18 and CUMC12. The paper attributes the edge to labels letting networks exceed the mutual-information limit, meaning how much intensities reveal about anatomy. It attributes the collapse to networks memorizing how a specific training distribution's intensities map to anatomy. It offers no fix.
Our claim is that the collapse is not inherent to label supervision. It comes from the fact that label losses are only ever applied to source-domain images. A label-closure loss routed through target-domain images, used either as unsupervised domain adaptation over a pool of unlabeled target scans or as a short per-pair test-time adaptation, should recover most of the in-distribution edge out of distribution. The gains should be concentrated on low-contrast, shape-variable organs, measured by boundary metrics and not only by volume overlap.
Mechanism
Let $f_\theta$ be a pairwise registration network. Let $t$ be an unlabeled target-domain scan, and let $\{(a_i, s_i)\}$ be labeled source-domain reference scans with their organ masks. Write $W_{i\to t}$ for the warp $f_\theta$ predicts from $a_i$ to $t$, and $W_{t\to j}$ for the warp from $t$ to $a_j$. The adaptation objective is
$$\mathcal{L}(t) = \sum_{i \neq j} \Big[1 - \mathrm{Dice}\big(W_{t\to j}(W_{i\to t}(s_i)),\, s_j\big)\Big] + \lambda_{c}\, \mathrm{Var}_i\big[W_{i\to t}(s_i)\big] + \lambda_{s}\sum_i S\big(W_{i\to t}(a_i), t\big) + \lambda_{r} R(\theta).$$
The first term is the label cycle. The second is a consensus term: masks propagated from different references into $t$ should agree voxelwise. $S$ is a modality-robust local similarity such as local self-similarity descriptors, and $R$ penalizes non-smooth warps through the spread of the log-Jacobian determinant.
The cycle term has a known degenerate solution. The network could map $t$ into some shared but wrong intermediate frame, and the errors would cancel along the path. The network that predicts $W_{t\to j}$ never sees $a_i$, so cancellation requires a frame that is consistent across all references. The similarity term and the consensus term tie that frame to $t$'s actual anatomy. We treat this degeneracy as an empirical risk to be monitored, not as something the method rules out.
The method combines techniques from several sources. It reuses the label-overlap training of VoxelMorph and the weakly supervised multimodal line (Label-driven weakly-supervised learning for multimodal deformable image registration). It takes the brief per-pair adaptation idea from Encoding Matching Criteria for Cross-domain Deformable Image Registration. That paper adapts a self-similarity encoder for 5 to 20 iterations using intensity matching only, so its adaptation signal cannot separate adjacent soft tissues with similar local texture. Our adaptation signal is anatomical. The difference from SynthMorph is that SynthMorph avoids shift by training on images synthesized from brain label maps with randomized intensities. It never sees the real target domain, and it presumes the label maps cover realistic shape variation. We use real target images and treat synthesis as a complementary baseline.
Research Program
We first build the abdominal analogue of the Jena et al. study. Without one, "out-of-distribution collapse" in the abdomen is an assumption, not a finding. The source domain is one center's or one scanner's CT from AMOS, which provides 15 organ masks from 8 scanners and both CT and MRI. Target domains are held-out AMOS scanners, AMOS MRI, and portal-venous versus arterial phase subsets of AbdomenCT-1K. These datasets were built for segmentation, but their masks give inter-patient registration ground truth. The Learn2Reg Abdomen CT-CT task from Learn2Reg serves as the in-distribution anchor. We report Dice, Normalized Surface Dice, 95th-percentile Hausdorff distance, and the 30th-percentile Dice robustness score for robustness. Following AbdomenCT-1K's finding that volumetric Dice hides boundary errors on tortuous organs, results are stratified by organ and by how far each patient's organ shape deviates from the reference population.
We then add the cycle objective in two modes. In the transductive mode, we adapt once on unlabeled target-domain scans. In the test-time mode, we adapt per pair for a fixed iteration budget. We vary the number and shape diversity of references. This is where morphological variability enters as a design variable: references spanning elongated, curved and compact pancreases should give the consensus term more to work with than references near the mean shape. Finally, we replicate the method on the brain protocol of Jena et al., OASIS to LPBA40, MGH10, IBSR18 and CUMC12, so the result can be compared directly with the one paper that established the collapse.
Baselines
The naive reference point is a label-supervised VoxelMorph-style network trained on source data and applied unchanged to the target domain, alongside affine-only alignment. These establish the size of the gap. The file reports no abdominal registration numbers under scanner or phase shift. The nearest evidence of collapse is from segmentation: AMOS reports a drop of over 20% mDice on unseen MRI scanners, and AbdomenCT-1K reports a 15% Dice drop from arterial to portal-venous phase.
The strongest existing competitor is classical optimization, specifically ConvexAdam, the overall Learn2Reg winner, along with FireANTs and Greedy. These are the methods Jena et al. show winning out of distribution. Jena et al. warn that classical tools are often run with poor default settings, so we tune their hyperparameters on the same source validation split with a search budget matched to the network's. The bar is the in-distribution numbers from that paper: about 0.90 Dice for supervised networks versus about 0.80 for classical tools on OASIS. Jena et al. give no per-dataset numbers out of distribution, and Learn2Reg reports none for the abdominal task in the file ("no reported result found on Lacuna" for per-task values). Beating this baseline means matching or exceeding tuned ConvexAdam in Normalized Surface Dice on pancreas, duodenum and adrenal glands in every target domain, while staying under the classical runtime.
Three further baselines target the most plausible shortcuts. The first is the intensity-only adaptation of Wang et al., run at the same iteration count. The second is a SynthMorph-style network trained on intensity-randomized images synthesized from AMOS source labels. The third is segment-then-register: an nnU-Net trained on the source domain predicts target masks, and registration is then guided by those predicted masks. All learned methods share one backbone, the same source labels, and matched wall-clock time at test.
The key ablation removes each loss term in turn: cycle only, consensus only, similarity only. Similarity-only adaptation reduces to Wang et al.'s regime. If it matches the full objective, the claim is false.
Why not just…?
Why not just use tuned classical optimization, which already wins out of distribution? Classical methods are bounded by how much intensity reveals about anatomy, which Jena et al. formalize as the mutual-information ceiling. The pancreas, duodenum and surrounding fat are exactly where that bound is tight, which is why the in-distribution label edge exists at all. Classical tools are also slow: VoxelMorph reports over 2 hours on CPU for ANTs, and Wang et al. report 114.7 seconds for SyN. Still, if tuned ConvexAdam matches us on low-contrast organs out of distribution, the honest conclusion is that the label edge does not survive shift. Our organ-stratified boundary comparison will show whether it does.
Why not just segment the target and register the masks? This is the most serious threat, because it also carries label knowledge into the target. Its weakness is that the segmenter shifts too: AbdomenCT-1K reports a 15% phase-shift drop, AMOS reports over 20% across MRI scanners, and AMOS finds small organs segmented worst. Mask errors then pass straight into the warp. Our method uses many references, and their disagreement provides a check that a single segmenter lacks. If segment-then-register with a segmenter adapted to the target domain ties with us, we will report it as such. The cycle loss could also be used to adapt the segmenter.
Why not just randomize intensities during training, as SynthMorph does? Synthesis handles contrast and scanner variation well in the brain. But it never sees real target appearance, such as phase-specific enhancement patterns or MRI vendor artifacts. It also cannot create shape variation beyond the label maps it is given, and abdominal shape variation is large. Synthesis is compatible with our method, so we test synthesis alone, cycle alone, and both together. If synthesis alone closes the gap, the cycle loss adds nothing.
Why not just do unsupervised test-time adaptation as in Wang et al.? Their adaptation loss is intensity-based. Jena et al. show that intensity-only training gives no accuracy gain over classical tools, so intensity-only adaptation should regain robustness at the cost of the label edge. The similarity-only ablation tests this directly.
Decisive Checks
The proposal succeeds if four things hold. First, in at least the phase-shift and the unseen-MRI-scanner targets, cycle adaptation beats both the unadapted supervised network and tuned ConvexAdam in Normalized Surface Dice on pancreas and duodenum. Second, the gain over intensity-only adaptation is larger on low-contrast, shape-variable organs than on liver and spleen; this is the prediction from the mutual-information argument. Third, the degeneracy diagnostic stays clean: on a labeled held-out subset of each target, cycle loss and true target Dice must fall together. If the cycle loss keeps dropping while true Dice stalls, that shows the network has found a shared but wrong intermediate frame. Fourth, on the brain replication, the method reverses the out-of-distribution ordering that Jena et al. report. As a by-product, we test whether consensus disagreement predicts per-case registration error, which would give a label-free quality-control score for deployment.
References
- Deep Learning in Medical Image Registration: Magic or Mirage?
- VoxelMorph: A Learning Framework for Deformable Medical Image Registration.
- Label-driven weakly-supervised learning for multimodal deformarle image registration
- Encoding Matching Criteria for Cross-domain Deformable Image Registration
- Anatomy-aware and acquisition-agnostic joint registration with SynthMorph
- AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation
- AbdomenCT-1K: Is Abdominal Organ Segmentation a Solved Problem?
- Learn2Reg: Comprehensive Multi-Task Medical Image Registration Challenge, Dataset and Evaluation in the Era of Deep Learning