Deformable image registration warps one medical scan onto another so that the same anatomy lines up. Networks trained with anatomical labels, meaning organ outlines used only during training, now beat classical optimizers on the data they were trained on. They lose that lead on scans from other hospitals. Every such network also hard-codes one trade-off between tight alignment and physically plausible, fold-free deformations. We propose to spend the labels differently. Instead of using them only to train the network that produces deformations, we also use them to train a small judge network. The judge looks at two candidate deformations of the same image pair and predicts how much better one is than the other, not merely which one wins. At deployment no labels exist, so the judge alone picks, for each new image pair, the trade-off setting a user would otherwise pick by hand.
The borrowed ingredient is magnitude-aware preference learning. This is a family of methods from language-model alignment that trains on pairs of outputs while also telling the model the size of the quality gap between them. Here it becomes the training objective for the judge, not for the registration network. The core bet is about what transfers. Relative, same-pair quality differences should carry over to new scanners and labeling protocols better than absolute quality scores do, because a comparison within one image pair cancels whatever makes that particular pair easy or hard. If that is true, the judge can recover much of the per-pair "oracle" choice out of distribution, which is exactly where label-supervised registration currently falls behind classical tools.
Why this intersection, and why not the obvious version
The two literatures meet around one shared complaint. Overlap losses such as Dice score a harmless boundary shift and a hole punched through an organ the same way, as Learning deformable registration of medical images with anatomical constraints (AC-RegNet, which adds an autoencoder shape prior to Dice training) argues. Binary preference objectives likewise treat a near-tie and a catastrophic failure alike, as Direct Preference Optimization with an Offset argues. That paper (ODPO) adds the quality gap as an offset in the loss.
The obvious combination is to run ODPO or Margin Matching Preference Optimization directly on the registration network. MMPO replaces hard preference labels with soft targets $\sigma(\gamma m)$, where $m$ is the quality gap. We think this direct version is the wrong idea, for two reasons. First, most registration networks are deterministic regressors with no likelihood for preference losses to act on. Second, and more decisive, preference learning earns its keep when the quality signal cannot be differentiated or is only available as comparisons. During label-driven training the Dice signal is already dense and differentiable, so a reviewer would rightly say "just reweight the Dice loss." The place where no usable signal exists is deployment, where labels are absent. That is where we put the magnitude-aware machinery.
The file reports no prior work combining these directions. Two adjacent works define the gap. Multi-Objective Learning for Deformable Image Registration produces a spread of deformations along the alignment/smoothness trade-off using hypervolume maximization, but states that steering toward preferred solutions without manual intervention is unsolved. HyperMorph trains one network conditioned on the regularization weight $\lambda$ and leaves $\lambda$ to a test-time slider. Both supply the candidate set that we propose to steer automatically. Deep Learning in Medical Image Registration: Magic or Mirage? supplies the motivation: supervised networks reach Dice near 0.90 on OASIS versus about 0.80 for classical tools, yet fall below FireANTs and Greedy on LPBA40, MGH10, IBSR18 and CUMC12. It also supplies a constraint. A judge trained on the same OASIS labels could inherit the same brittleness, and our experiments are designed to expose that.
Core Claim
A label-trained pairwise critic, meaning a network that scores candidate deformations, can choose the regularization trade-off per image pair without labels. We claim three things. First, training the critic with magnitude-aware soft targets beats both a binary ranking critic and a critic that regresses absolute quality. Second, the advantage of within-pair relative scoring over absolute regression grows out of distribution. Third, this closes a meaningful fraction of the gap between a single globally tuned $\lambda$ and the per-pair label oracle, in and out of distribution.
Research Program
Candidate generation. We train a HyperMorph-style network on Learn2Reg OASIS with Dice plus smoothness losses. For each training image pair, we sample candidate deformations $\phi_a, \phi_b$ at different $\lambda$ values, from different checkpoints, and from the multi-decoder front of Grewal et al.
Label-based quality score. Each candidate gets a quality score $q(\phi)$ computed from training labels. The score combines three terms. The first is Dice. The second is 95th-percentile Hausdorff distance, which measures boundary distance. The third is a penalty on folding, defined as the fraction of voxels whose Jacobian determinant is at most zero; a non-positive Jacobian determinant means the deformation turns tissue inside out. Candidates also receive the AC-RegNet autoencoder latent distance as a shape-plausibility term, so that topological tears cost more than boundary nudges.
Critic and training objective. The critic $s_\psi$ has parameters $\psi$ and returns a scalar score. Its inputs are the fixed image, the warped moving image and the deformation field; it receives no labels. It is trained on within-pair comparisons with the MMPO-style objective
$$\mathcal{L}(\psi) = -\,\mathbb{E}\big[\,p\log\sigma(\Delta s) + (1-p)\log\sigma(-\Delta s)\,\big],\quad p=\sigma\big(\gamma\,(q(\phi_a)-q(\phi_b))\big),$$
where $\Delta s = s_\psi(\phi_a)-s_\psi(\phi_b)$, the expectation runs over candidate pairs of the same image pair, and $\gamma$ sets how sharply quality gaps map to target probabilities. Near-ties produce targets near one half, so the critic is not pushed to saturate on them. Large gaps produce confident targets. We also test the ODPO offset form and a relative-regression form in the spirit of REBEL and value-based calibration, both of which regress score differences directly onto reward differences.
Deployment. At test time the critic scans the $\lambda$ family, or the multi-objective front, for each new image pair and selects the highest-scoring deformation. This adds a few forward passes and keeps inference fast.
Test-time fine-tuning variant. A secondary variant uses the critic's gradient for brief per-pair fine-tuning. Because this invites exploiting the critic's blind spots, we anchor it to the original prediction and report folding rates alongside accuracy.
Transfer to other settings. We repeat the recipe on chest X-rays, training on JSRT and testing on Montgomery and Shenzhen with the AC-RegNet codebase. We also run it on Learn2Reg Abdomen MR-CT. That task is multimodal, meaning MR is aligned to CT, so intensity-based selection rules are weakest there.
Baselines
The naive reference is a single $\lambda$ tuned on a labeled validation split and applied to every test pair. This is what practitioners do today. The ceiling is the per-pair oracle, which picks each pair's best candidate using the test labels. We report every method as the fraction of the naive-to-oracle gap it closes, alongside raw Dice, HD95, the standard deviation of the log Jacobian determinant (a smoothness measure) and folding rate. Beating the naive reference means closing a positive, statistically reliable fraction of that gap without raising folding.
The label-free selection competitor is HyperMorph with $\lambda$ chosen per pair by maximizing normalized cross-correlation (NCC), or mutual information in the multimodal task, between warped and fixed images. It uses the same candidate family, so the only difference from our method is the selection rule. The critic must beat it on Dice and HD95 at equal or lower folding.
The key ablations isolate magnitude awareness and relativity. All of them use the same critic architecture, the same candidate pairs, the same number of training steps and the same tuning budget, with $\gamma$ or its equivalent tuned on one validation split for every variant. The first is a binary Bradley–Terry critic trained only on which candidate won. The second is an absolute critic that regresses $q(\phi)$ directly, which is the standard registration quality-estimation approach. The third is a hybrid of preference and regression, motivated by HelpSteer2-Preference, which found the two signals complementary for reward models.
The strongest existing out-of-distribution methods are FireANTs and Greedy on the four brain transfer sets. The file gives no reported Dice values for them there; Jena et al. report only that they beat OASIS-trained networks. We will therefore run them with their default published settings. Beating them out of distribution would be the headline result, and matching them while keeping network inference speed would still be useful.
For in-distribution OASIS, the target reference is supervised networks at Dice near 0.90 (Jena et al.). For Abdomen MR-CT, the strongest reported number is LLM-Morph at Dice 80.11% and HD95 9.89 mm (Large Language Models for Multimodal Deformable Image Registration). Its public code allows a candidate family to be built on it. For JSRT, AC-RegNet reports DSC 0.943. For Montgomery and Shenzhen, no reported result was found on Lacuna for cross-dataset transfer, so we will measure AC-RegNet ourselves.
Why not just…?
Why not just apply ODPO or MMPO directly to the registration network? During training, labels give a dense, differentiable Dice signal, so a preference loss on the generator mostly re-expresses a loss we could optimize directly. Doing so would also require a probabilistic generator such as probabilistic diffeomorphic registration. We will include a Dice-reweighting baseline that upweights hard pairs, since sample reweighting is the direct registration analogue of RDO and VideoDPO. We expect it to help in distribution. It cannot help at deployment, because it provides nothing to select with when labels are absent.
Why not just tune one global $\lambda$ carefully? This is our naive baseline. HyperMorph already makes the search cheap. The open question is whether the best $\lambda$ varies per pair enough to matter. The oracle gap measures exactly that. If the gap is small, the whole proposal is moot, and we will report that honestly as a negative finding.
Why not just regress absolute quality from the images? This is the most serious threat, and it may well suffice in distribution. Our argument is that absolute Dice varies strongly across pairs and sites, for example with label protocol and anatomy size, whereas within-pair differences cancel those offsets. The absolute-versus-relative ablation, run separately in and out of distribution, settles it. If regression matches the pairwise critic out of distribution, the preference framing adds nothing and we will say so.
Why not just select by intensity similarity? Intensity similarity is exactly what the mutual-information ceiling limits (Jena et al.). It also breaks down across modalities, as Hu et al. note for MR-to-ultrasound registration, where intensities share no simple relationship. The NCC and mutual-information selection baseline tests this directly, and the MR-CT task is where we expect the largest margin over it.
Why not just use classical optimizers out of distribution? They are the bar to clear there, and we run them as baselines. Classical methods remain limited to Dice around 0.80 in distribution and are slower. If the critic cannot close the out-of-distribution gap to them, the fair conclusion is that label-trained components share one brittleness, whether they generate or judge.
Decisive Checks
Four results decide the proposal. The first is whether, on LPBA40, MGH10, IBSR18 and CUMC12, the magnitude-aware critic closes more of the oracle gap than both the binary and the absolute-regression critics, with non-overlapping bootstrap intervals over image pairs. The second is whether its advantage over absolute regression is larger out of distribution than in distribution, which is the specific prediction of the relativity argument. The third is whether critic-selected deformations keep folding at or below the naive baseline; if they do not, the critic has learned to reward tearing. The fourth is a $\gamma$ sweep showing whether soft targets collapse toward binary behavior at large $\gamma$, the scale sensitivity MMPO flags. Failure on the first check with success in distribution would itself be informative. It would suggest that the "mirage" of Jena et al. comes from the labels themselves rather than from how label supervision enters the network.
References
- Learning deformable registration of medical images with anatomical constraints
- Direct Preference Optimization with an Offset
- Margin Matching Preference Optimization: Enhanced Model Alignment with Granular Feedback
- Multi-Objective Learning for Deformable Image Registration
- HyperMorph: Amortized Hyperparameter Learning for Image Registration
- Deep Learning in Medical Image Registration: Magic or Mirage?
- REBEL: Reinforcement Learning via Regressing Relative Rewards
- Don't Forget Your Reward Values: Language Model Alignment via Value-based Calibration
- HelpSteer2-Preference: Complementing Ratings with Preferences
- Large Language Models for Multimodal Deformable Image Registration
- Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces
- Reward Difference Optimization For Sample Reweighting In Offline RLHF
- Label-driven weakly-supervised learning for multimodal deformarle image registration