Supervising the Landscape, Not the Warp: Star-Convex Features for Few-Label Deformable Registration

Image registration means finding the spatial warp that lines up one scan with another. Medical imaging has a well-documented puzzle here. A careful head-to-head study, Deep Learning in Medical Image Registration: Magic or Mirage?, found that on brain MRI, unsupervised deep networks and properly tuned classical optimizers both reach about 0.80 Dice overlap. Deep networks trained with anatomical label supervision climb above 0.90 Dice. Those supervised networks then collapse when the scanner changes (trained on OASIS, tested on IBSR18), while classical optimizers keep working. Supervision carries the accuracy, and pair-by-pair optimization carries the robustness. Today the two cannot be combined, because supervision is stored in a network that predicts warps directly. With few patients, that network memorizes its training distribution.

We propose to put the supervision somewhere else. We would use a handful of labeled scans, between 1 and 10, to train a feature extractor whose only job is to shape the objective that a classical optimizer minimizes. Each pair is still solved by explicit optimization at test time. The feature extractor is trained so that this optimization problem has one wide basin, centered on the anatomically correct warp. "Star-convex" is the precise version of this: from any starting warp, moving in a straight line toward the correct warp always lowers the objective. PRISE showed that such a constraint makes iterative alignment converge reliably, but only for 8-parameter homographies on 2D natural and satellite images. Our claim has two parts. First, a sampled, low-frequency version of the constraint can be imposed on dense deformable warps using very few labels. Second, because the learned object is a landscape rather than a warp predictor, it should keep most of the supervised accuracy gain under scanner shift, where supervised warp networks lose it.

Core Claim

Let $f_\theta$ be a 3D feature network with parameters $\theta$. Let $v$ be a low-resolution stationary velocity field, a smooth flow whose integration gives a fold-free warp $\phi_v$. For fixed image $F$ and moving image $M$, the registration energy is $E_\theta(v) = \lVert f_\theta(F) - f_\theta(M\circ\phi_v)\rVert^2 + \lambda R(v)$. Here $R$ is a smoothness penalty and $\lambda$ its weight. Let $v^*$ be a reference optimum, described below. Following PRISE, we ask that $E_\theta$ sit above a quadratic bowl centered at $v^*$, and that it decrease along the straight path toward $v^*$. We enforce both conditions with hinge penalties on sampled perturbations $v$:

$$ \mathcal{L}_{\text{star}} = \Big[\tfrac{\mu}{2}\lVert v-v^*\rVert^2 - \big(E_\theta(v)-E_\theta(v^*)\big)\Big]_+ + \Big[\mu\lVert v-v^*\rVert^2 - \langle \nabla E_\theta(v),\, v-v^*\rangle\Big]_+ , $$

where $\mu>0$ sets the bowl's curvature and $[\cdot]_+$ is $\max(0,\cdot)$.

The step beyond PRISE is in the choice of the sampled perturbations $v$. A dense field has far too many directions to constrain them all. We therefore sample perturbations from a low-frequency subspace of the velocity field, made of coarse grids and smooth random fields at several scales. Basins in registration are known to break mainly at large, coarse displacements. Lombaert et al. note that local gradients go blind once structures move past their own support. Fine-scale residuals are left to the ordinary smoothness term.

Reference optima come from two sources. The first is self-warps: a real scan is deformed by a known random diffeomorphism and given a randomized appearance, so the true $v^*$ is known exactly. The second is real inter-subject pairs among the few labeled scans, where $v^*$ is obtained offline by optimizing label overlap plus smoothness. That second source is how the "supervision secret sauce" enters, without ever training a warp predictor.

Research Program

The work has three parts.

The first part is the training loss and sampler. This includes multi-scale low-frequency perturbation sampling and efficient computation of the gradient term through a few steps of differentiable velocity-field integration. Training costs some memory, but test-time cost matches ConvexAdam-style GPU optimization. We initialize $f_\theta$ from the synthetic-pretrained network of Learning General-Purpose Biomedical Volume Representations using Randomized Synthesis. That paper trains a 3D U-Net on randomized shapes and textures and pairs it with ConvexAdam for zero-shot registration. It supplies our starting features and our test-time solver. Its features are shaped only by a contrastive loss that matches corresponding points; nothing constrains the landscape away from the optimum.

The second part measures landscape geometry directly instead of inferring it from Dice. On held-out pairs, we cast random coarse rays from perturbed starting points toward the label-optimal warp and record two quantities. One is the fraction of rays along which the energy is not monotone. The other is the capture range: the largest initial misalignment from which the solver still reaches a fixed fraction of best-case Dice.

The third part tests the robustness hypothesis. We run the shift protocol of Magic or Mirage while varying the number of labeled training scans.

Baselines

The naive reference point is a well-tuned classical optimizer on raw intensities, using Greedy or ANTs with the tuning protocol of Magic or Mirage to avoid the "instrumentation bias" that paper documents. On OASIS it scores about 0.80 Dice, the same as unsupervised TransMorph. It defines the floor our method must beat with labels in hand.

The strongest existing competitor for in-distribution accuracy is a label-supervised deep registration network, LKU-Net, trained on the same k labeled subjects. Magic or Mirage reports that such supervision lifts OASIS Dice "from ~0.80 to over 0.90" with full training labels. No reported result was found on Lacuna for the few-label (k ≤ 10) regime, so we will measure it ourselves. It is the threat on OASIS and, per that paper, the expected loser on IBSR18.

The strongest competitor on the optimization side is the synthetic-pretrained features with ConvexAdam. The randomized-synthesis paper reports a 26-point median Dice gain over standard methods on Learn2Reg abdominal MR-CT, but no absolute value is given in the file. ConvexAdam won Learn2Reg overall and transferred best to unseen data. SAME++, which combines self-supervised anatomical embeddings, fold-free velocity fields, and instance optimization, is reported as 4.2 to 8.2% Dice above ConvexAdam and LapIRN on abdominal and head-and-neck CT. It is the strongest label-free hybrid and is run as published. On cardiac CT with 4 labels, we compare against DeepRS, which reports 77.6% registration Dice versus VoxelMorph's 71.7%.

All learned methods get the same k labeled scans, the same unlabeled pool, and the same synthetic pretraining initialization where applicable. Each gets an equal hyperparameter search budget, using the same number of configurations evaluated on the same validation split, and the same test-time solver iterations wherever a solver is used.

The key ablation keeps everything fixed but replaces $\mathcal{L}_{\text{star}}$ with a loss that only makes features agree at $v^*$. That loss is the contrastive objective of the synthesis paper, fine-tuned on the same labeled pairs. This isolates whether shaping the landscape away from the optimum matters. Two further ablations remove one hinge at a time and remove the label-derived optima, leaving self-warps only.

We will count it as beating the baselines if the method does both of the following on IBSR18 after training on OASIS. It must exceed the tuned classical floor and the synthetic-features-plus-ConvexAdam baseline by a statistically significant margin under a Wilcoxon signed-rank test, the test Learn2Reg uses. It must also retain at least half of the in-distribution gain that supervised LKU-Net shows, while LKU-Net's gain shrinks. Fold rates must stay at the near-zero level of fold-free baselines.

Why not just…?

Why not just train the features by backpropagating Dice through a few unrolled ConvexAdam iterations? This is the most serious alternative. It also puts supervision into features rather than a warp predictor, and it is simpler. The difference is that it only shapes the landscape along the trajectory taken from the training initialization. Star-convexity constrains a whole neighborhood of starting points. That should matter under larger initial misalignment and under shift, when test trajectories leave the region seen in training. We will run unrolled training at matched compute as a co-primary baseline and compare capture-range curves and monotonicity rates. If it matches us on IBSR18 and at large perturbations, the star-convex machinery is unnecessary, and we will report that.

Why not just use the synthetic foundation features or SAME++ embeddings off the shelf? Both are label-free. Magic or Mirage argues that label-free registration is capped by how much the image intensities say about anatomy. SAME++ partly breaks that cap but needs four cascaded stages. If adding 4 to 10 labeled scans through our loss does not improve these features, the labels are not worth collecting, and the experiment is designed to show exactly that.

Why not just fine-tune a supervised registration network on the few labels and accept the shift penalty? Magic or Mirage documents the shift collapse with full data. With fewer labels the overfitting should be worse, not better. Our OASIS→IBSR18 sweep over k measures this directly rather than assuming it.

Proposed Experiments

The primary experiment trains on OASIS with k ∈ {1, 4, 10} labeled subjects plus unlabeled scans, then tests in-distribution on OASIS and under shift on IBSR18. We report Dice, HD95, SDlogJ, and fold rate. The second experiment uses Learn2Reg Abdomen CT-CT and MR-CT, where large soft-tissue motion and cross-modality intensity mismatch make basins narrow. We compare against ConvexAdam, synthetic features plus ConvexAdam, and SAME++, adding DSC30 to capture the worst-case tail. The third uses MM-WHS cardiac CT with 4 labels to compare against DeepRS on its own protocol.

The decisive check is the capture-range experiment. Each test pair is initialized with coarse perturbations of growing magnitude, and we plot final Dice against initial misalignment for our features, the matching-only ablation, and unrolled training. The proposal stands if two things hold together. Our method's curve must stay flat to larger misalignments. The ray-monotonicity rate on held-out and shifted pairs must explain that flatness. If our features improve Dice at the starting alignment but not the basin width, the gain comes from better features rather than from landscape shape, and the central claim fails.