Matching Sampling Geometry, Not Intensity: Deformable-Offset Correspondence for Registering Low-Contrast Boundaries

Deformable image registration finds a smooth warp that lines up one scan with another. Learned registration networks are trained mostly by rewarding intensity or feature similarity between the warped and fixed images. At a low-contrast boundary, such as the pancreas fading into duodenum or bowel loops pressed against the sigmoid colon, that reward is almost flat: moving the boundary a few millimetres barely changes local similarity. In these places the warp is set by the smoothness penalty, which extends the interior motion outward, and not by the image. Boundaries come out misaligned even when the volume-overlap scores look good.

The proposal uses an object that segmentation networks already compute but registration has never matched. A deformable convolution is a convolution whose sampling points are shifted by learned, image-dependent offsets. In segmentation networks for low-contrast organs, these offsets bend to follow organ contours, drawing on context wider than the local intensity step (DB-MSMUNet, DUNet). The offsets therefore form a dense description of local shape that survives where intensity gradients vanish. Our claim is that a registration loss requiring these offsets to transform consistently under the estimated warp, including through its local Jacobian, supplies the missing gradient at ambiguous boundaries. That signal fixes not just where the boundary lands but how the warp stretches and rotates there.

Core Claim

Let a shared encoder with deformable layers produce, at each location $x$ and for each kernel tap $p$ in the regular grid $\mathcal{R}$, a learned offset $\Delta(x,p)$. Write $\Delta_F$ for the offsets computed on the fixed image and $\Delta_M$ for those computed on the moving image. Let $\phi(x)=x+u(x)$ be the predicted warp from fixed to moving coordinates, with Jacobian $J_\phi(x)$. If the two scans show the same anatomy, a kernel placed at $x$ in the fixed image should map, to first order, onto the kernel the moving image places at $\phi(x)$. This gives the offset-equivariance loss

$$ \mathcal{L}_{\text{off}}=\sum_{x\in B}\;\sum_{p\in\mathcal{R}}\big\| p+\Delta_M(\phi(x),p)-J_\phi(x)\,\big(p+\Delta_F(x,p)\big)\big\|^2 , $$

where $B$ is a band of locations near predicted boundaries and $\Delta_M(\phi(x),\cdot)$ is read off by trilinear interpolation. The loss constrains $J_\phi$, the local stretching and rotation of the warp, at exactly the locations where intensity similarity gives no constraint on it.

There is a degenerate solution. If the offsets collapse to zero, the loss reduces to $\|p-J_\phi p\|^2$, which is just another smoothness penalty. We prevent collapse by never letting registration gradients reach the offset predictor. That predictor is trained only by a segmentation and contour objective on a labelled set, and by a contrast-consistency term described below. Registration may use the offsets but may not reshape them.

Relation to Prior Work

Prior work plays three roles here. Learned feature-space registration motivates the matching-in-features stance and supplies baselines. CAR and AC-CAR learn contrast-invariant encoders by randomly re-coloring images with 1×1 convolutions and penalizing the latent distance between versions. The Deep Propagative Network of Li et al. matches in a learned multi-scale feature space and tunes loss weights by bilevel optimization. All three use rigid-grid encoders and image-wide similarity, so neither offers any boundary-specific signal.

Composite-loss registration constrains the design and defines the threat. The multi-perspective loss for CT/MRI lung registration (Long-COVID MPL) adds Gaussian-pyramid label overlap at six scales. Multi-Objective DIR maps a Pareto front between similarity, smoothness and landmarks, but every objective is a global average. Label-overlap losses are the most direct route to boundary alignment, and we must beat them at an equal label budget. ShiftMorph shifts features along eight fixed axis-aligned offsets and adds a self-consistency loss across shifted groups. It is the closest discrete relative of our operator, but its shifts carry no image-dependent geometry to match.

Deformable segmentation supplies the method. DB-MSMUNet embeds deformable kernels at three scales in a pancreas encoder and adds an edge-enhancement decoder, working only on 2D slices of single images. Bounding Maps shows that soft, linearly decaying targets beat hard masks under boundary ambiguity, and we adopt that style of target for the contour branch that trains the offsets.

The file also identifies a tension that a reviewer will raise: offsets predicted from contrast-sensitive activations may change when contrast changes, which would make $\mathcal{L}_{\text{off}}$ compare contrast artifacts. Nobody has tested this. We confront it directly with an offset-consistency term, the squared distance between offsets computed on two random-convolution re-colorings of the same image, and we measure offset drift as a primary diagnostic.

Research Program

We build a 3D registration network in the VoxelMorph style. Its encoder has deformable layers only at 1/4 and 1/8 resolution, which keeps the 81 offsets per voxel of a 3×3×3 kernel within GPU memory. The offset predictor and encoder are first trained on a disjoint segmentation set with three terms: a soft contour target, Dice, and the offset-consistency term under random-convolution augmentation.

Registration then trains with four terms: local normalized cross-correlation (LNCC, a windowed correlation of intensities), diffusion smoothness, $\mathcal{L}_{\text{off}}$, and an optional feature-similarity term. The offset predictor stays frozen during this stage. The band $B$ comes from the contour branch's soft output, so registration needs no test-time labels.

We also run a label-free instance-optimization variant, which refines the warp per scan pair at test time using $\mathcal{L}_{\text{off}}$. This separates the value of the signal itself from the value of amortized training. For fairness, all loss weights are tuned with the same validation budget for every method. As an analysis we run the bilevel tuner of Li et al..

The program has three analyses. First, we measure where the gain comes from: improvement as a function of distance to the boundary and of local image contrast. Second, we measure offset stability under contrast shifts, with and without the consistency term. Third, we measure whether $\mathcal{L}_{\text{off}}$ changes the Jacobian at boundaries in a direction that agrees with expert landmarks, as opposed to merely smoothing.

Baselines

The naive reference is LNCC plus diffusion smoothness on the same rigid-grid backbone, trained with the same data and steps. Every other number is read relative to it.

The strongest competitor on boundary alignment is label-supervised registration in the style of the Long-COVID MPL, with a Gaussian-pyramid label loss added to LNCC. It is the strongest because it directly rewards boundary overlap. That paper reports Dice 0.913 on its CT/MRI lung data against 0.848 for CompNet and 0.693 for SyN. We give it access to exactly the labels our offset pretraining consumes, on the same scans, to keep the label budget equal.

On the pelvic MRI benchmark, Multi-Objective DIR reports a target registration error (TRE, the distance between corresponding landmarks after warping) falling from 3.97 mm to 3.74 mm, with a 0.05% folding rate. We re-run it with its landmark objective and use 3.74 mm as the bar. For contrast robustness we re-run AC-CAR, which reports Dice 0.808 on CamCAN against 0.784 for the prior best. On Learn2Reg abdominal MR-to-CT, no reported result was found on Lacuna for AC-CAR or any other method in the file, so all numbers there are our own re-runs under one protocol. Classical SyN serves as a non-learned reference.

Three ablations isolate the key ingredient. The first adds deformable layers but drops $\mathcal{L}_{\text{off}}$ (the naive Gap-1 idea). The second keeps the same frozen deformable encoder but matches its features instead of its offsets. The third uses $\mathcal{L}_{\text{off}}$ with the Jacobian term replaced by identity, which tests whether the first-order geometry matters.

Beating the field means three things. Boundary-band HD95 (the 95th-percentile surface distance) and band Dice must improve over the label-loss baseline at equal labels, with non-overlapping confidence intervals across test pairs. Pelvic TRE must fall below 3.74 mm. Folding must stay at or below 0.05%.

Why not just…?

Why not just add a label-overlap loss, since that already targets boundaries? This is the real threat. On structures that are labelled in training, it may match us. Our claim is narrower: offsets learned from contour supervision on some structures still describe boundary geometry for unlabelled ones, and they constrain the Jacobian, not only overlap. The decisive experiment hides the bowel and sigmoid labels from both methods during registration training and evaluates on those structures. If the label loss matches us there too, the offset idea adds nothing beyond its labels and we will say so.

Why not just match features from a segmentation network, a perceptual-style loss? SINCO shows that a downstream segmentation network's loss can protect boundaries in compression, so features alone might suffice. Feature distances, however, are compared point to point and are invariant to local rotation and stretch. Offsets are vectors that must transform with $J_\phi$. The features-versus-offsets ablation, run from the identical frozen network, settles which property matters.

Why not just upweight LNCC near edges? Upweighting a term whose gradient is near zero gives a scaled near-zero gradient. We test this anyway: edge-weighted LNCC with the same band $B$ is included, and we expect it to help only at high-contrast edges. The contrast-binned analysis will show whether that holds.

Why not just use dilated or larger rigid kernels? A larger rigid receptive field gives more context but no per-location geometry to match. The rigid-encoder variant of the feature-matching ablation answers this. Reported segmentation evidence (DB-MSMUNet) favors deformable kernels, but that evidence does not transfer automatically to registration.

Proposed Experiments

The primary benchmarks are Learn2Reg abdominal MR-to-CT and the 136-pair pelvic MRI set, both rich in low-contrast organ interfaces. Offset pretraining uses MSD Task07 pancreas and NIH pancreas CT, kept disjoint from registration test subjects. CamCAN T1w/T2w and CMRxRecon serve as contrast-shift sanity checks.

Metrics are band Dice and HD95 within 3 mm of reference boundaries, whole-organ Dice, landmark TRE on the pelvic set, the folding ratio $J<0$, and runtime per pair. All methods get the same GPU budget and validation-tuning budget.

Three checks are decisive. First, gain over the label-loss baseline should concentrate in low-contrast, near-boundary bins and vanish at high-contrast ones. If it is uniform, the mechanism is not what we claim. Second, on the pelvic set, landmark-predicted local Jacobians should agree better with $\mathcal{L}_{\text{off}}$-trained warps than with baseline warps. Third, offset drift under random-convolution re-coloring must be small with the consistency term and large without it, and registration accuracy should track that drift. If the third check fails, the tension in Gap 4 is real and the contrast-robust version of the method is not viable as designed.