Classical image registration aligns two 3D scans by running an optimizer over a dense deformation field until it converges. It is slow, but it is also the approach that holds up when scans come from a new scanner or protocol. Deep networks that predict the deformation in one forward pass are fast, but they lose accuracy once the test data shifts away from the training data. Today, the schedule that governs an iterative solver is fixed in advance and tuned by hand. That schedule covers the step size, how strongly smoothness is enforced, when to move from coarse to fine resolution, and when to stop. Our proposal is to replace it with a tiny controller that watches the solver's own progress signals and adjusts these settings on the fly. The controller sees only scale-free numbers such as the relative drop in loss, gradient-norm ratios, and the fraction of folded voxels. It never sees image intensities. We therefore expect it to be far less sensitive to scanner shift than a network that reads the images.
The second half of the idea concerns cost. Deep reinforcement learning (RL) on 3D solvers is expensive: every training episode is a full volumetric optimization, and standard actor-critic methods carry extra networks in memory. Our controller is small enough, at a few hundred parameters, to be trained by black-box evolution strategies on cheap downsampled 2D and low-resolution 3D registration episodes. We then transfer it without retraining to full-resolution 3D solves. The concrete claim is that this controller reaches the accuracy of an expert-tuned classical solver in substantially less wall-clock time on out-of-distribution brain MRI and on lung and abdominal CT, with its own compute overhead included. It should also beat a static schedule tuned for the same budget.
What exists and what is missing
Two papers show that learned, state-aware control of solver parameters can pay off. A Game of Bundle Adjustment - Learning Efficient Convergence trains a Soft Actor-Critic agent to set the scalar Levenberg-Marquardt damping from the last five reprojection errors. It reports a fivefold reduction in iterations on KITTI and a "3x Wall-Clock Speedup" after counting network inference. A reinforcement learning approach to parameter selection for distributed optimal power flow uses a Deep Q-Network to pick ADMM penalty values from a window of residuals, with a "59.3% reduction" in iterations on IEEE 9-bus. These papers supply the method template and motivate the proposal. Both control one or a few global scalars in low-dimensional problems. The first explicitly warns that per-step policy cost can exceed the savings. The second reports that its policies fail to transfer to larger systems. We keep their framing of "solver as environment" and take on the two weaknesses they expose: overhead, and transfer across scale.
Within registration, the hyperparameter-sensitivity work is entirely open-loop. HyperMorph trains a hypernetwork so that one feedforward model covers a range of smoothness weights chosen by the user. The weight stays fixed during the pass, and the model inherits the feedforward network's behavior under shift. Multi-Objective Learning for Deformable Image Registration balances loss weights during training to spread a Pareto front. Embedding Gradient-Based Optimization in Image Registration Networks (GraDIRN) learns step sizes and regularizers offline through unrolled gradient descent, and those values are frozen at test time. Deep implicit optimization (DIO) puts a classical solver inside a network through implicit differentiation, but runs that solver with a hand-set schedule for hundreds of iterations. None of these adapts the solver's schedule to the specific pair being registered while it runs. DIO is the most natural host for our controller, because its remaining cost is exactly the inner solve.
Two findings constrain the design. Deep Learning in Medical Image Registration: Magic or Mirage? shows that properly tuned classical solvers such as FireANTs and Greedy beat OASIS-trained deep models on LPBA40, IBSR18, CUMC12 and MGH10. It also shows that much of the apparent deep-learning advantage came from poorly tuned classical baselines. Separately, Learn2Reg reports that the GPU-based iterative method ConvexAdam won the overall ranking. Together these set the bar: the competitor is an expert-tuned GPU solver, not a default one. They also set the rule that the controller must never read image content.
Core Claim
The claim has three parts. A telemetry-only controller for a multi-scale GPU registration solver can decide five things each iteration: step-size scaling, regularization weight, resolution transitions, smoothing strength, and termination. Doing so, it reaches the accuracy of an expert-tuned static schedule in less wall-clock time. Its advantage over a static schedule tuned on the training distribution grows under domain shift, because it reacts to how this particular pair is converging. And it can be trained entirely on cheap surrogate episodes without an RL critic or reward model.
Research Program
The base solver will be a FireANTs-style multi-scale, Adam-driven dense deformation optimizer on GPU, with an NCC similarity and a diffusion or Gaussian-smoothing regularizer. The controller acts once per iteration. Its input is a short window of normalized statistics: the relative loss decrease, the ratio of similarity to regularizer gradient norms, the change in field magnitude, the fraction of voxels with negative Jacobian determinant, the current scale level, and the fraction of budget used. These are computed from tensors the solver already has, so they add negligible cost. The controller's outputs are multiplicative adjustments to the step size and regularization weight, plus discrete "advance scale" and "stop" decisions. The policy is a two-layer network with a few hundred parameters. Its forward pass costs microseconds, compared with milliseconds for a 3D solver iteration. This addresses the overhead Belder et al. identified by construction, and we will measure it.
Training optimizes episode-level return:
$$J(\pi) = \mathbb{E}_{(I_F, I_M)}\big[\,Q(\phi_T) - \alpha\, t_T\,\big],$$
where $\phi_T$ is the deformation at the step $T$ where the controller stops, $Q$ is a quality score, $t_T$ is elapsed wall-clock time, and $\alpha$ sets the price of a second. The primary quality score is training-label Dice minus a folding penalty. An ablation replaces it with a label-free score built from the similarity and folding fraction. Sweeping $\alpha$ traces a time-accuracy curve, not a single operating point.
The primary trainer is CMA-ES on the policy weights. The policy is tiny and the reward arrives only at the end of an episode, so evolution strategies avoid both the temporal credit-assignment problem and the extra critic and reference networks that make actor-critic RL memory-hungry. SAC, as in Belder et al., is run as a comparison trainer. Training episodes are 2D slice pairs and quarter-resolution 3D OASIS pairs, which cost roughly one to two orders of magnitude less per episode than full-resolution solves. The telemetry is scale-free, so the policy should transfer to full resolution unchanged. We test that directly. The analysis also asks what the controller learned. We will record where it spends iterations, when it relaxes regularization, and when it stops, and compare these behaviors across in-distribution and shifted data.
Baselines
The naive reference is the solver with its default published schedule and a fixed iteration count. It anchors how much any tuning buys.
The strongest competitor is an expert-tuned static schedule. We will run Bayesian optimization over the full set of static settings: per-scale iteration counts, step size, regularization weight and smoothing. It will use the same OASIS training pairs and the same number of full-resolution solver evaluations the controller consumes in training, counted in GPU-hours. The goal is to optimize the same objective $J$ at each $\alpha$. This is the baseline that Magic or Mirage argues the field usually skips, which makes it the relevant test. That paper reports classical tools at Dice "~0.80" on OASIS. On Lacuna we found no reported runtime for tuned FireANTs or Greedy on OASIS, so we will measure one ourselves. We also give the static baseline an unfair advantage on the shifted cohorts: a version re-tuned directly on each OOD dataset. This is an oracle upper bound for any fixed schedule. Beating the OASIS-tuned static schedule means matching its Dice within 0.005 at a clearly lower median wall-clock time, or reaching higher Dice at equal time, on OOD cohorts. Getting close to the per-dataset oracle would be a strong result.
ConvexAdam, the overall Learn2Reg winner, is the external iterative reference on the Learn2Reg lung and abdomen tasks. For it, Learn2Reg reports only best overall rank and runtimes "in the seconds", and we found no per-task number on Lacuna. As fast learned references we include VoxelMorph, which reports Dice "0.750" at "under 1 second on a GPU" in Balakrishnan et al., and DIO, which reports Dice "0.853 ± 0.018" on OASIS. For DIO we also test the controller plugged into its inner solver.
Two ablations isolate the key ingredient. The first is an open-loop learned schedule: the same trainer and budget, but the policy sees only the iteration index and scale level, so it can learn a good fixed schedule but cannot react. The second is the hand-crafted reactive rule of Levenberg-Marquardt style: halve the step on loss increase, grow it on decrease, and stop on a relative-decrease threshold. If the open-loop learned schedule matches the full controller, then feedback adds nothing.
Why not just…?
Why not just tune a static schedule carefully? This is the real threat. Magic or Mirage shows how much careful tuning closes gaps. Our argument is that one static schedule averages over pairs that converge very differently, and that the mismatch widens under shift. That is an empirical claim. The comparison against the OASIS-tuned static schedule, and the per-dataset oracle, settles it. If the static schedule matches the controller on the OOD cohorts, we will report that iterative registration does not benefit from closed-loop control, and that the useful result is the tuning protocol itself.
Why not just use the solver's built-in adaptivity (Adam, line search, a stopping threshold)? Adam already adapts per-parameter step sizes. It does not decide when to switch scales, how to trade similarity against smoothness partway through, or when further iterations are no longer worth their cost. Belder et al. found that greedy, rule-based damping left a fivefold iteration gap on KITTI, and that a policy looking beyond the next step beat greedy learned policies by "50% fewer iterations". The hand-crafted reactive rule is in our baselines so this question gets a direct answer in registration rather than by analogy.
Why not just warm-start the solver from a feedforward network like VoxelMorph and run a few iterations? This could cut iterations as much as a controller does, in distribution. Under shift the warm start is exactly the component that degrades, according to Magic or Mirage, and the solver then needs an unknown amount of repair. That is precisely where a reactive stopping and scheduling rule should help. The two ideas compose, so we run warm-start with a static schedule, warm-start with the controller, and each alone, on in-distribution and OOD data. If warm-start with a static schedule dominates everywhere, the controller is not needed.
Why not just use deep RL as in prior DAC work? Actor-critic training on full 3D episodes carries the rollout and multi-network costs the file documents for RL. Our policy is small enough that CMA-ES can train it on cheap surrogate episodes. The SAC comparison trainer reports, for equal training GPU-hours, the time-accuracy curve each trainer reaches. That shows whether the heavier method earns its cost.
Proposed Experiments
Training uses OASIS pairs only, as 2D slices and quarter-resolution 3D volumes. In-distribution evaluation uses held-out full-resolution OASIS pairs with Dice and HD95. Shift evaluation, the main test, uses LPBA40, IBSR18, CUMC12 and MGH10, the cohorts where Magic or Mirage found classical solvers ahead of deep models. For cross-anatomy transfer we evaluate zero-shot on Learn2Reg Lung CT and Abdomen CT-CT and on NLST, using TRE on keypoints, Dice, and SDlogJ. Every method is reported as a time-to-accuracy curve. Wall-clock time includes the controller's overhead and is measured on one fixed GPU, alongside iteration counts and folding fraction.
Three decisive checks follow. First, on OOD brain cohorts, does the controller dominate the OASIS-tuned static schedule's time-accuracy curve? Second, does the open-loop learned-schedule ablation lose most of that advantage, which would show that feedback, not just learned tuning, is doing the work? Third, does a controller trained only on low-resolution surrogates perform within noise of one fine-tuned on a small number of full-resolution episodes? If so, the training-cost argument holds. A secondary check puts the controller inside DIO's inner solver and asks whether DIO's reported OASIS accuracy can be kept at a fraction of its iterations.
References
- A Game of Bundle Adjustment - Learning Efficient Convergence
- A reinforcement learning approach to parameter selection for distributed optimal power flow
- HyperMorph: Amortized Hyperparameter Learning for Image Registration
- Multi-Objective Learning for Deformable Image Registration
- Embedding Gradient-based Optimization in Image Registration Networks
- Deep implicit optimization enables robust learnable features for deformable image registration
- Deep Learning in Medical Image Registration: Magic or Mirage?
- Learn2Reg: Comprehensive Multi-Task Medical Image Registration Challenge, Dataset and Evaluation in the Era of Deep Learning
- An Unsupervised Learning Model for Deformable Medical Image Registration