Teaching Slice-to-Volume Registration to Doubt Itself with the Errors It Actually Makes

Many imaging tasks require placing a flat 2D image inside a 3D volume. Examples include locating an ultrasound frame inside a pre-operative heart scan, or finding where a thin section cuts through a CT volume. Clinicians do this in their heads, and they fail in predictable ways: they misjudge which way a plane tilts in depth, they confuse mirror-image anatomy, and they mistake one slice for its nearly identical neighbour. Registration algorithms, the programs that estimate the pose of the 2D slice inside the 3D volume, fail in the same ways. Almost none of them report when they have failed. A registration that has settled confidently in the wrong place looks, to the downstream display, exactly like a correct one.

This note proposes a separate verifier. It is a small network that looks at a 2D slice, the 3D volume, and a proposed pose, and predicts how wrong that pose is in degrees and millimetres. The verifier is trained on injected errors, but not on the random pose jitter that current pipelines use. We build its negatives from the registration solvers' own converged failures, together with a few structured operators that mimic the known ambiguities of mental reverse-projection: depth-sign flips, mirror reflections, and adjacent-slice shifts. The central bet comes from language-model work showing that models trained on artificial mistakes miss their own natural ones. We expect that a verifier trained on random perturbations will look excellent on random perturbations and fail on the wrong answers solvers actually produce, and that matching the injected errors to the solver's failures closes that gap.

Core Claim

Synthetic error injection is already used in 2D-to-3D mapping, but only to train pose regressors. SVoRT trains an iterative Transformer on fetal MRI slices with simulated rigid motion. CU-Reg samples echocardiography slices under random rigid transforms and adds a regularizer that penalizes confusing adjacent parallel slices. Needles & Haystacks builds 158,436 synthetic slice-to-volume problems across 22 non-medical and medical volume datasets. None of these trains a component that judges whether a finished registration is correct.

The closest verifier is MATTER. It regresses continuous alignment error for 3D-to-3D LiDAR point clouds and uses the prediction to trigger re-registration. It does not handle 2D slices and does not ask whether its synthetic noise matches real failures.

The paper that constrains the design is Synthetic Error Injection Fails to Elicit Self-Correction In Language Models. In its Sudoku setting, models reached 94% recognition of injected errors but only 8% recognition of their own errors. The authors attribute this to synthetic errors being context-independent while natural errors cluster where the task is hard.

Our claim transfers that diagnosis to geometry and tests a specific fix. Solver failures in slice-to-volume registration are not uniformly distributed in pose space. They concentrate in a small number of basins: symmetric flips, depth-sign ambiguity, and neighbouring-slice lock-on. A verifier whose training negatives are sampled from those basins will detect real failures, including failures from solvers it never saw. A verifier trained on uniform perturbations of matched magnitude will not.

Research Program

Harvesting failures. On the Needles & Haystacks training split we run three solvers from many initializations: SLSQP and COBYLA, which minimize an intensity dissimilarity score, and LoFTR-S2V, a learned feature matcher followed by GC-RANSAC and SLSQP refinement. We record every converged pose together with its true rotation and translation error. Converged poses beyond 20° or a fixed translation threshold become "hard negatives." Correct convergences become positives.

Structured operators. We add three operators motivated by the cognitive-mapping literature's account of how humans fail at reverse-projection: reflecting the pose about the volume's principal symmetry planes, inverting the sign of out-of-plane tilt, and shifting the slice along its normal by one to a few slice spacings. These cover failure basins that a given solver may visit rarely on the training split.

The verifier. The verifier resamples the volume at the proposed pose and compares the resampled image with the observed slice. It uses a shared image encoder, cross-attention between the two images, and a coarse 3D context crop around the plane. It outputs a predicted rotation error, a predicted translation error, and a probability that the pose is beyond threshold. It sees only images and the pose, never solver identity, so that it can be applied to solvers it was not trained on.

Refreshing the negatives. Once the verifier is used to rank or stop solver restarts, the search visits a new distribution of wrong answers. This is the geometric analogue of the on-policy shift that broke error-injection fine-tuning in language models. We therefore run two or three refresh rounds. In each round we harvest the failures that the verifier-guided search accepts, add them as negatives, and retrain.

Second setting. The same recipe is applied to the CAMUS echocardiography volumes using the released CU-Reg code. There the dominant failure is adjacent-slice confusion in low-signal images. The verifier's job is to flag frames whose predicted pose is unreliable, within a real-time budget.

Baselines

Naive reference. The do-nothing reference trusts every solver output. On Needles & Haystacks this gives the reported mAA@20° (mean average accuracy at a 20° threshold) of 54.62% for SLSQP with 2048 random restarts, 52.58% for LoFTR-S2V, and 52.72% for LoFTR-S2V with SLSQP refinement, with LoFTR-S2V at 0.297 s per task (Needles & Haystacks). On CAMUS, CU-Reg reports a distance error of 3.91 mm at 37 frames per second, against 5.84 mm for FVR-Net (CU-Reg).

For verifiers themselves, no reported result was found on Lacuna for failure detection on Needles & Haystacks or CAMUS. We will therefore report every verifier baseline ourselves under a shared protocol.

Solver's own score. The strongest existing competitor is the dissimilarity value the solver already computes at convergence: normalized cross-correlation (ZNCC) or mean absolute error for SLSQP, and inlier count for LoFTR-S2V with RANSAC. It costs nothing and is what practitioners use today. For downstream search, the matching baseline is choosing the restart with the best dissimilarity. That is exactly how the 54.62% figure is obtained.

Uniform-perturbation verifier. This baseline has the same architecture, data volumes, and training budget as ours. Its negatives are uniform random pose perturbations whose error-magnitude histogram is matched to the harvested failures. It is the MATTER and CU-Reg style of synthetic noise and is the ablation that isolates our key ingredient. A further ablation removes the structured operators, and another removes the refresh rounds.

Uncertainty baseline. We use the variance of an ensemble of five pose regressors as an uncertainty score. Prior work suggests this is weak: Are Registration Uncertainty and Error Monotonically Associated found Spearman rank correlation between Gaussian-process uncertainty and error of only 0.2899 on RESECT and 0.4014 on MIBS, in non-rigid neurosurgical ultrasound, which is a different task. We run it directly rather than rely on transfer from that result.

Fairness and success criteria. All learned verifiers get the same training tasks, the same number of gradient steps, and the same hyperparameter sweep of eight configurations. Downstream comparisons are matched on wall-clock time per task.

Beating the baselines has two parts. The first is higher AUROC (area under the receiver operating characteristic curve, a threshold-free measure of how well a score separates good poses from bad ones) and higher Spearman correlation than the solver-score and uniform-perturbation baselines on failures from held-out solvers. The second is higher mAA@20° than 54.62% at equal or lower runtime than the 2048-restart SLSQP search.

Why not just…?

Why not just use the solver's own dissimilarity score as the verifier? This is the most serious threat. If wrong basins usually have visibly worse ZNCC than correct ones, a learned verifier adds little. The Needles & Haystacks result argues against it, because selecting by dissimilarity over 2048 restarts still leaves 54.62% mAA. That means many wrong poses score at least as well as the right one, which is expected under geometric symmetry and repetitive texture. Still, this is an inference, not a measurement. Our first experiment measures the AUROC of the raw score directly. If it matches the verifier on held-out solvers, we will report that and the project reduces to a calibration study.

Why not just train on random perturbations, as existing pipelines already do? Random perturbations mostly produce large, obviously wrong poses. The language-model evidence (Synthetic Error Injection Fails) shows that high accuracy on such errors can coexist with near-chance detection of natural ones. We do not assume this transfers to geometry. The magnitude-matched uniform-perturbation ablation tests it head on. If it performs as well as solver-matched injection, our central claim is wrong.

Why not just use uncertainty from an ensemble or a Gaussian process? Uncertainty measures disagreement among models. It does not measure distance from the truth. Where all models share the same symmetry ambiguity, they will agree on the wrong answer. The weak correlations reported in Are Registration Uncertainty and Error Monotonically Associated are consistent with this, and the ensemble baseline above tests it on our tasks.

Why not just spend the compute on more restarts or an iterative refiner like SVoRT? More restarts help only if the selection rule can recognize the correct basin once it is found, which is exactly the problem the verifier addresses. SVoRT and IterCAD both show that iteration helps. IterCAD raised CAD reconstruction IoU from 80.71% to 91.61% by learning repair from injected code errors. But both assume the loop knows when to stop. We compare our method against restart counts matched on compute. The verifier is justified only if it wins at matched wall-clock time.

Proposed Experiments

Cross-solver transfer. This is the decisive check. We train each verifier variant on failures harvested from SLSQP and COBYLA, then evaluate on failures of LoFTR-S2V, and repeat with the roles swapped. We also hold out whole source datasets among the 22 so that anatomy and material cannot be memorized. We report AUROC for detecting poses beyond 20° and Spearman correlation with true rotation error, broken down by failure type (symmetry flip, depth-sign flip, adjacent-slice shift, other). The prediction is that the uniform-perturbation verifier scores well on held-out uniform perturbations and poorly on real solver failures, and that solver-matched injection narrows the gap.

Downstream search. We use the verifier to rank restarts and to stop search early, sweeping the restart budget. We plot mAA@20° against runtime next to the reported SLSQP (54.62%) and LoFTR-S2V (52.58%, 0.297 s) points.

Refresh rounds. We measure verifier accuracy on the failures that verifier-guided search itself accepts, before and after each refresh. This is where an analogue of the "parrot" collapse would appear, if it exists.

CAMUS. On the 400 test pairs, we measure how well the verifier flags CU-Reg frames with distance error above 5 mm, and how much CU-Reg's 3.91 mm mean error drops when flagged frames are rejected, at a fixed retention rate. The verifier must add latency compatible with 24 frames per second.

The project succeeds if solver-matched negatives beat magnitude-matched random ones on held-out solvers. It fails informatively if they do not, in which case the result would show that, unlike language reasoning, geometric registration errors are well covered by random injection.