Reinforcement learning with verifiable rewards trains a model by letting it try many answers and rewarding only those an automatic checker accepts. It has transformed math and code reasoning in language models because those domains have cheap, exact checkers. Medical imaging seems to have nothing comparable, since the clean image a reconstruction should match is usually never acquired. Accelerated MRI does, however, contain a partial checker. The scanner's physics, the known Fourier sampling operator, can test any candidate reconstruction against raw measurements that the reconstructor was not allowed to see. This note proposes using that held-out physics check as the reward for adapting a generative image prior, a diffusion model trained on one anatomy or scanner, to a new target domain from a handful of unlabelled, undersampled scans. Afterwards, the adapted model reconstructs new target scans with a single sample and no per-scan optimization.
The opportunity is practical. Generative priors produce sharp reconstructions but can invent anatomy when they meet a domain they were not trained on. Methods that tune themselves on each scan, by contrast, need minutes of optimization per slice. A verifier built from physics can do two jobs: it filters the prior's inventions on the scans used for adaptation, and it lets the cost of adaptation be paid once, offline, for a whole target cohort. The open question this note is built to answer is whether such a verifier, which by construction checks only frequencies the scanner measured, rewards fidelity or rewards a more convincing kind of hallucination.
Core Claim
The claim is that group-relative policy optimization over posterior samples, scored by consistency on held-out k-space, can adapt a source-domain diffusion prior to a target domain using one to sixteen unlabelled target scans. Group-relative policy optimization (GRPO) draws several samples for the same input and pushes the model toward samples that score above the group average. K-space is the Fourier-domain raw data an MRI scanner records. The adapted prior should then match or beat per-scan test-time adaptation on an unseen target cohort at a small fraction of the inference cost, without increasing hallucination in unmeasured frequencies.
The mechanism is not "replace a differentiable loss with reinforcement learning." That version is a weak idea, and a reviewer would be right to reject it. The point is narrower. A diffusion sampler produces a distribution of reconstructions that differ mainly in the unmeasured part of k-space. Ranking a group of such samples by how well they predict withheld measurements is a direct test of which nullspace guesses the data supports. The "nullspace" here means the image content the measurements cannot constrain.
There is a second reason to adapt across scans rather than within one. When target scans use random sampling masks, the union of their measured coordinates covers far more of k-space than any single scan does. Amortized adaptation can therefore check frequencies that a per-scan method never sees. A per-scan method cannot exploit this cross-scan coverage.
Grounding in Prior Work
Three lines of work motivate the design. The first shows the verifier exists. In Self-Supervised Physics-Based Deep Learning MRI Reconstruction Without Fully-Sampled Data, the measured k-space indices are split into a set used for reconstruction and a disjoint set used only for the loss. They report a 40/60 split as a good trade-off. Zero-Shot Self-Supervised Learning for MRI Reconstruction adds a third validation split for early stopping and applies the idea scan by scan. Both use the held-out check as a regression loss on a deterministic unrolled network, a network that alternates learned denoising steps with physics-based data-consistency steps. Neither uses the check to choose among stochastic hypotheses.
The second line shows the regime matters. Reinforcement Learning for Reasoning in Large Language Models with One Training Example shows that GRPO on a single verified math problem lifts Qwen2.5-Math-1.5B on MATH500 from 36.0% to 73.6%. It keeps improving test accuracy after training reward saturates, while its outputs on the training problem degrade into gibberish. This note borrows the question of whether training past saturation on very few instances generalizes. It does not assume the answer transfers from language to images.
The third line constrains the design by naming the failure mode. CellDuality: Unlocking Biological Reasoning in LLMs with Self-Supervised RLVR builds a reward from forward–inverse self-consistency when no ground truth exists, and its authors concede that a model can learn a consistent but false cycle. The physics check here is stronger than learned self-consistency because the operator is fixed and not learned. It shares the same blind spot, though: it never sees the nullspace. Equivariant Splitting: Self-supervised learning from incomplete data addresses exactly this for gradient training. It uses an invariance of the image distribution under rotations or shifts to manufacture "virtual" operators that probe the nullspace. It reports 27.45 dB on DIV2K inpainting against 25.89 dB for equivariant imaging and 28.46 dB supervised. This note reuses that trick as an optional reward term rather than as a loss.
On adaptation, Test-Time Model Adaptation for Image Reconstruction Using Self-supervised Adaptive Layers freezes an unrolled network and tunes tiny linear blocks per test image. It uses a range-space loss together with a heuristic null-space loss anchored to the pre-trained output. CAT-SAM: Conditional Tuning for Few-Shot Adaptation of Segment Anything Model shows in segmentation that small isolated adapters beat full tuning and plain LoRA in one-shot transfer. These motivate restricting updates to low-rank adapters on the denoiser. The difference from this proposal is that AdaptNet adapts per image with a hand-designed nullspace term, whereas this proposal adapts once per domain and lets sample-level verification arbitrate the nullspace. Diffusion priors for MRI, such as Learning Fourier-Constrained Diffusion Bridges for MRI Reconstruction, supply the base model class but are trained with reference images.
Research Program
The source model is a diffusion prior trained on fully sampled fastMRI knee data fastMRI: An Open Dataset and Benchmarks for Accelerated MRI. At inference it is coupled to a standard data-consistency step, so that conditioning on a measured subset is built into sampling. The target domain is fastMRI brain, using FLAIR and T2. Fully sampled target images are hidden during adaptation and used only for evaluation.
Adaptation sees $K \in \{1, 4, 16\}$ target scans, retrospectively undersampled at $R = 4$, 6 and 8 with random masks. At each step, for a target scan with measured indices $\Omega$, a fresh random split $\Omega = \Theta \cup \Lambda$ is drawn. Here $\Theta$ is the subset the model may condition on and $\Lambda$ is the withheld subset. The model then draws $G$ reconstructions $\hat x_1, \dots, \hat x_G$ conditioned only on $y_\Theta$, the measurements in $\Theta$. Each sample is scored with
$$r_i = \mathbb{1}\left[\, \|A_\Lambda \hat x_i - y_\Lambda\|^2 \le \tau\,|\Lambda|\,\sigma^2 \right] - \lambda\,\frac{\|A_\Lambda \hat x_i - y_\Lambda\|^2}{\|y_\Lambda\|^2},$$
In this formula $A_\Lambda$ is the multi-coil Fourier sampling operator restricted to $\Lambda$, and $y_\Lambda$ is the measured data there. $\sigma^2$ is the receiver noise variance, estimated from noise-only k-space regions. $\tau$ sets a noise-consistency threshold, and $\lambda$ weights a continuous tie-breaker.
The binary term is the "verifiable" part. It asks whether the residual on withheld measurements is statistically indistinguishable from noise. This mirrors the exact-match reward of math RLVR while being grounded in a physical noise model. Advantages are computed relative to the group. Updates go to low-rank adapters on the denoiser, with a KL penalty, a divergence term that keeps the adapted sampler close to the source prior. The diffusion sampler is treated as a policy whose actions are denoising steps, so the reward lands at the end of a trajectory. The third, validation split from ZS-SSL is held fixed per scan and used only to decide when to stop or to detect over-training.
Three additions are tested as separate ingredients. First, a reward-split curriculum allocates more rollouts to scans whose group rewards have the highest variance, the analogue of the historical-variance selection in the one-shot RLVR paper. Second, an equivariant reward term scores each sample under a rotated virtual operator, following Equivariant Splitting, to put pressure on the nullspace. Third, a post-saturation phase continues training well after the binary reward reaches 1 on all adaptation scans, so that later behavior can be observed.
Baselines
The naive reference point is the source knee prior run on brain data with data consistency but no adaptation. This measures how large the domain gap actually is. A second naive point is classical compressed sensing with total variation regularization. The fastMRI paper reports that a U-Net improves normalized mean squared error by 40–50% relative to that baseline.
The strongest existing method is ZS-SSL with transfer learning from Zero-Shot Self-Supervised Learning for MRI Reconstruction. It initializes from a knee-trained unrolled network and optimizes per brain scan. The file reports 37.1 dB PSNR at $R = 4$ in this knee-to-brain setting, at 85 seconds per slice. It will be rerun on the identical held-out brain cohort, masks and accelerations, with the authors' splitting and stopping rule and the same GPU, and compared at matched per-slice wall-clock and at its own convergence. Beating it means equal or higher PSNR and SSIM on the target cohort at $R = 4$, at least 0.5 dB higher at $R = 8$, and inference cost at least an order of magnitude lower.
AdaptNet Test-Time Model Adaptation for Image Reconstruction Using Self-supervised Adaptive Layers is the strongest parameter-efficient per-image competitor. The file reports roughly 3 dB over MetaCS and DDSSL but no absolute fastMRI brain figure. It is run per test scan with its published 20 iterations and the same knee-pretrained backbone family.
An amortized SSDU baseline isolates whether the generative prior and the RL step matter at all. It fine-tunes the knee unrolled network on the same $K$ unlabelled brain scans with the SSDU split loss and is then applied to the cohort without per-scan updates. For diffusion posterior sampling on fastMRI, no reported result found on Lacuna, so it is run ourselves.
The key ablations hold the reward, the $K$ scans and the adapter fixed, and change one thing at a time. One replaces the group-relative policy gradient with direct gradient descent of the same reward through the last few sampler steps. One replaces the noise-calibrated binary reward with the continuous residual alone. One removes the equivariant term. All methods get the same target scans, and every method's hyperparameter search is capped at the same number of trial runs. Tuning decisions use only the validation split, never ground truth.
Why not just…?
Why not just use held-out consistency to pick the best of N samples at inference, with no training? This is the most serious threat, because it uses the same verifier with no learning. Best-of-N costs N full sampling runs per slice. It also cannot improve the prior's samples, only choose among them, so if the source prior rarely produces correct brain anatomy, selection cannot fix that. The experiments compare the adapted model at one sample against the unadapted prior at N = 4, 16 and 64 with selection by held-out consistency. If best-of-16 matches the adapted model, the honest conclusion is that the contribution is the verifier, not the adaptation, and the note will report it that way.
Why not just backpropagate the measurement-splitting loss, as SSDU does? For a deterministic unrolled network, that is clearly the right tool, and the amortized SSDU baseline is included precisely because it may win. For a diffusion sampler, backpropagating through dozens of denoising steps is memory-heavy and usually truncated, and a single-sample loss provides no contrast between competing nullspace guesses. The direct-gradient ablation uses the same reward and adapter, so it settles whether the policy-gradient form adds anything beyond the reward itself.
Why not just run ZS-SSL or AdaptNet on every scan? Both work, but they pay their cost on every scan. ZS-SSL takes 85 seconds per slice even with transfer, which means hours for a volume. Each also sees only one scan's measured set, so neither can exploit cross-scan k-space coverage. If per-scan methods still win clearly in PSNR at $R = 8$, amortization is buying speed at the price of fidelity, and the cost–quality curve in the experiments will show where that trade lies.
Why not just use cycle-consistency as CellDuality does? The MRI forward operator is not invertible, so a forward–inverse cycle can be closed by any image that agrees with the measured data. CellDuality's own authors flag this consistency-versus-correctness gap. Held-out splitting is strictly more demanding: the image must predict data it never saw. Even so, it shares the nullspace blind spot, which is why the nullspace diagnostics below exist.
Proposed Experiments and Decisive Checks
Evaluation uses a held-out fastMRI brain cohort with ground truth, scored by PSNR, SSIM and NMSE, plus per-slice inference time. Hallucination is measured two ways. First, error is decomposed into the part inside the measured set and the part in the never-measured complement, on the evaluation scans. Second, error is computed within radiologist-marked lesion boxes from fastMRI+: Clinical Pathology Annotations for Knee and Brain Fully Sampled Multi-Coil MRI Data. A method that raises global PSNR while worsening lesion-region error counts as failing.
The first decisive check concerns post-saturation behavior. After the binary reward saturates on the adaptation scans, the question is whether continued training keeps improving cohort PSNR and nullspace error, as in one-shot RLVR, or whether nullspace error rises while held-out reward stays perfect. The second outcome would be direct evidence of reward hacking in the unmeasured space. If that happens, the test becomes whether the equivariant term stops it.
The second check concerns coverage. It compares adaptation scans whose random masks are drawn independently against scans that share one fixed mask. If cross-scan coverage is the mechanism, gains should shrink sharply with the shared mask, and that difference should be largest at $R = 8$.
The third check is the scaling curve from $K = 1$ to 16 against best-of-N and amortized SSDU. This locates the regime, if any, where verified policy adaptation of a generative prior is worth its training cost.
A secondary transfer test repeats the pipeline on sparse-view CT from LIDC-IDRI, where the Radon operator has a much larger nullspace. No reported baseline number was found on Lacuna for that dataset, so all comparisons there are run in-house.
References
- Self-Supervised Physics-Based Deep Learning MRI Reconstruction Without Fully-Sampled Data
- Zero-Shot Self-Supervised Learning for MRI Reconstruction
- Reinforcement Learning for Reasoning in Large Language Models with One Training Example
- CellDuality: Unlocking Biological Reasoning in LLMs with Self-Supervised RLVR
- Equivariant Splitting: Self-supervised learning from incomplete data
- Test-Time Model Adaptation for Image Reconstruction Using Self-supervised Adaptive Layers
- CAT-SAM: Conditional Tuning for Few-Shot Adaptation of Segment Anything Model
- Learning Fourier-Constrained Diffusion Bridges for MRI Reconstruction
- fastMRI: An Open Dataset and Benchmarks for Accelerated MRI.
- fastMRI+: Clinical Pathology Annotations for Knee and Brain Fully Sampled Multi-Coil MRI Data