Letting Held-Out K-Space Decide How Much to Trust a Patient's Prior Scan

Patients with brain tumors or neurodegeneration are scanned again and again, and every follow-up MRI looks mostly like the one before. That makes the previous scan an obvious aid for speeding up the next one. An MRI scanner measures the image's frequency content (called k-space), and acquisition time grows with how much of it is measured. With a good guide, a reconstruction algorithm can fill in more of what was skipped. The danger is just as obvious. A reconstruction that leans on last year's scan can paint last year's anatomy over this year's new lesion, and a radiologist cannot tell. The risk is worse when the follow-up comes from a different scanner, because the prior then differs in contrast and intensity for reasons that have nothing to do with disease. No ground-truth image exists to catch these errors. In routine accelerated imaging, the fully sampled image is never acquired.

We propose to use the one honest signal that is available. Self-supervised measurement splitting is a way to train reconstruction networks without clean references. It divides the acquired k-space samples into a part the network sees and a part it is scored on. We split the follow-up scan's own measurements three ways and use a held-out portion as a referee. The referee decides, region by region and without labels, how much the reconstruction may borrow from the patient's earlier 3D scan. The system is native 3D for a concrete reason: patients are repositioned between visits, so prior and follow-up only line up after full volumetric registration, and slice-by-slice borrowing would match the wrong anatomy. The claim we test is that this held-out referee keeps the acceleration gains from using the prior while preventing it from erasing real change, including when the prior was acquired on a different scanner.

Core Claim

Measurement splitting has so far served to train reconstructors and to decide when to stop training. SSDU trains an unrolled physics network on one subset of k-space and scores it on the rest. ZS-SSL adds a third subset, which it uses to stop per-scan training before the network fits noise. Equivariant Splitting shows that, for reconstructors that respect the right symmetries, the splitting loss is an unbiased estimate of the supervised loss in expectation. That theory supports the motivating idea here: a held-out k-space residual is a usable proxy for reconstruction error. Our new step is to give that residual a different job. It becomes a per-scan arbiter between two sources of information: the current measurements and a registered prior from another visit and possibly another scanner.

The method has two parts. The follow-up's acquired sample set $\Omega$ is split into disjoint sets $\Theta$, $\Lambda$ and $\Gamma$. A 3D unrolled network $f_\theta$ receives $y_\Theta$, the measurements in $\Theta$, together with the registered prior volume $P$. Inside the network, a spatial gate $w \in [0,1]^{H \times W \times D}$ controls where $P$ may influence the regularizer. Training uses the standard splitting loss on $\Lambda$. At test time, the gate's strength is chosen from a small family $\{w_\alpha\}$, indexed by a global scale $\alpha$ and by sub-regions defined in coil-weighted hybrid space. The rule is to pick the member that minimizes the residual on $\Gamma$:

$$\hat{\alpha} = \arg\min_{\alpha} \| A_\Gamma f_\theta(y_\Theta, P; w_\alpha) - y_\Gamma \|_2^2 .$$

Here $A_\Gamma$ is the forward operator restricted to $\Gamma$, meaning coil sensitivities, Fourier transform and sampling. Because this is a selection among a handful of settings, the deployed network weights stay frozen. That matters for the regulatory concern raised in INPUTADAPTER, which adapts inputs rather than weights so that a certified predictor stays locked. Per-scan training in ZS-SSL does not have this property.

The method also yields an audit product. We reconstruct the scan with and without the prior and flag regions where the two disagree and the $\Gamma$ residual favors the no-prior version. These are candidate sites of real change that the prior would have hidden.

There is a limitation we state up front. $\Gamma$ can only expose errors that leave a footprint in sampled frequencies; errors confined to the unsampled null space are invisible to it. Our bet is that an erased or shrunken focal lesion has broad-band spectral content and therefore does leave a footprint. Whether that holds as a function of lesion size and acceleration is the central empirical question.

Research Program

Reconstructor. The first component is a 3D physics-guided unrolled reconstructor for Cartesian 3D acquisitions, with two phase-encode directions undersampled and the readout direction fully sampled. Only part of the work is unrolled in full 3D: we reconstruct in hybrid space (inverse transform along the readout) and use 3D convolutions with anisotropy-aware kernels. This keeps memory manageable on a single GPU and avoids the compute problem that pushed ZS-SSL and AeSPa to slice-wise processing at minutes per slice.

Prior pathway. The prior is registered with rigid and then affine 3D registration to a no-prior first-pass reconstruction. It is then passed through a contrast-agnostic encoder, so that the network borrows structure rather than raw intensities. This addresses the observation in K-space acquisition domain generalization that scanner shift is largely an acquisition-physics phenomenon, which image-space noise does not mimic and in fact made worse. It also replaces the scalar grayscale correction of LACS-MRI. LACS-MRI showed that a longitudinal reference permits reconstruction at 6% sampling where wavelet compressed sensing fails. It handles only global linear intensity rescaling across visits, and it relies on hand-crafted sparsity.

Analysis of the referee. For which lesion sizes, contrasts and acceleration factors does the $\Gamma$ residual detect prior-induced erasure? How large must $\Gamma$ be? Every sample moved into $\Gamma$ is taken from the network's input, and Klug et al. show that self-supervised splitting already pays a sample-complexity tax.

Data. No dataset in the file offers longitudinal raw multi-coil k-space. We therefore work at two levels. Retrospective simulation forward-models longitudinal 3D images from Lumiere (glioma follow-up) and ADNI (structural brain MRI over years) with simulated coil sensitivities and smooth phase. We acknowledge that this misses real phase and noise statistics. Raw-k-space realism is checked on fastMRI brain and on the fastMRI-to-M4Raw scanner shift used by Wattad et al. On those raw datasets we pair subjects with synthetically deformed copies or with inserted lesions to create controlled "follow-ups."

Baselines

Naive references. The first is zero-filled reconstruction, and the second is the supervised oracle trained on fully sampled references. Between them they bracket the achievable range. For change-sensitive questions we also report the last-context-image heuristic: simply copying the registered prior. CRONOS shows this heuristic is hard to beat on image fidelity, yet it fails on real tumor change (7.55 vs 8.38 NRMSE on Lumiere for CRONOS vs persistence). It is the extreme of "trust the prior completely," and it gives the worst case for lesion erasure.

Strongest no-prior self-supervised method. This is Multi-Operator Equivariant Imaging from SSIBench. It combines physical splitting operators with virtual rotations and warps, and it reports 32.14 dB PSNR on 6× single-coil fastMRI brain against a 33.15 dB supervised oracle. We run it through DeepInverse with the same 3D backbone, the same data and the same tuning budget, measured in GPU-hours and hyperparameter trials. We also run a 3D SSDU without prior, with the split ratio near the $\rho=0.4$ reported for SSDU, as a simpler reference.

Slice-wise baseline. A 2D-SSDU-then-stack version isolates the cost of losing through-plane context, both in reconstruction and in prior alignment.

Classical longitudinal reference method. LACS-MRI with grayscale correction, run on identical masks.

Key ablation. The most important comparison isolates the referee itself. We use the identical prior-conditioned network with a single global prior weight, tuned once on a validation set with full references, which is generous to the baseline. A second variant uses a gate learned only through $\Lambda$, with no $\Gamma$ selection.

What counts as winning. Beating these baselines means matching or exceeding the fixed-weight prior model's global PSNR at 8–12× acceleration. At the same time, the method must reduce lesion-volume error on changed regions to the level of the no-prior MO-EI model or below. On the robustness side, it must not lose more than a small, pre-registered margin (0.5 dB) to MO-EI when the prior comes from a shifted scanner.

Why not just…?

Why not just add the registered prior as a fixed-weight regularizer to SSDU? This is the real threat, and it is exactly the key ablation. We expect a fixed weight to face an unavoidable trade-off. A weight strong enough to help at high acceleration in stable anatomy should erase small new lesions, and a weight weak enough to preserve them forfeits most of the gain. The experiment that settles this sweeps the weight and plots global PSNR against lesion-volume error. If some fixed weight dominates the $\Gamma$-selected gate on both axes, the referee is unnecessary, and we would report that.

Why not just skip the prior and use the best self-supervised reconstructor? At moderate acceleration, MO-EI may already preserve lesions well, which would make the prior pointless. LACS-MRI suggests the prior pays off at very low sampling rates, so we test 6× through 12×. We will report the crossover point at which priors start to help. If none appears, the proposal reduces to a negative result about longitudinal priors in deep self-supervised reconstruction.

Why not just harmonize the prior's intensities and keep a classical or fixed-weight method? Histogram matching or grayscale correction fixes global intensity scaling. It does not fix contrast-weighting changes, bias fields or the non-linear vendor differences behind the >20% mDice collapse on unseen MRI scanners reported in AMOS. We include harmonized-prior variants of both the fixed-weight and LACS baselines. If either of them closes the gap under scanner shift, then the contrast-agnostic prior encoder is the useful ingredient and the referee is not, and the ablation will show it.

Why not just fine-tune per scan as in ZS-SSL? Per-scan fine-tuning does protect against domain shift, and ZS-SSL-TL holds 37.1 dB under knee-to-brain transfer. It changes weights at deployment, however, and costs 85–640 seconds per slice, which becomes hours for a 3D volume. We include ZS-SSL-TL with prior input as an upper-effort comparator and report both wall-clock time and accuracy.

Proposed Experiments

Lesion-erasure test. On Lumiere and on fastMRI subjects with inserted lesions, we vary lesion size from a few voxels to centimeters and acceleration from 6× to 12×. For each method we measure whether the new lesion survives reconstruction: segmentation Dice against the fully sampled follow-up, and error in inter-visit lesion volume. In the same setting we measure how often the $\Gamma$-residual audit map flags erased lesions and how often it raises false alarms in stable tissue. This test is decisive. If $\Gamma$ cannot detect erasure of lesions of clinically relevant size, the core claim fails.

Pseudo-change test. Using ADNI subjects judged clinically stable, we feed priors whose contrast and bias field have been deliberately altered, or which come from a different visit protocol where one is available. We measure regional brain-volume change between visits. A method that leaks the prior's scanner signature will produce spurious atrophy or growth. This is the confound that LNE and the cross-encoder leave unaddressed when they treat inter-visit differences as biology.

Raw-k-space realism. We repeat the scanner-shift arm on fastMRI-to-M4Raw, where real low-field acquisition statistics replace simulation. We also report global PSNR and SSIM alongside the change metrics, because CRONOS notes that pixel fidelity and clinical utility can diverge.

Practicality. We report per-volume reconstruction time and memory for every method, and we vary the size of $\Gamma$ to expose its cost in input samples.