Measurement-Checked Peer Learning for Label-Free MRI Reconstruction

Accelerated MRI scanners record only a fraction of the frequency data (k-space) needed to form an image. The usual fix is a neural network that fills in the missing frequencies. Training such a network normally requires fully sampled "ground truth" scans, and those are often impossible to collect because patients move and signals decay. Self-supervised methods avoid this by holding out part of each scan's own measurements and training the network to predict them. What no measurement can teach is the content of frequencies that were never acquired, the nullspace of the scanner. There, a single network is free to invent structure.

We propose training two structurally different reconstruction networks, one convolutional and one attention-based, as peers. Each peer receives the other's guesses about the unmeasured frequencies, and each guess is weighted by how well that peer predicted real measurements it was never shown. The held-out measurements do the job that ground-truth labels do in existing peer-learning methods: they decide when to trust a peer. The proposal also targets the main practical objection to peer learning, which is that it needs several models resident in GPU memory at once. We show how to deliver the peer signal with the memory of a single model by caching stale peer predictions. If this works, practitioners get the gains of multi-model training without ground truth and without multi-model memory.

Core Claim

Networks with different built-in assumptions tend to invent different wrong content in the nullspace. Where they agree, and where the agreeing peer is demonstrably accurate on held-out real measurements, their consensus is a better training target than any single network's own output. Concretely, let $\Theta_k$ and $\Lambda_k$ be disjoint input and held-out subsets of the acquired k-space indices given to peer $k$, and let $\bar\Omega$ be the unacquired indices. Peer $k$, written $f_k$, minimizes

$$ \mathcal{L}_k = \big\|A_{\Lambda_k} f_k(y_{\Theta_k}) - y_{\Lambda_k}\big\|^2 + \lambda \sum_{j \ne k} w_j \,\big\|P_{\bar\Omega}\big(f_k(y_{\Theta_k}) - \mathrm{sg}[\tilde x_j]\big)\big\|^2 . $$

Here $A_{\Lambda_k}$ maps an image to the held-out frequencies, $P_{\bar\Omega}$ keeps only the unacquired frequencies, $\tilde x_j$ is peer $j$'s reconstruction (possibly cached from an earlier step), and $\mathrm{sg}$ blocks gradients into the peer. The weight $w_j$ is a per-scan, per-frequency-band score. It is computed from peer $j$'s error on its own held-out set $\Lambda_j$, times a term that shrinks when the two peers disagree. The first term is the standard measurement-splitting loss of SSDU. The second term is new. It is a mutual-learning term in the spirit of Deep Mutual Learning, restricted to where supervision is missing and gated by a label-free trust signal.

The file does not contain this combination. SiamRecon builds a two-branch self-supervised MRI method, but both branches share one set of weights, so there is no second model with different biases. It also needs an SSDU warm start to avoid collapse. FSMNet runs frequency and spatial branches that learn from each other, but it is trained against fully sampled targets. CTRCL pairs a CNN and a Transformer and corrects wrong peer predictions before they are exchanged, but that correction uses ground-truth annotations. Our contribution is to replace ground truth with held-out measurements as the gate. We also exchange information in k-space rather than aligning internal features. This avoids the CNN-versus-Transformer feature-matching problem that CTRCL addresses with a dedicated module.

Research Program

Two peers with different inductive biases. Both are unrolled networks, meaning an iterative reconstruction algorithm written out as a fixed number of network stages. Each stage alternates a learned denoiser with a step that enforces agreement with the measured data. Peer A uses a residual CNN denoiser, as in MoDL-style unrolling. Peer B uses a windowed-attention denoiser. The two peers receive different random splits $(\Theta_k, \Lambda_k)$ of each scan. This mirrors the per-student input distortions in KDCL, and it means each peer's held-out set can serve as an honest check on that peer.

The trust weight. KDCL-General weights students using a clean validation set, and CTRCL gates exchanges with labels. Our weight uses nothing beyond the scan itself. We will compare three versions. The first is uniform weighting. The second uses held-out error only. The third uses held-out error multiplied by a disagreement penalty. This comparison tells us whether measurement-based gating is the ingredient that prevents the peers from reinforcing each other's mistakes, the failure mode sometimes described as "the blind leading the blind."

Removing the memory cost. Online mutual learning keeps every model's weights, stored activations and optimizer state in GPU memory at the same time. We will test whether that is necessary for reconstruction. In the alternating-cache regime, only one peer trains at a time. The inactive peer's optimizer state moves to CPU memory. Its targets $\tilde x_j$ come from either a gradient-free half-precision forward pass or a cache that is refreshed every $T$ steps. Peak GPU memory then roughly equals one model in training plus a forward pass with nothing stored for backpropagation. We will sweep $T$ from 1 (the online case) upward to find the point at which stale targets start to hurt.

The CNN peer can additionally use memory-efficient invertible backpropagation (MEL), which trained 3D unrolled models on a 12GB GPU. MEL's requirement that every layer be invertible rules it out for the attention peer, so that peer uses gradient checkpointing instead. Because the two peers need different memory tools, the heterogeneous pairing is a real stress test of the memory claims.

Nullspace diagnostics. We will use ground truth only at evaluation time. The measurements are the error of each reconstruction restricted to $\bar\Omega$, and the correlation between peer disagreement and that error. If disagreement does not predict error, the core claim fails, whatever the headline PSNR shows.

Baselines

The naive reference point is a single unrolled network trained with the SSDU loss alone, at its reported best split ratio $\rho = 0.4$ (Yaman et al.). This is our first loss term with no peer term. No reported result was found on Lacuna for SSDU's absolute PSNR in the scenarios we use, so we will rerun it ourselves.

The strongest self-supervised competitor is Multi-Operator Equivariant Imaging (MO-EI) from SSIBench. MO-EI regularizes the nullspace by requiring consistent reconstructions under different sampling masks and geometric transforms. The benchmark reports 32.14 dB PSNR for MO-EI on 6× single-coil fastMRI brain, against a supervised oracle of 33.15 dB. MO-EI attacks the same nullspace problem we do, through symmetry instead of peers, so it is the method to beat. We will run it inside the same benchmark code with the same backbones.

A second strong competitor is SiamRecon, the closest two-branch design. It reports 36.68 dB on 4× multi-coil fastMRI knee, against 37.69 dB for supervised VS-Net. It also reports gains of more than 0.5 dB over SSDU on IXI brain.

Fairness is defined by total training compute and peak GPU memory, not by epoch count. The two-peer system is compared with a single model given twice the training steps, and also with a single model widened until its parameter count and FLOPs match both peers combined. Every method gets the same hyperparameter search budget. Beating the field means two things on SSIBench's fastMRI brain and SKM-TEA knee scenarios. The first is a PSNR and SSIM gain over compute-matched MO-EI that holds across seeds. The second is a gain of at least 0.3 dB when the peer term is added on top of MO-EI, which shows the two mechanisms complement each other.

The ablations that isolate our key ingredient are uniform weights versus measurement-gated weights, two identical CNN peers versus the heterogeneous pair, and online training versus the alternating cache.

Why not just…?

Why not just train one bigger self-supervised model with the same compute? This is the most serious threat. The peers see exactly the same measurements, so they cannot add information the data do not contain. Their only possible advantage is lower variance in how the nullspace gets filled. The compute- and parameter-matched single-model baselines described above settle this directly. If a widened MO-EI network matches the peer system, we will report that peer learning adds nothing in this setting.

Why not just ensemble independently trained models afterwards? Averaging outputs reduces variance at inference time. It costs K times the inference and never feeds agreement back into training. We will evaluate a post-hoc average of the two independently trained peers. Our claim requires that each individually deployed peer from our training beats that average, which it can only do if the training signal matters.

Why not just distil offline, training one model, freezing it, and teaching the second? Offline distillation already has single-model memory cost. It is, in effect, our cache with $T$ equal to the whole training run. The staleness sweep therefore contains this baseline. The DML results on CIFAR-100 (Zhang et al.) found peer training beating distillation from a larger pre-trained teacher. The sweep tests whether that holds in reconstruction.

Why not just use gradient checkpointing or MEL for memory? We do use both. They shrink the stored activations of one model, but not the duplicated weights and optimizer states of several models trained together. MEL also cannot handle attention layers. Our alternating cache targets the part of the memory cost that these tools leave untouched.

Proposed Experiments

All experiments run on SSIBench's scenarios: single-coil and noisy fastMRI brain, 32× SKM-TEA knee, and CMRxRecon cine. We add multi-coil fastMRI knee at 4× so we can compare against SiamRecon's reported numbers. The metrics are PSNR and SSIM, PSNR measured on the nullspace alone, peak GPU memory, and wall-clock time.

The first decisive check asks whether peer disagreement predicts nullspace error on held-out scans with ground truth. If it does not, we stop. The second asks whether the gated peer term improves over compute-matched MO-EI and SSDU, and whether it still improves when stacked on top of MO-EI. The third asks whether the alternating cache stays within 0.1 dB of online mutual learning while using roughly the memory of one model. The fourth asks whether heterogeneous peers beat identical-architecture peers, which tests whether diverse inductive biases are what drives the gain.

A 3D knee run on a single 12GB GPU, the setting MEL demonstrated for one supervised network, tests whether label-free two-peer training is practical at clinical dimensionality.