Does Accelerated MRI Fake Agreement? Measuring and Preserving Reader Disagreement Across Hospitals

MRI scans are slow because the scanner collects the image's frequency content one line at a time. To go faster, scanners skip lines. Skipping lines produces aliasing: overlapping "ghost" copies of anatomy folded across the image. Neural reconstruction networks are trained to remove these ghosts. Today they are judged by how closely their pixels match a fully sampled image, or by whether a single annotator's lesion is still detectable. This proposal asks a different question that matters clinically. When several radiologists read the same exam, they often disagree, and that disagreement is information: it marks the genuinely borderline cases. Does accelerated reconstruction keep that pattern intact? Or does it quietly change it?

Our hypothesis is that it changes it in two opposite ways. Coherent residual aliasing from regularly spaced undersampling creates spurious disagreement, because ghosts look like structures. Learned de-aliasing can create pseudo-consensus. By smoothing away ambiguous texture, it makes radiologists agree more confidently on borderline cases, sometimes on the wrong answer. We further expect pseudo-consensus to grow when a network trained at one hospital is deployed at another. Neither effect shows up in pixel similarity (the structural similarity index, SSIM) or in single-label detection accuracy. The work builds a multi-reader, two-institution knee MRI study that measures these effects directly. It then trains a surrogate panel of per-reader models so the effect can be audited without new reader studies. Finally, it tests whether a reconstruction loss that preserves the fully sampled disagreement pattern, rather than minimizing disagreement, transfers better across sites.

Core Claim

The central claim has three parts. First, the right yardstick for an accelerated reconstruction is the distribution of a reader panel's calls on each case, not one label. Second, the characteristic failure mode is a collapse of that distribution on borderline cases. Third, this collapse depends on how the scan was undersampled and which network reconstructed it, and it worsens under cross-institution transfer.

The connected literature motivates this from both sides without testing it.

On the reconstruction side, the 2020 fastMRI Challenge found two things. Networks trained on Siemens data degraded on GE and Philips scans. Reconstructions could hallucinate vessels or erase small lesions. Yet its six neuroradiologists' grades were simply averaged to rank teams, so the paper never looked at how they disagreed. fastMRI+ added pathology boxes and documented that SSIM rewards blurring away small tears. However, it used one radiologist per anatomy at one institution, so it cannot see disagreement at all.

On the subjectivity side, three pathology papers show that disagreement carries signal:

- The Latent Doctor Model learns reader-specific styles and reproduces panel distributions.

  • The virtual expert panel trains one network per pathologist and improves calibration over consensus training.
  • DiFF weights multi-rater masks by downstream diagnostic value.

All three operate on fixed, already-formed images. None involves an acquisition step that could itself manufacture or destroy ambiguity. We bring their machinery into the image-formation loop.

One design choice sets this work apart from the obvious combination. The file's "Gap 3" suggests training reconstructions to minimize the spread of reader disagreement. That objective would directly reward pseudo-consensus, which is the failure we are trying to expose. We instead ask reconstructions to match the panel distribution obtained on the fully sampled image.

Research Program

Measurement. The first component is a controlled multi-reader study. We use knee MRI because it has raw frequency-domain data (called k-space) with pathology labels at two different institutions.

  • Source site. The fastMRI multi-coil knee data with fastMRI+ boxes come from one institution and serve as the training site.
  • Target site. SKM-TEA provides raw k-space from 155 patients with 16 pathology categories, acquired at a different institution with a different protocol. It serves as the deployment site.

We are explicit that the shift from source to target bundles site, hardware, and sequence contrast. That bundle is what real deployment looks like. A condition fine-tuned on the target site, described below, separates "transfer" from "acceleration."

Case selection and reads work as follows:

  • Cases. About 150 source and 100 target exams, enriched for meniscal tears and cartilage lesions, which fastMRI+ highlights as easily blurred.
  • Readers. Six musculoskeletal radiologists read each case.
  • Response format. For each finding, readers give a five-point confidence rating. This yields a per-case panel distribution.
  • Experimental factors. Two acceleration factors (4x, 8x), two mask families, and four reconstructors.
  • Mask families. Equispaced masks produce coherent ghosts. Random variable-density masks produce noise-like aliasing. Contrasting them isolates the aliasing geometry.
  • Reconstructors.
  • Zero-filled inverse transform.
  • Compressed sensing with wavelet sparsity, i.e., a classical reconstruction that assumes the image is simple in a wavelet basis.
  • An unrolled variational network of the E2E-VarNet type, trained on source data only.
  • The same network fine-tuned on target-site data.
  • Reading schedule. An incomplete block design ensures each reader sees a given case in at most three conditions, separated by washout periods, which keeps reading load feasible.
  • Reference. The fully sampled image read by the same panel.

The pseudo-consensus quantity. For each case $c$ and condition $r$, let $p_c^{\text{full}}$ and $p_c^{r}$ be the empirical panel distributions over confidence levels. We record two quantities:

  • the entropy change $\Delta H_c = H(p_c^{r}) - H(p_c^{\text{full}})$;
  • the divergence $\mathrm{KL}(p_c^{\text{full}} \,\|\, p_c^{r})$.

A case counts as pseudo-consensus when $\Delta H_c$ is negative and the panel's modal call moves away from the fully sampled modal call.

A limitation applies here. Without arthroscopic ground truth, the fully sampled panel is the reference. The claim is therefore about fidelity to what experts see at full sampling, not about absolute diagnostic truth.

Surrogate panel. Reader studies cannot be repeated for every new network. We therefore train six per-reader networks on the fully sampled reads, in the style of the virtual expert panel. These are first pretrained as detectors on fastMRI+ boxes and then fine-tuned to each reader's ratings. We also train a single Latent Doctor Model as an alternative surrogate. The surrogate is useful only if its entropy shift across conditions tracks the human panel's shift. That is tested directly, not assumed.

Distribution-preserving reconstruction. If the surrogate is valid, we fine-tune the unrolled reconstructor $f_\theta$ on source data with the following objective:

$$ \mathcal{L}(\theta) = \lVert f_\theta(y) - x \rVert_1 + \lambda \sum_{k=1}^{6} \mathrm{KL}\big(g_k(x) \,\|\, g_k(f_\theta(y))\big) $$

The symbols are:

  • $y$ is the undersampled k-space;
  • $x$ is the fully sampled image;
  • $g_k$ is the frozen surrogate model of reader $k$, outputting a distribution over confidence levels;
  • $\lambda$ trades pixel fidelity against preservation of each reader's call.

Two design choices matter. The panel is frozen, so the reconstructor cannot win by moving the readers. The data-consistency step of the unrolled network still locks every acquired frequency, as in DuDoRNet. We then deploy the result on target-site k-space without further training.

Baselines

Naive references. Zero-filled reconstruction and wavelet compressed sensing are the do-nothing and classical references. They bound how much disagreement raw aliasing alone creates. They are run on identical masks, and the compressed-sensing regularization weight is tuned on source validation data only.

Pixel-loss unrolled network. The main competitor is the standard practice: an E2E-VarNet-style unrolled network trained with a pixel/SSIM loss. This is the class that won the fastMRI challenge, which reports SSIM around 0.964 at 4x on brain. On knee data at 6x, SKM-TEA reports about 35 dB peak signal-to-noise ratio and 0.84 SSIM for unrolled models.

Task-driven reconstruction. The strongest existing method that targets diagnosis is TACKLE. It trains sampler, reconstructor, and task head jointly on single consensus labels. It reports Dice 0.8532 at 16x on SKM-TEA knee segmentation, against 0.7843 for sequential reconstruction, and 91.36% classification accuracy at 64x. We run a TACKLE-style variant with the sampler fixed to our masks. This isolates the loss rather than the sampler. Its task head is trained on the fully sampled panel majority.

Ablations. The ablation that isolates our key ingredient is identical to our method except for the second term. It replaces the six-reader KL with cross-entropy to the panel majority. If it matches our method, the distributional ingredient adds nothing.

Fairness and the bar to clear. All learned models share architecture, source training data, and GPU budget, and each receives the same number of hyperparameter trials, selected on source validation. Beating the baselines means two things on target-site cases, judged by the human panel on a confirmatory subset:

  • a lower mean KL to the fully sampled panel distribution, with a significantly smaller pseudo-consensus rate;
  • no worse detection of the target-site pathology boxes.

No reported result was found on Lacuna for multi-reader agreement on accelerated MRI. The relevant baseline numbers for that quantity will therefore be produced by this study. The only pathology-domain reference points are those below:

Method Domain Reported result
Latent Doctor Model Pathology KL 0.461 on Gleason2019; entropy Spearman correlation up to 0.728 on ISUP
Virtual expert panel Pathology Expected calibration error 0.050 vs. 0.183 for consensus training

Why not just…?

Why not just measure detection accuracy against fastMRI+ boxes, as TACKLE does? Single-label accuracy can stay flat while the panel collapses onto a confident call. That is exactly the pseudo-consensus pattern. The study tests this directly. We ask whether the change in panel entropy predicts which cases' modal call flips, after controlling for the change in surrogate detection score. If it adds no predictive value, the multi-reader apparatus is unnecessary, and we will report that.

Why not just use perceptual or no-reference image metrics? The self-supervised validation study showed that perceptual distance tracks human quality ratings far better than SSIM. This is a real threat: if perceptual distance also predicts disagreement collapse, a cheap metric suffices. Our analysis regresses per-case KL on perceptual distance and SSIM alongside surrogate-panel shift. We report whether the image metrics alone explain it. A quality rating asks "does this look good?", whereas pseudo-consensus is the case where a smooth image looks better and is read more confidently. We therefore expect the two to diverge on borderline cases, but the experiment decides.

Why not just pool more multi-site data and retrain? The fastMRI challenge shows that roughly 7,000 training volumes did not prevent the cross-vendor drop. Universal Undersampled MRI Reconstruction found naive pooled training worse than anatomy-specific normalization (40.78 vs. 43.16 dB on 4x brain). Neither result says anything about reader disagreement, though. Our target-fine-tuned condition is the in-data upper bound for "just get target data." If it fully removes pseudo-consensus, the practical recommendation becomes site adaptation, and the preservation loss matters only where target labels are unavailable.

Why not rely on data consistency to prevent hallucination? Data consistency locks acquired frequencies but leaves every skipped frequency to the learned prior. The challenge's reported hallucinations occurred in such networks. The zero-filled-versus-unrolled contrast under identical masks shows how much of the panel shift the prior introduces.

Proposed Experiments and Decisive Checks

Test 1: does pseudo-consensus exist? On source-site cases, we compare the pseudo-consensus rate of pixel-loss unrolled reconstructions with that of zero-filled and compressed-sensing reconstructions at 8x. We use a mixed-effects model with reader and case as random effects. The hypothesis fails if learned reconstructions show no entropy drop relative to fully sampled reads on borderline cases. Borderline cases are those with fully sampled panel entropy in the top third.

Test 2: does aliasing geometry matter? We compare equispaced and random masks at equal acceleration. Coherent ghosts should raise spurious disagreement in classical reconstructions. Learned reconstructions should convert that excess disagreement into pseudo-consensus.

Test 3: does transfer amplify the effect? The source-trained network should show a larger pseudo-consensus rate on target cases than the target-fine-tuned network does on the same cases.

Test 4: is the surrogate valid? The surrogate panel must reproduce the human condition-wise entropy shifts. We use the 0.728 rank correlation reported for the Latent Doctor Model in pathology as an informal bar, since no MRI number exists. If the surrogate falls well short, the preservation loss is not trustworthy. In that case the project's contribution reduces to the measurement study and its released multi-reader k-space annotations.

Test 5: does the preservation loss help? On a confirmatory human read of target-site cases, the distribution-preserving reconstructor must beat both the pixel-loss and consensus-label variants on KL to the fully sampled panel without losing detection.