Lesion-Preserving Self-Supervised MRI Reconstruction from Radiologists' Bounding Boxes

Fast MRI scans skip most of the frequency measurements a scanner would normally collect. A neural network then fills in the missing data to produce an image. These networks are trained to make images look globally correct, and they tend to smooth away small, clinically decisive structures such as a meniscus tear. Fixing this normally takes two scarce resources: fully sampled reference scans to train the reconstruction, and dense expert outlines of the pathology. We propose to use neither. The reconstruction network is trained only on the undersampled scans themselves. In addition, the network is penalized whenever a lesion detector, trained on cheap rectangular boxes that radiologists have already drawn, cannot find the marked lesion in its output. Put in two sentences: self-supervised reconstruction already removes the need for clean reference scans, and coarse bounding boxes can supply the missing signal about where diagnostic detail must survive. We test whether that signal restores lesion visibility without teaching the network to invent lesions.

The need is well documented. When supervised reconstructions from the variational network (an unrolled, physics-guided reconstruction model) were fed to a YOLOv5 detector on fastMRI+ knee data, meniscus-tear sensitivity fell from 0.707 on fully sampled images to 0.429 at 4× and 0.366 at 8× acceleration. Detected and missed tears had nearly identical image-similarity scores (End-to-End AI-based MRI Reconstruction and Lesion Detection Pipeline). The standard metrics therefore cannot see the failure, and a training objective built only on those metrics cannot fix it.

Core Claim

A self-supervised reconstruction network, jointly trained with a detection loss computed from box-level annotations, will recover substantially more of the lost lesion sensitivity at 4× and 8× than the same network trained without the box signal. It should do so at roughly equal global image quality and without raising false detections on healthy slices. Crucially, the gain must be visible to an independent detector that never participated in training. That requirement separates "the image now shows the lesion" from "the network learned a private code that only its own detector reads."

The closest existing work differs on exactly the axes this proposal changes. Joint Calibrationless Reconstruction and Segmentation of Parallel MRI shares an encoder between reconstruction and brain-tissue segmentation and needs only 10% of segmentation labels, but it trains reconstruction with a mean-squared-error loss against fully sampled references, and its labels are dense masks. Transformer-Based Integrated Framework for Joint Reconstruction and Segmentation in Accelerated Knee MRI feeds segmentation back into reconstruction on SKM-TEA, but it is fully supervised on both tasks. Constrained Probabilistic Mask Learning (ProM) shows that task-driven sampling patterns differ from pixel-driven ones, but it steers the pattern with dense segmentations. Zhao et al. only evaluate the chain and never backpropagate from detection into reconstruction. None of these combines reference-free reconstruction training with coarse lesion labels. None targets focal pathology rather than tissue segmentation. None tests whether task feedback produces hallucinated findings.

Research Program

The reconstruction model is an unrolled network: a fixed number of stages that alternate a learned image-cleaning block with a data-consistency step. The data-consistency step overwrites the frequencies the scanner actually measured, so the network can only fill in the unmeasured ones. We reuse the variational-network design of End-to-End Variational Networks for Accelerated MRI Reconstruction, including its coil-sensitivity estimation.

For training without clean references we borrow the method of SSDU. SSDU splits the acquired measurements into a subset the network sees and a held-out subset on which the loss is computed in frequency space. We use its reported split of 40% held out. We will also run the multi-operator equivariant imaging loss (MO-EI), which the benchmark in Benchmarking Self-Supervised Methods for Accelerated MRI Reconstruction found strongest, as an alternative self-supervised term.

A detection head is attached to the reconstructed image and trained on fastMRI+ boxes (fastMRI+). Its loss is weighted by $\lambda$ and backpropagated through all unrolled stages. The total objective is

$$ \mathcal{L} = \mathcal{L}_{\text{self}}(y_\Lambda, \hat{y}_\Lambda) + \lambda \, \mathcal{L}_{\text{det}}(D(\hat{x}), B) + \mu \, \mathcal{L}_{\text{det}}(D(\hat{x}_h), \varnothing), $$

The symbols are as follows. $y_\Lambda$ are the held-out measured frequencies and $\hat{y}_\Lambda$ their predictions. $\hat{x}$ is the reconstruction of an annotated slice with box set $B$, and $D$ is the detection head. The last term applies the detection loss with an empty target $\varnothing$ to reconstructions $\hat{x}_h$ of slices the radiologists marked as lesion-free.

That last term is the ingredient we consider essential, and it gets its own ablation. It exists because the detection gradient can be satisfied in two ways. The network can preserve real lesion signal, or it can paint lesion-like texture into the unmeasured frequencies. Data consistency does not rule out the second option, because at 8× most high-frequency content is unmeasured.

The annotation budget becomes an explicit variable. We train with box labels from 10%, 25%, and 100% of the labeled knee studies. For comparison, we also train with the study-level labels alone, using a weakly supervised multiple-instance loss in place of the box loss. This locates where on the cost curve the reconstruction benefit appears.

Baselines

The naive reference point is zero-filled reconstruction (inverse Fourier transform of the undersampled data) followed by a detector trained on boxes. It sets the floor of what is lost when nothing fills in the unmeasured frequencies.

The first real baseline is the same variational network trained with SSDU alone, followed by a detector trained on its outputs with the same boxes. This is the two-stage cascade any reviewer would propose first. It uses the same unlabeled k-space, the same box budget, the same detector architecture, and the same hyperparameter search budget. We will give it 20 configurations and match that for our method.

The strongest existing reconstruction baseline is MO-EI. The file reports it at 32.14 dB PSNR against a 33.15 dB supervised oracle in a 6× single-coil brain setting (Benchmarking Self-Supervised Methods). No reported lesion-detection result for MO-EI or SSDU was found on Lacuna, so we will measure both ourselves.

The strongest detection-oriented reference is the fully supervised variational network. Zhao et al. report meniscus-tear sensitivity of 0.429 at 4× and 0.366 at 8×, against 0.707 on fully sampled images (End-to-End AI-based MRI Reconstruction and Lesion Detection Pipeline). This baseline uses fully sampled training data that our method does not. Beating it would therefore be a strong result, while matching it would already show that references can be replaced by boxes. We will re-run it in our pipeline rather than rely only on the quoted numbers, because detector differences would confound a direct comparison.

Two ablations isolate the key ingredients. The first removes the healthy-slice term ($\mu = 0$), testing whether the negative constraint is what prevents invented findings. The second replaces true boxes with randomly placed boxes of matching size and count, testing whether any extra image-domain loss would help regardless of where the lesions are.

Our method counts as beating the cascade if, at 8×, the independent judge detector's sensitivity improves by a margin larger than the patient-level bootstrap confidence interval. This must hold at a false-positive rate on healthy slices no higher than the cascade's, and with SSIM within 0.01 of SSDU.

Why not just…?

Why not just fine-tune the detector on the reconstructed images? This is the most serious threat. It is likely to recover much of the automated sensitivity, since information about a tear may survive in the reconstruction even when it is not visually obvious. We do not argue it away. We run it as the cascade baseline and report it honestly. The distinction lies in who reads the image. In clinical use, a radiologist reads the reconstruction, not our detector. The proposal's claim is therefore judged by a separate detector trained only on fully sampled images from disjoint patients, a stand-in for a reader who has never seen accelerated artifacts. This setup mirrors ProM's zero-shot evaluation with frozen downstream models (ProM). The outcome can go either way. A fine-tuned detector may recover sensitivity on SSDU images while the judge detector does not. In that case, the lesion information is present but not visible, and only reconstruction-side feedback helps human readers. If instead the judge detector also recovers on plain SSDU images, the smoothing problem is less severe than Zhao et al. suggest, and our method adds little.

Why not just use a better self-supervised reconstruction? If lesion loss were simply a symptom of weak reconstruction, then MO-EI, which nearly reaches the supervised oracle in PSNR, would close the gap on its own. The evidence argues against this. Zhao et al. found that detected and missed tears are indistinguishable by SSIM, even under a supervised network. SKM-TEA shows that biomarker errors vary across subregions at similar global quality (SKM-TEA). We test this directly by comparing detection sensitivity across methods at matched PSNR. If MO-EI alone matches our sensitivity, the box signal is unnecessary.

Why not just learn a task-specific sampling pattern instead? ProM showed that sampling patterns chosen for segmentation preserve far more downstream accuracy than standard patterns at 32× (Dice 0.727 versus 0.399 for Gaussian sampling on cardiac data). This is a complementary lever rather than a rival. It still needs dense labels, and it changes what is acquired rather than how the unacquired part is filled. We keep the standard fastMRI equispaced patterns fixed so that gains are attributable to reconstruction. As a stretch experiment, we will run ProM-style pattern learning driven by our box loss.

Why not just use promptable or gaze-guided segmentation at test time? Methods like GazeRefine need the lesion to be visible in the image to anchor on. They cannot restore detail that the reconstruction has already smoothed away. They operate downstream of the problem this proposal addresses.

Proposed Experiments

The primary experiment uses fastMRI multi-coil knee data (fastMRI) with fastMRI+ meniscus-tear boxes, at 4× and 8× with the standard equispaced masks. Reconstruction training sees only the retrospectively undersampled k-space. Fully sampled images are used only to train the judge detector, on a patient-disjoint split, and to compute image-quality metrics on the test set. Metrics are judge-detector sensitivity and false positives per healthy slice, reported as a sensitivity-versus-false-positive curve, alongside SSIM, PSNR, and normalized error. All are reported with patient-level bootstrap intervals.

The decisive checks follow directly from the design. The method must beat the SSDU cascade on judge-detector sensitivity without increasing false positives on healthy slices. The $\mu = 0$ ablation should show increased false positives, which would confirm that the hallucination risk is real and that the negative term controls it. The random-box ablation should show no gain, confirming that the effect comes from lesion location. A lesion-insertion probe provides a further check. We compare judge-detector responses on reconstructions of healthy slices against those same slices fully sampled, and any systematic new detections count as failure regardless of the sensitivity gains.

The secondary experiment uses SKM-TEA's 16 pathology box categories and its quantitative T2 maps. This tests whether lesion-driven feedback damages tissue biomarkers, measured against the reported unrolled-model patellar-cartilage T2 error of 0.563 ms at 6× (SKM-TEA).

A small blinded reading by two musculoskeletal radiologists, on 8× test slices from the SSDU cascade and from our method, would be the final arbiter of whether the judge detector is a faithful proxy for human visibility.