Telling Bad Scans from Hard Scans in Label-Efficient Medical Segmentation

When a segmentation model trains mostly on its own guesses, which is semi-supervised learning with pseudo-labels, or asks an expert to label only the scans it finds most confusing, which is active learning, it treats every uncertain prediction the same way. An uncertain prediction can mean two very different things. The model may not yet have learned that anatomy, in which case an expert label is valuable. Or the scan itself may be too corrupted to support any confident boundary, for example from motion ghosting, noise, or blur from accelerated acquisition. In that case a pseudo-label will teach the model noise, and an expert label is costly and may itself be unreliable. We propose to separate these two cases using no-reference image quality assessment: a model that estimates how degraded an image is without access to a clean version of it. The quality estimator will produce a map of degradation for each pixel. It will be trained without any human quality ratings, using simulated scanner corruptions as supervision. Its output will decide which pseudo-labels to trust and which scans are worth a radiologist's time.

The opportunity is that existing label-efficient methods already sense that model confidence is not enough, but they repair it from the wrong side. They work on the predicted mask rather than on the image that produced it. If we are right, a fixed expert budget buys more accuracy when corrupted scans are dominant in the unlabeled pool. That is the common case in multi-site clinical data.

Core Claim

A blind, spatially resolved estimate of image quality carries information about pseudo-label error that the segmentation model's own disagreement does not. That information can be put to work through a simple rule. Pixels where the model is uncertain on a clean image are learning opportunities and should be queried. Pixels where the model is uncertain on a degraded image should be down-weighted and not queried. Concretely, for pixel $i$ of an unlabeled scan, let $v_i$ be model disagreement and $q_i \in [0,1]$ the predicted local quality, where 1 means undegraded. The pseudo-label loss is weighted by $q_i(1-v_i)$, and a scan $s$ is ranked for expert annotation by

$$A(s) = \frac{1}{|s|}\sum_{i \in s} v_i \, q_i ,$$

so that uncertainty only counts where the image could have supported a confident answer. Disagreement $v_i$ is measured the way Enhancing Pseudo Label Quality for Semi-supervised Domain-Generalized Medical Image Segmentation does it: the variance between predictions on an image and on a Fourier style-swapped copy. The testable prediction is a dose-response effect. As the share of corrupted scans in the unlabeled pool grows, plain uncertainty sampling should waste more and more of its budget on corrupted scans. The quality-weighted rule should degrade much more slowly.

Relation to Prior Work

Three papers motivate the gap by gating pseudo-labels without looking at the input image. Quality-aware semi-supervised learning for CMR segmentation rejects masks using a classifier over physiological volume curves. That classifier needs 1,000 clinician-graded cases and a full cardiac cycle, and it simply discards bad scans without saying why they failed. Our quality signal is image-level, needs no human quality labels, and works on static slices. CACPS, from the domain-generalization paper above, measures disagreement under synthetic style shifts. It cannot tell sensor noise from anatomical difficulty, and we reuse its disagreement signal as one of our two axes. DiffRect repairs pseudo-labels with latent diffusion entirely in mask space and never asks whether the image intensities support the repaired boundary. A Semi-Supervised Framework for Cardiac MRI Segmentation via Multi-Constraint Collaborative Self-Training fuses confidence, entropy and prototypes, which are again all model-side signals.

The method for the quality estimator comes from Blind Predicting Similar Quality Map for Image Quality Assessment (BPSQM). That work trains a U-Net to predict pixel-wise quality maps produced by full-reference metrics, which compare an image against a clean original. It is limited by those teacher metrics. Image Quality Assessment for Magnetic Resonance Imaging constrains our choice of teacher. Using 14,700 radiologist ratings on FastMRI, it found that PSNR and SSIM are unreliable for MRI while DISTS, HaarPSI and VSI align much better. We therefore build teacher maps from HaarPSI and VSI rather than SSIM. Generative "hallucinated reference" approaches such as No-Reference Image Quality Assessment Based on Multi-Task GAN are known to break down under severe corruption, so we avoid reconstruction and predict quality directly. MedQ-Bench also constrains the design: frontier multimodal language models trail clinicians by about 13.53% on medical quality perception and are weakest on subtle degradations. That argues against using them as the gate, though we keep one as a baseline.

Research Program

The first component is a medical blind quality estimator trained without human ratings. We take the labeled and unlabeled training scans of ACDC and M&Ms and corrupt them in k-space with physically motivated operations. These are rigid and periodic motion simulated through phase-line shifts, undersampling at several acceleration factors, Rician noise, and smooth intensity bias fields, each at graded severities and sometimes confined to a region. For each corrupted–original pair we compute local HaarPSI and VSI similarity maps as targets. We then train a fully convolutional network, following BPSQM, to predict the map from the corrupted image alone. The "original" scans are not pristine, so the estimator learns relative degradation. Whether this transfers to real artifacts is checked externally on the FastMRI radiologist ratings, which are never used for training.

The second component plugs the map into two standard label-efficient pipelines. In semi-supervised training we use cross pseudo-supervision with Fourier augmentation, as in CACPS, and multiply its per-pixel weight by $q_i$. In active learning we start from a small labeled seed, train, and rank the unlabeled scans by $A(s)$. We query a fixed number of scans per round, using held-out ground truth as the simulated expert, and retrain.

The third component is analysis. Before any training gains are claimed, we measure on held-out labeled data whether $q_i$ predicts pseudo-label error beyond what $v_i$ predicts. This is the cheapest test of whether the idea has any substance.

Baselines

The naive reference is the same segmentation backbone trained on the labeled seed alone. For active learning, the naive reference is random scan selection at equal budget. Both define the floor and show how much any method gains from unlabeled data or from choosing what to label.

The strongest semi-supervised competitor on multi-vendor data is CACPS from Enhancing Pseudo Label Quality for Semi-supervised Domain-Generalized Medical Image Segmentation. It reports 84.52% Dice on M&Ms with 2% labels and 85.72% with 5%. It is the right competitor because it already filters pseudo-labels by a quality proxy, perturbation disagreement, on exactly the scanner-shift problem where acquisition quality varies. No code for it appears in the file. We will therefore reimplement it, report our reproduction alongside the published figures, and build our method on top of our reproduction, so the only difference is the $q_i$ factor. Beating it means a Dice gain over our reproduction that exceeds seed-to-seed variation across five seeds at both label fractions, with a larger gain on held-out vendors.

On ACDC the strongest reported numbers are DiffRect at 82.40% Dice with 1% labels and the multi-constraint self-training method at 0.730 Dice with 5% labels. We will run DiffRect from its public code on the same splits and the same corrupted unlabeled pools. We also test whether adding quality weighting to DiffRect's input pseudo-labels helps it. That checks whether our signal is complementary to label-space correction rather than a substitute for it.

For active learning the strongest simple competitor is uncertainty sampling ranked by mean $v_i$ alone. The key ablations isolate the ingredient. First, replace the spatial map with a single scalar quality score per scan, which tests whether the spatial resolution matters. Second, replace the learned estimator with classical global SNR and contrast heuristics. Third, replace $q_i$ with a shuffled map that keeps the marginal distribution, which tests for a pure regularization effect. All methods share backbone, augmentation, training schedule and a matched hyperparameter search of equal size. No reported result found on Lacuna for quality-aware or uncertainty-based active learning on ACDC or M&Ms, so these comparisons are internal.

Why not just…?

Why not just rely on the model's own disagreement, as CACPS does? This is the most serious threat. Corrupted scans may already produce high disagreement under style swaps, making $q_i$ redundant. The partial-correlation analysis settles it directly. If $q_i$ explains no variance in pseudo-label error once $v_i$ is controlled for, the weighting idea is dead, and we would report that. The case for a residual signal is that disagreement under global amplitude swaps does not exercise local motion or aliasing artifacts. The MRI quality study shows those are the artifacts that coarse measures miss.

Why not just train the segmenter with the same simulated corruptions as augmentation and skip gating? This is also a real threat. If the segmenter becomes robust to the corruptions, its pseudo-labels on degraded scans may simply be fine. We will run an augmentation-matched baseline that uses exactly the corruption library used to train the estimator. The proposal survives only if gating adds to augmentation, especially on real rather than simulated artifacts. Evidence from SpurCon that standard augmentation leaves shortcuts on acquisition artifacts intact (worst-group accuracy of 0% on ISIC 2020 for the baseline) is suggestive but not decisive for segmentation.

Why not just drop low-SNR scans with a scalar threshold? The MRI quality study found that simple pixel statistics misjudge clinical quality. A per-scan threshold also throws away the clean regions of partly corrupted scans. The scalar-quality and heuristic-SNR ablations test both points. If they match the spatial map, the simpler tool wins and we would say so.

Why not just ask a multimodal language model whether each scan is diagnostically usable? MedQ-Bench reports 68.97% perception accuracy for GPT-5 and worse for open models, with the largest failures on mild degradations. That is the regime that matters for borderline pseudo-labels. Such a model also cannot give pixel-level weights. We include an open model's scan-level quality judgment as one ranking baseline on a subset, at matched budget.

Proposed Experiments and Decisive Checks

The quality estimator is validated first. We report rank correlation (Spearman and Kendall) between its pooled scores and the FastMRI radiologist ratings for noise, contrast and artifacts. We compare it against the full-reference DISTS, HaarPSI and VSI scores that the MRI quality study found strongest. The file gives no numeric correlation for these, so no reported result found on Lacuna for an NR-IQA correlation on FastMRI. The bar is that a no-reference estimator approaches its full-reference teachers.

The main experiment runs semi-supervised training on M&Ms at 2% and 5% labels and on ACDC at 1% and 5% labels. We use the natural unlabeled pools, then pools in which 0%, 20%, 40% and 60% of scans carry injected k-space motion or undersampling at clinically plausible severity. We report Dice per structure and per held-out vendor. The active learning experiment uses five query rounds at equal scan budgets and plots Dice against labeled scans for each acquisition rule at each corruption prevalence.

Three outcomes would be decisive. The first is the residual-information test described above. The second is a growing gap between $A(s)$ and pure uncertainty sampling as corruption prevalence rises, together with an audit showing that uncertainty sampling spends a rising fraction of queries on corrupted scans. The third is that gains persist on M&Ms vendors with naturally lower quality, where no corruption was injected. That would show the estimator responds to real artifacts and not only to our simulator.

A negative result on the first test, or gains that appear only on simulated corruption, would tell us that blind quality assessment does not add to model-side uncertainty in this setting. That answer is itself useful to anyone designing annotation budgets for multi-site data.