Which Scales Did the Motion Touch? Separating Motion Blur from Rater Disagreement in MRI Segmentation

When several radiologists outline the same structure on an MRI scan, their contours differ. Recent segmentation models treat that spread as useful signal and learn to predict a distribution of plausible outlines rather than one answer. When a patient moves during a scan, the boundaries also become uncertain, but for a different reason. The anatomy is no longer faithfully recorded, so there is less information about where the edge is. Current models fold both effects into one "uncertainty" number. A clinician therefore cannot tell whether a wide confidence band means "experts would genuinely disagree here" or "this region was smeared by breathing and should be rescanned or read cautiously." The first calls for a judgment; the second calls for a better image.

We propose a segmentation model that keeps the two apart, using a physical fact about MRI. An MRI scanner records the image one frequency band at a time, sampling k-space, the image's spatial-frequency representation, over minutes. Motion during a given stretch of the acquisition corrupts the frequency bands recorded then, and frequency bands correspond to spatial scales. The model's multi-scale encoder is built on frequency bands of the image. A small module estimates how reliable each band is, and a second, separate latent variable captures each rater's habits. Boundary spread that comes from unreliable bands is attributed to motion; spread that remains when bands are reliable is attributed to raters. Motion can be simulated precisely on existing multi-rater datasets, so the attribution can be tested directly rather than merely asserted.

Core Claim

Our central claim is that motion uncertainty and rater uncertainty can be separated in a single forward-pass model, and checked against human data, under one condition: the multi-scale representation is aligned with the frequency bands that motion actually corrupts. Generic image pyramids do not meet this condition.

Write $r$ for a latent code describing a rater's style and $m$ for a sampled motion hypothesis consistent with the observed image. The predicted per-pixel foreground probability is $p(r,m)$. The total predicted variance then splits by the law of total variance:

$$ \mathrm{Var}[p] = \underbrace{\mathbb{E}_m\left[\mathrm{Var}_r[p \mid m]\right]}_{\text{rater part}} + \underbrace{\mathrm{Var}_m\left[\mathbb{E}_r[p \mid m]\right]}_{\text{motion part}}. $$

The novelty is not this identity. It is making both terms identifiable and checking each against an independent ground truth. The motion term should grow with simulated motion severity and track the extra disagreement real annotators show on corrupted images. The rater term should stay roughly constant across severities and match the clean multi-rater spread.

Where This Sits

Three lines of prior work supply the method and motivate the gap.

Multi-rater segmentation models supply the rater half. Pionono (Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation) gives each annotator a Gaussian in an 8-dimensional latent space, whose mean encodes personal bias and whose variance encodes inconsistency. We reuse this rater parameterization directly. MRNet (Learning Calibrated Medical Image Segmentation via Multi-Rater Agreement Modeling) reconstructs each rater's mask from multi-scale features and feeds the disagreement map back as attention. The variational model of Variational Inference for Quantifying Inter-observer Variability in Segmentation of Anatomical Structures samples coherent expert hypotheses from a 6-dimensional latent. All three were trained and evaluated on static, sharp images, so any boundary blur in their inputs is silently assigned to rater style. The QUBIQ challenge report (QUBIQ: Uncertainty Quantification for Biomedical Image Segmentation Challenge) constrains our design: it found that deep ensembles of ordinary networks often beat elaborate probabilistic models, though at high cost. Ensembles are therefore the competitor to beat, not a strawman.

Motion work supplies the corruption model and the motivating evidence. Simulating rigid head motion artifacts on brain magnitude MRI data–Outcome on image quality and segmentation of the cerebral cortex shows that structured "nodding" motion applied in pseudo k-space can be computed from magnitude images alone. On the MR-ART dataset it reproduces real-motion image quality, with a coefficient-of-joint-variation difference of 0.004 versus 0.015 for generic random simulation, and it reproduces real-motion segmentation degradation. This is what lets us corrupt multi-rater datasets without raw scanner data. Deformation-Aware Segmentation Network Robust to Motion Artifacts for Brain Tissue Segmentation Using Disentanglement Learning jointly corrects motion and segments. It does so through one deterministic deformation field against one reference mask, and so cannot express that a corrupted scan admits several anatomies. Multi-task Swin Transformer for Motion Artifacts Classification and Cardiac Magnetic Resonance Image Segmentation predicts a scan-level artifact grade alongside segmentation, but it skips segmentation training on severe cases. The CMRxMotion results (Extreme cardiac MRI analysis under respiratory motion: Results of the CMRxMotion challenge) withhold masks for non-diagnostic scans and report that left-ventricular volume errors more than double under severe motion. FeTA 2024 (Advances in automated fetal brain MRI segmentation and biometry: Insights from the FeTA 2024 challenge) finds image quality to be the largest single driver of Dice loss, about 0.10, while collapsing rater disagreement into single references. Together these show the gap is real: severe-motion cases, where calibrated uncertainty matters most, are the cases these benchmarks remove.

Multi-scale work motivates the encoder design but also constrains it. Deep Learning for Visual Recognition of Environmental Enteropathy and Celiac Disease found that naively concatenating zoom levels often fails to beat a single scale, so scale fusion needs structure. Hierarchical models such as A Multi-scale Graph Network with Multi-head Attention for Histopathology Image Diagnosisn and EccoMamba (EccoMamba: Enhanced Cross-hierarchical Continuity Orthogonal Mamba for Medical Image Segmentation) assume that a fine-scale feature belongs to the coarse region it sits inside. Motion ghosts break this assumption, because they are copies of distant structures projected across the field of view. We keep the idea of structured scale fusion. We replace spatial containment with frequency-band reliability as the routing signal, because band reliability is the quantity motion physically corrupts.

Research Program

The encoder. Each slice is transformed to pseudo k-space and split into $B$ concentric bands, about four, from the central low frequencies outward. Each band is inverse-transformed into a band-limited image, and band $b$ feeds pyramid level $b$ of an nnU-Net-style encoder. Coarse levels therefore see mostly low frequencies and fine levels mostly high ones, which is roughly what a U-Net pyramid does implicitly, made explicit here.

The band-reliability module. A small head predicts, for each band, a reliability weight $w_b \in [0,1]$ and a per-band spatial attribution map. It is supervised during training by the known corruption: for each simulated motion event we know which phase-encoding lines, and hence which bands, were displaced, and by how much. Features from each level are scaled by $w_b$ before fusion.

The motion hypotheses. These are produced by perturbing the band weights. We sample $w_b$ from a learned Beta distribution centered on the prediction, rather than taking one deterministic correction. A full motion-trajectory posterior is explicitly out of scope; the cheap version is sufficient to test the claim.

The rater side. This follows Pionono. Each rater has a Gaussian latent $q(z \mid r)$ injected at the decoder. Training labels are the clean multi-rater masks, while inputs are motion-corrupted versions of the same slices.

The label-target problem. Clean labels are not the right target for a corrupted input: a rater looking at the blurred image would have drawn something less certain. We therefore add a calibration objective on the motion term, fit on a small set where humans annotated the corrupted images themselves (described below). We also report results with and without this set, so the method's dependence on it is visible.

The annotation study. This is the empirical core. We take QUBIQ multi-rater MRI tasks (prostate, the 7-rater brain growth task, and brain tumor) and apply the nodding simulator at three severities, with motion timed to hit central or peripheral k-space in different conditions. Four to six raters then re-contour a stratified sample of roughly 150 slices across clean and corrupted conditions, blinded to condition. Each rater sees a given anatomy in only one condition, spaced in time to limit memory effects. We will also collect multi-rater contours on a sample of MR-ART real-motion scans (still, 5 nods, 10 nods) for the same subjects. To our knowledge from the file, no dataset measures how human disagreement grows with motion severity on matched anatomy; this study creates one, and it is a contribution independent of the model.

Baselines

The naive reference point is nnU-Net trained on clean multi-rater soft labels, the mean of the rater masks, with no motion augmentation. Its per-pixel entropy serves as the uncertainty. It shows how badly a standard pipeline under-reports uncertainty on corrupted inputs.

The first strong baseline is the same nnU-Net trained with the structured nodding augmentation from Simulating rigid head motion artifacts on brain magnitude MRI data–Outcome on image quality and segmentation of the cerebral cortex and soft multi-rater labels. This is what a careful practitioner would build first, and FeTA 2024 suggests heavily augmented nnU-Nets are hard to beat on overlap metrics.

The strongest competitor for uncertainty quality is a deep ensemble of five such augmented nnU-Nets. QUBIQ reports that ensembles frequently outperform probabilistic architectures. We give it the uncertainty decomposition it supports natively: within-member entropy versus across-member disagreement. That lets us test whether the standard split into data noise and model uncertainty already recovers the motion/rater split.

For rater-aware baselines we train Pionono and MRNet on the same motion-augmented data. For MRNet's prior-work numbers the file gives only its 29 FPS speed on QUBIQ, with no per-task Dice. The strongest reported number on our clean target task is the variational model's continuous Dice of 0.5372 on QUBIQ brain growth, against 0.5083 for MC dropout (Variational Inference for Quantifying Inter-observer Variability in Segmentation of Anatomical Structures). QUBIQ top teams reached average Dice of 0.80–0.85 across tasks. No reported result was found on Lacuna for any multi-rater method evaluated on motion-corrupted QUBIQ or MR-ART images, so every corrupted-condition number will be new and produced under our protocol.

All methods share the nnU-Net preprocessing, the same training slices and augmentation streams, an equal GPU-hour budget, and the same hyperparameter-search budget of 20 configurations each. The ensemble receives five times the training compute, which we report rather than hide.

We also run three ablations. In the first, the frequency-band encoder is replaced by a standard image-space pyramid with the same reliability head; this isolates whether band alignment matters. In the second, the band-reliability supervision is removed; this tests whether the split emerges unsupervised. In the third, the human-annotated corrupted set is removed.

To count as beating the baselines, the method must match the augmented ensemble's Q-Dice on corrupted inputs within one point while using one network. It must also beat every baseline on the attribution checks below by a margin outside bootstrap confidence intervals.

Why not just…?

Why not just train a deep ensemble on motion-augmented data? This is the real threat for total uncertainty, and we expect it may match or beat us on Q-Dice. But ensemble disagreement measures what the models do not know, and motion corruption is a property of the data, not of model ignorance. An ensemble trained on corrupted inputs with clean soft labels should therefore place motion spread inside each member's own predictive entropy, alongside rater spread, and not separate the two. Experiment 2 tests this directly: if ensemble disagreement correlates with motion severity as strongly as our motion term does, our attribution advantage disappears and we will say so.

Why not just predict a scan-level motion grade, as in the multi-task Swin model, and widen the uncertainty when the grade is high? This is cheap and probably good enough for scan-level quality control. The CMRxMotion top method reached Cohen's kappa of 0.631 for grading, while the multi-task Swin model reached only 0.184. A single scalar cannot say which boundary is unreliable, though. When motion hits only peripheral k-space, fine structures such as the thin right-ventricle wall, which CMRxMotion found fails most, degrade while coarse organ outlines survive. We implement this baseline by scaling Pionono's variance with the predicted grade. It must lose on the spatial attribution test, not merely on overall calibration, for our approach to be justified.

Why not just correct the motion first and then run a multi-rater model? A deterministic corrector commits to one anatomy and hands the segmenter a clean-looking image, so downstream uncertainty shrinks exactly where it should grow. Jung et al. also report that intermediate corrected images lose resolution. We run this pipeline with an image-to-image corrector trained on our simulated pairs, followed by Pionono. The prediction to check is that its rater spread on severe motion falls below the human disagreement measured in our annotation study. If it does not, the decoupled pipeline is a legitimate simpler answer.

Why not just use the ordinary U-Net pyramid? A U-Net's downsampled levels already approximate low-pass views, so the frequency-band input may add nothing. The image-space-pyramid ablation answers this question. If it performs as well, the paper's claim reduces to "supervise a reliability head with known corruption," and we would report it in that simpler form.

Proposed Experiments

Experiment 1: does human disagreement actually grow with motion, and where? Using the annotation study, we measure per-pixel rater entropy and pairwise boundary distance as functions of severity and of k-space timing. If disagreement does not rise measurably under nodding motion, Gap 1 is smaller than the literature implies, and the rest of the program should shrink to a calibration study.

Experiment 2: attribution. On simulated data, where the true corruption is known, we compute the Spearman correlation between the predicted motion term and the known displaced-band energy per region, and require the rater term to vary less than 10% across severities. On the human-annotated corrupted set, we check that the motion term predicts the increase in human entropy over the clean condition, while the rater term predicts the clean-condition entropy. On MR-ART real motion, we test the same relationships with no simulator in the loop, which guards against the model learning simulator quirks. On CMRxMotion, which has single annotations, we test only whether the per-scan motion term ranks scans in agreement with the expert artifact grades, including the severe scans that baselines never trained on.

Experiment 3: accuracy and calibration under corruption. We report Q-Dice, expected calibration error, and generalized energy distance to the human sample set on clean, mild, moderate, and severe inputs. FeTA 2024 serves as an out-of-domain check of the motion term on real fetal motion, scored against its image-quality strata.

The decisive outcome is the comparison against the ensemble's native decomposition and against the scalar-grade baseline on Experiment 2. If neither separates motion from raters but our model does, at matched accuracy, the claim holds. If either does as well, the simpler method is the result we report.