Phase-Locked Observer Disagreement in Free-Breathing Cardiac MRI

When several experts outline the heart on the same MRI frame, their outlines disagree. The medical imaging community usually treats that disagreement in one of two ways. It is either noise to be averaged away, or irreducible ambiguity to be reproduced by a probabilistic model. In real-time, free-breathing cardiac MRI, the heart is imaged continuously while the patient breathes and, in stress tests, exercises. Here a third possibility becomes testable: part of the disagreement may rise and fall with the breathing cycle and the heartbeat. If it does, the rhythm of the body can be measured from the images themselves with ordinary frequency filtering, the same band-limited trick used to pull a pulse out of face video. That measurement would let us split observer disagreement into a component locked to physiological phase, which is plausibly a motion artifact that correction can remove, and a residual component that plausibly reflects genuine anatomical ambiguity that should be kept.

The proposal is to measure this split directly, confirm causally that the phase-locked part is motion-induced, and build a segmentation model that predicts the two parts separately. The practical payoff is in exercise-stress cardiac MRI. There, automatic segmenters and humans both degrade, and clinical pipelines still depend on a person choosing "end-expiration" frames by hand. A decomposition of disagreement into motion-driven and anatomy-driven parts tells us which uncertainty a better acquisition or motion correction can buy back, and which uncertainty a report should carry forward.

Core Claim

In free-breathing real-time cardiac MRI, we hypothesize that a substantial and stress-dependent fraction of frame-to-frame inter-observer disagreement is predictable from respiratory and cardiac phase alone. Both phases can be estimated from the image sequence by adaptive frequency tracking, without extra hardware. We further hypothesize that correcting respiratory motion before annotation removes mostly this phase-locked fraction and leaves the residual largely unchanged.

Let $d(t)$ be a disagreement score for frame $t$, such as the mean pairwise boundary distance among raters. Let $\phi_r(t)$ and $\phi_c(t)$ be the respiratory and cardiac phases recovered by filtering. We fit

$$ d(t) = f(\phi_r(t)) + g(\phi_c(t)) + \epsilon(t), $$

where $f$ and $g$ are smooth periodic functions (low-order Fourier series in phase) and $\epsilon(t)$ is the residual. We then define the phase-locked fraction as $\rho_r = \mathrm{Var}[f(\phi_r)]/\mathrm{Var}[d]$, and analogously $\rho_c$ for cardiac phase. The claim is that $\rho_r$ is non-trivial at rest, grows under exercise stress, and collapses after respiratory correction while $\mathrm{Var}[\epsilon]$ does not. If $\rho_r$ is near zero even at maximal stress, the hypothesis is wrong, and that is itself a useful negative result: rhythm-aware processing would then not help explain observer variability.

Why This Gap Is Real

Three papers motivate the question. Assessment of deep learning segmentation for real-time free-breathing cardiac MRI at rest and under exercise stress segments left ventricle, myocardium and right ventricle in real-time scans of 15 volunteers up to 164 bpm. Human reference contours became hard to draw consistently at maximal stress, and end-expiration frames had to be picked by hand. The paper reports but does not decompose this disagreement, and it calls for automated phase isolation. Continuous Spatio-Temporal Memory Networks for 4D Cardiac Cine MRI Segmentation propagates masks across the whole cardiac cycle, but reports failure under inconsistent breathing. It also outputs deterministic masks and has no multi-rater labels between end-diastole and end-systole. Repetitive Motion Estimation Network supplies the method we reuse. It maps fluoroscopy video onto a one-dimensional motion signal, then separates cardiac motion with a 0.5–2.0 Hz band-pass and respiration with a sub-0.33 Hz low-pass. It never produces masks or touches observer variability. Its fixed cutoffs also assume resting physiology, and those cutoffs break at 164 bpm, when the cardiac fundamental approaches 2.7 Hz and fast breathing moves toward the cardiac band.

On the observer side, three latent-rater models reproduce multi-rater distributions on static images. Pionono gives each rater a Gaussian in an 8-dimensional latent space, with a mean for systematic bias and a variance for intra-rater inconsistency. The Latent Doctor Model learns rater styles from small support sets with a conditional variational autoencoder, which is a generator that samples plausible outputs from a learned latent distribution. Variational Inference for Quantifying Inter-observer Variability uses a 6-dimensional variational latent space on QUBIQ brain-growth MRI. None of the three has temporal structure. Disentangling Human Error from Ground Truth constrains the design in a different way: it treats all disagreement as annotator error around a single true map. Our decomposition takes a middle position. It treats the phase-locked part as removable corruption and the residual as legitimate ambiguity, and it tests that split empirically rather than assuming either extreme.

Research Program

The first component is phase extraction that survives stress. Following the image-derived surrogate idea of the motion estimation network, we compute a respiratory surrogate from each real-time sequence. Candidates are the first principal component of intensities in a liver–diaphragm region, or a small learned regressor. We compute a cardiac surrogate from ventricular blood-pool intensity. The real-time scans run at 33 ms per frame, so both rhythms sit well below the Nyquist limit. We track each fundamental frequency with a sliding short-time spectral peak and center a narrow band on the tracked frequency, rather than using fixed 0.5–2.0 Hz and 0.33 Hz cutoffs. Instantaneous phase comes from the analytic signal of the band-limited trace. We validate respiratory phase against the frames that human readers label as end-expiration. Tracking bands matter because two failure modes appear under stress. At rest, respiratory harmonics near 0.75–1.0 Hz can alias into a fixed cardiac band, and at maximal stress the two fundamentals move toward each other. The comparison between fixed and tracked bands is an explicit ablation.

The second component is a new multi-rater dataset over time, because no dataset in the file has multiple raters on intermediate frames of real-time scans. We would use real-time free-breathing scans from the public OCMR benchmark used by MoCo-INR. If access can be arranged, we would add the rest and stress cohort from the real-time assessment study. For each subject and a mid-ventricular slice, four readers segment about 60 consecutive frames covering at least three breaths, shown in shuffled order so that readers cannot copy contours between neighboring frames. A repeated subset measures intra-rater consistency. In the intervention arm, the same frames are shown again after respiratory motion correction. Correction is either a rigid in-plane shift predicted from the filtered respiratory surrogate or a MoCo-INR motion-compensated reconstruction, using the released MoCo-INR code.

The third component is a phase-conditioned rater model. We extend Pionono so that the decoder receives the image and a rater code, together with $(\sin\phi_r, \cos\phi_r, \sin\phi_c, \cos\phi_c)$ and a two-frame temporal context. The model has two outputs: a spread of samples across the rater distribution, and a per-pixel split of predicted disagreement into a phase-explained part and a residual. We fit this split with the same additive structure as the equation above, applied per pixel.

Baselines

The naive reference point is a single deterministic nnU-Net trained on ACDC breath-hold cine, as in the real-time assessment study. It uses zero predicted disagreement everywhere, and its only uncertainty proxy is the softmax entropy. The strongest reported number for this setting is from Assessment of deep learning segmentation for real-time free-breathing cardiac MRI. Under maximal stress, nnU-Net reaches Dice 0.92 for the left ventricle, 0.85 for myocardium and 0.83 for the right ventricle. Its ejection-fraction mean absolute difference is 2.6% at rest and 3.6% under stress. We reproduce this model on our data as the anchor.

The strongest competitor for predicting disagreement is a deep ensemble of five nnU-Nets trained on our multi-rater frames. The QUBIQ challenge found that simple ensembles often matched or beat probabilistic U-Nets and dropout, with top ensemble teams at average Dice 0.80–0.85 across its static tasks. For per-frame disagreement prediction on real-time cardiac multi-rater data, no reported result was found on Lacuna, so we will establish this number ourselves.

The strongest multi-rater method is per-frame Pionono without phase input. Its reported unweighted Cohen's kappa is 0.758 on Gleason 2019, from the Pionono paper. That figure comes from a different domain, and we cite it only to show the method is competitive where it was designed.

The key ablation isolates the physiological phase ingredient. It is identical to our model, with the same temporal context, parameter count and tuning budget, but receives no phase features. A second ablation feeds phase from fixed-cutoff filters instead of tracked bands.

For the ejection-fraction endpoint, we add a pipeline that selects end-expiration frames automatically using our respiratory surrogate and then runs the deterministic nnU-Net. This baseline also answers the manual bottleneck raised in the real-time study.

All models share the same training frames, the same subject-level cross-validation folds and the same hyperparameter search budget, with the ensemble given its natural five-times training compute. We count our model as winning if it meets all of the following. Its Spearman correlation between predicted and observed per-frame disagreement must beat the ensemble and the no-phase ablation, with confidence intervals from bootstrapping over subjects. Its generalized energy distance and QUBIQ-style threshold-averaged Dice to the rater set must be no worse. Its predicted phase-locked fraction must track the measured $\rho_r$ across rest and stress.

Why not just…?

Why not just select end-expiration frames automatically and ignore the rest? This is the most serious practical threat, because it may remove most motion-induced disagreement for volume measurement at very low cost. We include it as a baseline, and if it matches our ejection-fraction variability it wins on that endpoint. It still discards most frames, and at high breathing rates the quiet window shrinks to a few 33 ms frames. It also cannot say how much of the remaining disagreement is anatomical. That question is the scientific target here, and the intervention arm answers it regardless.

Why not just give a temporal network more frames and let it learn motion implicitly? A network given neighboring frames could learn phase without being told. The no-phase ablation has the same temporal context precisely to test this. If it matches our model, the explicit filtering adds nothing to prediction, and the contribution narrows to the measurement study and the tracked phase estimator. The prior evidence points the other way. In the Repetitive Motion Estimation Network work, adding frequency separation to a learned temporal representation produced 27 false positives out of 1,703 peaks, against 96 for an optical-flow baseline. That result is about peak detection, however, not segmentation uncertainty, so the question remains open.

Why not just use an ensemble's variance as the uncertainty? QUBIQ gives ensembles real standing. An ensemble variance is still a single undifferentiated number: it cannot say whether better motion correction would shrink it. The decisive comparison is whether ensemble variance drops by the same amount as rater disagreement under the correction intervention. If it does, the ensemble already implicitly contains the decomposition and our model's second output is redundant.

Proposed Experiments

The first decisive check is purely descriptive. Using the multi-rater time series, we estimate $\rho_r$ and $\rho_c$ with subject-level confidence intervals at rest, and at moderate and maximal stress where the cohort permits. We also check coherence between $d(t)$ and the respiratory surrogate at the tracked frequency. The hypothesis predicts $\rho_r$ clearly above a shuffled-phase null at rest and larger under stress.

The second check is causal. After respiratory correction, the phase-locked variance $\mathrm{Var}[f(\phi_r)]$ should fall substantially while $\mathrm{Var}[\epsilon]$ stays within its intra-rater noise band, measured from repeated frames. If the residual also collapses, the "anatomical ambiguity" reading of the residual is undermined. If nothing changes, motion is not what drives disagreement.

The third check validates phase extraction. We measure agreement of automatically detected end-expiration frames with reader-chosen frames, and compare tracked against fixed bands at maximal stress, where we expect fixed cutoffs to fail.

The fourth check covers prediction and clinical use. We evaluate per-frame disagreement correlation and distribution metrics against all baselines. We also compare the spread of ejection fraction across raters and model samples with nnU-Net's reported 3.6% mean absolute difference under stress, to ask whether the phase-explained component accounts for the excess over rest.