Does Seeing the Heart Move Shrink Annotator Disagreement?

When cardiologists outline the left ventricle on an ultrasound video, they usually trace only two frozen frames: end-diastole (maximum filling) and end-systole (maximum contraction). Different experts draw these outlines differently, and those differences carry into ejection fraction and chamber volume. Human vision science offers a sharp prediction here. A single frame of a point-light walker (a person shown only as moving dots at the joints) looks like random scatter, yet the same dots in motion are seen at once as a body. Motion lets the visual system group points that move together and separate them from points that do not. If echocardiographic boundaries behave the same way, part of what we call "inter-observer variability" is a side effect of asking experts to judge a moving structure from a snapshot. That part should shrink when experts see coherent motion. The rest reflects stable personal conventions, such as whether to include trabeculations and papillary muscles, which motion cannot fix.

We propose to measure this split directly and then model it. The human study is a controlled multi-rater annotation experiment on public echocardiography videos. The same experts trace the same frames under three viewing conditions: frozen frame, normal playback, and a frame-shuffled playback that shows the same images for the same time but destroys coherent motion. The modeling contribution is a video segmentation model with two separate sources of predicted disagreement. One is a motion-coherence map, which marks pixels whose intensity over time is poorly explained by the heart's low-dimensional periodic motion. The other is a per-rater style variable. The claim is falsifiable. Motion coherence should predict where disagreement collapses under playback, rater style should predict what survives, and neither prediction should be reproducible by an image-only ensemble.

Core Claim

Inter-observer boundary disagreement in cine cardiac imaging has two components that current models merge. The first is snapshot ambiguity: it is concentrated where tissue and noise are indistinguishable in a still frame, and it falls when coherent motion is visible. The second is convention: it is rater-specific, persists across viewing conditions, and lies outside the cardiac motion manifold. Every multi-rater segmentation model in the file treats images as static. Pionono (Probabilistic Modeling of Inter- and Intra-observer Variability) embeds each rater in an 8-dimensional Gaussian whose mean is bias and whose variance is inconsistency. The Latent Doctor Model conditions a variational model on a rater's example annotations. Liu et al.'s variational model uses a 6-dimensional latent on brain MRI. All three attribute every disagreement to either image ambiguity or rater style, and none can say which disagreements a different viewing protocol would remove. Our design does not assume an answer to the tension flagged by Latent Motion Profiling: cardiac motion fits in a 2-dimensional subspace, while rater style seems to need 6 to 8 dimensions. We measure how much disagreement each subspace accounts for.

Why Biological Motion Is the Right Lens

The perception literature motivates specific controls rather than supplying a method. Vangeneugden et al. found that about 65% of macaque temporal-cortex neurons were tuned to static body form and only about 18% distinguished forward from backward walking. Sequence information appeared only when responses were read out in 50 ms windows. This implies that most visual machinery works on snapshots, and that motion adds a separate, time-sensitive signal. A frozen-frame annotation protocol throws that signal away.

Feng et al. showed that area MST, but not MT, prefers intact over spatially scrambled walkers. Global integration of coherent local motion is the step that matters. This motivates our frame-shuffled condition as the analogue of a scrambled walker: it keeps exposure and image content fixed and removes coherence. It also motivates a reversed-playback condition. Their forward/backward manipulation suggests that if coherence alone drives the benefit, reversed playback should help as much as forward playback. If learned physiological expectations drive it, reversed playback should help less.

The zebrafish tectothalamic circuit (Kappel et al.) and diver tracking by swimming periodicity (Underwater human-robot interaction) are two other systems that separate animate structure from noise by rhythmic signature. That is the computational idea behind our coherence map.

Research Program

Human study. We draw about 150 apical four-chamber videos from EchoNet-Dynamic, stratified by image quality, and recruit six to eight sonographers and cardiologists. Each rater traces the end-diastolic and end-systolic left-ventricle boundary on every video under frozen, forward-playback, frame-shuffled, and reversed conditions. In the playback conditions the rater traces on a paused target frame but can scrub a loop. The design is counterbalanced and within-subject, with washout of at least three weeks between conditions for the same video. A 20% repeat subset estimates intra-observer consistency, following the re-annotation design of the echo quality study. That study measured one expert's grading inconsistency over two months but did not look at contours.

We will record gaze at low cost with a consumer tracker, as in Gaze-Assisted Domain Adaptation. That work used pooled gaze as a single teacher signal. We keep gaze per rater and treat it as a secondary outcome: does playback make experts fixate the same boundary segments?

Model. The backbone is a temporal segmenter pretrained on EchoNet-Dynamic with masked-frame reconstruction, following SimLVSeg (UniLVSeg). That model treats the single expert trace as ground truth, and we replace that assumption. We add two inputs. The first is a motion-coherence map computed from a latent-motion-profiling encoder with a rank-2 motion subspace. For each pixel it gives the residual left after the pixel's temporal trajectory is explained by that low-rank cardiac motion. The second is a Pionono-style rater latent. The model is trained on the multi-rater traces and outputs a per-rater contour distribution. Its structure attributes spread to coherence (shared across raters) and to style (rater mean and variance). For volumetric generalization we will test the same recipe on 3D cine CMR, using the propagation setting of Tayebi Arasteh et al.. That work cut volume bias to at most 5.2 ml but treated its anchor labels as single truths. We treat this CMR step as a stretch goal because no multi-rater cine CMR data exists in the file.

Baselines

The naive reference point is a fixed-width band of predicted disagreement around a single deterministic contour. Disagreement always concentrates near boundaries, so any claimed predictor of where experts disagree must beat "a constant ring of width w." We tune w on validation data. A second naive reference is majority-vote training, which the Latent Doctor Model showed to be overconfident on borderline cases.

The strongest existing competitor is a deep ensemble. QUBIQ found that simple deterministic ensembles often beat probabilistic architectures at matching multi-expert distributions, with top average Dice between 0.80 and 0.85 on static tasks. Because the question involves video, we build that ensemble from the same SimLVSeg-style temporal backbone, not from frame models. It gets the same multi-rater data, splits, and hyperparameter search budget as our model. We report its inference cost next to ours instead of capping its member count. For segmentation accuracy itself, the reference is SimLVSeg's reported Dice of 93.32% on EchoNet-Dynamic (UniLVSeg). Chao et al. reported Dice of 0.911 for fine-tuned SAM and 0.915 for the domain-specific EchoNet model. We do not expect to beat these numbers and do not aim to; we only require that we do not fall meaningfully below them. For predicting disagreement on echocardiography, no reported result was found on Lacuna for any method. On the static QUBIQ brain-growth task, Liu et al. report continuous Dice of 0.5372 against 0.5083 for Bayes dropout. That result shows these metrics discriminate between methods but is not directly comparable.

Pionono is the rater-conditioned competitor. We run it on the same temporal backbone without the coherence input.

The key ablation uses the same model in three versions: with the coherence map removed, with it computed from frame-shuffled video, and with it replaced by generic optical-flow magnitude. The shuffled version isolates coherent periodic motion from the mere presence of temporal input. The optical-flow version tests whether the low-rank periodic prior matters.

Beating the ensemble means two things. First, the partial correlation between predicted and observed frozen-condition disagreement must be higher after controlling for ensemble variance, with bootstrap confidence intervals over videos that exclude zero. Second, the coherence term must specifically predict the per-location drop in disagreement from frozen to playback, which the ensemble has no mechanism to predict.

Why not just…?

Why not just train a deep ensemble on the multi-rater labels? This is the real threat, given QUBIQ's result. An ensemble of temporal models may already absorb motion cues implicitly and track disagreement well. What an ensemble cannot do is separate disagreement that a protocol change removes from disagreement that it does not, because it never sees viewing condition or rater identity. The decisive experiment is whether ensemble variance predicts the frozen-to-playback reduction as well as the coherence map does. If it does, we will report that the ensemble suffices and treat the coherence map as an interpretation, not a method.

Why not just show raters the video and skip the modeling? If playback cuts disagreement, that protocol change is the cheapest possible outcome, and the human study reports it as the primary result. The shuffled-frame and reversed controls say why it works, which determines whether frozen-frame datasets such as EchoNet-Dynamic can be corrected after the fact. The model is needed for the existing millions of frozen-frame traces. It flags which existing labels are likely snapshot-ambiguous and which reflect convention, and that cannot be recovered by re-annotating.

Why not just use temporal flicker of a single model's predictions as the disagreement signal? Frame-to-frame instability of a segmenter is a cheap motion-aware uncertainty proxy, and we include it in the optical-flow ablation family. Flicker reflects the model's instability, not the image's motion structure. The test is whether flicker explains observed human disagreement beyond the band baseline as well as the coherence residual does.

Why not just regress ejection fraction directly and avoid boundaries? Direct regression forests reached a correlation of 0.922 for the left ventricle, but they were trained on volumes from manual traces. They therefore inherit the disagreement we are trying to explain, and they produce no contour for uses that need geometry. We will report the spread of ejection fraction across raters under each condition. Chao et al. cite 7 to 13% expert variation, and this measure shows whether the boundary effect matters clinically.

Decisive Checks

The primary pre-registered test compares median radial boundary spread across raters, measured in millimetres along spokes from the ventricle's centroid, between frozen and forward playback. Secondary outcomes are area coefficient of variation and ejection-fraction spread.

The snapshot hypothesis survives only if three conditions hold. Forward playback must reduce spread. Frame-shuffled playback, with matched exposure, must reduce it significantly less. The reduction must be concentrated where the coherence residual is high.

The convention hypothesis survives only if rater latents fitted on frozen-condition traces predict the same rater's playback traces better than a pooled model does. That would show that style persists when ambiguity is removed.

We also check whether reversed playback matches forward playback, which would indicate that coherence rather than physiological expectation drives the benefit, and whether gaze dispersion across raters falls under playback.

If playback leaves spread unchanged and the coherence map adds nothing beyond the ensemble, we conclude that echocardiographic disagreement is almost entirely convention. That is a useful negative result: it would mean that video-aware multi-rater models are not worth their cost, and that effort should go into standardizing annotation rules instead.