Motion as a Tie-Breaker: Resolving Annotator Disagreement at Low-Contrast Sliding Boundaries

Where two tissues have the same brightness on a scan, experts disagree about where one ends and the other begins. Those disagreements fall precisely on the thin boundary voxels that matter most for small, hard-to-segment structures. In many such places the image sequence still carries information that a single frame does not. If the tissues slide past one another during the heartbeat or a breath, the boundary is the line where motion abruptly changes. We propose to use that motion change as evidence for where the boundary lies, but only inside the band where annotators disagree. The result is a training target for segmentation that follows the experts where they agree and follows the physics of motion where they do not.

Concretely, an unsupervised registration network that learns where to relax its smoothness penalty already produces a map of sliding interfaces as a by-product, without any labels. We turn that map into a modified boundary-distance loss for segmenting thin, rare structures. Within the inter-observer disagreement band, the loss's zero level is pulled toward the detected motion discontinuity. Where no motion discontinuity exists, the loss stays agnostic instead of enforcing one rater's arbitrary line. The bet is that this gives sharper, more temporally stable, and more clinically consistent boundaries than averaging raters, modeling each rater's style, or simply feeding motion features to the network as an extra input.

Core Claim

The claim is specific. On sliding, low-contrast interfaces in cine imaging, the location of the motion discontinuity falls inside the inter-observer disagreement band. It also lies closer to a carefully adjudicated reference boundary than the typical individual rater does. Using it to place the supervision target inside that band improves thin-structure segmentation more than any approach that uses motion only as an input feature, or that uses multiple raters without motion. The novelty is on the label side. Motion is used to decide what the ground truth should be where humans cannot agree. It is not used as a feature, as a routing mask, or as an augmentation tool.

This connects the four directions without giving them equal weight. Sliding motion modeling supplies the evidence. Inter-observer variability defines where that evidence is allowed to act. Low-contrast ambiguity is the condition under which the evidence is needed. Extreme spatial imbalance dictates the loss family, because thin structures need boundary-distance supervision rather than overlap losses alone.

Why This Gap Is Real

Three findings in the literature motivate the proposal.

Discontinuity-preserving registration that routes motion through crisp masks is fragile. DDIR predicts a separate velocity field per segmented cardiac compartment and stitches them together. SDDIR learns the segmentation jointly, but its authors report that registration degrades once segmentation Dice falls below roughly 70–76%. That is exactly the accuracy range reported for low-contrast structures such as the esophagus on SegTHOR (Kondratenko et al., 0.80 Dice; EMSN, baseline 0.7664). We therefore deliberately avoid mask-routed motion.

A mask-free alternative exists. TM-SPR learns a per-voxel regularization weight under a Beta prior and lowers that weight at sliding interfaces such as the lung–chest wall, reaching 1.356 mm target registration error on lung 4DCT. Yet that map is discarded after alignment and never informs segmentation. MemWarp similarly learns anatomical filters, from single-annotator masks, and uses them only for warping. These papers supply our motion component and show that motion edges can be found without labels.

On the label side, the Boundary loss integrates the network's soft output against a signed distance map of the reference contour. It stabilizes training on tiny targets (ISLES Dice 0.575 to 0.656 when added to Generalized Dice). It assumes, however, that the contour is a single correct line, so any annotator's bias becomes a systematic gradient across the whole margin. Two works constrain how we treat multiple raters. Pionono models each rater as a Gaussian in a latent space and reproduces individual styles (Gleason 2019 κ 0.758). DiFF instead defines ground truth by downstream diagnostic utility. Both treat disagreement as something to reproduce or to resolve with an external classifier. Neither uses physical evidence in the images to decide which hypothesis is right. DiFF also needs diagnostic labels that cardiac cine datasets often lack.

Research Program

The method has three parts.

Motion evidence. First, a TM-SPR-style registration network is trained, without labels, on all frame pairs of each cine sequence. The output we keep is a kinematic edge map $\kappa(x) = 1 - \bar\Lambda(x)/\lambda_{max}$. Here $\bar\Lambda(x)$ is the learned regularization weight at voxel $x$, averaged across frame pairs, and $\lambda_{max}$ is its prior upper bound. Averaging over the cardiac cycle matters: a true sliding interface should reappear in many phases, while spurious drops from noise should not.

Disagreement band. Second, from $R$ rater masks we form the agreed foreground $A$ (the voxels all raters include), the agreed background (the voxels no rater includes), and the disagreement band $B$ in between.

Motion-adjudicated boundary loss. Inside $B$ we extract the ridge of $\kappa$ and keep it only where its strength exceeds a threshold $\tau$ chosen on a validation fold. The kept ridge, together with the band edges, defines a target surface $S^\*$; $\phi_{S^\*}$ is the signed distance to that surface. The loss is

$$\mathcal{L}(\theta) = \int_{\Omega\setminus B} \phi_{\partial B}(x)\, s_\theta(x)\,dx \;+\; \int_{B} m(x)\,\phi_{S^\*}(x)\, s_\theta(x)\,dx .$$

In this expression, $s_\theta(x)$ is the network's foreground probability. $\phi_{\partial B}$ is the signed distance to the nearer band edge, so it pushes outside-band voxels to agree with the raters. $m(x)\in[0,1]$ is the normalized ridge strength, set to zero where no motion discontinuity is found, so that those parts of the band receive no boundary gradient.

As in the original boundary loss, this term is scheduled against Generalized Dice so that the object is located before boundary terms dominate. In the band, the loss falls back to "tolerance" where motion is silent. This is intentional. It is how the method avoids enforcing an arbitrary rater line at non-sliding interfaces, such as the endocardium against blood, where motion is not a reliable boundary cue.

Primary setting. The main application is short-axis cardiac cine-MRI from ACDC. There, the thin right-ventricular free wall and the epicardium, which slides against the pericardium, lung and liver, combine sliding, low contrast and severe imbalance. ACDC as described in the file has single annotations, so we will collect three independent rater contours on a subset of about 40 subjects at end-diastole and end-systole. Raters will see single frames only. A separate adjudicated reference will be drawn by a panel reviewing the full cine loop. It serves only for evaluation, never for training. UK Biobank cine data (DDIR's setting) provides additional unlabeled sequences for training the registration component.

Positive control. Lung 4DCT is a high-contrast case where the lung–chest-wall motion edge and the intensity edge should coincide. Agreement there verifies the motion component before we trust it where contrast is absent.

Baselines

The naive reference is a 3D U-Net trained with Generalized Dice on a single randomly assigned rater per case. It represents what most pipelines do with one annotation. The consensus baseline uses the same network trained with the scheduled Generalized Dice plus boundary loss of Kervadec et al. against a STAPLE consensus of the three raters. This is the strongest simple competitor because it already addresses both imbalance and multiple raters.

The strongest existing multi-rater method is Pionono, re-implemented with a 3D backbone and evaluated through its consensus latent. Its reported result is κ 0.758 on Gleason 2019, but there is no reported result on Lacuna for it on cardiac cine data. On the motion side, a mask-routed joint model in the style of SDDIR tests whether coupling segmentation and discontinuous registration through masks already captures our benefit. No reported ACDC segmentation number for it was found on Lacuna. For context only, MemWarp reports 89.61% average registration Dice on ACDC, which is a registration metric and not directly comparable.

Two ablations isolate the key ingredient. The first, "band-tolerance only," sets $m\equiv 0$, which removes the motion evidence while keeping the band. The second, "motion as input," concatenates $\kappa$ to the image channels under the consensus loss.

All models share one backbone, the same patch size, the same training split and the same compute budget. Every model gets an equal hyperparameter search of 20 trials, with $\tau$ and the loss schedule counted inside our method's 20. To count as beating a baseline, our method must achieve a lower 95th-percentile Hausdorff distance and lower mean surface distance to the adjudicated reference on the thin structures (right-ventricular wall and myocardium). These differences must hold under a paired test across subjects. Dice must not drop more than one point.

Why not just…?

Why not just feed the motion map to the segmentation network as an extra input channel? This is the most serious threat, because a network could learn to read $\kappa$ by itself. The difference is what the network is rewarded for. With motion as an input but consensus labels as the target, the network is still trained to reproduce STAPLE's line inside the band. It will learn to ignore $\kappa$ wherever $\kappa$ disagrees with the label, which is exactly where we claim motion is informative. The "motion as input" ablation settles this directly. If it matches our method on distance to the adjudicated reference, the label-side contribution is not needed and we would report that.

Why not just form a STAPLE or majority-vote consensus and train with boundary loss? Consensus places the line at the raters' center of mass. Pionono and DiFF both argue that averaging discards real information, although neither measured distance to a physically adjudicated boundary. If single-frame raters are biased the same way at sliding interfaces, for instance all under-contouring a faint epicardium, the consensus inherits that bias. Motion can correct it. The consensus baseline tests this, and the analysis is stratified by whether the band lies at a sliding interface.

Why not just model each rater's style, as Pionono does? Rater modeling reproduces the distribution of human contours faithfully. It has no mechanism for saying which hypothesis is physically correct. At best its consensus latent approaches STAPLE. If Pionono's consensus beats ours on surface distance at sliding interfaces, our central premise is wrong.

Why not just enforce temporal consistency by propagating masks across frames with registration? Such a loss rewards contours that move coherently. A contour placed consistently but wrongly satisfies it equally well, so it does not resolve where inside the band the boundary sits. We include it as a control. We expect it to improve frame-to-frame stability without reducing distance to the adjudicated reference.

Proposed Experiments

Pre-training check. The first check runs before any segmentation training and could end the project. For each annotated subject, we measure the distance from the $\kappa$ ridge to the adjudicated reference, compared against the distance of each individual rater to that reference. We also measure the fraction of the band where a ridge passes $\tau$. If the motion ridge is not closer than the median rater at sliding interfaces, the proposal fails as stated. On lung 4DCT, the ridge should coincide with the intensity edge to within about one voxel. Our registration reimplementation should approach TM-SPR's reported 1.356 mm target registration error before we trust it.

Main comparison. The main experiment compares all baselines on the 40 multi-rater ACDC subjects. Metrics are surface distance and 95th-percentile Hausdorff distance to the adjudicated reference, analyzed separately for sliding interfaces (epicardium, right-ventricular free wall) and non-sliding interfaces (endocardium). The prediction is that gains appear only at sliding interfaces, and that we roughly tie the band-tolerance ablation at non-sliding ones. A uniform gain everywhere would suggest the improvement comes from band tolerance rather than motion.

Downstream checks. Two further checks probe downstream value. The first is the variability of myocardial mass, computed from automatic contours across end-diastole and end-systole, since true mass should not change across the cycle. This follows DDIR's use of clinical index preservation. The second is frame-to-frame boundary jitter across the full cine loop.

Robustness to segmentation quality. We will repeat the main comparison with weakened segmentation backbones. The question is whether our method, which never routes motion through masks, avoids the collapse SDDIR reports below 70–76% Dice. That would make it the more usable design for low-contrast organs.