Doctors can cheaply scribble a few strokes inside an organ, but they rarely draw near its edges, and the edges are where segmentation models fail. This is worst where neighbouring tissues look almost the same, such as a heart wall next to blood, or scar next to the blood pool. This note proposes to fill that gap by moving other patients' scribbles into each new scan. A registration network (a model that predicts a smooth deformation mapping one scan onto another) warps the sparse strokes of many patients into the target scan. Each patient scribbled at slightly different positions, so after warping their strokes collectively cover the target structure much more densely, including the boundary band that no single annotator touched. The segmentation network then learns from this pooled evidence. Lesions are not pooled, because their location is patient-specific. Instead they are constrained to lie inside the container organ whose boundary the pooled evidence pins down.
The idea matters because existing scribble-supervised methods recover the missing boundary from local intensity continuity: random walks, graph cuts, or conditional random fields (CRFs, smoothing models that pull labels along similar-intensity pixels). These leak exactly where contrast is low. Existing joint registration-segmentation methods need at least one densely labelled scan. This proposal replaces the local intensity heuristic with a cross-patient shape prior, and it asks for no dense label at all.
Core Claim
Registration-based pooling of scribbles from many patients, with three safeguards, recovers low-contrast container boundaries better than single-image propagation plus a containment loss. The first safeguard weights each warped vote by how reliable the registration is locally. The second trains the registration network only on container-level anatomy, with gradients from the pathology head blocked. The third gates pathology predictions by the container through a soft containment loss.
The claim is specifically about boundaries. The gain should appear in a band a few voxels wide around the true contour, and it should grow with the number of pooled neighbours $K$ until registration error near the boundary exceeds the typical distance from scribble to boundary. Past that point, pooling should hurt. That crossover is the falsifiable prediction.
The closest existing work differs in what it assumes. DeepAtlas couples a registration and a segmentation network so that each supervises the other, but it propagates dense labels from $N\ge1$ annotated scans. Bi-JROS stabilises the same coupling as a leader-follower game, but it also needs one fully annotated atlas and models no nesting of structures. Both use registration to copy a complete label from one scan. Here registration aggregates incomplete labels from many scans, and the aggregation is what produces boundary coverage. On the scribble side, Can et al. and Ji et al. expand scribbles within each image independently using random walks, graph cuts and CRFs. They are the methods whose boundary leakage this proposal targets.
Research Program
The system has three parts.
Segmentation network. A segmentation network $S$ predicts container classes (for example myocardium and ventricular cavity, or prostate central gland and peripheral zone) and, where applicable, nested pathology classes. On the MyoPS benchmark these are scar inside edema inside myocardium.
Registration network. A registration network $R$ maps each neighbour scan $k$ onto target $t$ with a deformation $\phi_{k\to t}$.
Pooled supervision. The pooled target for container class $c$ at voxel $v$ is a reliability-weighted vote over warped neighbour scribbles:
$$ \hat{y}_t(v,c)=\frac{\sum_{k=1}^{K} w_{k}(v)\,\mathbb{1}[s_k(\phi_{k\to t}(v))=c]}{\sum_{k=1}^{K} w_{k}(v)\,\mathbb{1}[s_k(\phi_{k\to t}(v))\neq\varnothing]}. $$
Here $s_k$ is neighbour $k$'s scribble map, with $\varnothing$ meaning unannotated. The weight $w_k(v)$ is a local reliability score computed from the inverse-consistency error of the deformation and from agreement between $S$'s container prediction on $t$ and the warped prediction from $k$. $S$ is trained with partial cross-entropy on the target's own scribbles. It is also trained with confidence-weighted cross-entropy on $\hat{y}_t$, only at voxels where the vote is confident.
Registration losses. $R$ is trained with an image similarity term and an anatomy term on $S$'s container predictions, in the spirit of DeepAtlas. It also gets a contradiction penalty that comes directly from scribbles: a warped neighbour scribble of class $c$ must not land on a target scribble of a different class. This gives registration a weak supervision signal that needs no dense masks.
Asymmetric coupling. Pathology logits never send gradients into $R$. This design follows two negative results. U-MyoPS reports that jointly optimising registration with scar and edema segmentation degraded pathology accuracy, which forced multi-stage training. APEx found that letting pathology information flow back into anatomy representations hurt, and that only anatomy-to-pathology flow worked. Our explicit hypothesis is that the conflict U-MyoPS observed comes from pathology gradients reaching registration, and that confining registration to containers removes it.
Containment. Pathology is bounded by a soft inclusion loss that penalises lesion probability exceeding container probability. This reuses the idea of the Pathology Inclusiveness loss and Myocardium Prior module from MyoPS-Net, which were developed under full supervision on pre-aligned images. The loss lets lesion errors near a container mistake be corrected through training, rather than truncated by a hard mask.
Neighbour selection and scheduling. Neighbours are chosen by image similarity after affine alignment, and $K$ is swept. Training starts with registration warmed up on intensity similarity alone, then alternates updates as in DeepAtlas.
Analysis. We will measure scribble coverage, meaning the fraction of boundary-band voxels that receive at least one confident vote, as a function of $K$. We will also measure the precision of those votes against held-out dense labels. Together these give a direct mechanistic account of whether pooling helps, rather than relying only on end-task Dice.
Baselines
Naive reference. The do-nothing reference is a U-Net trained with partial cross-entropy on the target's own scribbles only. It shows how much any propagation buys.
Strongest within-image scribble method. The strongest existing scribble competitor is Can et al.: random-walk expansion of scribbles, iterative U-Net retraining, CRF refinement, and Monte Carlo dropout filtering. It reports 88.0% mean Dice on ACDC (against 91.2% with full supervision) and 78.1% on NCI-ISBI prostate (against 81.8%). It is the strongest competitor because it attacks the same boundary gap with local intensity propagation. We reimplement it on our splits with the same backbone, and give its CRF kernel weights the same tuning budget as our method's hyperparameters ($K$, vote threshold, loss weights), measured in validation runs. To beat it, our method must improve boundary-band Dice and 95th-percentile Hausdorff distance with non-overlapping seed-level confidence intervals, and match or exceed its whole-structure Dice. A whole-structure gain alone is not enough, because the claim is about boundaries.
Strongest scribble method with hierarchy. Can et al. plus the soft containment loss, with no registration, isolates the contribution of registration. We consider it the most dangerous baseline; see below.
Dense-atlas joint methods. DeepAtlas and Bi-JROS are run with the number of dense atlases whose annotation time matches our scribble budget. Bi-JROS reports 82.8% Dice on brain MRI with one atlas. The file gives no reported result for DeepAtlas or Bi-JROS on ACDC, NCI-ISBI or MyoPS, so we must run them ourselves. Beating them at equal annotation time is the evidence that many patients' scribbles beat one patient's dense mask.
Full-supervision references for MyoPS. These are MyoPS-Net (scar Dice 0.661, edema Dice 0.742) and the 2020 challenge winner (scar 0.708, edema 0.731), as reported in MyoPS-Net and the MyoPS benchmark paper. They serve as upper references, not targets. No reported result was found on Lacuna for scribble-supervised MyoPS pathology segmentation.
Ablations. Three ablations isolate the key ingredients. The first uses a frozen, separately trained registration for pooling. The second removes the reliability weights $w_k$. The third lets pathology gradients reach $R$, which directly tests the U-MyoPS failure hypothesis.
Why not just…?
Why not just add a containment loss to an existing scribble method and skip registration? This is a real threat. A containment loss alone may already stop most lesion leakage, as the MyoPS-Net ablation suggests: scar Dice rose from 0.560 to 0.656 when its prior and inclusion terms were added. But containment only says lesions stay inside the container. It says nothing about where the container's own low-contrast boundary lies, and that is where scribbles provide no signal. The "Can et al. plus containment" baseline settles the question. If it matches us in the boundary band, the pooling idea is wrong and we will report that.
Why not just register with an off-the-shelf tool first, then pool scribbles? This sequential version is cheaper and could work well on cine MRI, where cross-subject shapes are regular. Evidence against it is partial. U-MyoPS found ANTs and VoxelMorph distorted difficult cross-sequence alignments, and DeepAtlas reports 1.8–2.7% higher registration Dice from segmentation-guided training over unsupervised registration. Those settings use dense labels, though. The frozen-registration ablation measures directly whether joint training improves vote precision near boundaries. If it does not, the simpler pipeline should be preferred.
Why not just annotate 5–10% of scans densely and use semi-supervised learning? On ACDC, Collaborative Self-Training reports Dice of 0.730 with 5% labels and 0.801 with 10%, with Mean Teacher at 0.654. Scribbles on every scan reach 88.0% in Can et al. That suggests breadth of coverage across patients matters more than depth on a few. We nonetheless include the equal-annotation-time comparison against dense-atlas joint methods, since annotation time per scan is the honest currency.
Why not just clip lesions with the predicted container mask after training? Hard clipping cannot repair a container that is under-segmented at a faint margin. It deletes true lesion voxels there, and the network's internal features never learn the constraint. We run clipping as a variant of the containment baseline so that the cost appears on MyoPS scar adjacent to the blood pool.
Proposed Experiments
Datasets and scribbles. Container-boundary experiments use ACDC (left and right ventricle and myocardium; myocardium surrounds the left ventricular cavity) and NCI-ISBI 2013 prostate (central gland and peripheral zone). Following Can et al., both use scribbles on all training subjects, so we can compare directly against their reported numbers. The full hierarchy is tested on MyoPS 2020: myocardium contains edema, which contains scar. Its 25 training cases have dense labels, so scribbles are synthesised by eroding and skeletonising the dense masks. They are placed only in structure interiors at controlled distances from the boundary, and a sensitivity study varies that distance.
Metrics. We report Dice and 95th-percentile Hausdorff distance on whole structures and within a 3-voxel boundary band. For the mechanism analysis we also report vote coverage and vote precision.
Decisive checks. There are three. First, boundary-band Dice must exceed "Can et al. plus containment" on ACDC and NCI-ISBI. Second, the curve of boundary error against $K$ must show the predicted rise and then fall, with the turning point tracking measured registration error. Third, on MyoPS, blocking pathology gradients into registration must outperform the unblocked variant on scar Dice, which would explain the U-MyoPS joint-training failure.
Scope. BraTS is deliberately left out of the main claim. Its container, the whole tumor, has no stable shape across patients, so pooling should not help there. We will run it once as a negative control and expect no gain.
References
- DeepAtlas: Joint Semi-supervised Learning of Image Registration and Segmentation
- Bi-level Learning of Task-Specific Decoders for Joint Registration and One-Shot Medical Image Segmentation
- Learning to Segment Medical Images with Scribble-Supervision Alone
- Scribble-Based Hierarchical Weakly Supervised Learning for Brain Tumor Segmentation
- Aligning Multi-Sequence CMR Towards Fully Automated Myocardial Pathology Segmentation
- Anatomy-Guided Pathology Segmentation
- MyoPS-Net: Myocardial pathology segmentation with flexible combination of multi-sequence CMR images
- MyoPS: A benchmark of myocardial pathology segmentation combining three-sequence cardiac magnetic resonance images
- A Semi-Supervised Framework for Cardiac MRI Segmentation via Multi-Constraint Collaborative Self-Training