Medical segmentation models are expensive to train because only specialists can draw the masks. So most label-efficient methods learn from cheaper marks: a few scribbles inside and outside an organ, or the two diameter lines radiologists already draw on tumors. These methods share one habit. They throw away the pixels they are unsure about, and those are almost always the pixels on the boundary. Image forensics, the field that finds edited regions in photos, has shown that this habit is costly. When a model learns to separate tampered from authentic patches, the patches that straddle the seam carry most of the useful signal about where the edge is. Dropping them makes the boundaries worse (NCL).
We propose to bring that lesson into medicine without importing its dependence on full masks. The idea is simple. Weak medical annotations already prove that certain patches contain both lesion and non-lesion tissue. Any patch that contains a foreground scribble pixel and a background scribble pixel must hold the boundary. So must any patch centred on an endpoint of a tumor diameter line, since those endpoints sit on the lesion edge by clinical definition. These "certified mixed patches" can be found from the annotation geometry alone, before the network has learned anything. That lets us train on boundary regions from the first step, and it avoids the trap in which a model's own early confidence decides which pixels count.
Core Claim
The weak supervision used in clinical practice carries free, label-certified boundary information. Feeding that information into a contrastive objective that lets mixed patches act as both positives and negatives will tighten boundaries, measured by 95th-percentile Hausdorff distance (HD95), the near-worst-case distance between predicted and true contours. It will also do better than methods that filter by uncertainty, and the advantage should be largest in the regime where those methods are known to fail, namely when early predictions are poor. What is new is not the role-switching loss itself but the source of the mixed patches. In forensics, they come from ground-truth masks. Here, they come from guarantees implied by sparse annotations.
Relation to Prior Work
Two lines of work motivate the proposal. In forensics, NCL sorts patches into tampered, authentic and contour classes. It uses a two-branch "pivot" design in which a contour patch serves as a hard positive against authentic patches in one branch and a hard negative against tampered patches in the other, with a consistency loss tying the two branches together. It reports 83.1% F1 on NIST16. However, NCL defines contour patches from full manipulation masks, so it is not a weakly supervised method at all. We reuse its loss but remove that requirement.
In medicine, PCLMix trains from scribbles on cardiac MRI with pixel-level contrastive learning. It selects anchor pixels by low predictive entropy and masks out ambiguous ones. Its authors report that when initial confidence is poor, noisy pixels pass the threshold and corrupt the representation. Learning to Segment Medical Images with Scribble-Supervision Alone likewise zeroes out uncertain labels using Monte Carlo Dropout, where predictions are repeated with random units switched off and their spread is read as uncertainty. Together these papers motivate the work: both document that boundaries are where uncertainty concentrates, and both respond by discarding them.
The research file names the tension directly. Medicine filters transition pixels to avoid confirmation loops, while forensics mines them. Our claim is that certification resolves this tension, because a certified mixed patch is ambiguous only about where inside it the edge lies, never about whether it contains one.
Two papers constrain the design. Weakly Supervised Lesion Co-Segmentation on CT Scans turns RECIST lines (the standard diameter measurements radiologists draw on tumors) into GrabCut pseudo-masks and reaches 90.3% Dice on DeepLesion. Their ablations also show that dense CRF post-processing (a pixel-smoothing step that pulls labels toward image edges) sharpens boundaries at an unacceptable cost in recall. Can et al. report that CRF refinement is highly sensitive to its settings. Both results argue against relying on post-hoc boundary cleanup, so we put the boundary signal into training instead.
Research Program
We first construct the patches. For scribbles, we slide a window at several scales and certify any window that contains pixels from at least two different scribble classes. We label it mixed for that class pair, which supports multi-class problems such as left ventricle, right ventricle and myocardium. For RECIST, the four diameter endpoints sit on the boundary by definition. Windows centred on them are certified mixed. Windows fully inside the convex hull of the two lines, shrunk by a margin, are certified lesion. Windows well outside the bounding extent are certified background. Formally, a window $W$ is certified mixed for classes $a$ and $b$ when
$$ W \cap S_a \neq \emptyset \;\;\text{and}\;\; W \cap S_b \neq \emptyset, $$
where $S_a$ and $S_b$ are the annotated pixel sets for those classes.
We then attach a projection head to a standard U-Net encoder. We add NCL's pivot loss over certified pure and certified mixed patch embeddings, and keep a partial cross-entropy on scribble pixels. We keep the backbone deliberately plain, without heterogeneous dual decoders, so that any gain can be attributed to the patch source. A second variant, used only as an ablation, adds model-proposed mixed patches once predictions stabilize. This tests whether certification still matters after the model becomes reliable.
We will also characterize the certified patches. For each scale we measure their coverage of the true boundary and their purity, meaning the fraction of certified pure patches that truly contain no edge. Both are measured against full masks held out for analysis only. We compare these against the purity of PCLMix's entropy-selected anchors at each training epoch. This directly tests the mechanism behind the claim: certified patches should stay correct early in training, while entropy selection should not.
Baselines
The naive reference point is a U-Net trained with partial cross-entropy on scribble pixels only. On DeepLesion, the equivalent is a U-Net trained on GrabCut pseudo-masks built from RECIST lines. This shows how much any added objective buys.
The strongest scribble competitor on ACDC is Can et al.'s random-walk seed expansion with Monte Carlo Dropout filtering and iterative relabeling. They report 88.0% Dice on ACDC against 91.2% with full supervision, and 78.1% Dice on NCI-ISBI prostate. We will also run PCLMix, since it is the closest contrastive method. The file states only its relative numbers: Dice 1.1% below full supervision and HD95 41.4% lower than full supervision. No absolute PCLMix Dice value was found on Lacuna, so we will reproduce it ourselves.
On DeepLesion the strongest method is the Siamese co-segmentation model of Agarwal et al., at 90.3% mean Dice on their 1,000 manually masked lesions. We will evaluate on that same masked subset.
All methods will share our data splits, scribble sets, backbone width, training epochs and a matched hyperparameter search of equal size. Beating a baseline means exceeding its reproduced Dice while reducing HD95. Equal Dice combined with a significantly lower HD95, measured over three seeds, also counts, since boundary quality is the claim.
The key ablation swaps certified mixed patches for entropy-selected mixed patches and leaves everything else unchanged. A second ablation drops mixed patches entirely and keeps the contrastive loss on pure patches. This is our method with PCLMix's habit restored.
Why not just…?
Why not just lower PCLMix's entropy threshold so boundary pixels are kept? Lowering the threshold admits more boundary pixels, but they are chosen by the same unreliable early predictions that PCLMix's authors identify as the failure mode. Our entropy-versus-certified ablation and the per-epoch purity curves test this directly. If a tuned threshold matches certified patches, we will say so.
Why not just use a multiple-instance constraint on mixed patches, requiring each certified window to predict at least one pixel of each class? This is the most serious threat, because it uses the same certification and is simpler than a contrastive pivot. We will run it as a full baseline. If it matches the pivot loss, the contribution narrows to the certification construction, which still stands on its own. We expect the pivot to help most on diffuse edges, where a hard "at least one pixel" constraint is satisfied trivially, but this is a hypothesis to test.
Why not just refine coarse predictions with a dense CRF? Agarwal et al. found that dense CRF refinement causes sharp recall loss on CT lesions, and Can et al. found CRFs fragile to their settings. We will still report the naive baseline plus a CRF so readers can see this on our splits.
Why not just train on GrabCut or random-walk pseudo-masks? Both cited medical papers needed substantial extra machinery on top of such masks to reach their reported numbers, which indicates the masks alone are noisy. The naive baseline quantifies this.
Proposed Experiments
The main experiments use ACDC and NCI-ISBI 2013 with scribble supervision, and DeepLesion with RECIST supervision. We report Dice, HD95, and a boundary F-score within a 2-pixel band.
The first decisive check is a low-supervision sweep. We thin the scribbles to 50% and 25% of their length and shorten the training warm-up. Our claim predicts that the gap over PCLMix and Can et al. widens as initial predictions get worse. If the gap shrinks instead, the certification argument fails.
The second check is the purity analysis described above. Certified pure-patch error should remain near zero throughout training, while entropy-selected anchor error should be high early on.
The third check tests whether the gain is specific to medical tissue. On NIST16 and CASIA we simulate scribble-like sparse annotations by sampling strokes from the true masks. Running our method there tells us whether it recovers a meaningful share of NCL's full-mask 83.1% F1 on NIST16. The diffuse-boundary hypothesis predicts a smaller pivot-over-multiple-instance advantage on sharp forensic seams than on infiltrating lesion edges. Observing that pattern would support the claimed mechanism, and its absence would weaken it.
References
- Pre-training-free Image Manipulation Localization through Non-Mutually Exclusive Contrastive Learning
- PCLMix: Weakly Supervised Medical Image Segmentation via Pixel-Level Contrastive Learning and Dynamic Mix Augmentation
- Learning to Segment Medical Images with Scribble-Supervision Alone
- Weakly Supervised Lesion Co-Segmentation on Ct Scans