Medical image segmentation needs experts to outline organs and tumors voxel by voxel. That labor is the bottleneck, so the field has turned to scribbles: a few quick strokes inside and along the edge of each structure. The surprising recent finding is that the simplest approach is the best one. A standard 3D U-Net that computes its loss only on scribbled voxels beats every elaborate "shape prior" method once evaluation moves beyond the heart. The remaining gap to full supervision sits mostly at boundaries and thin structures, where the scribbles say nothing. We propose closing part of that gap with a constraint that holds for every shape, rather than a prior about what shapes look like. The network predicts a distance-to-boundary field alongside its mask. Border scribbles must lie exactly on that field's zero level, and the field must change no faster than distance itself. Each labeled voxel then fixes a ball of guaranteed-foreground or guaranteed-background around it, which spreads supervision into unlabeled regions without assuming anything about cardiac anatomy.
The idea borrows a representational stance from visual neuroscience. There, an object's outline and its internal skeleton (medial axis) are encoded as separable shape dimensions. Scribble methods have used neither. In two sentences: treat the mask as the silhouette and a predicted distance field as its medial description, then use scribbles as exact geometric constraints on that field. Because those constraints are true of any solid region, they should transfer across organs and tumors where learned shape priors have collapsed.
Core Claim
Partial-loss scribble training wastes geometric information that scribbles carry. A border scribble is not just a foreground or background label; it is a set of points where the distance to the boundary is zero. An interior scribble point at distance $r$ from the nearest boundary implies that a whole ball of radius $r$ around it is foreground. A network that represents shape as a signed distance field $\phi(x)$ (positive inside, negative outside, magnitude equal to distance to the boundary) can turn these facts into losses on unlabeled voxels. We claim this will add to the strongest existing baseline on the 7-task ScribbleBench. The gain should concentrate on boundary accuracy and tumors. It should not reproduce the cardiac overfitting seen in earlier shape-prior methods, because it adds no anatomical assumption.
Grounding in Prior Work
The motivating evidence comes from the neuroscience side. Shape coding in occipito-temporal cortex relies on object silhouette, curvature, and medial axis used variance partitioning over model dissimilarity matrices to show that silhouette, curvature and medial axis account for separable parts of the fMRI signal, with a medial-axis-specific region in right lateral occipital cortex. We do not claim to model cortex. We take from this work only the claim that outline and skeleton are complementary descriptions worth representing separately. A related finding constrains our design. Material and shape perception based on two types of intensity gradient information shows that stable shape information lives in the ordering of intensities, not their magnitude. This warns us against tying the geometric loss to raw contrast, which varies across CT and MRI. Our constraints are therefore purely geometric and never touch image intensity.
The machine learning side supplies both the method we build on and the negative results that limit what we may try. Gotkowski et al. (Revisiting 3D Medical Scribble Supervision: Benchmarking Beyond Cardiac Segmentation, Embarrassingly Simple Scribble Supervision for 3D Medical Segmentation) built ScribbleBench, which simulates interior and border scribbles across seven CT and MRI tasks. On it, a 3D nnU-Net trained with a partial loss (nnU-Net+pL, which computes cross-entropy only on scribbled voxels) reaches 0.813 mean Dice against 0.856 for full supervision. ShapePU and CycleMix fall to 0.369 and 0.559. Their result is the hard constraint on this proposal: anything we add must at least preserve nnU-Net+pL's performance on every task.
The failed methods show why. CycleMix pulls predictions toward their largest connected component. That assumption is right for a ventricle and wrong for a multifocal liver tumor. ScribFormer relies on a transformer branch to learn shape implicitly. It reaches 0.888 Dice on ACDC but does not transfer beyond the heart. Both encode what anatomy typically looks like. Our losses encode only what any solid region must satisfy, and that is the specific difference we test.
Classical propagation, as in Learning to Segment Medical Images with Scribble-Supervision Alone, expands labels with random walks and conditional random fields. A conditional random field is a smoothing model that favors giving neighboring voxels of similar intensity the same label. Can et al. report that it is hyperparameter-sensitive and leaks across weak edges. Weakly Supervised Lesion Co-Segmentation on Ct Scans reports recall loss from such smoothing. Both failures are intensity-driven, which is what our intensity-free design avoids. Agarwal et al.'s use of RECIST diameter lines as spatial anchors is the closest precedent for reading a sketch as geometry. Those lines are a fixed two-segment skeleton with a single heuristic grab-cut step, whereas we use a general distance field with exact constraints.
Research Program
We add a second output head to the standard 3D nnU-Net, keeping its configuration, augmentation and schedule unchanged. The new head predicts $\phi(x)$ in millimeters. The mask head continues to receive the partial loss. Three terms tie the heads to each other and to the scribbles.
The first term is a zero-level constraint. Every border-scribble voxel $b$ incurs $|\phi(b)|$. Every interior-scribble voxel $x$ incurs a hinge on $\phi(x) < 0$, and every background-scribble voxel incurs a hinge on $\phi > 0$.
The second term is a slope constraint, which is what lets labels propagate. A true distance field changes by at most one millimeter per millimeter. We therefore penalize $\max(0, \|\nabla\phi(x)\| - 1)$ on all voxels, labeled or not. We also add a weak eikonal term pushing $\|\nabla\phi\| \to 1$ away from the predicted boundary. Given the slope constraint, the zero-level constraint implies an upper bound on how far inside any point can be:
$$\phi(x) \le \min_{b \in B} \|x - b\|,$$
where $B$ is the set of border-scribble voxels. We enforce this bound explicitly as a hinge on interior scribble voxels. The bound is what prevents a predicted region from swelling past a known boundary.
The third term couples the two heads. The mask head's foreground probability $p(x)$ is pulled toward $\sigma(k\,\phi(x))$ on unlabeled voxels, where $\sigma$ is the logistic function and $k$ sets the sharpness. This term carries the medial-ball effect. Wherever the field is confidently large at a scribbled interior point, the slope constraint keeps it positive throughout a ball of that radius, and the coupling turns that into foreground supervision for the mask.
Multi-class tasks get one field per class. Tumors inside organs are handled by nesting: the tumor field is constrained to be no larger than the organ field.
One assumption needs checking before anything else. We assume ScribbleBench's simulated border scribbles lie on or within about one voxel of the true boundary. We will measure their actual distance to the dense ground-truth boundary and set a tolerance band in the zero-level loss to match. If the border scribbles are too inexact, the zero-level constraint loses most of its force. In that case the method reduces to interior and background hinges plus the slope constraint, and we will report it as such.
Baselines
The naive reference point is nnU-Net+pL exactly as released with ScribbleBench. It is also the strongest existing method on this benchmark: 0.813 mean Dice over seven tasks, against a full-supervision ceiling of 0.856, both reported in Revisiting 3D Medical Scribble Supervision: Benchmarking Beyond Cardiac Segmentation. We will rerun it ourselves with the same folds, the same simulated scribbles, the same epoch budget and three seeds, rather than relying on the published figure. To count as beating it, our method must raise mean Dice by a margin larger than the seed-to-seed spread, and no individual task may drop by more than one Dice point. The file does not give per-task values or boundary-metric values for this baseline (no reported result found on Lacuna for ScribbleBench Hausdorff or surface distance). We will compute those metrics ourselves for every method.
The strongest simple competitor that adds supervision to unlabeled voxels is pseudo-label self-training on top of nnU-Net+pL. In this procedure we train, predict on the training volumes, keep high-confidence voxels and retrain once, with the total compute matched to our method. If self-training matches us, then any unlabeled-voxel signal is enough and the geometry is unnecessary.
The key ablation isolates the ingredient we claim matters. It keeps the distance head, the slope and eikonal regularizers and the head coupling, but removes every scribble-derived distance constraint, meaning the zero-level loss on border scribbles and the upper-bound hinge. This variant shows whether gains come from multitask regularization alone or from treating scribbles as distance facts. A second ablation keeps the scribble constraints but drops the slope penalty, which cuts propagation into unlabeled voxels. For context we also report the published CycleMix (0.559) and ShapePU (0.369) results on ScribbleBench, and the full-supervision ceiling. We will not retune those methods.
Why not just…?
Why not just use nnU-Net with partial loss and stop? This is the most serious threat, and the benchmark exists because complexity has repeatedly lost to it. Our answer is narrow. Its remaining 4.3-point gap comes from regions the scribbles never touch, and the partial loss by definition sends no gradient there. If our constraints give no measurable gain over the rerun baseline, especially on boundary metrics, then the baseline wins and we will report that.
Why not just self-train on pseudo-labels? Self-training does reach unlabeled voxels, and it may close much of the gap on its own. It is included as a compute-matched baseline for that reason. The difference is that pseudo-labels inherit the model's own boundary errors, while our zero-level constraints come from the annotator. The prediction that distinguishes the two is that our advantage should grow as the number of border scribbles increases. Self-training gains nothing from additional border scribbles beyond their ordinary use in the partial loss.
Why not just add a signed-distance head as a generic auxiliary task? Distance-field heads are a common regularizer in segmentation. Our main ablation is exactly this variant. If it matches the full method, then the representational idea of keeping a separate skeleton-like description is helping, but the claim that scribbles act as geometric constraints is not. That would be a smaller result than the one we propose.
Why not let a transformer learn shape implicitly, or use classical propagation? ScribFormer's learned prior and conditional-random-field propagation have both been tested beyond the heart, and both have documented failures. ScribFormer collapses on ScribbleBench according to Gotkowski et al. Conditional random fields leak and are sensitive to hyperparameters according to Can et al. and Agarwal et al. Both depend on anatomy or intensity, which our losses do not.
Proposed Experiments
The main experiment runs all methods on the seven ScribbleBench tasks: LiTS, BraTS2020, AMOS2022, KiTS2023, WORD, MSCMR and ACDC. We use the benchmark's simulated scribbles and report per-task and mean Dice, 95th-percentile Hausdorff distance and normalized surface distance.
The first decisive check concerns where the gain appears. If the mechanism works as claimed, the improvement over nnU-Net+pL should be larger on surface metrics than on Dice. It should also be largest on the tumor tasks (LiTS, KiTS, BraTS), where earlier shape priors failed worst, falling as low as 0.05 Dice on brain tumors. If instead the gains appear only on the two cardiac tasks, we have rebuilt the overfitting problem.
The second check tests the annotation-bottleneck claim directly. We thin the scribbles to 50% and 25% of their simulated length, separately for interior and border scribbles, and plot Dice against annotation budget for the baseline, self-training and our method. The constraints should matter more as scribbles become sparser. Removing border scribbles specifically should erase most of the advantage.
The third check is a diagnostic on the learned field. For voxels covered by the medial balls our method implies, we measure the fraction that the dense ground truth labels correctly. This tests whether the propagation is as safe as the geometry promises, or whether errors in the predicted field let mistakes spread.
A negative result is decisive in two cases. If the scribble-constraint ablation matches the full model, or if self-training matches it at equal compute, the core claim fails. We would then report that, on this benchmark, the information in scribbles beyond voxel labels is not usable through geometry.
References
- Shape coding in occipito-temporal cortex relies on object silhouette, curvature, and medial axis
- Material and shape perception based on two types of intensity gradient information
- Revisiting 3D Medical Scribble Supervision: Benchmarking Beyond Cardiac Segmentation
- Embarrassingly Simple Scribble Supervision for 3D Medical Segmentation
- CycleMix: A Holistic Strategy for Medical Image Segmentation from Scribble Supervision
- ScribFormer: Transformer Makes CNN Work Better for Scribble-based Medical Image Segmentation
- Learning to Segment Medical Images with Scribble-Supervision Alone
- Weakly Supervised Lesion Co-Segmentation on Ct Scans