Doctors rarely have time to outline every organ in a 3D scan, so a practical alternative is to let them draw a few quick strokes, called scribbles, inside and around each structure and train a segmentation network on those strokes alone. The best current recipe is plain. It trains a standard network only on the scribbled voxels and ignores everything else, and it comes within about four Dice points of full outlining. That gap, however, is not spread evenly. It sits where the scribbles stop and the image stops helping too: the faint border where the pancreas meets surrounding fat, or the adrenal gland pressed against neighbouring tissue. It also sits on scans from MRI machines the network has never seen. In both places, nothing in the training signal tells the network where the boundary should be.
We propose to fill that silence with anatomy. From a small set of fully outlined CT scans, we learn a smooth, non-linear shape model for each organ: a template that can bend, twist, and swell, but cannot tear, fold over itself, or sprout disconnected pieces. A model of this kind, with a smooth inverse and no tearing, is called diffeomorphic. While a scribble-supervised network trains on a new modality or scanner, the warped template supplies labels for unscribbled voxels. It does this only where the network itself is uncertain. Scribbles say where the organ is, and the shape model says how far it can plausibly extend. Organ shape depends on the patient and not on the scanner, so a prior learned on CT should still apply on MRI, where appearance-based cues fail.
Core Claim
Adding a non-linear, topology-preserving shape prior to scribble supervision should reduce boundary error on low-contrast structures and on unseen MRI scanners, compared with the plain partial-loss baseline. It should also leave high-contrast structures no worse, provided the prior acts only on unannotated voxels where the network is uncertain. The prediction can be falsified directly. The gain over the baseline should grow as local boundary contrast falls. If the gain is flat across contrast levels, or negative on easy organs, the idea is wrong.
What Prior Work Establishes
The baseline we must beat, and the main constraint on the design, comes from Embarrassingly Simple Scribble Supervision for 3D Medical Segmentation. It benchmarks scribble methods on seven 3D tasks (ScribbleBench) and finds that nnU-Net with a partial loss reaches a mean Dice of 0.813 against 0.856 for dense supervision. Elaborate regularizers collapse outside cardiac MRI: ShapePU averages 0.369 and CycleMix 0.559, and they fall to 0.05 on brain tumours and 0.36 on abdominal organs. This is the result most likely to contradict us, since a shape-aware method is exactly what failed. We take it as a design rule rather than a refutation. The prior must never override scribbles or confident predictions, so in the worst case the method reduces to the partial-loss baseline. We also evaluate on all seven tasks, so a collapse cannot hide.
Two scribble methods show the failure we target. Learning to Segment Medical Images with Scribble-Supervision Alone grows scribbles into labels using random walks and intensity-based conditional random fields. It loses 2.9 Dice on cardiac MRI and 4.5 on prostate, and its intensity cues leak across low-contrast borders. Scribble-based Domain Adaptation via Co-segmentation transfers from dense T1-contrast MRI to scribble-annotated T2 MRI through a graph of appearance and deep-feature affinities, and reaches 83.4% Dice on vestibular schwannoma. It has no geometric prior, and it was tested on a single focal tumour, not on deforming abdominal organs. Our proposal replaces appearance affinities with shape constraints in exactly the regions where appearance is uninformative.
The method comes from the shape-modelling literature. Landmark-Free Statistical Shape Modeling Via Neural Flow Deformations (FlowSSM) represents shapes by deforming a template along a learned continuous velocity field. Its average surface error is 0.24 mm on femurs, and it produces zero self-intersecting meshes on liver, where a linear point-distribution model produced 871. FlowSSM, however, requires clean surfaces and never touches raw images. LEDA shows that such deformations can be encoded with exact inverse consistency in about 0.02 seconds per pair, which makes it cheap to repeat inside a training loop. From Images to Probabilistic Anatomical Shapes: A Deep Variational Bottleneck Approach supports the claim that passing images through a compact shape code discards artifacts and low-contrast noise. Its uncertainty correlates with error at 0.782, against 0.278 for a linear baseline, with gains at 10–20% of training data. It predicts landmarks from images, though, and does not mix shape with sparse supervision.
The application setting comes from AMOS. AMOS contains 500 CT and 100 MRI scans of 15 abdominal organs, collected on eight scanners. On scanners unseen during training, MRI performance drops by more than 20%, and small low-contrast organs such as the adrenals and duodenum remain poorly segmented by every model it tests. Boundary-focused architectures such as DB-MSMUNet address pancreatic ambiguity with deformable convolutions and edge decoders and reach 89.47% Dice on NIH Pancreas. They do so with dense labels, and an architecture alone cannot supply information that the labels lack.
Research Program
First, we learn a shape prior for each organ. A FlowSSM-style model is fitted to dense masks from a small CT seed cohort: an organ template plus a low-dimensional latent code that drives a neural velocity field, so that each organ shape is a smooth warp of the template. We start with 50 CT scans, which mirrors the AMOS seed annotation stage. Training costs are modest; FlowSSM reports 2–4 hours on cohorts of this size.
Second, we train on the target domain with scribbles. An nnU-Net backbone receives the partial loss on scribbled voxels. A light shape head on the encoder predicts a pose and a latent code, and the frozen flow model warps the template into a full shape mask $T$. The head is supervised by the scribbles, and by the network's own confident voxels after a warm-up period. On unscribbled voxels $v$, the network output $p_v$ is pulled toward $T_v$, weighted by the network's predictive entropy $H_v$ computed with test-time augmentation:
$$ \mathcal{L} = \mathcal{L}_{\text{pL}} + \lambda \sum_{v \notin S} g(H_v)\,\mathrm{CE}(p_v, T_v), $$
Here $S$ is the set of scribbled voxels, $g$ is a monotone gate that is zero below an entropy threshold, $\mathrm{CE}$ is cross-entropy, and $\lambda$ is the prior weight. Because confident voxels and scribbles are never overridden, the method falls back to the baseline whenever the network is already sure. The prior shapes the output only where the image is ambiguous.
Third, we analyze where the prior helps. We bin ground-truth boundary voxels by local intensity contrast across the boundary, and we report the gain for each bin, separately for each organ and each scanner.
Baselines
The naive reference point is a nnU-Net trained densely on CT and applied directly to MRI with no target labels. It shows how much any target supervision buys. AMOS reports a drop of more than 20% on unseen MRI scanners but gives no per-organ value for this exact setting, so we measure it ourselves.
The strongest competitor is nnU-Net with the partial loss from Embarrassingly Simple Scribble Supervision, which reports a mean Dice of 0.813 on ScribbleBench. It is strongest because it is simple, robust across seven tasks, and has defeated every more elaborate method so far. We run it in two forms. One uses target scribbles only. The other is the more dangerous version: joint training on dense source CT plus target scribbles, with the partial loss applied to the scribbles. Both forms use the same nnU-Net configuration, the same simulated scribbles, the same epochs, and the same hyperparameter-search budget as our method. Beating it means a statistically significant improvement, paired across cases, in mean Dice and in 95th-percentile Hausdorff distance on the AMOS low-contrast organs (pancreas, adrenals, duodenum) and on held-out MRI scanners, with no significant loss on any ScribbleBench task. ScribbleBench reports no per-organ partial-loss result for AMOS organs, so we will compute those numbers ourselves. Dense supervision, with a mean of 0.856 on ScribbleBench, serves as the upper bound.
We also include the published shape-prior scribble method ShapePU (0.369 on ScribbleBench) and the co-segmentation domain adaptation of Scribble-based Domain Adaptation via Co-segmentation, reimplemented on AMOS CT-to-MRI. The latter isolates whether geometric constraints beat appearance affinities.
Three ablations isolate the key ingredients. Replacing the flow model with a linear PCA shape model tests whether non-linearity matters. Removing the entropy gate, so the prior acts on every unannotated voxel, tests whether gating is what prevents a ShapePU-style collapse. Fitting the shape model to the baseline's output after training, rather than during it, tests whether the prior must shape learning or only clean up predictions.
Why not just…?
Why not just use nnU-Net with the partial loss and accept the four-point gap? Four points is an average over seven tasks, many of them high-contrast. Our hypothesis is that the gap is concentrated on faint boundaries and unseen scanners, where it matters clinically. The contrast-stratified analysis settles this. If the baseline's deficit is flat across contrast levels, there is nothing for a shape prior to fix, and we would report that.
Why not just train jointly on dense CT and scribbled MRI? This is the real threat. Dense CT labels already carry implicit shape information that a shared network might transfer without any explicit prior. The difference is that joint training transfers shape entangled with CT appearance, which AMOS shows does not survive scanner changes. Our prior is appearance-free. The comparison on held-out MRI scanners is decisive: if joint training matches us there, the explicit prior is unnecessary.
Why not just fit a shape model to the network's output afterward? Post-hoc fitting is cheaper, and it fixes topology. It cannot, however, correct errors the network has learned to make confidently, such as a systematic spill into the duodenum. It also discards correct detail that falls outside the shape manifold. The post-hoc ablation measures this directly. If it matches in-training use, we would recommend the simpler pipeline.
Proposed Experiments
The main experiment uses AMOS. Shape priors come from 50 dense CT scans. Target training uses simulated scribbles on MRI from a subset of scanners, and evaluation uses MRI from held-out scanners in a leave-one-scanner-out protocol, with dense labels used for scoring only. Metrics are per-organ Dice, 95th-percentile Hausdorff distance, and topology errors, counted as extra connected components and holes. The second experiment reruns the full seven-task ScribbleBench with identical budgets, to verify that the method does not collapse where ShapePU did. The third evaluates pancreas segmentation on NIH Pancreas and MSD Pancreas with scribbles only, as a single-organ low-contrast test, using DB-MSMUNet's dense 89.47% and 87.59% as reference ceilings rather than as competitors.
The decisive check is the contrast-stratified gain curve on held-out-scanner MRI. The proposal succeeds if the improvement over joint-trained partial loss rises as boundary contrast falls, and if it is zero or positive on the highest-contrast bins. It fails if the curve is flat, or if the linear-PCA ablation performs as well as the diffeomorphic prior. The latter outcome would mean non-linear deformation modelling contributes nothing here.
References
- Embarrassingly Simple Scribble Supervision for 3D Medical Segmentation
- Learning to Segment Medical Images with Scribble-Supervision Alone
- Scribble-Based Domain Adaptation via Co-segmentation
- Landmark-free Statistical Shape Modeling via Neural Flow Deformations
- LEDA: Log-Euclidean Diffeomorphism Autoencoder for Efficient Statistical Analysis of Diffeomorphisms
- From Images to Probabilistic Anatomical Shapes: A Deep Variational Bottleneck Approach
- AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation
- DB-MSMUNet: Dual Branch Multi-Scale Mamba UNet for Pancreatic CT Scans Segmentation