Hard-Case Geodesic Augmentation for One-Shot Anatomical Segmentation

When a medical segmentation model has only one or a handful of labelled scans, the usual fix is to invent more training data. Off-the-shelf image generators turn out to be poor at this. They produce clean, typical-looking examples that the model already handles, and they fail to beat simple appearance augmentation. A better source of synthetic anatomy is to warp a labelled scan along smooth, invertible deformations computed from real unlabelled patients. Each synthetic image then stays a valid anatomy, and its label mask moves with it. That method exists and works well. It samples those deformations at random, though, so it inherits the generators' failure mode in milder form: most samples land in regions the network already handles.

We propose to keep the guarantee that every sample is a valid anatomy and to change only which samples are drawn. During training, augmentation is steered toward the deformations the current network segments worst, inside a bounded neighbourhood of the real cohort. In two sentences: build the space of valid anatomical warps from real patients, as prior work does, then repeatedly train on the warps the current segmenter gets most wrong, without straying outside the range of shapes actually seen. The bet is that geometric validity together with targeted difficulty beats both validity alone (random geodesic sampling) and difficulty alone (adversarial but unconstrained warps).

Core Claim and Relation to Prior Work

The method we build on is Anatomical Data Augmentation via Fluid-based Image Registration. It uses LDDMM (large deformation diffeomorphic metric mapping), a registration method in which one image is warped onto another by a smooth, invertible flow. A flow is summarised by an initial "momentum" vector, and following the flow forward from that momentum is called geodesic shooting. The paper computes the momentum that warps a labelled atlas onto each of $K$ unlabelled scans. It then forms random convex combinations of those momenta and shoots them for random durations, including extrapolation past the observed targets. The result is synthetic image–mask pairs. On OAI knee MRI with one labelled scan it reports 88.31% Dice, against 79.94% for Brainstorm and a fully supervised upper bound of 90.01%. This paper supplies our deformation space. Our proposal does not change its geometry. It changes only the rule for choosing samples.

Two other papers motivate the change.

Limitations of Synthetic Data Generation in Specialized Data-Scarce Domains tests StyleGAN2, fine-tuned Stable Diffusion, DreamBooth and IP-Adapter in data-scarce classification. None consistently beats AugMix. Classifiers trained on real data reach 11% to 45% higher recall on synthetic images than on real ones. That paper names this the "canonical trap": synthetic data is too easy. Their test covers only learned image generators. Whether geometrically constrained warps also fall into the trap has not been measured, and we will measure it.

Interpolation for Robust Learning: Data Augmentation on Geodesics supplies the targeting idea. It interpolates between whole class distributions along optimal-transport paths and trains on the point where the classifier's loss is highest. On CIFAR-100 it reports a 16.8% gain in robustness to a standard single-step gradient attack (FGSM) over the best baseline. That work lives in feature space for natural-image classification and has no notion of an invertible anatomical deformation. We transplant its "worst point along the path" principle into the LDDMM momentum space, where every candidate is a valid warped anatomy with an exact label.

Two papers constrain the design.

A GMM based algorithm to generate point-cloud and its application to neuroimaging generates corpus callosum shapes along geodesics on a product manifold of Gaussian-mixture parameters. Its evaluation is only a 1-nearest-neighbour classifier, so it gives us no evidence about deep segmenters.

Geodesic Regression Characterizes 3D Shape Changes in the Female Brain During Menstruation shows two things. Straight-line approximations agree with true geodesics only when deformations are small relative to curvature. Exact geodesic computations are costly. Together these push us to keep the true diffeomorphic shooting and to avoid backpropagating through it.

Research Program

We first reproduce the geodesic subspace from the fluid augmentation paper. One labelled atlas is registered by LDDMM to every unlabelled training scan, which yields momenta $M_1,\dots,M_K$. A candidate warp is indexed by mixing weights $\alpha$ on the simplex and a shooting time $t$. We precompute a large offline bank of candidates, a few thousand per atlas. The bank spans $t$ from interpolation into bounded extrapolation. Precomputing removes any need to differentiate through the shooting equations.

During training, the method scores a random subset of the bank at each epoch with the current segmenter's loss and trains on the highest-loss candidates. A diversity penalty discourages repeatedly selecting the same neighbourhood. To stop hardness-seeking from drifting into implausible anatomy, candidates are restricted to a Riemannian ball around the cohort:

$$ (\alpha^*, t^*) = \arg\max_{(\alpha,t)\in\mathcal{B}} \; \mathcal{L}\big(f_\theta(I_c\circ\phi_{\alpha,t}^{-1}),\, S_c\circ\phi_{\alpha,t}^{-1}\big), \quad \mathcal{B}=\{(\alpha,t): d(\phi_{\alpha,t}, \mathcal{T}) \le \rho\}. $$

Each symbol is as follows:

  • $I_c$ and $S_c$ are the atlas image and its mask.
  • $\phi_{\alpha,t}$ is the diffeomorphism shot from momentum $t\sum_k \alpha_k M_k$.
  • $f_\theta$ is the segmenter and $\mathcal{L}$ its training loss.
  • $d(\cdot,\mathcal{T})$ is the geodesic distance, measured by momentum norm in the LDDMM metric, to the nearest observed target warp.
  • $\rho$ is set to the largest such distance between any two real scans.

The constraint encodes the rule that a synthetic case may be as unusual as the most unusual real patient, but no more. Training mixes selected hard samples with random bank samples and real data in a fixed ratio. Random samples are kept so that coverage does not collapse onto a few adversarial modes.

The second part of the program is diagnostic. We adapt the hardness-gap test from the canonical-trap paper to segmentation. A segmenter trained only on real labelled data is scored on real held-out scans and on each type of synthetic sample. If synthetic samples score clearly higher Dice than real held-out scans, they are "too easy." The prediction is that random geodesic samples show a positive gap, and that hard-selected samples shrink it toward zero without overshooting into a negative gap. A negative gap would signal implausible cases.

Baselines

The naive reference is a 3D U-Net trained on the single labelled atlas, or on $K$ labelled scans in the multi-shot setting, with the standard random affine and intensity augmentation used in the fluid augmentation paper. Every other method must beat it. It also anchors the absolute scale.

The strongest existing method is random geodesic-subspace augmentation from Anatomical Data Augmentation via Fluid-based Image Registration. It is the strongest competitor for two reasons: it uses the same deformation space, and it is the best number in the file on this exact task, 88.31% one-shot Dice on OAI. For LPBA40 there is no reported numerical result on Lacuna (the paper shows it only graphically), so we reproduce it ourselves. For fairness, it uses the identical precomputed bank, the same unlabelled scans, the same segmenter and schedule, the same number of synthetic samples seen per epoch, and the same tuning budget for the mixing ratio. The only difference is how bank entries are chosen. Beating it means a mean Dice improvement over at least five seeds and several choices of atlas, with confidence intervals excluding zero, on both OAI and LPBA40. Since the gap to the 90.01% supervised bound is under two points on OAI, we will report the fraction of that gap closed. We also expect larger, easier-to-detect gains on LPBA40's many small brain structures and in worst-structure Dice.

Three ablations isolate the key ingredient. Removing the constraint set $\mathcal{B}$ tests whether bounding by the cohort's spread matters. Using hard-selected samples without the random portion tests coverage. Selecting hard samples from a bank built with random B-spline free-form deformations, instead of geodesic warps, tests whether geometric validity matters once hardness is targeted.

Why not just…?

Why not just draw more random geodesic samples? This is the most serious threat. If hard samples matter only because they are rare, a larger random budget might find them anyway. Our comparison equalises samples seen per epoch, but we will also run random sampling with two and four times the budget. If the larger random budget matches hard selection, the honest conclusion is that compute, not targeting, was the bottleneck, and we will report it that way.

Why not just use adversarial elastic deformations, which are cheaper? The fluid augmentation paper notes that strong B-spline warps fold or tear tissue, and it reports that geodesic augmentation outperformed B-spline augmentation. Adversarial search makes the problem worse, because it actively seeks extreme warps. The hard-selected B-spline ablation tests this directly. If it matches our method, geometric validity adds nothing beyond hardness, and the contribution shrinks to "hard-example mining for augmentation."

Why not just use self-supervised pretraining instead? Contrastive Pretraining for Echocardiography Segmentation with Limited Data reaches 0.9125 Dice with 5% of labels on EchoNet-Dynamic. However, it relies on more than ten thousand unlabelled videos, and nothing in it guarantees anatomically valid outputs. Pretraining improves the starting features, while augmentation changes the training distribution, so the two are not competitors. We will run both with and without a contrastively pretrained encoder trained on the unlabelled scans. That shows whether the augmentation gain survives a stronger starting point.

Why not just use a diffusion model? The canonical-trap study already found that learned generators do not beat AugMix in data-scarce settings, and that their outputs are easier than real data. Those generators also provide no exact masks for synthetic anatomy. Our hardness-gap diagnostic gives the constructive version of that finding.

Decisive Checks

We evaluate on OAI knee MRI and LPBA40 brain MRI in the one-shot setting, and again with 5 and 10 labelled scans. Metrics are Dice per structure, the 95th-percentile Hausdorff distance, and the hardness gap described above.

The idea is supported if three things hold. Hard-constrained selection beats random geodesic sampling at an equal budget and still at twice the budget. The hard-selected B-spline ablation falls behind. The hardness gap shrinks toward zero for selected samples.

The idea fails if any of three results appears. Hard selection gains nothing over random sampling once the sample budget is equalised. Removing the bound $\rho$ helps, which would mean the constraint does not express what we claim. Gains vanish when a pretrained encoder is added.

Training cost is reported throughout. Bank construction requires $K$ LDDMM registrations plus offline shooting, which is the same order as the baseline. Online selection adds only forward passes.