When a segmentation model trained at one hospital is run on scans from another hospital's scanner, it often fails silently. It produces confident outputs that are wrong, so the usual way of choosing what a radiologist should label next, which is to ask for the cases where the model is least confident, misses exactly the errors that matter. This proposal uses a different signal for what to label, one that does not depend on the model's confidence: whether the predicted 3D shape is anatomically plausible from one slice to the next. Real organs change smoothly along the scan's depth axis. A model that has lost its footing on a new scanner tends to produce boundaries that jump, vanish, or reappear between adjacent slices. We propose scoring every slice of every unlabeled target scan by how far its slice-to-slice boundary change departs from what was observed in the labeled source data. Clinicians then annotate only a few high-scoring slices, spread across many target scans, instead of whole volumes.
The practical stakes are large. The closest existing method asks experts to segment entire 3D volumes of hundreds of slices each. In-domain studies show that spreading sparse slice labels across many volumes beats dense labeling of a few. Whether which slices you pick matters at all is currently an open and somewhat discouraging question. The project is designed so that a negative answer is also informative: it would show where cheap generic recipes are enough and active annotation adds nothing.
Core Claim
Under cross-scanner and cross-protocol shift, a slice's deviation from source-derived through-plane shape statistics predicts that slice's true segmentation error better than foreground entropy or ensemble variance. This holds most clearly for confidently wrong predictions. Selecting sparse target slices by this score, at a fixed budget of annotated slices, should close more of the remaining domain gap than three alternatives: fixed-interval slice selection, entropy-based slice selection, and full-volume active selection. All of these run on top of the strongest unsupervised recipe, which combines heavy augmentation with entropy minimization.
The score is simple. For organ $k$ in a target volume with slice spacing $\Delta$, let $d_z$ be the symmetric boundary distance in millimetres between the predicted contours on slices $z$ and $z+1$, and let $r_z \in [0,1]$ be the relative position of slice $z$ within the predicted organ extent. From source ground truth we fit a density $p_{src}(d \mid k, r, \Delta)$ of adjacent-slice boundary change. Each target slice is then scored as
$$s_z = \sum_k -\log p_{src}(d_z \mid k, r_z, \Delta) \;+\; \lambda \cdot \mathbb{1}[\text{a connected component of organ } k \text{ appears or vanishes at } z \text{ away from the organ's ends}],$$
where $\lambda$ weights topological breaks. Fitting the density conditioned on $\Delta$ is what lets the score cope with strongly anisotropic data. Thick-slice cardiac MRI and thin-slice CT get different expected displacements, which addresses the anisotropy weakness noted for 2.5D models such as CSAM and Inter-slice Context Residual Learning. The score needs one forward pass per volume. That matters because Bayesian acquisition with interacting ensembles, as in Diminishing Uncertainty Within the Training Pool, or ASSFT's pool-wide voxel margins and embeddings, are acknowledged to be expensive.
How This Relates to Prior Work
Two lines of work motivate the signal. Deep convolutional networks for pancreas segmentation in CT imaging showed that slice-wise predictions fragment along the depth axis. That paper treats the fragmentation as a defect to remove; we treat it as a measurement. Under shift, Efficient Bayesian Uncertainty Estimation for nnU-Net found confident boundary errors on unseen M&Ms vendors, and Improved post-hoc probability calibration for out-of-domain MRI segmentation recalibrated outputs using a shape autoencoder. That calibration work is 2D-only, limited to regular cardiac shapes, and was never used to choose what to label. Our score is a shape check along the depth axis and is fed directly into acquisition. Unlike a learned whole-shape prior, it needs only local boundary statistics, so it should extend to irregular organs such as the pancreas, though this has to be tested.
Two papers constrain the design. How to select slices for annotation found, in single-centre data, that masked-autoencoder-based slice selection did not beat random or fixed-interval selection, and that 3D models trained only on very sparse slices ($p<0.05$) failed. We respond in three ways. We select across the whole target pool rather than within a single volume. We use a model-error signal rather than feature diversity. Most importantly, the 3D model keeps its dense source supervision, so sparse target slices only have to correct it. M3DA constrains us further. Heavy augmentation plus entropy minimization closes about 62% of the gap on average, so every active method here must be layered on that recipe, not compared against a weak baseline.
The closest competitor is ASSFT. It adapts a 3D medical foundation model by querying whole target volumes, ranked by foreground-masked entropy and by how far the adapting encoder's features drift from the frozen pre-trained encoder. We borrow its foreground focus but change two things: the query unit becomes a slice, and the signal becomes through-plane shape plausibility instead of entropy. PixelPick supports the bet that sparse, spread-out labels can go far, reaching 96% of full supervision with 0.06% of pixels on CamVid. FairAP does active domain adaptation on 2D images with CLIP features and has no notion of depth.
Research Program
The base model is a 3D nnU-Net trained on labeled source data with nnU-Net's augmentation. Adaptation runs in rounds. In each round the model segments all unlabeled target volumes and computes $s_z$ for every slice. It then selects the top slices under two constraints: no more than $m$ slices per volume per round, which follows the in-domain finding that spreading labels across volumes helps, and a minimum gap between selected slices of the same volume. The selected slices receive ground-truth masks. These come from existing dense labels, so the simulation is exact.
Training continues on dense source volumes, target volumes with a loss applied only on labeled slices, and entropy minimization on unlabeled target voxels. The design guards against two failure modes. Imaging artifacts can make a slice implausible for reasons no label will fix. Following the observation in Peng et al. that extreme-uncertainty samples are often unlearnable, we report results with and without excluding the top percentile of scores. Separately, a prediction that misses an organ completely but smoothly has low $s_z$. For that reason we also test a combined score that adds foreground entropy, and we report which component carries the gains on each shift.
Before any active learning loop is run, we perform a cheap diagnostic. For each target slice we compute the true per-slice Dice error. We then measure how well each candidate score ranks slices with high error, and in particular the confidently wrong slices where foreground entropy is low but error is high. If $s_z$ does not beat entropy and checkpoint-ensemble variance on this ranking, the selection hypothesis is unlikely to hold and the project pivots to characterizing where augmentation saturates (Gap 4).
Baselines
The do-nothing reference is the source-trained 3D nnU-Net with nnU-Net augmentation, evaluated on target. Directly above it is the MinEnt plus nnAugm recipe from M3DA, which uses no target labels and is the strongest unsupervised method in that benchmark. On the MRI-to-CT shift M3DA reports a best UDA Dice of 0.388, up from a baseline of 0.032; on CT to low-dose CT it reports 96% gap closure. Any active method must be judged on how much of the remaining gap it closes above this level.
The strongest simple active competitor is fixed-interval and uniform-random slice selection spread across target volumes, with the same per-volume cap. This is the strategy that matched learned selection in How to select slices for annotation. The strongest existing method is ASSFT. It reports an AMOS2022-CT Dice of 80.51 at a 5% volume budget, against 73.31 for random volume selection. That number comes from a foundation-model backbone and a different protocol. We will rerun ASSFT's two selection criteria on our nnU-Net with our training loop and will not compare against the reported figure directly.
Every method gets the same source data, the same target pool, the same training schedule and compute per round, and the same budget measured in annotated slices. For ASSFT, a 5% volume budget is converted into the equivalent total number of slices. We also report an annotation-time proxy: foreground slices are weighted by contour length, so that choosing hard slices is not counted as free.
Two ablations isolate the key ingredient. The first is foreground-masked entropy computed per slice and run through the identical sparse-slice pipeline. The second is per-slice variance from cyclical-learning-rate checkpoint ensembles, following Efficient Bayesian Uncertainty Estimation for nnU-Net. Both change only the score. Beating them shows that the through-plane signal itself, and not merely slice-level querying, drives any gain.
The proposal counts as beating the baselines if three conditions hold at two or more budgets on at least two of the three shift families: it closes a statistically significant additional fraction of the remaining gap over fixed-interval selection; it matches or beats ASSFT at an equal slice budget; and it beats both uncertainty ablations.
Why not just…?
Why not just use heavy augmentation and self-training, with no labels at all? On mild shifts this may be enough. M3DA reports 96% gap closure on CT to low-dose CT, and we expect active labeling to add little there. That outcome is part of the map we want to draw. On severe shifts the same recipe leaves most of the gap open, reaching 0.388 Dice from MRI to CT. AMOS reports a drop of more than 20% on unseen MRI scanners. Because the recipe is a component of every arm, the experiment directly measures what labels add on top of it, shift by shift.
Why not just label slices at fixed intervals across many volumes? This is the real threat, given the in-domain null result in How to select slices for annotation. Our reason to expect a difference is that errors under shift cluster. M&Ms names the basal and apical cardiac slices as the persistent failures, and fixed intervals sample mostly easy mid-organ slices. The diagnostic ranking test and the head-to-head comparison at matched budgets settle the question. If fixed-interval selection ties, we will report that as the finding.
Why not just pick slices by foreground entropy or ensemble disagreement? These are the obvious substitutes, and they are cheap to run per slice. The evidence that they fall short under shift is indirect: models that are confidently wrong on new scanners, as reported in the calibration and nnU-Net uncertainty papers. The two ablations test this directly. If entropy matches $s_z$, the depth-axis signal is redundant and the contribution reduces to slice-level querying.
Why not just query whole volumes as ASSFT does? Whole volumes cost hundreds of slices each, and dense labels within one volume are largely redundant. At a matched slice budget, ASSFT can cover only a few target scans. The matched-budget comparison shows whether that coverage gap matters.
Proposed Experiments
The shifts come from three families. The first is the M3DA abdominal shifts on AMOS, including CT to MRI, MRI to CT, and CT to low-dose CT. The second is M&Ms cardiac cine MRI, trained on vendors A and B and adapted to C and D. Its roughly 10 mm slices stress the anisotropy handling. The third is a held-out-scanner split within AMOS MRI. Budgets range from 0.5% to 10% of target slices.
Metrics are per-organ Dice; the fraction of the baseline-to-oracle gap closed, as defined in M3DA; and for slices, the ranking of high-error and confidently wrong slices. We also report HD95, a boundary-distance error metric that How to select slices for annotation omitted, and through-plane boundary smoothness on the final predictions.
Three checks decide the outcome. Does $s_z$ identify confidently wrong slices better than entropy on M&Ms vendors C and D, particularly at the basal and apical slices? Does it beat fixed-interval selection at matched budgets on MRI to CT, where unsupervised adaptation leaves the most room? Does the ranking of methods remain stable when the budget is measured in contour-weighted annotation cost rather than slice count? A positive answer to all three would establish depth-axis implausibility as a cheap, calibration-free acquisition signal. A negative answer on the second check, once the first has passed, would mean that knowing where errors are does not translate into better labels. That would be a useful correction to how active domain adaptation is currently argued.
References
- CSAM: A 2.5D Cross-Slice Attention Module for Anisotropic Volumetric Medical Image Segmentation
- Inter-Slice Context Residual Learning for 3D Medical Image Segmentation
- Diminishing Uncertainty within the Training Pool: Active Learning for Medical Image Segmentation
- Deep convolutional networks for pancreas segmentation in CT imaging
- Efficient Bayesian Uncertainty Estimation for nnU-Net
- Improved Post-hoc Probability Calibration for Out-of-Domain MRI Segmentation
- How to select slices for annotation to train best-performing deep learning segmentation models for cross-sectional medical images?
- M3DA: Benchmark for Unsupervised Domain Adaptation in 3D Medical Image Segmentation
- Adapting Medical Vision Foundation Models for Volumetric Medical Image Segmentation via Active Learning and Selective Semi-supervised Fine-tuning
- All you need are a few pixels: semantic segmentation with PixelPick
- Toward Fair and Accurate Cross-Domain Medical Image Segmentation: A VLM-Driven Active Domain Adaptation Paradigm
- Privacy-Preserving and Uncertainty-Aware Federated Trajectory Prediction for Connected Autonomous Vehicles
- AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation
- Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation: The M&Ms Challenge