Many clinical CT and MRI scans are sharp within each slice but coarse between slices. A slice might be 0.7 mm across but 5 mm thick, a condition called resolution anisotropy. Segmentation models trained at one hospital face two problems at a new one. The scanner changes the noise and contrast, and the thick slices smear anatomy along the depth axis. Small, thin structures such as the adrenal glands and duodenum then break into disconnected fragments across slices. Earlier work has used a scan's own sharp slices as free training data, but only to sharpen images. Our proposal uses them to adapt the segmentation model itself. At test time, on one unlabeled volume, we blur the sharp in-plane slices so they look like the scan's thick direction. We then train the segmenter to label the blurred slice the same way it labels the sharp original. A second term makes predictions from the three orthogonal viewing planes agree in 3D.
The opportunity is that each anisotropic scan carries an internal answer key. Its in-plane slices show this patient's anatomy, with this scanner's noise, at full sharpness. The degraded through-plane views show the same anatomy blurred by a slice profile that can be estimated from the scan itself. If a segmenter learns to be invariant to that specific blur on that specific scan, then it can read the coronal and sagittal reformats, the side views of the stack. Fusing those views restores depth-wise continuity without any source data, any target cohort, or a 3D network that has to fit the full-resolution volume in memory.
Core Claim
Internal reference training — building training pairs from a volume's own sharp plane — can supervise semantic predictions, not just pixels. Doing so on a single test volume reduces both scanner-induced error and slice-direction fragmentation. We expect it to do this more effectively than two-stage "super-resolve then segment" pipelines and than training-time thick-slice augmentation. The new element is the label-level pseudo-pair: the segmenter's prediction on a sharp in-plane slice is the target for its prediction on a version of that slice degraded with the scan's estimated slice profile. This is combined with a cross-plane agreement term computed over the reassembled volume.
Position Relative to Prior Work
The closest work is DA-VSR. It pre-trains a volumetric super-resolution network on lung CT and adapts its shared feature extractor to an unseen target volume by reconstructing synthetically downsampled in-plane slices. On KiTS19 at 4x, this raises PSNR from 37.60 to 38.07 dB. DA-VSR provides our mechanism, the in-plane pseudo-pair, but it optimizes and evaluates only pixel fidelity. It never asks whether the adapted features produce better or more continuous segmentations.
Two other papers share that pixel-level focus. The filter-bank formulation for slices with and without gaps estimates the slice profile with ESPRESO and regresses missing detail coefficients. We reuse its profile estimation to build our degradation operator. The inter-slice SA-INR method fine-tunes on each subject in about one minute and reaches 30.88 dB on SKI10. Its fast per-subject fine-tuning motivates our compute budget, but it also stops at pixel metrics.
Two volumetric adaptation methods motivate the context term, and they also constrain the claim. SDC-UDA shows that stacking 2D domain-adapted predictions causes slice-direction flickering. Its three-slice attention lifts CrossMoDA mean Dice to 84.8, against 33.5 for SIFA. However, it needs a labeled source cohort and the target cohort, and it uses only adjacent axial slices. MAPSeg obtains 3D context through masked autoencoding and global-local features. It reaches 81.2% Dice on MMWHS CT-to-MRI, but it assumes near-isotropic inputs and large unlabeled 3D cohorts. Neither paper exploits the orthogonal sharp plane, and neither works from one volume.
On the single-sample side, Single Image Test-Time Adaptation for Segmentation shows that per-image adaptation is feasible and that an IoU loss beats cross-entropy for pseudo-labeling, by 3.51% on GTA5-C. We adopt that loss. That work, however, is 2D and natural-image only, and its strongest gains needed a source-trained mask refiner.
Roth et al. motivate the problem: 3D smoothing of 2D predictions lifts pancreas Dice from 56% to 68%, yet boundaries stay unresolved. AMOS also motivates it, with a drop of more than 20% Dice on MRI from unseen vendors and weak small-organ results. Finally, shape-prior corrections such as Ouyang et al.'s calibration and ErrorNet assume regular shapes. That is why we avoid a learned shape prior and rely on the scan's own sharp slices instead.
Research Program
We start from a 2D segmentation network $f$ trained on source-site slices. Given one unlabeled target volume $V$ whose depth axis $z$ is coarse, we first estimate its slice profile $h$ with ESPRESO-style estimation, as in the filter-bank paper. We then define a degradation $D_h$ that blurs an in-plane slice $x$ along one in-plane axis with $h$ and resamples it at the through-plane spacing. Applying $D_h$ to a sharp axial slice makes it statistically resemble a coronal or sagittal reformat of this scan.
Adaptation then minimizes the following objective over a small set of parameters $\theta$, namely the normalization layers and the final decoder block:
$$ \mathcal{L}(\theta) = \sum_{x \in \mathcal{A}(V)} \ell_{\text{IoU}}\big(f_\theta(D_h(x)),\ \mathrm{sg}[\,f_\theta(x)\,] \odot m(x)\big) + \lambda \sum_{v \in V} \big\| p^{\text{ax}}_\theta(v) - \bar p_\theta(v) \big\|^2 + \lambda \sum_{v}\sum_{k \in \{\text{cor},\text{sag}\}} \big\| p^{k}_\theta(v) - \bar p_\theta(v)\big\|^2 . $$
In this objective, $\mathcal{A}(V)$ is the set of sharp axial slices. $\mathrm{sg}$ stops gradients so that the sharp-slice prediction acts as a fixed target. $m(x)$ is a confidence mask that keeps only pixels whose sharp-slice prediction has low entropy. $\ell_{\text{IoU}}$ is the soft IoU loss recommended by SITTA. $p^{\text{ax}}, p^{\text{cor}}, p^{\text{sag}}$ are the voxelwise class probabilities obtained by running $f_\theta$ on the axial slices and on the coronal and sagittal reformats and restacking the results. $\bar p$ is their average. $\lambda$ weights the volumetric agreement term.
The first term is the internal reference: it teaches $f$ how this scanner's anatomy looks when blurred by this slice profile. The second term addresses volumetric context loss directly, without a memory-hungry 3D network. The final mask is the fused $\bar p$. Because only a few parameters are updated, we target adaptation in minutes on one GPU. That is well below the 127 minutes reported for filter-bank super-resolution on a V100.
The analysis has three parts. We measure how much the gain grows with the anisotropy ratio. We test whether an estimated profile beats a generic Gaussian blur. We test whether the agreement term by itself, without internal reference pairs, collapses toward trivially smooth masks.
Baselines
The naive reference point is the source-trained model with no adaptation. We run it in two forms: the 2D model on axial slices, and a 3D U-Net in the vanilla, carefully preprocessed style that won KiTS19 (KiTS19 results), applied after trilinear resampling to its training spacing. This 3D U-Net is the strongest supervised competitor. On in-domain AMOS CT it reports 89.04% mean Dice (AMOS), and on KiTS19 the winning entry reports a composite Dice of 0.912. Under cross-vendor MRI shift, AMOS reports a drop of more than 20%. No reported result was found on Lacuna for single-volume test-time adaptation on anisotropic AMOS or KiTS19 targets, so we will produce those numbers ourselves. Beating this baseline means a higher held-out-vendor mean Dice and better small-organ Dice at matched anisotropy, with paired per-case tests.
The second baseline is post-hoc 3D Gaussian smoothing of the stacked 2D probabilities, following Roth et al., with the smoothing width tuned on source validation data. It isolates whether simple spatial regularization accounts for any continuity gain.
The third baseline is super-resolve-then-segment. We first upsample the target with an in-plane self-supervised super-resolution model, either DA-VSR-style backbone adaptation (DA-VSR) or the profile-aware filter-bank method. The frozen segmenter is then applied to the near-isotropic result. This baseline gets the same per-volume wall-clock budget as our method.
The fourth baseline is source-time thick-slice augmentation. Here the source segmenter is trained with random anisotropic blur and resampling drawn from a range of slice profiles, and no test-time step is used.
The fifth baseline is SITTA-style IoU pseudo-label adaptation on axial slices alone (SITTA), with the same parameter subset and iteration count.
Ablations remove each loss term in turn. They also replace the estimated profile $h$ with a generic Gaussian, and degrade a randomly chosen axis instead of matching $z$. We additionally report SDC-UDA and MAPSeg in their published multi-volume settings as references that use more data. They are not treated as competitors under the same budget.
Why not just…?
Why not just super-resolve the volume and run the existing segmenter? This is the most serious threat. Super-resolution uses the same internal reference signal, and Jeong et al. show that thinning slices from 5 mm to 1 mm raises CAD detection AUC from 0.867 to 0.902. However, those reconstructions had no paired ground truth. The super-resolution objective also rewards matching pixels, not matching labels. DA-VSR gains only about 0.5 dB, and pixel accuracy says nothing about boundaries between low-contrast organs such as the pancreas and duodenum. Most importantly, a frozen segmenter applied to super-resolved output still faces the scanner shift. The matched-budget comparison in our experiments settles this question directly. If super-resolve-then-segment equals our method on small-organ Dice, we will report that the semantic pseudo-pair adds nothing.
Why not just augment source training with simulated thick slices? This would give degradation invariance once, at no test-time cost. It also nearly dissolves the novelty if it works. The argument for test-time adaptation is that the target's slice profile, gap, and noise texture are unknown and interact with vendor-specific contrast. The internal reference sees the actual combination rather than a sampled guess. The profile-mismatch ablation and the held-out-vendor MRI split are designed to show whether that matters. If augmentation matches us on vendor-shifted MRI, the honest conclusion is that a broad augmentation range is enough, and we would recommend it instead.
Why not just smooth 2D predictions in 3D, or use a 3D U-Net? Roth et al. found that smoothing helps but leaves boundaries where visceral fat is scarce unresolved. AMOS shows that a strong 3D U-Net still loses more than 20% on unseen MRI vendors and struggles on small organs. Both baselines are run as described above, so the question is answered empirically rather than assumed.
Proposed Experiments
The main testbed is AMOS. It covers 15 abdominal organs, 500 CT and 100 MRI scans, and eight scanner models. Source models are trained on CT plus MRI from a subset of vendors, and targets are MRI from held-out vendors. To obtain ground truth under controlled anisotropy, we take target volumes with thin slices and simulate slice-thickness profiles with FWHM of 2, 4, and 6 mm, with and without gaps, mirroring the OASIS-3 configurations in the filter-bank paper. Original labels are kept at full resolution for evaluation. We also evaluate on natively thick KiTS19 cases, using kidney and tumor Dice against the challenge labels. This tests pathology with irregular shapes, where shape priors fail.
Metrics are per-organ Dice and 95th-percentile Hausdorff distance, with small organs reported separately. We also report a continuity measure: the number of connected components per organ relative to ground truth, and Dice between adjacent predicted slices relative to the same quantity in the ground truth. Wall-clock adaptation time is reported for every method.
The decisive checks are as follows. First, the gain over the no-adaptation baseline should rise with the anisotropy ratio and vanish on near-isotropic inputs. If it does not vanish, something other than the cross-plane reference is responsible. Second, removing the internal-reference term while keeping cross-plane agreement should hurt small organs. Agreement alone can be satisfied by over-smoothing. Third, an estimated profile should beat a generic blur on gap configurations. Fourth, we deliberately report the expected failure mode: structures whose fine detail runs only along $z$ and is therefore absent from the sharp plane. This limitation is noted for SMORE-style methods, and our method should inherit it.
References
- DA-VSR: Domain Adaptable Volumetric Super-Resolution for Medical Images
- Self-Supervised Super-Resolution for Anisotropic MR Images with and Without Slice Gap
- Inter-Slice Super-Resolution of Magnetic Resonance Images by Pre-Training and Self-Supervised Fine-Tuning
- SDC-UDA: Volumetric Unsupervised Domain Adaptation Framework for Slice-Direction Continuous Cross-Modality Medical Image Segmentation
- MAPSeg: Unified Unsupervised Domain Adaptation for Heterogeneous Medical Image Segmentation Based on 3D Masked Autoencoding and Pseudo-Labeling
- Single Image Test-Time Adaptation for Segmentation
- Deep convolutional networks for pancreas segmentation in CT imaging
- AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation
- Improved Post-hoc Probability Calibration for Out-of-Domain MRI Segmentation
- ErrorNet: Learning Error Representations from Limited Data to Improve Vascular Segmentation
- The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challenge
- Deep Learning-Based Slice Thickness Reduction for Computer-Aided Detection of Lung Nodules in Thick-Slice CT