Hospitals receive certified medical AI models that are effectively frozen. Changing their weights counts as a new device, and in practice almost none are retrained after clearance. Yet these models meet scans that differ from their training data, most visibly in resolution: thick slices, different scanners, different slice profiles. We propose to leave the certified model untouched and instead put a small super-resolution network in front of it. Super-resolution means a network that sharpens a coarse image into a finer one. This front-end is adapted separately for each new scan, needs no labels from the new hospital, and is optimized for one thing: the frozen model should behave on the sharpened thick-slice scan the way it would behave on a truly thin-slice scan.
The trick that makes this label-free is that clinical volumes are anisotropic. They are sharp within each slice and blurry between slices. Within a single target scan, we can therefore take its sharp in-plane direction, blur it so it looks like the between-slice direction, sharpen it back with the front-end, and ask that the frozen model's output on the restored version match its output on the untouched original. The frozen model's output on genuine sharp data serves as the training target. This turns the per-scan self-supervision used by existing medical super-resolution methods into a task-driven signal. It also turns the synthetic-to-real gap into something we can measure directly: the question is whether the simulated blur we apply resembles the real slice blur the scanner produced.
Core Claim
We claim that a per-scan super-resolution adapter trained on a frozen segmenter's own output consistency recovers more of the cross-site segmentation drop than two alternatives: per-scan adaptation driven by pixel fidelity, and interpolation. We further claim that how well the simulated degradation matches the real slice degradation, rather than the capacity of the adapter, determines how much of the gain survives on real thick-slice acquisitions. The first claim is about which loss to adapt with. The second is about where the synthetic-to-real gap actually bites.
Where This Sits in Prior Work
Three methods motivate the proposal by showing pieces of it separately.
DA-VSR adapts a volumetric super-resolution backbone at test time by downsampling high-resolution in-plane slices and reconstructing them. It reports 38.07 dB PSNR versus 37.60 dB unadapted at 4x on KiTS19 kidney CT. Its adaptation target is purely pixel fidelity, so we cannot tell whether it helps or hurts a downstream model. The filter-bank self-supervised method trains entirely within each MRI scan, estimating the slice profile from in-plane data. It also optimizes only reconstruction, and it costs 127 minutes per volume at the hardest setting. We reuse the per-scan proxy idea from both papers and replace their objective.
Task-driven restoration supplies the objective. SR4IR uses the downstream network's feature space as the super-resolution loss and reports +3.6 mIoU on 4x segmentation, but it alternately trains the task network and relies on paired data from natural images. HQG-Net penalizes an enhancer when a downstream segmenter degrades (+6.3% Dice on nerve fibers), but it does 2D contrast enhancement, not resolution recovery, and it needs a trainable task model. Both papers also warn that task-optimized outputs can have lower PSNR. This is a constraint for us, because clinicians will look at the images, so we keep a pixel term and report fidelity.
The regulatory framing comes from two papers. Regulating AI Adaptation finds that only 6 of 416 cleared devices were retrained within two years. It also finds that site-specific retraining of a chest X-ray model regained up to 0.23 AUC locally but cost an average of 0.176 AUC on the original site. InputAdapter shows that a trainable front-end driven by a frozen predictor's loss can beat fine-tuning on ICU time series, with +14.1 AUCPR on acute kidney injury. It needs target labels, however, and it handles only 1D data.
Our proposal differs from InputAdapter in being label-free and volumetric. It differs from DA-VSR and the filter-bank method in what the adaptation optimizes. It differs from SR4IR and HQG-Net in never touching the task model.
SR-MPMGen is the closest physics-aware competitor. It maps brain MRI to quantitative tissue maps with a Bloch-equation loop and point-spread-function matching, which improves out-of-distribution tumor classification from 0.692 to 0.753 ROC-AUC. It is decoupled from task gradients and depends on sequence metadata and closed-form physics. We take from it the lesson that the degradation model matters, but we estimate that model from the scan rather than from acquisition parameters.
Method
Let $x$ be a 3D patch from a target scan, reoriented so that one of its sharp in-plane axes plays the role of the "slice" axis. Let $D_{\hat h}$ blur and subsample that axis with a slice profile $\hat h$ estimated from the scan itself. Let $G_\theta$ be the super-resolution adapter with parameters $\theta$, and let $S$ be the frozen segmenter returning class logits. The per-scan objective is
$$ \mathcal{L}(\theta) = \lambda \,\mathrm{KL}\big(S(x)\,\|\,S(G_\theta(D_{\hat h}(x)))\big) + \|G_\theta(D_{\hat h}(x)) - x\|_1 , $$
Here KL is the divergence between voxelwise softmax outputs and $\lambda$ trades task consistency against pixel fidelity. After adaptation, $G_\theta$ is applied along the true thick axis and the result is passed to $S$.
Both sides of the consistency term keep the scan's original thick axis. The loss therefore isolates what the adapter does to the one axis it restores, not what the segmenter makes of residual anisotropy. Before adaptation, $G_\theta$ is pre-trained on source data with the same loss so that per-scan adaptation is a short fine-tune rather than training from scratch.
The topology concern raised by FeTA 2024 is that equal Dice can hide spurious holes and bridges. To address it, we optionally add the segmenter's soft output on a skeleton-weighted region as a third term. We treat this as an ablation, not a core ingredient.
Research Program
The work has three parts.
The first part builds the adapter and the per-scan adaptation loop around frozen segmenters. These are 3D U-Net/nnU-Net-style models trained once on source data and never updated.
The second part runs the degradation-fidelity study. We vary how $D_{\hat h}$ is built along a ladder of increasing realism and measure how the gap between proxy-task gains and real-thick-slice gains changes. The rungs, in order, are idealized Gaussian blur, a per-scan estimated slice profile, and a k-space-domain degradation. For the last rung we truncate and re-sample along the slice direction in the complex multi-coil data of Diff5T, so that the simulated thick slices carry real coil noise rather than an i.i.d. Gaussian approximation. This study is where the synthetic-to-real direction enters. It is a measurement of how wrong the simulated degradation can be before the task signal stops transferring.
The third part is a decomposition. On a cross-site benchmark, we ask what fraction of the frozen model's out-of-distribution drop a resolution-only front-end can recover. Contrast and anatomy shifts are outside what such an adapter can fix, and we would rather report the resolution-attributable share honestly than claim to fix all shift.
Baselines
The naive reference is the frozen segmenter applied to target scans resampled by trilinear or cubic B-spline interpolation. This is what a deployed device does today, and every gain is reported relative to it.
The strongest existing per-scan method is DA-VSR-style pixel-fidelity adaptation, combined with the filter-bank method of the anisotropic MRI paper for MRI targets. These are the closest competitors because they already adapt to each target scan without labels. Our claim is specifically that their objective is the wrong one for a downstream model.
We run each competitor through the identical adapter architecture, pre-training, per-scan iteration budget, and slice-profile estimate, setting $\lambda = 0$ so that the only difference is the loss. The reported number for this family is the 38.07 dB PSNR on KiTS19 at 4x from DA-VSR. No reported result was found on Lacuna for DA-VSR's effect on downstream segmentation, so we measure it ourselves. Beating this baseline means a higher frozen-model mDice and lower HD95 on real thick target scans, with PSNR no more than modestly lower.
The second strong competitor is the label-using upper bound. This is an InputAdapter-style front-end trained with the frozen segmenter's supervised loss on target labels, the same setting as InputAdapter but in 3D. It tells us how much of the labeled gain the label-free version captures.
We also report fine-tuning the segmenter on target labels. This serves as a regulatory-forbidden reference that should win locally, and we measure its backward drop on the source test set, which is expected from Regulating AI Adaptation.
Finally, we train a separate segmenter with random slice-thickness augmentation. This is the "retrain once with augmentation" competitor.
The key ablations sweep $\lambda$ from pixel-only to task-only, and swap the degradation model along the realism ladder while holding everything else fixed. All methods get the same tuning budget, selected on a held-out source-site split, never on target labels.
Why not just…?
Why not just fine-tune the segmenter on the new site? It is the most effective local fix and we include it. However, it changes a certified device, and the multi-site chest X-ray evidence shows average source-site losses of 0.176 AUC after local retraining Regulating AI Adaptation. It also needs target labels. Our comparison reports both its local gain and its backward loss next to the adapter, whose source performance is unchanged by construction.
Why not just run a good super-resolution model and then segment? This decoupled pipeline is exactly the pixel-only baseline. Natural-image evidence shows that PSNR-optimized super-resolution underserves recognition SR4IR, and image-first pipelines collapse at high acceleration in cardiac MRI DiSK. This is a real threat in our setting, though, because medical anatomy is smoother than natural scenes and the pixel and task objectives may largely agree. If the $\lambda$ sweep shows no task gain over $\lambda = 0$, the core claim fails and we will report that.
Why not just train the segmenter with thick-slice augmentation? For a vendor building a new model, this may be the cheapest answer, and it could match our method. It does not help already-cleared devices, and it relies on the same idealized blur whose fidelity we question. We train this model and compare. If it matches the adapter on real thick scans, the adapter's value reduces to the locked-model setting, and we will say so.
Why not just rely on a stronger architecture? AMOS reports 3D-UNet at 89.04% CT mDice above Swin-UNetr at 86.32%, and a drop of over 20% on unseen MRI scanners. TransNetR gains under multi-center testing, but its gains are modest. Architecture alone does not remove scanner shift. We test both a convolutional and a transformer frozen segmenter to show whether the adapter's gain is architecture-agnostic.
Proposed Experiments
AMOS. The main benchmark is AMOS, which has 500 CT and 100 MRI scans with 15 organs from eight scanner models at two centers. We train the frozen segmenter on a subset of CT scanners and evaluate on held-out scanners and on thick-slice cases. Evaluation is against native-grid labels, which needs no high-resolution ground truth. The in-distribution reference is 89.04% mDice for 3D-UNet. Metrics are mDice, HD95, PSNR on held-out in-plane proxies, and adaptation time per scan.
KiTS19. A secondary CT setting reuses the LIDC-IDRI-to-KiTS19 split of DA-VSR with a frozen kidney segmenter. It allows a direct PSNR comparison against the reported 38.07 dB alongside task metrics.
FeTA 2024. For topology we use FeTA 2024, training on KISPI and Vienna data and testing on the 0.55T KCL cohort. The reported reference there is Dice 0.86, HD95 1.69, and ED 6.26, where ED is the Euler characteristic difference, a measure of topological error. We also report biometry MAPE against the reported best of 7.72% and the human inter-rater 5.38%. The question is whether task-consistency adaptation lowers ED without raising MAPE.
Diff5T. The degradation ladder uses Diff5T's 0.5 mm structural scans and raw k-space to produce physically realistic thick-slice versions with known thin-slice truth. We pair it with a frozen brain-tissue segmenter.
Decisive checks. Three outcomes settle the proposal. The first is whether task-consistency adaptation beats pixel-only adaptation on real thick-slice AMOS targets at matched compute. The second is whether the gap between proxy gain and real gain shrinks as the degradation model moves from Gaussian to estimated to k-space-derived. The third is whether the label-free adapter recovers a substantial share of the labeled InputAdapter-style upper bound while the source-site performance of the frozen model stays exactly unchanged.
Hallucination check. To guard against the adapter producing images that fool the segmenter without being faithful, we report PSNR and difference maps on the Diff5T known-truth volumes for every $\lambda$.
References
- DA-VSR: Domain Adaptable Volumetric Super-Resolution for Medical Images
- Self-Supervised Super-Resolution for Anisotropic MR Images with and Without Slice Gap
- Beyond Image Super-Resolution for Image Recognition with Task-Driven Perceptual Loss
- HQG-Net: Unpaired Medical Image Enhancement With High-Quality Guidance
- Regulating AI Adaptation: An Analysis of AI Medical Device Updates
- Adapt the Data, Not the Model: Input-Space Adaptation for Frozen Time-Series Predictors
- Using MR Physics for Domain Generalisation and Super-Resolution
- Advances in automated fetal brain MRI segmentation and biometry: Insights from the FeTA 2024 challenge
- Diff5T: Benchmarking human brain diffusion MRI with an extensive 5.0 Tesla k-space and spatial dataset
- Direct Cardiac Segmentation from Undersampled K-Space using Transformers
- AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation
- TransNetR: Transformer-based Residual Network for Polyp Segmentation with Multi-Center Out-of-Distribution Testing