When a segmentation model trained at one hospital is sent to another, it often fails. The anatomy is the same, but the images look different: another scanner vendor, another MRI sequence, another camera. The usual fixes pull in opposite directions. Unsupervised test-time adaptation needs no annotation, but it collapses when the shift is severe, as with CT to MRI or with tissue contrast inverting between sequences. Cross-domain methods that do use a little target annotation need the original training scans, which hospitals usually cannot share. We propose a middle path. A clinician draws rough scribbles on a handful of scans from the new site. We use those scribbles to learn a small, site-wide correction to the image intensities. The correction maps the new site's images back into the style the frozen model expects, and it is then applied to every later scan from that site.
The key design choice is what the correction is allowed to do. It is a per-pixel intensity remapping (a learned curve from input brightness to output brightness) multiplied by a smooth, low-frequency brightness field. By construction, such a transform cannot move an edge. It can only change which intensities tissues take and how brightness drifts slowly across the image. So a few dozen scribbled pixels cannot teach it new shapes, and the source model's anatomical knowledge stays intact. The claim we test is this: under a fixed, small annotation budget, restricting adaptation to this "style-only" family beats fine-tuning the model's weights on the same scribbles. It also recovers most of the gap to source-dependent scribble adaptation, without ever touching the source data.
Why this gap is real
Three bodies of work each cover part of the problem. The closest prior work is Scribble-based Domain Adaptation via Co-segmentation. It adapts vestibular schwannoma segmentation from contrast-enhanced T1 to T2 MRI using scribbles on 30 target scans covering about 1% of the volume. It reaches 83.4% Dice, above a model fully supervised on target data (81.6%). The same paper reports that unsupervised adaptation "largely failed." This result motivates us: sparse target scribbles are enough signal to bridge a cross-modality shift. The paper also sets our constraint, because its method jointly trains on 150 fully annotated source volumes and matches deep features across the whole dataset. Neither is possible when the model arrives at a new hospital as a frozen checkpoint.
The second body of work is source-free and test-time adaptation. DPL filters pseudo-labels, meaning the model's own guesses used as training targets, by uncertainty and class prototypes. GraTa aligns gradients from entropy and consistency losses. PTTEA minimizes the output of a learned shape-energy model. EDCP and per-image frequency prompts keep the model frozen and adapt small input-level modules. These papers supply our frozen-model and input-adaptation methodology. They also carry a warning: DPL and PTTEA both state that unsupervised signals break down under severe cross-modality shift. The histology audit in Source-Free Domain Adaptation of WSOL Models shows the same thing, with self-training locked into error loops when initial target accuracy is near 37%. Our method replaces the self-generated signal with a small amount of real human signal. That is the difference.
The third body of work treats annotation as the bottleneck. Can et al. train entirely from scribbles within one domain, losing only 2.9 Dice points on ACDC cardiac MRI and 3.7 on prostate. GazeRefine uses clinician gaze as a prompt over frozen DINOv3 features. These show that weak signals carry most of the information in dense masks. Both, however, stay within one domain or rely on generic features, and GazeRefine struggles on low-contrast prostate boundaries.
We chose not to use a Fourier amplitude adapter, the natural tool from FAN-Net and the frequency-mixing methods, as the core mechanism. Negating an image leaves its Fourier amplitude unchanged and shifts its phase by π. So a pure amplitude adapter cannot represent contrast inversion, which is exactly the T1-to-T2 case where unsupervised methods fail. A pointwise intensity curve can represent it. We keep the frequency insight only in the smooth brightness field, which plays the role of FAN-Net's low-frequency band.
Core Claim
Let $f$ be the frozen source segmenter and $x$ a target image. The corrected image is
$$\tilde{x}(r) = b_\phi(r)\, g_\theta\big(x(r)\big),$$
Here $r$ is a pixel location, $g_\theta$ is a piecewise-linear intensity curve with $K$ learnable knots shared across the site, and $b_\phi$ is a positive smooth field parameterized by a low-order polynomial or B-spline with a few dozen coefficients. The parameters $\theta, \phi$ are fit once per site by minimizing
$$\mathcal{L} = \sum_{i \in \mathcal{S}} \mathrm{pCE}\big(f(\tilde{x}_i), s_i\big) + \lambda \sum_{j \in \mathcal{U}} \mathrm{Cons}\big(f(\tilde{x}_j), f(\tilde{x}_j^{\mathrm{aug}})\big).$$
In this objective, $\mathcal{S}$ is the small set of scribbled scans with scribbles $s_i$, and pCE is cross-entropy computed only on scribbled pixels. $\mathcal{U}$ is the unlabeled scans from the same site, Cons is agreement under geometric augmentation, and $\lambda$ weights the two terms. Only about $K + 30$ numbers are learned per site, and the segmenter's weights never change, which also suits regulatory settings where the validated model must stay fixed, as argued for time series in INPUTADAPTER. The prediction we will test is that this constrained family has too little capacity to overfit five scribbled scans. It should therefore beat weight fine-tuning below roughly ten annotated scans, while matching unsupervised methods on mild shifts and clearly beating them on inversion-type shifts.
Research Program
The work has three parts. First, we build the adapter with a monotone curve and with a free, non-monotone curve. The non-monotone version is needed when tissue ordering changes, for example fluid dark in T1 and bright in T2. Second, we measure annotation-budget curves: Dice against the number of scribbled scans (1, 3, 5, 10, 30) at each target site. The unit of cost is scans scribbled, not pixels, because clinician time is spent per scan. Where datasets lack native scribbles, we synthesize them from ground-truth masks by skeletonizing eroded regions and sampling background strokes. We follow the roughly 1%-of-volume density reported in the co-segmentation paper. We will validate the synthetic scribbles against the paper's real ones on the vestibular schwannoma dataset if they are available. Third, we probe where the style-only assumption breaks. When the shift includes real anatomical or annotation-protocol differences, which a style-only transform by definition cannot fix, the adapter should plateau. We will report that plateau as a finding.
Baselines
The naive reference is the frozen source model applied without adaptation. On the five-domain fundus benchmark, GraTa reports this baseline trails GraTa by 5.87 Dice points on average and by 12.77 on Domain D. On AMOS, the benchmark paper reports drops of over 20% mDice on unseen MRI scanners. We rerun both rather than copy them.
The strongest annotation-free competitors are GraTa and EDCP. They represent the best way to adapt with zero human input, so they answer whether the scribbles are worth their cost. We run them on the same frozen checkpoints, with their published hyperparameters, and give them access to the same unlabeled pool $\mathcal{U}$. We beat them if we match them on fundus sites and exceed them by at least 5 Dice points on T1-to-T2 and CT-to-MRI transfer.
The strongest method that uses scribbles is scribble co-segmentation with source access, which reports 83.4% Dice on T2 vestibular schwannoma. It has privileged access to source data, so we treat it as an upper reference rather than a competitor. Recovering most of the gap between no-adaptation and 83.4% without source data would count as success.
The most dangerous fair competitor is fine-tuning the segmenter on the same scribbles, with the same pCE-plus-consistency loss. We run three versions: batch-norm parameters only, decoder only, and all weights. Each gets the same scribbled scans and the same number of optimization steps, and early stopping is chosen on a held-out scribbled scan. We also run scribble-only target training in the style of Can et al. to show what the source model contributes. Finally, as a training-free prompt baseline, we run GazeRefine-style prototype propagation over DINOv3 features, with scribbles in place of gaze.
The key ablation replaces the intensity-curve-and-bias-field family with an unconstrained small convolutional input adapter of similar size. This isolates whether the benefit comes from the edge-preserving constraint or merely from adapting the input instead of the weights. A second ablation compares per-image fitting with per-site fitting.
Why not just…?
Why not just fine-tune the model on the scribbles? This is the real threat, and we treat it as the central experiment rather than a strawman. The fine-tuned model has far more capacity and will win once enough scans are annotated. Our bet is limited to the small-budget regime, where fine-tuning can learn scribble-shaped rather than organ-shaped outputs. If batch-norm fine-tuning matches our adapter at 3 to 5 scans, the edge-preserving constraint adds nothing. We would then reframe the contribution as a budget study with fine-tuning as the recommendation.
Why not just use unsupervised test-time adaptation and skip annotation entirely? On mild camera and site shifts, it may suffice, and our budget curves will show where it does. The evidence in the file points the other way for large shifts. The co-segmentation paper reports that unsupervised adaptation failed for T1 to T2. DPL and PTTEA state their methods collapse under cross-modality shift. The WSOL audit shows self-training stuck in error loops from low starting accuracy.
Why not just prompt a frozen foundation model with the scribbles? This avoids adaptation altogether, but it depends on generic features separating the tissue. GazeRefine's 80.28% on prostate MRI and its reported weakness at low-contrast boundaries suggest they often do not. Such models also need prompts on every scan, while our correction is paid for once per site. The scribble-prompted DINOv3 baseline tests this directly, and we report per-scan annotation cost alongside Dice.
Why not just standardize intensities with histogram matching? Histogram matching is a fixed, monotone version of our intensity curve that ignores labels. It is the zero-parameter special case of our adapter, so we include it as a baseline. We expect it to fail under contrast inversion, because it cannot reorder tissues, and to fail under MRI bias fields, because it is global.
Proposed Experiments
We use four benchmarks. The first is vestibular schwannoma T1-c to T2, the headline severe shift, compared against the 83.4% co-segmentation reference. The second is the five-domain fundus optic disc and cup benchmark, a mild shift where GraTa and EDCP are strong. The third is multi-center NCI-ISBI prostate MRI, where Can et al. report 78.1% scribble-supervised Dice within one domain. The fourth is AMOS, with a model trained on CT and adapted to MRI from each unseen scanner. We report Dice and 95th-percentile Hausdorff distance against the number of scribbled scans, and add expected calibration error to check that corrected inputs do not produce overconfident outputs.
Two checks decide the project. The first is the crossover point between our adapter and batch-norm fine-tuning on the annotation-budget curve. The idea is justified only if that point lies at five or more scans on at least the two severe-shift benchmarks. The second is the non-monotone-curve ablation on T1-to-T2. If the monotone curve and histogram matching do as well, contrast inversion was not the obstacle and the mechanism story is wrong. We also check whether a per-site correction fit on five scans still helps on the site's remaining scans, because that amortization is what makes the annotation cost acceptable.
References
- Scribble-Based Domain Adaptation via Co-segmentation
- Source-Free Domain Adaptive Fundus Image Segmentation with Denoised Pseudo-Labeling
- Gradient Alignment Improves Test-Time Adaptation for Medical Image Segmentation
- Progressive Test Time Energy Adaptation for Medical Image Segmentation
- Efficient Deformable Convolutional Prompt for Continual Test-Time Adaptation in Medical Image Segmentation
- Each Test Image Deserves A Specific Prompt: Continual Test-Time Adaptation for 2D Medical Image Segmentation
- Source-Free Domain Adaptation of Weakly-Supervised Object Localization Models for Histology
- Learning to Segment Medical Images with Scribble-Supervision Alone
- GazeRefine: Expert Gaze as a Test-Time Prompt for Training-Free Medical Image Segmentation
- Fan-Net: Fourier-Based Adaptive Normalization for Cross-Domain Stroke Lesion Segmentation
- Adapt the Data, Not the Model: Input-Space Adaptation for Frozen Time-Series Predictors
- AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation