Re-acquire, Don't Repeat: Simulated Scanner Variation for Rare Disease Classes Trained at One Hospital

When a diagnostic model is trained on images from a single hospital, it has two problems at once. First, rare diseases are rare: a chest X-ray dataset can hold tens of thousands of normal films and only seven examples of its rarest finding. Second, every image carries that hospital's hardware signature, meaning its noise texture, contrast curve, and resolution. The standard fix for the first problem makes the second one worse. Class-balanced resampling shows each of the handful of rare-disease images to the model hundreds of times, which gives the classifier every opportunity to memorize the exact pixels, scanner texture included, of those few cases.

We propose a direct remedy. Whenever a rare-class image is resampled, the model should see a physically plausible re-acquisition of it rather than a copy: the same anatomy passed through a freshly drawn simulated scanner, with a different noise level, detector blur, exposure curve, and resolution. The claim is specific. Physics-grounded artifact simulation matters most at the point in a long-tailed training pipeline where tail images are duplicated, which is the classifier re-balancing stage. Applying the same simulation uniformly during representation learning, the obvious combination of the two literatures, leaves most of the cross-hospital tail-class gap in place. We will test this by training at one center and evaluating, without any retraining, at another.

Background and the gap

Two literatures meet here without having talked to each other. The first, domain randomization, trains networks on deliberately varied or even unrealistic synthetic corruptions so they stop relying on scanner texture. SynthSeg trains a brain segmenter only on synthetic scans with random contrasts, bias fields (smooth brightness gradients across the image), and resolutions, and it works on roughly 5,000 real scans from six modalities. AnyStar goes further and trains on synthetic blobs corrupted with scanner noise, ringing, and bias fields. Hoffmann's review distills the training choices behind this approach. These papers supply our method: a library of physically motivated corruptions. They also share a blind spot. They work on healthy anatomy or generic shapes, and no rare pathology or skewed class distribution appears anywhere in them.

The second literature, long-tailed classification, finds that networks learn usable features for rare classes and that the final linear layer is what fails, because its weights become biased toward frequent classes. On the benchmark it introduces, Holste et al. show that freezing the encoder and retraining only the head with class-balanced sampling (classifier re-training, cRT) raises tail-class accuracy on NIH-CXR-LT from 1.7% to 30.0%. MONICA confirms across 12 datasets that decoupled and hybrid methods are the most reliable. Both papers test within a single institution, so they motivate our proposal by leaving the cross-site question open. Holste et al. also constrain it: they report that MixUp, which blends two images pixel by pixel, hurts radiographs because it washes out thin, high-contrast diagnostic cues. Any artifact we simulate must therefore be bounded so that it does not do the same thing.

The closest work that combines directions stops short of this question. Adversarial Consistency for Single Domain Generalization synthesizes hard style shifts for single-source medical segmentation, but it uses style transfer rather than scanner physics and ignores class imbalance. Center-aware Adversarial Augmentation perturbs features away from class centroids. With seven tail examples those centroids are mostly noise, and that paper is tested only on natural images. Multi-domain long-tailed recognition and sharpness-aware long-tailed domain generalization both need several source domains, which a single hospital does not have. Artifact-Based Domain Generalization of Skin Lesion Models gives the cautionary result that shapes our design: debiasing against one annotated artifact does not transfer to others. That is why we randomize over families of unannotated acquisition parameters instead of targeting named artifacts.

Core Claim

In single-center long-tailed training, scanner-shortcut learning is concentrated in the tail classes. The re-balancing step is where it gets locked in, because that step repeats a few images many times. Replacing repetition with simulated re-acquisition should therefore close more of the external-site gap for tail classes than the same simulation applied only in representation learning. It should also do so without the cue destruction that MixUp causes.

Concretely, let $A_\theta$ be a corruption operator with acquisition parameters $\theta$: noise level, detector blur, exposure and gamma curve, bias field, and downsampling factor. In stage 2, each class-balanced draw of an image $x$ from class $c$ is replaced by

$$\tilde{x} = A_{\theta}(x), \quad \theta \sim p_c(\theta), \quad \theta \text{ drawn fresh at every draw},$$

where $p_c$ is a parameter distribution whose severity is capped per class. The cap is set by a cue-survival check: the largest severity at which a frozen reference classifier's confidence on held-out class-$c$ examples drops by less than a fixed tolerance. Rare classes with fine cues get gentler corruption than classes with coarse, visible findings. This per-class bound is the second new ingredient and is our answer to the MixUp failure.

Research Program

We will build modality-specific corruption libraries for two settings where a long-tailed train-at-one-site, test-at-another split already exists in the file. For chest radiographs, the library covers photon-count (Poisson) noise, detector blur, exposure and gamma variation, windowing, scatter-like low-frequency haze, and resampling. For fundus photographs, it covers illumination bias fields, defocus blur, and color and exposure shifts. These are implemented as fast on-the-fly transforms in the style of SynthSeg.

The analysis has three parts. The first checks the premise of Gap 2 directly. We freeze an ERM encoder trained at one center and fit linear probes to predict acquisition attributes, using view position where metadata allows or a synthetic center signature otherwise. We ask whether this information is more linearly available in tail-class images than in head-class ones. The second is a controlled trap experiment modeled on Bissoto et al.'s trap sets. We inject a mild synthetic scanner signature, such as a characteristic noise texture and contrast curve, that correlates with tail labels in training and reverse the correlation at test time. This isolates shortcut learning from all other cross-site differences. The third is the main comparison on real cross-site transfer.

Baselines

The naive reference point is ERM with standard augmentation (flips, crops, mild intensity jitter) and no re-balancing. It is trained at the source site and evaluated at the target site. Holste et al. report 1.7% tail accuracy for this setting on NIH-CXR-LT in-domain. No reported cross-site number found on Lacuna for NIH-CXR-LT to MIMIC-CXR-LT.

The strongest long-tailed competitor is cRT from Holste et al., which reports 30.0% tail accuracy and 0.369 group-wise average accuracy on NIH-CXR-LT within the same institution. Because it is the method our approach modifies, it is the fairest direct test. We rerun it with identical backbone, ImageNet initialization, epochs, and hyperparameter search budget. To beat it, our method must improve external-site tail accuracy by more than the seed-to-seed spread, estimated over at least five seeds, without lowering head-class accuracy by more than one point.

On fundus, the strongest reported external result is Balanced-MixUp, with a quadratic weighted kappa of 85.14 on Messidor-2 without retraining after training on Eyepacs. We rerun it at the same resolution and backbone (ResNeXt-50). Beating it means exceeding its rerun kappa on Messidor-2 and also improving recall on the rarest severe grades.

We also run GCL as benchmarked in MONICA, which reports 71.33% tail accuracy in-domain on ISIC-2019-LT at imbalance ratio 500. It is the most robust hybrid long-tailed method we know of. For style-based single-source generalization, we adapt the AdaIN adversarial synthesizer of Xu et al. as a training-time augmentation, since it is the strongest non-physics shift simulator in the file.

Three ablations isolate the key ingredient. The first applies physics corruption in stage 1 only, which is the obvious combination. The second applies it in stage 2 only, and the third in both. A fourth variant removes the per-class severity cap and uses a single global severity. A fifth replaces the physics library with generic heavy augmentation (random Gaussian noise, color jitter, RandAugment-style policies) at matched corruption strength, measured by pixel-space distortion. All variants share data, schedule, and a fixed tuning budget of the same number of trials each.

Why not just…?

Why not just use strong generic augmentation plus cRT? This is the most serious threat. If our gain comes only from breaking duplication, generic noise at matched strength could do as well. The fifth ablation settles it. If generic augmentation in stage 2 matches physics augmentation on external tail accuracy, we will report the finding as "diversify tail resamples" and drop the physics claim. The evidence that it may fall short is indirect: in Xu et al., ERM with conventional augmentation reaches 76.60% Dice from CT to MRI against 85.93% with synthesized shifts. That comparison is style-based rather than physics-based, and it concerns segmentation rather than tail classes.

Why not just apply artifact simulation during representation learning, as SynthSeg does? That is the default combination. The failure mode we predict is that a stage-1 encoder can become invariant on average while the stage-2 head, trained on hundreds of repeats of seven clean images, relearns the source site's signature through small residual feature directions. The stage-1-only versus stage-2-only ablation, together with the trap experiment, will show whether this happens. If stage 1 alone closes the gap, our core claim is wrong.

Why not just use Balanced-MixUp? It already attacks duplication by blending tail images with others, and it transfers well to Messidor-2. However, Holste et al. find that pixel blending hurts radiographs by erasing subtle findings, and blending does not model a different scanner. We run it on both modalities. We expect it to remain competitive on fundus and to lose on chest X-ray tail classes.

Why not just fine-tune or adapt to each target hospital? Target data is often unavailable during development, and validated models are often frozen, as argued by Xu et al. and Adapt the Data, Not the Model. Adaptation also needs rare-class target examples, which one hospital's small calibration set seldom contains. We still report a few-shot fine-tuned upper bound so readers can see how much of that gap zero-shot simulation recovers.

Proposed Experiments

The primary experiment trains on NIH-CXR-LT and evaluates zero-shot on MIMIC-CXR-LT over their shared label subset, and then in the reverse direction. Metrics are head, medium, and tail accuracy, group-wise average accuracy, and macro AUC. The secondary experiment trains on Eyepacs and tests on Messidor-2, reporting quadratic weighted kappa and per-grade recall. A tertiary check uses ISIC-2019-LT with external PH2, Derm7pt, and PAD-UFES-20, with the caveat that dermoscopy artifacts are less physics-driven.

The main outcome is the external-minus-internal accuracy gap for each frequency group. The claim predicts that the tail gap shrinks most under stage-2 re-acquisition and least under stage-1-only simulation.

Three checks decide the question. The first is the trap set: under reversed signature correlation, stage-2 re-acquisition should keep tail accuracy near its unbiased value while cRT collapses. The second is the probe: acquisition information in frozen features should be higher for tail classes under ERM and fall after our training. If neither of these holds, the shortcut mechanism is not what drives the external gap, and the proposal's premise fails. The third is cue survival: without the per-class severity cap, chest X-ray tail accuracy should drop in the way Holste et al. report for MixUp. That drop would confirm that the bound is necessary and not decorative.