Rebalancing Without Borrowing the Hospital: Rare Resistance Mechanisms Across Centers

When a lung tumor stops responding to a targeted drug, what the oncologist does next depends on why it stopped. The most common reason is one secondary mutation. Rarer reasons, such as amplification of a second growth gene or transformation into a different cancer type, each occur in a few percent of patients and call for different treatments. Today the reason is found by re-biopsy, which is invasive and can miss the resistant part of the tumor. A CT-based classifier of resistance mechanisms would be useful. It would also have to solve two known machine-learning problems at the same time. The rare mechanisms are a textbook long tail: a few classes dominate and many are scarce. And such a model is only credible if it works at hospitals it was not trained on.

This note argues that these two problems interact in a harmful way. In multi-center data, rare classes tend to come disproportionately from a few hospitals, usually the referral centers that re-biopsy and sequence. Standard class rebalancing gives each scarce example more weight during training, so it also gives more weight to that hospital's scanner signature. We predict that rebalancing raises internal tail accuracy partly by teaching the model "this scanner means rare mechanism," and that this gain shrinks or reverses at new hospitals. We propose to measure this effect on public chest X-ray benchmarks where it can be controlled. We then propose a specific fix, which is to rebalance over class-by-hospital cells rather than over classes. We test that fix, together with longitudinal within-patient contrast, on a multi-center cohort of EGFR-mutant lung cancer patients who progressed on tyrosine kinase inhibitors (TKIs, small-molecule drugs that block the mutated growth receptor).

Core Claim

Our claim is that for rare classes whose labels are confounded with acquisition site, the choice of what to rebalance over matters more for external validity than the choice of rebalancing algorithm. We expect three things. Class-balanced classifier retraining will show a larger internal-to-external drop on tail classes than on head classes. Balancing jointly over (class, site) cells, combined with acquisition-targeted augmentation, will recover most of that drop. For the resistance task specifically, representing each patient by the change between the pre-treatment and progression scans will reduce site dependence, because both scans usually come from the same hospital and share its artifacts.

The biological motivation comes from the somatic-evolution view summarized in Computational Cancer Biology: An Evolutionary Perspective. That work argues resistant subclones usually pre-exist and expand under drug pressure. What changes between scans is therefore more informative than any single snapshot. Prior CT and PET/CT work on EGFR looks only at baseline scans and only at head classes. A Computer Tomography Radiomics-based model to Predict Survival of Patients with EGFR‑Mutated Non‑small‑cell Lung Cancer predicts survival at a single center. Advancing EGFR mutation subtypes prediction in NSCLC by combining 3D pretrained ConvNeXt, radiomics, and clinical features discards all rare subtypes and reaches an area under the ROC curve (AUC) of 0.775 on ex19del versus L858R, again at one hospital. PET/CT Based EGFR Mutation Status Classification of NSCLC Using Deep Learning Features and Radiomics Features predicts binary mutation status at one center, and its radiomics model already drops from 0.82 to 0.68 AUC on an internal split. None of these papers classifies acquired mechanisms, handles the tail, or tests on an outside hospital. Machine learning based personalized drug response prediction for lung cancer patients models 33 EGFR mutations structurally but uses no imaging and no clinical cohort. It supports the premise that the mechanism determines drug response, but it offers no route from scans.

Research Program

The method starts from decoupled training, the approach of Long-Tailed Classification of Thorax Diseases on Chest X-Ray: A New Benchmark Study. An image encoder is first trained on the natural, imbalanced data. It is then frozen, and only the final classifier is retrained with class-balanced sampling (classifier re-training, cRT). That paper showed the tail failure lives mostly in the final layer's bias toward frequent classes, which makes stage two the natural place to intervene. It also found that MixUp-style pixel blending destroys fine radiographic detail. That finding constrains our design: we do not rely on blending for CT.

Our change to stage two is site-aware sampling. Let $n_{y,s}$ be the number of training examples with class $y$ at site $s$. Each example from an observed cell is drawn with probability

$$ p(x) \propto \frac{1}{n_{y,s}}, $$

so every observed class-site pair contributes equally. Tail examples from an over-represented referral center are therefore no longer multiplied into dominance. Where a tail class is observed at only one site, sampling alone cannot break the confound. For that case we add acquisition-targeted augmentation in the spirit of Out-of-Distribution Robustness via Targeted Augmentations. That paper showed that randomizing only the spurious acquisition factors, stain color in its case, beats generic invariance methods. For CT, our analogue simulates reconstruction-kernel sharpness, slice-thickness resampling, and noise level, the factors that Radiomics in prostate cancer imaging for a personalized treatment approach - current aspects of methodology and a systematic review on validated studies identifies as the main destabilizers of texture features.

For the resistance task, each patient has a pre-treatment scan $x_0$ and a scan at progression $x_1$. With encoder $f$, the classifier sees $[f(x_1),\, f(x_1)-f(x_0)]$, so a single ablation can isolate the value of the difference term. Mechanisms can co-occur, and "no mechanism identified" is itself a large class. We therefore treat the labels as multi-label, following the point raised in Towards long-tailed, multi-label disease classification from chest X-ray: Overview of the CXR-LT challenge. The system also reports a calibrated abstention option. A 30% tail accuracy is not clinically usable, but a model that flags which patients most need re-biopsy for a rare mechanism could be. Throughout, we quantify confounding as the mutual information between tail labels and site in the training pool, so that effects can be plotted against a measured quantity.

The closest methodological prior is Tackling Long-Tailed Category Distribution Under Domain Shifts. It names the problem of tail classes co-occurring with domains and addresses it with meta-learning, word-embedding semantics, and feature augmentation borrowed from head classes. It was evaluated only on natural images with synthetic style shifts. We differ in three ways: we use real hospital shifts, we use a much simpler intervention placed where the tail bias actually sits, and we add a longitudinal signal that is specific to the clinical problem. Semantic word embeddings of mechanism names ("MET amplification") seem unlikely to carry visual similarity, so we would not expect their method's central ingredient to transfer.

Baselines

The naive reference point is plain empirical risk minimization with cross-entropy on the natural distribution, meaning every example weighted equally. On NIH-CXR-LT, Long-Tailed Classification of Thorax Diseases on Chest X-Ray: A New Benchmark Study reports only 1.7% tail accuracy for this baseline. It anchors how much any rebalancing buys.

The strongest competitor is class-balanced cRT from the same paper, which reports 30.0% tail accuracy on NIH-CXR-LT and 36.7% on MIMIC-CXR-LT, and raises group-average accuracy from 0.164 to 0.369. It is the right competitor because our proposal changes only its stage two. Any gain we show is a gain over the best version of the same recipe. We run it with the identical frozen encoder, the same number of stage-two epochs, and the same hyperparameter search budget on the same internal validation split. To beat it, we must improve external tail accuracy (or balanced accuracy) by a margin exceeding bootstrap confidence intervals, with no significant loss on external head accuracy.

We also run Balanced-MixUp from Balanced-MixUp for Highly Imbalanced Medical Image Classification. It is the one imbalance method in the file with a true external test, reporting a quadratic weighted kappa of 80.78 on Eyepacs and 85.14 on the external Messidor-2. For the long-tail-under-shift problem we include the method of Tackling Long-Tailed Category Distribution Under Domain Shifts, which reports 42.1% versus 27.0% on AWA2-LTS. We include group distributionally robust optimization over sites as the standard domain-robust option. WILDS: A Benchmark of in-the-Wild Distribution Shifts found it and similar methods did not beat ERM on 8 of 10 datasets, so we expect it to be weak, but it must be shown. Every baseline receives the same backbone, data, and tuning budget.

Three ablations isolate our ingredients: site-aware sampling without augmentation, augmentation without site-aware sampling, and, for the resistance cohort, the progression scan alone versus the progression scan plus the difference term. For the clinical resistance task itself, no reported result was found on Lacuna for CT-based classification of acquired resistance mechanisms. The Zhang et al. baseline-subtype AUC of 0.775 is the nearest reported point, though it measures a different task.

Why not just…?

Why not just apply cRT and report an external test? This is the obvious combination and is our main baseline. Our hypothesis is precisely that it will look good internally and transport poorly on tails. Long-Tailed Classification of Thorax Diseases on Chest X-Ray: A New Benchmark Study never tested NIH-to-MIMIC transfer, so there is currently no evidence either way. If cRT's tail gains transport intact even under deliberately induced confounding, our core claim is false. The proposal would then shrink to a clinical application paper, and we would report it that way.

Why not just use a domain-generalization method such as invariance penalties or group robustness? WILDS: A Benchmark of in-the-Wild Distribution Shifts found these rarely beat ERM on real shifts. Out-of-Distribution Robustness via Targeted Augmentations explains why: with few training sites the problem is underspecified, and enforcing invariance discards useful domain context. They are nonetheless included as baselines, so this is settled empirically rather than by citation.

Why not just recalibrate locally at each hospital? All models are local: time to replace external validation with recurrent local validation argues exactly this, and it is a real threat. Retraining the final layer on a few local labels might erase any site-aware advantage. We therefore run a few-shot local head-retraining arm at each held-out site, with $k$ labeled local patients. For rare mechanisms, however, a new site may have zero or one local tail example, so local recalibration cannot fix the tail. We will report the $k$ at which local recalibration catches up. If it catches up at small $k$ even for tails, local recalibration is the better recommendation and we will say so.

Why not just harmonize radiomic features, or use Balanced-MixUp? Harmonization acts only on features, not on the classifier's tail bias. It will be tested as a preprocessing variant under every arm. Balanced-MixUp is a baseline, but Holste et al.'s finding that blending degrades radiographs suggests it will struggle on CT texture.

Proposed Experiments

The first experiment is fully public and decisive for the core claim. We use NIH-CXR-LT and MIMIC-CXR-LT from Long-Tailed Classification of Thorax Diseases on Chest X-Ray: A New Benchmark Study, restricted to their shared findings, with dataset as "site." We build training pools in which tail-class examples come from one source with controlled proportions, from 50% (unconfounded) to 100% (fully confounded), and test on held-out data from the other source. Every arm is evaluated on internal and external head, medium, and tail accuracy, balanced accuracy, and calibration error. The predicted signature has two parts. First, cRT's external tail accuracy falls off as confounding rises while its internal tail accuracy stays flat or rises. Second, site-aware sampling flattens that curve. We also run the natural, unmanipulated transfer in both directions, which Holste et al. left open.

The second experiment checks generality on other modalities. We use MONICA from MONICA: Benchmarking on Long-tailed Medical Image Classification, which reports 71.33% versus 12.67% ERM tail accuracy on ISIC-2019-LT at imbalance ratio 500, wherever site or source metadata can define domains. We also use Camelyon17-WILDS with an artificially long-tailed label subsample, where the stain-jitter result of 75.3% to 90.5% out-of-distribution accuracy gives a known-good targeted augmentation to reuse.

The third experiment is the clinical target. It is a retrospective multi-center cohort of EGFR-mutant non-small-cell lung cancer patients with progression on first- or third-generation TKIs, re-biopsy or sequencing-confirmed mechanism labels, and both pre-treatment and progression CT. Evaluation uses leave-one-center-out testing, per-mechanism AUROC, tail recall at fixed specificity, and selective risk under abstention. This cohort does not exist in the file and must be assembled. Tail classes may contain only tens of patients. We therefore pre-register bootstrap intervals and treat the public experiments as the evidence for the mechanism, with the clinical cohort as the test of usefulness.

The decisive checks are two. First, does the internal-minus-external tail gap of cRT grow with measured label-site confounding, and does site-aware sampling shrink it without hurting heads? Second, on the clinical cohort, does adding the within-patient difference term improve leave-one-center-out tail AUROC more than it improves internal AUROC? That pattern would indicate it removes site signal rather than merely adding capacity.