Medical images are usually shrunk to a few hundred pixels before a neural network sees them. Shrinking works like a blur: it removes fine detail, and fine detail is where early or subtle disease often lives. The usual remedy is to keep more resolution, through high-resolution crops, image tiles, or teacher–student schemes. This note argues that the remedy is not free when data are scarce. The fine detail that shrinking removes also contains each scanner's and each hospital's fingerprint: noise grain, compression patterns, detector texture. With a few hundred images drawn from a few sources, a model fed full resolution can sort cases by where they were acquired instead of by disease. That is exactly the shortcut already documented for small chest X-ray cohorts, where networks cluster cases by hospital and scanner rather than by pathology (Tartaglione et al.).
We propose to measure, separately for each band of image detail from coarse to fine, how much that band says about the disease and how much it says about the acquisition site. We then build a small-sample classifier that keeps the bands and scales carrying disease signal and actively removes the site signal from them. The test case is look-alike diseases, such as COVID-19 versus other pneumonias, or normal lungs with unusual markings versus early infiltrates. In these cases the separating evidence is subtle texture, so resolution should matter most, and so does the risk that texture means "which hospital" rather than "which disease." Put in two sentences: added resolution helps small-data medical classifiers only to the extent that the added detail carries disease information rather than scanner information. We can estimate that split per frequency band from a few labeled images plus free acquisition metadata, and use it to decide what detail to keep.
Core Claim
Our hypothesis is that under small samples, external (other-hospital) performance as a function of input resolution has an interior optimum, and that this optimum shifts toward lower resolution as the number of labeled cases falls and as the number of acquisition sources in the training set shrinks. Internal cross-validation will systematically pick too high a resolution, because scanner detail helps on held-out cases from the same sources. A classifier that keeps fine-detail bands but projects out the directions that predict acquisition site should recover the benefit of resolution without the shortcut. The pieces involved, multi-scale inputs, frozen pretrained features and site-removal, are not new individually. What is new is treating resolution as a quantity whose value depends on both sample size and site leakage, and measuring the two per band.
Relation to Prior Work
Three lines of work motivate the gap. First, work on downsampling loss documents that resizing erases small or texture-defined lesions. Shrinking acts as a low-pass filter that keeps anatomy and discards local pathology (Sze-To et al.), suppresses early-stage textures in chest X-rays (Gabruseva et al.), and blinds staging when biopsy slides are reduced to 224 pixels (Dossou et al.). Second, the mitigations for this loss all assume plenty of data. SDFN fuses a downsampled whole image with high-resolution lung crops but trains two DenseNet-121 networks on 112,120 images (SDFN). Resolution distillation needs large unlabeled pools; its weakest student improves by 4% when more unlabeled data are added (DiPalma et al.). AdaptivePath trains a reinforcement-learning zoom policy on thousands of slide patches (Agentic Visual Reasoning). Our proposal adopts SDFN's global-plus-local idea, but trains only a linear head on frozen features, so it remains viable at tens of cases. Third, small-sample work on ambiguous chest images, including COVID-CAPS (Afshar et al.), Deep Pneumonia (Wei et al.) and the CORDA study (Tartaglione et al.), resizes all inputs uniformly and never asks what resolution should be.
Two results constrain the design. Tartaglione et al. show that site shortcuts survive both augmentation and lung segmentation, which rules out relying on cropping alone. Afshar et al. show that pretraining on 94,323 NIH X-rays raised accuracy while lowering COVID sensitivity from 90% to 80%, a warning that imported features carry their own biases. For this reason we measure site leakage of the frozen features directly instead of assuming pretraining is neutral. Feature-subtraction ideas from few-shot segmentation, which remove background-similar components from foreground features (Xu et al.), are the closest methodological relative. We subtract a site direction instead of a background prototype.
Research Program
The first stage is a measurement study. Each training image is decomposed into a Laplacian pyramid, a stack of images each holding one band of detail from coarse anatomy to finest grain. Each band is also evaluated at several input scales (whole image at 224, 512 and 1024 pixels, plus native-resolution lung tiles). For each band and scale we extract frozen features from a chest-X-ray-pretrained backbone and fit two linear probes with nested cross-validation. One probe predicts the disease contrast of interest. The other predicts acquisition source, which is free metadata in multi-source collections like COVIDx. This yields, per band $b$, a disease score $D_b$ and a site score $S_b$ (cross-validated balanced accuracy above chance). We repeat the measurement at labeled sizes of 25, 50, 100 and 400 cases per class to trace how both scores change with $n$. A key confound will be logged explicitly: native resolution itself differs across sources, so image size alone can reveal the site.
The second stage is the classifier. Tile and global features from the retained bands are pooled with a small attention head. Before classification, the features are projected onto the subspace orthogonal to the linear site-probe directions; this is what we mean by site-nulling. Band and scale weights are chosen to maximize
$$J = \sum_b w_b \,(D_b - \lambda S_b),$$
where $w_b \in [0,1]$ is the weight on band $b$, and $\lambda$ trades disease signal against site leakage. $\lambda$ is selected by leave-one-source-out validation rather than random splits, since random splits reward leakage. The trainable parameters number in the thousands, which is the reason this can work at $n \le 100$.
The third stage is analysis. We fit the crossover sample size $n^*$ below which full resolution underperforms 224-pixel input on external data, and test whether $n^*$ grows with the number of training sources.
Baselines
The naive reference point is the standard recipe: resize to 224 pixels, fine-tune ResNet-18 with lung segmentation and RSNA pretraining, as in Tartaglione et al. On CORDA, that paper reports sensitivity 0.54 and specificity 0.80. Beating it means higher sensitivity at matched specificity on CORDA under identical splits.
The strongest resolution-preserving competitor is an SDFN-style dual pathway, combining a downsampled whole image with high-resolution lung crops (SDFN). It is the published method that most directly answers "keep more detail." SDFN reports mean AUC 0.815 versus 0.804 for whole-image-only on ChestX-ray14 at full data size. We will run it with the same backbone initialization, the same labeled subsets, and the same hyperparameter search budget, fine-tuned end to end and also with frozen features. Beating it means higher external AUC at $n \le 100$ per class, without loss on internal AUC larger than the confidence interval.
For the multi-source COVIDx setting, COVID-Net reports 93.3% accuracy and 91.0% COVID sensitivity on its full test split (Wang & Wong). COVID-CAPS reports 90% sensitivity without pretraining and 80% with it (Afshar et al.). We rerun both on our subsampled, source-held-out splits, because their reported numbers come from random splits and full data and are not comparable as-is.
Three ablations isolate the key ingredient: the full model without site-nulling, all bands with equal weight, and random band selection. A further baseline selects resolution by ordinary internal cross-validation.
Why not just…?
Why not just feed the full-resolution image and fine-tune? This is the hypothesis under test, not something to argue around. If full-resolution fine-tuning matches our model on held-out sources at $n \le 100$, the site-leakage story is wrong for this setting and we will report that. The prediction is that it wins internally and loses externally, consistent with the source clustering Tartaglione et al. observed even at standard resolution.
Why not just pick the resolution by cross-validation? With under 100 cases, random-split cross-validation shares sources between folds, so scanner detail looks like signal. We include CV-tuned resolution as a baseline. The decisive comparison is whether it selects a higher resolution than leave-one-source-out selection does, and whether that choice costs external performance.
Why not just suppress scanner detail with noise, blur and intensity-normalization augmentation? This is a real threat. Randomizing fine grain could destroy site cues more cheaply than projection does. Tartaglione et al. found augmentation insufficient, but they did not target high-frequency site texture specifically. We therefore include a strong high-resolution baseline with band-specific noise and blur augmentation plus histogram normalization. If it closes the gap, the contribution narrows to the measurement study and the $n^*$ analysis, which remain useful as a guide for practitioners.
Why not just crop the lungs? Cropping removes background markers but not the scanner texture inside the lungs. Tartaglione et al.'s segmented model reached only 0.54 sensitivity, and SDFN shows that crops alone lose context (0.807 versus 0.815 fused). Lung masks are applied in every condition, so cropping is held constant.
Proposed Experiments
The primary setting is COVIDx, using its five constituent repositories as site labels, with COVID-19 versus non-COVID pneumonia as the look-alike contrast. Training subsets of 25 to 400 per class are drawn from a subset of sources, and testing is on a held-out source. CORDA (447 images, a single emergency hospital) serves as a fully external clinical test after training on public data, and RSNA Pneumonia serves as pretraining and as a source of opacity boxes. JSRT's 2048-pixel radiographs give a clean native-resolution set for nodules versus non-nodules, where the disease signal is known to be fine-scale. The NCI Guanacaste colposcopy set (Zhang et al.) acts as a contrasting regime, where resolution is known to help and site diversity is low. There we expect $S_b$ to be small and full resolution to win, which is a falsifiable prediction in the other direction. Metrics are AUC, sensitivity at 90% specificity, the internal-minus-external gap, and site-probe accuracy on the final features, all averaged over at least ten subsamples per size with bootstrap intervals.
Three checks decide the proposal. First, $S_b$ should rise in the finest bands and with source heterogeneity, and fall in Guanacaste; if site information is flat across bands, the premise fails. Second, external AUC versus resolution should show an interior optimum at small $n$ that moves upward as $n$ grows. Third, at $n \le 100$, the site-nulled multi-band model should beat both full-resolution fine-tuning and the targeted-augmentation baseline on held-out sources while keeping site-probe accuracy near chance. If the augmentation baseline equals it, we will say so and present the band measurements as the result.
References
- Unveiling COVID-19 from CHEST X-Ray with Deep Learning: A Hurdles Race with Small Data
- Binary Codes for Tagging X-Ray Images via Deep De-Noising Autoencoders
- Deep Learning for Automatic Pneumonia Detection
- Pretrained Vision Models for Predicting High-Risk Breast Cancer Stage
- SDFN: Segmentation-based Deep Fusion Network for Thoracic Disease Classification in Chest X-ray Images
- Resolution-based distillation for efficient histology image classification
- Agentic Visual Reasoning in Whole-Slide Pathology Images via Active Perception
- COVID-CAPS: A capsule network-based framework for identification of COVID-19 cases from X-ray images
- Deep Pneumonia: Attention-Based Contrastive Learning for Class-Imbalanced Pneumonia Lesion Recognition in Chest X-rays
- Eliminating Feature Ambiguity for Few-Shot Segmentation
- COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest Radiography Images
- A Spatial Regulated Patch-Wise Approach for Cervical Dysplasia Diagnosis