Scale-Stratified Texture Statistics for Label-Efficient Cross-Site Disease Grading

Medical image graders that must separate adjacent disease stages, such as "mild" from "moderate" diabetic retinopathy, fail in two linked ways. They lose accuracy when moved to a new hospital's camera, and they need many expert grade labels to learn the fine distinctions. Neuro-symbolic diabetic retinopathy work adds a useful observation about which cues survive a camera change. In Single Domain Generalization in Diabetic Retinopathy: A Neuro-Symbolic Learning Approach, detecting isolated lesions such as hemorrhages and exudates helped cross-site transfer. Tracing thin continuous vessels hurt it, because vessel tracings are sensitive to focus and resolution. The authors call this the "vein paradox." Texture research offers a matching observation: statistics pooled over space are far more stable under resampling than point-by-point structure. DISTS scores image quality by comparing spatial means of deep features at several network depths. It kept a rank correlation of 0.928 with human judgments under shifts and rotations, while SSIM fell to about 0.19.

We propose to turn this observation into a grader. We build a small grading head on top of a frozen, self-supervised image encoder. Its inputs are per-scale spatial statistics of the encoder's feature maps: mean, variance, and lacunarity, a measure of how "gappy" or clustered activations are. Each statistic is weighted by a label-free estimate of how much it moves when acquisition conditions are perturbed, balanced against how well it separates grades on the few labeled images available. Only the head is trained, so it needs few expert labels, and the encoder stays frozen, as certified clinical models often must. The label-free sensitivity score lets the model drop cues that are both scale-specific and fragile before it ever sees the new hospital.

Core Claim

For grading tasks where adjacent stages differ by small, scattered lesions, the cues that are predictive of grade and robust across sites sit mostly in higher-order statistics of mid-resolution feature maps. The fragile cues sit in spatial structure and in the finest scales. We further claim that a cheap perturbation test on unlabeled images can tell the two apart. The test is to apply realistic blur, defocus, gamma, and compression changes and measure how much each statistic moves. The prediction is falsifiable in two ways. The perturbation sensitivity of a feature should correlate with that feature's actual drop on held-out sites. A head that weights features by this criterion should beat, on unseen sites and at small label budgets, a head that merely sees the same perturbations as training augmentation.

This differs from the closest work in specific ways. Lacunarity Pooling Layers supplies the multi-scale lacunarity operator: Gaussian-pyramid box-counting, fused by a 1×1 convolution. That paper was trained fully supervised, tested on in-distribution plant images, and never examined which scales transfer across sites. Samba makes embeddings cluster by severity rather than by hospital, using learnable Gaussian-mixture severity bases, but it needs fully supervised fine-grained labels. Its authors report that it degrades when severe grades are rare. DG-ADR aligns grades across sites with diffusion-generated images and an alignment loss. It warns that too much synthetic data teaches the model generative artifacts, which constrains us to avoid synthetic lesion generation entirely. Our contribution is not a new backbone. It is an explicit account of which scale-and-statistic features transfer, estimated without labels and applied to a frozen encoder.

Research Program

The encoder is a ResNet-50 pretrained with SimCLR (a contrastive method that learns by matching two augmented views of the same image) on unlabeled EyePACS fundus images. This follows Gupta et al., who showed such features transfer across cameras better than ImageNet features. Gupta et al. also note that SimCLR's blur and color augmentations may erase fine lesion texture. We therefore also test a variant pretrained with blur removed from the augmentation set.

From each of four encoder stages, and from two Gaussian-pyramid levels within each stage, we compute channel-wise features over space. Following DISTS, we pool with an anti-aliased Hanning-window pooling to reduce the texture flicker that plain strided pooling causes. The features are the spatial mean, the spatial variance, and multi-scale box-counting lacunarity. As a deliberate contrast, we also compute DISTS-style spatial cross-correlations between channels, which capture structure. Call the resulting feature $z_{j}$, where $j$ indexes a combination of scale, statistic type, and channel.

For each feature we measure two quantities. The first is susceptibility $S_j$: the variance of $z_j$ across a bank of acquisition perturbations applied to unlabeled images from several public sites. The perturbations are defocus blur, resolution rescaling, illumination gamma, vignetting, and JPEG compression, in the spirit of the augmentation-susceptibility maps of Ouyang et al.. This variance is normalized by the feature's natural variance across images. The second is grade discriminability $G_j$: a Fisher ratio of between-grade to within-grade variance, computed on the small labeled source set. The head applies the weight

$$w_j = \frac{G_j}{G_j + \lambda S_j},$$

where $\lambda$ trades off predictiveness against fragility. It then fits an ordinal regression head on the weighted features: $K-1$ cumulative "grade exceeds $k$" logits, which enforce the ordering of stages. The value of $\lambda$ is chosen on a held-out source site, never on a test site.

Susceptibility deliberately uses perturbations rather than raw between-site variance. Sites differ in how common each grade is, which would confound a site-difference measure with real disease signal. We report the between-site version as a secondary check.

The analysis program asks three questions, in order. Does $S_j$ predict the real per-feature shift between sites? Is this robustness concentrated in the variance and lacunarity statistics at mid scales, as the vein-paradox reading predicts, rather than in structure features and the finest scales? Does the weighting improve grading of adjacent stages on unseen sites, especially separating grade 1 from grade 2? If lesion-scale statistics carry the signal, this is where the gain should appear.

Baselines

The naive reference is a linear probe on globally average-pooled features from the same frozen SimCLR encoder, which reproduces the Gupta et al. setup. That paper reports 83.43% five-class accuracy on APTOS 2019 when transferring from EyePACS. Its protocol differs from a leave-one-site-out evaluation, so we rerun it inside ours rather than copy the number. Beating this baseline means higher quadratic weighted kappa and macro-F1 on unseen sites at every label budget.

The strongest existing domain-generalization method with a reported number in our protocol is DG-ADR, from Divergent Domains, Convergent Grading. In its seven-dataset leave-one-out benchmark it reports an average of 56.4% accuracy, 81.9% AUC, and 43.2% F1 across unseen domains. We adopt that protocol so this comparison is direct. DG-ADR fine-tunes the whole network with all source labels. We will match it at 100% labels, and we count parity there as sufficient if our method uses a frozen encoder and a head with a few thousand parameters. We count beating it at 10% and 25% of source labels as the main result.

Samba is the strongest architecture for cross-domain grading. The file reports only its gains relative to VMamba (a 4.1% average accuracy improvement on diabetic retinopathy) and no absolute number, so we will retrain it from released code where available under the same splits and label budgets. KG-DG is reported to reach 84.65% out-of-distribution accuracy with its symbolic branch alone, under its own single-source protocol. We run it only in that protocol and note that it consumes lesion boxes and vessel masks, which our method does not use.

Four ablations isolate the proposal's key ingredient. The first uses unweighted multi-scale statistics, which amounts to lacunarity-style pooling alone and tests whether the weighting matters. The second uses mean-only statistics at all scales, testing whether higher-order statistics matter. The third uses structure-only cross-correlation features, testing the vein-paradox prediction. The fourth replaces the learned sensitivity weights with random weights of the same sparsity.

All methods share the same splits, label subsets (three random draws per budget), preprocessing, and a matched hyperparameter search budget of 30 trials each, tuned on the same held-out source site.

Why not just…?

Why not just train the head with the perturbations as data augmentation? This is the most serious threat. Augmentation also teaches invariance to the same perturbation bank, with no susceptibility estimate needed. Our argument is that with only a few hundred labeled images, a head can fit a fragile but predictive feature despite augmentation. Down-weighting that feature explicitly spends no labels. The argument could be wrong. We therefore run the augmented head, and the augmented head combined with weighting, at every budget. If augmentation alone matches the weighted head on unseen sites, the proposal reduces to an analysis paper about which statistics transfer, and we would report it as such.

Why not just use lacunarity pooling as published? On plant data, Lacunarity Pooling gave small in-distribution gains: 94.88% to 95.00% on LeavesTex1200 and 73.27% to 73.77% on DeepWeeds. It had no domain-shift test. Its unweighted version is our first ablation. If it matches the weighted head across sites, weighting adds nothing.

Why not just fine-tune a full cross-domain grader like Samba or DG-ADR? These methods need full fine-grained labels and full fine-tuning. Samba's authors report failure on skewed grade distributions, and DG-ADR's authors report sensitivity to the volume of synthetic data. Both are costs in exactly the low-label, frozen-model regime we target. At 100% labels, either may win outright, and we will say so. The claim concerns the label-scarce, frozen regime.

Why not just rely on better self-supervised features? Stronger pretraining may shrink every gap. We test this by applying the same head to both SimCLR variants. The weighting's benefit should persist when the encoder changes, because the claim concerns pooling and feature selection, not the encoder.

Proposed Experiments and Decisive Checks

The main benchmark is the seven-dataset leave-one-out diabetic retinopathy protocol of DG-ADR. It spans APTOS 2019, DeepDR, Messidor-2 and the other public fundus sets listed there, with source label budgets of 10%, 25%, and 100%. We also run the single-source EyePACS-to-{APTOS, Messidor-1, Messidor-2} protocol of KG-DG. Unlabeled images for susceptibility estimation come only from source sites. The held-out site is never touched.

Metrics are quadratic weighted kappa, macro-F1, accuracy, and AUC, plus the grade-1-versus-grade-2 confusion rate and calibration error on unseen sites. As a secondary test of the scale hypothesis, we use the breast cancer histology grading task in Samba, which has a 20× to 40× magnification shift, if its data are obtainable. There, a scale-indexed representation should transfer better when pyramid levels are offset by one octave. That prediction is cheap to check.

Three results would be decisive.

First, the rank correlation between susceptibility $S_j$ and each feature's actual source-to-target shift must be clearly positive on held-out sites. If it is near zero, the mechanism is wrong and we stop.

Second, the weighted head must beat the perturbation-augmented head on unseen-site kappa at the 10% and 25% budgets, across three label draws.

Third, structure features and the finest scale should show the largest cross-site drops, and variance and lacunarity at mid scales the smallest, reproducing the vein paradox within a single network. If the pattern reverses, the texture-versus-structure reading of the paradox is false. That would itself be worth reporting.