Abstract
We summarise experiments evaluating cross-domain few-shot learning (CDFSL) with feature extractors trained on ImageNet. The work explores the transfer performance of extracted features on five target domains with different degrees of similarity to ImageNet. These experiments compare robust classifiers and normalisation methods, consider multi-instance learning algorithms, and evaluate the effect of using features extracted by different ResNet backbones at various levels of their convolutional hierarchies. The cosine similarity classifier and 1-vs-rest logistic regression with $\ell_2$ regularisation are the top-performing robust classifiers in the evaluation, and $\ell_2$ normalisation improves performance on all five target domains when using LDA as the robust classifier. The results also show that feature extractors with the highest capacity do not always achieve the best CDFSL performance. Lastly, simple multi-instance learning methods are shown to improve classifier accuracy.
Introduction
Modern computer vision relies on deep convolutional neural networks (CNNs) that require massive amounts of labeled data. However, in fields like medicine or specialized agriculture, obtaining thousands of labeled images is often impossible.
Cross-domain few-shot learning (CDFSL) attempts to solve this by taking a model pretrained on a massive source dataset (ImageNet) and "refitting" it to a target domain where data is scarce (often just a few dozen examples). The challenge is the "domain shift": ImageNet contains everyday objects, while the target might be X-rays or satellite photos. This study investigates which specific combinations of model architectures, normalization techniques, and classifiers work best when the gap between source and target is large.
Methodology
The authors employed a "frozen backbone" approach. Instead of retraining the entire neural network—which often leads to overfitting on small datasets—they used pretrained ResNet models as fixed feature extractors.
Key experimental components included:
- Backbone Architectures: Comparing different ResNet sizes (ResNet-101 vs ResNet-152) and extracting features from different levels of the convolutional hierarchy (e.g., the final layer vs. the penultimate block).
- Robust Classifiers: Testing various heads on top of frozen features, including Logistic Regression, Linear Discriminant Analysis (LDA), and Cosine Similarity classifiers.
- Normalization: Evaluating the impact of $\ell_2$ normalization on feature vectors.
- Multi-instance Learning (MIL): Testing whether treating "weakly augmented" versions of a single image as a "bag" of instances improves classification accuracy.
Datasets
Models were pretrained on ImageNet and evaluated on five target domains representing varying degrees of similarity to the source:
- CropDisease: Plant pathology images.
- EuroSAT: Satellite imagery.
- ISIC: Skin cancer (dermoscopy) images.
- ChestX: Chest X-ray images.
- Food101: Food images (most similar to ImageNet).
Results and Discussion
The experimental results, summarized in the table below, reveal several critical insights into model selection for CDFSL.
The Capacity Paradox
A higher-capacity model does not always guarantee better transfer performance. For instance, on the CropDisease dataset, features from the penultimate block (conv4) of a ResNet-101 achieved 95.55% accuracy, significantly outperforming the deeper ResNet-152's final block (conv5), which reached only 92.51%. This suggests that the deepest layers of a network may become too specialized to the source data (ImageNet) to remain useful for distant domains.
Classifier Performance and Normalization
While specialized "few-shot" algorithms like Cosine Similarity are popular, standard 1-vs-rest Logistic Regression with $\ell_2$ regularization remained highly competitive, even leading on the difficult ISIC dataset (43.28% vs 42.98%). Furthermore, applying $\ell_2$ normalization to feature vectors consistently improved the performance of LDA across all five target domains. For example, on Food101, normalization boosted LDA performance from 77.90% to 79.06%.
Multi-instance Learning (MIL)
Using simple multi-instance learning—providing the classifier with multiple "views" of the same image—provided a consistent boost in accuracy. This was particularly evident in the most challenging domain, ChestX, where accuracy improved from a baseline of 25.29% to 25.81%.
Conclusion
This study demonstrates that "off-the-shelf" features can be effectively adapted for diverse domains, but success depends heavily on the choice of layer and normalization. Performance on highly specialized domains like ChestX remains low (~25%), indicating that ImageNet-pretrained features may lack the necessary "vocabulary" for medical imaging. Future work should explore more sophisticated fine-tuning strategies that can operate under few-shot constraints without collapsing.
Summary updated