Experiments in Cross-domain Few-shot Learning for Image Classification: Extended Abstract

Hongyu Wang, Huon Fraser, Henry Gouk, Eibe Frank, Bernhard Pfahringer, Michael Mayo, Geoff HolmesMeta-Knowledge Transfer @ ECML/PKDD 2022 · OpenReview

Abstract

We summarise experiments evaluating cross-domain few-shot learning (CDFSL) with feature extractors trained on ImageNet. The work explores the transfer performance of extracted features on five target domains with different degrees of similarity to ImageNet. These experiments compare robust classifiers and normalisation methods, consider multi-instance learning algorithms, and evaluate the effect of using features extracted by different ResNet backbones at various levels of their convolutional hierarchies. The cosine similarity classifier and 1-vs-rest logistic regression with $\ell_2$ regularisation are the top-performing robust classifiers in the evaluation, and $\ell_2$ normalisation improves performance on all five target domains when using LDA as the robust classifier. The results also show that feature extractors with the highest capacity do not always achieve the best CDFSL performance. Lastly, simple multi-instance learning methods are shown to improve classifier accuracy.

Introduction

Modern computer vision relies on deep convolutional neural networks (CNNs) that require massive amounts of labeled data. However, in fields like medicine or specialized agriculture, obtaining thousands of labeled images is often impossible.

Cross-domain few-shot learning (CDFSL) attempts to solve this by taking a model pretrained on a massive source dataset (ImageNet) and "refitting" it to a target domain where data is scarce (often just a few dozen examples). The challenge is the "domain shift": ImageNet contains everyday objects, while the target might be X-rays or satellite photos. This study investigates which specific combinations of model architectures, normalization techniques, and classifiers work best when the gap between source and target is large.

Methodology

The authors employed a "frozen backbone" approach. Instead of retraining the entire neural network—which often leads to overfitting on small datasets—they used pretrained ResNet models as fixed feature extractors.

Key experimental components included:

  • Backbone Architectures: Comparing different ResNet sizes (ResNet-101 vs ResNet-152) and extracting features from different levels of the convolutional hierarchy (e.g., the final layer vs. the penultimate block).
  • Robust Classifiers: Testing various heads on top of frozen features, including Logistic Regression, Linear Discriminant Analysis (LDA), and Cosine Similarity classifiers.
  • Normalization: Evaluating the impact of $\ell_2$ normalization on feature vectors.
  • Multi-instance Learning (MIL): Testing whether treating "weakly augmented" versions of a single image as a "bag" of instances improves classification accuracy.

Datasets

Models were pretrained on ImageNet and evaluated on five target domains representing varying degrees of similarity to the source:

  1. CropDisease: Plant pathology images.
  2. EuroSAT: Satellite imagery.
  3. ISIC: Skin cancer (dermoscopy) images.
  4. ChestX: Chest X-ray images.
  5. Food101: Food images (most similar to ImageNet).

Results and Discussion

The experimental results, summarized in the table below, reveal several critical insights into model selection for CDFSL.

Experimental results comparing normalization, backbones, classifiers, and MIL across five target domains.

The Capacity Paradox

A higher-capacity model does not always guarantee better transfer performance. For instance, on the CropDisease dataset, features from the penultimate block (conv4) of a ResNet-101 achieved 95.55% accuracy, significantly outperforming the deeper ResNet-152's final block (conv5), which reached only 92.51%. This suggests that the deepest layers of a network may become too specialized to the source data (ImageNet) to remain useful for distant domains.

Classifier Performance and Normalization

While specialized "few-shot" algorithms like Cosine Similarity are popular, standard 1-vs-rest Logistic Regression with $\ell_2$ regularization remained highly competitive, even leading on the difficult ISIC dataset (43.28% vs 42.98%). Furthermore, applying $\ell_2$ normalization to feature vectors consistently improved the performance of LDA across all five target domains. For example, on Food101, normalization boosted LDA performance from 77.90% to 79.06%.

Multi-instance Learning (MIL)

Using simple multi-instance learning—providing the classifier with multiple "views" of the same image—provided a consistent boost in accuracy. This was particularly evident in the most challenging domain, ChestX, where accuracy improved from a baseline of 25.29% to 25.81%.

Conclusion

This study demonstrates that "off-the-shelf" features can be effectively adapted for diverse domains, but success depends heavily on the choice of layer and normalization. Performance on highly specialized domains like ChestX remains low (~25%), indicating that ImageNet-pretrained features may lack the necessary "vocabulary" for medical imaging. Future work should explore more sophisticated fine-tuning strategies that can operate under few-shot constraints without collapsing.

Summary updated

Abstract

We summarise experiments evaluating cross-domain few-shot learning (CDFSL) with feature extractors trained on ImageNet. The work explores the transfer performance of extracted features on five target domains with different degrees of similarity to ImageNet. These experiments compare robust classifiers and normalisation methods, consider multi-instance learning algorithms, and evaluate the effect of using features extracted by different ResNet backbones at various levels of their convolutional hierarchies. The cosine similarity classifier and 1-vs-rest logistic regression with $\ell_2$ regularisation are the top-performing robust classifiers in the evaluation, and $\ell_2$ normalisation improves performance on all five target domains when using LDA as the robust classifier. The results also show that feature extractors with the highest capacity do not always achieve the best CDFSL performance. Lastly, simple multi-instance learning methods are shown to improve classifier accuracy.
BibTeX
@inproceedings{wang2022experiments,
  author = {Hongyu Wang and Huon Fraser and Henry Gouk and Eibe Frank and Bernhard Pfahringer and Michael Mayo and Geoff Holmes},
  title = {{Experiments in Cross-domain Few-shot Learning for Image Classification: Extended Abstract}},
  booktitle = {Meta-Knowledge Transfer @ ECML/PKDD},
  year = {2022},
  url = {https://openreview.net/forum?id=akB5jVQmzb},
}
Figure