TL;DR
This paper tackles the challenge of Cross-Domain Few-Shot Classification (CDFSC) by proposing a novel approach, ISSNet, that leverages multiple unlabeled source domains alongside one labeled source domain. By stylizing the labeled source domain with features from unlabeled sources, ISSNet effectively expands the data distribution and significantly improves classification performance on unseen target domains, outperforming existing methods.
What Problem Does This Paper Address?
Few-Shot Classification (FSC) aims to train models that can classify new categories with very few labeled examples, typically by leveraging knowledge from a large, labeled auxiliary dataset. However, a more challenging scenario, Cross-Domain Few-Shot Classification (CDFSC), arises when the auxiliary and target datasets come from vastly different domains (e.g., medical images vs. natural images). This domain shift makes it difficult for models trained on one domain to generalize to another.
Existing CDFSC methods often require labeled data from multiple source domains, which can be costly to obtain. This paper addresses the problem of CDFSC when only one source domain is fully labeled, and several other source domains are available but unlabeled. The core challenge is to effectively utilize these unlabeled source domains to bridge the domain gap and improve generalization to a target domain without incurring additional labeling costs.
Methods
The proposed solution is the Inter-Source Stylization Network (ISSNet), which operates in three stages:
- Inter-Source Style Transfer Learning: This stage uses a "teacher" network to transfer the visual style from the unlabeled source domains (Dsu) to the labeled source domain (Dsl). This process generates a new, pseudo-labeled dataset (Dal) that retains the content of Dsl but adopts the stylistic characteristics of Dsu. This effectively expands the data distribution of Dsl. The style transfer is guided by perceptual loss, style loss (using Gram matrices to capture texture similarity), and a cross-entropy loss for recognition.
- Cross-Domain Distilled Consistency Learning: Here, a "student" encoding model (M) is trained on both the original labeled source domain (Dsl) and the newly generated pseudo-labeled dataset (Dal). Knowledge distillation is employed to transfer knowledge from the teacher network to the student model. This stage also incorporates a batch spectral regularization (BSR) loss to regularize the singular values of the feature matrix, further enhancing generalization.
- Target Prototypical Classifier Learning: In the final stage, the pre-trained student model M is fine-tuned on the target dataset (Dt) using a standard N-way K-shot setting. A new classifier is trained to categorize the target domain samples. This stage also benefits from data augmentation (DA) and label propagation (LP) strategies.
Datasets
The paper utilizes a benchmark setup with:
- One fully labeled source domain: miniImageNet.
- Six unlabelled source domains: DIARETDB1, Swedish Leaf, FLIR 1, SUIM, CCTSDB, and UC Merced Land Use Dataset (UCMLUD). These are chosen to represent diverse practical scenarios.
- Eight target datasets: CropDisease, EuroSAT, ISIC, ChestX, CASIA NIR Database (faceNIR), Fish Recognition data (Fish), Colorectal Histology MNIST (CHM), and Audio-Visual Vehicle (AVV) Dataset. These datasets span near-domain and distance-domain scenarios.
Key Insights and Ideas
- Leveraging Unlabeled Data is Key: The core insight is that unlabeled data from diverse sources can be highly valuable for bridging domain gaps in CDFSC, even without explicit labels.
- Stylization as a Distribution Expansion Tool: The inter-source stylization technique effectively expands the data distribution of the labeled source domain by imbuing it with styles from unlabeled domains. This makes the model more robust to domain shifts.
- Consistency Learning Bridges the Gap: Distilling knowledge from the stylized data and the original labeled data into a single student model helps enforce consistency across domains, improving generalization.
- BSR Loss for Robustness: The study suggests that the Batch Spectral Regularization (BSR) loss is more effective than standard CE loss for improving CDFSC performance, likely by promoting more stable feature representations.
Key Results
- ISSNet achieves competitive performance across 8 target datasets, often securing the top two results.
- The paper demonstrates that ISSNet significantly reduces the performance loss caused by domain shifts compared to several baseline methods.
- Ablation studies show that the combination of KL loss and BSR loss ("KL+BSR") yields the best overall performance.
- The introduction of strategies like data augmentation (DA) and label propagation (LP) further boosts performance, with LP+DA being the most effective.
Limitations and Future Work
- Data Scarcity in Target Domain: The paper notes that for the AVV dataset, only a 5-way 1-shot task was evaluated due to extremely limited data, suggesting that performance might be further constrained in such scenarios.
- Cost of Multiple Sources: While the paper focuses on using unlabeled sources, the requirement for multiple source domains (even if unlabeled) might still be a practical consideration.
- Future Work: The paper implicitly suggests exploring more sophisticated style transfer techniques or different ways to combine information from multiple unlabeled sources. Further investigation into the effectiveness of BSR loss in other domain adaptation tasks could also be a direction.
Summary updated