Abstract
Cross-Domain Few-Shot Learning (CD-FSL) aims to transfer knowledge from a source domain to novel target domains with limited data. A primary challenge in CD-FSL is the significant domain gap caused by shifts in visual styles. While existing methods like wave-SAN attempt to bridge this gap by swapping styles between images, they are limited to "real" and "easy" styles already present in the source data. This paper proposes StyleAdv, a model-agnostic meta-learning framework that synthesizes "virtual" and "hard" adversarial styles. By perturbing style statistics with signed gradients, StyleAdv forces the model to become robust against drastic visual shifts. Extensive experiments on eight target datasets demonstrate that StyleAdv achieves state-of-the-art performance using both CNN and Vision Transformer (ViT) backbones.
The Challenge of Domain Gaps
Few-Shot Learning (FSL) typically assumes that training and test data come from the same distribution. In contrast, CD-FSL involves moving from general domains (e.g., ImageNet) to specialized ones like medical X-rays or satellite imagery. The "style" of an image—its texture, lighting, and color distribution—is the primary driver of this domain gap. To build a robust model, we must move beyond simple style-swapping and create synthetic styles that specifically challenge the model's current feature representations.
Methodology: The StyleAdv Framework
StyleAdv operates as a minimax game within a meta-learning loop, consisting of an inner loop for style attack and an outer loop for model optimization.
1. Style Extraction
Inspired by Adaptive Instance Normalization (AdaIN), the model extracts the mean ($\mu$) and standard deviation ($\sigma$) of feature maps. For CNNs, these are calculated across spatial dimensions. For Vision Transformers, the framework reshapes patch tokens into spatial feature maps to compute equivalent channel-wise statistics.
2. Inner Loop: Style Adversarial Attack
The core innovation is a novel Style-FGSM (Fast Gradient Sign Method). Instead of perturbing raw pixels, it adds noise to the extracted style parameters ($\mu, \sigma$) in the direction that increases classification loss. This process creates "hard" adversarial styles that do not exist in the training set but are mathematically designed to expose model weaknesses.
3. Outer Loop: Optimization
The model is trained to minimize a multi-part loss function:
- Standard FSL Loss: Accuracy on clean images.
- Adversarial Loss: Accuracy on images with synthesized hard styles.
- Consistency Loss: A KL-Divergence term that ensures the model's predictions remain stable even when the style changes, preventing "semantic drift."
Key Results and Insights
StyleAdv was evaluated on the BSCD-FSL and FWT benchmarks, covering eight diverse target domains. It achieved new state-of-the-art results, improving ResNet-10 base models by up to 8.44% in 5-way 5-shot settings.
As illustrated in Figure 6, the adversarial styles (a) significantly alter the visual appearance while preserving the underlying object structure. The training curves (b) confirm that adversarial styles are indeed "harder" to learn, leading to higher loss during training but ultimately better generalization. T-SNE visualizations (c) show that StyleAdv produces much tighter and more separable class clusters compared to baseline methods, which is critical for few-shot classification under heavy domain shift.
Conclusion
StyleAdv demonstrates that attacking the latent style space is more effective for cross-domain generalization than attacking raw pixels. By forcing the model to focus on core semantic features rather than superficial style cues, StyleAdv provides a robust solution for CD-FSL across both traditional and transformer-based architectures.
Summary updated