Introduction: The Challenge of Single Domain Generalization
Deep learning models are notoriously brittle when they encounter "distribution shift"—situations where test data looks different from training data. While standard Domain Generalization (DG) involves learning from multiple sources (like photos, paintings, and cartoons) to find invariant features, Single Domain Generalization (single-DG) is significantly harder: the model is trained on only one source domain.
Existing single-DG methods typically use adversarial attacks or feature perturbations to "stress-test" models. However, these often produce "unnatural" images that lose their original meaning (semantic distortion) or fail to capture the complex stylistic shifts found in the real world. ProMEA (Prompt-driven Expansion and Alignment) addresses this by using rich, diverse descriptions available in language to guide data expansion.
Methodology: The ProMEA Framework
The authors propose a two-stage approach to bridge the gap between a single source and diverse unseen target domains.
1. Domain Prompt Expansion (DPE) - Training Phase
During training, ProMEA expands the single source domain into a diverse set of "virtual" domains. It utilizes a pre-trained Vision-Language Model (CLIP) and a lexical substitution model to create diverse text prompts (e.g., "a sketch of a dog" or "a painting of a dog").
A core technical innovation is Fourier Decoupling. Using a Fast Fourier Transform (FFT), the system separates an image into its Phase (carrying structural/semantic info) and its Amplitude (carrying style/low-level statistics). By combining the "style" (amplitude) from generated text prompts with the "structure" (phase) of source images, the model creates synthetic samples that adopt new styles while retaining the exact geometry of the original objects.
2. Domain Prompt Alignment (DPA) - Inference Phase
When the model encounters a completely new target image (e.g., a cartoon), it calculates the similarity between that image and the various "source prompts" learned during training. It then adaptively aligns the target sample's style toward the space spanned by the expanded source domains, making unseen data look more familiar to the classifier without needing to update model weights.
Experimental Results
ProMEA consistently outperformed state-of-the-art (SOTA) methods across three standard DG benchmarks:
- PACS (Photo, Art, Cartoon, Sketch): Achieved an average accuracy of 71.91%, beating the previous best (StyDeSty) by 0.56%. It saw a notable 2.52% jump in the "Art" domain.
- VLCS: Reached 67.50% accuracy, a significant 2.78% improvement over previous methods.
- OfficeHome: Achieved 60.29%, outperforming other single-DG methods by over 1%.
Ablation studies confirmed that the "Alignment" module during testing was particularly crucial for the "Sketch" domain, which often lacks the rich stylistic information found in photos or paintings.
Conclusion and Future Work
ProMEA demonstrates that language-guided augmentation is more semantically stable and realistic than raw pixel perturbation. By keeping the Phase of the original image constant, the model learns that an object's identity is independent of its stylistic Amplitude. While the method relies on the quality of the pre-trained CLIP model, its success in single-DG scenarios suggests that Fourier-based prompt guidance could be effectively applied to more complex tasks like object detection or segmentation.
Summary updated