ProMEA: Prompt-driven Expansion and Alignment for Single Domain Generalization

Yunyun Wang, Yi Guo, Xiaodong Liu, Songcan ChenIJCAI 2025 · DOI

Introduction: The Challenge of Single Domain Generalization

Deep learning models are notoriously brittle when they encounter "distribution shift"—situations where test data looks different from training data. While standard Domain Generalization (DG) involves learning from multiple sources (like photos, paintings, and cartoons) to find invariant features, Single Domain Generalization (single-DG) is significantly harder: the model is trained on only one source domain.

Conceptual comparison between existing single-DG methods (a) and the proposed ProMEA method (b).

Existing single-DG methods typically use adversarial attacks or feature perturbations to "stress-test" models. However, these often produce "unnatural" images that lose their original meaning (semantic distortion) or fail to capture the complex stylistic shifts found in the real world. ProMEA (Prompt-driven Expansion and Alignment) addresses this by using rich, diverse descriptions available in language to guide data expansion.

Methodology: The ProMEA Framework

The authors propose a two-stage approach to bridge the gap between a single source and diverse unseen target domains.

1. Domain Prompt Expansion (DPE) - Training Phase

During training, ProMEA expands the single source domain into a diverse set of "virtual" domains. It utilizes a pre-trained Vision-Language Model (CLIP) and a lexical substitution model to create diverse text prompts (e.g., "a sketch of a dog" or "a painting of a dog").

Detailed architecture of ProMEA, showing Domain Prompt Expansion (DPE) and Domain Prompt Alignment (DPA) modules.

A core technical innovation is Fourier Decoupling. Using a Fast Fourier Transform (FFT), the system separates an image into its Phase (carrying structural/semantic info) and its Amplitude (carrying style/low-level statistics). By combining the "style" (amplitude) from generated text prompts with the "structure" (phase) of source images, the model creates synthetic samples that adopt new styles while retaining the exact geometry of the original objects.

2. Domain Prompt Alignment (DPA) - Inference Phase

When the model encounters a completely new target image (e.g., a cartoon), it calculates the similarity between that image and the various "source prompts" learned during training. It then adaptively aligns the target sample's style toward the space spanned by the expanded source domains, making unseen data look more familiar to the classifier without needing to update model weights.

Experimental Results

ProMEA consistently outperformed state-of-the-art (SOTA) methods across three standard DG benchmarks:

  • PACS (Photo, Art, Cartoon, Sketch): Achieved an average accuracy of 71.91%, beating the previous best (StyDeSty) by 0.56%. It saw a notable 2.52% jump in the "Art" domain.
  • VLCS: Reached 67.50% accuracy, a significant 2.78% improvement over previous methods.
  • OfficeHome: Achieved 60.29%, outperforming other single-DG methods by over 1%.

Ablation studies confirmed that the "Alignment" module during testing was particularly crucial for the "Sketch" domain, which often lacks the rich stylistic information found in photos or paintings.

Conclusion and Future Work

ProMEA demonstrates that language-guided augmentation is more semantically stable and realistic than raw pixel perturbation. By keeping the Phase of the original image constant, the model learns that an object's identity is independent of its stylistic Amplitude. While the method relies on the quality of the pre-trained CLIP model, its success in single-DG scenarios suggests that Fourier-based prompt guidance could be effectively applied to more complex tasks like object detection or segmentation.

Summary updated

BibTeX
@inproceedings{wang2025promea,
  author = {Yunyun Wang and Yi Guo and Xiaodong Liu and Songcan Chen},
  title = {{ProMEA: Prompt-driven Expansion and Alignment for Single Domain Generalization}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/225},
}
Figure