Pseudo Multi-Source Domain Generalization: Bridging the Gap Between Single and Multi-Source Domain Generalization

Shohei EnomotoSubmission to ICLR 2026 · Withdrawn

TL;DR

Training models on multiple environments (Multi-Source Domain Generalization, or MDG) makes them robust to distribution shifts, but collecting multi-domain data is expensive. Pseudo Multi-Source Domain Generalization (PMDG) bridges this gap by generating synthetic "pseudo-domains" from a single source domain via transformations like style transfer and data augmentation, allowing standard MDG algorithms to run effectively in single-source settings.

What Problem Does This Paper Address?

Deep learning models typically degrade when deployed on distributions that differ from their training environment (e.g., changes in lighting, weather, or visual style).

While Multi-Source Domain Generalization (MDG) tackles this by training across diverse environments (e.g., learning domain-invariant features across photos, sketches, and paintings), gathering labeled datasets across multiple domains is prohibitively expensive. In contrast, Single-Source Domain Generalization (SDG) reflects real-world constraints—you only have one dataset—but cannot directly leverage the rich ecosystem of MDG algorithms because it lacks distinct domain partitions.

This paper asks: Can we artificially synthesize multi-domain splits from a single dataset to unlock the power of MDG algorithms for SDG?

Methods

The authors propose PMDG alongside an open-source evaluation suite, PseudoDomainBed (adapted from the popular DomainBed benchmark):

  • Pseudo-Domain Generation: Rather than feeding simple augmentations directly to Empirical Risk Minimization (ERM), PMDG creates $K$ distinct parallel pseudo-domains:
  • Style transformations: AdaIN Style Transfer (artistic style), CartoonGAN (cartoon style), and edge detection (sketch-like representations).
  • Data augmentations: Modern augmentations treated as domain shifts, including IPMix, RandConv, AugMix, RandAugment, and TrivialAugment.
  • MDG Training on Pseudo-Domains: Standard MDG algorithms (e.g., Spectral Decoupling [SD], RIDG, CORAL, IRM) are trained directly on these synthetic domain batches as if they were naturally collected, distinct data sources.
  • PseudoDomainBed Benchmark: Modifies DomainBed to standardize both dataset-level and mini-batch-level transformations, removing validation-set augmentations so out-of-distribution evaluation remains uncorrupted.
Single Source Domain ──► [ Transformation 1 (e.g., IPMix)  ] ──► Pseudo-Domain 1 ┐
                    ──► [ Transformation 2 (e.g., IPMix)  ] ──► Pseudo-Domain 2 ┼─► MDG Algorithm (e.g., SD)
                    ──► [ Unaltered Source                ] ──► Source Domain   ┘

Datasets

The authors benchmarked PMDG across multiple standard DG testbeds:

  • Small/Medium DG benchmarks: PACS, VLCS, OfficeHome, and TerraIncognita.
  • Large-scale validation: ImageNet (single source) evaluated on seven out-of-distribution variants: ImageNet-C, ImageNet-R, ImageNet-A, ImageNet-Sketch, ImageNet-V2, and Stylized-ImageNet.
  • PseudoDomainBed: The paper's modified adaptation of DomainBed, providing a unified testbed for single-source pseudo-domain evaluation.

Key Insights and Ideas

  • MDG performance correlates with PMDG performance: Algorithms that excel in genuine multi-domain setups (such as Spectral Decoupling) generally perform best when paired with pseudo-domains.
  • Pseudo-domains can match or beat real domains: In controlled experiments where total sample size $n$ was held constant, the gap between PMDG (trained on 1 domain + pseudo-domains) and MDG (trained on 3 actual domains) narrowed as $n$ increased. On certain target domains (e.g., Caltech101 on VLCS, Sketch on PACS), PMDG actually outperformed real MDG.
  • Consistency trumps kitchen-sink diversity: Applying a single high-quality augmentation multiple times (e.g., Original + IPMix + IPMix) outperformed mixing several different augmentations. Notably, some augmentations like CutMix degraded MDG performance.
  • Architectures matter: While convolutional backbones (ResNet50) favored augmentation-based pseudo-domains (Org + IM + IM), Vision Transformers (ViTs) benefited significantly more from shape-preserving style transfers (Org + ST + ED + CG + IM + IM), aligning with ViT's inherent shape biases.

Key Results

  • ResNet50 Benchmark: Combining Spectral Decoupling (SD) with Org+IM+IM achieved 55.9% average accuracy across PACS, VLCS, OfficeHome, and TerraIncognita, outperforming the strongest SDG baseline (IPMix at 55.2%) and standard ERM (50.6%).
  • Vision Transformer (ViT): SD paired with style-based pseudo-domains (Org+ST+ED+CG+IM+IM) reached 62.0% average accuracy, outperforming the baseline IPMix (60.5%) and ERM (59.3%).
  • ImageNet Out-of-Distribution: When scaling to ImageNet, PMDG (SD + Org+IM+IM) reached 37.20% OOD average accuracy versus 36.90% for IPMix, showing notable improvements on tough subsets like ImageNet-A (6.13% vs. 4.27%) and Stylized-ImageNet (12.44% vs. 11.54%).

Limitations and Future Work

  • The "Uniform Distribution" Assumption: PMDG assumes transformed samples always form a distinct, meaningful domain distribution. Weak or uncalibrated transformations may fail to produce a meaningful distribution shift, potentially adding noise rather than useful invariance.
  • Sensitivity to Transformation Choice: Certain MDG algorithms (like MLDG) deteriorated across nearly all pseudo-domain transformations, demonstrating that not every MDG formulation tolerates synthetic domain shifts.
  • Lack of Theoretical Guarantees: The paper is predominantly empirical. A formal theoretical analysis defining when and why synthetic pseudo-domains can mathematically substitute for real-world distribution shifts remains an open research direction.

Summary updated

Abstract

Deep learning models often struggle to maintain performance when deployed on data distributions different from their training data, particularly in real-world applications where environmental conditions frequently change. While multi-source domain generalization (MDG) has shown promise in addressing this challenge by leveraging multiple source domains during training, its practical application is limited by the significant costs and difficulties associated with creating multi-domain datasets. To address this limitation, we propose pseudo multi-source domain generalization (PMDG), a novel framework that enables the application of sophisticated MDG algorithms in a more practical single-source domain generalization setting. PMDG generates multiple pseudo-domains from a single source domain through style transfer and data augmentation techniques, creating a synthetic multi-domain dataset that can be used with MDG algorithms. Through extensive experiments with PseudoDomainBed, our modified version of the DomainBed benchmark, we analyze the effectiveness of PMDG across multiple datasets and architectures. Our analysis reveals several key findings, including a positive correlation between MDG and PMDG performance and the potential of pseudo-domains to match or exceed actual multi-domain performance with sufficient data. These comprehensive empirical results provide valuable insights for future research in domain generalization.
BibTeX
@misc{enomoto2025pseudo,
  author = {Shohei Enomoto},
  title = {{Pseudo Multi-Source Domain Generalization: Bridging the Gap Between Single and Multi-Source Domain Generalization}},
  year = {2025},
  eprint = {2505.23173},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.23173},
}

Loading related content...

Figure