Abstract
Machine learning models frequently struggle with domain shift, where test data differs statistically from training data. This paper introduces a novel Domain Generalization (DG) approach that utilizes the single target sample available at inference time to bridge this gap. By projecting unseen target features onto a "label-preserving" source-feature manifold via a generative model, the method effectively "pulls" out-of-distribution samples back into a space the classifier understands. This approach achieves state-of-the-art results across multiple benchmarks, including PACS, Office-Home, and CIFAR-10-C, demonstrating enhanced robustness and data efficiency.
Introduction: The Challenge of Domain Shift
Machine learning models often fail when deployed in the real world because the target domain differs statistically from the source domains used during training. Traditional Domain Generalization (DG) methods attempt to address this by learning domain-invariant features during the training phase. However, these methods typically overlook the valuable information contained within the actual test sample available at inference time.
The authors argue that even a single target example can be related to previously perceived source samples, much like how humans generalize from known experiences to new ones. By leveraging this inference-time information, models can become significantly more robust to unseen variations.
Methodology
The proposed method consists of a three-stage pipeline designed to align target samples with a well-understood source manifold.
- Label-Preserving Transformation: During training, a feature extractor ($f_\theta$) is learned to cluster source data based on class labels while ignoring domain labels. A cosine-similarity-based loss ensures that samples with the same label are proximal in the feature space, minimizing the $\mathcal{H}$-divergence between source domains.
- Generative Manifold Learning: A generative model (VAE or GAN) is trained on these clustered source features. This model learns to map a simple latent distribution to the manifold of "valid" source features ($\mathcal{Z}_s$).
- Inference-time Target Projection: When a new target sample ($\mathbf{x}_t$) arrives, it may fall outside the source manifold. The core innovation is solving an optimization problem to find a point on the source manifold ($\mathbf{z}_t^*$) that is closest to the target feature ($\mathbf{z}_t$) in terms of cosine distance. This is achieved by optimizing the input to the generative model. The resulting "projected" feature is then passed to the final classifier.
By design, the projected feature is "familiar" to the classifier because it resides on the source manifold, yet it preserves the semantic label of the original target sample.
Experimental Setup
The method was evaluated on several standard DG benchmarks to test its versatility:
- PACS: High stylistic variance (Photo, Art, Cartoon, Sketch).
- VLCS: Real-world photographic datasets with varying camera biases.
- Office-Home: 65 classes across four distinct domains.
- Digits-DG: Digit recognition across multiple synthetic and real datasets.
- CIFAR-10-C: Robustness to 19 different types of image corruptions.
Results and Discussion
The proposed method demonstrates superior performance across all evaluated benchmarks, particularly in scenarios with significant domain gaps.
Key Performance Metrics
- State-of-the-Art Accuracy: On the PACS dataset, the method achieved 90.15% accuracy using a ResNet-50 backbone, significantly outperforming established methods like RSC (87.83%).
- Handling Dissimilar Domains: On Office-Home, the method showed a 5.6% improvement on the "Clipart" domain, proving its strength when the target domain is substantially different from the source.
- Robustness to Corruption: On CIFAR-10-C, the method consistently outperformed the M-ADA baseline across all 19 corruption types and severity levels.
- Data Efficiency: The approach remains robust even with limited training data, outperforming baselines when trained on only 20% of the available source data.
Insights
The success of this approach lies in "pushing" target data into the classifier's comfort zone. Instead of forcing the classifier to be a generalist, the authors use the generative model to translate unfamiliar target features into familiar source-like features. This "test-time" optimization is a powerful tool that complements traditional training-time invariance.
Conclusion and Future Work
While the method achieves impressive results, it introduces a computational overhead of approximately 50-100ms per image due to the iterative optimization step at inference time. This may limit its use in real-time video processing or low-power edge devices. Future work could focus on developing more efficient stopping criteria for the optimization process or extending this projection logic to complex tasks like semantic segmentation and zero-shot learning.
Summary updated