CycleMix: Mixing Source Domains for Domain Generalization in Style-Dependent Data

Aristotelis Ballas, Christos Diou

Abstract

Deep learning systems often struggle with Domain Generalization (DG) because they rely on spurious correlations, such as associating specific image styles with target classes. This paper introduces CycleMix, a novel data augmentation strategy designed to mitigate this issue. By utilizing CycleGANs to learn and blend stylistic attributes from multiple source domains, CycleMix creates synthetic training samples with randomized styles. This process forces the feature extractor to disregard style-dependent features in favor of style-invariant representations. Experimental results on the PACS benchmark demonstrate that CycleMix significantly improves generalization to unseen domains, particularly in cases with high stylistic variance.

Introduction

Machine learning models typically operate under the i.i.d. (independent and identically distributed) assumption, where training and testing data are expected to share the same distribution. In real-world applications, this assumption frequently fails. A primary cause of this failure in image classification is the model's tendency to prioritize "spurious correlations"—shortcuts like background textures or artistic styles—over the actual semantic content of an object.

To address this, the goal of Domain Generalization is to develop a feature extractor that is robust to domain shifts. CycleMix approaches this by treating style as a nuisance variable that can be manipulated to teach the network to focus on the "essence" of an object regardless of its visual medium.

Methodology: The CycleMix Framework

CycleMix employs a two-stage approach to diversify the training distribution and encourage style-invariance.

1. Style Capture via CycleGAN

The first stage involves training a suite of CycleGAN models to map transitions between every pair of source domains. For a dataset with multiple domains (e.g., Art, Cartoon, Photo, Sketch), these models learn to translate an image from one domain's style into another while preserving the underlying content.

2. The Mix Operation and Randomized Blending

During the training of the image classifier, CycleMix generates stylistic variations of each input image. For any given source image, the pre-trained GANs produce versions of that image in all other available styles.

Instead of simply adding these translated images to the dataset, CycleMix performs a Randomized Blending operation. For each image in a mini-batch, the algorithm assigns a random weight ($a_j$) to each style. The final training sample is a weighted sum of the original image and its various stylistic translations. By constantly varying the magnitude of these style components, the model is exposed to an infinite variety of synthetic "novel" domains, preventing it from latching onto any single stylistic signature.

To maintain the integrity of the learning process and prevent the model from being overwhelmed by synthetic data, the authors apply CycleMix to 50% of each mini-batch, while the remaining 50% consists of original source data.

Experimental Setup

The proposed method was evaluated using the PACS benchmark, a standard for DG research. PACS contains images across four domains:

  • Photo
  • Art Painting
  • Cartoon
  • Sketch

The dataset includes seven object classes (dog, elephant, giraffe, guitar, horse, house, person). Following the "leave-one-domain-out" protocol, the model is trained on three domains and tested on the fourth, which remains entirely unseen during the training phase.

Results and Discussion

CycleMix demonstrated superior performance compared to established baselines such as Mixup, CutMix, and SagNet.

  • Accuracy Gains: CycleMix achieved an average top-1 accuracy of 86.6% across all PACS domains, outperforming the standard Empirical Risk Minimization (ERM) baseline of 84.8%.
  • Robustness in Hard Domains: The method was particularly effective in the "Cartoon" and "Sketch" domains, which are traditionally difficult for models trained on photographic data. In the Sketch domain, CycleMix reached 79.9% accuracy, a significant improvement over the 77.1% achieved by standard training.
  • Style-Invariance: The results suggest that by treating style as noise through the mixing process, the model is forced to extract deeper, more reliable content features.

Limitations and Future Work

A significant limitation of the current CycleMix implementation is scalability. Because the method requires a CycleGAN for every pair of domains, the number of models needed grows quadratically ($S(S-1)/2$). For a dataset with 10 domains, 45 separate CycleGANs would be required, creating a heavy computational burden.

Future research will explore the integration of StarGAN architectures, which can learn mappings between multiple domains within a single model. This would drastically reduce computational overhead and allow CycleMix to be applied to larger, more complex datasets beyond the PACS benchmark.

Summary updated

Abstract

As deep learning-based systems have become an integral part of everyday life, limitations in their generalization ability have begun to emerge. Machine learning algorithms typically rely on the i.i.d. assumption, meaning that their training and validation data are expected to follow the same distribution, which does not necessarily hold in practice. In the case of image classification, one frequent reason that algorithms fail to generalize is that they rely on spurious correlations present in training data, such as associating image styles with target classes. These associations may not be present in the unseen test data, leading to significant degradation of their effectiveness. In this work, we attempt to mitigate this Domain Generalization (DG) problem by training a robust feature extractor which disregards features attributed to image-style but infers based on style-invariant image representations. To achieve this, we train CycleGAN models to learn the different styles present in the training data and randomly mix them together to create samples with novel style attributes to improve generalization. Experimental results on the PACS DG benchmark validate the proposed method.
BibTeX
@inproceedings{ballas2024cyclemix,
  author = {Aristotelis Ballas and Christos Diou},
  title = {{CycleMix: Mixing Source Domains for Domain Generalization in Style-Dependent Data}},
  booktitle = {Setn},
  year = {2024},
  doi = {10.1145/3688671.3688758},
}
Figure