Conditional Coupled Generative Adversarial Networks for Zero-Shot Domain Adaptation

Jinghua Wang, Jianmin Jiang

Abstract

Machine learning models trained in one domain often perform poorly in others due to domain shift. Domain adaptation (DA) techniques address this by training transferable models from a label-rich source domain to a label-scarce target domain. However, most existing DA methods require target-domain data during training. This paper tackles the challenging problem of Zero-Shot Domain Adaptation (ZSDA), where target-domain data is completely unavailable during the training stage. We propose Conditional Coupled Generative Adversarial Networks (CoCoGAN), which extends the CoGAN architecture into a conditioning model. CoCoGAN captures the joint distribution of dual-domain samples across a relevant task (RT) and an irrelevant task (IRT). By training on source-domain RT samples and dual-domain IRT samples, CoCoGAN learns to "hallucinate" target-domain data for the RT. A novel representation alignment signal is introduced to ensure consistency across tasks. Extensive experiments demonstrate that CoCoGAN significantly outperforms state-of-the-art methods in image classification tasks.

Introduction: The Challenge of Zero-Shot Domain Adaptation

Domain shift is a fundamental challenge in machine learning, where a model's performance degrades when tested on data from a distribution different from its training set. While standard Domain Adaptation (DA) mitigates this using target-domain samples, Zero-Shot Domain Adaptation (ZSDA) addresses the more extreme scenario where no target data is available for the task at hand.

Conceptual framework of ZSDA using representation alignment across relevant and irrelevant tasks.

The proposed CoCoGAN framework, developed by researchers at Shenzhen University, leverages a secondary "Irrelevant Task" (IRT) for which dual-domain data is available. By learning the shared correlation between domains in the IRT, the model can apply this "bridge" to the "Relevant Task" (RT), effectively hallucinating the missing target data for training effective classifiers.

Methodology: The CoCoGAN Framework

CoCoGAN builds upon the Coupled GAN (CoGAN) architecture, designed to learn a joint distribution of multi-domain images without requiring corresponding image pairs.

Architecture of the Conditional Coupled Generative Adversarial Network (CoCoGAN) showing weight sharing and task conditioning.

Architecture and Weight Sharing

The framework consists of two parallel GANs: $GAN_s$ for the source domain and $GAN_t$ for the target domain. To force the networks to learn common semantic concepts (e.g., the shape of a digit), the generators and discriminators share weights in their high-level layers. Low-level layers remain unshared to capture domain-specific details like color or texture.

Task Conditioning and Representation Alignment

A binary conditioning variable $c$ allows the network to switch between the RT and the IRT. A critical innovation is the Representation Alignment supervisory signal. Task classifiers are integrated into the network, and the generators are trained to produce features that "fool" these classifiers. This ensures that the internal representations for both tasks are aligned within each domain, preventing the model from becoming biased toward the IRT and ensuring the domain transformation is universal.

Experimental Results

CoCoGAN was evaluated on datasets including MNIST, NIST, and Fashion-MNIST, with synthetic domain shifts such as colorization, edge detection, and intensity inversion.

  • Performance Gains: CoCoGAN consistently outperformed the ZDDA baseline. In NIST letter classification using Fashion-MNIST as the IRT, it achieved a 34.0% improvement.
  • Ablation Study: The representation alignment signal proved vital; its removal led to an average performance drop of 15.6% across all experiments.
  • Qualitative Success: The model successfully generated synthetic target-domain samples that were visually indistinguishable in style from real target data while maintaining correct semantic labels from the source domain.

Conclusion and Future Work

CoCoGAN provides a robust solution for Zero-Shot Domain Adaptation by transferring domain knowledge across tasks. While highly effective, the method assumes a consistent domain transformation across tasks ($T_r = T_{ir}$). Future work may explore more complex domain shifts and higher-resolution natural scenes to further test the framework's scalability and robustness in diverse real-world applications.

Summary updated

Abstract

Machine learning models trained in one domain perform poorly in the other domains due to the existence of domain shift. Domain adaptation techniques solve this problem by training transferable models from the label-rich source domain to the label-scarce target domain. Unfortunately, a majority of the existing domain adaptation techniques rely on the availability of the target-domain data, and thus limit their applications to a small community across few computer vision problems. In this paper, we tackle the challenging zero-shot domain adaptation (ZSDA) problem, where the target-domain data is non-available in the training stage. For this purpose, we propose conditional coupled generative adversarial networks (CoCoGAN) by extending the coupled generative adversarial networks (CoGAN) into a conditioning model. Compared with the existing state of the arts, our proposed CoCoGAN is able to capture the joint distribution of dual-domain samples in two different tasks, i.e. the relevant task (RT) and an irrelevant task (IRT). We train the CoCoGAN with both source-domain samples in RT and the dual-domain samples in IRT to complete the domain adaptation. While the former provide the high-level concepts of the non-available target-domain data, the latter carry the sharing correlation between the two domains in RT and IRT. To train the CoCoGAN in the absence of the target-domain data for RT, we propose a new supervisory signal, i.e. the alignment between representations across tasks. Extensive experiments carried out demonstrate that our proposed CoCoGAN outperforms existing state of the arts in image classifications.
BibTeX
@inproceedings{wang2019conditional,
  author = {Jinghua Wang and Jianmin Jiang},
  title = {{Conditional Coupled Generative Adversarial Networks for Zero-Shot Domain Adaptation}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  url = {https://openreview.net/forum?id=vOCf4fbN4C},
}
Figure