CatGAN: Coupled Adversarial Transfer for Domain Generation.

Shanshan Wang, Lei Zhang

Abstract

This paper introduces a Coupled adversarial transfer GAN (CatGAN), an efficient solution to domain alignment. The basic principles of CatGAN focus on the domain generation strategy for adaptation which is motivated by the generative adversarial net (GAN) and the adversarial discriminative domain adaptation (ADDA). CatGAN is structured by shallow multilayer perceptrons (MLPs) for adversarial domain adaptation. The CatGAN comprises of two slim and symmetric subnetworks, which then formulates a coupled adversarial learning framework. With such symmetry, the input images from source/target domain can be fed into the MLP network for target/source domain generation, supervised by the coupled discriminators for confrontation. Notably, each generator contains GAN loss and domain loss to guarantee the simple network work well. The content fidelity term aims at preserving the domain specific knowledge during generation. Another finding is that the class-wise CatGAN is an effective alternative to conditional GAN without label constraint in generative learning. We show experimentally that the proposed model achieves competitive performance with state-of-the art approaches.

Introduction

In machine learning, the domain shift problem presents a significant challenge: a classifier trained on one dataset (the source) often performs poorly on a related but different dataset (the target). For instance, a model trained on the clean MNIST digit dataset may struggle with the Street View House Numbers (SVHN) dataset due to differences in background, lighting, and noise.

While humans can easily generalize between these domains, machines require explicit Domain Adaptation (DA) to align these distributions. Many existing DA methods rely on complex, deep neural networks that are computationally expensive. The authors of CatGAN argue that for effective domain alignment, high-fidelity image generation is secondary to distribution alignment. Consequently, they propose a "slim" architecture that achieves state-of-the-art results without the heavy overhead of deep generative models.

The CatGAN Framework

The core of CatGAN is its Coupled and Symmetric architecture. Unlike standard GANs that map random noise to images, CatGAN maps data from one domain directly to another to bridge the gap between datasets.

Comparison of GAN architectures: (a) Standard GAN, (b) Conditional GAN, and (c) the proposed CatGAN.

As shown in the comparison above, CatGAN utilizes a two-way generation strategy. The model consists of two sub-networks: one learns the mapping from Source to Target ($S \to T$), and the other learns the reverse ($T \to S$). This bidirectional approach creates a stable mapping that constrains the latent space more effectively than one-way models.

Architecture and Optimization

CatGAN is designed for efficiency, utilizing shallow multilayer perceptrons (MLPs) rather than deep convolutional layers. The generators typically consist of 2-layer MLPs, while the discriminators use 3-layer structures.

Internal structure of the CatGAN generator and discriminator, highlighting the use of shallow MLPs and the dual loss strategy.

To ensure the generated data is useful for alignment, the framework optimizes a triple loss function:

  1. Adversarial Loss: The standard GAN objective where generators attempt to fool discriminators.
  2. Domain Loss: A specific term designed to minimize the distance between the generated data's distribution and the target domain's center.
  3. Content Fidelity Loss: A constraint that ensures the "content" of the data is preserved during transfer. If a source image is transformed to the target domain and then back again, it should return to its original form.

Additionally, the authors propose a Class-wise CatGAN. By training separate small networks for each class, they provide an effective alternative to Conditional GANs, which can be difficult to optimize under certain label constraints.

Experimental Results

The researchers evaluated CatGAN across several classic domain adaptation benchmarks, including COIL-20, MSRC vs. VOC 2007, and the Office-Caltech (4DA) dataset. The results demonstrate that CatGAN's focus on distribution alignment over visual realism pays off:

  • COIL-20: Achieved 92.4% average accuracy, an 8% improvement over previous shallow methods.
  • MSRC vs. VOC: Reached 63.6% accuracy, significantly outperforming other shallow DA approaches.
  • Digits: On the challenging SVHN to MNIST transfer, CatGAN achieved 92.1% accuracy, notably outperforming deeper adversarial methods like ADDA (76.0%).
  • Deep Benchmarks: Using DeCAF features on the 4DA task, CatGAN reached 92.3%, proving it can compete with deep transfer learning models like the Residual Transfer Network (RTN).

Conclusion

CatGAN demonstrates that shallow, symmetric MLP structures can be highly effective for domain adaptation. By prioritizing distribution alignment and utilizing a coupled adversarial framework, it offers a lightweight alternative to deep generative models. While the class-wise approach may face scaling challenges with very large numbers of categories, the flexibility of the CatGAN framework provides a strong foundation for future research in efficient domain generation and open-set adaptation.

Summary updated

Abstract

This paper introduces a Coupled adversarial transfer GAN (CatGAN), an efficient solution to domain alignment. The basic principles of CatGAN focus on the domain generation strategy for adaptation which is motivated by the generative adversarial net (GAN) and the adversarial discriminative domain adaptation (ADDA). CatGAN is structured by shallow multilayer perceptrons (MLPs) for adversarial domain adaptation. The CatGAN comprises of two slim and symmetric subnetworks, which then formulates a coupled adversarial learning framework. With such symmetry, the input images from source/target domain can be fed into the MLP network for target/source domain generation, supervised by the coupled discriminators for confrontation. Notablely, each generator contains GAN loss and domain loss to guarantee the simple network work well. The content fidelity term aims at preserving the domain specific knowledge during generation. Another finding is that the class-wise CatGAN is an effective alternative to conditional GAN without label constraint in generative learning. We show experimentally that the proposed model achieves competitive performance with state-of-the art approaches.
BibTeX
@misc{wang2017catgan,
  author = {Shanshan Wang and Lei Zhang},
  title = {{CatGAN: Coupled Adversarial Transfer for Domain Generation.}},
  year = {2017},
  eprint = {1711.08904},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1711.08904},
}
Figure