IntersectGAN: Learning Domain Intersection for Generating Images with Multiple Attributes

Zehui Yao, Boyan Zhang, Zhiyong Wang, Wanli Ouyang, Dong Xu, Dagan Feng

Abstract

Generating images with multiple specific attributes typically requires large datasets containing those exact combinations. However, as the number of attributes increases, available real-world samples decrease exponentially. We present IntersectGAN, a generative framework that learns to synthesize images at the intersection of multiple domains using only single-attribute datasets. By training an intersection generator to simultaneously satisfy multiple discriminators—each specialized in a single attribute—the model effectively learns complex attribute combinations without direct multi-attribute supervision.

Introduction

The primary bottleneck in training Generative Adversarial Networks (GANs) for multi-attribute image synthesis is the scarcity of data. For instance, while thousands of images of "smiling people" or "people with black hair" exist, the subset of "smiling people with black hair" is significantly smaller. As shown in Figure 1, the number of available samples drops sharply as more attributes are required, making it difficult to scale traditional supervised models.

Attribute distribution showing the scarcity of multi-attribute samples as complexity increases.

Methods

IntersectGAN employs an "intersecting architecture" to bypass the need for multi-attribute training data. The framework consists of:

  • $n$ Parallel Generators ($G_1, ..., G_n$): Each generator is dedicated to a single attribute domain (e.g., $G_1$ for "smiling").
  • One Intersection Generator ($G_Y$): This generator receives the same noise input as the specialized generators but is tasked with producing images that belong to the intersection of all $n$ domains.
  • $n$ Discriminators ($D_1, ..., D_n$): Each $D_i$ is trained to distinguish real samples of attribute $i$ from fake samples produced by both the specialized generator $G_i$ and the intersection generator $G_Y$.

To succeed, $G_Y$ must minimize a combined adversarial loss across all discriminators. Because it must fool the "smiling" expert and the "male" expert simultaneously, it is forced to generate an image that satisfies both criteria at once, as illustrated in Figure 2.

Conceptual diagram of IntersectGAN finding the intersection of 'smiling' and 'male' domains.

Key Results

The model was evaluated on synthetic colored squares, CelebA, and MNIST datasets. Key findings include:

  • Implicit Learning: The model successfully learns the overlap between independent concepts without seeing them together in the training set.
  • Quantitative Performance: IntersectGAN outperformed Conditional GAN (cGAN) and D2GAN in FID scores. For the "Black hair + Male" category, IntersectGAN achieved an FID of 77.9, compared to cGAN's 84.6.
  • Human Evaluation: In Amazon Mechanical Turk studies, users preferred IntersectGAN's realism. For the "Black hair + Smiling" combination, the model achieved a 52.1% realism preference, nearly double that of some baselines.
  • Attribute Accuracy: The model's "Attribute Score" (the percentage of generated images that humans agreed had the target features) was often double or triple that of the ground truth distribution in the original dataset.

Conclusion and Future Work

IntersectGAN demonstrates a robust method for multi-attribute image generation using limited, single-attribute supervision. While the current architecture faces potential scalability challenges regarding the number of parallel components, it offers a significant step toward more data-efficient generative modeling. Future work will explore more scalable network architectures to reduce the overhead of the parallel generator/discriminator design.

Summary updated

BibTeX
@inproceedings{yao2019intersectgan,
  author = {Zehui Yao and Boyan Zhang and Zhiyong Wang and Wanli Ouyang and Dong Xu and Dagan Feng},
  title = {{IntersectGAN: Learning Domain Intersection for Generating Images with Multiple Attributes}},
  booktitle = {ACM Multimedia},
  year = {2019},
  doi = {10.1145/3343031.3350908},
}
Figure