Selective Focusing Learning in Conditional GANs

Kyeongbo Kong, Kyunghun Kim, Woo-Jin Song, Suk-Ju KangarXiv 2021 · arXiv · OpenReview

Abstract

Conditional generative adversarial networks (cGANs) have demonstrated remarkable success due to their class-wise controllability and superior quality for complex generation tasks. Typical cGANs solve the joint distribution matching problem by decomposing two easier sub-problems: marginal matching and conditional matching. From our toy experiments, we found that it is the best to apply only conditional matching to certain samples due to the content-aware optimization of the discriminator. This paper proposes a simple (a few lines of code) but effective training methodology, selective focusing learning, which enforces the discriminator and generator to learn easy samples of each class rapidly while maintaining diversity. Our key idea is to selectively apply conditional and joint matching for the data in each mini-batch. We conducted experiments on recent cGAN variants in ImageNet (64x64 and 128x128), CIFAR-10, and CIFAR-100 datasets, and improved the performance significantly (up to 35.18% in terms of FID) without sacrificing diversity.

Introduction

Conditional Generative Adversarial Networks (cGANs) are essential for modern image synthesis, enabling the generation of high-quality images conditioned on specific class labels. However, training cGANs remains challenging, often plagued by "mode collapse"—where the generator produces limited varieties of images—or a difficult trade-off between image fidelity and diversity.

Standard cGAN training typically addresses two sub-problems: marginal matching (ensuring realism regardless of class) and conditional matching (ensuring images correspond to their labels). Most frameworks employ "joint matching," attempting to solve both simultaneously for every sample. However, discriminators naturally prioritize easy, common patterns. Forcing joint matching on all samples can hinder the generator's ability to master high-fidelity, class-specific details, leading to training instability.

Methodology: Selective Focusing Learning (SFL)

To address these challenges, we introduce Selective Focusing Learning (SFL). SFL is a training methodology that prioritizes "easy" samples for class-specific learning while maintaining standard joint matching for the remainder of the dataset.

Conceptual overview of matching strategies: Marginal, Conditional, Joint, and the proposed Selective Focusing Learning (SFL).

The SFL process involves three key steps:

  1. Selection: In each training iteration, samples in a mini-batch are ranked based on their "conditional scores" from the discriminator, identifying the "easiest" samples.
  2. Selective Matching:
    • Easy Samples: These are updated using only conditional matching. By removing the marginal matching constraint, the generator can more rapidly learn high-fidelity, class-specific features.
  3. Remaining Samples: These undergo standard joint matching (marginal + conditional) to ensure the model captures the full diversity of the data distribution.
  4. SFL+ Guidance: In early training stages where the discriminator may be unreliable, a pretrained classifier can be used to rank real samples, providing more stable guidance.

Experimental Results

We evaluated SFL across several benchmark datasets, including ImageNet (64×64 and 128×128), CIFAR-10, and CIFAR-100. The methodology was integrated into state-of-the-art architectures such as SA-GAN and BigGAN.

Quantitative comparison of FID scores across different matching strategies, showing SFL achieves the lowest (best) score.

The results demonstrate significant improvements in image quality. On ImageNet 128×128, SFL+ reduced the Fréchet Inception Distance (FID) from 17.55 to 10.34. Furthermore, SFL proved robust across multiple trials and showed particular strength in scenarios with coarse labels (e.g., broad categories rather than specific species), where it helped the model find stable ground more effectively than standard joint matching.

Discussion and Conclusion

SFL leverages the content-aware nature of discriminators to accelerate the learning of high-quality features without sacrificing diversity. Its primary advantage lies in its simplicity; it requires minimal code changes and adds negligible computational overhead (approximately 0.3% extra training time).

While SFL is highly effective, its performance can degrade in the presence of high label noise. Future work will focus on dynamic adjustment of the focusing rate and extending this selective logic to other generative frameworks. By stabilizing the training process and enhancing fidelity, SFL represents a significant step forward in conditional image synthesis.

Summary updated

Abstract

Conditional generative adversarial networks (cGANs) have demonstrated remarkable success due to their class-wise controllability and superior quality for complex generation tasks. Typical cGANs solve the joint distribution matching problem by decomposing two easier sub-problems: marginal matching and conditional matching. From our toy experiments, we found that it is the best to apply only conditional matching to certain samples due to the content-aware optimization of the discriminator. This paper proposes a simple (a few lines of code) but effective training methodology, selective focusing learning, which enforces the discriminator and generator to learn easy samples of each class rapidly while maintaining diversity. Our key idea is to selectively apply conditional and joint matching for the data in each mini-batch. We conducted experiments on recent cGAN variants in ImageNet (64x64 and 128x128), CIFAR-10, and CIFAR-100 datasets, and improved the performance significantly (up to 35.18% in terms of FID) without sacrificing diversity.
BibTeX
@misc{kong2021selective,
  author = {Kyeongbo Kong and Kyunghun Kim and Woo-Jin Song and Suk-Ju Kang},
  title = {{Selective Focusing Learning in Conditional GANs}},
  year = {2021},
  eprint = {2107.08792},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2107.08792},
}
Figure