Label Geometry Aware Discriminator for Conditional Generative Networks

Suman Sapkota, Bidur Khanal, Binod Bhattarai, Bishesh Khanal, Tae-Kyun KimarXiv 2021 · OpenReview

TL;DR

This paper introduces a novel "Label Geometry Aware Discriminator" for conditional GANs that improves synthetic image quality for downstream tasks. By replacing the standard softmax loss with an additive angular margin (AAM) loss in the discriminator's auxiliary classifier, the model better captures the intrinsic angular distribution of deep features, leading to more faithful and diverse image generation.

What Problem Does This Paper Address?

Conditional Generative Adversarial Networks (cGANs) are great at generating realistic images for specific tasks, like translating a face image to show a different emotion. However, a big challenge is that these synthetic images, even if they look good, don't always help improve performance on downstream tasks like image classification. The core issue is that current methods often struggle to generate images that are both high-fidelity (look real and match the target class) and diverse (cover the full range of the target distribution). Existing approaches often use an auxiliary classifier within the discriminator, typically with a softmax loss. The authors argue that this softmax loss, operating in Euclidean space, doesn't fully leverage the inherent angular relationships within deep features, which are crucial for distinguishing subtle variations in attributes like facial expressions.

Methods

The paper's main contribution is the introduction of an Additive Angular Margin (AAM) loss for the auxiliary classifier within the discriminator of a conditional GAN.

Here's a breakdown:

  • Standard cGAN Setup: The framework uses a generator (G) and a discriminator (D). G takes an input image and a target class label to produce a synthetic image. D tries to distinguish real from fake images and also classifies the target attribute of an image.
  • Auxiliary Classifier: In many cGANs (like StarGAN), the discriminator includes an auxiliary classifier to ensure generated images adhere to the target class. This is typically trained with a softmax cross-entropy loss.
  • The Novelty: AAM Loss: The authors propose replacing this standard softmax loss with their AAM loss. The key idea is to leverage the geometric properties of deep features. They normalize feature vectors and weight matrices to lie on a hypersphere. The AAM loss then introduces an angular margin (m) between the feature vector and the correct class's weight vector, and also pushes it away from incorrect class weight vectors. This encourages:
  • Intra-class compactness: Features belonging to the same class are clustered closer together.
  • Inter-class separation: Features from different classes are pushed further apart.
  • Benefits: This geometric constraint is hypothesized to help the generator produce images with higher fidelity to the target class and better coverage of the conditional distribution. It also makes the discriminator more robust.

Datasets

The authors evaluate their method on two distinct datasets:

  • RaFD (Real-world Affective Faces Database): This dataset contains facial expression images, making it suitable for evaluating attribute translation tasks (e.g., changing a neutral face to an angry one).
  • CIFAR-100: A standard natural image dataset with 100 classes, used to test the generalizability of the approach on a more diverse and challenging classification task.

Key Insights and Ideas

  • Geometry Matters: The core insight is that the angular relationships between deep features are critical for accurate attribute classification and, by extension, for generating high-quality synthetic images that preserve these attributes. Standard Euclidean-space losses might miss this crucial geometric information.
  • "Seeing is Not Believing": The paper reinforces the idea that visually pleasing synthetic images don't automatically translate to improved downstream task performance. This highlights the need for metrics and methods that specifically assess the utility of synthetic data for training.
  • AAM Loss for GANs: Adapting angular margin losses, which have shown success in supervised classification, to the discriminator of a cGAN is a novel approach. The authors demonstrate its effectiveness in improving the generator's output.
  • Robustness to Noise: The proposed method shows improved robustness to label distortion in synthetic data, suggesting that the learned representations are more stable and less susceptible to noisy labels.

Key Results

The proposed method, incorporating the AAM loss, consistently outperforms baseline methods, particularly StarGAN, across various metrics:

  • CIFAR-100:
    • GAN-Train: Achieved a significant improvement of +7% over StarGAN.
  • GAN-Test: Showed a substantial gain of +13.5% over StarGAN.
  • FID: Achieved a lower FID score (53.73 vs. 66.74), indicating better image quality.
  • RaFD:
    • GAN-Train: Improved by +1.87% over StarGAN.
  • GAN-Test: Showed a significant improvement of +13.5% over StarGAN.
  • FID: Achieved a lower FID score (5.35 vs. 6.14), indicating better image quality.
  • Qualitative Results: Figures show that the AAM-based method generates images with fewer artifacts, better contrast, and more intense attribute preservation (e.g., more distinct facial expressions).
  • Data Augmentation: When used for data augmentation, synthetic images generated by the proposed method led to better attribute classification accuracy on real test data compared to those generated by StarGAN.

Limitations and Future Work

  • Computational Cost: While not explicitly detailed as a limitation, the paper mentions that TAC-GAN requires extra learnable parameters. The AAM loss, while effective, might add some computational overhead compared to a simple softmax.
  • Understanding GAN Dynamics: The authors acknowledge that a complete understanding of the complex interactions between generators and discriminators in non-stationary environments, especially when equipped with classifiers, remains an open problem.
  • Future Work:
    • Combining regularization techniques (like those used in SNGAN) with their proposed AAM loss could potentially further improve performance.
  • A deeper analysis of the interplay between different components of the GAN and the impact of various regularization constraints through ablation studies is suggested.
  • Exploring the dynamics of conditional GAN training with these improved discriminators is an interesting future direction.

Summary updated

Abstract

Multi-domain image-to-image translation with conditional Generative Adversarial Networks (GANs) can generate highly photo realistic images with desired target classes, yet these synthetic images have not always been helpful to improve downstream supervised tasks such as image classification. Improving downstream tasks with synthetic examples requires generating images with high fidelity to the unknown conditional distribution of the target class, which many labeled conditional GANs attempt to achieve by adding soft-max cross-entropy loss based auxiliary classifier in the discriminator. As recent studies suggest that the soft-max loss in Euclidean space of deep feature does not leverage their intrinsic angular distribution, we propose to replace this loss in auxiliary classifier with an additive angular margin (AAM) loss that takes benefit of the intrinsic angular distribution, and promotes intra-class compactness and inter-class separation to help generator synthesize high fidelity images. We validate our method on RaFD and CIFAR-100, two challenging face expression and natural image classification data set. Our method outperforms state-of-the-art methods in several different evaluation criteria including recently proposed GAN-train and GAN-test metrics designed to assess the impact of synthetic data on downstream classification task, assessing the usefulness in data augmentation for supervised tasks with prediction accuracy score and average confidence score, and the well known FID metric.
BibTeX
@misc{sapkota2021label,
  author = {Suman Sapkota and Bidur Khanal and Binod Bhattarai and Bishesh Khanal and Tae-Kyun Kim},
  title = {{Label Geometry Aware Discriminator for Conditional Generative Networks}},
  year = {2021},
  eprint = {2105.05501},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2105.05501},
}
Figure