UGAN: Untraceable GAN for Multi-Domain Face Translation

Defa Zhu, Si Liu, Wentao Jiang, Chen Gao, Tianyi Wu, Guodong GuoarXiv 2019 · OpenReview

TL;DR

This paper introduces UGAN, a novel Generative Adversarial Network (GAN) for multi-domain image-to-image translation, specifically focusing on face editing. UGAN tackles the common problem of "source retaining" where translated images still exhibit characteristics of the original domain, by employing a unique source classifier and a target prototype injection mechanism to ensure the translated image is truly "target-only."

What Problem Does This Paper Address?

Multi-domain image-to-image translation aims to transform an image from one domain to another (e.g., changing a face's age, applying makeup, or altering expression). The ideal outcome is an image that fully embodies the target domain's characteristics while completely shedding those of the source domain. However, existing methods often fail at this, a phenomenon termed "source retaining." For instance, when aging a face, the translated image might still show features of the original age, like wrinkles or beard stubble, even when it's supposed to represent a much younger person. This is because standard domain classifiers used in these models are not sensitive enough to detect these lingering source characteristics.

Methods

UGAN introduces two key innovations to combat source retaining:

  1. Untraceable Constraint with a Novel Source Classifier: Instead of a standard domain classifier that just identifies the target domain, UGAN uses a discriminator with a specialized "source classifier." This classifier is trained to determine which domain an image was translated from. The generator is then trained to "fool" this source classifier, making it impossible to trace the original source domain of the generated image. This forces the generator to effectively erase source characteristics.
  2. Prototype Injection for Target Characteristics: To ensure the generated image accurately reflects the target domain, UGAN incorporates "prototypes." These are essentially average images representing the essential characteristics of each target domain (e.g., the typical round face of a child). An "Adaptive Prototype Injection" (API) module is used to intelligently blend these prototype features with the source image's features, ensuring the target characteristics are naturally integrated.

Datasets

The paper evaluates UGAN on three established face editing datasets:

  • Face Aging: A dataset with 15,030 face images divided into 7 age groups.
  • MAKEUP-A5: A dataset of 6,095 Asian women's faces with 5 makeup categories.
  • CFEE: A dataset of 22 facial expressions with 5,060 images.

Key Insights and Ideas

  • The "Source Retaining" Problem is More Than Just Misclassification: The paper highlights that standard domain classifiers are insufficient because they don't penalize generated images that look like the target domain but still contain source domain artifacts.
  • Explicitly Erasing Source is Key: The core idea is that simply aiming for the target isn't enough; actively forcing the removal of source attributes is crucial for high-quality translation.
  • Prototypes as "Essence" of a Domain: Using average images (prototypes) as a guide for target characteristics is an effective way to inject domain-specific features, especially when dealing with subtle but important attributes like facial structure.
  • Adaptive Injection Matters: Simply concatenating or summing features from the source and prototype doesn't work well. An adaptive mechanism is needed to blend them harmoniously.

Key Results

UGAN consistently outperforms the state-of-the-art StarGAN across all three tasks:

  • Face Aging: UGAN achieved a significant drop in mean Intra-FID scores (from 41.9 for StarGAN to 29.2 for UGAN‡, and 27.4 for the full UGAN model), indicating much better age translation. Qualitative results show UGAN producing more realistic children with round faces and older individuals with wrinkles and white hair, while StarGAN often retains adult features.
  • Makeup Editing: UGAN also showed superior performance with a mean Intra-FID drop from 89.6 (StarGAN) to 71.3 (UGAN). Qualitative results demonstrate UGAN's ability to remove makeup naturally and apply specific makeup styles accurately.
  • Expression Editing: UGAN demonstrated strong performance in editing 22 fine-grained facial expressions, capturing subtle differences and accurately synthesizing expressions without retaining original facial cues.
  • AMT User Studies: Human evaluations confirmed UGAN's superiority, with users consistently preferring UGAN's results over StarGAN's across all tested categories.

Limitations and Future Work

  • Computational Cost: While not explicitly detailed as a limitation, the paper mentions using API on low-resolution feature maps due to the computational expense of correlation matrix computation.
  • Broader Applications: The authors suggest that the core ideas of UGAN – specifically the untraceable constraint and prototype injection – could be applied to other domains beyond face editing, such as language translation, to improve quality by reducing source language interference. They plan to explore these broader applications.

Summary updated

Abstract

The multi-domain image-to-image translation is a challenging task where the goal is to translate an image into multiple different domains. The target-only characteristics are desired for translated images, while the source-only characteristics should be erased. However, recent methods often suffer from retaining the characteristics of the source domain, which are incompatible with the target domain. To address this issue, we propose a method called Untraceable GAN, which has a novel source classifier to differentiate which domain an image is translated from, and determines whether the translated image still retains the characteristics of the source domain. Furthermore, we take the prototype of the target domain as the guidance for the translator to effectively synthesize the target-only characteristics. The translator is learned to synthesize the target-only characteristics and make the source domain untraceable for the discriminator, so that the source-only characteristics are erased. Finally, extensive experiments on three face editing tasks, including face aging, makeup, and expression editing, show that the proposed UGAN can produce superior results over the state-of-the-art models. The source code will be released.
BibTeX
@misc{zhu2019ugan,
  author = {Defa Zhu and Si Liu and Wentao Jiang and Chen Gao and Tianyi Wu and Guodong Guo},
  title = {{UGAN: Untraceable GAN for Multi-Domain Face Translation}},
  year = {2019},
  eprint = {1907.11418},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1907.11418},
}
Figure