TL;DR
This paper introduces UGAN, a novel Generative Adversarial Network (GAN) for multi-domain image-to-image translation, specifically focusing on face editing. UGAN tackles the common problem of "source retaining" where translated images still exhibit characteristics of the original domain, by employing a unique source classifier and a target prototype injection mechanism to ensure the translated image is truly "target-only."
What Problem Does This Paper Address?
Multi-domain image-to-image translation aims to transform an image from one domain to another (e.g., changing a face's age, applying makeup, or altering expression). The ideal outcome is an image that fully embodies the target domain's characteristics while completely shedding those of the source domain. However, existing methods often fail at this, a phenomenon termed "source retaining." For instance, when aging a face, the translated image might still show features of the original age, like wrinkles or beard stubble, even when it's supposed to represent a much younger person. This is because standard domain classifiers used in these models are not sensitive enough to detect these lingering source characteristics.
Methods
UGAN introduces two key innovations to combat source retaining:
- Untraceable Constraint with a Novel Source Classifier: Instead of a standard domain classifier that just identifies the target domain, UGAN uses a discriminator with a specialized "source classifier." This classifier is trained to determine which domain an image was translated from. The generator is then trained to "fool" this source classifier, making it impossible to trace the original source domain of the generated image. This forces the generator to effectively erase source characteristics.
- Prototype Injection for Target Characteristics: To ensure the generated image accurately reflects the target domain, UGAN incorporates "prototypes." These are essentially average images representing the essential characteristics of each target domain (e.g., the typical round face of a child). An "Adaptive Prototype Injection" (API) module is used to intelligently blend these prototype features with the source image's features, ensuring the target characteristics are naturally integrated.
Datasets
The paper evaluates UGAN on three established face editing datasets:
- Face Aging: A dataset with 15,030 face images divided into 7 age groups.
- MAKEUP-A5: A dataset of 6,095 Asian women's faces with 5 makeup categories.
- CFEE: A dataset of 22 facial expressions with 5,060 images.
Key Insights and Ideas
- The "Source Retaining" Problem is More Than Just Misclassification: The paper highlights that standard domain classifiers are insufficient because they don't penalize generated images that look like the target domain but still contain source domain artifacts.
- Explicitly Erasing Source is Key: The core idea is that simply aiming for the target isn't enough; actively forcing the removal of source attributes is crucial for high-quality translation.
- Prototypes as "Essence" of a Domain: Using average images (prototypes) as a guide for target characteristics is an effective way to inject domain-specific features, especially when dealing with subtle but important attributes like facial structure.
- Adaptive Injection Matters: Simply concatenating or summing features from the source and prototype doesn't work well. An adaptive mechanism is needed to blend them harmoniously.
Key Results
UGAN consistently outperforms the state-of-the-art StarGAN across all three tasks:
- Face Aging: UGAN achieved a significant drop in mean Intra-FID scores (from 41.9 for StarGAN to 29.2 for UGAN‡, and 27.4 for the full UGAN model), indicating much better age translation. Qualitative results show UGAN producing more realistic children with round faces and older individuals with wrinkles and white hair, while StarGAN often retains adult features.
- Makeup Editing: UGAN also showed superior performance with a mean Intra-FID drop from 89.6 (StarGAN) to 71.3 (UGAN). Qualitative results demonstrate UGAN's ability to remove makeup naturally and apply specific makeup styles accurately.
- Expression Editing: UGAN demonstrated strong performance in editing 22 fine-grained facial expressions, capturing subtle differences and accurately synthesizing expressions without retaining original facial cues.
- AMT User Studies: Human evaluations confirmed UGAN's superiority, with users consistently preferring UGAN's results over StarGAN's across all tested categories.
Limitations and Future Work
- Computational Cost: While not explicitly detailed as a limitation, the paper mentions using API on low-resolution feature maps due to the computational expense of correlation matrix computation.
- Broader Applications: The authors suggest that the core ideas of UGAN – specifically the untraceable constraint and prototype injection – could be applied to other domains beyond face editing, such as language translation, to improve quality by reducing source language interference. They plan to explore these broader applications.
Summary updated