Unpaired Faces to Cartoons: Improving XGAN

Stev H. Ramos, Joel Cabrera, Daniel Ibáñez, Alejandro B. Jiménez-Panta, César Beltrán Castañón, Edwin VillanuevaWorkshops @ CVPR 2022 · DOI

Abstract

Unsupervised image-to-image translation, such as converting human photos into cartoons, is a complex task due to the lack of paired training examples. This paper introduces W-XDGAN, an evolution of the Cross-GAN (XGAN) architecture. By integrating Wasserstein GAN with Gradient Penalty (WGAN-GP), a Wasserstein-based domain adversarial loss, and a specialized denoising autoencoder, the proposed model overcomes common issues like training instability and visual artifacts. Evaluated on VGG-Face and CartoonSet, W-XDGAN achieves a Fréchet Inception Distance (FID) of 60.43, significantly outperforming the original XGAN and UNIT models.

Introduction

The goal of unsupervised image-to-image translation is to learn a mapping between two domains without direct supervision. The original XGAN framework attempted this by using a shared latent space to capture domain-invariant semantic features. However, it frequently suffered from "mode collapse" due to the standard Binary Cross Entropy loss and produced outputs marred by speckle noise and checkerboard artifacts. W-XDGAN addresses these limitations through architectural refinements and more stable loss functions.

Methodology

W-XDGAN (Wasserstein-Cross-D-GAN) implements several key upgrades to the baseline XGAN framework to improve stability and output quality:

  1. WGAN-GP: The model replaces the standard binary classifier with a "critic" that approximates the Wasserstein distance. This provides smoother gradients, making training much more stable.
  2. Wasserstein Domain Adversarial Loss ($L_{dann}$): In the shared latent space, the authors implemented a network that learns the Wasserstein distance between the face and cartoon distributions. This ensures that internal representations of both domains overlap more effectively.
  3. Post-Generator Denoiser: A dedicated denoising autoencoder was added specifically to clean up the cartoon generator's output. This component removes speckle noise and corrects color bleeding without interfering with the primary generator's gradients.
  4. Spectral Normalization: This technique was applied to the convolutional layers of the discriminator to further stabilize the adversarial training process.
The W-XDGAN architecture, highlighting the shared embedding space and the additional denoising flow.

Datasets

The study utilized two primary datasets, both resized to 64x64 pixels:

  • VGG-Face: A collection of 2,361 high-quality real human frontal-face images.
  • CartoonSet: A massive collection of 35,134 2D cartoon avatar images.

The significant size imbalance (roughly 15:1) between the two datasets presented a unique challenge for domain adaptation, requiring the model to map features accurately despite the disparity in sample availability.

Results and Analysis

W-XDGAN demonstrated substantial improvements both quantitatively and qualitatively.

  • Quantitative Performance: The model achieved a Fréchet Inception Distance (FID) of 60.43, a massive improvement over the original XGAN (159.36) and the competitive UNIT model (114.25).
  • Qualitative Quality: Visual comparisons show that W-XDGAN produces cartoons with much cleaner lines, more accurate skin tones, and fewer artifacts like random glasses or blurred hair compared to its predecessors.
Comparison of face-to-cartoon translation results between XGAN and various W-XDGAN configurations.

Key insights revealed that latent space alignment via the Wasserstein-based $L_{dann}$ loss is critical; without it, the domains fail to align semantically. Furthermore, t-SNE visualizations confirmed that W-XDGAN creates a much tighter and more overlapping cluster of face and cartoon embeddings than the original XGAN, proving superior semantic alignment.

Conclusion and Future Work

W-XDGAN represents a significant step forward in stable, high-quality unsupervised image translation. By addressing the fundamental instabilities of the original XGAN, the authors have created a robust pipeline for stylized generation. While challenges like color bias and low resolution (64x64) remain, the integration of Wasserstein objectives and post-processing denoisers provides a strong foundation. Future work may explore Diffusion Models, which have recently shown even greater potential for stability and diversity in image generation.

Summary updated

BibTeX
@inproceedings{ramos2022unpaired,
  author = {Stev H. Ramos and Joel Cabrera and Daniel Ibáñez and Alejandro B. Jiménez-Panta and César Beltrán Castañón and Edwin Villanueva},
  title = {{Unpaired Faces to Cartoons: Improving XGAN}},
  booktitle = {Workshops @ CVPR},
  year = {2022},
  doi = {10.1109/cvprw56347.2022.00158},
}
Figure