Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigation

Peiye Zhuang, Oluwasanmi Koyejo, Alexander G. Schwing

Abstract

Controllable semantic image editing enables a user to change entire image attributes with a few clicks, e.g., gradually making a summer scene look like it was taken in winter. Classic approaches for this task use a Generative Adversarial Net (GAN) to learn a latent space and suitable latent-space transformations. However, current approaches often suffer from attribute edits that are entangled, global image identity changes, and diminished photo-realism. To address these concerns, we learn multiple attribute transformations simultaneously, integrate attribute regression into the training of transformation functions, and apply a content loss and an adversarial loss that encourages the maintenance of image identity and photo-realism. We propose quantitative evaluation strategies for measuring controllable editing performance, unlike prior work, which primarily focuses on qualitative evaluation. Our model permits better control for both single- and multiple-attribute editing while preserving image identity and realism during transformation. We provide empirical results for both natural and synthetic images, highlighting that our model achieves state-of-the-art performance for targeted image manipulation.

Introduction

Generative Adversarial Networks (GANs), such as StyleGAN2, have demonstrated remarkable capabilities in synthesizing high-resolution, photorealistic images. However, performing precise semantic edits—such as adjusting a person's age or changing the weather in a landscape—remains a significant challenge. Existing latent-space navigation methods often suffer from attribute entanglement (where changing one feature inadvertently alters others), identity loss, and the introduction of visual artifacts. This paper proposes a supervised framework to discover disentangled, continuous editing directions in the latent space of pre-trained GANs.

Methodology

The proposed approach navigates the latent space by training transformation functions ($T$) guided by three primary components:

  1. Attribute Regression: A pre-trained regressor ($R$) provides a continuous "ground truth" for target attributes. By optimizing the latent direction to move the regressor's output by a specific scalar $\epsilon$, the model enables fine-grained, non-binary control over edits.
  2. Joint-Distribution Training: Rather than optimizing directions in isolation, the framework trains multiple attribute transformations simultaneously. This joint optimization forces the model to find orthogonal directions, effectively minimizing entanglement between correlated features.
  3. Multi-Part Loss Function:
    • Regression Loss ($L_{reg}$): Drives the intended semantic change.
  4. Adversarial Loss ($L_{disc}$): Leverages the GAN's discriminator to maintain photorealism and prevent artifacts.
  5. Content/Perceptual Loss ($L_{content}$): Ensures that the core identity and background of the image remain stable during the editing process.

Additionally, the authors explore Local transformations, where the editing direction is a function of the specific input image (modeled via an MLP), allowing for more nuanced, personalized edits compared to standard global directions.

Datasets and Experimental Setup

The framework was validated across diverse domains using pre-trained StyleGAN2 and PGGAN weights. Human face editing was evaluated on FFHQ, CelebA, and CelebA-HQ. Natural scene manipulation focused on the Transient Attribute Database (8,571 images with 40 attributes) and MIT Places2.

Results and Discussion

The proposed method demonstrates state-of-the-art performance in both quantitative and qualitative evaluations.

Quantitative Performance

Unlike prior work that relies heavily on visual inspection, this study introduces rigorous metrics for identity preservation and attribute independence. Using cosine similarity from face recognition models (e.g., VGGFace2), the authors show that their method maintains subject identity significantly better than baselines. Furthermore, tests for attribute independence confirm that target edits (e.g., "Smile") result in minimal unintended fluctuations in non-target features (e.g., "Gender" or "Age").

Qualitative Realism

Visual comparisons highlight the model's ability to produce clean transitions. While competing methods often introduce "orange spots" in weather edits or "glasses" during age transitions due to dataset correlations, the joint training strategy successfully disentangles these features, preserving the integrity of the original image and avoiding un-photorealistic artifacts.

Conclusion and Future Work

This work establishes that disentanglement in GAN latent spaces is not merely a property of the space itself but a result of the training strategy used to navigate it. By treating the regressor as a "compass" and training attributes jointly, the model achieves unprecedented control. Future research will address the dependence on GAN inversion for real-world photo editing and the potential for algorithmic bias inherited from pre-trained regressors.

Summary updated

Abstract

Controllable semantic image editing enables a user to change entire image attributes with a few clicks, e.g., gradually making a summer scene look like it was taken in winter. Classic approaches for this task use a Generative Adversarial Net (GAN) to learn a latent space and suitable latent-space transformations. However, current approaches often suffer from attribute edits that are entangled, global image identity changes, and diminished photo-realism. To address these concerns, we learn multiple attribute transformations simultaneously, integrate attribute regression into the training of transformation functions, and apply a content loss and an adversarial loss that encourages the maintenance of image identity and photo-realism. We propose quantitative evaluation strategies for measuring controllable editing performance, unlike prior work, which primarily focuses on qualitative evaluation. Our model permits better control for both single- and multiple-attribute editing while preserving image identity and realism during transformation. We provide empirical results for both natural and synthetic images, highlighting that our model achieves state-of-the-art performance for targeted image manipulation.
BibTeX
@inproceedings{zhuang2021enjoy,
  author = {Peiye Zhuang and Oluwasanmi Koyejo and Alexander G. Schwing},
  title = {{Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigation}},
  booktitle = {International Conference on Learning Representations},
  year = {2021},
  url = {https://openreview.net/forum?id=KSGbhMtfZi},
}
Figure