SpaceEdit: Learning a Unified Editing Space for Open-Domain Image Color Editing

Jing Shi, Ning Xu, Haitian Zheng, Alex Smith, Jiebo Luo, Chenliang Xu

Abstract

SpaceEdit introduces a unified, semantic latent space for open-domain image color and tone editing. By training an encoder-decoder model on real-world user edits, the authors create a "style-only" space that enables diverse tasks—from language-guided adjustments to exemplar-based style transfers—while perfectly preserving the original image's structure and content.

Introduction

Traditional image editing tools, such as Adobe Lightroom, rely on an "operation space" consisting of sliders for contrast, brightness, and curves. While powerful, these interfaces can be unintuitive for novices. Conversely, modern generative AI approaches like StyleGAN excel at image synthesis but often struggle with "open-domain" photos—complex, real-world scenes—and frequently fail to preserve the exact structural content of the original image during editing.

SpaceEdit bridges this gap by providing a system that is open-domain, content-preserving, and unified. It handles multiple tasks, including text-to-edit, style transfer, and style retrieval, within a single framework.

Overview of SpaceEdit capabilities including multimodal editing, language-guided edits, and exemplar-based transfer.

Methodology: Decoupling Content and Style

The core of SpaceEdit is an image-to-image translation framework designed to decouple structural content from stylistic transformations.

Architecture

The model utilizes an image encoder paired with a StyleGAN2-based decoder. To ensure high-fidelity content preservation, the standard StyleGAN "constant" input is replaced with the encoder's lowest-resolution feature map. Furthermore, skip connections between the encoder and decoder preserve fine-grained spatial details, preventing the warping of objects during color adjustments.

The SpaceEdit architecture featuring an encoder-decoder setup with skip connections and a conditioned editing space W.

The Unified Editing Space (W)

Unlike standard StyleGAN, where the latent space encodes both content and style, SpaceEdit’s W space is conditioned on the input image. This design ensures that the W space focuses exclusively on the transformation (the edit). This "style-only" space allows for:

  • Supervised Language Editing: Mapping text and image features into a latent code w via an MLP.
  • Zero-Shot Language Editing: Leveraging CLIP to optimize w to match text prompts like "gloomy" or "aurora."
  • Exemplar-based Editing: Extracting a w vector from a pair of before/after images to apply that specific style to new content.

Datasets and Training

The model is trained on the Adobe Discover Dataset, which contains 60,000 pairs of before-and-after photos uploaded by real users to Lightroom. This dataset captures professional-grade, diverse, real-world editing styles. Training employs a conditional GAN loss, and diversity in multimodal editing is encouraged by mapping random noise vectors into the W space.

Key Insights and Results

SpaceEdit demonstrates significant improvements in both quality and semantic alignment:

  • Multimodal Superiority: SpaceEdit achieves an FID of 5.17, significantly outperforming previous state-of-the-art models like BiCycleGAN (FID 12.28).
  • Semantic Factorization: Using unsupervised methods (SeFa), the authors found that the learned space naturally organizes into interpretable axes (e.g., brightness, softness) without explicit supervision.
  • Style Retrieval and Clustering: The latent space achieves a clustering purity score of 12.76, nearly triple that of the raw Lightroom operation space (4.25), proving its alignment with human perception of style.
  • Fidelity: Inversion experiments show the model can reconstruct target edited images with minimal error (2-4 pixels), confirming the W space's completeness for color and tone transformations.

Limitations

While highly effective for color and tone, SpaceEdit is strictly limited to these domains. It cannot perform geometric or content-based edits, such as object removal or changing facial expressions. Additionally, zero-shot language editing is dependent on the internal knowledge of the CLIP model.

Conclusion

SpaceEdit represents a significant step toward intuitive, professional-grade image editing. By creating a unified latent space that separates style from content, it enables a wide array of editing tasks while maintaining the integrity of the original photograph. Future work may extend this framework to video color grading and more complex image harmonization tasks.

Summary updated

Abstract

Recently, large pretrained models (e.g., BERT, Style-GAN, CLIP) show great knowledge transfer and generalization capability on various downstream tasks within their domains. Inspired by these efforts, in this paper we propose a unified model for open-domain image editing focusing on color and tone adjustment of open-domain images while keeping their original content and structure. Our model learns a unified editing space that is more semantic, intu-itive, and easy to manipulate than the operation space (e.g., contrast, brightness, color curve) used in many existing photo editing softwares. Our model belongs to the image-to-image translation framework which consists of an image encoder and decoder, and is trained on pairs of before-and-after edited images to produce multimodal outputs. We show that by inverting image pairs into latent codes of the learned editing space, our model can be leveraged for vari-ous downstream editing tasks such as language-guided image editing, personalized editing, editing-style clustering, retrieval, etc. We extensively study the unique properties of the editing space in experiments and demonstrate superior performance on the aforementioned tasks <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Code and supplementary material can be found at the project page https://jshi31.github.io/SpaceEdit.
BibTeX
@inproceedings{shi2022spaceedit,
  author = {Jing Shi and Ning Xu and Haitian Zheng and Alex Smith and Jiebo Luo and Chenliang Xu},
  title = {{SpaceEdit: Learning a Unified Editing Space for Open-Domain Image Color Editing}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2022},
  doi = {10.1109/cvpr52688.2022.01911},
}
Figure