WINE: Wavelet-Guided GAN Inversion and Editing for High-Fidelity Refinement

Chaewon Kim, Seung Jun Moon, Gyeong-Moon ParkWACV 2025 · OpenReview

TL;DR

This paper introduces WINE, a novel GAN inversion and editing method that leverages wavelet transforms to explicitly address the low-frequency bias in existing models. By focusing on high-frequency details through a new "wavelet loss" and "wavelet fusion" technique, WINE achieves state-of-the-art results in reconstructing and editing images with remarkable fidelity.

What Problem Does This Paper Address?

Generative Adversarial Networks (GANs) have become incredibly powerful at generating realistic images. However, the inverse problem – taking a real image and finding the latent code that a GAN would use to generate it – is challenging. Existing GAN inversion methods often struggle to reconstruct fine-grained details like textures, sharp edges, or small features. This is because many methods, particularly those relying on pixel-wise losses (like L1 or L2), are inherently biased towards preserving low-frequency information (overall shapes and colors) while neglecting high-frequency details. This "frequency bias" leads to blurry results or loss of intricate image information, especially in complex images. While some methods try to mitigate this by using higher-dimensional latent spaces or fine-tuning generators, they still fall short in preserving these crucial high-frequency details.

Methods

WINE tackles the frequency bias by operating in the wavelet domain. Here's a breakdown of their key contributions:

  • Wavelet Transform for Analysis: They use the Discrete Wavelet Transform (DWT) to decompose images into different frequency sub-bands (LL, LH, HL, HH). This allows them to analyze and manipulate information at various frequency levels.
  • Wavelet Loss: Instead of relying solely on L1 or L2 pixel losses, WINE introduces a "wavelet loss" that specifically penalizes errors in the high-frequency sub-bands (LH, HL, HH). This explicitly encourages the model to preserve fine details. They also propose a multi-level wavelet loss to capture a broader range of high-frequency information.
  • Wavelet Fusion: To effectively transfer high-frequency information from the input image's residual (the difference between the input and the initial low-fidelity reconstruction) to the generator, WINE employs "wavelet fusion." This technique directly manipulates wavelet coefficients, allowing for a more precise and less lossy transfer of high-frequency details compared to traditional feature fusion methods.
  • Adaptive Distortion Alignment (ADA): To handle potential misalignments in the input image or its residual, WINE incorporates an ADA module trained with a self-supervised task to align these features before fusion.

Datasets

The paper uses standard datasets for training and evaluation:

  • Flickr-Faces-HQ (FFHQ): Used for training the generator and encoder components.
  • CelebA-HQ: Used for evaluating inversion and editing performance on human faces.
  • Animal-Faces-HQ (AFHQ): Used to demonstrate the model's generalizability beyond human faces.

Key Insights and Ideas

  • The "Frequency Bias" is Real and Problematic: The paper convincingly demonstrates that standard pixel-wise losses (L1, L2) and even some high-rate inversion methods are heavily biased towards low-frequency information, leading to a loss of fine details.
  • Wavelets are Key to High-Frequency Preservation: By decomposing images into frequency sub-bands, wavelet transforms provide a powerful tool to isolate and explicitly address the preservation of high-frequency details, which are crucial for realistic image reconstruction and editing.
  • Direct Wavelet Manipulation is More Effective: Instead of just using wavelet information indirectly, WINE's "wavelet fusion" directly manipulates wavelet coefficients, enabling a more effective transfer of high-frequency information to the generator.
  • WINE is the First to Interpret GAN Inversion in the Frequency Domain: This work pioneers the use of frequency domain analysis (specifically via wavelets) to understand and improve GAN inversion.

Key Results

  • State-of-the-Art Performance: WINE consistently outperforms existing GAN inversion methods (including pSp, e4e, ReStyle, HyperStyle, HFGI, and StyleRes) across various quantitative metrics like L2, L1, LPIPS, and SSIM. For instance, WINE achieves the lowest L2 loss (0.011) and highest SSIM (0.753) on the CelebA-HQ dataset.
  • Superior Detail Preservation: Qualitatively, WINE excels at reconstructing intricate details such as eyelashes, accessories, and text, as shown in Figure 4 and Figure 6.
  • High-Fidelity Editing: WINE demonstrates a strong balance between editability and reconstruction quality, successfully preserving details even after attribute manipulation (Figure 5).
  • Ablation Studies Confirm Effectiveness: Experiments show that both the wavelet loss and wavelet fusion components significantly contribute to WINE's superior performance.

Limitations and Future Work

  • Generator Dependency: WINE's wavelet fusion mechanism is designed for generators that provide intermediate wavelet coefficients. While they used SWAGAN as a proof-of-concept, generalizing this to other wavelet-based generators is a natural next step.
  • Extending to Other Models: The authors suggest future work in applying their method to wavelet-based diffusion models and other generators that leverage wavelet expansions.
  • Computational Cost: While WINE is efficient for inference, the training process for GAN inversion models can still be computationally intensive. Further optimizations might be explored.

Summary updated

Abstract

Recent advanced GAN inversion models aim to convey high-fidelity information from original images to generators through methods using generator tuning or high-dimensional feature learning. Despite these efforts, accurately reconstructing image-specific details remains as a challenge due to the inherent limitations both in terms of training and structural aspects, leading to a bias towards low-frequency information. In this paper, we look into the widely used pixel loss in GAN inversion, revealing its predominant focus on the reconstruction of low-frequency features. We then propose WINE, a Wavelet-guided GAN Inversion aNd Editing model, which transfers the high-frequency information through wavelet coefficients via newly proposed wavelet loss and wavelet fusion scheme. Notably, WINE is the first attempt to interpret GAN inversion in the frequency domain. Our experimental results showcase the precision of WINE in preserving high-frequency details and enhancing image quality. Even in editing scenarios, WINE outperforms existing state-of-the-art GAN inversion models with a fine balance between editability and reconstruction quality. Pre-trained model and codes will be publicized after the review process.
BibTeX
@inproceedings{kim2025wine,
  author = {Chaewon Kim and Seung Jun Moon and Gyeong-Moon Park},
  title = {{WINE: Wavelet-Guided GAN Inversion and Editing for High-Fidelity Refinement}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
  url = {https://openreview.net/forum?id=d9xgv09iQJ},
}
Figure