Abstract
Cycle consistency is widely used for face editing. However, we observe that the generator tends to find a tricky way to hide information from the original image to satisfy the constraint of cycle consistency, making it impossible to maintain the rich details (e.g., wrinkles and moles) of nonediting areas. In this work, we propose a simple yet effective method named HifaFace to address this problem from two perspectives. First, we relieve the pressure of the generator to synthesize rich details by directly feeding the high-frequency information of the input image into the end of the generator. Second, we adopt an additional discriminator to encourage the generator to synthesize rich details. Specifically, we apply wavelet transformation to transform the image into multi-frequency domains, among which the high-frequency parts can be used to recover the rich details. We also propose a novel attribute regression loss for fine-grained and wider-range control. Powered by this framework, we achieve high-fidelity and arbitrary face editing, outperforming other state-of-the-art approaches.
Introduction
Face editing aims to modify specific attributes—such as adding glasses or a smile—while preserving the subject's identity and the fine details of the original image. Most state-of-the-art models rely on cycle consistency: if you turn a person with no glasses into someone with glasses, and then turn them back, you should recover the original image.
However, the authors identify a "tricky" behavior: the generator often learns to perform steganography. Instead of realistically rendering the edited face, it hides the original face's fine details (like a mole or a specific wrinkle) as imperceptible high-frequency signals in the edited image. This satisfies the mathematical cycle but results in an edited image that looks blurry or loses the "soul" of the original subject, as illustrated in Figure 1.
Methodology
HifaFace introduces three primary innovations to force the generator to be honest and high-fidelity:
1. Wavelet-Based Generator
Instead of standard skip connections, the model uses Haar Wavelet Transformation. It decomposes the input image into four frequency bands: one low-frequency (LL) and three high-frequency (LH, HL, HH). The high-frequency bands, which contain the "rich details," are fed directly into the end of the decoder. This relieves the generator from the "pressure" of having to reinvent these details from scratch.
2. High-Frequency Discriminator ($D_H$)
To ensure the high-frequency parts of the generated image look like real skin texture and not hidden "steganographic" noise, the authors add a second discriminator that specifically inspects the wavelet-decomposed high-frequency components.
3. Attribute Regression Loss
Most models treat attributes as binary (0 or 1). HifaFace introduces a regression loss that allows for a continuous scale ($\alpha$). This enables "arbitrary" editing—for example, making a smile wider or turning regular eyeglasses into dark sunglasses by increasing the attribute intensity beyond the training labels.
Experiments and Results
The researchers evaluated HifaFace on two major benchmarks: CelebA-HQ and FFHQ. The model significantly outperformed existing approaches like StarGAN, RelGAN, and StyleFlow across several metrics:
- FID (Fréchet Inception Distance): Achieved a score of 4.04, significantly lower (better) than the next best competitor (SSCGAN at 4.69).
- Accuracy: Reached 97.5% attribute classification accuracy, showing that the edits are highly effective.
- SRE (Self-Reconstruction Error): Achieved the lowest error (0.021), proving it preserves the original image's identity better than other methods.
Qualitative results (e.g., Figure 2) demonstrate that while other models "wash out" the skin or lose hair detail, HifaFace maintains photographic sharpness even during extreme attribute changes.
Conclusion
HifaFace addresses the steganography problem in cycle-consistent GANs by separating structure and texture through frequency domains. By injecting high-frequency information directly and employing a specialized discriminator, the framework achieves high-fidelity results that maintain the "rich details" of the original subject. This approach not only improves visual quality but also allows for fine-grained, arbitrary control over facial attributes.
Summary updated