Abstract: High quality facial image editing is a challenging problem in the movie post-production industry, requiring a high degree of control and identity preservation. Previous works that attempt to tackle this problem may suffer from the entanglement of facial attributes and the loss of the person's identity. Furthermore, many algorithms are limited to a certain task. To tackle these limitations, we propose to edit facial attributes via the latent space of a StyleGAN generator, by training a dedicated latent transformation network and incorporating explicit disentanglement and identity preservation terms in the loss function. We further introduce a pipeline to generalize our face editing to videos. Our model achieves a disentangled, controllable, and identity-preserving facial attribute editing, even in the challenging case of real (i.e., non-synthetic) images and videos. We conduct extensive experiments on image and video datasets and show that our model outperforms other state-of-the-art methods in visual quality and quantitative evaluation.
Introduction
Facial attribute editing is a cornerstone of digital retouching and film post-production. However, performing these edits automatically on real-world images remains a significant technical challenge. Most existing tools struggle with two primary issues: entanglement and identity loss.
Entanglement occurs when facial features are coupled within the model's representation; for instance, attempting to change a subject's age might inadvertently alter their perceived gender or skin tone. Identity loss occurs when the editing process degrades the unique facial characteristics of the individual, resulting in a subject that no longer resembles the original person. While many generative models perform well on synthetic data, they often fail when applied to "in-the-wild" photos or high-resolution video where temporal consistency is paramount to avoid visual artifacts like flickering.
Methodology
This research leverages the structured latent space of StyleGAN2. To edit a real image, the system first "projects" the image into the extended latent space ($W+$), identifying the specific mathematical vector that represents the target face.
The core of the proposed architecture is the Latent Transformer ($T$). Unlike previous methods that search for global linear directions in latent space, this model trains a dedicated network to learn a non-linear transformation function. The effectiveness of the transformer is driven by a multi-component loss function:
- Attribute Classification Loss ($L_{cls}$): Drives the target attribute toward the desired state (e.g., inducing a smile).
- Attribute Regularization ($L_{attr}$): Acts as the disentanglement engine. It utilizes a pre-trained classifier to ensure that non-target attributes remain constant. A "correlation-weighted" term is included to prevent over-constraining features that are naturally statistically linked.
- Identity Preservation ($L_{rec}$): Ensures the modified latent code remains in close proximity to the original, preserving the subject's unique identity.
For video applications, the pipeline incorporates face alignment, per-frame latent transformation, and Seamless Cloning (Poisson blending) to integrate edited faces back into original frames with high temporal stability.
Experimental Results
The model was evaluated using the CelebA-HQ and FFHQ datasets for training and image evaluation, while FILMPAC and RAVDESS were used to demonstrate performance on real-world video.
Quantitative Performance
When compared against state-of-the-art baselines like GANSpace and InterFaceGAN, the Latent Transformer consistently achieved higher attribute preservation rates and identity preservation scores. Specifically, when modifying complex attributes like age or gender, the model was significantly more effective at keeping unrelated features (such as eyeglasses or facial hair) unchanged.
Qualitative Performance
A key strength of the Latent Transformer is its ability to perform sequential editing. Because the model preserves identity and disentangles attributes so effectively, a single image can undergo multiple transformations in a row without the quality degrading or the identity shifting.
All results are produced at a high resolution of 1024 x 1024, meeting the standards required for professional post-production.
Discussion and Insights
The Non-Linearity of Latent Space
The authors demonstrate that the assumption of linearity in latent space—a common hypothesis in earlier works—is a limiting factor for real-world image editing. The Latent Transformer's ability to navigate the $W+$ space non-linearly allows for much more accurate reconstructions and edits.
Temporal Stability
By operating within the latent space rather than at the pixel level, the model inherently maintains temporal stability in video. Because the latent codes of consecutive frames are naturally similar, the resulting edits do not suffer from the "jitter" or flickering common in other video editing techniques.
Limitations
Despite its success, the model faces challenges with extreme poses, such as side profiles. This is primarily a limitation of the underlying StyleGAN generator and the encoder used for projection; if the initial representation of the face is incomplete, the subsequent edits cannot be fully accurate. Additionally, attributes that are underrepresented in the training data (e.g., specific types of headwear) are more difficult for the transformer to learn effectively.
Conclusion
The Latent Transformer represents a significant step forward in disentangled facial editing. By combining the generative power of StyleGAN with a specialized transformation network and identity-preserving loss functions, the model provides a robust tool for high-resolution image and video manipulation that meets the rigorous demands of the media industry.
Summary updated